This blog is only to remind myself of what I've learned (from others or by myself) and only for knowledge sharing. External sources will be clearly stated in [Reference] of each article. I hope this blog won't infringe any copyrights and that it can be useful for interested blog readers.

2008年7月5日 星期六

DirectShow a/v sync notes

Preface

Reference DVB-H CRTPSourceFilter source code

Direct Show

!!NOTES:

when checking a/v sync, there are three cases to be tested.

a) From no signal to having signal (e.g. Mute->unMute)

b) fast-backward/re-wind, fast-forward (DISCONTINUITY)

c) Device lost (ex: Ctrl-Alt-Del)

4) buffering mechansim

1) a/v PTS

Real a/v PTS should come from a source filter,

ex: FileSourceFilter, DVBHRTPSourceFilter

a/v PTS should be used in MediaSample::SetTime(..) and

MediaSample::SetSyncPoint(..) with 100ns-based unit.


Pseudo code:

HRESULT CRTPSourcePin::FillBuffer(IMediaSample *pSample) {

// Notes:

// (1) pSample may be audio or video samples.

// (2) m_rtTimeBase and pPacket->pts are micro-second based.

1 micro-sec = 1000 nano-sec.

rtStart = (m_pSourceFilter->m_rtTimeBase + pPacket->pts) * 10 ;
rtEnd = rtStart + 1 ;

pSample->SetTime( &rtStart, &rtEnd );
pSample->SetSyncPoint( TRUE ) ;

if ( pPacket->pts > m_pFilter->m_rtCurrMaxPTS) {
m_pFilter->m_rtCurrMaxPTS = pPacket->pts;
}

}

!!NOTES:

MediaSample PTS value has to always move upward.

Otherwise, there might be frame drops.

2) DirectShow System Clock

When running a DirectShow Graph, System clock will also move forward

together with MediaSample PTS.


There will be following different cases

a) When PTS of MediaSample is equal to or bigger than System Clock

Audio MedisaSample PTS

|----------------|-------------|----------------------

100 200 300 or 500 (meant to be assigned)

Video MediaSample PTS

|----------------|-------------|-----------------------

100 200 300 or 500 (meant to be assigned)

System clock

|----------------|-------------|-----------------------

1000 1100 1200

Result: a/v will be in sync as long as PTS for MediaSample is assigned synchronously for a/v.


b) The speed of MediaSample is slower than System Clock

Audio MedisaSample PTS

|----------------|-------------|----------------------

100 200 210 (meant to be assigned)

Video MediaSample PTS

|----------------|-------------|-----------------------

100 200 210 (meant to be assigned)

System clock

|----------------|-------------|-----------------------

1000 1100 1200

Result:

might have video frame drops even though a/v PTS is in sync

because System Clock runs faster


3) Discontinuity

When discontinuity happens, what should be done

a) Use the last MAX PTS as the next PTS base

(ps: this is to make sure PTS value is moving upward)

b) Calculate how many seconds are passed between discontinuity event and

the last packet PTS

(ps: this is to make sure the following MedisaSample PTS value still

follows System Clock speed)

Pseudo code:

s = GetMediaPacket(...);

if (s == REAL_PACKET) {

lastPacket_tickCount = GetTickCount();

} else if (s == DISCONTINUITY) {

m_pSourceFilter->m_rtTimeBase += pRTPSourceFilter->m_rtCurrMaxPTS;

// extra seconds to add

m_pSourceFilter->m_rtTimeBase += (GetTickCount() - lastPacket_tickCount ) *1000;
m_pSourceFilter->m_rtCurrMaxPTS = 0;

}

 

4) Buffering mechanism

When it comes to DVB-T, buffering may not be needed.

But when it comes to DVB-H burst streaming, deciding how many packets should be buffered

in advance is very important because it will affect the playback quality.