GB2437122A

GB2437122A - Method and apparatus for measuring audio/video sync delay

Info

Publication number: GB2437122A
Application number: GB0607214A
Authority: GB
Inventors: Matthew Alan Bowers; Scott Griffiths
Original assignee: Tektronix International Sales GmbH; VQUAL Ltd; Tektronix Bristol Ltd
Current assignee: Tektronix International Sales GmbH
Priority date: 2006-04-10
Filing date: 2006-04-10
Publication date: 2007-10-17
Anticipated expiration: 2026-04-10
Also published as: GB2437122B; GB0607214D0

Abstract

A method of determining the delay between an audio and visual signal, the method comprising providing a video signal having a predetermined sequence of visual events, each visual event occurring at a predetermined point in the sequence, providing an audio signal having a predetermined sequence of audio events, each audio event occurring at a predetermined point in the sequence, wherein the audio and video events are in synchronicity with one another, digitally encoding the video and audio signals to generate encoded video and audio data streams, detecting the occurrence of each event in the encoded video and audio data streams and their relative time of occurrence and from the detected time of occurrence of the audio and visual events measuring any time delay between the encoded video and audio data streams.

Description

I

METHOD AND APPARATUS FOR MEASURING AUDIO/VIDEO SYNC DELAY In the field of providing video content it is common to provide that content as digital data.

In the context of this application the term "video" shall be taken to refer to the combination of audio and visual information. A common requirement is to pass the digital video data through one or more encoding processes, for example prior to the broadcast transmission of the digital video data. The coding processes habitually involve data compression and the use of digital audio filters to process the audio signal. The encoding process may also typically involve multiplexing a plurality of separate data streams together. Since it will commonly be the case that the audio data will be processed differently from the image data and each of the different stages in the encoding process can potentially introduce a time delay to the digital data signal, the overall encoding process can potentially introduce a loss of synchronisation between the audio and image data, which will be most unnoticeable as a loss of lip-sync in video footage of the speaking characters. The human brain can perceive even quite small time delays between the video and image data, with the circumstances in which the audio signal leads the video signal being most noticeable. For this reason, the applicable encoding and transmission standards stipulate maximum time delays between the audio and video data. For example, according to some standards the audio signal must not lead the corresponding video signal by a time delay greater than 4Oms.

It is therefore advantageous to be able to determine in advance the precise amount of time delay any given encoding system is likely to introduce between the audio and visual parts of the overall video signal.

According to a first aspect of the present invention there is provided a method of determining the delay between an audio and visual signal, the method comprising: Providing a video signal having a predetermined sequence of visual events, each visual event occurring at a predetermined point in the sequence; Providing an audio signal having a predetermined sequence of audio events, each audio event occurring at a predetermined point in the sequence, wherein the audio and video events are in synchronicity with one another; Digitally encoding the video and audio signals to generate encoded video and audio data streams; Detecting the occurrence of each event in the encoded video and audio data streams and the relative time of occurrence; and From the detected time of occurrence of the audio and visual events measuring any time delay between the encoded video and audio data streams.

Preferably an alert may be generated if the measured time delay between the encoded audio and video data streams exceeds a predetermined value.

Preferably each event within both sequences of audio and visual events has a different duration. The duration of the events may sequentially increase through the sequence.

Preferably each event within both sequences of audio and visual events is spaced apart by a time periods of differing duration. The time periods between events may sequentially increase throughout the sequence of events.

Preferably each video event comprises illuminating a predetermined display sector.

Preferably each audio event comprises generating an audible tone.

According to a second aspect of the present invention there is provided apparatus for determining the delay between encoded audio and video signals, wherein the video signal has a predetermined sequence of visual events, each visual event occurring at a predetermined point in the sequence and the audio signal has a predetermined sequence of audio events, each audio event occurring at a predetermined point in the sequence, the audio and video events being in synchronicity with one another, the apparatus comprising; A video event detector arranged to detect each video event and provide a first time signal corresponding to each video event; An audio event detector arranged to detect each audio event and provide a second time signal corresponding to each audio event; and A time comparator arranged to receive the first and second time signals and to measure any time delay between the signals.

The apparatus preferably further comprises an alert generator arranged to receive the measured time delay from the time comparator and generate an alert if the time delay exceeds a predetermined value.

Embodiments of the present invention will now be described below, by way of illustrative example only, with reference to the accompanying figures, of which: Figure 1 schematically illustrates the timing and duration of audio and video events included in a test signal according to embodiments of the present invention; Figure 2 schematically illustrates a time delay analysis system for determining the time delays between the audio and video signals shown in Figure 1; and Figure 3 schematically illustrates the relative timings of an audio and video signal as shown in Figure 1 in which there is a delay between the audio and video signals.

In embodiments of the present invention any time delay between audio and video data subsequent to an encoding process being performed on the originally available audio and video data is determined utilising a predetermined video sequence having known timing properties. The video/audio data sequence is provided in either an unencoded data format, such as RAW, or in a "basic" standard encoded data format, such as for example MPEG.

The predetermined video sequence comprises a series of visible "flashes" having a predetermined duration and time interval between each flash. The sequence also comprises a corresponding number of audible tones whose duration and time interval between tones exactly corresponds to the occurrences of the visible flashes. An example of an appropriate timing diagram for the visible and audible signals is schematically illustrated in Figure 1. Tn Figure 1 the upper signal trace 2 represents the binary levels for the visible signal, with the signal either being totally black or totally white in visible appearance. The lower signal trace 4 represents the audible signal, with the upper signal level representing a production of an audible tone and the lower signal level representing the absence of a tone.

As can be seen from Figure 1, after an initial time period of 1 unit, for example I second, during which neither a visible flash is produced nor an audible tone produced, a visible flash and audible tone of duration of 1 unit is subsequently produced. This is followed by a further time period during which no visible flash or audible tone is produced, this second time period having a duration of 2 units. This is then followed by the production of a visible flash and audible tone having a duration of 2 units, followed by a period of no visible flash or audible tone of duration 3 units and so on in the sequence illustrated in Figure 1, the total sequence comprises five periods during which a visible flash and audible tone are produced, each period lasting one time unit longer than the preceding period, with correspondingly increasing time periods in between during which no visible flash or audible tone is produced. In the example shown therefore the entire sequence lasts for a total of 30 time units, which will typically be 30 seconds. The entire sequence preferably continually repeats.

In a preferred embodiment of the present invention the visible flash is produced in at least the macro block, or at least an integer part thereof, that is shown at the top left hand corner of the display screen. Preferably a 2x2 array of blocks, i.e. 32x32 pixels, is used to encode the visible flash. This location is carefully chosen since, due to the scanning method of generating a displayed image as will be appreciated by those skilled in the art, the digital data representing this part of the display screen will occur very early in the relevant data stream and will consequently practically always be correctly encoded. The selection of the visible flash as a 32x32 pixel area will also tend to ensure the correct encoding of this video data. Similarly, the use of only black and white shades for the visible flash will maximise the likelihood of the video data being correctly encoded since these are "basic" digital values. In a similar fashion, the audio tone is provided as a tone whose only frequency allowance is at 10KHz, or some other single frequency. Since only a single frequency component is utilised for the audio tone it should be faithfully encoded by any audio encoder included within the encoding system under test.

Further visual data may be provided to the user, for example a larger visual representation of the visible flash, for example as a series of rotating circular segments, each segment being representative of a single time unit such that a complete sequence requires a full "revolution" through the multiple segments. It will of course be appreciated that such visual enhancements are merely for the convenience of the human operator and are not a necessary part of embodiments to the present invention.

The predetermined audio visual sequence is passed through the encoding system under test and the encoded digital data stream subsequently analysed. The analysis process comprises detecting one or both of the beginning and end of one of the visible flashes by detecting the point in time within the encoded data stream at which the 32x32 pixel macro block integer changes from "black" to "white" or vice versa. The time at which this occurs is accurate to within the duration of 1 frame of audio visual data, since the display is only refreshed every frame. A typical frame rate is 25 frames per second, concurrently the encoded audio signal is analysed to determine one or both of the beginning and end of the audio tones. A preferred method of detecting the beginning or end of the audio tone is to detect the sharply rising or falling amplitude of the tone as each transition from "tone" to "no tone" or vice versa occurs. The analysis process can thus determine any time delay between the video and audio "events" (an event being rising or falling audio or video signal edge). In preferred embodiments any determined delay that falls outside a predetermined set of parameters, such as those set by one or more transmission standards, causes an alert to be automatically generated.

A system in accordance with an embodiment of the present invention for determining any loss of video/audio synchronisation in an encoded data stream is schematically represented in Figure 2. A predetermined video stream 10 as described above in an un-encoded state is stored on a data storage medium such as a hard disk 12 and is provided as an input to the encoding system 14 to be tested. The encoding system will generally output an encoded data stream that can be decomposed to separate video 16 and audio 18 streams. Each of the video and audio streams are provided as inputs to an analysis engine 20 and input to separate video and audio event detection units 22, 24. It will be appreciated that although the video and audio streams are shown in Figure 2 as discrete inputs to the analysis engine, the decomposition of the encoded data stream provided by the encoding system 14 may equally be accomplished within the analysis engine, for example by means of a wrapper demux. Each event detection unit is arranged to detect the relevant video or audio events' of the encoded test data stream, these being the beginning or ends, or both, of the visible flashes' and audio tones as discussed above in relation to Figure 1, and to provide an output signal indicative of when each event occurs. The output signals from each of the audio and video event detection units 22, 24 is provided to a time comparison unit 26 that is arranged to measure any time interval present between the output signals from the event detection units and thus any time interval, be it lag or lead, between the occurrences of the audio and video events'. This time interval data is provided from the time comparison unit 26 to an output interface unit 28 that is arranged to provide the time interval data to an appropriate user interface. In preferred embodiments the output interface unit 28 is also arranged to compare any time delay between the audio and video signals with defined maximum permitted delays that may be stored in a further data storage area 30 or may be stored internally to the output interface unit. If a detected time interval exceeds a predefined value then the output interface unit may be arranged to provide an alarm signal.

As previously mentioned with reference to Figure 1, the sequence of visible flashes and audible tones comprises events' of increasing duration and with increasing time intervals between each event'. This ensures that should the time delay between the video and audio signals be great enough for one of the video events to coincide with an audio event this false' synchronisation, which would not cause the analysis engine to generate a report or alarm, will not be maintained at the next occurrence of a video and audio event. This is schematically illustrated in Figure 3, in which the upper trace 32 represents the video event signal and the lower trace 34 represents the audio event signal. As can be seen from Figure 3, the audio event signal has been delayed relative to the video signal by the encoding process by a time period of 3 time units, say seconds, as represented by arrow A. Consequently, the beginning of the second video event 36 occurs at the same time as the beginning of the first audio event 38. If these are the first video and audio events detected by the analysis engine then a false report of synchronisation between the audio and video streams may be provided. However, at the beginning of the next video event 40 it can be seen that the audio stream is out of synchronisation, since the events are not evenly spaced apart and do not have a constant duration. Consequently the analysis engine is able to determine that in fact the video and audio streams are not in synchronisation. If the analysis engine detects both the beginning and end of the video and audio events then the loss of synchronisation will be detected sooner since the end of the first audio event 38 will occur before the end of the second video event 36, even though the beginning of both events coincided. In this instance the loss of synchronisation between the audio and video streams is detected by the analysis engine within one time unit, for example one second.

The method and apparatus of embodiments of the present invention is thus able to detect and measure any time delay between encoded digital video and audio data streams quickly and reliably.

Claims

CLAIMS

I. A method of determining the delay between an audio and visual signal, the method comprising: providing a video signal having a predetermined sequence of visual events, each visual event occurring at a predetermined point in the sequence; providing an audio signal having a predetermined sequence of audio events, each audio event occurring at a predetermined point in the sequence, wherein the audio and video events are in synchronicity with one another; digitally encoding the video and audio signals to generate encoded video and audio data streams; detecting the occurrence of each event in the encoded video and audio data streams and their relative time of occurrence; and from the detected time of occurrence of the audio and visual events measuring any time delay between the encoded video and audio data streams.

2. The method of claim 1, wherein an alert is generated if the measured time delay between the encoded audio and video data streams exceeds a predetermined value.

3. The method of claim I or 2, wherein each event within both sequences of audio and visual events has a different duration.

4. The method of claim 3, wherein the duration of the events sequentially increases through the sequence.

5. The method of any preceding claim, wherein each event within both sequences of audio and visual events is spaced apart by a time period, each respective time period having a different duration.

6. The method of claim 5, wherein the time periods between events may sequentially increase throughout the sequence of events.

7. The method of any preceding claim, wherein each video event comprises illuminating a predetermined display sector.

8. The method of any preceding claim, wherein each audio event comprises generating an audible tone.

9. Apparatus for determining the delay between encoded audio and video signals, wherein the video signal has a predetermined sequence of visual events, each visual event occurring at a predetermined point in the sequence, and the audio signal has a predetermined sequence of audio events, each audio event occurring at a predetermined point in the sequence, the audio and video events being in synchronicity with one another, the apparatus comprising: a video event detector arranged to detect each video event and provide a first time signal corresponding to each video event; an audio event detector arranged to detect each audio event and provide a second time signal corresponding to each audio event; and a time comparator arranged to receive the first and second time signals and to measure any time delay between the signals.

10. The apparatus of claim 9 further comprising an alert generator arranged to receive the measured time delay from the time comparator and generate an alert if the time delay exceeds a predetermined value.