GB2273215A

GB2273215A - Synchronising video and audio signals

Info

Publication number: GB2273215A
Application number: GB9324187A
Authority: GB
Inventors: David Graham Kirby; Matthew Ralph Marks; Larry Goodson
Original assignee: British Broadcasting Corp
Current assignee: British Broadcasting Corp
Priority date: 1992-11-24
Filing date: 1993-11-24
Publication date: 1994-06-08
Anticipated expiration: 2013-11-24
Also published as: GB2273215B; GB9324187D0; GB9224622D0

Abstract

A delay signal is associated with a video and an audio signal and represents the delay between the video and audio signal. Each time one of the video and audio signals is subject to further processing some additional delay occurs. The delay signal is modified at 2 in response to this. Synchronisation of the video and audio signals is restored by reading the delay signal and delaying either the video or audio signal, as appropriate, to resynchronise the signals. The invention is of particular use in television production. <IMAGE>

Description

Synchronisation Of Audio And Video Signals Field of the Invention This invention relates to a method and apparatus for synchronising a video signal with an associated audio signal in situations where at least one of the signals has a delay introduced into it by, for example, video or audio processing. The invention is of particular use in the field of television production where a finished programme is produced from frequently changing combinations of many different signals. Certain ones of these will at times be passing through various items of processing equipment which introduce fixed or variable delays to either the audio or the video signal.

Background of the invention As television technology has progressed an ever increasing number of video and audio processing devices are being used in television production. Usually these act on the video signal but not on its associated audio signal.

There are, however, an increasing number of devices which act on the audio signal and delay this without causing delay to the video signal. The result of the use of such video and audio processing devices is a loss of synchronisation between audio and video (lip-sync) which is noticed particularly when a subject is speaking to a camera.

One remedy to this problem would be to intercept the program and introduce a fixed delay into the audio or video signal to resynchronise them. However, some of the processing devices used in television production introduce variable delays and where several different program sources are being combined each source will have been exposed to different delay values. This makes synchronisation by introduction of a fixed delay impractical.

An alternative would be to insert a compensating fixed or variable delay to either the video or the audio signal at every point where the other is delayed. This requires the audio path through the production equipment to correspond physically with the video path and can result in degradation of the signal because of the amount of additional processing involved. Furthermore this is expensive to implement.

In United Kingdom Patent Application No. GB-A-2181325 a representation of an audio signal is introduced into a video signal whilst in the United Kingdom Patent Application No. GB-A-2244184 a representation of aspects of a video signal are introduced into an associated audio signal. At the end of the program production chain synchronisation is restored by correlating the actual signal with its representation. The correlation process is, however, somewhat complex and suffers from reliability problems which vary with program content, as a result of the effect of video or audio processing on both the actual signal and its representation and as a result of the addition of other signals to the final output signal.

Furthermore, the system can add information to the audio signal and thus generate audible artefacts under certain circumstances.

Summary of the Invention In a preferred embodiment of this invention a video signal and its associated audio signal have a delay value inserted into the video and/or the audio signal corresponding to any lag of the video/audio signal. Modification of the delay value takes place in dependence on further delays introduced into the audio or video paths by audio or video processing devices. The accumulated delay value can then be extracted and applied to compensating delays to re-synchronise the audio and video signals.

The invention is defined with more precision in the appended claims to which reference should now be made.

Brief Description of the Drawings The invention will now be described in more detail by way of example with reference to the accompanying drawings in which: Figure 1 is a block diagram of a video processing device and an associated synchronising device embodying the invention for use in a television production chain; Figure 2 is a block diagram of the synchronising apparatus of figure 1; and Figure 3 illustrates the format of delay data used in an embodiment of the invention.

Detailed description of preferred Embodiments Figure 1 shows a block diagram of an embodiment of the invention in which a representation of a variable delay can be inserted into a video signal in dependence on a delay introduced into the video signal by a video processing device. The delay representation is then used to re-synchronise the video signal with an associated audio signal after the video signal has passed through the video processing device.

Figure 1 shows a synchronising device 2 which can receive up to four video signals and pass them to a video processing device 4 with which it is associated. In this particular example the video processing device 2 has only two inputs 6 and thus only two incoming video signals 8 are shown entering the synchronising device. The synchronising device has four high impedance looped inputs 10 and the incoming video signals 8 pass through these before passing to inputs 6 on the video processing device 4. Selection or cueing information 12 associated with the incoming video signals is also supplied to the synchronising device and the video processing device.

By passing the video signal through the synchronising device any delay values which are already present in an incoming video signal can be read. These will usually be included in an unused part of the video signal such as a field blanking interval. This delay value is extracted by the synchronising device 2. Any additional delay value introduced by the video processing device into the video signal is added to the received delay value in the synchronising device 2.

The processed video signal leaves the video processing device at an output 14. This output is coupled to a further input 16 of the synchronising device which then supplies it to an output 18 via an electronic switch 20.

The switch 20 is operable to insert new delay data 22 at appropriate points in the video signal. This can then be used for resynchronsation of the video signal with its associated audio signal in compensating delay devices.

Because the delay data is extracted from the video signal prior to processing in the processing device 4 and reinserted after processing it is of no consequence if the video processing device destroys delay data contained within the video signal. One of the functions of the synchronising device is thus to "bridge" the data across a video processor which would otherwise destroy it.

In the embodiment illustrated here there are three ways in which the delay introduced into the video signal path by the video processing device 4 can be made known to the synchronising device 2 for modification of the extracted delay value. Firstly, a direct connection 24 can supply a delay input to the synchronising device from the video processing device. This is particularly appropriate where the video processing device introduces a variable delay.

Secondly, if the delay introduced by the video processing device is fixed it can be set manually by switches 26 on the synchronising device.

Thirdly, if the inputs and outputs of the video processor are asynchronous (as would occur with a video synchroniser), the delay introduced is always less than four video fields and the input and output signals conform to the standard phase alternate line (PAL) definition.

The synchronising device can then measure the delay from the synchronising pulses and colour burst sequence at its inputs 10 relative to its input 16.

In the case of an audio processing device the synchronising device can use either of the first two methods above.

Audio signals do not, generally, contain unused space such as the blanking intervals which are present in video signals. Therefore, when an audio processing device is used and has an associated synchronising device 2 such as that in figure 1 the associated video signal is looped through one of the high impedence inputs 10 and then passed straight to the input 16. The delay value embedded in the video signal is modified by subtracting rather than adding data and the modified delay value reinserted in a blanking interval by the electronic switch 20. The delay value is subtracted rather than added since delaying the audio signal has an opposite effect to delaying the video signal. Thus the compensating delays effect the same type of correction for an audio delay as they do for a video delay.

In the absence of any cueing information 12, the active input 10 of the synchronising device can either be set manually, or the device can select, from those of its inputs which carry a video signal, the input which has the greatest value of embedded delay. This is done because, for a given discrepancy between audio and video timings, the effect is less noticeable if the audio lags the video than vice versa.

In operation the delay introduced by the processing device is numerically added to the value of the delay extracted from the selected incoming video signal and the modified value is inserted into the video signal provided by the output of the processing device.

In practice the updating of the delay code carried out by the synchronising device may itself require delaying by the same amount as the video (or audio) signal has been delayed in passing through its associated hardware. For example, if a signal reaching the circuitry of Figure 1 has a delay code embedded in it representing previous delays, the synchroniser will read the delay code and update it immediately. However, the frame in the input video signal is being delayed by the video processor.

Thus, unless correction is made, the delay code will propogate through the system faster than the frame of the video signal with which it is associated. For this reason it is preferable to delay the insertion of the delay code in correspondence with the delay to which the video signal is currently subjected.

The first synchronising device in a production chain for a video programme will usually encounter no delay data at its input 10 providing that audio and video are co-timed.

This is interpreted as zero incoming delay. Synchronsing devices accumulate the delays introduced by their accompanying processors, and because the delay indication is embedded in the video signal itself, it will always correspond correctly to the delay which has been suffered by that signal. Different signals selected by a video mixer will include different embedded data if they have incurred delay, so the final mixed-down programme will also contain all the correct delay information. All that is then required is for a final synchronising device to be fed with this completed programme and the associated audio signals. It will provide delay data via connections 27 to variable audio and/or video delays. The delays will then be adjusted in dependence on the data, using strategies to limit audibility and visibility of adjustment, to resynchronise the programme sound to its picture.

It is currently envisaged that, in most applications, the delay suffered by a video signal will always be greater than that of an audio signal due to the multiplicity of video processors in use in programme production. Because of this an adjustable video delay will not usually be needed. However, if a video delay is used, it may only be adjustable in coarse steps in order to maintain synchronisation between its input and output. An audio delaying device could therefore be used to restore synchronisation more precisely.

The synchronising device shown in figure 2 comprises four high impedance looped video inputs 10. The inputs have high impedances so that they can sample video signals without degrading them to any significant degree. An electronic switch 28 selects a video signal supplied to one of the inputs under the control of a control circuit 30. The selected signal is then passed to both a clamp circuit 32 and a sync. separator 34. The sync. separator 34 controls the clamp 32 via an input 36 to ensure that the selected video signal is referenced to a known DC voltage level. The clamped video signal and the separated synchronising pulses from the sync. separator 34 are then fed to a data receiver 38. The data receiver 38 expects to encounter data of a predetermined format such as that shown in figure 3.This data is carried on a video line during a vertical blanking interval (VBI) between successive video fields.

The data is represented by eight binary digits. In this particular example the 256 binary numbers that these eight bits provide are implemented in 4 millisecond steps to give delay values from -124 to +892 milliseconds. The delay data is preceded in the (vBI) by eight bits of address data in a fixed pattern. These are used to identify the data as delay data as opposed to other data which may be carried on this television line.

Address and data bits are represented by a voltage near to the maximum "peak white" level of the video signal for a logical 1 and the black level of the video signal for a logical 0. The stream of bits is filtered prior to insertion into the video signal to ensure that it does not contain any information outside the permitted bandwidth.

The first address bit is inserted llys after the start of the horizontal synchronising pulse to avoid interference with line blanking period. The eight address and eight data bits are then contiguous on the video line and each has a duration of lps. In use the data receiver 38 waits for a predetermined number of horizontal sync. pulses after a vertical sync. pulse for the line on which the data is carried. It then waits 11.5us from the beginning of the horizontal sync. pulse before sampling the video signal on that line for the level of the first address bit. In this way the sample is taken in the middle of the bit period thus providing maximum protection against jitter.

A window comparator in the data receiver is used to determine whether the voltage level is below a lower threshold representing logical 0 or above an upper threshold representing logical 1. If any bit is between these two thresholds the delay value is discarded because the data may have become corrupted as a result of mixing of two video signals.

Data extracted by the data receiver 38 is passed to the control circuit 30. The receiver can be arranged to provide an additional connection to the control circuit 30 for use in the situation where data does not appear on the expected video line or contains errors.

The sync. pulses 40 and the clamped video signal 42 provided at the outputs of the sync. separator 34 and the clamp circuit 32 respectively, are also sent to a colour burst detector 44. This contains a line counter and charge pump which operate in a well known manner to produce an output signal 46 if a colour burst is present on a line of the input video field. This is used by the control circuit 30 to determine, with the aid of sync.

pulses 40 from the sync. separator 34, the timing of standard PAL colour television signals within a four field (ATMS) sequence. It is also able to detect whether video is present on the selected input 10 by the presence or absence of sync. pulses.

A second video path through the synchronising device passes from the video intercept input 16 to the video output 18. The input 16 receives the video signal output by the video processing device 14 and the path it then follows is used to enable modified delay data to be inserted into the video signal. This path is completely independent of signals from inputs 10 and thus the signal may be asynchronous with the signals at inputs 10.

Video received on the input 16 passes to a sync. separator 48 and a clamp 50. The clamp is used to ensure that DC level of the video signal is the same as that of any data to be inserted. The signal then passes from the clamp circuit output via a switch 20 to the video output 18.

Synchronising pulses 52 from the sync. separator are passed to a data transmitter 54. This includes a line counter. When a line in the video signal into which modified delay data is to be inserted is encountered an output 56 of the data transmitter supplies a signal to control the electronic switch 20 to receive delay data from a second output 58 of the data transmitter. Delay data to be provided at output 58 comes from the control circuit via a connection 60.

When data is to be inserted in a line of the video signal the switch 20 is activated l.Ops before the first address bit to be written and is released l.Ops after the last data bit to be written. This ensures that any data already present will be overwritten and prevents harmonics being generated from clipping of the data. A burst detector 62 receiving the sync. pulses and the clamped video signal operates in a similar manner to the colour burst detector 44. The timing of the output video signal within a four field PAL sequence can be determined using the sync. pulses which are produced at output 64 to the control circuit if the colour burst is present on a predetermined line of a video signal.This output is fed to the control circuit 40 which can then, if required, use this to compare the timing of the signal received at input 16 with that received at one of the inputs 10-to calculate the delay being introduced by the video processing device.

The presence of absence of sync. pulses also enable it to determine whether or not a video sign is in fact present at the input 16.

External delay data 24 generated automatically by a video or audio processing device passes to the control circuit 30 via an interface 64 for addition to delay data extracted from a video signal. Cueing data 12 associated with the input video signals also passes through this interface 64.

A further input to the control circuit 30 is provided by the delay setting switches 26. These are used for setting manually a delay value to be added to any delay already present in the signal.

Delay value outputs 66 are provided via an interface 68 coupled to the control circuit 30. These are used for resynchronisation of an audio and video signal. These outputs 66 provide inputs to audio and video compensating delays.

The control circuit 30 uses the information provided to it to control the select switch 28 to select one of the looped inputs 10 as a video signal source. From this, video signal delay values are extracted by the data receiver 38 and are provided to the control circuit 30.

Displays on the synchronsing device can be used to show the status and delay of the signal presented to each input 10.

The control circuit operates to add the delay introduced between the input 10 and the input 16 to the delay value detected by the data receiver 38. This total delay value is then written into unused space in the video signals provided at input 16 by means of the data transmitter 54 and the electronic switch 20 to provide a signal including the correct delay value at output 18.

In a programme production chain every piece of video or audio processing circuitry will have a synchronising device associated with it. Thus any delay caused to the audio or video signal by any of the processing circuitry will be added to the delay value embedded in the video signal.

At the end of a production chain a further synchronising device will receive the final video signal on one of its inputs 10 and from this will provide delay outputs 66 to control audio and video compensating delays. The arrangement is such that the output 66 are only updated after a minimum number of consecutive identical delay values have been detected by the data receiver 38. This is to minimise the risk of errors in delay data effecting the synchronisation of the audio and video signals.

In the production chain it is preferable that the first synchronising device in the chain uses such as "confidence counting" process in generating its output data to embed in a video signal. Subsequent units are, however, permitted to change their embedded output data whenever the incoming embedded data changes and further confidence counting is thus only performed at the final synchronising device controlling compensating delays. This avoids generating any cumulative lag in the response time of the system which would result if confidence counting were employed at every synchronising device before embedded data is updated. Preferably the compensating delays controlled by outputs 66 operate to implement adjustments in the delay values received in small steps in order to make transitions more smooth.This small step transition could also be implemented in the synchronising device by, for example the interface 68.

One of the delay setting switches 26 can be used to insert a "hold delay" into a video signal. The purpose of this is to be associated with video signals that have no associated audio signal, e.g. a still picture, and which are temporarily cut over a main video signal while it retains its audio. The code is used by the system to prevent any delay in adjustment while the still picture is present in the video signal and will thereby avoid any temporary misalignment when the still picture is removed and the previous delay is restored, i.e. the delay implemented remains the same throughout the period when the still picture is displayed.

In some digital audio signals e.g. AES/EBU digital audio, an independent data stream is provided for user data. In an alternative embodiment the delay code could be included in this user data channel associated with the audio signal. Thus it would not be necessary to access the video signal to modify the delay code in response to an audio delay. To compensate with the correct delay the audio delay is subtracted from the video delay. The difference in the delay codes is used, as before, at approriate points to control a compensating delay and thus resynchronise the audio and video signals.

Claims

1. A method for synchronising a video signal with an associated audio signal comprising the steps of providing a delay signal associated with the video and audio signals, the delay signal representing a delay between the video and audio signals, reading the delay signal, and delaying at least one of the video and audio signals in response to the delay signal to-restore synchronisation.

2. A method according to claim 1 in which the delay signal is included in one of the video and audio signals.

3. A method for synchronising a video signal with an associated audio signal according to claim 1 or 2 in which the delay providing step includes the step of modifying the delay signal in dependence on additional delay caused by processing of one of the video and audio signals.

4. A method for synchronising a video signal with an associated audio signal according to claim 3 including the steps of reading the delay signal prior to modification and replacing the delay signal with the modified value.

5. A method for synchronising a video signal with an associated audio signal according to claim 2, 3 or 4 in which the delay signal is provided in the video signal.

6. A method for synchronising a video signal with an associated audio signal according to claim 5 in which the delay signal is provided in a field blanking interval in the video signal.

7. A method for synchronising a video signal with an associated audio signal according to claim 5 or 6 in which delay of the audio signal causes the delay signal to be modified in an opposite sense to modification caused by delay of the video signal.

8. A method for synchronising a video signal with an associated audio signal according to any preceding claim in which the delay signal may comprise a predetermined code indicating that a previously read delay signal should be utilised for synchronisation.

9. A method for synchronising a video signal and an associated audio signal according to any preceding claim in which the step of delaying one of the video and audio signals is responsive to a change in the delay signal after a new delay signal has been read a predetermined number of times.

10. Apparatus for synchronising a video signal and an associated audio signal comprising means for providing a delay signal associated with the video and audio signals, the delay signal representing a delay between the video and audio signals, means for reading the delay signal, and means for delaying at least one of the video and audio signals in response to the delay signal to restore -synchronisation.

11. Apparatus for synchronising a video signal and an associated audio signal according to claim 10 in which the delay providing means comprises means for including the delay signal in one of the video and audio signals.

12. Apparatus for synchronising a video signal and an associated audio signal according to claim 10 or 11 comprising means for modifying the delay signal in dependence on additional delay caused by processing of one of the video and audio signals.

13. Apparatus for synchronising a video signal and an associated audio signal according to claim 12 including means for reading the delay signal prior to modification and means for replacing the delay signal with the modified signal.

14. Apparatus for synchronising a video signal and an associated audio signal according to claim, 10, 11 , 12 or 13 in which the delay signal is included in the video signal.

15. Apparatus for synchronising a video signal and an associated audio signal according to claim 14 in which the delay signal is included in a field blanking interval in the video signal.

16. Apparatus for synchronising a video signal and an associated audio signal according to claim 10 in which delay of the audio signal causes the modifying means to modify the delay signal in an opposite sense to modification caused by delay of the video signal.

17. Apparatus for synchronising a video signal and an associated audio signal-according to any of claims 10 to 16 responsive to a predetermined code in the delay signal to utilise a previously detected delay signal for synchronisation of video and audio signals.

18. Apparatus for synchronising a video signal and an associated audio signal accord-Lng to any of claims 10 to 16 responsive to a predetermined code to identify the delay signal.

19. Apparatus for synchronising a video signal and an audio signal according to any of claims 10 to 18 in which the delaying means is responsive to a change in the delay signal after a new delay value has been received a predetermined number of times.

20. A method for modifying a delay signal associated with and representing a delay between a video signal and an associated audio signal comprising the steps of reading the delay signal, reading delay data representing additional delay caused by processing of one of the video and audio signals, and modifying the delay signal in response to the delay data.

21. A method according to claim 20 in which the delay signal is included in the audio signal.

22. A method according to claim 20 in which the delay signal is included in the video signal.

23. A method for modifying a delay signal included in one of and representing a delay between a video signal and an associated audio signal according to claim 22 in which the delay signal is included in a field blanking interval of the video signal.

24. Apparatus for modifying a delay signal associated with and representing a delay between a video signal and an associated audio signal comprising means for reading the delay signal, means for reading delay data representing additional delay caused by processing of one of the video and audio signals, and means for modifying the delay signal in response to the delay data.

25. Apparatus according to claim 24 in which the delay signal is included in the audio signal.

26. Apparatus according to claim 24 in which the delay signal is included in the video signal.

27. Apparatus according to claim 24 in which the delay signal is included in a field blanking interval in the video signal.

28. A method for synchronising a video signal with an associated audio signal substantially as herein described.

29. Apparatus for synchronising a video signal with an associated audio signal substantially as herein described.

30. A method for modifying a delay signal associated with and representing a delay between a video signal and an associated audio signal substantially as herein described.

31. Apparatus for modifying a delay signal associated with and representing a delay between a video signal and an associated audio signal substantially as herein described.