JP3199046B2 - Delay compensation method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a delay correction method in a multiplex transmission system that encodes, multiplexes, and transmits at least a video signal and an audio signal.

[0002]

2. Description of the Related Art FIG. 4 is a block diagram showing a conventional multiplex transmission system. In the figure, 100 ′ is a transmitting device,
Reference numeral 200 'denotes a receiving device, which is a transmission line (public line) 300.
Connected to each other. In this multiplex transmission system, a transmitting device 100 'and a receiving device 200'
Is a multimedia terminal device conforming to ITU-T H.320. Details of each block in the multimedia terminal device are shown in ITU-T H.261, H.221, I.430, and G.711.

In this multiplex transmission system, a transmitting device 10
At 0 ', the video signal and the audio signal are encoded and multiplexed, and transmitted to the receiving apparatus 200' via the transmission path 300. The receiving device 200 'separates and decodes the multiplexed video signal and audio signal sent from the transmitting device 100' .

In this case, a delay correction circuit 30 (30-1) is provided in the transmission device 100 'to correct the delay difference between the coded video signal and audio signal. Also, the receiving device 2
00 ′ is provided with a delay correction circuit 30 (30-2).
The delay difference between the decoded video signal and audio signal is corrected. In this case, in the delay correction circuit 30, the audio signal is fixedly delayed, or the delay amount is fixed according to the speed of the line interface.

[0005]

However, since the video encoding circuit 7 performs high-efficiency encoding according to the information amount of video data as described later, the delay is not constant. For this reason, depending on the video source, the video signal and the audio signal may not match with only the fixed delay correction, which may be unnatural.

SUMMARY OF THE INVENTION The present invention has been made to solve such a problem, and an object of the present invention is to achieve more natural delay synchronization between a video signal and an audio signal in the multiplex transmission system described above. It is to provide a possible delay correction scheme.

[0007]

In order to achieve the above object, a first invention (an invention according to claim 1) comprises a correcting means for correcting a delay difference between an encoded video signal and an audio signal.
In the column, this period is set at intervals of a predetermined frame.
Of the average value of the frame decimation factor of the video signal in
Then, the average value is compared with a predetermined threshold, and the average
For video signals whose value is larger than the threshold, increase the delay correction.
For video signals whose average value is smaller than the threshold,
It is provided with a frame rate calculating means for reducing the frame rate.
You. According to the present invention, the average value of the frame thinning number is equal to the threshold value.
For video signals larger than the value,
Video signal that frequently causes frame thinning (delay
Is large, the delay correction is increased. This
On the other hand, the average value of the frame decimation number is smaller than the threshold.
For video signals that are not
Video signal with a small delay (a video signal with a small amount of delay)
The delay correction is reduced.

[0008]

[0009]

[0010] The second invention (the invention according to claim 2) is the first invention.
In the present invention, the magnitude of the delay correction is switched only when the level of the audio signal is low. According to the present invention, only when the level of the audio signal is low,
The magnitude of the delay correction is switched.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail based on embodiments. FIG. 1 is a block diagram of a multiplex transmission system showing an embodiment of the present invention. In FIG. 1, reference numeral 100 denotes a transmitting device, and 200 denotes a receiving device, which are mutually connected via a transmission line (public line) 300.

[Transmitting Device 100] In the transmitting device 100, 1 is a video signal input terminal to which a video signal (analog signal) is input, 2 is an audio signal input terminal to which an audio signal (analog signal) is input, and 5 is a video signal. A / D conversion of the video signal 0105 from the signal input terminal 1 is performed, and CIF (Common Intermedi
ate Format), a video A / D conversion circuit that outputs video data 0507 in a format, 7 is a video encoding circuit that performs high-efficiency compression encoding of the video data 0507 and outputs video coded data 0709, and 6 is an audio signal input terminal. This is an audio A / D conversion circuit that A / D converts the audio signal 0206 from the A / D converter 2, further performs high-efficiency compression encoding, and outputs encoded audio data 0608.

Reference numeral 8 (8-1) denotes an encoding control information from the video encoding circuit 7 in which the audio encoded data 0608 is transmitted from the video encoding circuit 7 in order to adjust the delay between the encoded video data 0709 and the encoded audio data 0608.
A delay correction circuit for delaying according to 0708, 9 is a multiplexing circuit that multiplexes the video coded data 0709 and the delay corrected audio coded data 0809 and outputs multiplexed data 0910, and 10 is a multiplexed data 0910 to the transmission line 300. This is a transmission line interface circuit which converts the transmission line data 1016 into an appropriate line interface and outputs the transmission line data 1016.

[Receiving device 200] In the receiving device 200, reference numeral 11 denotes a receiving line interface circuit for outputting multiplexed data 1112 from received transmission line data 1611 transmitted via a transmission line (public line) 300; Data
1112 to video encoded data 1213 and audio encoded data 12
A demultiplexing circuit for demultiplexing the video data into 08, a video decoding circuit 13 for performing high-efficiency compression decoding of the video coded data 1213 and outputting the video decoded data 1314, and a D / A conversion of the video data 1314 in the CIF format for video Video D that outputs signal 1403
/ A conversion circuit.

8 (8-2) is a video signal 1403 and an audio signal 15
A delay correction circuit for delaying the audio coded data 1208 in accordance with the coding control information 1308 in order to match the delay with 04;
Reference numeral 5 denotes an audio D / A conversion circuit which performs high-efficiency compression / decoding of the delay-corrected audio encoded data 0815 and further D / A-converts and outputs an audio signal 1504. Reference numeral 3 denotes a video signal (analog signal) 1403.
Is an audio signal output terminal from which an audio signal (analog signal) 1504 is output.

[Delay Correction Circuit 8] FIG. 2 is a block diagram showing details of the delay correction circuit 8 (8-1) in FIG. In the figure, reference numeral 21 denotes a voice coded data 060 according to a write control signal 2521 (1) and a read control signal 2521 (2).
8 to control the delay amount, and
A memory circuit for outputting 809;
This is a silence detection circuit which judges silence of voice from 8 and outputs a silence control signal 2224 as logic "1" when there is no sound.

A frame rate calculation circuit 23 calculates an average coding frame rate from the coding control signal 0708, and outputs a frame rate control signal 2324 when the frame rate is large as logic "1". twenty two
Calculate the logical product of 24 and the frame rate control signal 2324,
An AND circuit 25 outputs a memory control signal 2425, and a memory control circuit 25 outputs a write control signal 2521 (1) and a read control signal 2521 (2) according to the memory control signal 2425.

The circuits other than the delay correction circuit 8 in FIG. 1 are well known to those skilled in the art, and are not directly related to the present invention. Hereinafter, a characteristic operation in the multiplex transmission system will be described with reference to a timing chart shown in FIG.

FIG. 3A shows, with time, video data 0507 that has been A / D converted by the video A / D conversion circuit 5 and then CIF-converted.
F2,...). FIG.
(B) shows, in chronological order, coded video data 0709 which has been subjected to high-efficiency compression coding by the video coding circuit 7 and in which video frames have been thinned out. The details of the CIF conversion and the high-efficiency compression encoding are described in ITU-T H.261 as an example, and thus the description is omitted here.

FIG. 3C shows video coded data 07 as an example of a coding control signal 0708 output from the video coding circuit 7.
It indicates the frame thinning number which is the frame thinning information of No. 09, and is an amount indicating how long the frame time of the coded frame has been efficiently encoded.

That is, when one frame of video data is highly efficient coded within one frame time, the decimation factor is set to "0", one frame of video data takes two frame times, and subsequent frame data is decimate. In this case, the thinning number is defined as "1".

In the frame rate calculation circuit 23,
The average value of the frame thinning numbers in this period is calculated every predetermined period (a period of 16 frames in FIG. 3), and when the average value is larger than a predetermined threshold value, the frame thinning is large (occurrence information The video source is determined to be a video source (having a large amount), and a frame rate control signal 2324 of logic “1” is output. When the average value is smaller than a predetermined threshold value, it is determined that the image source is a video source in which frame thinning is small (the amount of generated information is small), and a frame rate control signal 2324 of logic “0” is output (FIG. )).

On the other hand, the silence detection circuit 22 determines a level below a predetermined threshold as a silence signal from the contents of the audio coded data 0608, and only during that period, the silence control signal 22 of logic "1".
24 is output (FIG. 3 (e)). The AND circuit 24 calculates the logical product of the frame rate control signal 2324 and the silence control signal 2224, and outputs a memory control signal 2425 of logic "1" to the memory control circuit 25 when the frame source is a video source with large frame thinning and a silence period. (FIG. 3F).

The memory control circuit 25 has a memory control signal 24
According to 25, the magnitude of the delay of the memory circuit 21 is controlled.
Specifically, when the memory control signal 2425 is “1”, the memory circuit 21 is controlled so as to increase the delay, and when the memory control signal 2425 is “0”, the memory circuit 21 is controlled so as to reduce the delay.

The write control signal 2521 (1) and the read control signal 2521 output from the memory control circuit 25
The details of (2) and the operation of the memory circuit 21 are well known to those skilled in the art, and thus description thereof will be omitted.

As described above, in the present embodiment, the audio coded data 0608 is controlled by the memory circuit 21 so that the audio delay amount is large when the frame source is a video source with large frame thinning (when the video signal delay amount is large). In the case of a video source with a small frame thinning (when the delay amount of the video signal is small), the audio delay amount is controlled to be small, that is, the delay compensation of the video signal and the audio signal is performed by the statistical property of the video source ( This is adjusted according to whether the amount of generated information is large or small, etc.), so that the delay synchronization between the video signal and the audio signal can be more naturally achieved.

Further, the switching of the audio delay amount is performed only when the audio level is low and it can be determined that there is no audio by the silence detection circuit 22 and the AND circuit 24. Therefore, it is possible to eliminate unnaturalness such as audio discontinuity. Become. In the present embodiment, the period for calculating the average coding frame rate, the various threshold values, and the details of the delay correction amount can be arbitrarily determined, and the details are omitted because they are not essential to the present invention. Also, in the delay correction circuit 8-2 in the receiving device 200, the same operation as the delay correction circuit 8-1 in the transmitting device 100 is performed to adjust the delay between the video signal 1403 and the audio signal 1504 so that the audio encoded data 1208 is adjusted.
Delay.

[0028]

As is apparent from the above description, according to the present invention, the delay compensation of the coded video signal and audio signal is performed by using the average value of the frame decimation factor of the video signal and the threshold value.
Thus, the delay synchronization between the video signal and the audio signal can be more naturally achieved.
Further, according to the present invention, by switching the magnitude of the delay correction only when the level of the audio signal is low, it is possible to eliminate unnaturalness such as audio breaks.

[Brief description of the drawings]

FIG. 1 is a block diagram of a multiplex transmission system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing details of a delay correction circuit in the multiplex transmission system.

FIG. 3 is a timing chart for explaining the operation of the delay correction circuit.

FIG. 4 is a block diagram showing a conventional multiplex transmission system.

[Description of Signs] 1 ... video signal input terminal, 2 ... audio signal input terminal, 3 ... video signal output terminal, 4 ... audio signal output terminal, 5 ... video A / D conversion circuit, 6 ... audio A / D conversion circuit. , 7: video encoding circuit, 8 (8-1, 8-2): delay correction circuit, 9: multiplexing circuit, 10: interface circuit, 11: reception interface circuit, 12: demultiplexing circuit, 13: video decoding circuit,
14: video D / A conversion circuit, 15: audio D / A conversion circuit, 21: memory circuit, 22: silence detection circuit, 23: frame rate calculation circuit, 24: AND circuit, 25: memory control circuit, 100: transmission Device, 200 ... receiving device, 3
00: Transmission line (public line).

Continuation of the front page (58) Field surveyed (Int. Cl. ⁷ , DB name) H04B 14/04-14/06 H04J 3/00-3/22 H04N ⁷ /04-7/045

Claims (2)

(57) [Claims] 1. A multiplex transmission system that encodes, multiplexes, and transmits at least a video signal and an audio signal, wherein a delay difference between the encoded video signal and the audio signal is determined.
Correction means for correcting, wherein the correction means is provided at intervals of a predetermined frame period.
Calculating the average value of the frame thinning number of the video signal,
This average value is compared with a predetermined threshold value,
For video signals whose value is larger than the threshold, increase the delay correction.
However, in the case of a video signal in which the average value is smaller than a threshold value, delay compensation is performed.
Delay correction method characterized by comprising a frame rate calculation means for reducing a positive. 2. The method according to claim 1, wherein the correction unit is a delay unit.
Switching the size of the delay correction is performed when the audio signal level is low.
A delay correction method characterized in that only the correction is performed .