JP2648095B2 - Image encoding and decoding device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding and decoding apparatus, and more particularly to an image encoding and decoding apparatus used for a video conference or the like.

[0002]

2. Description of the Related Art A conventional image coding and decoding apparatus of this kind will be described with reference to the drawings.

FIG. 5 is a block diagram showing an example of a conventional image encoding device, and FIG. 6 is a block diagram showing an example of a conventional image decoding device.

[0004] In FIG. 5, this conventional image coding apparatus is disclosed in Japanese Patent Laid-Open No. 4-257184: "encoding apparatus", receives digital image data, and codes the digital image data. Encoding circuit 34 for highly efficient encoding for each encoded frame and outputting it as encoded image data, and a buffer memory for temporarily storing the encoded image data based on a write clock and reading it out as buffered image data at a predetermined reading speed 35, a delay circuit 38 for receiving the digital audio data, delaying the digital audio data, and outputting it as delayed audio data
And a multiplexing circuit 39 for multiplexing the buffer image data and the delayed audio data and transmitting the multiplexed signal as a multiplexed signal, wherein the delay circuit 38 is a delay amount variable delay circuit, A delay amount calculating circuit 37 for calculating the delay amount of the buffer image data as an average delay amount for each of N (N is an integer equal to or more than 2) encoded frames predetermined based on the digital audio data based on the average delay amount. And a delay control circuit 37 for changing the delay amount of the delay circuit 38 according to the input level of the delay circuit 38. Here, the delay amount control circuit 3
7: determining means for determining the presence or absence of the digital audio data based on the input level, and transmitting a determination signal;
First generating means for generating a delay control signal indicating a voice delay amount based on the average delay amount when the determination signal indicates no voice; Second generation means for generating a delay hold signal indicating the amount hold, and when receiving the delay hold signal, the delay circuit 38 holds the delay amount.

In FIG. 6, this conventional image decoding apparatus is disclosed in Japanese Patent Application Laid-Open No. Hei 4-82433: "Image Encoding Apparatus".
A separating circuit 42 for inputting the received signal a and separating it into digitally encoded image data b and digital audio data e, and a buffer memory for writing the digitally encoded image data b at a constant speed and temporarily storing the same. 43
A delay circuit 44 for inputting and delaying digital audio data e; a predictive decoding circuit 45 for efficiently decoding buffer memory output data c output from the buffer memory 43 and outputting digital image data d; A delay amount calculation circuit 4 for inputting a buffer memory read clock g generated in a decoded frame from the memory 43, calculating a delay amount of the digitally encoded image data, and outputting the calculated delay amount.
6 and the delay amount i of the digitally encoded image data from the delay amount calculation circuit 46, and the decoded frame N (N ≧ N)
A delay control circuit 47 for obtaining an average delay amount every (integer of 1) times and controlling the delay amount of the delay circuit 44 with the delay control signal j from the average delay amount.

[0006]

In the conventional image coding and decoding apparatus, since the image and audio delays are corrected by both the coding apparatus and the decoding apparatus, the respective apparatuses are provided with the same configuration. There is a problem that an audio delay circuit, a delay calculation circuit and a delay control circuit are required.

In this conventional example, a variable delay time is calculated from the relationship between the amount of data stored in a buffer for transmitting or receiving an image signal and the transmission speed, and the delay is calculated based on the result of adding a fixed delay that is fixed. When calculating the quantity, if the calculation method is complicated and realized by hardware, there is a problem that the circuit scale becomes large.

[0008]

According to the present invention, there is provided an image coding apparatus comprising: a coding circuit having a buffer for smoothing an image signal obtained by performing high-efficiency coding on an input moving image signal; An audio encoding circuit that encodes the input audio signal, and a multiplexing circuit that assembles a transmission line frame and multiplexes the image signal and the audio signal and sends the multiplexed image signal and the audio signal to a transmission line. A frame position of a multiplexing circuit is sampled and a frame head identification flag is inserted into the moving image signal when the frame head is at the head of the frame, and the frame head identification flag delayed by the encoding circuit and the multiplexing circuit is set to the frame position. Flag sending means to send to the transmission line
Have .

An image decoding apparatus according to the present invention comprises a demultiplexing circuit for establishing transmission frame synchronization from data transmitted via a transmission path and separating a multiplexed image signal and an audio signal, and a delay circuit for delaying the data. 2. An audio decoding circuit for decoding the decoded audio signal, decoding the image signal to obtain a reproduced moving image signal, and detecting a frame head identification flag inserted by the image encoding device according to claim 1. and an image decoding circuit, when the image decoding circuit detects the frame head identification flag, Ru determined decoding delay amount of the moving image signal from the time calculated from the frame position information of the demultiplexer Having delay amount calculating means,
The delay amount calculating means detects the level of the audio signal decoded by the audio decoding circuit, and controls the delay circuit when it is determined that there is no sound when the audio signal is lower than a predetermined threshold value.
The delay correction between the moving image signal and the audio signal is performed.

An image encoding apparatus according to the present invention encodes an input image signal, adds a flag signal to a header indicating the start of image encoding, and outputs image data. A buffer circuit for smoothing the amount of coding information from the circuit, a voice coding circuit for voice coding an input voice signal and outputting voice data, and assembling and multiplexing a transmission frame with the image data and the voice data. A multiplexing circuit for converting and transmitting, and a frame head detection circuit for notifying the image coding circuit of a flag signal for frame head identification when the transmission frame number is input from the multiplexing circuit and the head of the transmission frame is detected. And

An image decoding apparatus according to the present invention includes a demultiplexing circuit for separating multiplexed image data and audio data from data transmitted via a transmission line, and a demultiplexer for storing the demultiplexed image data. A buffer circuit that outputs the image data each time a header indicating the start of the frame is detected, and an image decoding that decodes the image data to output a reproduced image and detects a frame head identification flag to output a flag detection notification And a delay amount setting circuit that calculates a delay amount by sampling frame position information from the demultiplexing circuit when receiving the flag detection notification and calculates a delay amount in the case of silence from silence determination information. A delay circuit for delaying the separated audio data in accordance with the delay amount, decoding the delayed audio data to output a reproduced audio, and The lower the reproduced sound than the meta threshold was determined to silence and a speech decoding circuit for outputting the sound determination information.

[0012]

In general, the amount of delay generated in image encoding and decoding is not uniquely determined because it dynamically changes depending on the movement of an input image, the size and fineness of a subject, and the transmission speed. Therefore, there is a drawback in that a fixed delay correction method between an image signal and an audio signal or a method in which communication between an image and an audio signal is periodically interrupted to measure a delay cannot be accurately corrected.

The present invention is indispensable for multiplexing and transmitting images and voices, and always uses a multiplexed frame structure that allows a common time observation on the transmitting and receiving sides to reduce the amount of delay between images and voices at all times. By measuring and controlling the delay circuit of the received voice, dynamic delay correction can be performed.

In addition, it is possible to avoid a sound abnormality at the time of delay correction by detecting the silence of the received sound and controlling the delay circuit of the received sound when there is no sound.

[0015]

Next, the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing one embodiment of the present invention.

In FIG. 1, this embodiment comprises an image encoding device 1 and an image decoding device 2, and the image encoding device 1 encodes an input image signal 100, and includes a header portion indicating the start of image encoding. Image data 1 by adding flag signal 103
02, a buffer circuit 11 for smoothing the amount of encoded information from the image encoding circuit 10, and an audio signal for encoding the input audio signal 105 and outputting audio data 106. An encoding circuit 14, a multiplexing circuit 12 for assembling and multiplexing a transmission frame with the image data 102 and the audio data 106, and transmitting the transmission frame number 104 from the multiplexing circuit 12 to detect the head of the transmission frame And a frame head detection circuit 13 for notifying the image coding circuit 10 of a flag signal 103 for frame head identification in the event that the frame coding has been performed.
A demultiplexing circuit 20 for separating the multiplexed image data 200 and the audio data 207 from the data transmitted via the P.00, and a header indicating the start of the image encoding that stores the separated image data 200 is detected. Image data 2 for each
01 for outputting the image data 201
And an image decoding circuit 22 that outputs a reproduced image 203 and outputs a flag detection notification 202 by detecting a frame head identification flag, and a frame position from the demultiplexing circuit 20 when the flag detection notification 202 is received. A delay amount setting circuit 23 for calculating a delay amount by sampling the information 204 and calculating a delay amount 205 in the case of silence from the silence determination information 206;
7 according to the delay amount 205,
The audio data 208 after the delay is decoded to reproduce the audio 209.
And a speech decoding circuit 25 that outputs reproduced sound 209 lower than a predetermined threshold value as silence and outputs silence judgment information 206.

Next, the operation of this embodiment will be described.

The image encoding circuit 10 receives the input image signal 10
0 is input, and a flag signal 103 from a frame head detection circuit 13 to be described later is added to a header portion indicating the start of image encoding to perform high-efficiency encoding. High-efficiency coding is described in, for example, ITU-T Recommendation H.264. The information compression is performed by performing motion compensation interframe coding, DCT transform coding, quantization, and variable length coding adopted in H.261. The image data 101 obtained by the image encoding circuit 10 is input to a buffer circuit 11 for smoothing the amount of generated information, and the smoothed data 102 is input to a multiplexing circuit 12 described in detail later.

A voice encoding circuit 14 to which a voice signal 105 is input performs voice coding and outputs voice data 106 to a multiplexing circuit 12.

The multiplexing circuit 12 assembles a transmission frame and multiplexes the image data 102 and the audio data 106 for transmission. The frame start detection circuit 13 is
When the transmission frame number 104 is input and the head of the transmission frame is detected, the frame head identification flag 103
To the image coding circuit 10 described above. The image encoding circuit 10 inserts the information into the above-described header portion only when receiving the notification of the frame head identification flag. The frame head identification flag is delayed by the image encoding circuit 10, the buffer circuit 11, and the multiplexing circuit 12, and transmitted together with the image signal.

On the other hand, the demultiplexing circuit 20
The transmission frame synchronization is established from the data transmitted from, and the multiplexed image data 200 and audio data 207 are separated. The buffer circuit 21 stores the separated image data 200 and outputs the image data 201 to the image decoding circuit 22 every time a header indicating the start of image encoding is detected. The image decoding circuit 22 decodes the image, outputs a reproduced image 203, detects the frame head identification flag inserted in the header, and outputs a flag detection notification 202 to the delay amount setting circuit 23 when the flag is detected. I do.

The delay amount setting circuit 23 outputs a flag detection notification 2
02, the frame position information 204 notified from the demultiplexing circuit 20 is sampled to calculate the delay amount, and the silence determination information 2 from the speech decoding circuit 25 is sampled.
06, and outputs the delay amount 205 calculated in the case of silence to the delay circuit 24. The delay circuit 24 is a circuit for delaying the audio data 207 separated by the demultiplexing circuit 20 in accordance with the delay amount 205, and the delayed audio data 208
Is output to the audio decoding circuit 25. Voice decoding circuit 25
Performs decoding, outputs the reproduced voice 209, and observes the level of the reproduced voice 209. When the level is lower than a predetermined threshold value, it determines that there is no sound and outputs the silence determination information 206.

FIG. 2 is a diagram showing a frame bit position in this embodiment, FIG. 3 is a diagram showing a synchronization word and a frame / multiframe in this embodiment, and FIG. 4 is a diagram showing correspondence between multiframe numbers in this embodiment. FIG.

Next, the operation of the multiplexing circuit 12 and the demultiplexing circuit 20 will be described in detail with reference to FIGS. 1, 2, 3 and 4.

The multiplexing circuit 12 is provided, for example, in accordance with ITU-T Recommendation H. The transmission frame is constructed such that 80 octets adopted in H.221 and 1 frame constitute 80 frames, and 16 frames constitute 1 multiframe, and image data and audio data are multiplexed and transmitted at predetermined bit positions. On the other hand, the demultiplexing circuit 20 is based on the ITU-T Recommendation H. 221 frame synchronization is established, and multiplexed image data and audio data are separated.

In the following, ITU-T Recommendation H. A detailed description of the H.221 multiplexing method and an example of an automatic image and sound delay correction operation using this method will be described.

FIG. FIG. 221 is a diagram illustrating a frame bit position of frame 221. A frame bit (FAS) is inserted from the first to eighth octets of an eighth bit, and one octet constitutes one frame. Therefore, the time of one frame is 125 μsec × 80 = 10 msec.

FIG. 221 frame and a multi-frame configuration, and 16 frames (0F to 0F)
15F) forms one multi-frame. Therefore,
The time of one multiframe is 10 msec × 16 = 16
It is 0 msec. N1 to N4 represent multiframe numbers, and 16 multiframes (0MF to 15M
F) can be identified. FIG. 4 shows the correspondence between the multi-frame numbers.

With the above frame configuration, 10 msec
Time observation of up to 2560 msec is possible in steps.

The multiplexing circuit 12 of the image encoding apparatus 1 according to the present embodiment has a multi-frame number (MF) of 0 to 15 and 0
From the frame number (F) to the frame head detection circuit 1
3 is output to flag 0 and time is set to 0 at 0MF and 0F.
3 is input to the image encoding circuit 10 and transmitted to the image decoding device 2 after each processing.

On the other hand, when the time 0 flag inserted by the image encoding device 1 is notified to the delay amount setting circuit 23 of the image decoding device 2 of the present embodiment, the multi-frame number (MF) of the demultiplexing circuit 20 and By reading the frame number (F), the delay time D (MF, F) of the image can be calculated by equation (1).

D (MF, F) = 160 msec × MF + 10 msec × F (1) For example, when MF = 3 and F = 12, the delay time D (3,
12) is D (3,12) = 160 msecx3 + 10 msecx
12 = 600 msec.

The delay amount setting circuit 23 has a delay time D (M
(F, M), it is confirmed that there is no sound, and a delay time is set in the delay circuit 24. This is an operation of delaying the delay time setting until a silent state can be confirmed, and setting a new delay amount if the next delay time is obtained before the setting.

[0035]

As described above, the present invention provides an encoding circuit having a buffer for smoothing an image signal obtained by performing a high-efficiency encoding on a moving image signal inputted to an image encoding apparatus. , An audio encoding circuit for encoding an input audio signal, and a multiplexing circuit for assembling a transmission line frame, multiplexing an image signal and an audio signal, and sending the multiplexed image signal and the audio signal to the transmission line, and an encoding circuit and a multiplexing circuit. Demultiplexing for transmitting the frame head identification flag delayed by the above to the transmitter, and establishing a transmission frame synchronization from the data transmitted through the transmission path by the image decoding device and separating the multiplexed image signal and audio signal. Circuit, an audio decoding circuit for decoding the audio signal delayed by the delay circuit, and a decoding of the image signal to obtain a reproduced moving image signal, and a frame head identification inserted by the image encoding device. An image decoding circuit for detecting a lag, at the time when the image decoding circuit detects the frame head identification flag, the encoding and decoding delay of the moving image signal from the time calculated from the frame position information of the demultiplexing circuit. By calculating the amount, the amount of delay caused by image encoding and decoding that dynamically changes depending on the movement of the input image, the size and fineness of the subject, and the transmission speed, without interrupting the image and audio, And the delay of the audio can be automatically corrected, so that there is an effect that image communication with lip sync can be always obtained.

Further, by detecting the silence of the received voice and controlling the received voice delay circuit when there is no voice, it is possible to avoid voice abnormalities at the time of delay correction.

[Brief description of the drawings]

FIG. 1 is a block diagram showing one embodiment of the present invention.

FIG. 2 is a diagram showing a frame bit position in the embodiment.

FIG. 3 is a diagram showing a synchronization word and a frame / multiframe in the embodiment.

FIG. 4 is a diagram showing correspondence of multi-frame numbers in the embodiment.

FIG. 5 is a block diagram illustrating an example of a conventional image encoding device.

FIG. 6 is a block diagram illustrating an example of a conventional image decoding device.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 image encoding device 2 image decoding device 10 image encoding circuit 11 buffer circuit 12 multiplexing circuit 13 frame start detection circuit 14 audio encoding circuit 20 demultiplexing circuit 21 buffer circuit 22 image decoding circuit 23 delay amount setting circuit 24 Delay circuit 25 Voice decoding circuit

 ────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-2-305278 (JP, A) JP-A-6-343165 (JP, A) JP-A-4-100430 (JP, A)

Claims (5)

(57) [Claims] An encoding circuit having a buffer for smoothing an image signal obtained by performing high-efficiency encoding on an input moving image signal, and an audio encoding circuit for encoding an input audio signal. And a multiplexing circuit for assembling a transmission line frame, multiplexing the image signal and the audio signal, and transmitting the multiplexed image signal and the audio signal to a transmission line. the insert the frame start identification flag to the video signal when it is, a flag sent to sending the frame head identification flag which is delayed by the encoding circuit and the multiplexing circuit on the transmission line
Image encoding device characterized by having a means. 2. A demultiplexing circuit for establishing transmission frame synchronization from data transmitted via a transmission line and separating a multiplexed image signal and an audio signal, and decoding the audio signal delayed by a delay circuit. And an image decoding circuit for decoding the image signal to obtain a reproduced moving image signal and for detecting a frame head identification flag inserted by the image encoding device according to claim 1. the when the picture decoding circuit detects the frame head identification flag, slow seek decrypt delay amount of the moving image signal from the time calculated from the frame position information of the demultiplexer
An image signal device comprising a delay amount calculating means . 3. The delay amount calculating means detects a level of an audio signal decoded by the audio decoding circuit, and when the audio signal is lower than a predetermined threshold value, the delay circuit is activated when it is determined that there is no sound. 3. The image decoding apparatus according to claim 2, wherein the control is performed to correct a delay between the moving image signal and the audio signal. 4. An image encoding circuit which encodes an input image signal, adds a flag signal to a header indicating the start of image encoding, and outputs image data, and an image encoding circuit for encoding information from the image encoding circuit. A buffer circuit for smoothing the amount, a voice coding circuit for voice coding an input voice signal and outputting voice data, and a multiplexing circuit for assembling, multiplexing and transmitting a transmission frame with the image data and the voice data. When,
A frame head detection circuit for notifying the image encoding circuit of a flag signal for frame head identification when a transmission frame number is input from the multiplexing circuit and a head of the transmission frame is detected. Encoding device. 5. A demultiplexing circuit for separating multiplexed image data and audio data from data transmitted via a transmission path, and a header for storing the separated image data and indicating the start of image encoding. A buffer circuit that outputs the image data each time the image data is detected, an image decoding circuit that decodes the image data to output a reproduced image, detects a frame head identification flag, and outputs a flag detection notification, A delay setting circuit that samples frame position information from the demultiplexing circuit when receiving the detection notification to calculate a delay and calculates a delay in the case of silence from silence determination information; A delay circuit that delays the audio data according to the delay amount, decodes the delayed audio data to output a reproduced audio, and outputs a reproduced audio lower than a predetermined threshold. An audio decoding circuit for judging raw speech as silence and outputting said silence judgment information.