JPH0620306B2 - Image encoding / decoding device - Google Patents

Description

TECHNICAL FIELD The present invention relates to an image encoding / decoding device used in a video conference, a video telephone, and the like.

[Conventional technology]

7 and 8 are block diagrams of an encoding unit and a decoding unit in a conventional image encoding / decoding device. In the figure, (1)
Is an input video signal, (2) is an A / D converter, (3) is a digital video signal, (14) is an interframe coding circuit, (15) is a locally decoded signal, and (16) is a locally decoded value. A frame memory that can store each video frame, (17) is a locally decoded value that has undergone frame delay, (18) is encoded information, (19)
Is a transmission buffer, (20) is a switch, (44) is a transmission buffer # 1, (45) is a transmission buffer # 2, (23) is a selector, and (25)
Is the transmission signal, (30) is the reception buffer, (46) is the switch, and (4
7) is the reception buffer # 1, (48) is the reception buffer # 2, (49)
Is a selector, (31) is a received coded signal, (32) is an interframe decoding circuit, (33) is a decoded signal, and (34) is a frame memory capable of storing the decoded signal in video frame units, (3
5) is a decoded signal which has undergone frame delay, (42) is a D / A converter, and (43) is a decoded video signal.

Next, the operation will be described. Input video signal (1) is A /
It is digitalized by the D converter (2). The interframe coding circuit (14) is a signal that has already been coded and locally decoded.
The digitalized signal (3) is interframe-encoded using an interframe prediction signal (17) formed by giving a frame delay to the frame memory (16) with respect to (15). The encoded information (18) becomes a transmission signal (25) via the transmission buffer (19) and is sent to the transmission line. Transmission buffer (19)
Has a double buffer structure, and includes a switch (20), a transmission buffer # 1 (44), a transmission buffer # 2 (45) and a selector (23).
Consists of Each of the buffers (# 1 and # 2) has a capacity enough to store the coding information for one video frame, and while one buffer is writing the coding information (18), the other By controlling the switch (20) and the selector (23) so that the buffer is reading and transmitting information, the transmission buffer (19) operates as if both writing and reading. When one video frame is coded and its information is written to one buffer, if the other buffer has finished reading, the writing / reading of both buffers is switched and the next image frame is coded. However, if another buffer is being read, the double buffer cannot be switched. Therefore, the next input video frame is not encoded. That is, the piece is played. In the decoding section, the reception buffer
Using the inter-frame prediction signal (35) formed by adding a frame delay to the already-decoded signal by the frame memory (34) for the received coded signal (31) via (30), Decoding is performed in the inter-decoding circuit (32). Decoded signal
(33) becomes a decoded video signal by the D / A converter (42). The reception buffer (30) has a double buffer structure, and includes a switch (46), a reception buffer # 1 (47), a reception buffer # 2 (48) and a selector (49). By controlling so that one buffer is reading while the other buffer is writing, the receiving buffer operates as if writing and reading at the same time. Writing / reading of the two buffers is switched in units of video frames. The above flow is shown in FIG. Both the transmission buffer and the reception buffer are switched in image frame units. The delay time from when a certain image frame is encoded and the encoded information is written in the transmission buffer to when the same image frame is decoded and reproduced is approximately one cycle of the encoded transmission cycle of one image frame and the time required for two cycles. Stipulated in. Therefore, when the transmission speed is low or the amount of information generated is large, it takes time to read the transmission buffer, and there is a risk that the delay time becomes large. Also, due to the frequent occurrence of frame frames, the time interval of encoded video frames becomes large, so that the inter-frame correlation becomes small and the inter-frame encoding efficiency decreases.

[Problems to be solved by the invention]

Since the conventional encoding / decoding device is configured as described above, the encoding processing unit needs to operate in real time in synchronization with the video signal, which increases the device scale and delay time. In addition, when a frame sound is generated, there are problems such as deterioration of folding due to sub-sampling in the time direction, deterioration of coding efficiency due to deterioration of inter-frame correlation, and increase of information generation amount.

The present invention has been made in order to solve the above-mentioned problems. By setting the encoding processing speed to a speed commensurate with the transmission capacity, the device scale is reduced, the delay time is shortened, It is an object of the present invention to provide an image coding apparatus capable of improving the coding efficiency by maintaining the inter-frame correlation even when an audio signal occurs and removing the aliasing component.

[Means for solving problems]

An image coding / decoding apparatus according to the present invention includes a control circuit for placing a buffer for each video frame between a preprocessing unit and a coding unit in an encoder and performing time integration of a video frame using the buffer. And a circuit for controlling the read / write to the buffer in the video frame unit according to the storage amount of the buffer for smoothing the transmission information amount,
A buffer for each video frame is placed between the decoding unit and the video output unit in the decoder.

[Action]

The addition of buffers in video frame units according to the present invention operates as follows. That is, since the encoding processing speed can be reduced according to the transmission speed, time division processing can be introduced to reduce the device scale. Since the video frame to be encoded is captured in the buffer, the amount of transmission information can be smoothed in small steps by intermittently encoding, and the delay time can be shortened. Further, the time integration function at the time of writing to the video frame buffer according to the present invention maintains the inter-frame correlation even when a frame sound is generated, and improves the coding efficiency by removing the aliasing, and the reproduced image quality. Has the effect of improving.

Example of Invention

An embodiment of the present invention will be described below with reference to the drawings. First
FIG. 3 is a block diagram showing a configuration example of the encoding unit according to the present invention. In the figure, (1) is an input video signal, (2) is an A / D converter, (3) is a digital video signal, (4) is a time integration circuit,
(5) is the front frame memory, (6) is the digital video signal (3) by sequentially reading the contents of the buffer on the write side of the front frame memory (5) having a double buffer structure.
And a signal in which the pixel positions on the video frame are matched, (7) time-integrated input signal, (8) switch, (9) front frame memory # 1, (10) front frame memory # 2. ， (11)
Is a switch, (12) is a control signal of the front frame memory, and (1
3) is the controller of the front frame memory, (14) is the inter-frame coding circuit, (15) is the locally decoded signal, (16) is the frame memory that can store the locally decoded value in video frame units, (17) is a locally decoded value subjected to frame delay, (18) is coding information, (19) is a transmission buffer, (20) is a switch, (21) is a transmission buffer # 1, and (22) is a transmission buffer #. 2, (23) is a selector, (24) is a signal indicating the accumulated amount and status of the transmission buffer having a double buffer structure, and (25) is a transmission signal. FIG. 2 is a block diagram showing a configuration example of the time integration circuit in FIG. In the figure,
(26) is a subtractor, (27) is an inter-frame difference signal, (28) is a weighting circuit, and (29) is a subtractor. FIG. 3 is a block diagram showing a configuration example of the decoding unit according to the present invention. In the figure, (2
5) is a transmitted signal, (30) is a received buffer, (31) is a received coded signal, (32) is an interframe decoding circuit, (33) is a decoded signal,
(34) is a frame memory that can store decoded signals in video frame units, (35) is a decoded signal that has undergone frame delay, (36) is a post-frame memory, (37) is a switch, (3)
8) Rear frame memory # 1, (39) is rear frame memory # 2, (40) is rear frame memory # 3, (41) is selector, (42) is D / A converter, (43) Is the decoded video signal. Also, FIGS. 4, 5, and 6 are explanatory views showing the operation of the embodiment of the present invention.

Next, the operation will be described. Input video signal (1) is A /
It is digitalized by the D converter (2). The time integration circuit (4) uses the signal (6) that sequentially reads the contents of the buffer on the write side of the front frame memory (5), which has a double buffer structure, and uses it for multiple frames of the input signal (3). The time-integrated signal is written into the buffer on the front writing side. In the time integration, the difference between the input signal (3) and the signal (6), that is, the interframe difference (27) is obtained, the value is converted by the weighting circuit (28), and the difference with the input signal (3) is calculated. The data is written in the buffer as a signal that is time-integrated in frame units. Now, input the signal of the n-th frame to Sn, the n-th
The signal that is written in the buffer after time integration up to 1 frame If the conversion by the weighting circuit (28) is represented by multiplication by k times, The time integration is represented by the calculation. The front frame memory (5) is provided with a switch (8) so that the video frame data is read out from another buffer and is encoded and transmitted while the video frame data is being written into one buffer while being time-integrated. The front frame memory # 1 (9), the front frame memory # 2 (10), and the switch (11) are operated. Switch
The frame memory read signal necessary for time integration of the encoded video frame signal and the writing side by switching by (11).
(6) and are output. The encoded video frame signal output from the read side of the front frame memory (5) is read intermittently by the control signal (12). This is for smoothing the information generation amount in small steps, which will be described in detail later. When one video frame is read out completely, the double buffer is switched, and the contents of the buffer on the side that has been time-integrated until then are read out as a coded video frame.
(See FIG. 4) That is, the time integration is interrupted by the switching of the double buffer, in other words, the capture of the encoded video frame. The number of video frames that are continuously time-integrated depends on how long it takes for a video frame during coded transmission. The encoding required time corresponds to the amount of information generated in the video frame. A large amount of information generation means a large motion in interframe coding. That is, the number of video frames that are continuously time-integrated increases in a scene with a lot of movement. That is, a blurred image in the time direction is captured. Therefore, the longer the interval between the coded video frames, the longer the time integration, so that the inter-frame correlation of the coded video frames is maintained and the inter-frame coding efficiency is improved. At the same time, it has the effect of eliminating aliasing due to subsamples in the time direction and the effect of noise reduction, and consequently improves the quality of the reproduced image. As explained previously, since the time integration is interrupted by the switching of the double buffer, it is necessary to set the output of the weighting circuit (28) to zero and refresh the contents of the frame memory when the time integration is newly started. is there. It is also effective that the parameter k described above is set to 0 immediately after the double buffer switching, and thereafter, the value is exponentially increased according to the input video frame sequence.

The encoded video frame that starts encoding by double buffer switching is a time-integrated video frame and is stored in the frame memory. Therefore, the encoding processing speed can be arbitrarily selected, which is low. By setting the processing time commensurate with the transmission speed, there is a margin to multiplex the circuit, and the device scale can be reduced.

The interframe coding circuit (14) uses the interframe prediction signal (17) formed by adding a frame delay to the already encoded and locally decoded signal (15) by the frame memory (16). , Inter-frame encoding the encoded video frame signal. The encoded information (18) becomes a transmission signal (25) via the transmission buffer (19) and is sent to the transmission line.

The transmission buffer (19) is a signal (24) indicating the accumulated amount and status.
Is sent to the controller (13) of the front frame memory, and the information generation amount is smoothed by indirectly stopping / restarting the encoding. The transmission buffer (19) has a double buffer structure. The manner of control is shown in FIG. Now, it is assumed that the video frame having the front frame memory # 1 (9) is being read. The read data is encoded and written in the transmission buffer # 1 (21). When the storage amount of the buffer (21) exceeds a preset threshold value Th1, the front frame memory controller (13) causes the front frame memory # 1.
Stop reading from (9). And send buffer # 2
When the accumulated amount of (22) decreases by reading and becomes 0, the double buffer of the transmission buffer (19) is switched. As a result of this switching, the transmission buffer # 2 is vacant, so reading from the front frame memory # 1 (9) is restarted and the coding information (18) is written in the transmission buffer # 2 (22).
The encoded information stored in the transmission buffer # 1 (21) is sent to the transmission line. In another case, the storage amount of the transmission buffer # 2 (22) being read may be less than Th2 before the storage amount of the transmission buffer # 1 (21) being written reaches Th1. Also in this case, first of all, the front frame memory # 1 (9)
The read buffer is stopped and the double buffer of the transmission buffer (19) is switched when the accumulated amount of the buffer (22) becomes zero. By the same switching, the frame memory # 1
Resume reading from (9). By repeating this,
Since a large amount of encoded information is sent to the transmission buffer (19) without being accumulated, the delay time is reduced. Also, the transmission buffer capacity can be small. The threshold value Th1 regulates the amount of information accumulated on one side of the transmission buffer and is determined from the delay time and the transmission rate. The threshold Th2 is a case where the amount of information is not sufficient for the transmission framing, so it is determined by the transmission rate and the format of the transmission framing.

In the decoding unit, the received coded signal (31) that has passed through the reception buffer (30) is converted into a frame memory that has already been decoded.
Decoding is performed in the interframe decoding circuit (32) using the interframe prediction signal (35) formed by giving a frame delay by (34). Decoded signal (33) is post frame memory
Written in (36). The post-frame memory (36) has a triple buffer structure. As shown in FIG. 6, while one buffer is reading according to the frame synchronization of the image output system, the other buffer is The decoded signal (33) is being written, and the remaining buffer is stopped. The third buffer is necessary because the writing of the received and decoded video frame signal and the reading according to the frame synchronization of the image output system are completely asynchronous, and neither interruption is allowed. The read signal from the rear frame memory (36) becomes the decoded video signal (43) by the D / A converter (42).

In the above control, the time from the reading of a video frame from the front frame memory (5) to the decoding and reproduction of the video frame is approximately one cycle of the coding transmission cycle of one video frame. Stipulated in. Therefore, the lower the transmission speed, the greater the effect of shortening the delay time compared to the conventional device.

〔The invention's effect〕

As described above, according to the present invention, the image frame to be coded is first written in the frame memory while being time-integrated, and thereafter the coded transmission is performed while the information generation amount is smoothed in small steps. Time can be shortened, and even if a frame sound occurs, there is no aliasing, and an image frame with inter-frame correlation can be encoded. Therefore, high quality images can be efficiently encoded and transmitted.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an image encoding / decoding device encoding unit according to an embodiment of the present invention, FIG. 2 is a block diagram showing the configuration of the time integration circuit in FIG. 1, and FIG. FIG. 4 is a block diagram showing a configuration of a decoding unit of an image encoding / decoding device according to an embodiment of the invention, FIG. 4 is an explanatory diagram showing an operation of a front frame memory according to an embodiment of the invention, and FIG.
FIG. 6 is an explanatory view showing the relationship between the front frame memory and the transmission buffer according to one embodiment of the present invention, FIG. 6 is an explanatory view showing the operation of the rear frame memory in one embodiment of the present invention, and FIG. FIG. 8 is a block diagram showing the configuration of an image encoding / decoding device encoding unit, FIG. 8 is a block diagram showing the configuration of a conventional image encoding / decoding device decoding unit, and FIG. 9 is a conventional image encoding / decoding device. It is explanatory drawing which shows operation | movement of the transmission buffer and the reception buffer of an apparatus. In the figure, (1) is the input video signal, (2) is the A / D converter, and (3)
Is a digital video signal, (4) is a time integration circuit, (5) is a front frame memory, (6) is a front frame memory writing side output, (7) is a time integrated signal, (8) is a switch, (9)
Is front frame memory # 1, (10) is front frame memory # 2, (11) is selector, (12) is control signal, (13) is front frame memory controller, (14) is interframe coding Circuit, (15) locally decoded signal, (16) frame memory, (17)
Is an inter-frame prediction signal, (18) is coding information, (19) is a transmission buffer, (20) is a switch, and (21) is a transmission buffer # 1,
(22) is a transmission buffer # 2, (23) is a selector, (24) is a transmission buffer status signal, (25) is a transmission signal, (26) is a subtractor, (27) is an interframe difference signal, and (28) ) Is a weighting circuit,
(29) is a subtractor, (30) is a receive buffer, (31) is a received coded signal, (32) is an interframe decoding circuit, (33) is a decoded signal,
(34) is a frame memory, (35) is an inter-frame prediction signal, and (3
6) is a rear frame memory, (37) is a switch, (38) is a rear frame memory # 1, (39) is a rear frame memory # 2.
(40) is the post frame memory # 3, (41) is the selector, (42)
Is a D / A converter, (43) is a decoded video signal, (44) is a transmission buffer # 1, (45) is a transmission buffer # 2, (46) is a switch, and (47) is a reception buffer # 1, (48). ) Is the receiving buffer #
2, (49) are selectors. In the drawings, the same reference numerals indicate the same or corresponding parts.

Claims (4)

[Claims] 1. A first and a second frame memory capable of storing at least one image frame of data, a time integration circuit for time-integrating image signal sequences continuously input in image frame units, A switching circuit for switching the connection so that the time-integrated image frame is written into the first or second frame memory and at the same time, reading is performed from the second or first frame memory which is not being written; A controller that controls writing and reading to and from the first and second frame memories, a third frame memory that can store data of at least one image frame, and the second or first frame memory Using the signal read from the input signal as the input signal, the already-encoded pixels are locally decoded and stored in the third frame memory. An interframe coding circuit for forming a measurement signal and performing interframe coding by a method capable of sequential coding / decoding in units of a fixed amount of pixels, and coding obtained by the interframe coding circuit First and second buffers capable of storing information, and a second or first buffer which is not writing at the same time as writing the sequentially encoded information into the first or second buffer. A switching circuit that switches the connection so as to read from the buffer, a third buffer that can sequentially store and read the received coded signal, and a third buffer that can store at least one image frame of data. 4 frame memory and the signal read from the third buffer as an input signal, the already decoded pixels are stored in the fourth frame memory to form an inter-frame prediction signal, And inter-frame decoding circuit for sequentially performing frame-time decoding based on the coding method implemented in serial interframe coding apparatus, the fifth and sixth capable of storing at least one image frame of data
And a seventh frame memory, and a decoded signal output from the inter-frame decoding circuit is written into the fifth, sixth or seventh frame memory, and at the same time, no writing is performed in the seventh or sixth frame memory. An image encoding / decoding device comprising: a switching circuit for switching the connection so that the decoded image frame stored in the fifth frame is read in synchronization with the frame synchronization of the image output system. 2. In the first or second frame memory for alternately writing the image frames time-integrated for a plurality of frame periods, the n-th (n positive integer) image frames are time-integrated. A subtractor for reading out an image frame and obtaining an inter-frame difference with the (n + 1) th image frame which is an input image signal sequence, a weighting circuit for weighting an output signal of the subtractor, and the weighted inter-frame difference A subtracter that obtains an image frame that is time-integrated up to the (n + 1) th image frame by subtracting from the (n + 1) th image frame. In the second frame memory, control is performed so that the image frame time-integrated up to the nth image frame is written to the stored one. Claims having a stage subsection (1) image encoding and decoding apparatus according. 3. A time integration process is initialized by setting the weight of the weighting circuit to 0 when the first or second frame memory for writing the image frame time-integrated for a plurality of frame periods is switched. The image coding / decoding apparatus according to claim (2), characterized in that it has means for changing weights according to successively input image frame sequences. 4. In a first or second buffer in which information sequentially encoded by an interframe encoding circuit is written alternately, the first amount of information stored on the writing side is predetermined. When the value becomes larger than the threshold value or when the amount of information left unread on the reading side for sending to the transmission line becomes smaller than the predetermined second threshold value. When, the controller for controlling the first and second frame memories,
In order to perform the encoding, the reading from the frame memory on the reading side is stopped, the writing to the buffer on the writing side is stopped, and the data is left on the buffer on the reading side. By immediately switching the first and second buffers when the amount of information becomes 0, the reading from the buffer is continuously performed, and the reading from the frame memory and the encoding are controlled to be restarted. Then, the amount of generated information is smoothed,
The image coding / decoding apparatus according to claim (3), which outputs the coding information to the transmission path at a constant speed.