JPH06133288A - Moving picture coding display system - Google Patents

Abstract

PURPOSE:To effectively prevent disturbance of a picture by using a motion vector of a block near a block decoded just precedingly in place of a block in which an error takes place so as to compensate the motion and displaying the result. CONSTITUTION:In the independent coding mode, a switch 111 is open and a difference device 102 is not in use, then an input picture is subject to discrete cosine transformation in an orthogonal transformation device 103. A transformation coefficient is fed to a digital storage medium or the like from a terminal 106 via a quantization device 104 and a variable length coder 105. In the case of the prediction coding mode, a difference between the input picture and a predicted picture is formed by the difference device 102, transformed, quantized and coded and the result s outputted. The prediction picture in the mode is formed by a local decoding picture stored in a frame memory and prediction device 110. The local decoding picture is a picture decoded locally by an inverse quantization device 107 and an orthogonal inverse transformation device 108. The picture is formed by adding other local decoded picture stored in the memory 110 and a predicted picture by the prediction device 110 at an adder 109. Thus, the improved picture quality is obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention sets an independent frame from continuous frames of image signals continuously input,
These independent frames are independently encoded within a frame, a prediction signal of a non-independent frame between the independent frames is formed based on signals of preceding and succeeding independent frames, and the signal of the non-independent frame corresponds to the corresponding In order to decode the signal of the inter-frame predictive coding method encoded based on the prediction signal, the first decoding means for independently decoding the independent frame within the frame and the non-independent frame from the prediction signal of the non-independent frame The present invention relates to a display method of an adaptive interframe predictive decoding method provided with a second decoding means for decoding. Such a method can be applied to a videophone, a video conference system, an MPEG method, and the like.

[0002]

2. Description of the Related Art In order to achieve high efficiency coding by reducing the code amount when coding a moving image signal, interframe predictive coding is performed by utilizing the correlation between frames of the image signal.
This is because a normal moving image is quite similar between each frame, so the signal of the frame to be encoded can be predicted from the signal of the previous frame that was encoded, and the signal between the prediction result and the actual image can be predicted. Only the prediction error (residual error) is encoded. The code amount can be further reduced by combining this predictive coding with orthogonal transform and variable length coding. However, if there are errors or errors in decoding the prediction error, these errors may accumulate and a large difference may occur between the original signal and the decoded signal.

Therefore, independent frames are set at predetermined intervals in continuous frames of continuous image signals, and these independent frames are independently coded within the frame.
Then, the non-independent frame to be predictively coded forms a prediction signal based on the signals of these independent frames, and codes based on that. If an error occurs when decoding the image signal encoded in this way, the frame including this error cannot form a proper image. Therefore, various methods have been proposed to compensate the error in such a case. According to Hamano et al., "Study of ATM-compatible variable rate image communication", the SAT90-55 communication technique, when an error occurs, the image is divided into important blocks and non-important blocks, and the DC coefficients of the important blocks are roughly quantized. Compensation is performed by substituting with data and clearing the AC coefficient to 0, and non-important blocks are compensated by substituting with data of the previous frame.

According to the invention disclosed in Japanese Patent Laid-Open No. 63-016767, at the time of decoding encoded color image data, at least one of a plurality of color element signals forming a color image has an error. Occurs, the erroneous color component signal is replaced with another color component signal. Also,
According to the invention disclosed in Japanese Patent Laid-Open No. 63-016768, at the time of decoding similarly encoded color image data, at least one of a plurality of color element signals forming a color image is sub-scanned. When an error occurs in a block in the line direction, the error block is replaced with the image signal of the adjacent block.

[0005]

In the known technique described in the above-mentioned "Consideration of ATM compatible variable rate image communication", the procedure of appropriately compensating the DC coefficient and the AC coefficient respectively is rather complicated, and therefore non-existent. This is done in a simplified way for the important blocks, thus necessitating the extra means of dividing the image into important and non-important blocks, further complicating the technique. Since the error compensation does not include a dynamic element, it is inevitable that an error occurs and the motion of the compensated image is disturbed. The known techniques described in JP-A-63-016767 and JP-A-63-016768 are all related to a still image and can be used for compensation because they are processes performed only within one frame. The amount of information is limited, and high quality video compensation cannot be expected. Therefore, an object of the present invention is to provide a simple and high-quality error compensation that can prevent image distortion when an error occurs without changing the encoding system when compensating for an error in the interframe predictive encoding system. It is to provide a decoding method.

[0006]

According to the invention described in claim 1, this object is achieved as follows. That is, independent frames are set from consecutive frames of image signals that are continuously input, these independent frames are independently coded in the frames, and prediction signals of non-independent frames between the independent frames are set to the independent frames before and after. Independent frames are formed independently within a frame in order to decode a signal of the non-independent frame and a signal of an inter-frame predictive coding method in which the signal of the non-independent frame is encoded based on the corresponding prediction signal. In the display method of the adaptive inter-frame predictive decoding method, which is provided with the first decoding means for decoding the non-independent frame and the second decoding means for decoding the non-independent frame from the prediction signal of the non-independent frame, an error occurs during decoding Sometimes, instead of the block in error in the frame, the corresponding block in the last decoded frame until the next decodable block is decoded. , And motion compensation is displayed using a motion vector of neighboring blocks of the corresponding block. In that case, if the block that has been motion-compensated using the motion vector of the corresponding block of the previous frame and the neighboring block that could be decoded is compared with multiple blocks in the vicinity of the relevant block of the current frame, and the one with the closer correlation is displayed. It is advantageous.

According to the invention described in claim 3, the above object is achieved as follows. That is, independent frames are set from consecutive frames of image signals that are continuously input, these independent frames are independently coded in the frames, and prediction signals of non-independent frames between the independent frames are set to the independent frames before and after. Independent frames are formed independently within a frame in order to decode a signal of the non-independent frame and a signal of an inter-frame predictive coding method that is encoded based on the corresponding prediction signal. In the display method of the adaptive inter-frame predictive decoding method, which is provided with the first decoding means for decoding the non-independent frame and the second decoding means for decoding the non-independent frame from the prediction signal of the non-independent frame, an error occurs during decoding Sometimes, the corresponding blocks of the immediately preceding decoded frame and the immediately following decodable frame are interpolated and displayed.
In that case, it is advantageous to use the corresponding block of the immediately decodable frame for motion compensation using a motion vector during interpolation.

[0008]

It is apparent that the motion vector can be detected in each block that is normally decoded during decoding. These motion vectors are considered to represent the amount of change between two consecutive frames, so to speak. This motion vector cannot be detected for a block in which an error has occurred, but if the average value of the motion vectors of blocks in the vicinity of this block, such as two adjacent blocks, is calculated, the motion vector of the block in which the error occurred can be approximated. The value is obtained. Therefore, the block in which the error has occurred can be estimated with considerable accuracy by performing motion compensation on the corresponding block of the previous frame using the approximate value of the motion vector.
Substituting this estimated block for the erroneous block provides high quality error compensation.

[0009]

Embodiments of the present invention will be described in more detail below with reference to FIGS. FIG. 1 schematically shows an example of an encoder used in a moving image coded display method according to the present invention. The coding mode is roughly divided into two modes: an independent coding mode and a non-independent coding mode. In the independent coding mode, the input image is coded as it is. In the non-independent coding mode, that is, the predictive coding mode, the predicted image is formed by referring to the already coded image in any of the three predictive modes of before, after, or these interpolations. The actual image difference is encoded. The encoder is a differentiator 1 that forms the difference between the input image and the predicted image.
02, orthogonal transformer 103 for reducing code amount, quantizer 1
04 and variable length encoder 105. Independent coding can be performed only by this, but since predictive coding is performed, the inverse quantizer 10 is additionally provided after the quantizer 104.
7, the orthogonal inverse transformer 108, the adder 109, and the frame memory & predictor 110 are connected, and the output terminal of the frame memory & predictor 110 is connected to the difference unit 102 and the adder 109 via the switch 111. Has been done.

In the independent coding mode, the switch 111
Is opened and the subtractor 102 is disabled, the image signal input from the input terminal 101 is sent to the orthogonal transformer 103 as it is. The input image is discrete cosine transformed here, and the resulting transform coefficients are quantized by a quantizer 104 and variable length coded by a variable length encoder 105,
It is sent from the output terminal 106 to a digital storage medium (DSM) or the like. In the case of the predictive coding mode, the difference from the predicted image according to the predictive mode is formed by the differentiator 102,
This difference is converted, quantized and encoded in the same manner as in the case of encoding an independent frame, and is output. The predictive image in the predictive coding mode is formed by the predictor from the locally decoded images accumulated in the frame memory of the frame memory & predictor 110. These locally decoded images are images locally decoded by the inverse quantizer 107 and the orthogonal inverse transformer 108 connected after the quantizer 104. This decoded image is used as a predictor 11 from another locally decoded image that has already been created and is stored in the frame memory 110.
The predicted image formed by 0 is added by the adder 109 to be formed. The formed image is stored as a predicted image in the frame memory 110 and is continuously used for forming a predicted image in the future.

FIG. 2 shows a schematic configuration of the decoding device. The decoding apparatus includes a variable length decoder 113 and an inverse quantizer 1
14, an orthogonal inverse transformer 115 and a decoder 116, the operation of which is basically the same as that of the local decoding part of the encoder. That is, the data input from the input terminal 112 is decoded by the variable length decoder 113,
Inverse quantizer 114 inversely quantizes and orthogonal inverse transformer 115
Is inverse discrete cosine transformed by. This signal is the decoder 11
In step 6, the original image signal is restored, and if there is an error, the error is compensated and output.

FIG. 3 shows the detailed structure of the decoder 116. The decoder 116 includes a decoder unit 1, a motion compensation unit 2, a work memory 3, frame memories 4, 6, 8, 10 for storing decoded images, memory units 5, 7, 9, 11, VRAM for storing motion vectors. DM to transfer data to
It is composed of an A section 12, a VRAM section 13, and a control section 14, and these pixels are connected to each other via a main bus in the present embodiment. FIG. 4 is a flowchart showing the operation of the decoder 116, and FIG. 4 (a) shows the normal operation. First, the bit stream data enters the decoder unit 1 where the difference data between the motion vector and the image is decoded (step 21). Here, the end of the process is determined (step 22), and if not completed (step 22; N), the following operation is maintained. The decrypted data is temporarily stored in the work memory 3. The data in the work memory 3 is sent to the motion compensator 2, where motion compensation is performed (step 23), and the image data formed at that time is sent to the frame memories 4, 6, 8, 10. The motion vector is sent to the memory units 5, 7, 9, 11 which store the motion vector. Then, the image data in the frame memory units 4, 6, 8 and 10 are divided to form pixel blocks (step 24). When the image data for one sheet is accumulated, the DMA unit 12 is activated, and these data are transferred to the VRAM and displayed.

FIG. 4B shows the decoder 116 when an error occurs.
Shows the operation of. When an error occurs, the control unit 14
Transfers the image data and motion vector of the immediately preceding decoded image to the work memory 3. The image data of the block corresponding to the block in which the error has occurred is selected from these image data, and the motion vector of the block in the vicinity of the relevant block is selected from the motion vectors at the time of the error. One motion vector is calculated from the plurality of selected motion vectors by an appropriate calculation (step 25).
The motion compensation unit 2 performs motion compensation on the selected image data with this motion vector (step 26).

The motion-compensated image data is replaced with the image data of the corresponding block of the image in which the error has occurred to form the pixel block of that block (step 27). The operation then returns to normal operation. All these operations are controlled by the control unit 14. FIG. 5 shows an example of calculating a motion vector of a block in which an error has occurred. When an error occurs, the motion vector is lost together with the image data of the block. Therefore, the motion vector is calculated from the motion vectors of neighboring blocks.
Here, the motion vector of the block on which the error has occurred and the motion vector of the left block are extracted, and the motion vector is calculated by taking the arithmetic mean of these motion vectors.

[0015]

According to the present invention, the image data to be filled in the place where the error occurs is the compensation data by the motion compensation, so that the image quality is much improved as compared with the conventional method. The error compensation according to the present invention does not need to change the encoding method, and can prevent the image from being disturbed when an error occurs only on the decoding side. Moreover, the structure of the device is much simpler than the conventional one. Either the invention according to claims 1 and 2 or the one according to claims 3 and 4 may be optimal depending on the nature of the moving image at the time when an error occurs. The image quality can be further improved by selectively applying the above method.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of an encoder used in a moving image coded display system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing an embodiment of a decoding device used in the moving picture coded display system according to an embodiment of the present invention.

FIG. 3 is a block diagram showing an embodiment of a decoder that implements a moving image coded display method according to an embodiment of the present invention.

FIG. 4 is a flowchart showing an operation of the decoder of FIG.

FIG. 5 is a diagram showing a method of calculating a motion vector of a block in which an error has occurred.

[Explanation of symbols]

 1 Decoder Section 2 Motion Compensation Section 3 Work Memory 4, 6, 8, 10 Frame Memory 5, 7, 9, 11 Memory 12 DMA Section 13 VRAM Section 14 Control Section 103 Orthogonal Transformer 104 Quantizer 105 Variable Length Encoder 107 Inverse quantizer 108 Orthogonal inverse transformer 110 Frame memory & predictor

Claims (4)

[Claims] 1. An independent frame is set from consecutive frames of image signals continuously input, these independent frames are independently coded within the frame, and a prediction signal of a non-independent frame between the independent frames is set, The independent frames are formed based on the signals of the independent frames before and after, and the independent frames are decoded in order to decode the signal of the non-independent frame based on the prediction signal corresponding to the signal of the interframe predictive coding method. In a display method of an adaptive inter-frame predictive decoding method, a first decoding means for independently decoding in a frame and a second decoding means for decoding a non-independent frame from a prediction signal of a non-independent frame are provided. When an error occurs in the frame, the corresponding block of the immediately preceding frame is decoded until the next decodable block is decoded instead of the block in which the error occurred in the frame. The click, the moving picture coding display method and displaying by motion compensation using the motion vectors of neighboring blocks of the corresponding block. 2. The method according to claim 1, wherein a block that has been motion-compensated by using a motion vector of a corresponding position block of a previous frame that has been decoded and a neighboring block, and a plurality of blocks near the corresponding block of the current frame are A moving image coding display method characterized by comparing and displaying the one with a closer correlation. 3. An independent frame is set among consecutive frames of image signals that are continuously input, these independent frames are independently coded within the frame, and a prediction signal of a non-independent frame between the independent frames, The independent frames are formed based on the signals of the independent frames before and after, and the independent frames are decoded in order to decode the signal of the non-independent frame based on the prediction signal corresponding to the signal of the interframe predictive coding method. First decoding means for independently decoding in a frame, and second decoding means for decoding a non-independent frame from a prediction signal of the non-independent frame
In the display method of the adaptive inter-frame predictive decoding method, which is provided with the decoding means of (1), when the error occurs during decoding, the corresponding blocks of the immediately preceding decoded frame and the immediately following decodable frame are interpolated and displayed. Video coding display method. 4. The moving picture coded display method according to claim 3, wherein the corresponding block of the immediately decodable frame is motion-compensated using a motion vector at the time of interpolation.