JPH0818975A - Encoder/decoder for dynamic image - Google Patents

Abstract

PURPOSE:To provide an encoding system and a decoding system with superior encoding efficiency even in a part where a motion is changing discontinuously by analyzing the object structure of an image and performing motion detection and motion compensation at every object area. CONSTITUTION:An analysis circuit 101 performs the area division of an input image 121, and outputs before and behind relational data 125 and motion data 124 at every area. A circuit 102 applies motion compensation to the composite image data 122 of a preceding frame by referring to the data 124, and outputs inter-motion compensation frame data 126 at every area. A differentiator 103 outputs the differential data 127 of input image data 123 and the data 126. An adder 106 adds data 128 in which the data 127 is quantized 104 on the data 128, and outputs composite image data 130. A decoder part performs the inverse code conversion 109 of encoded data 131 from a circuit 108, and performs the inverse quantization 110 of quantized data 132, and adds 112 it on inter-motion compensation frame predictive data from a circuit 111, and outputs a composite image 139 by synthesizing 114 image data 138 composited at every area.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to coding and decoding of moving images.

[0002]

2. Description of the Related Art FIG. 9 shows a block configuration of an embodiment for realizing a conventional moving picture encoding and decoding system. In the encoding unit of FIG. 9, the motion detection circuit 901 first inputs the input image 92
The motion between frames of 1 and the decoded image 922 locally decoded in the previous frame is detected. Next detected motion data 9
23, the motion-compensated interframe prediction data 924 is generated from the locally decoded image 922 of the previous frame. The series of processes of motion detection and motion compensation described above can be performed in data units of various sizes, but in general, they are often performed in block units of a predetermined size. For example,
ITU-T / H. In the international standard encoding method such as H.261 or ISO-IEC / MPEG, the input image is 16 pixels x 1
It is divided into blocks of 6 line units, and independent motion detection and motion compensation processing is performed for each block. In FIG. 9, next, the difference between the input image 921 and the motion-compensated inter-frame prediction data 924 is calculated in the same data unit as this motion compensation processing. The difference data 925 is quantized, code-converted together with the motion data 923, and transmitted to the decoding unit. The quantized data 926 is dequantized and added to the motion-compensated interframe prediction data 924 to obtain a locally decoded image 928.
The locally decoded image 928 is held in the memory 907, and is output as reference data when the input image of the next frame is encoded. In the decoding unit, the supplied data 929 is subjected to inverse code conversion, and the same motion compensation processing, dequantization processing and addition processing as in the encoding unit are performed using this data, and a decoded image 935 is obtained. The decoded image 935 is held in the memory 913 as one piece of image data, and is used as reference data when decoding the image of the next frame.

When motion compensation is performed on a block-by-block basis, block distortion may occur in a decoded image if the motions of adjacent blocks differ greatly. Therefore, in recent years, in order to remove this block distortion, a method of correcting motion data so that it continuously changes between adjacent pixels and using it for motion compensation is also known. For example, in the method described in IEICE Technical Report IE90-106 “Basic Study of Motion Compensation Using Triangular Patches”, first, representative points are set on an image at predetermined intervals, and movements between frames on the representative points are performed. Detect a vector. Next, a motion vector is interpolated and calculated from a plurality of neighboring representative points for every pixel. The motion-compensated inter-frame prediction is performed for each pixel with reference to the calculated motion vector. Even in this method, the decoded image is stored in the memory as one piece of image data and is used as reference data when encoding and decoding the image of the next frame.

[0004]

In the conventional moving picture coding and decoding methods, motion detection and motion compensation are performed by referring to one piece of decoded picture data. For this reason, the coding efficiency may be reduced near the boundary between subjects having different movements.

For example, when motion detection is performed in block units, an erroneous motion vector different from the motion of any subject region may be detected in a block spanning a plurality of subjects having different motions. In this block, motion compensation different from the actual motion is performed, and the effect of motion-compensated interframe prediction cannot be fully exerted. Further, in this block, motion compensation which is discontinuous with that of the adjacent block is performed, so that block distortion occurs in the decoded image, causing a noticeable deterioration. Alternatively, in a block that spans multiple subjects with different movements, even if the movement of any one subject area in the block can be accurately detected, accurate motion compensation can be realized for the remaining areas. However, the same problem as described above may occur.

As another example of the conventional motion compensation method, there is a problem in a method of detecting a motion vector on a predetermined representative point and interpolating and calculating the motion vector from every neighboring representative point for every pixel. May occur. in this way,
The motion vector is interpolated at all pixels regardless of the content of the image. Therefore, in the vicinity of the boundaries of subjects with different movements,
The interpolation calculation is performed with reference to the neighboring representative points in a region different from the pixel of interest, and a vector different from the actual motion is given. As a result, the effects of motion-compensated inter-frame prediction may not be fully exerted, or unnatural deformation distortion may occur at the boundary portion of the subject.

An object of the present invention is to realize a device having excellent coding efficiency even in an image portion in which the motion changes discontinuously, by performing motion detection and motion compensation in consideration of the subject structure of the image. Especially.

[0008]

In the first moving picture coding / decoding apparatus of the present invention, the coding section analyzes the input image with reference to the decoded image data locally decoded in the previous frame, and inputs it. While dividing the image into regions and outputting the divided regions, reference is made to the motion data, and means for detecting and outputting the data indicating the front-rear relationship between the divided regions and the data indicating the movement between the frames for each region, A unit for generating and outputting motion-compensated interframe prediction data from the decoded image data, and referring to the contextual data, an adaptive difference between the region-divided input image and the motion-compensated interframe prediction data For each, the means for outputting the difference data, the means for quantizing the difference data, and outputting the quantized data,
Dequantized the quantized data, a unit for outputting the dequantized data, the dequantized data and the motion compensation inter-frame prediction data is added for each region, the decoded image data locally decoded for each region A means for outputting, a means for holding the decoded image data separately for each area, and outputting as reference data at the time of encoding the next frame, a code conversion of the contextual data, the motion data and the quantized data. And a means for outputting, the decoding unit inverse-code-converts the encoded data supplied from the encoding unit, and outputs the quantized data and motion data for each area and the contextual data of the areas. , Means for dequantizing the quantized data and outputting the inverse quantized data, and motion-compensated inter-frame prediction data from the decoded image data decoded in the previous frame with reference to the motion data. A unit for generating and outputting the data, a unit for adding the dequantized data and the motion-compensated interframe prediction data for each region, and a unit for outputting decoded image data decoded for each region, and a unit for outputting the decoded image data. A unit for holding each area separately and outputting it as reference data when decoding the next frame and the context data are referred to, and the decoded image data decoded for each area is combined to generate one decoded image. And means for outputting.

In the second moving picture coding / decoding apparatus of the present invention, the coding section analyzes the input image with reference to the decoded image data locally decoded in the previous frame, and sets the input image as the moving area. A unit for dividing and outputting the background region and a unit for detecting and outputting data indicating the movement between frames in each of the divided regions, and a motion compensation frame from the decoded image data with reference to the motion data. Means for generating and outputting inter-prediction data, means for obtaining an adaptive difference between the region-divided input image and the motion-compensated inter-frame prediction data for each area, and outputting difference data, and the difference data Means for quantizing and outputting quantized data; means for dequantizing the quantized data and outputting dequantized data; and means for storing the dequantized data and the motion-compensated interframe prediction data. Means for outputting the decoded image data that is locally decoded for each area, and means for holding the decoded image data separately in the moving area and the background area and outputting as reference data when encoding the next frame A decoding unit that reverse-codes the coded data supplied from the coding unit to convert the motion data and the quantized data, and outputs the motion data and the quantized data. Means for outputting quantized data and motion data for each;
Means for dequantizing the quantized data and outputting the dequantized data; means for referring to the motion data and generating and outputting motion-compensated interframe prediction data from the decoded image data decoded in the previous frame; The dequantized data and the motion-compensated inter-frame prediction data are added for each area,
Means for outputting the decoded image data decoded for each area, means for holding the decoded image data separately in the moving area and the background area, and outputting as reference data when decoding the next frame, and decoded image data in the background area And means for synthesizing the decoded image data of the moving area and generating and outputting one decoded image.

[0010]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the first moving picture coding / decoding apparatus of the present invention will be described with reference to FIG.

FIG. 1 shows a first moving image encoding /
It is a block diagram of one Example of a decoding device. The moving picture coding / decoding apparatus is composed of a coding unit and a decoding unit.

The encoding unit analyzes the input image 121 by referring to the decoded image data 122 locally decoded in the previous frame, outputs the input image data 123 divided into regions, and the context and regions of the divided regions. A motion image analysis circuit 101 which detects a motion between each frame and outputs the detected data, and motion data of the decoded image data 122 of the previous frame is compensated for each region by referring to the motion data 124. The motion compensation prediction circuit 102 that outputs the inter prediction data 126, and the difference unit 103 that obtains the adaptive difference between the input image data 123 and the motion compensation inter prediction data 126 for each region and outputs the difference data 127.
And the difference data 127 is quantized to obtain the quantized data 128.
A dequantization circuit 104 that outputs the dequantized data 128 and dequantized the quantized data 128 and outputs the dequantized data 129, the dequantized data 129, and the motion-compensated interframe prediction data 126 for each region. Adder 10 that outputs the decoded image data 130 locally decoded for each area
6 and the decoded image data 130 are held separately for each area,
A region-by-region memory 107 that outputs as reference data that is different for each region when the next frame is encoded, and contextual data 12
5 and the motion data 124 and the quantized data 128 are code-converted, and the code conversion circuit 108 that outputs the coded data 131 to the decoding unit.

The decoding unit inverse-codes the encoded data 131 supplied from the encoding unit, and outputs the context data 134, the motion data 133, and the quantized data 132, and the quantized data. Dequantization circuit 1 for dequantizing 132 and outputting dequantized data 135
10 and the motion data 133, the motion-compensated prediction circuit 1 that performs motion compensation on the decoded image data 136 of the previous frame for each region and outputs the motion-compensated inter-frame prediction data 137.
11, the dequantized data 135, and the motion-compensated interframe prediction data 137 are added for each area, and the adder 112 that outputs the decoded image data 138 decoded for each area, and the decoded image data 138 are separated for each area. , And the decoded image data 138 for each area is synthesized by referring to the contextual data 134 and the area-based memory 113 that outputs separate reference data for each area when decoding the next frame, and one decoded image 139 Area synthesis circuit 114 for generating and outputting
Consists of

The operation of this embodiment will be described.

In the encoding unit of FIG. 1, the moving image analysis circuit 101 first divides the input image 121 into regions and outputs the divided regions. Here, more accurate division can be realized by using the inter-frame correlation between the input image 121 and the decoded image data 122 locally decoded in each region in the previous frame. The divided areas are output in association with the areas of the decoded image data 122. For example, the same method is used in the encoding unit and the decoding unit, and numbering is performed for each area of the decoded image data of the previous frame.
The area-divided input image data of the current frame is output by rearranging the areas corresponding to the numbers. In this way, it is not necessary to separately encode and transmit which area the encoded data for each area corresponds to. When a new area that does not correspond to any area of the decoded image data 122 is detected on the input image, it is added to the end of the output order.
The moving image analysis circuit 101 also detects the front-back relationship between the divided areas and the movement between frames of each area, and outputs each as the front-back relationship data 125 and the movement data 124.

FIG. 2 is a block diagram for explaining an example of the internal configuration of the moving picture analysis circuit of FIG. In FIG. 2, the area dividing circuit 201 divides the input image 221 into areas. An example of a series of processes for this area division will be described below. First, an edge in which the brightness or chromaticity is rapidly changing is detected from the input image 221, and the initial area division is realized by using this edge as a boundary line of the area division. Next, of the many areas obtained by the division, a plurality of areas overlapping one area on the decoded image data 222 are integrated and output as a final area division result. At this time, when a certain area on the input image 221 spans a plurality of areas on the decoded image data 222, the areas having the largest number of overlapping pixels are associated with each other. The motion detection circuit 202 of FIG. 2 detects the motion between frames for each of the divided areas. Motion detection can be easily performed by a block matching method or a brightness gradient method, which are conventional techniques. At this time, the image data for each region has only the pixel data of the region of interest, and does not include the other surrounding pixel data. Alternatively, the pixel data outside the attention area is fixed to one predetermined value, such as zero. By doing so, even when the motion is detected near the outer periphery of the attention area, more accurate motion detection can be realized without causing erroneous detection due to the influence of the image content of the adjacent area. In addition, since a part of the attention area is hidden behind another adjacent area or appears from the back on the contrary, there is no correlation between frames, and an accurate motion may not be detected. The motion data is extrapolated to such a portion from the portion where the motion data can be accurately detected in the same attention area. Alternatively, a zero vector may be given as the motion data of the portion. Figure 2
The front-rear relationship detection circuit 203 detects the front-rear relationship between the areas when the divided areas overlap each other. An example of this detection procedure will be described with reference to FIGS. 3 and 4. FIG. 3 shows a case where the two areas A and B overlap each other. In such a case, it can be determined that the one in which the region shape changes little between frames, that is, the region A is in front. Alternatively, it can be determined by measuring the motion compensation inter-frame prediction error for each region. FIG. 4 shows prediction errors when motion-compensated interframe prediction is performed assuming that each of the areas A and B is in the front. A large error occurs when the region B is assumed to be the front surface, but the error is small when the region A is assumed to be the front surface. Therefore, it can be determined that the area A is on the front side in the input image.

FIG. 5 is a block diagram for explaining another example of the internal structure of the moving picture analysis circuit of FIG. In FIG. 5, the motion detection circuit 501 first detects a motion between frames of the input image 521 and the decoded image data 522 locally decoded in the previous frame. Motion detection can be easily realized by the block matching method and the brightness gradient method, which are conventional techniques. Next, the area dividing circuit 502 divides the input image 521 into areas. The area division can be realized by the same method as described above with respect to the area division circuit 201 of FIG. 2, but more accurate area division can be realized by referring to the detected motion data. An example of this processing will be described below. Input image 5
21 is divided into initial areas by the same intraframe processing as described above. Next, using the detected motion data 523,
Motion compensation is performed on each area on the decoded image data 522.
Among the plurality of regions obtained by initially dividing the input image 521, the plurality of regions overlapping the same region after the motion compensation are integrated and output as a final region division result. At this time, when a certain area on the input image 521 extends over the plurality of areas after the motion compensation, the areas having the largest number of overlapping pixels are associated with each other. By using this method, it is possible to sufficiently utilize the inter-frame correlation for each area even in an image having a relatively large motion, and obtain a more accurate area division result. The context detection circuit 503 shown in FIG. 5 refers to the input image data 524, the decoded image data 522, and the motion data 523, which have been divided into regions, and detects the context between regions. The front-back relation between the areas is the front-back relation detection circuit 20 shown in FIG.
3 can be realized by the same method as described above.
The motion correction circuit 504 of FIG. 5 corrects the motion data 523 detected by the motion detection circuit 501 and outputs final motion data 526. The reason for adding the correction is that the motion detection circuit 501 detects the motion from the input image 521 before the region division, and therefore erroneous detection is likely to occur near the boundary between the subject regions having different motions. For example, when the motion is detected by the block matching method, a relatively accurate motion can be detected inside the subject region, but in a block straddling the subject region where the motion is different, the influence of the image content of the adjacent region other than the attention region, It may be erroneously detected. Therefore, the value of the motion data is extrapolated from the part where the motion can be accurately detected to the part where the possibility of erroneous detection is high. Alternatively, only the motion data of the portion is input image data 5 after the area division.
24 may be used for re-detection.

Now, returning to FIG. 1 again, the moving image analysis circuit 1
The operation after 01 will be described.

The motion compensation prediction circuit 102 of FIG. 1 refers to the motion data 124, performs motion compensation processing on the decoded image data 122 locally decoded for each region in the previous frame, and performs motion compensation interframe prediction data 126 for each region. Is output.
When the motion data 124 is supplied in block units, the motion compensation process is performed on a block basis. When the motion data 124 is supplied in block units or representative points, a motion vector for each pixel is interpolated from a plurality of motion data in the vicinity of the target pixel, and motion compensation is performed for each pixel using this. You can go. Or movement data 1
When 24 is given by a conversion parameter that describes the motion of the entire region of interest, conversion processing using the parameter is performed on all pixels in the region to obtain motion-compensated interframe prediction data 126.

The difference unit 103 refers to the contextual data 125 to adaptively obtain the difference between the region-divided input image data 123 and the motion-compensated inter-frame prediction data 126,
The difference data 127 is output. Area A on the input image
Before performing the difference processing on the difference image, the difference device 103 is supplied with the prediction data from the corresponding area A on the decoded image data 122. Here, the prediction data is the area A on the input image.
When it is given to the pixel position of, the difference data between the input image data and the prediction data is output. If the value of the prediction data for the pixel of interest on the input image is zero or if the prediction data is not given, the input image data is output as it is as difference data. On the other hand, when the prediction data is given to a pixel position outside the area A on the input image, that is, a pixel position on another area B on the input image, it is adaptive depending on the context of the areas A and B. The difference processing is switched to. When the area A is in front of the area B, the value of the input image data at the pixel position is set to zero, and the difference between this and the prediction data is output. When the area A is behind the area B, the value of the difference data is output as zero, or the difference data is excluded from the encoding target.
By doing so, it is possible to avoid unnecessary encoding of a portion that is hidden behind another adjacent area and cannot be seen on the input image of the current frame, that is, a so-called uncovered portion.

In the quantization circuit 104, the difference data 12
7 is quantized and quantized data 128 is output.

In the inverse quantization circuit 105, the quantized data 12
8 is inversely quantized, and inversely quantized data 129 is output.

In the adder 106, the dequantized data 129 and the motion-compensated interframe prediction data 126 are added separately for each area, and the decoded image data 130 locally decoded for each area is added.
Is output. Here, when the value of the dequantized data is zero or the motion-compensated inter-frame prediction data is given to the pixel which is not coded, the prediction data is directly used as the locally decoded image data. By doing this, for the part hidden behind another adjacent area and not visible on the input image of the current frame, the so-called uncovered part,
The image data locally decoded in the previous frame can be held without being erased. This image data can be utilized as effective data for motion-compensated interframe prediction when the portion appears again on the input image.

The regional memory 107 holds the locally decoded decoded image data 130 for each region separately, and separately for each region as reference data for motion detection and motion compensation inter-frame prediction at the time of encoding the next frame. Output.

In the code conversion circuit 108, the quantized data 128, the motion data 124, and the contextual data 1 for each area.
25 is code-converted, and the encoded data 131 is transmitted to the decoding unit.

In the decoding section, first, the inverse code conversion circuit 109 reverse-codes the encoded data 132 supplied from the encoding section, and the quantized data 132 and the motion data 133 for each area are converted.
And the context data 134 are output.

In the inverse quantization circuit 110, the quantized data 13
2 is inversely quantized, and inversely quantized data 135 is output.

In the motion compensation prediction circuit 111, the motion data 1
33, the decoded image data 136 decoded for each area in the previous frame is subjected to motion compensation processing, and motion-compensated interframe prediction data 137 is output. Motion compensation prediction circuit 11
The motion compensation processing in 1 is realized by the same method as that of the motion compensation prediction circuit 102 of the encoding unit.

In the adder 112, the inverse quantized data 135
And the motion-compensated inter-frame prediction data 137 are added separately for each area, and decoded image data 138 for each area is output. Here, when the value of the inverse quantized data is zero, or the motion-compensated inter-frame prediction data is given to a pixel that is not coded, the prediction data is directly used as the decoded image data.

The memory 113 for each area stores the decoded image data 13
8 is separately held for each area and is output as decoded image data 136 for each area when the next frame is decoded.

In the area synthesizing circuit 114, the context data 1
34, determine which region is more front,
The decoded image data 138 for each area is overlapped to synthesize one image. The combined image is output as a decoded image 139 to the outside.

A second moving picture coding / decoding apparatus of the present invention will be described with reference to FIG.

FIG. 6 shows a second moving image coding / encoding method according to the present invention.
FIG. 10 is a block diagram of an example of an apparatus that realizes a decoding apparatus. The moving picture coding / decoding apparatus is composed of a coding unit and a decoding unit.

The encoding unit analyzes the input image 621 by referring to the decoded image data 622 locally decoded in the previous frame, outputs the input image data 623 divided into the moving area and the background area, and also divides it. A motion image analysis circuit 601 which detects a motion between frames for each region and outputs motion data 624, and motion data of the decoded image data 622 of the previous frame with reference to the motion data 624 is motion-compensated for each region.
A motion-compensated prediction circuit 602 that outputs motion-compensated interframe prediction data 625, and region-divided input image data 62
3 and the motion-compensated inter-frame prediction data 625 are obtained for each region, a difference unit 603 that outputs difference data 626, and a quantization circuit 604 that quantizes the difference data 626 and outputs quantized data 627. And the inverse quantization circuit 605 that inversely quantizes the quantized data 627 and outputs the inverse quantized data 628, the inverse quantized data 628 and the motion compensation inter-frame prediction data 625 are added for each region, and each region is added. An adder 606 that outputs the decoded image data 629 that has been locally decoded, and a region that holds the decoded image data 629 separately in the moving region and the background region and outputs them as separate reference data for each region when the next frame is encoded. Memory 607
And a code conversion circuit 608 that performs code conversion of the motion data 624 and the quantized data 627 and outputs the coded data 630 to the decoding unit.

The decoding unit inverse-codes the encoded data 630 supplied from the encoding unit, and outputs the motion data 632 and the quantized data 631. The inverse-code conversion circuit 609 and the quantized data 631 are inversely quantized. , Dequantized data 633
And the motion compensation prediction circuit 611 that outputs motion compensation interframe prediction data 635 by performing motion compensation on the decoded image data 634 of the previous frame for each region by referring to the motion data 632. Quantized data 63
3 and motion-compensated inter-frame prediction data 635 are added for each area, and an adder 612 that outputs decoded image data 636 decoded for each area, and the decoded image data 636 are separately held in the moving area and the background area. , A memory 61 for each area which is output as reference data for each area when the next frame is decoded
3 and the decoded image data of the moving area on the decoded image data of the background area are combined to generate one image, and the decoded image 63
7 and an area synthesizing circuit 614 for outputting to the outside.

The operation of this embodiment will be described.

In the encoding unit, first, the moving image analysis circuit 601.
The input image 621 is analyzed by and the input image data 623 divided into a moving area and a background area is output. The area division is realized by utilizing inter-frame correlation with the decoded image data 622 which is locally decoded separately in the moving area and the background area in the previous frame. Further, the moving image analysis circuit 601 detects and outputs the movement between frames for each area. In addition,
If the output order of the moving area and the background area of each data is determined in advance, the decoding unit can uniquely determine which area the data belongs to.

FIG. 7 is a block diagram for explaining an example of the internal configuration of the moving image analysis circuit 601 shown in FIG. In FIG. 7, the area dividing circuit 701 divides the input image 721 into a moving area and a background area. An example of this processing will be described below. First, an edge in which the luminance or chromaticity is rapidly changing is detected from the input image 721, and this edge is used as a boundary line for area division to implement initial area division. Next, with reference to the decoded image data 722 locally decoded in the previous frame, the area division is corrected. For example, of the initial divided areas,
An area that overlaps the moving area on the decoded image 722 of the previous frame is integrated into a moving area in the current frame. Alternatively, the following method may be used. First, a simple interframe difference between the input image 721 and the decoded image 722 of the previous frame is obtained. A region in which pixels with large difference values are dense is regarded as a region boundary portion, and a plurality of initial divided regions inside the region boundary portion are integrated to form a moving region. In either case, the initial divided areas that do not belong to the moving area are integrated to form the background area. Next, the motion detection circuit 702 of FIG. 7 detects the motion between frames separately for the divided motion area and background area.
Motion detection can be easily realized by the block matching method and the brightness gradient method, which are conventional techniques. Input image 72 at this time
3 and the decoded image data 722 have only the pixel data of the region of interest and are not affected by the other region even in the vicinity of the region boundary, so that the motion can be detected more accurately.

FIG. 8 is a block diagram for explaining another example of the internal structure of the moving image analysis circuit 601 shown in FIG. In FIG.
The motion detection circuit 801 detects motion data 823 indicating the motion between frames from the input image 821 and the decoded image data 822 of the previous frame. Motion detection can be easily realized by the block matching method and the brightness gradient method, which are conventional techniques. Next, the area dividing circuit 802 divides the input image 821 into areas. The area division can be realized by the same method as described above for the area division circuit 701 of FIG. 7, but the motion data 8 detected by the motion detection circuit 801
By using 23, more accurate area division can be realized. An example of this processing will be described below. First input image 82
1 is divided into initial areas by intra-frame processing. Next, referring to the motion data 823, the decoded image data 822 of the previous frame
Is motion-compensated for each area. Among the plurality of initially divided regions, a plurality of regions overlapping the motion-compensated motion region are integrated to form a motion region in the current frame. In addition, the remaining initial divided areas are integrated to form a background area. Here, when a certain area which is initially divided extends over both the motion area and the background area after the motion compensation, the area is integrated into one having a larger number of overlapping pixels. Finally, the motion correction circuit 803 corrects the motion data 823 detected by the motion detection circuit 801.
The final motion data 825 is output. The motion detection circuit 801 detects motion on the input image before area division. Therefore, in the vicinity of the boundary between the moving area and the background area, mutual image data influence each other, and there is a high possibility that erroneous detection occurs. Therefore, the motion data detected inside the area is extrapolated to such a portion. Alternatively, only the motion data of the portion may be detected again by using the input image after the area division.

Returning to FIG. 6 again, the moving image analysis circuit 6
The operation after 01 will be described.

The motion compensation prediction circuit 602 refers to the motion data 624, performs motion compensation processing on the decoded image data 622 locally decoded for each region in the previous frame, and outputs motion compensation interframe prediction data 625 for each region. . When the motion data 624 is supplied in block units,
Motion compensation processing is performed on a block basis. When the motion data 624 is supplied in block units or representative points, a motion vector for each pixel is interpolated from a plurality of motion data in the vicinity of the target pixel, and motion compensation is performed for each pixel using this. You can go. Alternatively, when the motion data 624 is given by a conversion parameter that describes the motion of the entire region of interest, conversion processing using the parameter is performed on all pixels in the region, and the motion-compensated interframe prediction data 625 is obtained. To get

The difference calculator 603 adaptively finds the difference between the input image data 623 divided into regions and the motion-compensated interframe prediction data 625, and outputs difference data 626. Here, when the prediction data is given from the area corresponding to the input image data of interest, the difference data is output as it is. In other cases, the following processing is performed. In the difference processing in the moving area, if the value of the prediction data is zero or no prediction data is given to a certain pixel of interest on the input image, the input image data is output as it is as difference data. Similarly, in the differential processing in the moving area, when the prediction data is given to the pixel position outside the moving area on the input image, that is, the pixel position in the background area, the value of the input image data is set to zero and the prediction is performed. Output the difference from the data. On the other hand, in the difference processing in the background area, if prediction data is given to a pixel position outside the background area on the input image, the value of the difference data is output as zero, or the difference data is to be encoded. Outside. By doing so, it is not necessary to unnecessarily encode a portion hidden behind the moving area and invisible on the input image of the current frame, a so-called uncovered portion.

The quantization circuit 604 quantizes the difference data 626 and outputs quantized data 627.

In the inverse quantization circuit 605, the quantized data 62
7 is inversely quantized, and inversely quantized data 628 is output.

In the adder 606, the dequantized data 628 and the motion-compensated interframe prediction data 625 are added separately for each area, and the decoded image data 629 locally decoded for each area is added.
Is output. Here, when the value of the inverse quantized data is zero, or the motion-compensated inter-frame prediction data is given to a pixel that is not coded, the prediction data is directly used as the decoded image data. By doing so, the image data locally decoded in the previous frame can be retained without being erased in a part of the background region hidden behind the moving region and invisible in the current frame, that is, an uncovered part. This image data can be used as effective data for motion-compensated interframe prediction when the portion appears again on the input image.

The area-based memory 607 holds the decoded image data 629 locally decoded separately in the moving area and the background area,
When the next frame is encoded, it is output as reference data for motion detection and motion compensation inter-frame prediction which is different for each area.

The code conversion circuit 608 performs code conversion on the quantized data 627 and the motion data 624 for each area, and transmits the coded data 630 to the decoding unit.

In the decoding section, first, the encoded data 630 supplied from the encoding section is subjected to inverse code conversion by the inverse code conversion circuit 609, and the quantized data 631 and the motion data 632 for each area are converted.
And output.

In the inverse quantization circuit 610, the quantized data 63
1 is inversely quantized, and inversely quantized data 633 is output.

In the motion compensation prediction circuit 611, the motion data 6
32, the motion compensation processing is performed on the decoded image data 634 decoded for each area in the previous frame, and the motion compensation inter-frame prediction data 635 is output. Motion compensation prediction circuit 61
The motion compensation processing in 1 is realized by the same method as the motion compensation prediction circuit 602 of the encoding unit.

In the adder 612, the inverse quantized data 633
And motion-compensated inter-frame prediction data 635 are added separately for each area, and decoded image data 636 for each area is output. Here, when the value of the inverse quantized data is zero, or the motion-compensated inter-frame prediction data is given to a pixel that is not coded, the prediction data is directly used as the decoded image data.

The area-based memory 613 stores the decoded image data 63.
6 is held separately for the moving area and the background area, and is output as decoded image data 634 for each area when the next frame is decoded.

The area synthesizing circuit 614 superimposes the decoded image data of the moving area on the decoded image data of the background area to synthesize one image. This image is output as a decoded image 637 to the outside.

[0054]

As described above, according to the present invention, the object structure of the image is analyzed, and the motion detection and the motion compensation are performed for each object region, so that an excellent code can be obtained even in a portion where the motion changes discontinuously. It is possible to realize a coding and decoding method with high coding efficiency.

For example, the motion detection is performed using the input image divided into regions and the decoded image data locally decoded in each region in the previous frame. Therefore, when the movement of the attention area is detected, the movement can be detected more accurately without being affected by the image data of the other surrounding areas.

The motion compensation processing is also performed using the decoded image data decoded in each area in the previous frame. Therefore, the image data of the surrounding area is not erroneously mixed in the motion compensation inter-frame prediction data, and more accurate motion compensation processing can be realized.

Further, a part of the image data once decoded is
Even when the image data is hidden behind the adjacent regions having different movements, the image data of the portion can be retained in the region-based memory without being erased. This image data can be effectively used as motion-compensated interframe prediction data when the portion appears again on the input image. Therefore, the prediction efficiency can be improved as compared with the conventional method.

[Brief description of drawings]

FIG. 1 is a basic block diagram of a first embodiment according to the present invention.

FIG. 2 is a block diagram showing an example of an internal configuration of a moving image analysis circuit according to the first embodiment of the present invention.

FIG. 3 is a diagram illustrating a method of detecting a front-back relationship between divided areas.

FIG. 4 is a diagram illustrating a method of detecting a front-back relationship between divided areas.

FIG. 5 is a block diagram showing an example of an internal configuration of a moving image analysis circuit according to the first embodiment of the present invention.

FIG. 6 is a basic block diagram of a second embodiment according to the present invention.

FIG. 7 is a block diagram showing an example of an internal configuration of a moving image analysis circuit according to a second embodiment of the present invention.

FIG. 8 is a block diagram showing an example of an internal configuration of a moving image analysis circuit according to a second embodiment of the present invention.

FIG. 9 is a basic block diagram of an embodiment of a conventional moving picture coding and decoding apparatus.

[Explanation of symbols]

101,601 Moving image analysis circuit 102,111,602,611,902,911 Motion compensation prediction circuit 103,603,903 Difference device 104,604,904 Quantization circuit 105,110,605,610,905,910 Inverse quantum Circuit 106,112,606,612,906,912 adder 107,113,607,613 area memory 907,913 image memory 108,608,908 code conversion circuit 109,609,909 reverse code conversion circuit 114,614 Region synthesizing circuit 201, 502, 701, 802 Region dividing circuit 202, 501, 702, 801, 901 Motion detection circuit 203, 503 Front-rear relationship detection circuit 504, 803 Motion correction circuit 121, 221, 521, 621, 721, 821 9
21 Input image 122, 136, 222, 522, 622, 634, 7
22, 822, 922, 933 Decoded image data of previous frame 123, 223, 524, 623, 723, 824 Input image data for each area 124, 133, 224, 523, 526, 624, 6
32, 724, 823, 825, 923, 931 Motion data 125, 134, 225, 525 Front-rear relationship data 126, 137, 625, 635, 924, 934 Motion-compensated inter-frame prediction data 127, 626, 925 Difference data 128, 132 , 627, 631, 926, 930 Quantized data 129, 135, 628, 633, 927, 932 Dequantized data 130, 138, 629, 636 Decoded image data for each region 131, 630, 929 Encoded data 139, 637 , 928, 935 Decoded image

Claims (2)

[Claims] 1. A moving picture coding / decoding apparatus comprising a coding section and a decoding section, wherein the coding section analyzes an input image with reference to decoded image data locally decoded in a previous frame. , A unit for dividing the input image into regions and outputting the same, and detecting and outputting data indicating the front-back relation between the divided regions and data indicating the movement between frames in each region, and refer to the movement data And a means for generating and outputting motion-compensated interframe prediction data from the decoded image data, and an adaptive difference between the region-divided input image and the motion-compensated interframe predicted data with reference to the contextual data. For each region, and means for outputting difference data, means for quantizing the difference data and outputting quantized data, and means for dequantizing the quantized data and outputting dequantized data. Means, means for adding the dequantized data and the motion-compensated interframe prediction data for each area, and outputting decoded image data locally decoded for each area, and the decoded image data held separately for each area And a unit that outputs as reference data when encoding the next frame, and a unit that performs code conversion of the contextual data, the motion data, and the quantized data, and outputs, and the decoding unit includes: Means for inverse-code-converting the encoded data supplied from the unit, and outputting quantized data and motion data for each area and contextual data between areas; and dequantizing the quantized data to dequantize the data. Means for outputting the motion-compensated inter-frame prediction data from the decoded image data decoded in the previous frame with reference to the motion data, and the dequantization data. Data and the motion-compensated inter-frame prediction data are added for each area and the decoded image data decoded for each area is output, and the decoded image data is held for each area separately, and is referred to when decoding the next frame. With reference to the contextual data and the means for outputting as data, the decoded image data decoded for each of the areas is combined,
An encoding / decoding apparatus for a moving image, comprising: a unit for generating and outputting one decoded image. 2. A moving image encoding / decoding device comprising an encoding unit and a decoding unit, wherein the encoding unit analyzes an input image with reference to decoded image data locally decoded in a previous frame. , A unit for dividing the input image into a moving region and a background region and outputting the divided region, and detecting and outputting data indicating a movement between frames in each of the divided regions; and referring to the movement data, A unit for generating and outputting motion-compensated interframe prediction data from decoded image data, and an adaptive difference between the region-divided input image and the motion-compensated interframe prediction data is obtained for each region, and the difference data is output. Means, means for quantizing the difference data and outputting quantized data, means for dequantizing the quantized data and outputting dequantized data, the dequantized data and the motion compensation Inter-frame prediction data is added for each area, means for outputting the decoded image data locally decoded for each area, and the decoded image data is held separately in the moving area and the background area, and at the time of encoding the next frame. The decoding unit includes means for outputting as reference data, means for performing code conversion of the motion data and the quantized data, and outputting, wherein the decoding unit performs inverse code conversion of the encoded data supplied from the encoding unit. , Means for outputting quantized data and motion data for each area of the moving area and the background area, means for dequantizing the quantized data, and means for outputting dequantized data, and referring to the motion data, Means for generating and outputting motion-compensated interframe prediction data from decoded image data decoded in the previous frame, and adding the dequantized data and the motion-compensated interframe prediction data for each region Means for outputting the decoded image data decoded for each area, means for holding the decoded image data separately in the moving area and the background area, and outputting as reference data when decoding the next frame, and decoded image data in the background area And a means for synthesizing the decoded image data of the moving area and generating and outputting one decoded image.