JP2500439B2 - Predictive coding method for moving images - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to motion-compensated interframe predictive coding of moving images.

[0002]

2. Description of the Related Art In the prior art, it is general to divide an input image into blocks of a predetermined size and execute motion compensation inter-frame prediction in block units.
As an example, in the moving picture coding / decoding method (ISO-IEC IS11172) internationally standardized by ISO-IEC / JTC1 SC29, motion-compensated interframe prediction processing is executed in fixed-size blocks of 16 pixels and 16 lines. .

[0003]

In the conventional motion-compensated inter-frame predictive coding system, the motion-compensated inter-frame prediction process is performed by dividing the block into blocks of a predetermined size regardless of the contents of the image. Therefore, even if there is a discontinuous movement change within a frame, it can be dealt with only in block units. Since the position of each block is fixed regardless of the position of the subject in the image, a block that spans between subjects with different movements can only respond to the movement of one subject at most and the prediction efficiency decreases. was there.

[0004]

SUMMARY OF THE INVENTION An object of the present invention is to subdivide a block that spans a plurality of subjects having different movements,
An object of the present invention is to provide a motion-compensated interframe predictive coding method for a moving image with high prediction efficiency by performing motion-compensated interframe prediction using different motion vectors for each of the divided regions.

A predictive coding method for a moving image according to the present invention, in motion-compensated interframe predictive coding of a moving image, means for detecting a boundary line between different subject regions from an input image, and a frame for a reference image of the input image. Means for detecting the inter-motion vector and the motion-compensated inter-frame prediction error on a block-by-block basis, and a plurality of blocks adjacent to the block that are subdivided at the boundary line with respect to the block having the prediction error larger than a predetermined threshold value. Means for determining the effectiveness of separately predicting motion-compensated inter-frame prediction of the divided portion using the motion vector of the block, and using the determination result by the motion vector, the boundary line, and the determination means, Means for performing adaptive motion compensation interframe predictive coding in block units or in subdivided blocks Characterized in that it comprises.

According to the predictive coding method for a moving picture of the present invention, in the motion-compensated inter-frame predictive coding for a moving picture, the inter-frame motion vector and the motion-compensated inter-frame prediction error for the reference picture of the input picture are detected in block units. Means for subdividing a block having a prediction error larger than a predetermined threshold value and using the motion vectors of a plurality of blocks adjacent to the block to separately predict the motion-compensated inter-frame Detecting the division boundary line that minimizes the prediction error, and determining the effectiveness of the subdivision of the block based on the minimum prediction error, the motion vector, the boundary line and the determination result by the determination means. Using adaptive motion compensation inter-frame prediction in block units of the input image or in sub units of subdivided blocks. Characterized in that it comprises a means for performing the coding.

[0007]

The first motion-compensated inter-frame predictive coding system for moving pictures according to the present invention will be described with reference to FIG. First, the area boundary detection circuit 101 detects the boundary lines of a plurality of subject areas from the input image 121 and outputs it as area boundary information 122.

The motion detection circuit 102 detects the inter-frame motion vector 124 of the input image 121 with respect to the reference image 123 in block units of a predetermined size. When the detected motion vector 124 is used, the motion compensation inter-frame prediction error data 125 of the block is output at the same time.

The block division decision circuit 103 is arranged to
For a block in which 125 is larger than a predetermined threshold, it is determined whether or not motion compensation is performed by subdividing the block, and prediction control information 126 is output.

The adaptive motion compensation predictive coding circuit 104 generates a predicted image from the reference image 123 by adaptive motion compensation interframe prediction using the detected region boundary information 122, motion vector 124 and prediction control information 126. , And outputs coded data 127 obtained by code-converting the difference value between this and the input image 121. If block division is instructed in the control information 126, the block is subdivided by referring to the boundary information 122, and the motion vector of the area is calculated from the value of the motion vector of the adjacent block in each divided area. Motion-compensated interframe prediction is performed. On the other hand, when the block division is not instructed, the motion compensation is executed in block units. Also, the adaptive motion compensation predictive coding circuit 104 outputs the image 128 obtained by locally decoding the coded data 127. This is held in the image memory 105 as the reference image 123 for the input image 121 of the next frame.

A second motion-compensated inter-frame predictive coding system of the present invention will be described with reference to FIG. First, the motion detection circuit 401 detects the inter-frame motion vector 423 of the input image 421 with respect to the reference image 422 in block units of a predetermined size. When the detected motion vector 423 is used, the motion compensation inter-frame prediction error data 424 of the block is simultaneously output.

Next, in the block division control circuit 402, it is judged whether or not to perform motion compensation by subdividing the block for which the error data 424 is larger than a predetermined threshold value, and prediction control information 426 is output. To do. At the same time, the optimum boundary line for subdividing the target block is detected and output as the division boundary information 425.

The adaptive motion compensation predictive coding circuit 403 generates a predicted image from the reference image 422 by adaptive motion compensation inter-frame prediction using the detected division boundary information 425, motion vector 423 and prediction control information 426, The difference value between this and the input image 421 is code-converted to output encoded data 427. At the time of generating the predicted image, if the control information 426 indicates block division, the area boundary information 425
And the motion vector 423 of the adjacent block are referred to and the block is subdivided and motion-compensated inter-frame prediction is executed separately.
If no block division is instructed, motion compensation is executed in block units. The adaptive motion compensation predictive coding circuit 403 may be realized by combining the uncovered background processing function as described above with reference to FIGS. Also, the adaptive motion compensation predictive coding circuit 403 outputs the locally decoded image 428 obtained by decoding the coded data 427. The locally decoded image 428 is held in the image memory 404 and used as the reference image 422 for the input image 421 of the next frame.

[0014]

1 is a block diagram of an embodiment for realizing a first motion-compensated inter-frame predictive coding system for moving pictures according to the present invention. In FIG. 1, the area boundary detection circuit 101 first inputs the input image.
The boundary lines of a plurality of subject areas are detected from 121 and output as area boundary information 122. Here, the boundary line of the area can be detected as a boundary line of a portion where the luminance or the hue changes abruptly on the input image. Since the area boundary information 122 is encoded as additional information, the area boundary line actually detected may be replaced with a simple pattern to form the area boundary information 122. FIG. 5 is a diagram illustrating an example of the simplified area boundary information 122. Case 1 in FIG. 5 is an example in which the shape of the detected boundary line is used as it is. Case 2 in FIG. 5 is an example in which the detected boundary line is replaced with the most similar straight line. Also, case 3 in FIG.
Is an example in which it is replaced with the most similar horizontal or vertical straight line. When it is desired to suppress the code amount of the area boundary information 122, the simplified pattern of Case 2 or Case 3 of FIG. 5 is used.

The motion detection circuit 102 detects the inter-frame motion vector 124 of the input image 121 with respect to the reference image 123 in block units of a predetermined size. At the same time, when the motion vector 124 is used, the motion compensation inter-frame prediction error data 125 of the block is output. FIG. 1
The image memory 10 is used as the reference image 123 for motion detection.
Although the locally decoded image 128 of the previous frame held in 5 is used, the input image of the previous frame may be used with a delay of one frame period.

Next, the block division decision circuit 103 decides whether or not to perform motion compensation by subdividing the block for which the error data 125 is larger than a predetermined threshold value, and outputs prediction control information 126. To do. The operation of this determination process will be described with reference to FIGS. 6 and 7. FIG. 6 shows a case where the prediction error Dc when the target block Bc is subjected to motion-compensated inter-frame prediction in block units is larger than a predetermined threshold Th and a boundary line Ec is detected in the block. In the present invention, blocks that do not satisfy one of the conditions are not subject to division determination. In addition, the motion vectors Vl, V in the blocks Bl, Br, Bu, Bd adjacent to Bc are
It is assumed that r, Vu, Vd have been detected. Case 1 in FIG. 6 shows a case where the block Bc is divided into two parts on the left and right with a vertical boundary line Ec. The motion vector Vl is assigned to the left side of the block Bc with respect to the Ec, and the motion vector Vr is assigned to the right side from the adjacent blocks Bl and Br, respectively, and the motion-compensated inter-frame prediction is executed separately. At this time, if the prediction error Dd in the block Bc portion is smaller than the prediction error Dc, it is determined that the block Bc is divided into two and motion compensation is performed. Case 2 in FIG. 6 is a case where the block Bc is vertically divided into two at the horizontal boundary line Ec. Substituting the motion vector Vu on the upper side of the block Bc and the motion vector Vd on the lower side from the adjacent blocks Bu and Bd at the boundary of Ec, the motion compensation inter-frame prediction error Dd is measured, and the block is divided into two. Determine whether to execute. Further, FIG. 7 shows a case where a block Bc having a motion-compensated inter-frame prediction error Dc in units of blocks larger than a predetermined threshold Th is divided into two by an oblique boundary line Ec. Two
The motion vector values Vs1 and Vs calculated from the motion vectors of the blocks adjacent to Bc in each of the divided areas
Substituting 2, Vs3 and Vs4, motion compensation interframe prediction is executed. At this time, if the prediction error Dd of the block Bc portion is smaller than Dc, the block is divided into two. The function f (a, b, c) for calculating the motion vector in FIG. 7 is a function for selecting the weighted average value of a, b, c or any one value. In the block whose subdivision is determined by the block division determination circuit 103 in FIG. 1, the area boundary information 122 is encoded as the additional information of the block, but the motion vector 124 of the block is not encoded. On the other hand, in the block that is not subdivided, the motion vector 12
4 is encoded, and the area boundary information 122 is not encoded. It should be noted here that when the target block Bc is subdivided, the motion vector is substituted from the adjacent reference block Bref, so that it is necessary to encode the motion vector without subdividing Bref. Therefore, before determining the subdivision of the block of interest Bc, whether or not the subdivision of the adjacent reference block Bref is necessary is checked in advance. Specifically, the improvement amount of the motion compensation prediction error when Bc and a plurality of Brefs are respectively subdivided is examined, and the subdivision of Bc is determined only when the improvement amount in Bc is the largest.
The adjacent reference block Bref for the target block Bc can be limited as follows. Like case 1 in Figure 6
When Bc is divided into two parts, left and right adjacent blocks Bl, Br,
When Bc is divided into upper and lower halves as in case 2 of FIG. 6, upper and lower adjacent blocks Bu and Bd are Brefs that need verification. Further, in the case of diagonally dividing into two as shown in FIG.
The adjacent block having the motion vector actually referred to in the calculation using the function f (a, b, c) of is Bref.

The adaptive motion compensation prediction circuit 104 detects the area boundary information 122, the motion vector 124, and the prediction control information.
A prediction image is generated from the reference image 123 by adaptive motion compensation inter-frame prediction using 126, the difference value between the input image 121 and the prediction image is code-converted, and coded data 127 is output. At the same time, the locally decoded image 128 obtained by decoding the encoded data 127 is output. This is used as a reference image 123 for the new input image 121 held in the image memory 105.

FIG. 2 shows the adaptive motion compensation predictive coding circuit of FIG.
FIG. 4 is a basic configuration diagram showing a first exemplary embodiment of the present invention that realizes an uncovered background processing function in 104. In FIG. 2, the prediction image generation portion of the adaptive motion compensation predictive coding circuit 204 is composed of a block unit motion compensation prediction circuit 205, a block division motion compensation prediction circuit 206, and an adaptive selection circuit 207. The motion compensation prediction circuit 205 outputs a motion compensation interframe prediction value using the motion vector 124 of the block of interest. Further, the block division motion compensation prediction circuit 206 refers to the area boundary information 122 indicating the subdivision position of the block and the value 124 of the motion vector of the block adjacent to the block of interest, and performs different motion compensation for each subdivided area. Output the inter-predicted value. In the adaptive selection circuit 207, one of the interframe prediction values or 0 according to the prediction control information 126.
A value is selected and output as the predicted image 227. Selecting a 0 value here means executing intraframe coding processing without interframe prediction. The 0 value is selected when an uncovered background area is generated near the area boundary line that subdivides the block.

FIG. 8 is a view for explaining the mechanism for discriminating the uncovered background area. In the block of interest Bc, the motion compensation inter-frame prediction error in block units is large,
Moreover, it is assumed that the area boundary line Ec is detected. The block Bc is divided into two parts on the left and right with Ec, and the same motion vector as that of the adjacent block is assigned to each divided region to execute motion-compensated inter-frame prediction. Here, when the space between the two areas Sl and Sr adjacent to each other with Ec as a boundary spreads, it is determined that an uncovered background area occurs around Ec. Specifically, when the horizontal components of the motion vectors of Sl and Sr are v _xl and v _xr , v _xl is leftward and v _xr
Is to the right, or both are to the left and v _xl is greater, or both are to the right and v _xr is greater, the uncovered background region occurs. On the other hand, in other cases, the uncovered background area does not occur. Case 1 in FIG. 8 shows the left area Sl.
Is larger than the area Sr on the right side and is near the camera that captured the image, an uncovered background area UB having the width of the vector Vub is generated on the right side of the boundary line Ec. The vector Vub can be obtained as the difference vector between the motion vectors Vl and Vr of the two adjacent blocks on the left and right.
In case 2 of FIG. 8, the left side area Sl is larger than the right side area Sr but is far from the camera.
An uncovered background area UB having the width of the vector Vub occurs on the left side of the boundary line Ec. In these uncovered background areas, inter-frame correlation is low in principle, and therefore motion compensation inter-frame prediction is not performed and the process is switched to intra-frame coding processing. To determine which of the left and right sides of the area boundary line Ec is the uncovered background area, the area of the width of the vector Vub with Ec on the right side and the area of the width of the vector Vub with Ec on the left side are cut out, respectively. This can be easily achieved by selecting one with a smaller correlation between motion compensation frames.
This determination information may be encoded as additional information, and in FIG. 8, the boundary line El on the left side of the uncovered background area UB is replaced with the area boundary line Ec regardless of the case.
May be fixedly encoded (or the right boundary Er may be fixedly encoded). The vector Vub indicating the width of the uncovered background area is calculated from the motion vector values Vl and Vr of the two adjacent blocks, and therefore need not be separately encoded. In FIG. 8, the block of interest Bc is defined as the area boundary line Ec.
Although only the case of dividing into two parts on the left and right is shown in, the uncovered background area can be discriminated by the same procedure also in the case of dividing into two parts vertically or diagonally.

FIG. 3 is also a basic configuration diagram for explaining the second embodiment of the present invention for realizing the uncovered background processing function in the adaptive motion compensation predictive coding circuit 104 of FIG. In FIG. 3, the prediction image generation part of the adaptive motion compensation predictive coding circuit 304 is a background prediction circuit 305, a block-based motion compensation prediction circuit 306, and a block division motion compensation prediction circuit 30.
7 and the adaptive selection circuit 308. Background prediction circuit 305
Outputs the background image stored in the built-in background memory as a predicted value. The adaptive selection circuit 308 selects one of the inter-frame predicted values or the background predicted value according to the prediction control information 26 and outputs it as the predicted image 327. Background prediction is selected for the uncovered background area in the subdivision block. The uncovered background area is detected by the method described with reference to FIG. The background image stored in the background memory is updated by using the pixel data of the portion determined to be the still area or the uncovered background area in the locally decoded image 128.

FIG. 4 is a block diagram of an embodiment for realizing the second motion compensation interframe predictive coding system of the present invention. Reference image of input image 421 in motion detection circuit 401
An inter-frame motion vector 423 for 422 is detected in block units of a predetermined size. At the same time, when the motion vector 423 is used, the motion compensation inter-frame prediction error data 424 of the block is output. Although the locally decoded image 428 of the previous frame held in the image memory 404 is used as the reference image 422 for motion detection in FIG.
The input image of the previous frame may be used with a delay of one frame period.

Next, the block division control circuit 402 determines whether or not to perform motion compensation by subdividing the block for which the error data 424 is larger than a predetermined threshold value, and outputs prediction control information 426. To do. At this time, the optimum division boundary line for subdividing the target block is also detected and output as division boundary information 425. Specifically, the block of interest is subdivided at an arbitrary boundary line, and a motion vector is assigned to each of the divided region parts by the method described with reference to FIG. 6 or 7, and a motion compensation inter-frame prediction error is calculated. . Perform this trial on various boundaries,
Select the boundary line that minimizes the prediction error. If this minimum prediction error is smaller than the error data 424, it is finally decided to subdivide the block. Since the division boundary line is encoded as additional information of the block of interest, the division boundary line is limited to only straight lines as in cases 2 and 3 of FIG. 5 in order to suppress the code amount and reduce the number of trial operations. be able to. Further, in the block whose subdivision is determined by the block division control circuit 402, the area boundary information 425 is encoded as additional information of the block, and the motion vector 423 of the block is encoded.
Is not encoded. On the other hand, in the block that is not divided, the motion vector 423 is encoded and the region boundary information 425 is not encoded. Therefore, when the target block Bc is subdivided, it is necessary to encode the motion vector without subdividing the adjacent block Bref that refers to the motion vector. Therefore, before determining the division determination of the block of interest Bc, the adjacent reference block Bref
The necessity of subdivision of is checked beforehand. Specifically, as described above with reference to FIGS. 6 and 7, Bc and a plurality of Brefs are each subdivided to examine the improvement amount of the motion compensation inter-frame prediction error, and only when the improvement amount in Bc is the largest. Bc
Determine the subdivision of.

The adaptive motion compensation predictive coding circuit 403 generates a predicted image from the reference image 422 by adaptive motion compensation inter-frame prediction using the detected division boundary information 425, motion vector 423 and prediction control information 426, Input image 421
And the coded data 427 obtained by code conversion of the difference value between the predicted image and the predicted image is output. At the same time, the locally decoded image 432 obtained by decoding the encoded data 427 is output. This is used as the reference image 422 for the new input image 421 held in the image memory 404.

[0024]

As described above, according to the present invention, a block that spans a plurality of subjects having different motions is subdivided, and motion compensation interframe prediction is performed by using different motion vectors in the divided regions, thereby performing prediction. Effective in improving efficiency. In addition, since the division boundary line is encoded in the divided block as the additional information in block units, and only one of the motion vectors is encoded in the non-divided block, there is no significant increase in the additional information, and there is an improvement effect in the overall encoding efficiency. .

[Brief description of drawings]

FIG. 1 is a basic configuration diagram of a first moving image predictive coding system according to the present invention.

FIG. 2 is a configuration diagram of a first embodiment of the present invention.

FIG. 3 is a configuration diagram of a first embodiment of the present invention.

FIG. 4 is a basic configuration diagram of a second moving image predictive coding method according to the present invention.

FIG. 5 is a diagram illustrating a method of simplifying a boundary line that divides a plurality of subject regions in a block.

FIG. 6 is a diagram illustrating a procedure of subdivision of a block and motion compensation of a divided region when there are a plurality of subject regions having different movements in the block.

FIG. 7 is a diagram illustrating a procedure of subdivision of a block and motion compensation of a divided region when there are a plurality of subject regions having different movements in the block.

FIG. 8 is a diagram illustrating a procedure of detecting an uncovered background area near a division boundary line when there are a plurality of subject areas having different movements in a block.

[Explanation of symbols]

 101 Area boundary detection circuit 102,401 Motion detection circuit 103 Block division determination circuit 402 Block division control circuit 104,204,304,403 Adaptive motion compensation prediction coding circuit 305 Background prediction circuit 205,306 Motion compensation prediction circuit 206,307 Block division motion compensation prediction circuit 207,308 Adaptive selection circuit 105,404 Image memory 121,421 Input moving image 122 Region boundary information 123,422 Reference image 425 Division boundary information 124,423 Motion vector 125,424 Motion compensation inter-frame prediction error data 126,426 Prediction control information 227,327 Predicted image 127,427 Encoded data 128,428 Local decoded image

Claims (2)

(57) [Claims] 1. In motion-compensated interframe predictive coding of a moving image, means for detecting a boundary line between different subject regions from an input image, interframe motion vector with respect to a reference image of the input image, and motion-compensated interframe prediction error. A block unit for detecting the block and the block having the prediction error larger than a predetermined threshold, the block is subdivided at the boundary line, and the block is divided by using motion vectors of a plurality of blocks adjacent to the block. A unit for determining the effectiveness of predicting motion-compensated inter-frames separately, and a unit of a block of an input image or a unit of a sub-divided block of the input image using the motion vector, the boundary line, and the determination result by the determination unit. And a means for performing adaptive motion-compensated interframe predictive coding in video. Measurement coding method. 2. In motion-compensated inter-frame predictive coding of a moving image, a unit for detecting an inter-frame motion vector and a motion-compensated inter-frame prediction error for a reference image of an input image in block units, and a threshold value determined in advance A division boundary line that minimizes a prediction error when a block having a large prediction error is subdivided and the motion vectors of a plurality of blocks adjacent to the block are used to separately predict the motion-compensated inter-frame And a unit that determines the effectiveness of subdivision of a block based on the minimum prediction error, and the motion vector, the boundary line, and the determination result by the determination unit, the block unit or block of the input image is determined. Means for performing adaptive motion-compensated interframe predictive coding in units of subdivided parts. A predictive coding method for moving images characterized by and.