JPH09182082A - Movement compensation prediction encoding method for moving image and device therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the prediction of an overlap movement compensation by compensating deformations such as not only a parallel movement but also a rotation, a magnification, a reduction and a shearing, etc. SOLUTION: The compensation encoding of a moving image becomes possible because the movements of various kinds of deformation movings such as not only a parallel moving but also a magnification, a reduction, a shearing and a rotation, etc., are calculated, a predictive image is prepared and each difference is quantized and encoded by dividing an encoding object image 1 into small areas, comparing the image 1 with a reference image to be positional information immediately proceeding and using the motion parameter 4 for every small area as an affine conversion parameter.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for digitally compressing and encoding an image signal used for image communication, image recording, etc. Regarding

[0002]

2. Description of the Related Art In digital compression encoding of moving images, motion-compensated interframe prediction is used as a means for suppressing temporal redundancy of moving image signals. In this inter-frame prediction, an encoding target image is usually divided into blocks such as 16 pixels × 16 lines, a motion amount (motion vector) between each block and the reference image is detected, and the reference image is shifted by the motion vector. A difference (motion compensation prediction error) signal between the prediction image generated in this way and the encoding target image is encoded. The motion-compensated inter-frame prediction dramatically improves the inter-frame correlation of a moving image, and provides a great amount of information compression as compared with the simple inter-frame prediction. Furthermore, a DCT (Discrete Cosine Transfor) is applied to the motion compensation prediction error signal.
By applying m) and sub-band division, the spatial redundancy is also suppressed and further information compression is achieved. Therefore, videophone, video coding for conference (ITU-T Recommendation H.261), video coding for storage (ISO / IEC
11172 (MPEG-1), general purpose video coding for storage, broadcasting and communication (ISO / IEC 13818 (MP
In EG-2)) and the like, a hybrid coding configuration for DCT-coding a residual signal by motion-compensated interframe prediction is adopted. Hereinafter, as a representative of these, H.264. The encoding algorithm of H.261 will be briefly described with reference to FIG.

First, the image to be coded 1 is inputted to the motion vector detecting section 31 and divided into square blocks called macroblocks of 16 pixels × 16 lines. The motion vector detection unit 31 detects the amount of motion with respect to the reference image for each macroblock in the encoding target image 1,
The obtained motion vector 32 is sent to the block motion compensator 41. Here, the motion vector of each macroblock is obtained as a translation amount in the reference direction x and the direction y perpendicular thereto. That is, in the reference image, it is represented as a displacement between the coordinates of the block having the highest degree of matching with the macroblock of interest and the coordinates of the macroblock of interest. The motion vector search range is limited to the coordinates of the macroblock of interest and ± 15 pixels × ± 15 lines around it.

Next, in the block motion compensating section 41, the locally decoded image 6 located at the coordinate shifted by 32 motion vectors for each macroblock is used as the prediction value of the block,
A motion compensation predicted image 9 is generated. The motion-compensated predicted image 9 obtained here is input to the subtractor 10 together with the encoding target image store King 1. Difference between the two, that is, motion compensation prediction error 11
Is subjected to DCT conversion in the DCT / quantization unit 42 and further quantized to become compressed difference data 13. here,
The block size of DCT is 8 × 8. The compressed difference data 13 (quantization index) is stored in the difference data encoding unit 14
In (1), the data is compressed and becomes the differential image coded data 15. On the other hand, the motion vector 32 is coded by the motion vector coding unit 34, and the obtained motion vector coded data 35 is multiplexed by the multiplexing unit 22 together with the difference image coded data 15 and transmitted as multiplexed data 23. To be done.

Since the same decoded image and encoder are obtained in the decoder, the compressed difference data 13 (quantization index) is returned to the quantized representative value by the inverse quantization / inverse DCT unit 43, and the inverse DCT conversion is performed. Then, the decoded difference image 17 is obtained. The decoded difference image 17 and the motion compensation image 9 are added by an adder 18
Are added to form a locally decoded image 19. The locally decoded image 19 is stored in the frame memory 5 and is used as a reference image when the next frame is encoded.

[0006]

The above-described conventional motion compensation prediction method is to provide a set of motion vectors for each block such as 16 pixels × 16 lines and to compensate the translation amount for each block. .

Therefore, when the motion vectors of adjacent blocks are different, block-like discontinuous distortion occurs in the predicted image. This discontinuous distortion becomes particularly noticeable in a portion with a lot of movement, and at the time of low-rate encoding in which a sufficient code amount cannot be allocated for encoding the prediction error image,
It is a great visual obstacle. Furthermore, when motion compensation is performed using the motion vector obtained in block units, the prediction error at the center of the block is small, but the prediction error increases as the block boundary approaches.

As a means for solving the above problems, an overlap motion compensation method has been proposed in which prediction blocks are overlapped with each other and a weighted sum of each overlap block is used to generate a prediction image. This method is based on "Entropy Coding for Waveleet Tr
ansform of Image and Its Application for Motion Pi
cture Coding "(SPIE Symposium on Visial Communicati
ons and Image Processing '91, pp. 456, 466, 1991
November) and "Windowed Motion Com by H. Watanabe and others.
pensation "(SPIE Symposium on Visial Communications
and Image Processing '91, pp. 582-589, 1991 11
It was proposed at the same time.

Further, the low-rate coding ITU-TH. It is also used in 263. The overlap motion compensation will be briefly described below with reference to FIGS.

As shown in FIG. 4, the encoding target image 1 is set to 1
It is divided into macroblocks of 6 pixels × 16 lines, and the motion vector detection unit 31 obtains a motion vector 32 for each macroblock. Next, in the overlap motion compensator 33, the motion vector of the macroblock in question and the motion vectors of the three macroblocks around it are combined into a total of 4
The final predicted value is calculated by using the motion vector of the book and taking the weighted average of the four predicted values. Hereinafter, j th row and the i-th column of the macroblock B _i in the coordinates (x,
The procedure for calculating the predicted value of the pixel located at y) will be described in detail with reference to FIG.

1. The motion vector (Vx _{i-1, j-1} , Vy _{i-1, j-1} ) of the upper left macroblock B _{i-1, j-1} is used to correspond to the pixel P (x, y) of the current frame. The first predicted value P '(x + Vx _{i-1, j-1} , y + Vy _{i-1, j-1} ) is obtained.

2. Using the motion vector (Vx _{i, j-1} , Vy _{i, j-1} ) of the macroblock B _{i, j-1} immediately above, the second prediction value P ′ (for the pixel P (x, y) of the current frame is x + V
_{x i, j -1, y +} Vy i, j-1) obtained.

3. Using the motion vector (Vx _{i-1, j} , Vy _{i-1, j} ) of the immediate left macroblock B _{i-1, j} , the third prediction value P ′ for the pixel P (x, y) of the current frame (X + V
x _{i−1 , j} , y + Vy _{i−1, j} ) is obtained.

4. Using the motion vector (Vx _{i, j} , Vy _{i, j} ) of the macroblock B _{i, j} , the pixel P of the current frame
The fourth predicted value P ′ (x + Vx _{i, j} , for (x, y)
y + Vy _{i, j} ) is obtained.

5. Weighting factors W _{i-1, j-1} , (x, y), W _{i, j-1} (x, y), for each of the above four predicted values
W _{i-1, j} (x, y) and W _{i, j} are multiplied, and the sum of these is taken as the predicted value P ″ (x, y) for the pixel P (x, y).

[0016]

[Equation 1] Becomes However, W _{i-1, j-1} (x, y) + W _{i, j-1} (x, y) + W _{i-1, j} (x, y) + W _{i, j} (x, y) = 1 (2 ) Is. The weighting factors for the four predicted values are, for example, IT
U-TH. In H.263, it is defined as shown in FIG. In H 263, when the predicted value of the pixel in the second row and third column of the block is calculated _,
Of the motion vector (Vx _{i, j} , Vy _{i, j} ) of P '(x +
Vx _{i, j} , y + Vy _{i, j} ) ni weighting coefficient W
_{i, j} (x, y) becomes 5/8. Further, from FIG. 7 (b), the immediately above block B _i, the motion vector of the _{j over 1 (Vx i, j-1} ,
Py (x + Vx _{i, j-1} , y + V) indicated by Vy _{i, j-1} )
The weighting coefficient W _{i, j-1} (x, y) for y _{i, j-1} ) is 2/8
Becomes Similarly, as shown in FIG. 7C, the block on the immediate left B _{i-1, y}
Of the motion vector (Vx _{i-1, j} , Vy _{i-1, j} ) of P
Weighting coefficient W for (x + Vx _{i-1, j} , y + Vy _{i-1, j} )
_{i-1, j} (x, y) becomes 1/8. In addition, H. 263
Then, the prediction using the motion vector of the macroblock located in the diagonal direction of the block is not performed. That is, W _{i-1, j-1} (x, y) = 0 (3).

As shown in FIG. 6, P '(x + Vx_{i, j}, Y
+ Vy_{i, j}) Is excluded, P '(x + Vx _{i-1, j-1}, Y + Vy
_{i-1, j-1}), P '(Vx_{i, j-1}, Y + Vy_{i, j-1}), P '
(X + Vx_{i-1, j}, Y + Vy_{i-1, j}) Are three
Macro block B_{i-1, j-1}, B_{i-1 ,},, B_{i, j-1}Corresponds to
It is located outside the block. In other words, Macroblo
24 pixels with 4 pixels overlapping each outside
It is assumed that prediction is performed using blocks of × 24 lines.
You. This is why it is called overlap motion compensation.
is there.

The above is an example of the case where (x, y) is at the upper left of B _{i, j} . For example, when (x, y) is at the upper right of B _{i, j} , the macroblock B _{i, j-1} immediately above, the macroblock B _{i + 1, j-1 at the} upper right _{, and} the motion of the macroblock B _{i, j} . Use a vector. Similarly, when (x, y) is at the lower left of B _{i, j} , the immediate left macroblock B _{i-1, j} , the macroblock B _{i, j} , and the lower left macroblock B _{i-1, j +. 1} , using the immediate lower macroblock B _{i, j + 1} and (x, y) is at the lower right of B _{i, j} , the macroblock B _{i, j} immediately right macroblock B _{i + 1, j} , The immediately lower macroblock B _{i, j + 1} and the lower right macroblock B _{i + 1j +1} are used.

However, the above motion compensation prediction method compensates only the parallel movement of each block. Therefore, it is impossible to compensate for the deformation of the subject such as rotation and enlargement / reduction. That is, when the object is deformed, there is a problem that the prediction efficiency is significantly reduced.

An object of the present invention is to solve the above problems and to provide a motion compensation predictive coding method for a moving image, which further improves the prediction performance of overlap motion compensation by compensating for deformations such as rotation and enlargement / reduction. It is to provide the device.

[0021]

A motion-compensated predictive coding method for a moving picture according to the present invention divides a coding target image into a plurality of small areas, and calculates a motion amount between the coding target image and a prediction reference image. Is calculated for each of the small areas, and in obtaining the predicted value of each pixel in the small area, the gray value of the position indicated by the motion vector of the small area and the gray value of the position indicated by the motion vector of the adjacent small area are respectively set. A motion-compensated predictive coding method for a moving image, wherein a predictive image is generated by multiplying a weighting coefficient, and a value obtained by adding these is used as a predictive value, and a difference between the predictive image and an encoding target image is encoded. Depending on the respective motion parameters, the movement of the small area expresses at least two of translation, rotation, enlargement, reduction, shearing, and deformation in the reference direction (x) and the direction (y) perpendicular thereto. The steps described It is.

The motion-compensated predictive coding apparatus for a moving image according to the present invention divides a coding target image into a plurality of small regions, and calculates a motion amount between the coding target image and the prediction reference image for each of the small regions. When obtaining the predicted value of each pixel in the small area, multiply the gray value of the position indicated by the motion vector of the small area and the gray value of the position indicated by the motion vector of the adjacent small area by a weighting factor, In the motion-compensated predictive coding apparatus for a moving image, which generates a predicted image by setting a value obtained by adding the predicted value and codes a difference between the predicted image and the image to be coded, It has means for describing at least two of translation, rotation, enlargement, reduction, and deformation in a reference direction (x) and a direction (y) perpendicular to the reference direction by each motion parameter. .

Further, an embodiment of the motion compensation predictive coding apparatus for a moving picture according to the present invention comprises means for dividing a coding target picture into predetermined small areas, and means for immediately preceding each small area. Means for reproducing an image of a state as a reference image, means for detecting a motion vector of each small area by comparing it with the reference image, and the motion vector being decomposed into components according to the motion state to best match the reference image. A means for selecting a small area having a high degree of displacement and a displacement for each small area, a gray value of a position indicated by a motion vector of the small area, and a weighting coefficient for each gray value of a position indicated by a motion vector of an adjacent small area. Multiply
A means for generating a compensation prediction value for each small area by using a value obtained by adding these as a prediction value, and a motion compensation prediction error is calculated from the difference between the compensation prediction value for each small area and the image to be encoded. Means, means for compressing the spatial redundancy of the motion compensation prediction error, means for encoding the quantized compression difference, and means for encoding the motion vector for each small region,
In the motion compensation predictive coding apparatus for a moving image, having means for multiplexing and outputting the coded difference data and the motion vector coded data, a means for generating the compensation prediction for each of the small regions, Translation, enlargement, reduction,
The motion compensation prediction value is generated by a motion parameter expressing at least two of inversion, shearing, rotation, and deformation.

In a preferred mode, the means for generating the motion-compensated predicted value is calculated by using an affine transformation parameter.

[0025]

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram of a coding device to which the motion compensation predictive coding method of the present invention is applied.

In this encoding apparatus, the image to be encoded 1 is first input to the motion parameter detector 3 together with the motion model 2 and the motion parameter 4 for each small area is obtained.
Here, the size and shape of the small area is generally 1
It is a rectangular block of 6 pixels × 16 lines, but it is not always necessary.

An example of the motion model 2 is projective transformation. The formula of the projective transformation is shown below.

[0029]

[Equation 2]

[0030]

(Equation 3) The projective transformation interval is a general expression of a two-dimensional transformation, and p = q = 0,
The one with s = 1 is called an affine transformation. The affine transformation formula is shown below.

[0031] x '= ax + by + t x (6) y' = cx + dy + t y (7) is as follows rewrite this matrix notation.

[0032]

(Equation 4) Here, t _x and t _y respectively represent parallel movement amounts in the x direction and the y direction. When a = d = 1 and b = c = 0, the conventional motion compensation is performed only by the parallel movement. Ie

[0033]

(Equation 5) The following conversion can be realized by the affine conversion parameter a.

A> 1: enlargement in the x-axis direction a = 1: no change 0 <a <1: reduction in the x-axis direction a = 0: projection on the y-axis-1 <a <0: after reduction in the x-axis direction , Inversion with respect to y-axis a = -1: inversion with respect to y-axis a <-1: after expansion in the direction to x, then inversion with respect to y-axis Similarly, the conversion by the affine transformation parameter d expresses expansion / contraction in the y-axis direction. ,, d is negative, with inversion about the x-axis.

The affine transformation parameter b represents shear in the x-axis direction. For example, a = d = 1 and c = t _x
= T _y = 0, the conversion formula is as follows.

X ′ = x + by (10) y ′ = y (11)

As can be seen from the above equation, this conversion results in x
Points on the axis do not move, and points on the x-axis shift to the right by b × y (FIG. 5 (a)). This meaning is the same as in the case where the transformation figure in which this transformation is applied to a square (FIG. 5B) is applied with a shearing force on a square elastic body. Similarly, the affine transformation parameter c represents shear in the y-axis direction. For a = d = 1 and b = 0, points on the y-axis do not move, points not on the y-axis shift up by c × x. Furthermore, a = cos θ, b = sin θ, c = −sin θ,
When d = cos θ, the rotation of the angle θ can be expressed. As described above, various transformations such as enlargement, reduction, inversion, shearing, rotation, and combinations thereof can be freely expressed by the affine transformation.

Simplifying the affine transformation by the equation (8),
The equation without shearing is shown below.

[0039]

(Equation 6) Can also be used (H. Jozawa "Segment-Based Vid
eo Coding Using An Affine Model ", SPIE Symposium o
n visial Communications and Image Processing'94pp.
1605-1614, September 1994). The above is an example of the motion model used for the overlap motion compensation of the present invention.

In the motion detector 3, for example, Japanese Patent Application No. 6-11
The motion parameter 4 can be obtained by the method applied in 6260 “Motion Parameter Search Method”. Japanese Patent Application No. 6
The method of generating motion parameters in -116260 will be described below. Here, the motion model is the five-parameter affine transformation of equation (12).

As shown in FIG. 2, the motion parameter detector
3 is a translation parameter for the input image 1 and the reference image 102.
Input to the data search unit 103, and the block matching method is used.
Translation parameter (t_x, T_y) Search results, error evaluation
Starting from the smallest value, a predetermined number of initial translation
The parameter 104 is required. Initial translation parameter
Reference numeral 104 denotes a translation and expansion together with the input image and the reference image 102.
It is input to the large, reduction, and rotation parameter search unit 105.
In parallel translation, enlargement, reduction, rotation parameter search unit 105
Is the initial value of the initial translation parameter 104.
The translation parameter is changed within a predetermined minute range including
And enlargement / reduction parameter (C _x, C_y)and
Rotation parameter (θ) is changed, resulting in the smallest
A combination of parameters that gives a matching error evaluation value
Output as the final affine transformation parameter 106
You. An example of the search method is shown below.

[0042] First, in the first stage (translation parameter search unit 103), ± 15 pixels are searched in 1-pixel increments in the x and y directions, and from 31 × 31 = 961 search points, the one with the smallest error evaluation is selected. The four points are set as the initial translation parameter 104 in order. Next, the second stage (parallel movement / enlargement / reduction / rotation parameter search unit 105) performs ± 0.5 around the four points.
The pixel range is changed in 0.5 pixel steps, and the enlargement / reduction parameter is in the range of 0.8 to 1.2 in 0.1 steps and the rotation parameter is in the range of 0 to 2π in π / 4 steps. Change all search points (9 × 5 × 8
= 360 points), the smallest error evaluation value is obtained as an affine transformation parameter. 6 parameters (a,
b, c, d, t _x , t _y ) is exactly the same.

The motion parameter 4 for each small area obtained by the motion detector 3 is input to the overlap motion compensator 7 together with the locally decoded image 6 stored in the frame memory 5. The overlap motion compensation unit 7 performs prediction with deformation such as rotation, enlargement, and reduction in addition to parallel movement in the x and y directions for each small area (hereinafter referred to as macroblock), and a motion compensation predicted image 9 To generate. FIG. 8 shows an example of generation of a predicted image in the overlap motion compensation unit 7. The procedure for calculating the pixel located at the coordinate (x, y) in the macroblock B _{i, j} on the j-th row and the i-th column will be described in detail below.

1. The motion parameters (a _{i-1, j-1} , b _{i-1, j- 1} , c _{i-1, j-1} , d of the upper left macroblock B _{i-1, j-1} are
_{i-1, j-1} , tx _{i-1, j-1} , ty _{i-1, j-1} ), and the first predicted value P'corresponding to the pixel P (x, y) of the current frame
(X ' _{i-1, j-1} , y'i _{-1, j-1} ) is calculated.

[0045]

(Equation 7) FIG. 8 shows an example of enlargement in the x and y directions (a, d> 1, b = c = 0).

2. The motion parameters (a _{i, j-1} , b _{i, j-1} , c _{i , j-1} , d _{i, j-1} , t of the macroblock B _{i, j-1} immediately above it.
x _{i, j−1} , ty _{i, j−1} are used for the pixel P (x,
second predicted value P '( _{x'i, j-1} , y'corresponding to y)
_{i, j-1} ).

[0047]

(Equation 8) FIG. 8 shows reduction in the x and y directions (0 <a, d <1, b = c =
0) is shown.

3. The motion parameters (a _{i-1, j} , b _i-1.j , c _{i -1, j} , d _{i-1, j} , t of the macro blocks B _{i-1, j on} the immediate left are
x _i−1.j , ty _{i−1, j} , the pixel P (x,
The third predicted value P '(x' _{i-1, j} , y'corresponding to y)
_{i-1, j} ) is obtained.

[0049]

[Equation 9] FIG. 8 shows an example in which reduction and rotation in the x and y directions are combined (a = d = cos θ, b = sin θ, c = −si).
nθ and 0 <a, d <1).

4. The motion parameters (a _{i, j} , b _{i, j} , c _{i, j} , d _ij , tx _{i, j} , ty) of the macro block B _{i, j.
i, j} ) is used to obtain the fourth predicted value P ′ (x ′ _{i, j} , y ′ _{i, j} ) corresponding to the pixel P (x, y) of the current frame.

[0051]

(Equation 10) FIG. 8 shows an example of rotation (a = d = cos θ,
b = sin θ, c = −sin θ).

The corresponding point (x ', y') of (x, y) obtained by affine transformation is not necessarily an integer. Therefore, it is necessary to perform interpolation calculation in the reference image. In this case, for example, as shown in FIG. 9, pixel values P ′ (x ₁ , y ₁ ), P ′, and 4 ′ around the coordinates of the matching destination are shown.
(X ₁ +1, y ₁ ), P '(x ₁ , y ₁ +1), P' (x ₁ +
1, y ₁ +1) is interpolated (x ₁ = x ′ integer part, y ₁ = y ′ integer part). wx ₁ , wx ₂ , wy ₁ , w
y ₂ is a weighting coefficient, which is the sum of values multiplied by these weights.

[0053]

[Equation 11]

[0054]

(Equation 12)

[0055]

(Equation 13)

[0056]

[Equation 14]

[0057]

(Equation 15) Is the predicted value.

5. Weighting factors W _{i-1, j-1} (x, y), W _{i, j-1} (x, y), W for each of the above four predicted values
_{i-1, j} (x, y) and W _{i, j} (x, y) are multiplied, and the sum of these is taken as the predicted value P ″ (x, y) for the pixel P (x, y).

[0059]

(Equation 16) Becomes However,

[0060]

[Equation 17] It is. The weighting factors for the four predicted values are, for example, IT
U-TH. In H.263, it is defined as shown in FIG.
H. In the case of H.263, the motion vector (Vx _{i, j} , Vy _{i, j} ) of the block B _{i, j} is calculated from FIG. 7A when the prediction value of the pixel in the second row and third column of the block is calculated. Pointed to by P '
Weighting coefficient W _{i, j} for (x + Vx _{i, j} , y + Vy _{i, j} )
(X, y) becomes 5/8. Also, from FIG. 7 (b),
The motion vector (Vx _{i, j-1} , Vy of the immediately preceding block B _{i, j-1
i, j-1)} pointed _{P '(x + Vx i,} j -1, y + Vy i, j-1)
The weighting coefficient W _{i, j-1} (x, y) with respect to is 2/8. Similarly, as shown in FIG.
_{i-1, j} (x, y) becomes 1/8. H. 263
Then, the prediction using the motion vector of the macroblock located in the diagonal direction of the block is not performed. That is, W _{i-1, j-1} (x, y) = 0 (24).

The above is an example of the case where (x, y) is at the upper left of B _{i, j ,} . For example, when (x, y) is located at the upper right of B _{i, j} , the macro block B _{i, j−1} immediately above, the macro block B _{i + 1, j−1 at} the upper right, and the macro block B _{i, j} ,
The motion vector of the immediate right macroblock B _{i + 1, j} is used.
Similarly, (x, y) is the lower left macroblock B _{i−1 of} B _{i, j
, j} , the relevant microblock B _{i, j} , the lower left microblock B _{i-1, j + 1} , and the immediate lower macroblock B _{i, j + 1} ,
(X, y) is B _i, the macroblock B _i if immediately below the _{j, j,} straight right macroblock B _{i + 1, j,} directly under the macro block B _{i, j + 1} lower right macroblock B _{i Use + 1, j + 1} .
The above is the calculation method of the prediction compensation in the overlap motion compensation using the affine transformation. Now, the motion compensation prediction image 9 is input to the subtractor 10 together with the encoding target image 1, and their difference data, that is, the motion compensation prediction error 11 is input.
Is suppressed by the spatial redundancy compressing unit 12. On the other hand, in order to obtain the locally decoded image 19 of the current encoding target image 1, the compressed difference data 13 output from the spatial redundancy compression unit 12 is decoded by the difference data expansion unit 16 into the expanded difference image 17. The decompression difference image 17 is a motion compensation prediction error signal whose spatial redundancy is suppressed. The decompressed difference image 17 is added to the motion compensation predicted image 9 by the adder 18 and becomes the locally decoded image 19 of the current image to be encoded. The locally decoded image 19 is stored in the frame memory 5 and is referred to in the coding of subsequent frames.

The compressed difference data 13 corresponding to the motion compensation prediction error 11 is data-compressed and encoded by the difference data encoding unit 14 to be difference image encoded data. The motion parameter 4 for each small region is data-compressed and coded by the motion parameter coding unit 20 and becomes motion parameter coded data 21. The difference image coding 15 and the motion parameter coded data 21 are multiplexed in the multiplexing unit 22 and transmitted or stored as multiplexed data 23.

FIG. 10 shows a flowchart of the above-described motion compensation coding method for moving images according to the present invention. That is,
When the image to be encoded is input (step 51), it is divided into small areas (step 52), and the affine transformation parameter 106 is set to the motion parameter 4 for each small area compared to the reference image.
(Step 53), the motion-compensated predicted image 9 is compared with the motion parameter 4 in comparison with the reference diagram.
Is generated (step 54), the image to be coded at the time of input is subtracted from the motion compensation prediction image 9 to calculate a motion compensation prediction error (step 55), and the error is compressed to create difference data (step). 56). On the other hand, this difference data is added to the motion-compensated predicted image in step 54 to create a reference image (step 57), and this reference image is used for reference in the next process.

On the other hand, the difference data is encoded (step 58), multiplexed with the motion parameter 4 and output as multiplexed data 23 (step 59).

[0065]

As described above, the present invention is capable of compensating not only parallel movement but also various deformations such as enlargement, reduction, shearing, and rotation, and movements freely combining these for motion compensation of moving images. Therefore, the accuracy of the motion compensation prediction is improved, the prediction effect is significantly improved, and the total compression performance is further improved.

[Brief description of the drawings]

FIG. 1 is a configuration block diagram of an embodiment of a motion compensation predictive coding apparatus to which a moving image motion compensation predictive coding method of the present invention is applied.

FIG. 2 is a detailed configuration block diagram of a motion detection unit 3 shown in FIG.

FIG. 3 is a block diagram of an example of a conventional motion compensation predictive coding apparatus for a moving image.

FIG. 4 is a block diagram showing a configuration of a conventional overlap motion compensation predictive coding apparatus by parallel movement compensation.

FIG. 5 is an explanatory diagram of a conventional overlapping motion compensation prediction method by parallel movement compensation, in which (a) is a pixel P of a frame.
(X, y) is a diagram showing parallel movement in the x direction, and (b) is a diagram showing shearing showing the movement in the x direction of only the upper side QR.

FIG. 6 is a diagram showing an example of shear transformation that can be described by affine transformation.

FIG. 7 is a diagram showing an example of weighting factors of each block in overlap motion compensation, where (a) indicates the motion vector of the block, (b) indicates the immediately upper block, and (c) indicates the motion vector of the immediately left block. This is an example of the weighting coefficient for the predicted value of the pixel, and the normalization coefficient of 1/8 is omitted in the weighting coefficient in the figure, and the value obtained by multiplying each value by 1/8 is actually the weighting coefficient.

FIG. 8 is a diagram showing an overlap motion compensation method using affine transformation according to an embodiment of the present invention.

FIG. 9 is a diagram showing an example of a method of calculating a matching element by affine transformation.

[Fig. 10] Fig. 10 is a flowchart of a motion compensation predictive coding method for a moving image according to the present invention.

[Explanation of symbols]

 1 image to be encoded 2 motion model 3 motion parameter detection unit 4 motion parameter 5 frame memory 6 locally decoded image 7 overlap motion compensation unit 8 weighting function, 9 motion compensation predicted image 10 subtractor 11 motion compensation prediction error 12 spatial redundancy Compressor 13 Compressed difference data 14 Difference data encoder 15 Difference image coded data 16 Difference data decompressor 17 Expanded difference image 18 Adder 19 Locally decoded image 20 Motion parameter encoder 21 Motion parameter encoded data 22 Multiplexer 23 multiplexed data 31 motion vector detection unit 32 motion vector 33 overlap motion compensation unit 34 motion vector coding unit 35 motion vector coded data 41 block motion compensation unit 42 DCT / quantization unit 43 inverse DCT / quantization unit 101 frame Memory 102 Image 103 translation parameters searching unit 104 initial translation parameters 105 parallel movement, enlargement, reduction rotation parameter searching unit 106 affine transformation parameters

Claims (4)

[Claims] 1. An image to be encoded is divided into a plurality of small regions, a motion amount between an image to be encoded and a prediction reference image is obtained for each of the small regions, and a prediction value of each pixel in the small region is calculated. In obtaining, the grayscale value at the position pointed to by the motion vector of the small area and the grayscale value at the position pointed to by the motion vector of the adjacent small area are each multiplied by a weighting coefficient, and the sum of these values is used as the predicted value to obtain the predicted image. In the motion compensation predictive coding method for a moving image, wherein the motion of the small area is a reference direction (x) and a direction (x) that is perpendicular to the reference direction. translation to y), rotation, enlargement, reduction,
A motion-compensated predictive coding method for a moving image, characterized in that at least two of shear and deformation are described by respective motion parameters. 2. An image to be encoded is divided into a plurality of small regions, a motion amount between an image to be encoded and a prediction reference image is obtained for each of the small regions, and a prediction value of each pixel in the small region is calculated. In obtaining, the gray value at the position pointed to by the motion vector of the small area and the gray value at the position pointed to by the motion vector of the adjacent small area are each multiplied by a weighting coefficient, and the value obtained by adding these is used as the predicted value to predict. A motion-compensated predictive coding apparatus for generating a picture and coding a difference between the predicted picture and a coding target picture, in a direction (x) based on the motion of the small region and a direction perpendicular thereto. Translation to (y), rotation, enlargement, reduction,
A motion-compensated predictive coding apparatus for a moving picture, comprising means for describing at least two of the transformations by respective motion parameters. 3. A means for dividing an image to be encoded into predetermined small areas, a means for reproducing an image in a state immediately before each small area as a reference image, and a motion vector for each small area. Means for detecting by comparing with the reference image,
Means for decomposing the motion vector into components according to the state of motion, selecting a small area having the highest degree of matching in the reference image and setting the displacement for each small area, and a gray value of the position indicated by the motion vector of the small area. And a means for generating a compensation prediction value for each small area by multiplying each gray value of the position indicated by the motion vector of the adjacent small area by a weighting coefficient, and making the value obtained by adding these to the prediction value, Means for calculating a motion compensation prediction error from the difference between the compensation prediction value for each region and the image to be encoded, means for compressing the spatial redundancy of the motion compensation prediction error, and encoding the quantized compression difference. A motion compensation predictive coding device for a moving image, which has a means, a means for coding a motion vector for each of the small regions, and a means for multiplexing and outputting the coded difference data and the motion vector coded data. In the above, the means for generating the motion compensation prediction value for each small area generates the motion compensation prediction value by a motion parameter expressing at least two of translation, enlargement, reduction, inversion, shearing, rotation, and deformation. A motion-compensated predictive coding apparatus for a moving image. 4. The motion-compensated predictive coding apparatus for a moving picture according to claim 3, wherein the means for generating the motion-compensated predicted value is calculated by using an affine transformation parameter.