JP2008054341A - Method for compensating motion of moving picture - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve a problem in which when an evaluation is performed by subtracting a fixed amount from a basic evaluation value used for a motion estimation only in the case of specific motion vector so that a motion vector with a small amount of information at the time of coding is easily detected, an adverse effect of deterioration in predictive characteristics by a reduction of an amount of coding information of the motion vector when its value is always constant, can not be considered. <P>SOLUTION: A deterioration in predictive characteristics is prevented by performing setting so that a differential value to be subtracted from an evaluation value becomes smaller when a quantization step width becomes smaller by a quantization parameter for determining the quantization step width in quantizing a transform coefficient. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention belongs to a digital video encoding technique.

In high-efficiency coding of digital moving images, it is known that a motion compensation method using a correlation between temporally adjacent frames produces a large compression effect. For this reason, the half-pixel precision block matching technique is also used in the international standards MPEG1 and MPEG2 of the moving picture coding system. In this method, an image to be encoded is divided into a large number of blocks, and the motion vector for each block is obtained in the horizontal and vertical directions as a minimum unit, which is half the distance between adjacent pixels. This process is expressed as follows using mathematical formulas. The prediction image of the frame to be encoded (current frame) is P (x, y), and the reference image (the decoded image of the frame that is temporally close to P and already encoded) is R ( x, y). Further, assuming that x and y are integers, it is assumed that a pixel exists at a point where the coordinate value is an integer in P and R. At this time, the relationship between P and R is

It is represented by However, assuming that the image is divided into a blocks, Bi represents a pixel included in the i-th block of the image, and (ui, vi) represents a motion vector of the i-th block.

In block matching with half-pixel accuracy, ui and vi are each determined as a minimum unit of half of the inter-pixel distance, that is, 1/2 in this case. Therefore, the coordinate value is not an integer, and there is no actual image in the reference image (hereinafter, such a point is called an interpolation point)
It is necessary to obtain the luminance value of In this case, bilinear interpolation using four peripheral pixels is often used. When this interpolation method is described by a mathematical expression, the minority part of the coordinate value is α and β (0 ≦ α, β <1), and the luminance value R (at the interpolation point (x + α, y + β) of the reference image is set. x + α, y +
β) is

It is represented by

Note that a motion compensation method for each block such as block matching is hereinafter referred to as local motion compensation in the sense of detecting local motion.

This block matching is also used in the international standard system H.263. Also H.263
Then, a macro block of 16 pixels × 16 pixels can be divided into four small blocks of 8 pixels × 8 pixels, and block matching can be performed for each small block. The detailed algorithm of H.263 is described in “DRAFT ITU-T Recommendation H.263, Dec. 5, 1995”. Here, motion compensation is performed using only the pre-coded frame in the positive direction as a reference image. An encoding algorithm will be described for a P picture.

FIG. 2 shows the structure of the macroblock. Macroblock is 16 pixels x 16 pixels Y
The signal block 100 and two color difference signal blocks 5 and 6 of 8 pixels × 8 pixels corresponding to the signal block 100 are formed. Further, the Y signal block 100 includes four 8-pixel × 8-pixel blocks 1, 2.
3 and 4. In H.263, discrete cosine transform (DCT) is applied to the prediction error signal.
Then, the resulting transform coefficient is quantized and encoded, and this DCT is performed in units of 8 to 8 pixel blocks of 1 to 6. H.263 has three types of predictions, which are selected in units of macroblocks. Specific prediction types are as follows.

P-1. INTRA mode in which DCT is directly applied to pixels in a block without performing motion compensation.

P-2. INTER mode in which motion compensation is performed in a block of 16 pixels × 16 pixels and one motion vector is encoded.

P-3. INTER4V mode in which a block of 16 pixels × 16 pixels is divided into 4 small blocks of 8 pixels × 8 pixels, motion compensation is performed for each small block, and 4 motion vectors are encoded.

FIG. 3 shows an encoded block diagram in H.263. Here, a P picture that predicts motion in a positive direction in time will be described as an example. First, an input image is input to the local motion compensation processing unit 31, and local motion compensation of each Y signal macroblock is performed between the input image and a reference image extracted from the frame memory 32. As a result, the local motion vector accompanying the prediction of each macroblock is output to the encoder 38, and the motion compensated image is converted to INTRA /
It is output to the INTER switch 39-1. The details of the motion compensation processing unit 31 are shown in FIG.
Will be described. Next, INTRA / INTER switches 39-1 and 39-2 controlled by a control signal sent from a control device 33 that determines a coding block, a prediction method, a quantization method, and the like.
Accordingly, the encoding target image generated from the motion compensated image and the input image is input to the converter 34, and DCT is performed. The conversion coefficient of each macro block generated there is the control device 33.
Quantization parameter QP selected from requirements (encoding rate, playback image quality, etc.)
According to escription, it is described as QUANT but is described as QP in this specification). This QP is a positive integer from 1 to 31. Conversion factor COF
Is

It is quantized by. Here, LEVEL represents a quantized value, DC represents a direct current component of a conversion coefficient, and AC represents an alternating current component. The operator '/' represents rounding down after the decimal point, and the operator '//' represents rounding off after the decimal point.

The quantized value calculated here is sent to the encoder 38 and is inversely quantized by the inverse quantizer 36. Of the quantized value LEVEL, the direct current component of the INTRA mode is

Is dequantized. Other quantization values are

Is dequantized as an absolute value by

A positive or negative sign is given by. Here, DCOF represents an inverse quantization value. The calculated inverse quantization value is subjected to inverse DCT by an inverse transformer 37. The inverse transformation value is combined with the motion compensated image according to the selection result of the INTRA / INTER switch 39-3 controlled by the control device 33, and then input to the frame memory 32. In the encoder 38, information of the encoded block for each macroblock determined by the control device 33 (indicating which block has information to be encoded among 1 to 6 in FIG. 2), prediction type information, In addition to the quantization parameter, each
Multiplex coding is performed after a quantized value of a DCT coefficient and a local motion vector are input.

FIG. 4 is a diagram showing details of the local motion compensation processing unit 31 of FIG. The input image (current frame) is first divided into an INTER local motion estimation device 311 and an INTER4V local motion estimation device 31.
2 is input. In 311 and 312, local motion estimation in the INTER mode and local motion estimation in the INTER4V mode (16
The × 16 block is divided into small blocks 1 to 4 in FIG. Assuming that the input image is divided into a macroblocks (1 ≦ i ≦ a), the INTER local motion estimation device

In INTER4V local motion estimation device

For each macroblock is performed on the candidate local motion vectors within the search range,
Local motion vectors that minimize Ei (u, v) and 4Ei, j (uj, vj | 1 ≦ j ≦ 4) are detected, respectively. The meanings of the symbols used in Equations 7 and 8 are as follows.

F (x, y): Y signal of the original image of the frame to be encoded (current frame). x and y are integers.

R (x, y): Y signal of a reference image (decoded image of a frame that is temporally close to F and has already been encoded) extracted from the frame memory. x and y are integers. The interpolation method in which the local motion vector is a real number follows Formula 2.

Ei (u, v): An evaluation value for the local motion vector (u, v) of the i-th block when the image F is divided into a 16 × 16 blocks.

4Ei, j (uj, vj | 1 ≦ j ≦ 4): When the image F is divided into a 16 × 16 blocks, four 8 × 8 block local motion vectors (u1, v1) in the i-th block ), (u2, v2), (u3, v3),
Evaluation value for (u4, v4).

Bi: Pixels included in the i-th block when the image F is divided into a 16 × 16 blocks. Each pixel belongs to the range of Xi ≦ x <Xi + 16 and Yi ≦ y <Yi + 16 (x and y are integers). (Xi, Yi
) Is the upper left pixel of block i.

Bi, j: Pixels included in the j-th block when the block i is divided into four 8 × 8 blocks. The value of j is an integer of 1 to 4, and corresponds to 1 to 4 of the Y signal block in FIG. Each pixel belongs to the range of Xi, j ≦ x <Xi, j + 8, Yi, j ≦ y <Yi, j + 8 (x and y are integers). (Xi, j, Y
i, j) is the upper left pixel of the jth 8 × 8 block in block i.

Ai: Local motion vector within the motion search range of the i-th 16 × 16 block. Pixel accuracy is half pixel. The horizontal and vertical components u and v are real numbers limited to 0.5 increments.

Ai, j: Local motion vector within the motion search range of the j-th 8 × 8 block out of the i-th 16 × 16 block. The value of j is an integer of 1 to 4, and corresponds to 1 to 4 of the Y signal block in FIG. The horizontal and vertical components uj and vj are real numbers limited to 0.5 increments.

Next, the minimum value of the evaluation value Ei for local motion estimation in the INTER mode and the minimum value of the evaluation value 4Ei, j for local motion estimation in the INTER4V mode are transmitted to the control device 33. The control unit compares the two values, and if the minimum value of Ei is smaller than the minimum value of 4Ei, j or if the minimum value of Ei and the minimum value of 4Ei, j are the same value, the control unit sets the INTER mode to the value of macroblock i. When the minimum value of 4Ei, j is smaller than the minimum value of Ei, the INTER4V mode is selected as the motion prediction type of macroblock i. Furthermore, from the input image activity and the above evaluation values, INTRA / INTE
R is determined.

In response to this evaluation result, the control device 33 controls the prediction type switch 313, and a local motion vector is output to the local motion compensation device 314 according to the selected prediction type. The local motion compensation device 314 compensates the local motion vector of each macroblock for the reference image extracted from the frame memory 32, and generates a motion prediction image.
Then, the motion prediction image is output to the INTER / INTRA switch 39-1, and the local motion vector of each macroblock is output to the encoder 38.

In the motion estimation of FIG. 4, the absolute value sum of the prediction error signals is used as a basic evaluation value at the time of motion estimation, as shown in Equation 7. And, in order to detect a specific motion with a small amount of encoded information of the motion vector preferentially, means for subtracting a constant value from the basic evaluation value only for the motion is used (in this case, 258 is subtracted when the motion vector is 0 in both the horizontal and vertical directions, and 129 is subtracted when the motion vector is not 0). This is a method of suppressing the encoded information amount of the motion vector by allowing a slight increase in the encoded information amount in the prediction error signal. Such a technique is effective when the ratio of the amount of information required for encoding motion vectors is large with respect to the entire encoded information. In the case of Equation 7, the number of encoding bits required for encoding the 0 vector is smaller than the number of encoding bits required for encoding the other motion vectors, and the motion vector encoded in the INTER mode is 1. In contrast to the book, INTER4V pays attention to the fact that four motion vectors must be encoded.

Used in international standard formats H.261, H.263, MPEG1, MPEG2 (for example, H.261, MPEG1, MPEG2 is described in “Latest MPEG Textbook” (1994.8) supervised by Hiroshi Fujiwara). Motion compensation is a prediction method that uses all local motion compensation techniques. In contrast to this local motion compensation, it has been reported that a method of compensating for the overall motion of the entire image is effective in an image sequence that involves panning or zooming a camera like sports broadcasting (for example, Uekura et al., "Global Motion Compensation Method for Video Coding", IEICE Transactions,
Vol.J76-B-1, No. 12, pp944-952, Hira 5-12). This motion compensation method is called a global motion compensation method. There are several methods for compensating for global motion. Here, a method using bilinear transformation is taken as an example. In this conversion, it is allowed that adjacent pixels have different motion vectors. Therefore, in addition to translation, motion such as rotation, enlargement / reduction, and deformation can be handled. Here, a motion compensated image created by global motion compensation is called a global motion compensated predicted image, and a method for calculating a pixel (x, y) on the global motion compensated predicted image uses bilinear transformation. This will be explained with an example.

First, the pixel (x, y) on the current frame is spatially ((
Assume that the pixel position is represented by tx (x, y), ty (x, y)). At this time, the motion vector of the pixel (x, y) on the global motion compensated prediction image which is the prediction image of the current frame is (x-tx
(x, y), y-ty (x, y)) When bilinear transformation is used, spatial corresponding points (tx (x, y),
ty (x, y)) is

Is calculated using Therefore, in order to create a global motion compensated prediction image from the reference image, the motion estimation parameters b1 to b8 must be obtained. These parameters can be uniquely calculated from the motion vectors for the four pixels. Also, in encoding the motion estimation parameter, instead of directly encoding and transmitting b1 to b8, if the four motion vectors used for the calculation are encoded and transmitted, the global motion compensated prediction image Regeneration is possible. A motion estimation parameter generated by global motion compensation is defined as a global motion estimation parameter.

These four motion vectors do not need to be defined at their pixel positions as long as an agreement is made between the encoding side and the decoding side. However, when the motion vector of the pixel existing at the end of the global motion compensation prediction image exceeds the search range, the global motion compensation prediction image cannot be completely reproduced on the decoding side. Therefore, generally, motion vectors for pixels at the four corners of an image are often transmitted.

FIG. 5 shows an example of the global motion compensation method. 501 is the original image of the current frame, 50
Reference numeral 2 denotes the reference image. Reference numeral 503 denotes a patch when the entire original image is viewed as one area (patch), and reference numerals 504, 505, 506, and 507 denote the arrangement of the lattice points. Here, when the movement / deformation from 501 to 502 is compensated using the patch 503,
The reference image 502 is deformed like 508. In addition, the lattice points 504, 5 and 5 are obtained by this motion compensation.
05, 506, and 507 move to 509, 510, 511, and 512, respectively, and each lattice point holds a motion vector according to the amount of motion. Therefore, if these four motion vectors are transmitted to the decoding side, a global motion compensated prediction image common to the encoding side and the decoding side can be created using Equation 9. Note that a high-speed algorithm such as that referred to in Japanese Patent Application No. 8-60572 also exists in the method for creating the global motion compensation predicted image.

This global motion compensation is effective when the entire image moves uniformly in the same way, but when there is a region that moves differently in the image, only one pattern of these motions is detected. I can not cope. Therefore, in order to compensate for the spatial motion amount existing between the original image and the global motion compensated predicted image, a method of performing local motion compensation using the global motion compensated predicted image as a reference image is also used. Furthermore, in order to cope with still regions and conventional simple parallel movements, a method of performing local motion compensation on an image not subjected to global motion compensation and an image subjected to global motion compensation, respectively, and adapting in units of blocks. There is also.

When performing evaluation by subtracting a certain amount only for a specific motion vector from the basic evaluation value used for motion estimation so that a motion vector with a small amount of information at the time of encoding is easily detected, the value is always constant, The influence of deterioration in prediction characteristics due to reduction in the amount of motion vector encoded information cannot be considered.

By setting the quantization parameter that determines the quantization step width when quantizing the transform coefficient so that the difference value to be subtracted from the evaluation value is smaller as the quantization step width is smaller, deterioration of the prediction characteristics is prevented.

According to the present invention, it is possible to control the amount of motion vector encoded information associated with the motion compensation method in consideration of the entire encoded information.

The motion estimation evaluation value control method as shown in Equations 7 and 8 of “Prior Art” is effective when the ratio of the encoded information amount of motion vectors is larger than the total encoded information amount. To work. However, the motion vector information amount reduction technique of subtracting a certain amount from the basic evaluation value of motion estimation is accompanied by performance degradation of motion compensation. Therefore, when this method is used when the ratio of the encoded information amount of the motion vector is smaller than the encoded information amount of the prediction error signal, the encoding information of the prediction error signal is calculated from the reduction amount of the encoded information of the motion vector. In some cases, the amount of increase increases, and as a result, the coding efficiency deteriorates. Therefore, in the present invention, the quantization parameter QP (quantization parameter QP for QP is described as an example of H.263 in the “Prior Art” section) is involved in the encoded information amount of the prediction error signal in the present invention. The fact that (if QP is small,
The amount of code increases because the quantization step width becomes smaller. As shown in Equation 3, the H.263 QP has a mechanism in which the smaller the value, the smaller the quantization step width. Therefore, it is clear that the larger the QP, the smaller the amount of information required to encode the DCT coefficient. Along with this, it is clear that the amount of motion vector encoded information increases with respect to the total amount of code as QP increases. In the present invention, 0 vector or INTER4 at the time of motion estimation as shown in Equation 7 is used.
The difference value to the evaluation value of the INTER mode with respect to the V mode is small when QP is small, and large when QP is large, to cope with fluctuations in the encoded information amount of the DCT coefficient.

FIG. 6 shows a coding block diagram when the present invention is applied to the H.263 coding method. This diagram differs from the H.263 basic diagram FIG. 3 only in the local motion compensation processing unit 61 and the control device 63. This is only a difference in whether or not the local motion compensation processing unit 61 requires the quantization parameter determined by the control device 63. That means
There are very few additions and algorithmic additions to implement the present invention. 6
The description of FIG. 6 other than 1 and 63 overlaps with the description of FIG.

Next, the internal configuration of the local motion compensation processing unit 61 will be described with reference to FIG. FIG. 7 corresponds to FIG. 4 in the “prior art”. FIG. 7 differs from FIG. 4 only in that the INTER local motion estimation device 611 needs to be controlled by a quantization parameter.

Assuming that the input image is divided into a macroblocks (1 ≦ i ≦ a), IN
In the TER local motion estimation device 611

For each macroblock is performed on the candidate local motion vectors within the search range,
A local motion vector that minimizes Ei (u, v) is detected. The meanings of symbols used in Equation 10 are the same as those in Equation 7, except for n and m. n is a difference value subtracted from the sum of absolute values of prediction error signals in the block when the local motion vector is 0 in both the horizontal and vertical directions, and m is at least one of the local motion vectors in the horizontal and vertical directions is 0. If not, the difference value is subtracted from the absolute value sum of the prediction error signals in the block. This binary value is controlled by the quantization parameter QP. Normally, m should not be set larger than n for all QPs. If m is n
If it is larger, a motion vector with a large amount of encoded information of the motion vector is preferentially detected, so that the encoded information amount of the motion vector increases and the encoded information amount of the DCT coefficient also increases. become. For similar reasons, m must be a value greater than or equal to 0 for all QPs. As long as these two points are noted, various control methods of n and m can be considered. Here, for simplicity, m is fixed to 129 (similar to Equation 7), and a method for effectively controlling n

As an example. In this case, n = 201 when QP = 1, n = 258 when QP = 20, and n = 291 when QP = 31. In other words, when QP is larger than 20, compared with the case where n is fixed,
Further, the 0 vector is easily selected, and when the QP is smaller than 20, the 0 vector is difficult to select. This is consistent with the tendency that when the QP is small, fine quantization is performed, so that the effect of reducing the amount of motion vector information is offset by the increase in coding information of DCT coefficients. In addition, when QP is large, quantization is only roughly performed. Therefore, an increase in the coding information of the DCT coefficient due to a slight increase in the error signal is allowed by the quantization, and an effect of reducing the amount of motion vector information can be expected. .

The processing steps other than the INTER local motion estimation device 611 are the same as those in FIG. That is, INTER
The minimum value of the mode local motion estimation evaluation value Ei and the minimum value of the INTER4V mode local motion estimation evaluation value 4Ei, j are first transmitted to the controller 63. The control device 63 compares these two values, and if the minimum value of Ei is smaller than the minimum value of 4Ei, j or the minimum value of Ei and the minimum value of 4Ei, j are the same value, the INTER mode is changed to macroblock i. When the minimum value of 4Ei, j is smaller than the minimum value of Ei, the INTER4V mode is selected as the motion prediction type of macroblock i. Furthermore, from the input image activity and the above evaluation value, INTRA / INTER
Judgment is made. In response to the evaluation result, the control device 63 controls the prediction type switch 313, and the local motion vector according to the selected prediction type is output to the local motion compensation device 314. The local motion compensator 314 compensates the local motion vector of each macroblock for the reference image extracted from the frame memory 32 to generate a motion prediction image. The motion prediction image is output to the INTRA / INTER switch 39-1, and the local motion vector of each macroblock is output to the encoder 38.

Next, an example in which global motion compensation is incorporated in the H.263 encoding method will be described. FIG. 8 is an encoding diagram thereof. Here, a P picture that predicts motion in a positive direction in time will be described as an example. First, an input image is input to the local motion compensation processing unit 61 and the global motion compensation processing unit 81. Since the local motion compensation processing unit 61 operates in the same manner as in FIG. 6, the contents are as shown in FIG. 7 described above. I will omit the explanation here. The global motion compensation processing unit 81 first performs global motion compensation between the input image and the reference image extracted from the frame memory 32, and performs a new local motion estimation and motion compensation using the resulting image as a reference image. It has become. Although details will be described later, the evaluation value is controlled using the quantization parameter according to the present invention at the time of local motion estimation. Local motion compensation processing unit 61 and global motion compensation processing unit 8
1 is output to the GMC on / off switch 82-1, and the detected local motion vector is output to the GMC on / off switch 82-2 (GMC is a global motion).
Compensation (global motion compensation). These two switches are controlled by the control device 83. The control is controlled by INTER mode and INTER transmitted from 61 to the controller.
The minimum evaluation value in the 4V mode, the minimum evaluation value transmitted to the control device from 81, and the evaluation value in the INTRA mode calculated from the input image are compared, and the minimum value is selected. That is, global motion compensation is turned on when the minimum evaluation value transmitted from 81 is selected, and is turned off in other cases. The selection result is output from the control device 83 to the encoder 88 as GMC on / off information. In response to this evaluation result, the control device 83 switches to the switch 8.
2-1, 82-2 are controlled, the synthesized prediction image is output from the switch 82-1 to the INTRA / INTER switch 39-1, and the local motion vector of each macroblock is encoded from the switch 82-2 to the encoder 88. Is output. The global motion vector is output to 88 from the global motion compensation processing unit 81. Since other DCT and quantization methods are the same as those in FIGS. 3 and 6, they are omitted here.

FIG. 9 is a diagram showing details of the global motion compensation processing unit 81 in FIG. First, the input image (current frame) is input to the global motion estimation device 811, and global motion estimation is performed between the input image and the reference image extracted from the frame memory 32. Specifically, the four global motion vectors GMV1, GMV2, GMV3, and GMV4 shown in FIG. 5 are estimated and output to the encoder 88. There are many methods, for example, there is means for calculating an optimum value after predicting a global motion from a local motion vector in units of blocks. If four motion vectors can be estimated, b1 to b8 in Equation 9 can be uniquely calculated.
The global motion compensation image 812 can be used to generate a global motion compensation image.
Next, the INTER local motion estimation device 813 performs local motion estimation in the INTER mode between the global motion compensated image and the input image. Here, assuming that the input image is divided into a macroblocks (1 ≦ i ≦ a), the INTER local motion estimation device 8
13

For each macroblock is performed on the candidate local motion vectors within the search range,
A local motion vector that minimizes GEi (u, v) is detected. The meanings of the symbols used in Equation 12 are as follows.

F (x, y): Y signal of the original image of the frame to be encoded (current frame). x and y are integers.

G (x, y): Y signal of global motion compensation image. x and y are integers. The interpolation method in which the local motion vector is a real number follows Formula 2.

GEi (u, v): an evaluation value for the local motion vector (u, v) of the i-th block when the image F is divided into a 16 × 16 blocks.

GBi: Pixels included in the i-th block when the image F is divided into a 16 × 16 blocks. Each pixel belongs to the range of Xi ≦ x <Xi + 16 and Yi ≦ y <Yi + 16 (x and y are integers). (Xi, Yi
) Is the upper left pixel of block i.

GAi: Local motion vector within the motion search range of the i-th 16 × 16 block. Pixel accuracy is half pixel. The horizontal and vertical components u and v are real numbers limited to 0.5 increments.

k is a difference value subtracted from the sum of absolute values of prediction error signals in the block when the local motion vector is 0 in both the horizontal and vertical directions, and h is a case where at least one of the horizontal and vertical local motion vectors is not 0. The difference value subtracted from the absolute value sum of the prediction error signals in the block. This binary value is controlled by the quantization parameter QP. However, like n and m in Equation 10, normally, h should not be set larger than k for all QPs. In general

As shown, k is n, and h is the same QP control as m. The detected motion vector is output to the GMC on / off switch 82-2, and the minimum evaluation value is output to the control device 83. The local motion compensated image is output to the GMC on / off switch 82-1.

The local motion vector for the global motion compensated image has a characteristic that it has many small values, in particular, many zero vectors. Therefore, it is conceivable to perform local motion estimation for the global motion compensated image only when the local motion vector is 0 in both the horizontal and vertical directions. FIG. 1 shows the encoding diagram. 8 is that there is no output of a local motion vector from the global motion compensation processing unit 11 (81 in FIG. 8) to the GMC on / off switch 82-2. Therefore, when global motion compensation is turned on in 82-2, the local motion vector of the macroblock is not encoded. FIG. 10 shows the internal configuration of the global motion compensation processing unit 11. As in FIG. 9, the global motion estimation device 811 and the global motion compensation device 812 detect a global motion vector and generate a global motion compensated image. The encoder 18 and the GMC on / off switch 8 respectively.
2-1. Next, based on the control of the quantization parameter QP, the global motion compensation evaluation device 113 calculates an evaluation value for the zero vector by Equation 12. The information is transmitted to the control device 83. It is effective to set k in Equation 12 larger than n set in Equation 11. This is clear from the fact that local motion vectors do not need to be encoded when global motion compensation is turned on with the GMC on / off switch. Here, a method for effectively controlling k with respect to n and h

As an example. In this case, k = 216 when QP = 1, k = 258 when QP = 15, and k = 306 when QP = 31. In other words, when QP is greater than 15, compared to the case of fixed k,
Furthermore, the 0 vector is easily selected, and when the QP is smaller than 15, the 0 vector is difficult to select. Also, k is larger than n for all QPs, and the 0 motion vector when global motion compensation is on is more easily selected than the 0 motion vector when global motion compensation is off. . Thereby, when the QP is large, the information amount of the local motion vector is effectively reduced, and when the QP is small, the encoded information amount of the DCT coefficient quantization value accompanying the control giving priority to the specific motion vector is reduced. The increase is also suppressed.

Up to this point, the H.263 coding method has been described as an example. However, the present invention provides a technique for performing an orthogonal transform coefficient on a prediction error signal generated as a result of motion compensation and then quantizing the coefficient. All are applicable when used. Therefore, the quantization parameter QP of the present invention includes not only QUANT of H.263 but also H.261, MPQG1, and MPEG2 (for example, with regard to H.261, MPEG1, and MPEG2, for example, “Latest MPEG Textbook” supervised by Hiroshi Fujiwara (1994.8 MQUANT used in) is also included. In addition, if the encoding method has two or more patterns with different quantization step widths, the pattern itself or the specific quantization step width is set to the quantization parameter QP.
It is also possible to use as. Furthermore, there are two directions used in standard encoding schemes such as MPEG1 and MPEG2 that do not have a mode that requires two or more motion vectors in one macroblock such as INTER4V (normal The present invention can also be applied to a moving picture coding scheme including various motion prediction schemes, such as a coding scheme including bi-directional prediction that performs an operation of averaging images predicted in the opposite direction. For example, when applied to bi-directional prediction, the present invention can be used in motion compensation in the forward direction and in the reverse direction.

Since the present invention is mainly intended to provide two or more difference values from the basic evaluation function at the time of evaluating a specific motion vector depending on the value of QP, the control method of QP is not defined in the present invention. Therefore, as a control method, in addition to a method of handling QP like a function variable as shown in Equations 11, 12, and 13, a method of dividing QP into several cases may be considered. For example, the difference value from the basic evaluation function is 161 when QP is 1 to 5, 193 when 6 to 10, 257 when 11 to 20, and 2
The case where 1 to 25 is set to 321 and 26 to 31 is set to 387 is also included in the present invention.
In the above, m in Equation 11 and h in Equation 12 are fixed values (129), but a method of controlling them with QP is also included in the present invention. In this case, it is also assumed that m and h are larger than 0, m is not larger than n, and h is not larger than k for all QPs. And 2
When comparing the difference values, the difference value when the QP is small is set not to be larger than the difference value when the QP is large. When m or h is controlled in this way, if QP is large, INTER4V
There is an effect of reducing the four motion vectors required in the mode to one of INTER, and when QP is small, an increase in prediction error power by limiting the number of motion vectors to one can be reduced.

So far, motion compensation in a square block has been described, but the present invention can also be realized in a frame including a rectangular block or an arbitrarily shaped block. Therefore, such a case is also included in the present invention.

It is the figure which showed the example of the encoding block diagram containing the global motion compensation in this invention. It is the figure which showed the structure of the macroblock. It is the figure which showed the encoding block diagram of H.263 encoding system. It is the figure which showed the detail of the local motion compensation process part in the encoding block diagram of a H.263 encoding system. It is the figure which showed the principle of the global motion compensation method. It is the figure which showed the example of the encoding block diagram in the case of applying this invention to a H.263 encoding system. It is the figure which showed the detail of the local motion compensation process part in the example of an encoding block diagram in the case of applying this invention to a H.263 encoding system. It is the figure which showed the example of the encoding block diagram containing the global motion compensation in this invention, and the local motion compensation to the global motion compensation image. It is the figure which showed the detail of the global motion compensation process part in the example of the encoding block diagram containing the global motion compensation in this invention, and the local motion compensation to the global motion compensation image. It is the figure which showed the detail of the global motion compensation process part in the example of the encoding block diagram containing the global motion compensation in this invention.

Explanation of symbols

11, 81... Global motion compensation processing unit 31, 61... Local motion compensation processing unit, 32
... Frame memory, 33, 63, 83 ... Control device, 82-1, 82-2 ... GMC on / off switch, 113 ... Global motion compensation evaluation device, 311, 611 ... INTER local motion estimation device, 312 ... INTER4V local motion estimation device 313 ... INTER / INTER4V switch, 31
4 ... local motion compensation device, 504 to 507 ... grid point of original image of current frame, 509 to 5
12 ... Grid points after global motion estimation, 313 ... INTER / INTER4V switch, 314 ... Local motion compensation device, 811 ... Global motion estimation device, 812 ... Global motion compensation device.

Claims (10)

The smaller the value of the motion compensation performed by dividing an image into a (a is a positive integer) arbitrary shape block and the transform coefficient after orthogonal transformation is performed on the prediction error signal in the resulting block. In a video encoding method including a procedure of performing quantization using a quantization parameter QP (QP is a positive integer) that is set so that an average value of quantization step widths is small.
When both the horizontal and vertical motion vectors are 0, the evaluation value is a value obtained by subtracting n (n is an integer) from the sum of absolute values or sum of squares of the prediction error signals in the block. When at least one of them is not 0, an absolute value sum or a square sum of prediction error signals in the block is used as an evaluation value, and evaluation values for all candidate motion vectors to be searched for all motion compensation methods used for prediction are obtained. In comparison, when the motion vector when the value is the minimum is the motion vector of the block,
The value of n is divided into r (r is a positive integer of 2 or more) according to the value of QP, and 2
When comparing the values of n for two different QPs, the smaller n value of QP is the larger n value of QP.
A motion compensation method characterized in that the motion compensation method does not exceed the value of. The image is divided into a (a is a positive integer) arbitrary shape block, motion compensation for providing one motion vector in the block, and the block is divided into two or more small blocks, and the motion vector is divided for each small block. And a means for selecting one for each block from all motion compensation methods used for prediction including the above two types,
The quantization parameter QP (QP is a positive value), which is set so that the smaller the value, the smaller the average quantization step width is, after the orthogonal transformation is performed on the prediction error signal in the resulting block. In a moving image encoding method including a procedure of quantization using an integer of
When the block has one motion vector and both the horizontal and vertical motion vectors are 0, the evaluation value is a value obtained by subtracting n (n is an integer) from the absolute value sum or square sum of the prediction error signals in the block. When a block has one motion vector and at least one of the horizontal and vertical motion vectors is not 0, m (m is an integer) is subtracted from the sum of absolute values or sum of squares of prediction error signals in the block. When the value is an evaluation value and the motion vector of the block is two or more, the absolute value sum or square sum of the prediction error signals in the block is the evaluation value,
When comparing the evaluation values for all candidate motion vectors to be searched for all motion compensation methods used for prediction, and setting the motion vector at the minimum value as the motion vector of the block,
The value of n is divided into r (r is a positive integer of 1 or more) according to the value of QP.
When comparing the values of n for two different QPs, the smaller n value of QP is the larger n value of QP.
Will never be greater than
The value of m is divided into s (s is a positive integer of 1 or more) according to the value of QP.
When comparing the values of m for two different QPs, the value of m with the smaller QP is the value of m with the larger QP.
Will never be greater than
Any one of the above r and s is a value of 2 or more. The image is divided into a (a is a positive integer) arbitrarily shaped block, and the reference image is subjected to global motion compensation that compensates for the motion of the entire image, and the reference image is subjected to global motion compensation. Motion compensation performed as a non-image and means for selecting one for each block from all motion compensation methods used for prediction including the above two types;
The quantization parameter QP (QP is a positive value), which is set so that the smaller the value, the smaller the average quantization step width is, after the orthogonal transformation is performed on the prediction error signal in the resulting block. In a moving image encoding method including a procedure of quantization using an integer of
An image that has undergone global motion compensation is a reference image, and when both the horizontal and vertical motion vectors are 0, a value obtained by subtracting k (k is an integer) from the sum of absolute values or sum of squares of prediction error signals in the block Is the evaluation value,
An image that is not subjected to global motion compensation is a reference image, and when the horizontal and vertical motion vectors are both 0, the absolute value sum or square sum of the prediction error signals in the block
The value obtained by subtracting (n is an integer) is the evaluation value,
An image that is not subjected to global motion compensation is a reference image, and when at least one of the horizontal and vertical motion vectors is not 0, an absolute value sum or a square sum of prediction error signals in the block is used as an evaluation value.
For all motion compensation methods used for prediction, evaluation values for all candidate motion vectors to be searched are compared, and the motion vector and reference image when the value is the minimum are compared with the motion vector and reference of the block. When making an image,
The value of n is divided into r (r is a positive integer of 1 or more) according to the value of QP.
When comparing the values of n for two different QPs, the smaller n value of QP is the larger n value of QP.
Will never be greater than
The k value is divided into t (t is a positive integer of 1 or more) according to the value of QP.
When comparing the values of k for two different QPs, the value of k with the smaller QP is the value of k with the larger QP.
Will never be greater than
Any one of the above r and t is a positive integer value of 2 or more. The image is divided into a (a is a positive integer) arbitrarily shaped block, and the reference image is subjected to global motion compensation that compensates for the motion of the entire image, and the reference image is subjected to global motion compensation. Motion compensation performed as a non-image and means for selecting one for each block from all motion compensation methods used for prediction including the above two types;
The quantization parameter QP (QP is a positive value), which is set so that the smaller the value, the smaller the average quantization step width is, after the orthogonal transformation is performed on the prediction error signal in the resulting block. In a moving image encoding method including a procedure of quantization using an integer of
An image that has undergone global motion compensation is a reference image, and when both the horizontal and vertical motion vectors are 0, a value obtained by subtracting k (k is an integer) from the sum of absolute values or sum of squares of prediction error signals in the block Is the evaluation value,
When the image subjected to global motion compensation is a reference image, and at least one of the horizontal and vertical motion vectors is not 0, h (h is an integer) is subtracted from the absolute value sum or square sum of the prediction error signals in the block. The obtained value as the evaluation value,
An image that is not subjected to global motion compensation is a reference image, and when the horizontal and vertical motion vectors are both 0, the absolute value sum or square sum of the prediction error signals in the block
The value obtained by subtracting (n is an integer) is the evaluation value,
An image that is not subjected to global motion compensation is a reference image, and when at least one of the horizontal and vertical motion vectors is not 0, an absolute value sum or a square sum of prediction error signals in the block is used as an evaluation value.
For all motion compensation methods used for prediction, evaluation values for all candidate motion vectors to be searched are compared, and the motion vector and reference image when the value is the minimum are compared with the motion vector and reference of the block. When making an image,
The value of n is divided into r (r is a positive integer of 1 or more) according to the value of QP.
When comparing the values of n for two different QPs, the smaller n value of QP is the larger n value of QP.
Will never be greater than
The value of h is divided into u cases (u is a positive integer of 1 or more) according to the value of QP, and 2
When comparing the values of h for two different QPs, the value of h with the smaller QP is the value of h with the larger QP.
Will never be greater than
The k value is divided into t (t is a positive integer of 1 or more) according to the value of QP.
When comparing the values of k for two different QPs, the value of k with the smaller QP is the value of k with the larger QP.
Will never be greater than
Any one of the r, u, and t is a positive integer of 2 or more. The image is divided into a (a is a positive integer) arbitrarily shaped block, and the reference image is subjected to global motion compensation that compensates for the motion of the entire image, and the reference image is subjected to global motion compensation. A motion compensation in which one motion vector is provided in a block, and a reference image in which a global motion compensation is not performed, a block is divided into two or more small blocks, and a motion vector is assigned to each small block. Motion compensation to be provided, and means for selecting one for each block from all motion compensation methods used for prediction including the above three types;
The quantization parameter QP (QP is a positive value), which is set so that the smaller the value, the smaller the average quantization step width is, after the orthogonal transformation is performed on the prediction error signal in the resulting block. In a moving image encoding method including a procedure of quantization using an integer of
An image that has undergone global motion compensation is a reference image, and when both the horizontal and vertical motion vectors are 0, a value obtained by subtracting k (k is an integer) from the sum of absolute values or sum of squares of prediction error signals in the block age,
An image that is not subjected to global motion compensation is a reference image, and there is one block motion vector. When both horizontal and vertical motion vectors are 0, the sum of absolute values or square sums of prediction error signals in the block Subtracting n (n is an integer) from
An image that is not subjected to global motion compensation is a reference image, and there is one block motion vector. When at least one of the horizontal and vertical motion vectors is not 0, the sum of absolute values of prediction error signals in the block or Subtract m (m is an integer) from the sum of squares,
An image that has not been subjected to global motion compensation is a reference image, and when the motion vector of the block is two or more, the absolute value sum or square sum of the prediction error signals in the block is used as the evaluation value,
For all motion compensation methods used for prediction, evaluation values for all candidate motion vectors to be searched are compared, and the motion vector and reference image when the value is the minimum are compared with the motion vector and reference of the block. When making an image,
The value of n is divided into r (r is a positive integer of 1 or more) according to the value of QP.
When comparing the values of n for two different QPs, the smaller n value of QP is the larger n value of QP.
Will never be greater than
The value of m is divided into s (s is a positive integer of 1 or more) according to the value of QP.
When comparing the values of m for two different QPs, the value of m with the smaller QP is the value of m with the larger QP.
Will never be greater than
The k value is divided into t (t is a positive integer of 1 or more) according to the value of QP.
When comparing the values of k for two different QPs, the value of k with the smaller QP is the value of k with the larger QP.
Will never be greater than
Any one of said r, s, and t is a positive integer greater than or equal to 2, The motion compensation method characterized by the above-mentioned. The image is divided into a (a is a positive integer) arbitrarily shaped block, and the reference image is subjected to global motion compensation that compensates for the motion of the entire image, and the reference image is subjected to global motion compensation. A motion compensation in which one motion vector is provided in a block, and a reference image in which a global motion compensation is not performed, a block is divided into two or more small blocks, and a motion vector is assigned to each small block. Motion compensation to be provided, and means for selecting one for each block from all motion compensation methods used for prediction including the above three types;
The quantization parameter QP (QP is a positive value), which is set so that the smaller the value, the smaller the average quantization step width is, after the orthogonal transformation is performed on the prediction error signal in the resulting block. In a moving image encoding method including a procedure of quantization using an integer of
An image that has undergone global motion compensation is a reference image, and when both the horizontal and vertical motion vectors are 0, a value obtained by subtracting k (k is an integer) from the sum of absolute values or sum of squares of prediction error signals in the block age,
When the image subjected to global motion compensation is a reference image, and at least one of the horizontal and vertical motion vectors is not 0, h (h is an integer) is subtracted from the sum of absolute values or sum of squares of prediction error signals in the block. Value and
An image that is not subjected to global motion compensation is a reference image, and there is one block motion vector. When both horizontal and vertical motion vectors are 0, the sum of absolute values or square sums of prediction error signals in the block Subtracting n (n is an integer) from
An image that is not subjected to global motion compensation is a reference image, and a block has a single motion vector. When at least one of the horizontal and vertical motion vectors is not 0, the sum of absolute values of prediction error signals in the block or Subtract m (m is an integer) from the sum of squares,
An image that has not been subjected to global motion compensation is a reference image, and when the motion vector of the block is two or more, the absolute value sum or square sum of the prediction error signals in the block is used as the evaluation value,
For all motion compensation methods used for prediction, evaluation values for all candidate motion vectors to be searched are compared, and the motion vector and reference image when the value is the minimum are compared with the motion vector and reference of the block. When making an image,
The value of n is divided into r (r is a positive integer of 1 or more) according to the value of QP.
When comparing the values of n for two different QPs, the smaller n value of QP is the larger n value of QP.
Will never be greater than
The value of m is divided into s (s is a positive integer of 1 or more) according to the value of QP.
When comparing the values of m for two different QPs, the value of m with the smaller QP is the value of m with the larger QP.
Will never be greater than
The k value is divided into t (t is a positive integer of 1 or more) according to the value of QP.
When comparing the values of k for two different QPs, the value of k with the smaller QP is the value of k with the larger QP.
Will never be greater than
The value of h is divided into u cases (u is a positive integer of 1 or more) according to the value of QP, and 2
When comparing the values of h for two different QPs, the value of h with the smaller QP is the value of h with the larger QP.
Will never be greater than
Any one of the above r, s, t, and u is a positive integer of 2 or more. 7. The motion compensation method according to claim 1, wherein the value of r is 5 or more. 7. The motion compensation method according to claim 1, wherein the value of r is 3 or more.   A moving picture encoding method including the motion compensation method according to claim 1. A video encoding apparatus using a video encoding method including the motion compensation method according to claim 1.

Images