JP2011151775A - Image decoder, method of decoding image, and image decoding program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To increase compression efficiency of a coding amount by an image coding system using motion compensation prediction based on translation and motion compensation prediction based on geometrical conversion in combination. <P>SOLUTION: A decoding unit 250 decodes prediction method information, a motion vector, and a prediction error signal included in coding stream. A change control unit 214 refers to the prediction method information decoded by the decoding unit 250, and designates use of one of a prediction method by a translational motion compensation prediction unit 204 and a prediction method by a geometrical conversion motion compensation prediction unit 205 for each target block inside a target image. The decoding unit 250 decodes the motion vector of a predictively coded target block by referring to the motion vector of an adjacent block regardless of the motion compensation prediction method of the adjacent block. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image decoding apparatus, an image decoding method, and an image decoding program for decoding an encoded stream encoded using motion compensation prediction.

  As a typical moving image compression coding method, there is a standard of the MPEG series. In the MPEG series standard, motion compensation that divides a frame into a plurality of blocks and predicts motion from other frames is used. MPEG-4 and AVC / H. In H.264, a mechanism for switching and using an optimum one from a plurality of motion compensation block sizes is introduced.

  In motion compensation prediction in units of blocks, a method of compensating for parallel movement between a target block and a reference block is common. In addition to this, a scheme for compensating for block deformation (for example, enlargement, reduction, rotation) has been studied. For example, in Patent Document 1, as an image encoding method using prediction between frames, a mode in which a predicted image is obtained by parallel movement and a mode in which a predicted image is obtained by geometric transformation are adaptively switched for each block, thereby predicting efficiency. We are trying to improve. In this method, a translation motion vector and a lattice point motion vector (that is, a motion vector used in geometric transformation) are encoded.

JP-A-8-68680

  Under such circumstances, the present inventor further reduced the overall code amount by compressing the motion vector information in an image coding method that uses both motion compensation prediction by parallel movement and motion compensation prediction by geometric transformation. I found a method to compress.

  The present invention has been made in view of such a situation, and an object of the present invention is to improve the compression efficiency of a code amount in an image coding method using both motion compensation prediction by parallel movement and motion compensation prediction by geometric transformation. Is to provide.

  An image decoding apparatus according to an aspect of the present invention includes a prediction method information, a motion vector, and a prediction error signal included in an encoded stream encoded by combining motion compensation prediction by parallel movement and motion compensated prediction by geometric transformation. A prediction signal is generated from a decoding unit that decodes the motion block, a motion block between the target block in the target image, and a reference block in the reference image that is translated with the target block, and the image signal of the reference block A prediction signal is obtained from the translation motion compensation prediction unit, the motion block between the target block in the target image, and the reference block in the reference image that is geometrically transformed with the target block, and the image signal of the reference block. For each target block in the target image with reference to the prediction method information decoded by the geometric transformation motion compensation prediction unit to be generated and the decoding unit A control unit that specifies which one of a prediction method by a translational motion compensation prediction unit and a prediction method by a geometric transformation motion compensation prediction unit is used, a prediction signal generated by a prediction method specified by the control unit, and decoding An image signal generation unit that generates an image signal from the prediction error signal decoded by the unit. The motion vector of the target block is predictively encoded with reference to the motion vector of the adjacent block regardless of whether the motion compensation prediction method of the adjacent block is translation motion compensation prediction or geometric conversion motion compensation prediction. The decoding unit decodes the motion vector of the target block subjected to predictive encoding with reference to the motion vector of the adjacent block regardless of the motion compensation prediction method of the adjacent block.

  A plurality of representative pixels are selected for the target block, and motion vectors of the representative pixels are predictively encoded and included in the encoded stream. The geometric transformation motion compensation prediction unit includes a plurality of representative pixels of the target block. The motion vectors of the other pixels may be calculated by at least one of interpolation and extrapolation using the decoded motion vectors of the plurality of representative pixels.

  The representative pixel may be selected as a pixel corresponding to the approximate center or the approximate center of gravity of the small block when the target block is divided into small blocks.

  The target block is a square area, and the prediction method information for designating motion compensated prediction by the geometric transformation motion compensated prediction unit includes four representatives of four square small blocks obtained by dividing the target block into four equal parts. A first mode in which a motion vector is detected for each pixel, a second mode in which a motion vector is detected for each of two representative pixels of two vertically long rectangular blocks obtained by dividing the target block into two equal parts, and a target Any of the third modes in which a motion vector is detected for each of two representative pixels of two horizontally-long rectangular small blocks obtained by dividing a block into two vertically may be specified.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  The compression efficiency of the code amount can be improved by an image coding method using both motion compensation prediction by parallel movement and motion compensation prediction by geometric transformation.

It is a block diagram which shows the structure of the image coding apparatus which concerns on Embodiment 1 of this invention. 2A to 2H are diagrams for explaining a macroblock partition and a sub-macroblock partition. FIGS. 3A to 3C are diagrams for explaining representative points of the first mode, the second mode, and the third mode of the geometric transformation. 4A to 4D are diagrams for explaining motion vector predictive encoding. It is a figure for demonstrating an example of the scaling process of a motion vector value. It is a flowchart which shows the encoding process procedure of a macroblock in the image coding apparatus which concerns on Embodiment 1 of this invention. It is a block diagram which shows the structure of the image decoding apparatus which concerns on Embodiment 2 of this invention. It is a flowchart which shows the decoding process procedure of a macroblock in the image decoding apparatus which concerns on Embodiment 2 of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following embodiments, AVC / H. An example of encoding / decoding based on the H.264 encoding method will be described.

  FIG. 1 is a block diagram showing a configuration of image coding apparatus 100 according to Embodiment 1 of the present invention. The image encoding device 100 includes an image buffer 101, a translational motion compensation prediction unit 102, a geometric transformation motion compensation prediction unit 103, a prediction method determination unit 104, a prediction error signal generation unit 105, and an encoding unit 150. The encoding unit 150 includes a prediction error signal encoding unit 106, a first encoded bit sequence generation unit 107, a second encoded bit sequence generation unit 108, a third encoded bit sequence generation unit 109, a prediction error signal decoding unit 110, and a decoded image. A signal generation unit 111, a decoded image buffer 112, and an output switch 113 are included.

  These configurations can be realized by an arbitrary processor, memory, or other LSI in terms of hardware, and can be realized by a program loaded in the memory in terms of software, but here by their cooperation. Draw functional blocks. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The image buffer 101 temporarily stores image signals to be encoded supplied in the order of shooting / display time. The image buffer 101 converts the stored image signal to be encoded in predetermined pixel block units (here, macroblock units) into a translational motion compensation prediction unit 102, a geometric transformation motion compensation prediction unit 103, and a prediction error. The signal is supplied to the signal generator 105 in parallel. At this time, the images supplied in the order of shooting / display time are rearranged in the encoding order and output from the image buffer 101.

  In the MPEG series, a macroblock refers to a luminance signal of 16 × 16 pixels and two corresponding color difference signal blocks. When YUV of the color difference format is 4: 2: 0, the color difference signal has a size of 8 × 8 pixels.

  In the present embodiment, an intra coding method that encodes within a screen without using a reference image, a motion compensation prediction method by translation using a reference image, and a motion compensation prediction method by geometric transformation using a reference image Is used. Here, the reference image is a decoded image obtained by local decoding. In the present embodiment, since attention is not paid to the intra coding method, the configuration is omitted in FIG. These coding system modes are adaptively switched individually or in combination in units of macroblocks. Note that a method of encoding all macroblocks using a motion compensated prediction method based on geometric transformation using a reference image is also possible.

  The translational motion compensation prediction unit 102 generates a motion vector and a prediction signal between the target block in the target image and the reference block in the reference image that is in a translational relationship with the target block. In the present embodiment, the translational motion compensation prediction unit 102 performs motion compensation prediction based on translation similar to the existing motion compensation prediction stipulated in the AVC / H.264 scheme or the like.

  In motion compensation prediction, a forward or backward decoded image is used as a reference image in the display order supplied from a decoded image buffer 112 described later. The translational motion compensation prediction unit 102 performs block matching between the macroblock signal supplied from the image buffer 101 and the reference image signal supplied from the decoded image buffer 112 within a predetermined detection range in the reference image. Do. The translational motion compensation prediction unit 102 specifies a reference block signal in the reference image signal having the smallest error from the macroblock signal by block matching, and a motion vector between the macroblock signal and the reference block signal Is detected.

  This block matching is performed in a plurality of prescribed modes. Each of the plurality of modes has a different reference index, motion compensation block size, L0 / L1 prediction, and the like. The reference index is an index indicating a reference picture. Note that L0 / L1 prediction can be selected only in the B slice. A specific example of the block size will be described later.

  In addition, when performing motion compensation prediction, motion compensation can be performed with a pixel accuracy of less than one pixel. For example, in the AVC / H.264 system or the like, motion compensation can be performed up to an accuracy of 1/4 of a luminance signal and 1/8 of an accuracy of a color difference signal. When motion compensation is performed with pixel accuracy of less than one pixel, a signal with pixel accuracy of less than one pixel is generated from the surrounding signals by interpolation using a signal of an integer pixel in the reference image.

  The translational motion compensation prediction unit 102 performs motion compensation prediction in each mode, and sends a prediction signal (more specifically, a motion compensation prediction block signal) and a motion vector in each mode to the prediction method determination unit 104. Supply.

Next, AVC / H. The motion compensation block size in the H.264 system will be described.
FIG. 2 is a diagram for explaining a macroblock partition and a sub-macroblock partition. Here, in order to simplify the description, only the pixel block of the luminance signal is drawn. AVC / H. In the MPEG series including the H.264 system, the macroblock is defined by a square area. Generally, AVC / H. In the MPEG series including the H.264 system, a block defined by 16 × 16 pixels (16 horizontal pixels and 16 vertical pixels) is called a macro block. Furthermore, AVC / H. In the H.264 system, a block defined by 8 × 8 pixels is called a sub macroblock. The macroblock partition refers to each small block obtained by further dividing the macroblock for motion compensation prediction. The sub-macroblock partition refers to each small block obtained by further dividing the sub-macroblock for motion compensation prediction.

  FIG. 2A is a diagram showing that a macroblock is composed of one macroblock partition composed of a luminance signal of 16 × 16 pixels and two corresponding color difference signals. Here, this configuration is referred to as a 16 × 16 mode macroblock type.

  In FIG. 2B, the macroblock is composed of two macroblock partitions each composed of a luminance signal of 16 × 8 pixels (horizontal 16 pixels, vertical 8 pixels) and two corresponding color difference signals. FIG. The two macroblock partitions are arranged vertically. Here, this configuration is called a 16 × 8 mode macroblock type.

  In FIG. 2 (c), the macroblock is composed of two macroblock partitions composed of a luminance signal of 8 × 16 pixels (horizontal 8 pixels, vertical 16 pixels) and two corresponding color difference signals. FIG. The two macroblock partitions are arranged side by side. Here, this configuration is referred to as an 8 × 16 mode macroblock type.

  FIG. 2D is a diagram showing that a macroblock is composed of four macroblock partitions each composed of a luminance signal of 8 × 8 pixels and two corresponding color difference signals. These four macroblock partitions are arranged vertically and horizontally. This configuration is referred to as an 8 × 8 mode macroblock type. Note that a macroblock partition including the luminance signal of 8 × 8 pixels and two corresponding color difference signals is referred to as a sub macroblock.

  FIG. 2 (e) is a diagram showing that the sub macroblock is composed of one submacroblock partition composed of a luminance signal of 8 × 8 pixels and two corresponding color difference signals. Here, this configuration is referred to as an 8 × 8 mode sub-macroblock type.

  In FIG. 2F, the sub-macroblock is composed of two sub-macroblock partitions each composed of a luminance signal of 8 × 4 pixels (horizontal 8 pixels, vertical 4 pixels) and two corresponding color difference signals. FIG. These two sub-macroblock partitions are arranged vertically. This configuration is referred to as an 8 × 4 mode sub-macroblock type.

  In FIG. 2G, the sub-macroblock is composed of two macroblock partitions each composed of a luminance signal of 4 × 8 pixels (horizontal 4 pixels, vertical 8 pixels) and two corresponding color difference signals. FIG. The two macroblock partitions are arranged side by side. Here, this configuration is called a 4 × 8 mode sub-macroblock type.

  FIG. 2 (h) is a diagram showing that the sub-macroblock is composed of four sub-macroblock partitions composed of a luminance signal of 4 × 4 pixels and two corresponding color difference signals. These four sub-macroblock partitions are arranged two by two vertically and horizontally. Here, this configuration is called a 4 × 4 mode sub-macroblock type.

  AVC / H. In the H.264 encoding method, a mechanism for switching and using an optimum one from the above motion compensation block sizes is adopted. First, one of macroblock types of 16 × 16, 16 × 8, 8 × 16, and 8 × 8 modes is selected as the motion compensation block size for each macroblock. When the 8 × 8 mode macroblock type is selected, the motion compensation block size in units of submacroblocks is among the 8 × 8, 8 × 4, 4 × 8, and 4 × 4 mode sub macroblock types. Is selected.

  The luminance signal is motion-compensated with the number of pixels of the selected size. When the color difference format is 4: 2: 0, the color difference signal is motion-compensated with half the number of pixels both horizontally and vertically. In this way, motion compensation block size information is encoded with syntax elements called macroblock types and sub-macroblock types. The syntax is an expression rule of the encoded bit string, and the syntax element is information that is specified to be transmitted in the syntax.

  For any macroblock type of 16 × 16, 16 × 8, 8 × 16 and 8 × 8 modes, one motion vector is encoded / decoded for each macroblock partition in the case of L0 prediction and L1 prediction. . That is, one motion vector for the 16 × 16 mode macroblock type, two motion vectors for the 16 × 8 and 8 × 16 mode macroblock types, and 4 for the 8 × 8 mode macroblock type. Each motion vector is encoded and decoded.

  Each pixel of the luminance signal and chrominance signal of each macroblock partition is motion compensated according to one motion vector of the macroblock partition. That is, each pixel is motion-compensated using the same motion vector.

  Returning to FIG. 1, the geometric transformation motion compensation prediction unit 103 performs motion vector and prediction signal between the target block in the target image and the reference block in the reference image having a geometric transformation relationship with the target block. Is generated. More specifically, the geometric transformation motion compensation prediction unit 103 sets a representative pixel for each small block when the target block is virtually divided into a plurality of small blocks, and the motion vector of each representative pixel is set. Is detected. At that time, the geometric transformation motion compensation prediction unit 103 sets the pixel at the approximate center or the approximate center of gravity of the small block as the representative pixel.

  Further, the geometric transformation motion compensation prediction unit 103 calculates a motion vector of pixels other than the plurality of representative pixels of the target block using at least one of interpolation and extrapolation from the motion vectors of the plurality of representative pixels. In the following description according to the embodiment of the present invention, the notation of interpolation / extrapolation is used to mean at least one of interpolation and extrapolation. The geometric transformation motion compensation prediction unit 103 encodes and decodes motion vectors of four representative pixels a, b, c, and d of the target block (see FIG. 3A described later), and the target block. The second mode for encoding and decoding the motion vectors of the two representative pixels e and f in the vertical direction (see FIG. 3B described later), and the motion of the two representative pixels g and h in the horizontal direction of the target block A third mode for encoding / decoding vectors (see FIG. 3C described later) can be executed. Here, in the first mode, when the target block is virtually divided into four crosses and divided into four square small blocks, the center or approximately center of each of the four small blocks is set as a representative pixel, and the second mode Is a case where the target block is virtually divided into two vertically divided and divided into two horizontally-long rectangular small blocks, and the center or approximate center of each of the two small blocks is used as a representative pixel. Specifically, the center or approximately center of each of the two small blocks when divided into two vertically long rectangular small blocks by dividing into two horizontally is set as a representative pixel.

  Here, in the description according to the embodiment of the present invention, the target block is described as a macro block of 16 × 16 pixels, but the size of the target block is not limited to 16 × 16 pixels. It may be a sub-macroblock of 8 pixels, or a block of 32 × 32 pixels, 48 × 48 pixels, 64 × 64 pixels, or the like. Although the small block is described as a macroblock partition, the present invention is not limited to this, and the block may be a sub macroblock and the small block may be a sub macroblock partition.

  The prediction method determination unit 104 can employ any of the first mode, the second mode, and the third mode described above as the prediction method by the geometric transformation motion compensation prediction unit 103. Detailed processing of the prediction method determination unit 104 will be described later.

  Hereinafter, the geometric transformation motion compensation prediction unit 103 will be described more specifically. The geometric conversion motion compensation prediction unit 103 is different from the existing motion compensation prediction based on the parallel movement defined in the AVC / H.264 method, etc., and in addition to the parallel movement, the geometric transformation accompanied by deformation including enlargement / reduction / rotation, etc. Perform motion compensation prediction by scientific conversion.

  In motion compensated prediction by geometric transformation according to the present embodiment, each pixel of the luminance signal and chrominance signal of the macroblock, macroblock partition, sub-macroblock is not motion-compensated with the same motion vector, but for each pixel. Different motion vectors are created for motion compensation. The first mode of geometric transformation motion compensated prediction corresponds to the macroblock type of 8x8 mode, the second mode corresponds to the macroblock type of 16x8, and the third mode corresponds to the macroblock type of 8x16 To do. In other words, in motion compensated prediction by geometric transformation, the macroblock type is not defined by the shape of the pixel block, but by the number of motion vectors, representative pixel positions, and motion vector interpolation / extrapolation methods. Is done. The motion vector of the non-representative pixel is calculated by at least one of interpolation and extrapolation based on the motion vector of the representative pixel.

  The geometric transformation motion compensation prediction unit 103 selects a pixel at the center or near the center of each macroblock partition including the sub macroblock as a representative pixel, and obtains a motion vector thereof. In the case of a macroblock partition, since the number of vertical and horizontal pixels is an even number, there is no exact center pixel. Therefore, a specific one of the four pixels located at the center is set as a representative pixel, or the coordinates of the exact center pixel are obtained by interpolation to set a virtual pixel as the representative pixel.

  The geometric transformation motion compensated prediction unit 103 may use a motion vector calculated in units of pixels by an optical flow method or the like as a motion vector of a representative pixel of each macroblock partition, or a reliability such as an edge or corner of an image. You may correct | amend by the interpolation and extrapolation calculation from the motion vector of the feature point which can be judged to have high property. The motion vector of the corresponding macroblock / partition generated by the translational motion compensation / prediction unit 102 may be used. In addition, when using the motion vector of the corresponding macroblock partition, the motion vector value is applied to the representative pixel, and the motion vector value is verified by adjusting the motion vector value in the increasing direction and decreasing direction. May be corrected.

  Next, the geometric transformation motion compensation prediction unit 103 performs linear interpolation / extrapolation of the motion vector of the non-representative pixel in the macroblock from the motion vector of the representative pixel (in this embodiment, bilinear interpolation / extrapolation). Calculated by

  In the case of the second mode in which the motion vectors of the two representative pixels e and f in the vertical direction are encoded and decoded, the geometric transformation motion compensation prediction unit 103 calculates the motion vectors of the two representative pixels e and f in the vertical direction from A motion vector of a pixel on a straight line connecting these two points is calculated by at least one of interpolation and extrapolation. Then, the motion vectors of the other pixels and the motion vectors of the respective pixels subjected to linear interpolation / extrapolation are applied as they are in the horizontal direction. It is assumed that an operation in which a motion vector is directly applied is included in the interpolation / extrapolation. In the case of the third mode in which the motion vectors of the two representative pixels g and h in the horizontal direction are encoded and decoded, the geometric transformation motion compensation prediction unit 103 calculates these two points from the motion vectors of the two representative pixels in the horizontal direction. The motion vector of the pixels on the straight line connecting the two is calculated by at least one of interpolation and extrapolation, and the motion vectors of the other pixels are used as they are in the vertical direction. Apply.

  In the first mode in which the motion vectors of the four representative pixels a, b, c, and d are encoded / decoded, the geometric transformation motion compensation prediction unit 103 performs interpolation and extrapolation both in the horizontal direction and in the vertical direction. The motion vector of the non-representative pixel is calculated by at least one of them. Interpolation and extrapolation may be performed at once in both the horizontal and vertical directions by the method described later, or the motion vectors of pixels on a straight line connecting these two points are interpolated from the motion vectors of two representative pixels in the horizontal direction. By calculating the motion vector of the non-representative pixel by interpolation / extrapolation, and further interpolating / extrapolating the motion vector of the other pixels in the vertical direction using the motion vector of each pixel that has already been calculated. It may be calculated.

  In the second mode in which the motion vectors of the two representative pixels e and f in the vertical direction are encoded and decoded, the value of the motion vector of the representative pixel e is set to the value of the motion vector of the representative pixels a and b. The value of the motion vector of the representative pixel f is set to the value of the motion vector of the representative pixels c and d. Then, similarly to the first mode in which the motion vectors of the four representative pixels a, b, c, and d are encoded and decoded, the motion vectors of the non-representative pixels can be calculated.

  Similarly, in the third mode in which the motion vectors of the two representative pixels g and h in the horizontal direction are encoded and decoded, the value of the motion vector of the representative pixel g is set to the value of the motion vector of the representative pixels a and c. Then, the value of the motion vector of the second representative pixel h is set to the value of the motion vector of the representative pixels b and d. Then, similarly to the first mode in which the motion vectors of the four representative pixels a, b, c, and d are encoded and decoded, the motion vectors of the non-representative pixels can be calculated.

  The geometric transformation motion compensation prediction unit 103 performs motion compensation for each pixel using the calculated motion vector for each pixel. In the above description, the example of calculating the motion vector of each pixel included in the macroblock has been described. However, the motion vector of each pixel included in the sub-macroblock can be calculated in the same manner. By such processing, deformation including enlargement / reduction / rotation can be expressed, and processing equivalent to geometric transformation can be realized.

  The pixel-by-pixel motion vector calculation process and the motion compensation process for each pixel are performed in a plurality of prescribed modes. Each of the plurality of modes has a different reference index, motion compensation block size, L0 / L1 prediction, and the like. Note that L0 / L1 prediction can be selected only in the B slice.

  The geometric transformation motion compensation prediction unit 103 performs motion compensation prediction in each mode, and predicts a prediction signal (more specifically, a motion compensation prediction block signal) and a motion vector in each mode. To supply.

Hereinafter, the calculation method of the motion vector of each pixel in the geometric conversion motion compensation prediction will be described with a specific example.
FIG. 3A illustrates a method of calculating the motion vector of each pixel of the macroblock type in the first mode that encodes and decodes the motion vectors of the four representative pixels a, b, c, and d of the target block. It is a figure for doing. In FIG. 3A, the representative pixel is represented by a black circle, the non-representative pixel is represented by a white circle, and the representative pixel is represented by a pixel a (3, 3), a pixel b (11, 3), which is near the center of each macroblock partition. Pixel c (3, 11) and pixel d (11, 11) are set. Here, the coordinate of each pixel is represented by (i, j), the coordinate in the horizontal direction is represented by i in the unit of one pixel, and the coordinate in the vertical direction is represented by j in the unit of one pixel. The upper left pixel in the macroblock is the origin (0, 0), and the right direction and the downward direction are positive.

  First, motion vectors of the pixel a, pixel b, pixel c, and pixel d are assigned, and then motion vectors of other pixels are calculated by linear interpolation / extrapolation.

  In the first mode in which the motion vectors of the four representative pixels a, b, c, and d are encoded and decoded, the motion vectors of the pixel a, the pixel b, the pixel c, and the pixel d are assigned. In the second mode in which the motion vectors of the two representative pixels e and f are encoded and decoded, the motion vectors of the pixel e and the pixel f are assigned, and the motion vector of the pixel e is also the motion vector of the pixels a and b. The motion vector of the pixel f is also the motion vector of the pixels c and d. In the third mode in which the motion vectors of the two representative pixels g and h are encoded and decoded, the motion vectors of the pixel g and the pixel h are assigned, and the motion vector of the pixel g is also the motion vector of the pixels a and c. The motion vector of pixel h is also the motion vector of pixels b and d.

Next, the motion vectors V (i, j) of the other pixels are calculated from the motion vectors Va, Vb, Vc, Vd of the four representative pixels a, b, c, d by linear interpolation / extrapolation. .
V (i, j) = {(15-i-3) (15-j-3) Va + i (15-j-3) Vb + (15-i-3) (j-3) .Vc + (i-3) (J-3) Vd} Expression (1)
As described above, the motion vector for each pixel in the first mode, the second mode, and the third mode is calculated. It can also be calculated by the following method.

Pixel a motion vector Va = V (3,3), pixel b motion vector Vb = V (11,3), pixel c motion vector Vc = V (3,11), and pixel d motion vector Vd = Assuming V (11,11), the motion vector V (i, 3) of each pixel on the line passing through the pixel a and the pixel b is calculated by the following equation (2).
V (i, 3) = Va + (Vb−Va) * (i−3) / 8 Expression (2)

Similarly, the motion vector V (i, 11) of each pixel on the line passing through the pixels c and d is calculated by the following equation (3).
V (i, 11) = Vc + (Vd−Vc) * (i−3) / 8 Expression (3)

Furthermore, the motion vectors V (i, j) of the remaining pixels are calculated by the following equation (4).
V (i, j) = V (i, 3) + (V (i, 11) −V (i, 3)) * (j−3) / 8 Expression (4)

  As described above, the motion vector of each pixel in the first mode, the second mode, and the third mode can be calculated. In the second mode and the third mode, it can also be calculated by the following method.

  FIG. 3B is a diagram for explaining a method of calculating the motion vector of each pixel of the macroblock type in the second mode that encodes and decodes the motion vectors of the two representative pixels e and f of the target block. It is. In FIG. 3B, the representative pixels are set to the pixel e (7, 3) and the pixel f (7, 11) that are near the center of each macroblock partition.

  First, motion vectors of these pixels e and f are assigned, and then motion vectors of other pixels are calculated by linear interpolation / extrapolation.

When the motion vector Ve = V (7,3) of the pixel e and the motion vector of the pixel f are Vf = V (7,11), the motion vector V (7,3) of each pixel on the line passing through the pixel e and the pixel f. j) is calculated by the following equation (5).
V (7, j) = Ve + (Vf−Ve) * (j−3) / 8 Expression (5)

Next, as shown in the following formula (6), the value of the motion vector V (7, j) calculated by the formula (5) is expanded in the horizontal direction, and the value of the motion vector V (7, j) is left as it is. Is substituted into the motion vector V (i, j) of the pixel.
V (i, j) = V (7, j) (6)

The motion vector V (i, j) of the remaining pixels may be calculated by the following equation (7).
V (i, j) = Ve + (Vf−Ve) * (j−3) / 8 Expression (7)

  FIG. 3C illustrates a method of calculating the motion vector of each pixel of the third mode macroblock type that encodes and decodes the motion vectors of the two representative pixels g and h in the horizontal direction of the target block. FIG. In FIG. 3C, the representative pixels are set to the pixel g (3, 7) and the pixel h (11, 7) that are near the center of each macroblock partition.

  First, the motion vectors of these pixels g and h are assigned, and then the motion vectors of the other pixels are calculated by linear interpolation / extrapolation.

If the motion vector Vg = V (3,7) of the pixel g and the motion vector Vh = V (11,7) of the pixel h, the motion vector V (i, 7) of each pixel on the line passing through the pixel g and the pixel h. ) Is calculated by the following equation (8).
V (i, 7) = Vg + (Vh−Vg) * (i−3) / 8 Expression (8)

Next, as shown in the following equation (9), the value of the motion vector V (i, 7) calculated by the equation (8) is expanded in the vertical direction, and the value of the motion vector V (i, 7) is left as it is. Is substituted into the motion vector V (i, j) of the pixel.
V (i, j) = V (i, 7) (9)

The motion vector V (i, j) of the remaining pixels may be calculated by the following equation (10).
V (i, j) = Vg + (Vh−Vg) * (i−3) / 8 Expression (10)

  The geometric transformation motion compensation prediction unit 103 performs motion compensation prediction on each pixel according to the calculated motion vector of each pixel. More specifically, motion compensation prediction is performed by generating an interpolation signal from the reference image pixel indicated by the motion vector of each pixel. It should be noted that the accuracy of the motion vector of each pixel, the rounding of numerical values required in the calculation process, and the like need to be defined so that the same value is obtained regardless of which decoding device is used. In addition, when the coordinate of the prediction pixel designated by the motion vector is expressed by a decimal point, the pixel is interpolated from surrounding pixels at the time of motion compensation. As the interpolation method, 4 to 6 tap filtering, linear interpolation, or the like can be used.

  In the first mode that encodes and decodes the motion vectors of the four representative pixels a, b, c, and d, it cannot be expressed in the second mode or the third mode, such as enlargement / reduction, rotation, translation, etc., with the four motion vectors. Deformation can be represented.

  In the second mode in which the motion vectors of the two representative pixels e and f in the vertical direction of the target block are encoded and decoded, the two motion vectors can represent uniform deformation in the vertical direction in addition to translation. Although the expressive power of deformation is limited compared to the first mode, the number of motion vectors to be encoded can be reduced. In the third mode in which the motion vectors of the two representative pixels g and h in the horizontal direction of the target block are encoded and decoded, the two motion vectors can represent uniform deformation in the horizontal direction in addition to translation. Although the expressive power of deformation is limited compared to the first mode, the number of motion vectors to be encoded can be reduced.

  Returning to FIG. 1, the prediction method determination unit 104 employs either the prediction method by the translational motion compensation prediction unit 102 or the prediction method by the geometric transformation motion compensation prediction unit 103 for each target block in the target image. To decide. More specifically, it is determined which mode of which prediction method is adopted. Hereinafter, prediction method selection including mode selection is referred to.

  That is, the prediction method determining unit 104 selects which reference image is used to encode motion compensation prediction based on parallel movement or motion compensated prediction based on geometric transformation in which pixel block unit. Decide how. In the case of motion compensation prediction by geometric transformation, one of the first mode, the second mode, and the third mode is selected. At that time, the prediction method is determined by determining which combination of these items is the prediction method capable of realizing the most efficient encoding. As a criterion for determining the prediction method, for example, rate distortion theory considering the relationship between the code amount and distortion can be used. More specifically, the code amount of the macroblock (that is, the prediction method information, the motion vector, and the total code amount of the prediction signal) is calculated, and the distortion amount is calculated from the difference between the encoding target image and the decoded image. A prediction method is selected that minimizes a code amount and a rate distortion function using the distortion amount as an input variable. Note that the prediction method is described in a macroblock type. In the case of translation motion compensation, it is described in the macroblock type as to which of the 16 × 16/16 × 8/8 × 16/8 × 8 mode is used, and in the case of geometric transformation motion compensation, the first mode, the first mode The macroblock type is described as to whether the second mode or the third mode is used. The first mode of geometric transformation motion compensated prediction is an 8 × 8 macroblock type that encodes four motion vectors, and the second mode is a 16 × 8 macroblock that encodes two motion vectors. In type, the third mode is described as an 8 × 16 macroblock type that also encodes two motion vectors.

  The prediction method determination unit 104 sends the prediction method information described in the adopted macroblock type and the motion vector generated by the prediction method to the first encoded bit sequence generation unit 107 and the second encoded bit sequence generation unit 108. The motion vector corresponding to the employed prediction method is supplied to the second encoded bit string generation unit 108. At the same time, the prediction method determination unit 104 supplies the prediction signal generated by the employed prediction method to the prediction error signal generation unit 105.

  The prediction error signal generation unit 105 calculates a difference between the prediction signal generated by the prediction method adopted by the prediction method determination unit 104 and the image signal of the target block, and generates a prediction error signal. More specifically, the prediction error signal generation unit 105 subtracts the prediction signal supplied from the prediction method determination unit 104 from the image signal to be encoded supplied from the image buffer 101 to obtain a prediction error signal. Generated and supplied to the prediction error signal encoding unit 106.

  The encoding unit 150 includes prediction method information for specifying the prediction method adopted by the prediction method determination unit 104, a motion vector generated by the prediction method, and a prediction error signal generated by the prediction error signal generation unit 105. Is encoded. When encoding the motion vector of the target block, the second encoded bit string generation unit 108 of the encoding unit 150 performs predictive encoding with reference to the motion vector of the adjacent block. At that time, regardless of whether the motion compensation prediction method of the adjacent block is translation motion compensation prediction or geometric conversion motion compensation prediction, both are handled in common and the motion vector of the adjacent block to be referenced is determined. To do. Therefore, the reference relationship between the motion vector of the target block and the motion vector of the adjacent block is the difference between the motion vector of the block that uses the parallel motion compensation prediction method and the motion vector that uses the geometric transformation motion compensation prediction. But it holds. That is, in the motion vector predictive coding, the relationship is a mixture of both.

  Here, in a target block in which motion compensation prediction by geometric transformation is adopted, a motion vector of the representative pixel is a target of predictive encoding. For example, when predictively coding a motion vector of a target block adopting motion compensation prediction based on parallel movement with reference to a motion vector of an adjacent block adopting motion compensated prediction based on geometric transformation, a representative of the adjacent block Refer to the motion vector of the pixel.

  Hereinafter, the processing of the encoding unit 150 will be described in detail. The prediction error signal encoding unit 106 performs compression encoding processing such as orthogonal transform and quantization on the prediction error signal supplied from the prediction error signal generation unit 105 to generate an encoded prediction error signal. The third encoded bit string generator 109 and the prediction error signal decoder 110 are supplied.

  The first encoded bit string generation unit 107 encodes the prediction method information described by the macroblock type supplied from the prediction method determination unit 104 by entropy encoding such as arithmetic encoding, and generates an encoded bit string. . In addition, the prediction block size to be included in the prediction method information, L0 / L1 prediction distinction, and the like are combined and encoded as a macroblock type. The following description method can be used as information for identifying which of motion compensation prediction by parallel movement and motion compensation prediction by geometric transformation to be included in the prediction method information. For example, the macroblock type encoding method may be shared, and syntax elements for determining motion compensation prediction by parallel movement and motion compensation prediction by geometric transformation may be separately prepared and described. The macroblock type may be extended and described by combining with information to be encoded as the macroblock type.

  The second encoded bit string generation unit 108 encodes the motion vector supplied from the prediction method determination unit 104 to generate an encoded bit string. At that time, the motion vector is predictively encoded. When the motion vector of the target block is encoded, the prediction vector is calculated by predicting the motion vector from the adjacent block using the correlation with the motion vector of the neighboring block already encoded or decoded. Note that the correlation with the motion vectors of neighboring neighboring blocks can be specified based on the prediction method information supplied from the prediction method determination unit 104. Then, a difference vector that is a difference between the prediction vector and the motion vector of the target block or the target pixel is calculated, and the difference vector is encoded, thereby reducing the amount of coding of the motion vector. Here, in a target block in which motion compensation prediction by geometric transformation is adopted, a motion vector of the representative pixel is a target for encoding.

  FIG. 4 is a diagram for explaining a motion vector prediction method. FIG. 4A shows an example of predicting a motion vector between macroblocks in which no partition is set. FIG. 4B shows an example in which motion vectors are predicted between macroblocks in which partitions are set. FIG. 4C shows an example in which a motion vector is predicted in a 16 × 8 pixel macroblock. FIG. 4D shows an example in which a motion vector is predicted in a macroblock of 8 × 16 pixels. Hereinafter, referring to FIGS. 4A to 4D, AVC / H. A motion vector prediction method employed in the H.264 system will be described. In this motion vector prediction method, a motion vector of a target block is predicted using a median value of motion vectors of neighboring neighboring blocks.

  4A to 4D, the motion vector of a block painted in gray is an encoding target. In FIG. 4A, the motion vector of the target block is predicted using three motion vectors of the block A adjacent to the left of the target block, the block B adjacent to the upper right, and the block C adjacent to the upper right. More specifically, a median value is taken for each of the horizontal component and the vertical component from these three motion vectors to obtain a prediction vector. In the B picture, a motion vector for L0 prediction and a motion vector for L1 prediction are handled separately. Prediction for the L0 prediction of the target block by performing prediction using three motion vectors for the L0 prediction of the block A on the left of the target block, the block B on the upper right, and the block C on the upper right. Calculate the vector. Similarly, by using the motion vector for the L1 prediction of the block A adjacent to the left of the target block, the block B adjacent to the upper right, and the block C adjacent to the upper right, the prediction of the L1 prediction of the target block is performed. To calculate a prediction vector. Here, when the block B on the upper right and the block C on the upper right side cannot be used together and only the block A can be used, the motion vector of the block A is adopted as the prediction vector. If only one of the reference indexes of the block A on the left, the block B on the upper right, and the block C on the upper right is equal to the reference index of the current block (the reference pictures are equal), that block Are used for prediction.

  As shown in FIG. 4B, when a partition is set in the macroblock, the motion vector is different for each small block of the macroblock. In that case, in the block adjacent to the left of the target block, the motion vector of the uppermost small block A among the small blocks in contact with the target block is adopted. In the upper adjacent block, the leftmost block B among the small blocks in contact with the target block is adopted. In the block adjacent to the upper right, the lower left small block C is adopted.

  As shown in FIG. 4C, when the block to be encoded is 8 × 16, the left block is not the median value of the motion vectors of the three blocks, the left block is the block A on the left, and the right block is on the upper right side. The motion vector of the adjacent block C is adopted as the prediction vector. Also, as shown in FIG. 4D, when the block to be encoded is 16 × 8, the upper block is not the median of the motion vectors of the three blocks, the upper block is the upper adjacent block B, and the lower block is The motion vector of the block A on the left is adopted as the prediction vector.

  Note that the motion vector prediction method shown in FIG. 4 is an example and is not limited thereto. As long as the encoding method and the decoding side prescribe the motion vector prediction method identically, other methods can be used. For example, the position and number of adjacent blocks may be different. Further, an average value may be used instead of the median value of a plurality of motion vectors of adjacent blocks. A predetermined condition or priority order may be provided, and a motion vector of a single adjacent block may be used as it is. Moreover, the adjacent block does not necessarily have to contact the target block. In addition, although an example of predicting a motion vector in units of macroblocks has been described with reference to FIG. 4, the same processing can be performed when a motion vector in units of sub-macroblocks is predicted.

In the motion vector prediction method described with reference to FIG. 4, the motion vector value Vabcd ′ that has been scaled may be used for calculation of the prediction vector, instead of using the motion vector of the adjacent pixel as a candidate as it is. The scaled motion vector value Vabcd ′ includes the distance (time) T1 between the encoding target image and the reference image indicated by the motion vector of the encoding target block, and the motion vector of the encoding target image and the adjacent block or adjacent pixel. The motion vector value is scaled according to the difference from the distance (time) T2 from the reference image indicated by Vabcd. The scaled motion vector value Vabcd ′ is calculated by the following equation (11).
Vabcd ′ = Vabcd * (T1 / T2) (11)

  If the reference image referenced in the motion-compensated prediction of the current block is different from the reference image referenced in the motion-compensated prediction of the adjacent block, there will be a difference in the size of each other's motion vector. Scale the motion vector. For example, if the object is not deformed and moves at a constant speed, the magnitude of the motion vector increases as the frame interval increases. Vabcd 'is calculated by scaling the motion vector Vabcd of the adjacent block (adjacent pixel) according to the ratio of the frame intervals T1 and T2 between the encoding target image and the reference image.

  FIG. 5 is a diagram for explaining an example of motion vector value scaling processing. FIG. 5 illustrates an example in which the reference image of the encoding target block of the encoding target image is the image Ref2, and the reference image for motion compensation prediction of the adjacent block (adjacent pixel) is the image Ref3. In FIG. 5, since T1: T2 = 2: 3, the motion vector of the adjacent block (adjacent pixel) referring to the image Ref3 is scaled to 2/3. As a result, the adjacent block (adjacent pixel) is close to the value of the motion vector when motion compensation prediction is performed with reference to the image Ref2, and as a result, the value of the motion vector of the block to be encoded that refers to the image Ref2 is obtained. Get closer. In the example of FIG. 5, when the participating image in the motion compensation prediction of the adjacent block is the image Ref3 and the value of the motion vector is (24, −9), it is scaled to 2/3 (16, 6). The motion vector value of the image Ref2 referred to by the encoding target block may be used.

  Returning to FIG. 1, the second encoded bit string generation unit 108 encodes the difference vector obtained by calculating the difference between the prediction vector and the motion vector of the target block by entropy encoding such as arithmetic encoding. At this time, a prediction vector is calculated by mixing a motion vector used in motion compensation prediction by parallel movement and a motion vector used in motion compensation prediction by geometric transformation. Therefore, a motion vector used in motion compensated prediction based on parallel movement is predicted and encoded from a motion vector used in motion compensated prediction based on geometric transformation. In some cases, a motion vector used in motion compensation prediction by conversion is predicted and encoded. By calculating a prediction vector by mixing a motion vector used in motion compensated prediction by translation and a motion vector used in motion compensated prediction by geometric transformation, there is an effect of improving the encoding efficiency of the difference vector.

  As described above, the motion vector to be encoded / decoded in motion compensated prediction by geometric transformation is set to the motion vector of the pixel located at the center of each macroblock partition or sub-macroblock partition. . The motion vector of the pixel located at each center can be regarded as an average value of the motion vectors of all the pixels included in each partition, and has a strong correlation with the motion vector used in the motion compensation prediction by the parallel movement. Therefore, the difference vector between the motion vector used in the motion compensation prediction based on the geometric transformation and the motion vector used in the motion compensation prediction based on the parallel movement is likely to be a small value.

  The third encoded bit sequence generation unit 109 sequentially encodes the encoded prediction error signal supplied from the prediction error signal encoding unit 106 using entropy encoding such as arithmetic encoding to generate an encoded bit sequence To do.

  The encoded bit strings generated by the first encoded bit string generation unit 107, the second encoded bit string generation unit 108, and the third encoded bit string generation unit 109 are encoded by information other than prediction method information, motion vectors, and prediction error signals. The other encoded bit strings are multiplexed via the output switch 113 to generate an encoded stream.

  Further, the prediction error signal decoding unit 110 performs decompression decoding processing such as inverse quantization and inverse orthogonal transform on the prediction error signal encoded by the prediction error signal encoding unit 106, and decodes the prediction error signal. To do. The prediction error signal decoding unit 110 supplies the decoded prediction error signal to the decoded image signal generation unit 111. The decoded image signal generation unit 111 superimposes the prediction error signal supplied from the prediction error signal encoding unit 106 and the prediction signal supplied from the prediction method determination unit 104 to generate a decoded image signal. The decoded image signal generation unit 111 sequentially stores the decoded image signal in the decoded image buffer 112 in units of pixel blocks. The decoded image stored in the decoded image buffer 112 is used as a reference image when performing motion compensation prediction on the subsequent image in the encoding order, if necessary.

  FIG. 6 is a flowchart showing a macroblock encoding processing procedure in image encoding apparatus 100 according to Embodiment 1 of the present invention. First, the translational motion compensation prediction unit 102 and the geometric transformation motion compensation prediction unit 103 extract a macroblock signal from the image buffer 101 (S101).

  The translational motion compensation prediction unit 102 performs motion compensation prediction by translation between the macroblock signal supplied from the image buffer 101 and the reference image signal supplied from the decoded image buffer 112 (S102). The motion compensation prediction based on the parallel movement is performed for each mode. The geometric transformation motion compensation prediction unit 103 performs motion compensation prediction by geometric transformation between the macroblock signal supplied from the image buffer 101 and the reference image signal supplied from the decoded image buffer 112 (S103). . Motion compensation prediction by this geometric transformation is performed for each mode.

  The prediction method determination unit 104 determines which one of the motion compensation prediction method based on the parallel movement and the motion compensation prediction based on the geometric transformation is adopted (S104). At that time, it is also decided which mode is adopted. The prediction error signal generation unit 105 subtracts the prediction signal supplied from the prediction method determination unit 104 from the target image signal supplied from the image buffer 101 to generate a prediction error signal (S105). The prediction error signal encoding unit 106 encodes the prediction error signal (S106).

  The first encoded bit string generation unit 107 encodes the prediction method information supplied from the prediction method determination unit 104, and generates an encoded bit string (S107). The second encoded bit string generation unit 108 encodes the motion vector supplied from the prediction method determination unit 104 and generates an encoded bit string (S108). The third encoded bit string generation unit 109 entropy-encodes the prediction error signal encoded by the prediction error signal encoding unit 106 using arithmetic encoding or the like to generate an encoded bit string (S109).

  The prediction error signal decoding unit 110 decodes the prediction error signal encoded by the prediction error signal encoding unit 106 (S110). The decoded image signal generation unit 111 generates a decoded image signal by superimposing the prediction error signal decoded by the prediction error signal decoding unit 110 and the prediction signal supplied from the prediction method determination unit 104 (S111). The decoded image signal generation unit 111 accumulates the generated decoded image signal in the decoded image buffer 112 (S112).

  As described above, according to the first embodiment, it is possible to improve the compression efficiency of the code amount by the image coding method using the motion compensated prediction based on the geometric transformation. That is, the amount of codes can be reduced by predictively encoding motion vectors that use motion compensated prediction by geometric transformation. Also, if the motion compensation prediction based on the parallel movement and the motion compensation prediction based on the geometric transformation are used in combination, the compression efficiency of the code amount can be further improved. At that time, by sharing a motion vector related to motion compensation prediction based on parallel movement and a motion vector encoding method related to motion compensation prediction based on geometric transformation, even if these two prediction methods are mixed, The motion vector predictive coding method can be used as it is.

  In addition, in a pixel block in which a motion compensation prediction method based on geometric transformation is adopted, the motion vector of the pixel at or near the center of the partition into which the pixel block is divided is treated as the motion vector of the pixel block. The motion vector of the pixel located at each center can be regarded as an average value of the motion vectors of all the pixels included in each partition. Therefore, the correlation with the motion vector of the pixel block adjacent to the pixel block and adopting the motion compensation prediction method by parallel movement is also high. Therefore, even if the motion compensation prediction based on the parallel movement and the motion compensation prediction based on the geometric transformation are used together, an increase in the code amount of the motion vector can be suppressed.

  In addition, motion compensated prediction by geometric transformation according to the present embodiment can be applied to 8 × 16 mode and 16 × in the 8 × 8 mode macroblock type (in this type, four motion vectors are generated). An 8-mode macroblock type (in these types two motion vectors are generated) can also be supported. Similarly, 4 × 4 mode sub-macroblock types (in which four motion vectors are generated) are also applied to 4 × 8 mode and 8 × 4 mode sub-macroblock types (in these types). , Two motion vectors are generated). Thus, AVC / H. It corresponds to all macroblock partitions and sub-macroblock partitions adopted in H.264, etc., and is highly harmonious and compatible with existing coding schemes. Therefore, the introduction cost can be suppressed.

  FIG. 7 is a block diagram showing a configuration of image decoding apparatus 200 according to Embodiment 2 of the present invention. The image decoding apparatus 200 decodes the encoded stream generated by the image encoding apparatus 100 according to Embodiment 1. In the encoded stream, as described above, motion compensated prediction based on parallel movement and motion compensated prediction based on geometric transformation may be used together, or motion compensated prediction based on geometric transformation is used alone. There is also. Note that intra coding is not considered.

  The image decoding apparatus 200 includes an input switch 209, a decoding unit 250, a translational motion compensation prediction unit 204, a geometric transformation motion compensation prediction unit 205, a decoded image signal generation unit 207, a decoded image buffer 208, a switching control unit 214, A prediction unit switch 210, a second prediction unit switch 211, a third prediction unit switch 212, and a fourth prediction unit switch 213 are provided. The decoding unit 250 includes a first encoded bit string decoding unit 201, a second encoded bit string decoding unit 202, a third encoded bit string decoding unit 203, and a prediction error signal decoding unit 206.

  These configurations can be realized by an arbitrary processor, memory, or other LSI in terms of hardware, and can be realized by a program loaded in the memory in terms of software, but here by their cooperation. Draw functional blocks. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The decoding unit 250 decodes prediction method information, motion vectors, and prediction error signals included in the encoded stream. As described above, the difference vector is encoded in the encoded stream. The difference vector is a difference between the motion vector of the target block and a prediction vector predicted from the motion vector of the adjacent block or the adjacent pixel. When decoding the motion vector, the decoding unit 250 calculates a prediction vector from the motion vector of the adjacent block or the adjacent pixel, and adds the decoded difference vector to the prediction vector, thereby performing prediction encoding. The motion vector of the target block is decoded. As described above, the motion vector of the target block is predictively encoded with reference to the motion vector of the adjacent block regardless of the motion compensation prediction method of the adjacent block. When decoding the motion vector, the decoding unit 250 decodes the motion vector of the target block subjected to predictive encoding with reference to the motion vector of the adjacent block regardless of the motion compensation prediction method of the adjacent block. In the case of geometric conversion motion compensation, the motion vector of the target block becomes the motion vector of the representative pixel.

  The translational motion compensation prediction unit 204 obtains a prediction signal from the motion vector between the target block in the target image and the reference block in the reference image that is translated with the target block, and the image signal of the reference block. Generate. The geometric transformation motion compensation prediction unit 205 performs prediction based on the motion vector between the target block in the target image, the reference block in the reference image having a geometric transformation relationship with the target block, and the image signal of the reference block. Generate a signal.

  As described above, the target block is divided into a plurality of small blocks, and a representative pixel is selected for each small block. The representative pixel is selected as a pixel at the approximate center or the approximate center of gravity of the small block, and the encoded stream includes a motion vector of each representative pixel. The geometric transformation motion compensation prediction unit 205 calculates a motion vector of pixels other than the plurality of representative pixels of the target block by at least one of interpolation and extrapolation using the motion vectors of the plurality of representative pixels. For example, the motion vectors of the pixels other than the representative pixel are calculated by the arithmetic expressions shown in the above expressions (1) to (10).

  The switching control unit 214 refers to the prediction method information decoded by the decoding unit 250, and for each target block in the target image, the prediction method by the translational motion compensation prediction unit 204 and the geometric transformation motion compensation prediction unit 205. Specify which of the prediction methods is used.

  At that time, the switching control unit 214 also designates which mode is designated. For example, in the prediction method information for designating motion compensation prediction by the geometric transformation motion compensation prediction unit 205, the target block is divided into four equal parts in a cross and divided into four square small blocks, and four motion vectors are detected. The first mode, the target block is horizontally divided into two vertically divided small rectangular blocks, and two motion vectors are detected. The target block is vertically divided into two equal parts. Any one of the third modes in which two motion vectors are detected after being divided into two horizontally long rectangular small blocks is designated.

  More specific description will be given below. The first encoded bit string decoding unit 201, the second encoded bit string decoding unit 202, and the third encoded bit string decoding unit 203 are included in the encoded stream generated by the image encoding device 100 according to Embodiment 1. The encoded bit string to be transmitted is selectively input via the input switch 209.

  The first encoded bit string decoding unit 201 decodes the encoded bit string supplied via the input switch 209 by entropy decoding such as arithmetic decoding, and acquires prediction method information. As described above, the prediction method information includes whether the encoding is performed by translation / geometric transformation, and in the case of translation motion compensation, 16 × 16/16 × 8/8 × 16. Information on which of the / 8 × 8 mode is used is included. In the case of geometric transformation, a first mode for encoding / decoding the motion vectors of the four representative pixels a, b, c, d, and encoding the motion vectors of the two representative pixels e, f in the vertical direction of the target block The information includes information on the second mode for decoding and the third mode for encoding / decoding the motion vectors of the two representative pixels g and h in the horizontal direction of the target block. The second encoded bit string decoding unit 202 decodes the encoded bit string supplied via the input switch 209 by entropy decoding such as arithmetic decoding to obtain a motion vector. As described above, when encoding the motion vector, the motion vector is described by a difference vector between the prediction vector calculated from the motion vector of the adjacent block and the motion vector of the decoding target block. Specifically, the second encoded bit string decoding unit 202 decodes the encoded bit string supplied via the input switch 209 by entropy decoding such as arithmetic decoding, and obtains a difference vector. Further, the second encoded bit string decoding unit 202 performs motion vectors of neighboring blocks or adjacent pixels that have already been encoded / decoded according to the prediction method information supplied from the first encoded bit string decoding unit 201. The prediction vector is calculated by predicting the motion vector of the target block. The calculation of the prediction vector in the second encoded bit string decoding unit 202 is performed by the same method as the calculation of the prediction vector in the second encoded bit string generation unit 108 of the image encoding device 100. Then, the motion vector is obtained by adding the difference vector decoded from the encoded bit string to the prediction vector. Note that the encoding side calculates the prediction vector by mixing the motion vector used in the motion compensation prediction by translation and the motion vector used in the motion compensation prediction by geometric transformation. A motion vector used in motion compensated prediction and a motion vector used in motion compensated prediction based on geometric transformation are mixed to calculate a predictive vector, and the motion vector is obtained by adding the decoded difference vector to the predicted vector. In the case of parallel motion compensation compensation, the motion vector of the block can be acquired, and in the case of geometric transformation motion compensation prediction, the motion vector of the representative pixel of the block can be obtained.

  Based on the prediction method information decoded by the first encoded bit string decoding unit 201, which pixel block to be decoded is intra-coded, motion-compensated prediction by parallel movement, and motion-compensated prediction by geometric transformation, and which reference image is used. By using this, it can be seen what pixel block unit is selected and combined.

  The switching control unit 214 performs the first prediction unit switch 210, the second prediction unit switch 211, the third prediction unit switch 212, and the fourth prediction unit according to the prediction method information supplied from the first encoded bit string decoding unit 201. Switch 213 is switched. When the motion compensation prediction method by translation is selected as the prediction method of the target block, switching is performed so that the path of the translation motion compensation prediction unit 204 is selected, and the motion compensation prediction method by geometric transformation is selected. In this case, the path of the geometric transformation motion compensation prediction unit 205 is switched to be selected.

  The translation motion compensated prediction unit 204 includes a decoded image to be a reference image supplied from the decoded image buffer 208 via the fourth prediction unit switch 213, and a second encoded bit string decoding unit 202 to the second prediction unit switch 211. Using the decoded motion vector supplied via, motion compensated prediction by translation is performed.

  The geometric transformation motion compensated prediction unit 205 includes a decoded image to be a reference image supplied from the decoded image buffer 208 via the fourth prediction unit switch 213, and a second prediction bit switch from the second encoded bit string decoding unit 202. Using the decoded motion vectors of a plurality of representative pixels supplied via 211, the motion vectors of all the pixels other than the representative pixels are interpolated or extrapolated using the above equations (1) to (4). And motion compensation prediction by geometric transformation is performed by performing motion compensation for each pixel according to the motion vector for each pixel.

  The third encoded bit string decoding unit 203 sequentially decodes the encoded bit string supplied via the input switch 209, and obtains an encoded prediction error signal. The prediction error signal decoding unit 206 performs decompression decoding processing such as inverse quantization and inverse orthogonal transform on the encoded prediction error signal supplied from the third encoded bit string decoding unit 203, and performs the decoded prediction Get the error signal.

  The decoded image signal generation unit 207 generates an image signal from the prediction signal and the prediction error signal. More specifically, the decoded image signal generation unit 207 switches from the translation motion compensation prediction unit 204 or the geometric transformation motion compensation prediction unit 205 to the third prediction unit switch according to the prediction method specified by the switching control unit 214. The prediction error signal supplied from the prediction error signal decoding unit 206 is superimposed on the prediction signal supplied via 212 to generate a decoded image signal. The decoded image signal generation unit 207 sequentially stores the decoded image signal in the decoded image buffer 208 in units of pixel blocks.

  FIG. 8 is a flowchart showing a macroblock decoding process procedure in image decoding apparatus 200 according to Embodiment 2 of the present invention. The first encoded bit string decoding unit 201 decodes the encoded bit string supplied via the input switch 209 and acquires prediction method information (S201). The second encoded bit string decoding unit 202 decodes the encoded bit string supplied via the input switch 209 and obtains a difference vector (S202). The second encoded bit string decoding unit 202 calculates a prediction vector from the neighboring blocks or neighboring pixels (S202), adds the difference vector supplied from the prediction method determining unit 104 to the prediction vector, and obtains the target block or target pixel. Is calculated (S202).

  The switching control unit 214 refers to the decoded prediction method information and identifies the motion compensation prediction method of the target block (S203). When the prediction method is a motion compensation prediction method based on translation (translation in S203), the translation motion compensation prediction unit 204 determines a decoded image signal to be a reference image signal supplied from the decoded image buffer 208 as a reference image signal. The motion vector supplied from the 2-encoded bit string decoding unit 202 is used to perform motion compensation by translation and generate a prediction signal (S204).

  When the prediction method specified by the switching control unit 214 is a motion compensation prediction method based on geometric transformation (geometric transformation in S203), the geometric transformation motion compensation prediction unit 205 uses the reference image supplied from the decoded image buffer 208. The decoded image signal to be a signal is motion-compensated by geometric transformation using the motion vector supplied from the second encoded bit string decoding unit 202 to generate a prediction signal (S205).

  The third encoded bit string decoding unit 203 decodes the encoded bit string supplied via the input switch 209 and obtains an encoded prediction error signal (S206). The decoded image signal generation unit 207 decodes the acquired prediction error signal (S207). The decoded image signal generation unit 207 superimposes the prediction error signal decoded by the decoded image signal generation unit 207 and the prediction signal generated by the translational motion compensation prediction unit 204 or the geometric transformation motion compensation prediction unit 205. Then, a decoded image signal is generated (S208). The decoded image signal generation unit 207 accumulates the generated decoded image signal in the decoded image buffer 208 (S209). The decoded image signal stored in the decoded image buffer 208 is used as a reference image signal in the translational motion compensation prediction unit 204 and the geometric transformation motion compensation prediction unit 205.

  As described above, according to the second embodiment, the encoded stream generated by the image encoding apparatus 100 according to the first embodiment can be efficiently decoded. Thereby, the effect implement | achieved by the image coding apparatus 100 which concerns on Embodiment 1 mentioned above is supported from the decoding side, and the effect can be ensured. That is, in the image coding method using motion compensation prediction by geometric transformation, the effect of improving the compression efficiency of the code amount can be supported from the decoding side, and the effect can be ensured. Also, in the image coding method that uses motion compensation prediction by parallel movement and motion compensation prediction by geometric transformation, the effect of improving the compression efficiency of the code amount can be supported from the decoding side, and the effect can be guaranteed. it can. Moreover, the harmony and compatibility with the existing image decoding apparatus are high, and the introduction cost can be suppressed.

  The present invention has been described based on some embodiments. It is understood by those skilled in the art that these embodiments are exemplifications, and that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. By the way.

  In the above-described embodiment, the example in which the shape of the target block is a square has been described. In this regard, the shape of the target block may be other shapes such as a triangle, a parallelogram, and a trapezoid. In the above-described embodiment, the shape of the small block (that is, the macroblock partition and the sub-macroblock partition) is not limited to a square and a rectangle, but other shapes such as a triangle, a parallelogram, and a trapezoid. But you can. In this case, it is preferable that the representative pixel is set at or near the center of gravity of the shape.

  In the embodiment described above, the approximate center and approximately center of gravity pixel of the small block (that is, the macroblock partition and the sub macroblock partition) are used as the representative pixels. However, the present invention is not limited to this and corresponds to the center or center of gravity. The interpolation pixel located at the coordinates may be used as the representative pixel, or the interpolation pixel corresponding to the vertex of the macroblock or its surrounding pixels may be used as the representative pixel.

  DESCRIPTION OF SYMBOLS 100 Image coding apparatus, 101 Image buffer, 102 Translational motion compensation prediction part, 103 Geometric transformation motion compensation prediction part, 104 Prediction method determination part, 105 Prediction error signal generation part, 106 Prediction error signal coding part, 107 1 encoded bit string generation unit, 108 second encoded bit string generation unit, 109 third encoded bit string generation unit, 110 prediction error signal decoding unit, 111 decoded image signal generation unit, 112 decoded image buffer, 113 output switch, 150 code , 200 image decoding device, 201 first encoded bit string decoding unit, 202 second encoded bit string decoding unit, 203 third encoded bit string decoding unit, 204 translation motion compensation prediction unit, 205 geometric conversion motion compensation prediction , 206 prediction error signal decoding unit, 207 decoded image No. generation unit, 208 decoded image buffer 209 input switch, 210 a first prediction unit switch, 211 the second prediction unit switch, 212 the third prediction unit switch, 213 the fourth prediction section switch, 214 switch controller, 250 decoding unit.

Claims (6)

A decoding unit that decodes prediction method information, a motion vector, and a prediction error signal included in an encoded stream that is encoded by jointly using motion compensated prediction by parallel movement and motion compensated prediction by geometric transformation;
A translation motion compensated prediction unit that generates a prediction signal from a motion vector between a target block in the target image and a reference block in a reference image that has a translational relationship with the target block, and an image signal of the reference block When,
A motion vector between a target block in the target image and a reference block in a reference image that is geometrically transformed with the target block, and a geometric conversion motion compensation that generates a prediction signal from the image signal of the reference block A predictor;
With reference to the prediction method information decoded by the decoding unit, for each target block in the target image, either the prediction method by the translation motion compensation prediction unit or the prediction method by the geometric transformation motion compensation prediction unit A control unit for specifying whether to use,
A prediction signal generated by a prediction method specified by the control unit, and an image signal generation unit that generates an image signal from the prediction error signal decoded by the decoding unit,
The motion vector of the target block is a prediction code by referring to the motion vector of the adjacent block regardless of whether the motion compensation prediction method of the adjacent block is translation motion compensation prediction or geometric conversion motion compensation prediction. Has been
The said decoding part decodes the motion vector of the object block by which the prediction encoding was carried out with reference to the motion vector of the said adjacent block irrespective of the motion compensation prediction method of an adjacent block. In the target block, a plurality of representative pixels are selected, and a motion vector of each representative pixel is predictively encoded and included in the encoded stream.
The geometric transformation motion compensation prediction unit is configured to at least one of interpolation and extrapolation using a motion vector of a pixel other than the plurality of representative pixels of the target block using a decoded motion vector of the plurality of representative pixels. The image decoding device according to claim 1, wherein the image decoding device is calculated by:   3. The image decoding apparatus according to claim 2, wherein the representative pixel is selected as a pixel corresponding to a substantial center or a substantial center of gravity of the small block when the target block is divided into small blocks. The target block is a square area;
In the prediction method information for designating motion compensated prediction by the geometric transformation motion compensated prediction unit, a motion vector is detected for each of the four representative pixels of four square small blocks obtained by dividing the target block into four equal crosses. A second mode in which a motion vector is detected for each of the two representative pixels of two vertically long rectangular small blocks obtained by equally dividing the target block into two horizontally, and the target block is vertically 4. The third mode in which a motion vector is detected for each of the two representative pixels of two horizontally-divided small rectangular blocks divided into two equal parts is specified. 5. Image decoding device. A decoding step of decoding prediction method information, a motion vector, and a prediction error signal included in an encoded stream that is encoded by jointly using motion compensated prediction by parallel movement and motion compensated prediction by geometric transformation;
A translation motion compensation prediction step for generating a prediction signal from a motion vector between a target block in the target image and a reference block in a reference image in a translational relationship with the target block, and an image signal of the reference block When,
A motion vector between a target block in the target image and a reference block in a reference image that is geometrically transformed with the target block, and a geometric conversion motion compensation that generates a prediction signal from the image signal of the reference block A prediction step;
With reference to the prediction method information decoded in the decoding step, it is specified whether to use the prediction method based on the translation motion compensation or the prediction method based on the geometric transformation motion compensation for each target block in the target image. A designated step to
A prediction signal generated by the prediction method specified by the specifying step, and an image signal generation step of generating an image signal from the prediction error signal decoded by the decoding step,
The motion vector of the target block is a prediction code by referring to the motion vector of the adjacent block regardless of whether the motion compensation prediction method of the adjacent block is translation motion compensation prediction or geometric conversion motion compensation prediction. Has been
The image decoding method characterized in that the decoding step decodes the motion vector of the target block subjected to predictive coding with reference to the motion vector of the adjacent block regardless of the motion compensation prediction method of the adjacent block. A decoding process for decoding prediction method information, a motion vector, and a prediction error signal included in an encoded stream encoded by jointly using motion compensated prediction by parallel movement and motion compensated prediction by geometric transformation;
Translation motion compensation prediction processing for generating a prediction signal from a motion vector between a target block in a target image and a reference block in a reference image in a translational relationship with the target block, and an image signal of the reference block When,
A motion vector between a target block in the target image and a reference block in a reference image that is geometrically transformed with the target block, and a geometric conversion motion compensation that generates a prediction signal from the image signal of the reference block The prediction process,
With reference to the prediction method information decoded by the decoding process, it is specified whether to use the prediction method based on the translational motion compensation or the prediction method based on the geometric transformation motion compensation for each target block in the target image. Specified processing to
Causing the computer to execute a prediction signal generated by the prediction method specified by the specifying process and an image signal generation process for generating an image signal from the prediction error signal decoded by the decoding process;
The motion vector of the target block is a prediction code by referring to the motion vector of the adjacent block regardless of whether the motion compensation prediction method of the adjacent block is translation motion compensation prediction or geometric conversion motion compensation prediction. Has been
The image decoding program characterized in that the decoding processing decodes a motion vector of a target block subjected to predictive encoding with reference to a motion vector of the adjacent block regardless of a motion compensation prediction method of the adjacent block.

Images