JP5716438B2 - Image decoding apparatus, image decoding method, and image decoding program - Google Patents

Description

  The present invention relates to an image decoding apparatus, an image decoding method, and an image decoding program that decode an image using motion compensation prediction.

  As a typical moving image compression coding method, there is a standard of the MPEG series. In the MPEG series standard, motion compensation that divides a frame into a plurality of blocks and predicts motion from other frames is used. MPEG-4 and AVC / H. In H.264, a mechanism for switching and using an optimum one from a plurality of motion compensation block sizes is introduced.

  In motion compensation prediction in units of blocks, a method of compensating for parallel movement between a target block and a reference block is common. In addition to this, a scheme for compensating for block deformation (for example, enlargement, reduction, rotation) has been studied. For example, in Patent Document 1, as an image encoding method using prediction between frames, a mode in which a predicted image is obtained by parallel movement and a mode in which a predicted image is obtained by geometric transformation are adaptively switched for each block, thereby predicting efficiency. We are trying to improve. In this method, a translational motion vector and a lattice point motion vector (that is, a motion vector used in geometric transformation) are encoded.

JP-A-8-65680

  Under such circumstances, the present inventor has found a method of further compressing the entire code amount by compressing motion vector information by an image encoding method using motion compensated prediction by geometric transformation.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technique for improving the compression efficiency of a code amount in an image coding method using motion compensated prediction by geometric transformation.

One embodiment of the present invention is an image decoding device. This apparatus includes prediction method information for specifying a prediction mode, a differential motion vector of a representative pixel corresponding to the prediction mode, and a prediction included in an encoded stream that is encoded using motion compensated prediction by geometric transformation. In accordance with the decoding unit for decoding the error signal and the prediction mode specified by the prediction method information, the predicted motion vector of the motion vector of the representative pixel is calculated using the motion vectors inside and outside the target block, and the predicted motion vector is used as the predicted motion vector. A motion vector generation unit for generating a motion vector of the representative pixel by adding the difference motion vector of the representative pixel, a target block in the target image, and a reference in the reference image that is geometrically transformed with the target block Motion other than the representative pixel calculated by interpolation using the motion vector of the representative pixel between the block and the motion vector of the representative pixel Geometric transformation motion compensated prediction unit for generating a prediction signal from a vector and an image signal of a reference block, and an image signal generation unit for generating an image signal from the prediction signal and a prediction error signal decoded by the decoding unit And comprising. For the representative pixel, a pixel located at the vertex constituting the target block, a pixel located near the vertex, or an interpolation pixel located near the vertex is selected, and the target block is a rectangular region, and the geometric transformation motion compensation prediction unit Are the first mode for decoding the differential motion vectors of the upper left, upper right, lower left and lower right representative pixels among the four vertices of the target block, and the upper left, upper right and lower left representatives of the four vertices of the target block. The second mode in which the difference motion vector of the pixel is decoded and the difference motion vector of the lower right representative pixel is not decoded, and the difference motion vector of the upper left and lower left representative pixels among the four vertices of the target block is decoded, and the upper right and a third mode in which no decoded differential motion vector of the representative pixel of the lower right, of the four vertices of the target block, difference motion of the upper left and upper right of the representative pixel vectors In the fourth mode in which the differential motion vectors of the lower left and lower right representative pixels are not decoded, and the differential motion vectors of the upper left and lower right representative pixels of the four vertices of the target block are decoded, and the upper right and lower left And the differential motion vector of the upper left and lower right representative pixels from the four vertices of the target block, and the differential motion vector of the upper left and lower right representative pixels are decoded. A sixth mode in which decoding is not performed, a seventh mode in which the difference motion vector of the upper left, lower left and lower right representative pixels is decoded among the four vertices of the target block, and the target motion is not decoded Of the four vertices of the block, the differential motion vector of the upper right representative pixel is decoded, and the differential motion vector of the upper left, lower left and lower right representative pixels is not decoded. 8 mode, the ninth mode in which the differential motion vector of the lower left representative pixel among the four vertices of the target block is decoded, and the differential motion vector of the upper left, upper right and lower right representative pixels is not decoded, and 4 of the target block The tenth mode in which the differential motion vector of the lower right representative pixel among the vertices is decoded, and the differential motion vector of the upper left, upper right and lower left representative pixels is not decoded, and the upper left and upper right of the four vertices of the target block Among the eleventh mode in which the differential motion vectors of the lower left and lower right representative pixels are not decoded, at least two prediction modes are provided, and the at least one prediction mode includes a fifth mode, a sixth mode, a seventh mode, At least one of an eighth mode, a ninth mode, a tenth mode, and an eleventh mode is included.

  In the first mode, the second mode, the fourth mode, the sixth mode, or the eighth mode, the motion vector generation unit outputs the predicted motion vector of the representative pixel corresponding to the upper right vertex in the rectangular target block to the upper adjacent block. Adaptively selecting the motion vector value of the representative pixel corresponding to the lower right vertex of the block or the motion vector value of the upper adjacent block and the motion vector value of the representative pixel corresponding to the upper left vertex of the target block, Calculating based on the value of the selected motion vector, adding the calculated predicted motion vector and the difference motion vector to calculate the motion vector of the representative pixel corresponding to the upper right vertex in the target block; In the mode, the fifth mode, the seventh mode, the ninth mode, the tenth mode, or the eleventh mode, the representative pixel corresponding to the upper right vertex in the rectangular target block For the predicted motion vector, the motion vector value of the representative pixel corresponding to the lower right vertex of the upper adjacent block or the motion vector value of the upper adjacent block and the motion vector value of the representative pixel corresponding to the upper left vertex of the target block And the value of the selected motion vector may be used as the motion vector of the representative pixel corresponding to the upper right vertex in the target block.

  In the first mode, the second mode, the third mode, the sixth mode, or the ninth mode, the motion vector generation unit outputs the predicted motion vector of the representative pixel corresponding to the lower left vertex in the rectangular target block to the block on the left Adaptively select the motion vector value of the representative pixel corresponding to the lower right vertex of the block or the motion vector value of the left adjacent block and the motion vector value of the representative pixel corresponding to the upper left vertex of the target block. Calculate based on the value of the selected motion vector, add the calculated predicted motion vector and the difference motion vector to calculate the motion vector of the representative pixel corresponding to the lower left vertex in the target block, In the mode, the fifth mode, the seventh mode, the eighth mode, the tenth mode, or the eleventh mode, the representative pixel corresponding to the lower left vertex in the rectangular target block For the predicted motion vector, the motion vector value of the representative pixel corresponding to the lower right vertex of the left adjacent block or the motion vector value of the left adjacent block and the motion vector value of the representative pixel corresponding to the upper left vertex of the target block And the value of the selected motion vector may be used as the motion vector of the representative pixel corresponding to the lower left vertex in the target block.

  Translational motion compensated prediction for generating a prediction signal from a motion vector between a target block in the target image and a reference block in a reference image that is translated with the target block, and an image signal of the reference block And the prediction method information decoded by the decoding unit, for each target block in the target image, either the prediction method by the translation motion compensation prediction unit or the prediction method by the geometric transformation motion compensation prediction unit And a control unit for designating whether to use or not. The data included in the encoded stream is encoded by using motion compensated prediction based on parallel movement and motion compensated prediction based on geometric transformation, and the motion vector generation unit converts the predicted motion vector of the target block into a prediction mode, And, based on the motion compensation prediction method of the adjacent block adjacent to the target block, calculated based on the motion vector value of the representative pixel corresponding to the adjacent vertex of the adjacent block or the motion vector value of the adjacent block Also good.

Yet another embodiment of the present invention is also an image decoding method. This method includes prediction method information for specifying a prediction mode, a differential motion vector of a representative pixel corresponding to the prediction mode, and a prediction included in an encoded stream encoded by using motion compensated prediction by geometric transformation. In accordance with the decoding step for decoding the error signal and the prediction mode specified by the prediction method information, the predicted motion vector of the motion vector of the representative pixel is calculated using the motion vectors inside and outside the target block, and the predicted motion vector is used as the predicted motion vector. A motion vector generation step for generating a motion vector of the representative pixel by adding the differential motion vector of the representative pixel, a target block in the target image, and a reference in the reference image that is geometrically transformed with the target block Representative pixel motion vector between blocks, representative calculated by interpolation using representative pixel motion vector A geometric transformation motion compensated prediction step for generating a prediction signal from a non-prime motion vector and an image signal of a reference block, and a prediction signal and a prediction error signal decoded by the decoding step for generating an image signal And an image signal generation step. As the representative pixel, a pixel located at the vertex constituting the target block, a pixel located near the vertex, or an interpolation pixel located near the vertex is selected, and the target block is a rectangular region, and the geometric transformation motion compensation prediction step Are the first mode for decoding the differential motion vectors of the upper left, upper right, lower left and lower right representative pixels among the four vertices of the target block, and the upper left, upper right and lower left representatives of the four vertices of the target block. The second mode in which the difference motion vector of the pixel is decoded and the difference motion vector of the lower right representative pixel is not decoded, and the difference motion vector of the upper left and lower left representative pixels among the four vertices of the target block is decoded, and the upper right and a third mode in which no decoded differential motion vector of the representative pixel of the lower right, of the four vertices of the current block, the upper left and upper-right differential motion of the representative pixel A fourth mode in which the difference motion vector of the lower left and lower right representative pixels is not decoded, and the difference motion vector of the upper left and lower right representative pixels of the four vertices of the target block is decoded; The fifth mode in which the differential motion vector of the lower left representative pixel is not decoded, and the differential motion vector of the upper right and lower left representative pixels among the four vertices of the target block is decoded, and the differential motion vector of the upper left and lower right representative pixels A seventh mode in which the difference motion vector of the upper left representative pixel is decoded among the four vertices of the target block, and the difference motion vector of the upper right, lower left and lower right representative pixels is not decoded, Among the four vertices of the target block, the differential motion vector of the upper right representative pixel is decoded, and the differential motion vector of the upper left, lower left and lower right representative pixels is decoded. The eighth mode, the ninth mode in which the differential motion vector of the lower left representative pixel is decoded among the four vertices of the target block, and the differential motion vector of the upper left, upper right and lower right representative pixels is not decoded, and the target block 10th mode in which the differential motion vector of the lower right representative pixel is decoded and the differential motion vectors of the upper left, upper right and lower left representative pixels are not decoded, and the upper left of the four vertices of the target block Among the eleventh modes in which the differential motion vectors of the upper right, lower left and lower right representative pixels are not decoded, at least two prediction modes are provided. The at least one prediction mode includes the fifth mode, the sixth mode, the seventh mode, and the like. At least one of a mode, an eighth mode, a ninth mode, a tenth mode, and an eleventh mode is included.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  The compression efficiency of the code amount can be improved by the image coding method using the motion compensation prediction by the geometric transformation.

It is a block diagram which shows the structure of the image coding apparatus which concerns on Embodiment 1 of this invention. 2A to 2H are diagrams for explaining a macroblock partition and a sub-macroblock partition. It is a figure for demonstrating the representative pixel corresponding to four vertices of an object block. 4A to 4K are diagrams for explaining eleven modes of geometric conversion motion compensation prediction. It is a block diagram which shows the structure of the geometric conversion motion compensation prediction part which concerns on Embodiment 1 of this invention. It is a figure which shows an example of a syntax structure. FIGS. 7A to 7D are diagrams for explaining a method of predicting a motion vector of an encoding target block when motion compensated prediction based on parallel movement is selected for both the encoding target block and adjacent blocks. It is a figure for demonstrating an example of the scaling process of a motion vector value. It is a block diagram which shows the structure of the geometric conversion motion vector detection part which concerns on Embodiment 1 of this invention. FIGS. 10A to 10H are diagrams illustrating specific examples of a method for detecting a motion vector of a representative pixel in geometric transformation by correcting a translation motion vector generated by a translation motion compensation prediction unit. is there. It is a flowchart which shows the encoding process procedure of a macroblock in the image coding apparatus which concerns on Embodiment 1 of this invention. It is a flowchart which shows the procedure of the parallel movement motion compensation process of step S102 of the flowchart of FIG. It is a flowchart which shows the procedure of the geometric conversion motion compensation process of step S103 of the flowchart of FIG. It is a flowchart which shows the procedure which detects the motion vector of translation. It is a flowchart which shows the procedure which detects the motion vector of geometric transformation. It is a flowchart which shows the procedure of the geometric conversion motion compensation process of step S136 of the flowchart of FIG. It is a block diagram which shows the structure of the image decoding apparatus which concerns on Embodiment 2 of this invention. It is a flowchart which shows the decoding process procedure of a macroblock in the image decoding apparatus which concerns on Embodiment 2 of this invention. FIG. 19 is a flowchart illustrating a procedure when a process of switching a calculation method of a prediction vector of the second representative pixel b occurs according to the motion vector value of the first representative pixel a of the target block in S203 of the flowchart of FIG. 18. . FIG. 19 is a flowchart illustrating a procedure when a process of switching the calculation method of the prediction vector of the third representative pixel c occurs in S203 of the flowchart of FIG. 18 according to the motion vector value of the first representative pixel a of the target block. .

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following embodiments, AVC / H. An example of encoding / decoding based on the H.264 encoding method will be described.

  FIG. 1 is a block diagram showing a configuration of image coding apparatus 100 according to Embodiment 1 of the present invention. The image encoding device 100 includes an image buffer 101, a translational motion compensation prediction unit 102, a geometric transformation motion compensation prediction unit 103, a prediction method determination unit 104, a prediction error signal generation unit 105, a prediction error signal encoding unit 106, 1 coding bit stream generation part 107, 2nd coding bit stream generation part 108, 3rd coding bit stream generation part 109, prediction error signal decoding part 110, decoded picture signal generation part 111, decoded picture buffer 112, and output switch 113 are included. .

  These configurations can be realized by an arbitrary processor, memory, or other LSI in terms of hardware, and can be realized by a program loaded in the memory in terms of software, but here by their cooperation. Draw functional blocks. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The image buffer 101 temporarily stores image signals to be encoded supplied in the order of shooting / display time. The image buffer 101 converts the stored image signal to be encoded in predetermined pixel block units (here, macroblock units) into a translational motion compensation prediction unit 102, a geometric transformation motion compensation prediction unit 103, and a prediction error. The signal generation unit 105 is supplied in parallel. At this time, the images supplied in the order of shooting / display time are rearranged in the encoding order and output from the image buffer 101.

  In the MPEG series, a macroblock refers to a luminance signal of 16 × 16 pixels and two corresponding color difference signal blocks. When YUV of the color difference format is 4: 2: 0, the color difference signal has a size of 8 × 8 pixels.

  In the present embodiment, an intra coding method that encodes within a screen without using a reference image, a motion compensation prediction method by translation using a reference image, and a motion compensation prediction method by geometric transformation using a reference image Is used. Here, the reference image is a decoded image obtained by local decoding. In the present embodiment, since attention is not paid to the intra coding method, the configuration is omitted in FIG. These coding mode modes are adaptively switched individually or in combination in units of macroblocks. Note that a method of encoding all macroblocks using a motion compensated prediction method based on geometric transformation using a reference image is also possible.

  The translational motion compensation prediction unit 102 performs motion compensation prediction by translation between the macroblock signal to be encoded supplied from the image buffer 101 and the reference image signal supplied from the decoded image buffer 112. The translational motion compensation prediction unit 102 generates a motion vector and a prediction signal between the target block in the target image in each mode and the reference block in the reference image that is translated with the target block. To the prediction method determination unit 104. In the present embodiment, the translational motion compensation prediction unit 102 performs motion compensation prediction based on translation similar to the existing motion compensation prediction stipulated in the AVC / H.264 scheme or the like.

  In motion compensated prediction, a forward or backward decoded image is used as a reference image in the display order supplied from a decoded image buffer 112 described later. The translational motion compensation prediction unit 102 performs block matching between the macroblock signal supplied from the image buffer 101 and the reference image signal supplied from the decoded image buffer 112 within a predetermined detection range in the reference image. Do. The translational motion compensation prediction unit 102 specifies a reference block signal in the reference image signal having the smallest error from the macroblock signal by block matching, and a motion vector between the macroblock signal and the reference block signal Is detected.

  This block matching is performed in a plurality of prescribed modes. Each of the plurality of modes has a different reference index, motion compensation block size, L0 / L1 prediction, and the like. The reference index is an index indicating a reference picture. Note that L0 / L1 prediction can be selected only in the B slice. A specific example of the block size will be described later.

  In addition, when performing motion compensation prediction, motion compensation can be performed with a pixel accuracy of less than one pixel. For example, in the AVC / H.264 system or the like, motion compensation can be performed up to an accuracy of 1/4 of a luminance signal and 1/8 of an accuracy of a color difference signal. When motion compensation is performed with pixel accuracy of less than one pixel, a signal with pixel accuracy of less than one pixel is generated by interpolation from the signals of surrounding integer pixels in the reference image.

  The translational motion compensation prediction unit 102 performs motion compensation prediction in each mode, and sends a prediction signal (more specifically, a motion compensation prediction block signal) and a motion vector in each mode to the prediction method determination unit 104. Supply.

  Next, AVC / H. The motion compensation block size in the H.264 system will be described. 2A to 2H are diagrams for explaining a macroblock partition and a sub-macroblock partition. Here, in order to simplify the description, only the pixel block of the luminance signal is drawn. In the MPEG series, the macroblock is defined by a square area. Generally, AVC / H. In the MPEG series including the H.264 system, a block defined by 16 × 16 pixels (16 horizontal pixels and 16 vertical pixels) is called a macro block. Furthermore, AVC / H. In the H.264 system, a block defined by 8 × 8 pixels is called a sub macroblock. The macroblock partition refers to each small block obtained by further dividing the macroblock for motion compensation prediction. The sub-macroblock partition refers to each small block obtained by further dividing the sub-macroblock for motion compensation prediction.

  FIG. 2A is a diagram showing that a macroblock is composed of one macroblock partition composed of a luminance signal of 16 × 16 pixels and two corresponding color difference signals. Here, this configuration is referred to as a 16 × 16 mode macroblock type.

  In FIG. 2B, the macroblock is composed of two macroblock partitions each composed of a luminance signal of 16 × 8 pixels (horizontal 16 pixels, vertical 8 pixels) and two corresponding color difference signals. FIG. The two macroblock partitions are arranged vertically. Here, this configuration is called a 16 × 8 mode macroblock type.

  In FIG. 2 (c), the macroblock is composed of two macroblock partitions composed of a luminance signal of 8 × 16 pixels (horizontal 8 pixels, vertical 16 pixels) and two corresponding color difference signals. FIG. The two macroblock partitions are arranged side by side. Here, this configuration is referred to as an 8 × 16 mode macroblock type.

  FIG. 2D is a diagram showing that a macroblock is composed of four macroblock partitions each composed of a luminance signal of 8 × 8 pixels and two corresponding color difference signals. These four macroblock partitions are arranged vertically and horizontally. This configuration is referred to as an 8 × 8 mode macroblock type.

  FIG. 2 (e) is a diagram showing that the sub macroblock is composed of one submacroblock partition composed of a luminance signal of 8 × 8 pixels and two corresponding color difference signals. Here, this configuration is referred to as an 8 × 8 mode sub-macroblock type.

  In FIG. 2F, the sub-macroblock is composed of two sub-macroblock partitions each composed of a luminance signal of 8 × 4 pixels (horizontal 8 pixels, vertical 4 pixels) and two corresponding color difference signals. FIG. These two sub-macroblock partitions are arranged vertically. This configuration is referred to as an 8 × 4 mode sub-macroblock type.

  In FIG. 2G, the sub-macroblock is composed of two macroblock partitions each composed of a luminance signal of 4 × 8 pixels (horizontal 4 pixels, vertical 8 pixels) and two corresponding color difference signals. FIG. The two macroblock partitions are arranged side by side. Here, this configuration is called a 4 × 8 mode sub-macroblock type.

  FIG. 2 (h) is a diagram showing that the sub-macroblock is composed of four sub-macroblock partitions composed of a luminance signal of 4 × 4 pixels and two corresponding color difference signals. These four sub-macroblock partitions are arranged two by two vertically and horizontally. Here, this configuration is called a 4 × 4 mode sub-macroblock type.

  AVC / H. In the H.264 encoding method, a mechanism for switching and using an optimum one from the above motion compensation block sizes is adopted. First, one of macroblock types of 16 × 16, 16 × 8, 8 × 16, and 8 × 8 modes is selected as the motion compensation block size for each macroblock. When the 8 × 8 mode macroblock type is selected, the motion compensation block size in units of submacroblocks is among the 8 × 8, 8 × 4, 4 × 8, and 4 × 4 mode sub macroblock types. Is selected.

  The luminance signal is motion-compensated with the number of pixels of the selected size. When the color difference format is 4: 2: 0, the color difference signal is motion-compensated with half the number of pixels both horizontally and vertically. In this way, motion compensation block size information is encoded with syntax elements called macroblock types and sub-macroblock types. The syntax is an expression rule of the encoded bit string, and the syntax element is information that is specified to be transmitted in the syntax.

  For any 16 × 16, 16 × 8, 8 × 16 and 8 × 8 mode macroblock types, one motion vector is detected per macroblock partition. That is, one motion vector for the 16 × 16 mode macroblock type, two motion vectors for the 16 × 8 and 8 × 16 mode macroblock types, and 4 for the 8 × 8 mode macroblock type. Each motion vector is detected.

  Each pixel of the luminance signal and chrominance signal of each macroblock partition is motion compensated according to one motion vector of the macroblock partition. That is, each pixel is motion-compensated using the same motion vector.

  Returning to FIG. 1, the geometric transformation motion compensation prediction unit 103 performs parallel movement between the encoding target macroblock signal supplied from the image buffer 101 and the reference image signal supplied from the decoded image buffer 112. In addition, motion compensation prediction is performed by geometric transformation accompanied by deformation including enlargement / reduction / rotation. The geometric transformation motion compensated prediction unit 103 calculates a motion vector and a prediction signal between the target block in the target image in each mode and the reference block in the reference image that is geometrically transformed with the target block. Generated and supplied to the prediction method determination unit 104. More specifically, the geometric transformation motion compensation prediction unit 103 selects, as representative pixels, pixels that are located at the vertices constituting the target block, pixels that are located near the vertices, or interpolation pixels that are located near the vertices. A pixel motion vector is calculated. Here, since the target block is a quadrangular (for example, square, rectangular) region, the number of vertices in the region is four. Therefore, there are four representative pixels. Then, the geometric conversion motion compensation prediction unit 103 calculates a motion vector of a pixel other than the representative pixel by interpolation using the motion vector of the representative pixel. A prediction signal is generated for each pixel according to the motion vector for each pixel.

  FIG. 3 is a diagram for explaining representative pixels corresponding to four vertices of the target block. Here, the upper left vertex or the vertex vicinity of the target block is the vertex a, the upper right vertex or the vertex vicinity is the vertex b, the lower left vertex or the vertex vicinity is the vertex c, and the lower right vertex or the vertex vicinity is the vertex d. The block width (the number of pixels in the horizontal direction) W and the block height (the number of pixels in the vertical direction) H are both 16. In the example shown in FIG. 3, the pixel a located at the upper left vertex a and the pixels b, c, d located near the upper right, lower left, and lower right vertices are set as representative pixels a, b, c, d, respectively. doing. In FIG. 3, the representative pixel is represented by a black circle, the non-representative pixel is represented by a white circle, and the representative pixel is represented by a pixel a (0, 0) and a pixel b existing at the vertex of the target block (16 × 16 pixel macroblock in FIG. 3). (W, 0), pixel c (0, H) and pixel d (W, H) are set. Hereinafter, in the present specification, the coordinates of each pixel are represented by (i, j), the horizontal coordinate is represented by i, and the vertical coordinate is represented by j by one pixel. Further, the coordinate of the upper left pixel of the target block is set as the origin (0, 0). In the example shown in FIG. 3, the upper left first representative pixel a is included in the target block, but the upper right second representative pixel b, the lower left third representative pixel c, and the lower right fourth representative pixel d are targets. Not included in the block.

  The geometric transformation motion compensated prediction unit 103 encodes and decodes a difference vector of four representative pixels of a target block, a DMV4 mode that encodes and decodes a difference vector of three representative pixels of the target block, and A DMV2V mode for encoding / decoding a difference vector of two representative pixels in the vertical direction of the target block, a DMV2H mode for encoding / decoding a difference vector of two representative pixels in the horizontal direction of the target block, and a pair of target blocks DMV2A mode and DMV2B mode for encoding / decoding the difference vector of two representative pixels in the angular direction, DMV1A mode for encoding / decoding only the difference vector of the upper left representative pixel of the target block, and the upper right representative of the target block DMV1B mode, which encodes and decodes only the pixel difference vector, DMV1C mode for encoding / decoding only the difference vector of the lower left representative pixel of the block, DMV1D mode for encoding / decoding only the difference vector of the lower right representative pixel of the target block, and the difference vector of the representative pixel of the target block DMV0 mode that does not encode / decode.

  Note that the geometric transformation motion compensation prediction unit 103 does not necessarily need to execute all modes, and may use one mode in a fixed manner according to the processing capability and the frequency of occurrence of each mode. Moreover, the design provided with only one mode may be sufficient. Moreover, you may perform at least 2 mode among 11 modes. The at least two modes may include at least two modes of DMV2A mode, DMV2B mode, DMV1A mode, DMV1B mode, DMV1C mode, DMV1D mode, and DMV0 mode. In addition, the number of modes included in the geometric transformation motion compensation prediction unit 103 may be different from the number of modes to be executed. For example, at least two modes may be provided, and at least two of them may be executed. In these cases, it is possible to reduce the amount of code for identifying the calculation amount and the mode.

  In the description of the present embodiment, the target block is described as a 16 × 16 pixel macroblock. However, the size of the target block is not limited to 16 × 16 pixels, and an 8 × 8 pixel sub-macro. It may be a block or a block of 32 × 32 pixels, 48 × 48 pixels, 64 × 64 pixels, 128 × 128 pixels, or the like. In the description of the present embodiment, the target block is described as a square macroblock. However, the shape of the target block is not limited to a square, and a macro of 16 × 8 pixels or 8 × 16 pixels. It may be a block partition, a block of 8 × 4 pixels, a 4 × 8 pixel sub-macroblock partition, 32 × 16 pixels, 16 × 32 pixels, or the like.

  4A to 4K are diagrams for explaining eleven modes of geometric conversion motion compensation prediction. Black circles and white circles represent representative points. 4A to 4K illustrate examples in which the representative point and the representative pixel coincide with each other. A representative point drawn with a black circle encodes and decodes a difference vector, and indicates that the motion vector of this representative point is the sum of a prediction vector and a difference vector. For the representative points drawn with white circles, the difference vector is not encoded / decoded, and the prediction vector is used as it is as a motion vector.

  The DMV4 mode shown in FIG. 4A is a mode for encoding the difference vector of the representative pixels a, b, c, and d (corresponding to the first mode in the claims). The DMV3 mode shown in FIG. 4B is a mode in which the difference vectors of the representative pixels a, b, and c are encoded without encoding the difference vector of the representative pixel d (corresponding to the second mode of the claims). ). The DMV2V mode shown in FIG. 4C is a mode in which the difference vector of the representative pixels a and c is not encoded, but the difference vector of the representative pixels a and c is encoded (corresponding to the third mode of the claims). ). The motion vector values of the representative pixel a and the representative pixel b are equal, and the motion vector values of the representative pixel c and the representative pixel d are also equal. The DMV2H mode shown in FIG. 4D is a mode in which the difference vector of the representative pixels a and b is not encoded, but the difference vector of the representative pixels a and b is encoded (corresponding to the fourth mode in the claims). ). The motion vector values of the representative pixel a and the representative pixel c are equal, and the motion vector values of the representative pixel b and the representative pixel d are also equal.

  The DMV2A mode shown in FIG. 4 (e) is a mode that encodes the difference vectors of the representative pixels a and d without encoding the difference vectors of the representative pixels b and c (corresponding to the fifth mode of the claims). ). The average value for each component of the motion vectors of the representative pixel a and the representative pixel d is set as the value of the motion vector of the representative pixels b and c. The DMV2B mode shown in FIG. 4 (f) is a mode in which the difference vector of the representative pixels a and d is not encoded, and the difference vector of the representative pixels b and c is encoded (corresponding to the sixth mode of the claims). ). The average value for each component of the motion vectors of the representative pixel b and the representative pixel c is set as the value of the motion vector of the representative pixels a and d.

  The DMV1A mode shown in FIG. 4G is a mode that encodes the difference vector of the representative pixel a without encoding the difference vector of the representative pixels b, c, and d (corresponding to the seventh mode of the claims). ). The DMV1B mode shown in FIG. 4 (h) is a mode that encodes the difference vector of the representative pixel b without encoding the difference vector of the representative pixels a, c, and d (corresponding to the eighth mode in the claims). ). The DMV1C mode shown in FIG. 4 (i) is a mode that encodes the difference vector of the representative pixel c without encoding the difference vector of the representative pixels a, b, and d (corresponding to the ninth mode in the claims). ). The DMV1D mode shown in FIG. 4 (j) is a mode that encodes the difference vector of the representative pixel d without encoding the difference vector of the representative pixels a, b, and c (corresponding to the tenth mode in the claims). ). The DMV0 mode shown in FIG. 4 (k) is a mode in which the difference vectors of the representative pixels a, b, c, and d are not encoded (corresponding to the eleventh mode in the claims).

Table 1 below summarizes the above modes.

  In Table 1 above, “DMV encoding” represents encoding a difference vector. In this case, the motion vector MV is defined by the sum of the prediction vector PMV and the difference vector DMV. “PMV” represents that the difference vector DMV is not encoded, and the prediction vector PMV is directly used as the motion vector MV. A motion vector MV calculated from a motion vector MV of a neighboring block or another motion vector MV of the target block is set as a prediction vector PMV. In other cases (other than “DMV encoding” and “PMV”), the difference vector DMV is not encoded, and is calculated from the motion vectors MV of other vertices according to the expressions in Table 1. When encoding a motion vector, if all four difference vectors are encoded, the amount of code of the motion vector increases. Therefore, if it can be determined that it is not necessary to encode all four motion vectors, the number of difference vectors to be encoded is reduced. That is, a mode other than the DMV4 mode is adopted.

  Here, in motion vector encoding, a prediction vector is obtained by performing prediction from surrounding blocks or target blocks, and a code amount is obtained by encoding a difference vector that is a difference between the motion vector and the prediction vector. Reduce. Encoding / decoding a motion vector specifically means encoding / decoding a difference vector, and the motion vector MV is obtained by adding a prediction vector PMV and a difference vector DMV for each component. Specifically, not encoding / decoding a motion vector means that a prediction vector calculated from a surrounding block or a target block is used as it is as a motion vector without encoding / decoding a difference vector.

  Returning to FIG. 1, the prediction method determination unit 104 uses the above-described DMV4 mode, DMV3 mode, DMV2V mode, DMV2H mode, DMV2A mode, DMV2B mode, DMV1A mode, DMV1B mode as a prediction method by the geometric transformation motion compensation prediction unit 103. Any one of DMV1C mode, DMV1D mode, and DMV0 mode can be adopted. Detailed processing of the prediction method determination unit 104 will be described later.

  Hereinafter, the geometric transformation motion compensation prediction unit 103 will be described more specifically. FIG. 5 is a block diagram showing a configuration of the geometric transformation motion compensation prediction unit 103 according to Embodiment 1 of the present invention. The geometric transformation motion compensation prediction unit 103 includes a geometric transformation motion vector detection unit 121, a first non-representative point motion vector calculation unit 122, and a first prediction unit 123.

  In motion compensated prediction by geometric transformation according to the present embodiment, each pixel of the luminance signal and chrominance signal of the macroblock, macroblock partition, sub-macroblock is not motion-compensated with the same motion vector, but for each pixel. Different motion vectors are created for motion compensation. The geometric transformation motion vector detection unit 121 selects a vertex of each macroblock, a pixel near the vertex, or an interpolation pixel located near the vertex as a representative pixel, and obtains a motion vector thereof.

  The geometric transformation motion vector detection unit 121 detects or calculates the motion vector of the representative pixel for each mode of the geometric transformation motion compensation prediction shown in FIGS. Specifically, the representative pixel motion vector detection process in the geometric transformation motion vector detection unit 121 detects a motion vector of a representative point for encoding / decoding a difference vector in each mode, and encodes the difference vector. For the value of the motion vector of the representative point to be converted / decoded, a prediction vector is calculated, and the prediction vector is used as it is as a motion vector. For example, in the DMV4 mode in which the difference vector of four representative pixels of the first representative pixel a, the second representative pixel b, the third representative pixel c, and the fourth representative pixel d is encoded / decoded, Each motion vector is detected. In the DMV3 mode in which the difference vector of the three representative pixels of the first representative pixel a, the second representative pixel b, and the third representative pixel c is encoded / decoded, the respective motion vectors of the three representative pixels are detected. Then, the motion vector of the fourth representative pixel d is calculated as a motion vector, and the motion vector is calculated. Here, a method for calculating a prediction vector will be described by taking the case of representative pixels corresponding to the four vertices of the target block shown in FIG. 3 as an example.

In the DMV3 mode in which the motion vectors of the three representative pixels a, b, and c corresponding to the three vertices a, b, and c are encoded and decoded, the motion vectors of these pixels a, b, and c are assigned. A prediction vector of the representative pixel d corresponding to the vertex d is calculated from the three motion vectors, and the calculated prediction vector is set as the motion vector of the representative pixel d. The motion vector Va (= V (0,0)) of the pixel a, the motion vector Vb (= V (W, 0)) of the pixel b, and the motion vector Vc = (V (0, H)) of the pixel c From the relationship with the motion vector Vd = (V (W, H)) of the pixel d, the motion vector Vd of the representative pixel d corresponding to the vertex d is calculated by the following equation (1).
Vd = Vc + (Vb−Va) (1)
Or it calculates by following formula (2).
Vd = Vb + (Vc−Va) (2)

  In the case of the DMV2V mode in which the difference vector between two representative pixels in the vertical direction of the first representative pixel a and the third representative pixel c is encoded and decoded, the motion vectors of the two representative pixels are detected, For the motion vectors of the second representative pixel b and the fourth representative pixel d, a prediction vector is calculated and used as a motion vector. Here, the motion vector of the first representative pixel a is used as the motion vector as the prediction vector of the second representative pixel b (Vb = Va), and the motion vector of the third representative pixel c is used as the prediction vector of the fourth representative pixel d, ie, motion. Let it be a vector (Vd = Vc).

  Further, in the DMV2H mode in which the difference vector between the two representative pixels in the horizontal direction of the first representative pixel a and the second representative pixel b is encoded and decoded, the respective motion vectors of the two representative pixels are detected, and For the motion vectors of the three representative pixels c and the fourth representative pixel d, a prediction vector is calculated and used as a motion vector. Here, the motion vector of the first representative pixel a is used as the motion vector as the prediction vector of the third representative pixel c (Vc = Va), and the motion vector of the second representative pixel b is used as the prediction vector of the fourth representative pixel d, ie, motion. Let it be a vector (Vd = Vb).

Further, in the DMV2A mode in which the difference vector between two representative pixels in the diagonal direction of the first representative pixel a and the fourth representative pixel d is encoded / decoded, the respective motion vectors of the two representative pixels are detected, and For the motion vectors of the second representative pixel b and the third representative pixel c, a prediction vector is calculated and used as a motion vector. Here, an average value for each component of the motion vector of the first representative pixel a and the motion vector of the fourth representative pixel d is set as a prediction vector of the second representative pixel b and the third representative pixel c, that is, a motion vector.
Vb = Vc = (Va + Vd) / 2 Formula (3)

In the DMV2B mode in which the difference vector between two representative pixels in the diagonal direction of the second representative pixel b and the third representative pixel c is encoded and decoded, the respective motion vectors of the two representative pixels are detected, and For the motion vectors of the first representative pixel a and the fourth representative pixel d, a prediction vector is calculated and used as a motion vector. Here, an average value for each component of the motion vector of the second representative pixel b and the motion vector of the third representative pixel c is set as a difference vector of the first representative pixel a and the fourth representative pixel d, that is, a motion vector.
Va = Vd = (Vb + Vc) / 2 Formula (4)

  Further, in the DMV1A mode in which only the differential vector of the first representative pixel a is encoded / decoded, the motion vector of the representative pixel is detected, and the second representative pixel b, the third representative pixel c, and the fourth representative pixel d are detected. As a motion vector, a prediction vector is calculated and used as a motion vector. In the DMV1B mode in which only the differential vector of the second representative pixel b is encoded / decoded, the motion vector of the representative pixel is detected, and the first representative pixel a, the third representative pixel c, and the fourth representative pixel d are detected. As a motion vector, a prediction vector is calculated and used as a motion vector. In the DMV1C mode in which only the differential vector of the third representative pixel c is encoded / decoded, the motion vector of the representative pixel is detected, and the first representative pixel a, the second representative pixel b, and the fourth representative pixel d are detected. As a motion vector, a prediction vector is calculated and used as a motion vector. In the DMV1D mode in which only the difference vector of the fourth representative pixel d is encoded / decoded, the motion vector of the representative pixel is detected, and the first representative pixel a, the second representative pixel b, and the third representative pixel c are detected. As a motion vector, a prediction vector is calculated and used as a motion vector. Note that, in the DMV0 mode in which the difference vector of any representative pixel is not encoded / decoded, the motion vector of all the representative pixels is calculated as a motion vector. A method for calculating these prediction vectors will be described later.

  The geometric transformation motion vector detection unit 121 may use a motion vector calculated in units of pixels by an optical flow method or the like as a motion vector of a representative pixel of each macroblock, or may have reliability such as an edge or a corner of an image. You may use the motion vector which correct | amended the motion vector of the feature point judged to be high by the interpolation / extrapolation calculation. Further, the motion vector of the macroblock or macroblock / partition generated by the translational motion compensation prediction unit 102 may be corrected and used. In addition, when correcting and using a motion vector of a macroblock or a macroblock partition, the value of the motion vector is applied to the representative pixel, and verification is performed while adjusting the value of the motion vector in the increasing and decreasing directions. Thus, the value of the motion vector is corrected.

  Next, the first non-representative point motion vector calculation unit 122 interpolates the motion vectors of all the pixels in the macroblock from the motion vectors of the representative pixels supplied from the geometric transformation motion vector detection unit 121 by interpolation such as linear interpolation. Is calculated by using

  Hereinafter, the calculation method of the motion vector of each pixel other than the representative pixel in the geometric transformation motion compensation prediction by the first non-representative point motion vector calculation unit 122 will be described with a specific example.

The four motion vectors Va, Vb, Vc, Vd of the four representative pixels a, b, c, d detected and calculated by the geometric transformation motion vector detection unit 121 and supplied are the other pixels P (i, j ) Motion vector V (i, j) is generated by linear interpolation. When the block width (the number of pixels in the horizontal direction) is W and the height of the block (the number of pixels in the vertical direction) is H (W = H = 16 in the example of FIG. 3), the representative pixels a and b The motion vector V (i, j) of the pixel P (i, j) other than, c, d is calculated by the following equation (5).
V (i, j) = {(Wi) (H-j) Va + i (Hj) Vb + (Wi) j.Vc + i.j.Vd} / (W.H) (5) )

  The method for interpolating both in the horizontal direction and the vertical direction at the same time (that is, in two dimensions) has been described above. From the motion vectors of two representative pixels in the horizontal direction, the motion of pixels on a straight line connecting these two points The motion vector of the non-representative pixel may be calculated by interpolating the vector, and the motion vector of the other pixel may be calculated by further interpolating in the vertical direction using the motion vector of each pixel that has already been calculated. .

A calculation method in this case will be described. The motion vector V (i, 0) of each pixel P (i, 0) on the line passing through the pixel a and the pixel b is calculated by the following equation (6).
V (i, 0) = Va + (Vb−Va) * (i−0) / W Expression (6)

Similarly, the motion vector V (i, H) of each pixel P (i, H) on the line passing through the pixel c and the pixel d is calculated by the following equation (7).
V (i, H) = Vc + (Vd−Vc) * (i−0) / W (7)

Further, the motion vector V (i, j) of the remaining pixel P (i, j) is calculated by the following equation (8).
V (i, j) = V (i, 0) + {V (i, H) −V (i, 0)} * (j−0) / H (8)

  The first prediction unit 123 performs motion compensation prediction for each pixel using the calculated motion vector for each pixel. In the above description, the example of calculating the motion vector of each pixel included in the macroblock has been described. However, the motion vector of each pixel included in the sub-macroblock can be calculated in the same manner.

  The pixel-by-pixel motion vector calculation process and the motion compensation process for each pixel are performed in a plurality of prescribed modes. Each of the plurality of modes has a different reference index, motion compensation block size, L0 / L1 prediction, and the like. Note that L0 / L1 prediction can be selected only in the B slice.

  The first prediction unit 123 performs motion compensation prediction in each mode, and determines prediction mode information, a motion vector, and a prediction signal (more specifically, a motion compensation prediction block signal) in each mode as a prediction method. Supplied to the unit 104.

  The first prediction unit 123 performs motion compensation prediction on each pixel according to the motion vector of each pixel calculated by the first non-representative point motion vector calculation unit 122. More specifically, motion compensation prediction is performed by generating an interpolation signal from the reference image pixel indicated by the motion vector of each pixel. It should be noted that the accuracy of the motion vector of each pixel, the rounding of numerical values required in the calculation process, and the like need to be defined so that the same value is obtained regardless of which decoding device is used. In addition, when the coordinates of the predicted pixel specified by the motion vector are expressed with a decimal point, the pixel is interpolated from surrounding pixels at the time of motion compensation. As the interpolation method, 4 to 6 tap filtering, linear interpolation, or the like can be used.

  In the DMV4 mode, complex deformations that cannot be expressed in modes other than the DMV4 mode, such as enlargement / reduction, rotation, and parallel movement, can be represented by four motion vectors. In DMV3 mode, deformation by affine transformation can be represented by three motion vectors. The expressive power of deformation is limited compared to the DMV4 mode, but the number of motion vectors (more specifically, difference vectors) to be encoded can be reduced.

  In DMV2V mode, two motion vectors can represent different deformations in the vertical direction in addition to translation. The expressive power of deformation is limited compared to the DMV4 mode and DMV3 mode, but the number of motion vectors to be encoded can be reduced. In the DMV2H mode, two motion vectors can represent different deformations in the horizontal direction in addition to translation. The expressive power of deformation is limited compared to the DMV4 mode and DMV3 mode, but the number of motion vectors to be encoded can be reduced. In the DMV2A mode and the DMV2B mode, two motion vectors can represent different deformations in the oblique direction in addition to the parallel movement. The expressive power of deformation is limited compared to the DMV4 mode and DMV3 mode, but the number of motion vectors to be encoded can be reduced. In the DMV1A mode, the DMV1B mode, the DMV1C mode, the DMV1D mode, and the DMV0 mode, the number of motion vectors to be encoded can be greatly reduced when fitting to a deformation analogized from neighboring blocks.

  Returning to FIG. 1, the prediction method determination unit 104 employs either the prediction method by the translational motion compensation prediction unit 102 or the prediction method by the geometric transformation motion compensation prediction unit 103 for each target block in the target image. To decide. More specifically, it is determined which mode of which prediction method is adopted. Hereinafter, prediction method selection including mode selection is referred to.

  That is, the prediction method determining unit 104 selects which reference image is used to encode motion compensation prediction based on parallel movement or motion compensated prediction based on geometric transformation in which pixel block unit. Decide how. When selecting motion compensated prediction by geometric transformation, select one of DMV4 mode, DMV3 mode, DMV2V mode, DMV2H mode, DMV2A mode, DMV2B mode, DMV1A mode, DMV1B mode, DMV1C mode, DMV1D mode, and DMV0 mode . At that time, the prediction method is determined by determining which combination of these items is the prediction method capable of realizing the most efficient encoding. As a criterion for determining the prediction method, for example, rate distortion theory considering the relationship between the code amount and distortion can be used. More specifically, the code amount of the macroblock (that is, the prediction method information, the motion vector, and the total code amount of the prediction signal) is calculated, and the distortion amount is calculated from the difference between the encoding target image and the decoded image. A prediction method is selected that minimizes a code amount and a rate distortion function using the distortion amount as an input variable.

  The prediction method determination unit 104 supplies the adopted prediction method information to the first encoded bit string generation unit 107 and the difference vector calculation unit 114, and supplies a motion vector corresponding to the adopted prediction method to the difference vector calculation unit 114. At the same time, the prediction method determination unit 104 supplies the prediction signal generated by the employed prediction method to the prediction error signal generation unit 105.

  The first encoded bit string generation unit 107 encodes the prediction method information supplied from the prediction method determination unit 104 by entropy encoding such as arithmetic encoding, and generates an encoded bit string. The prediction block size and the L0 / L1 prediction distinction to be included in the prediction method information are combined and encoded as a macroblock type. Whether to use motion compensation prediction by parallel movement or motion compensation prediction by geometric transformation to be included in the prediction method information, in the case of geometric transformation, DMV4 mode, DMV3 mode, DMV2V mode, DMV2H mode, DMV2A mode, The following description method can be used for information for identifying which one of DMV2B mode, DMV1A mode, DMV1B mode, DMV1C mode, DMV1D mode, and DMV0 mode is used. For example, a syntax element may be separately prepared and described, or the macroblock type may be extended and described by combining with information to be encoded as another macroblock type.

  For example, a syntax element “geom_type” is prepared for each block unit for switching whether or not to perform motion compensation prediction by geometric transformation. “Geom_type” value is “0” for motion compensated prediction by translation, “geom_type” value is “1” for DMV0 mode of motion compensated prediction by geometric transformation, “geom_type” value is “2” for geometry DMV1A mode for motion compensated prediction by transformation, DMV1B mode for motion compensated prediction by geometric transformation when the value of “geom_type” is “3”, and DMV1C mode for motion compensated prediction by geometric transformation when the value of “geom_type” is “4” , The value of “geom_type” is “5” in the DMV1D mode of motion compensation prediction by geometric transformation, the value of “geom_type” is “6” in the DMV2V mode of motion compensation prediction by geometric transformation, and the value of “geom_type” is “ “7” is a DMV2H mode for motion compensated prediction by geometric transformation, “geom_type” is “8” for DMV2A mode for motion compensated prediction by geometric transformation, and “geom_type” is “9” by geometric transformation. DMV2B mode for compensated prediction, “10” for “geom_type” in DMV3 mode for motion compensated prediction by geometric transformation, and “DMV4” for motion compensated prediction by geometric transformation for “geom_type” value “11” Encode as representing.

  FIG. 6 is a diagram illustrating an example of the syntax structure. FIG. 6 shows an example in which one “geom_type” is prepared for each macroblock, and motion compensation prediction is performed in a mode common to both L0 prediction and L1 prediction. However, in the case of a mode in which prediction is performed from both L0 and L1. By preparing two, it is possible to perform different motion compensation predictions for each of the L0 prediction and the L1 prediction.

  When encoding the motion vector of the target block, the motion of the pixel of the adjacent block or the adjacent block is calculated using the correlation with the motion vector of the neighboring adjacent block or the pixel of the adjacent block that has already been encoded or decoded. A prediction vector can be calculated by predicting a motion vector from the vector. Then, by calculating a difference vector that is a difference between the prediction vector and the motion vector of the target block or the target pixel, the code amount of the motion vector of the target block or the target pixel can be reduced. Here, in a target block in which motion compensation prediction by geometric transformation is adopted, a motion vector of the representative pixel is a target for encoding.

  Returning to FIG. 1, the difference vector calculation unit 114 selects another representative pixel of the target block, a neighboring neighboring block that has already been encoded, or its neighboring block according to the prediction method information supplied from the prediction method determining unit 104. The prediction vector is calculated by predicting the motion vector of the target block or the target pixel from the motion vector of the target pixel. Note that when the prediction vector is calculated by the translational motion compensation prediction unit 102, the geometric transformation motion compensation prediction unit 103, and the prediction method determination unit 104, these values may be used by the difference vector calculation unit 114. Then, the difference vector calculation unit 114 calculates a difference between the prediction vector and the motion vector supplied from the prediction method determination unit 104, generates a difference vector, and supplies the difference vector to the second encoded bit string generation unit 108.

  In the DMV4 mode, the difference vector calculation unit 114 encodes and decodes the motion vectors of the four representative pixels a, b, c, and d corresponding to the four vertices a, b, c, and d. Vector and difference vector are calculated. In the DMV3 mode, in order to encode and decode the motion vectors of the three representative pixels a, b, and c corresponding to the three vertices a, b, and c, respective prediction vectors and difference vectors are calculated. In the DMV2V mode, in order to encode and decode the motion vectors of the two representative pixels a and c corresponding to the two vertices a and c, a prediction vector and a difference vector are calculated, respectively. In the DMV2H mode, in order to encode and decode motion vectors of two representative pixels a and b corresponding to two vertices a and b, respective prediction vectors and difference vectors are calculated. In the DMV2A mode, in order to encode and decode motion vectors of two representative pixels a and d corresponding to two vertices a and d, a prediction vector and a difference vector are calculated, respectively. In the DMV2B mode, in order to encode and decode the motion vectors of the two representative pixels b and c corresponding to the two vertices b and c, a prediction vector and a difference vector are calculated, respectively.

  In the DMV1A mode, only the prediction vector and the difference vector of the representative pixel a are calculated in order to encode / decode the motion vector of the representative pixel a corresponding to the vertex a. In the DMV1B mode, only the prediction vector and the difference vector of the representative pixel b are calculated in order to encode and decode the motion vector of the representative pixel b corresponding to the vertex b. In the DMV1C mode, only the prediction vector and the difference vector of the representative pixel c are calculated in order to encode and decode the motion vector of the representative pixel c corresponding to the vertex c. In the DMV1D mode, in order to encode and decode the motion vector of the representative pixel d corresponding to the vertex d, only the prediction vector and the difference vector of the representative pixel d are calculated. In the DMV0 mode, since no motion vector is encoded / decoded, a difference vector is not calculated.

  The difference vector calculation unit 114 holds the prediction method information and motion vector of the target block supplied from the prediction method determination unit 104, and uses the prediction vector information and the motion vector of the subsequent target block.

  FIGS. 7A to 7D are diagrams for explaining a method of predicting a motion vector of an encoding target block when motion compensated prediction based on parallel movement is selected for both the encoding target block and adjacent blocks. FIG. 7A shows an example of predicting a motion vector between macroblocks in which no partition is set. FIG. 7B shows an example in which motion vectors are predicted between macroblocks in which partitions are set. FIG. 7C shows an example in which a motion vector is predicted in a macro block of 8 × 16 pixels. FIG. 7D shows an example in which a motion vector is predicted in a 16 × 8 pixel macroblock. Hereinafter, a motion vector prediction method will be described with reference to FIGS. In this motion vector prediction method, a motion vector of a target block is predicted using a median value of motion vectors of neighboring neighboring blocks.

  In FIGS. 7A to 7D, the motion vector of a block painted in gray is an encoding target. In FIG. 7A, the motion vector of the target block is predicted by using three motion vectors of the block A adjacent to the left of the target block, the block B adjacent to the upper right, and the block C adjacent to the upper right side as candidates. To do. More specifically, a median value is taken for each of the horizontal component and the vertical component from these three motion vectors to obtain a prediction vector.

  In the B picture, a motion vector for L0 prediction and a motion vector for L1 prediction are handled separately. Prediction for the L0 prediction of the target block by performing prediction using three motion vectors for the L0 prediction of the block A on the left of the target block, the block B on the upper right, and the block C on the upper right. Calculate the vector. Similarly, by using the motion vector for the L1 prediction of the block A adjacent to the left of the target block, the block B adjacent to the upper right, and the block C adjacent to the upper right, the prediction of the L1 prediction of the target block is performed. To calculate a prediction vector. Here, when the block B on the upper right and the block C on the upper right side cannot be used together and only the block A can be used, the motion vector of the block A is adopted as the prediction vector. Also, only one of the reference indexes of the block A on the left, the block B on the upper right, and the block C on the upper right is equal in value to the reference index of the current block (that is, the reference picture is equal). In this case, the motion vector of the block is used for prediction.

  As shown in FIG. 7B, when a partition is set in an adjacent macroblock, the motion vector is different for each small block of the macroblock. In that case, in the block adjacent to the left of the target block, the motion vector of the uppermost small block A among the small blocks in contact with the target block is adopted as a candidate. In the upper adjacent block, the leftmost block B among the small blocks in contact with the target block is adopted as a candidate. In the block adjacent to the upper right, the lower left small block C is adopted as a candidate. In accordance with this rule, a prediction vector is calculated in the same manner as in FIG.

  As shown in FIG. 7C, when the block to be encoded is 8 × 16, the left block is not the median value of the motion vectors of the three blocks, the left block is the block A on the left, and the right block is on the upper right side. The motion vector of the adjacent block C is adopted as the prediction vector. In addition, as shown in FIG. 7D, when the block to be encoded is 16 × 8, the upper block is not the median value of the motion vectors of the three blocks, the upper block is the upper adjacent block B, and the lower block is The motion vector of the block A on the left is adopted as the prediction vector.

  Note that the motion vector prediction method shown in FIGS. 7A to 7D is an example, and is not limited thereto. As long as the encoding method and the decoding side prescribe the motion vector prediction method identically, other methods can be used. For example, the position and number of adjacent blocks may be different. Further, an average value may be used instead of the median value of a plurality of motion vectors of adjacent blocks. A predetermined condition or priority order may be provided, and a motion vector of a single adjacent block may be used as it is. Moreover, the adjacent block does not necessarily have to contact the target block. 7A to 7D, an example of predicting a motion vector in units of macroblocks has been described. However, the same processing can be performed when a motion vector in units of sub-macroblocks is predicted.

  Next, a method for predicting a motion vector of an encoding target block when motion compensated prediction by parallel movement is selected in the encoding target block and motion compensated prediction by geometric transformation is selected in an adjacent block will be described. When the adjacent block is motion compensated prediction by geometric transformation, in the block adjacent to the left of the target block, the motion vector of the second representative pixel b at the upper right of the uppermost small block among the small blocks in contact with the target block is calculated. It is adopted as a candidate when calculating a prediction vector. In the upper adjacent block, the motion vector of the lower left third representative pixel c of the leftmost block among the small blocks in contact with the target block is adopted as a candidate for calculating the prediction vector. In the block on the upper right side, the motion vector of the lower left third representative pixel c of the lower left small block is adopted as a candidate for calculating a prediction vector. In accordance with this rule, the prediction vector is calculated in the same manner as in FIG.

  Note that this motion vector prediction method is an example, and is not limited thereto. As long as the encoding method and the decoding side prescribe the motion vector prediction method identically, other methods can be used. For example, an average value of four representative pixels of the block adjacent to the left of the target block may be used as a motion vector candidate adjacent to the left of the target block. Further, an average value of four representative pixels of the block adjacent to the target block may be used as a motion vector candidate above the target block. Alternatively, the average value of the four representative pixels of the block on the upper right side of the target block may be used as a motion vector candidate on the upper right side of the target block. Furthermore, the positions and the number of pixels of adjacent blocks and adjacent blocks may be different. Further, an average value may be used instead of the median value of a plurality of motion vectors of pixels in adjacent blocks. You may use the motion vector of the pixel of a single adjacent block as it is. Further, the adjacent block or the pixel of the adjacent block may not necessarily be in contact with the target block. Although an example of predicting a motion vector in units of macroblocks has been described here, the same processing can be performed when a motion vector in units of sub-macroblocks is predicted.

  Next, a method for predicting a motion vector of an encoding target block when motion compensated prediction by geometric transformation is selected for both the encoding target block and adjacent blocks will be described. In this case as well, the prediction vector is calculated from the motion vector of the adjacent block. However, the difference vector between the second representative pixel b and the fourth representative pixel d in the DMV2V mode, the difference vector between the third representative pixel c and the fourth representative pixel d in the DMV2H mode, the second representative pixel b and the second representative pixel b in the DMV2A mode. The difference vector of the three representative pixels c and the difference vector of the first representative pixel a and the fourth representative pixel d in the DMV2B mode are calculated from the motion vector values of the target block by the method described above.

  The prediction vectors of the first representative pixel a, the second representative pixel b, and the third representative pixel c of the encoding target block excluding these modes are predicted from the motion vectors of the representative pixels of the adjacent blocks. The prediction vector of the first representative pixel a of the encoding target block is the motion vector of the second representative pixel b of the left adjacent block A, the motion vector of the third representative pixel c of the upper adjacent block B, and the upper left diagonal. The motion vector of the fourth representative pixel d of the block D is calculated with reference to a prediction vector candidate. When calculating the prediction vector, a rule is set in advance and selected according to the rule. For example, the motion vector of the second representative pixel b of the block A adjacent to the left, the motion vector of the third representative pixel c of the upper adjacent block B, and the motion vector of the fourth representative pixel d of the block D adjacent upper left obliquely Priorities are set, and motion vectors with the same reference image are selected as prediction vectors. When there are a plurality of motion vectors with the same reference image, the motion vector with the priorities is selected as the prediction vector. When the same motion vector does not exist in the reference image, scaling described later is performed.

  The prediction vector of the second representative pixel b of the encoding target block is the motion vector of the fourth representative pixel d of the upper adjacent block B, the motion vector of the third representative pixel c of the upper right block C, the target block The motion vector of the first representative pixel a is calculated with reference to a prediction vector candidate. Also in this case, when calculating the prediction vector, a rule is set in advance and selected according to the rule. For example, the motion vector of the fourth representative pixel d of the upper adjacent block B, the motion vector of the third representative pixel c of the upper right block C, and the motion vector of the motion vector of the first representative pixel a of the target block Priorities are set in order, and motion vectors with the same reference image are selected as prediction vectors. When there are a plurality of motion vectors with the same reference image, the motion vector with the priorities is selected as the prediction vector.

  The prediction vector of the third representative pixel c of the encoding target block refers to the motion vector of the fourth representative pixel d of the block A adjacent to the left and the motion vector of the first representative pixel a of the target block as prediction vector candidates. To calculate. Also in this case, when calculating the prediction vector, a rule is set in advance and selected according to the rule. For example, the priority order of the motion vector of the fourth representative pixel d of the upper neighboring block B, the motion vector of the third representative pixel c of the upper right block C, and the motion vector of the first representative pixel a of the target block. And a motion vector having the same reference image is selected as a prediction vector. When there are a plurality of motion vectors with the same reference image, the motion vector with the priorities is selected as the prediction vector. A flag for specifying a motion vector to be adopted as a prediction vector from a plurality of candidates may be prepared in the syntax. In this case, a candidate that selects the smallest code amount of the difference vector is selected for the representative point that encodes the difference vector, and a candidate that can obtain the best geometric transformation motion compensated image for the representative point that does not encode the difference vector. Is selected, and the flag that identifies the candidate selected for each representative point by the first encoded bit string generation unit 107 is encoded.

  Next, a motion vector prediction method for a coding target block when motion compensated prediction by geometric transformation is selected in the encoding target block and motion compensated prediction by parallel movement is selected in the adjacent block will be described. In the case of motion compensated prediction based on parallel movement of an adjacent block, the motion vector of the adjacent block is common to all pixels in the adjacent block. Therefore, the motion vector of the adjacent block is set as the motion vector of the representative pixel of the adjacent block, and the motion vector prediction of the encoding target block is selected when motion compensation prediction by geometric transformation is selected for both the encoding target block and the adjacent block. It is calculated by the same method as the method.

  Note that the motion vector prediction method when motion compensated prediction by geometric transformation is selected is an example, and is not limited thereto. As long as the encoding method and the decoding side prescribe the motion vector prediction method identically, other methods can be used. For example, the position and number of representative pixels in adjacent blocks may be different. Further, an average value may be used instead of the median value of a plurality of motion vectors of representative pixels of adjacent blocks. You may use the motion vector of the representative pixel of a single adjacent block as it is.

In addition, the prediction vector PVd of the fourth representative pixel d excluding the DMV2V mode, DMV2H mode, DMV2A mode, and DMV2B mode described above is the motion vector of each of the first representative pixel a, the second representative pixel b, and the third representative pixel c. It calculates from Va, Vb, Vc by the following formula (9).
PVd = Vc + (Vb−Va) (9)
Or it calculates by following formula (10).
PVd = Vb + (Vc−Va) Formula (10)

  The above formulas (9) and (10) that are the calculation formulas of the prediction vector of the fourth representative pixel d are the above formulas (1) and (10) that are the calculation formulas of the fourth representative pixel d calculated in the DMV3 mode. Each is the same formula as (2). Although transformations that cannot be expressed in the DMV2V mode, DMV2H mode, and DMV3 mode can be expressed, the conversion is often close to affine conversion that can be expressed in the DMV3 mode. The prediction vector that can be calculated by the above equations (9) and (10) is based on the idea that there is a strong correlation with the motion vector of the fourth representative pixel d that is encoded / decoded in the DMV4 mode.

  It should be noted that other methods can be used as long as the motion vector prediction method is defined identically on the encoding side and the decoding side. For example, for the fourth representative pixel d, the median value of the motion vectors of the first representative pixel a, the second representative pixel b, and the third representative pixel c can be used as the prediction vector. Alternatively, an average value may be used instead of the median value, or a motion vector of a single arbitrary pixel may be used as it is. Alternatively, the prediction vector calculated by Expression (9), the motion vector of the first representative pixel a, the motion vector of the second representative pixel b, the motion vector of the third representative pixel c, the first representative pixel a, and the second representative pixel The median of the motion vectors of b and the third representative pixel c, the average value of the motion vectors of the first representative pixel a, the second representative pixel b, and the third representative pixel c are used as prediction vector candidates, and at least two are adaptive. You may switch to. In this case, a flag for specifying a motion vector to be adopted as a prediction vector from a plurality of candidates is prepared in the syntax, and the first encoded bit string generation unit 107 encodes this flag.

  In the DMV2A mode, the motion vector Va of the first representative pixel a of the target block is set as the prediction vector PVd of the fourth representative pixel d. In the DMV2A mode, the motion vector Vd of the fourth representative pixel d is used when calculating the motion vectors Vb and Vc of the second representative pixel b and the third representative pixel c. This is because the motion vectors Vb and Vc of the second representative pixel b and the third representative pixel c cannot be used when calculating.

In the motion vector prediction method described above, the motion vector value Vabcd ′ that has been scaled may be used for calculation of the prediction vector, instead of using the motion vector of the adjacent block or the pixel of the adjacent block as it is as a candidate. The scaled motion vector value Vabcd ′ includes the distance (time) T1 between the encoding target image and the reference image indicated by the motion vector of the encoding target block, and the motion vector Vabcd of the encoding target image and the adjacent block or adjacent pixel. Is a motion vector value scaled according to the difference from the distance (time) T2 from the reference image indicated by. The scaled motion vector value Vabcd ′ is calculated by the following equation (11).
Vabcd ′ = Vabcd * (T1 / T2) (11)

  If the reference image referenced in the motion-compensated prediction of the current block is different from the reference image referenced in the motion-compensated prediction of the adjacent block, there will be a difference in the size of each other's motion vector. Scale the motion vector. For example, if the object is not deformed and moves at a constant speed, the magnitude of the motion vector increases as the frame interval increases. Therefore, Vabcd 'is calculated by scaling the motion vector Vabcd of the adjacent block (adjacent pixel) according to the ratio of the frame intervals T1 and T2 between the encoding target image and the reference image.

  FIG. 8 is a diagram for explaining an example of motion vector value scaling processing. FIG. 8 illustrates an example in which the reference image of the encoding target block of the encoding target image is the image Ref2, and the reference image for motion compensation prediction of the adjacent block (adjacent pixel) is the image Ref3. In FIG. 8, since T1: T2 = 2: 3, the motion vector of the adjacent block (adjacent pixel) referring to the image Ref3 is scaled to 2/3. As a result, the adjacent block (adjacent pixel) is close to the motion vector value when motion compensation prediction is performed with reference to the image Ref2, and as a result, close to the motion vector value of the encoding target block that refers to the image Ref2. In the example of FIG. 8, when the participating image in the motion compensation prediction of the adjacent block is the image Ref3 and the value of the motion vector is (24, −9), it is scaled to 2/3 (16, 6). The motion vector value of the image Ref2 referred to by the encoding target block may be used.

  As long as the prediction method of the motion vector is the same on the encoding side and the decoding side, the motion vector of the first representative pixel a is the target when calculating the prediction vector of the second representative pixel b of the target block. It is good also as a prediction vector of the 2nd representative pixel b of a block. Further, the motion vector of the fourth representative pixel d of the upper adjacent block B is set as the prediction vector of the second representative pixel b of the target block, or the motion vector of the first representative pixel a is set as the second representative pixel b of the target block. Whether the prediction vector is used may be adaptively switched.

  In the mode for encoding the difference vector of the first representative pixel a of the target block, when the width of the target block and the upper adjacent block B are the same, switching is performed according to the value of the motion vector of the first representative pixel a of the target block. You can also. The value of the motion vector of the first representative pixel a of the target block is the same as the value of the motion vector of the fourth representative pixel d of the upper adjacent block B scaled to correct the difference in distance from the reference image. If both values are less than a predetermined first set value (set to an extremely small value), the scaled motion vector of the fourth representative pixel d of the upper adjacent block B is adopted as a prediction vector, When the value is more than the first set value, the motion vector of the first representative pixel a of the target block is adopted as the prediction vector. If the value of the motion vector of the first representative pixel a of the target block is the same as or very close to the value of the scaled motion vector of the fourth representative pixel d of the upper adjacent block B, the difference vector of the second representative pixel b In the mode for encoding / decoding the motion vector, the value of the motion vector of the second representative pixel b of the target block is the same as or very close to the value of the scaled motion vector of the fourth representative pixel d of the upper adjacent block B. The code amount of the difference vector can be reduced. In the mode in which the difference vector of the second representative pixel b is not encoded / decoded, the motion vector value of the fourth representative pixel d of the upper adjacent block B is scaled more than the motion vector of the first representative pixel a of the target block. This is because the probability that a better geometric transformation motion compensated image can be obtained is higher.

  On the other hand, if the value of the motion vector of the first representative pixel a of the target block is relatively far from the value of the scaled motion vector of the fourth representative pixel d of the adjacent block B, the second representative pixel b In the mode for encoding / decoding the difference vector, the probability that the value of the motion vector of the second representative pixel b of the target block is relatively distant from the value of the motion vector of the fourth representative pixel d of the scaled upper neighboring block B In this case, the sign of the difference vector is greater when the motion vector value of the first representative pixel a of the target block is used as the prediction vector rather than the scaled motion vector of the fourth representative pixel d of the upper adjacent block B. It is likely that the amount can be reduced. In the mode in which the difference vector of the second representative pixel b is not encoded / decoded, the motion vector value of the first representative pixel a of the target block is set to be larger than the scaled motion vector of the fourth representative pixel d of the adjacent block B. The probability that a better geometric transformation motion compensation image can be obtained by using a prediction vector is higher. The first set value is determined based on an experiment or simulation by a designer.

  Further, when calculating the prediction vector of the third representative pixel c of the target block, the motion vector of the first representative pixel a may be used as the prediction vector of the third representative pixel c of the target block. Further, as described above, the motion vector of the fourth representative pixel d of the block A adjacent to the left is used as the prediction vector of the third representative pixel c of the target block, or the motion vector of the first representative pixel a of the target block is the target block. When adaptively switching whether to use the prediction vector of the third representative pixel c of

  In the mode for encoding the difference vector of the first representative pixel a of the target block, if the height of the target block and the block A adjacent to the left is the same, depending on the value of the motion vector of the first representative pixel a of the target block You can also switch. The value of the motion vector of the first representative pixel a of the target block is the same as the value of the motion vector of the fourth representative pixel d of the block A adjacent to the left that has been scaled to correct the difference in distance from the reference image. If both values are less than a predetermined second set value (set to a very small value), the scaled motion vector of the fourth representative pixel d of the block A on the left adjacent block A is adopted as a prediction vector, When the value is more than the second set value, the motion vector of the first representative pixel a of the target block is adopted as the prediction vector. If the value of the motion vector of the first representative pixel a of the target block is the same as or very close to the value of the motion vector of the fourth representative pixel d of the scaled left adjacent block A, the difference vector of the third representative pixel c In the mode for encoding / decoding the motion vector, the value of the motion vector of the third representative pixel c of the target block is the same as or very close to the value of the motion vector of the fourth representative pixel d of the scaled left neighboring block A. The code amount of the difference vector can be reduced. In the mode in which the difference vector of the third representative pixel c is not encoded / decoded, the motion vector value of the fourth representative pixel d of the block A adjacent to the left is scaled more than the motion vector of the first representative pixel a of the target block. This is because the probability that a better geometric transformation motion compensated image can be obtained is higher.

  On the other hand, when the value of the motion vector of the first representative pixel a of the target block is relatively far from the value of the motion vector of the fourth representative pixel d of the scaled left adjacent block A, the third representative pixel c In the mode of encoding / decoding the difference vector, the probability that the value of the motion vector of the third representative pixel c of the target block is relatively separated from the value of the motion vector of the fourth representative pixel d of the block A on the left adjacent scale In this case, the sign of the difference vector is greater when the motion vector value of the first representative pixel a of the target block is used as a prediction vector rather than the motion vector of the fourth representative pixel d of the scaled left adjacent block A. It is likely that the amount can be reduced. In the mode in which the difference vector of the third representative pixel c is not encoded / decoded, the value of the motion vector of the first representative pixel a of the target block is set to be larger than the motion vector of the fourth representative pixel d of the scaled left adjacent block A. The probability that a better geometric transformation motion compensation image can be obtained by using a prediction vector is higher.

  As described above, the case where the motion compensated prediction based on the geometric transformation is selected for the adjacent block has been described, but when the motion compensated prediction based on the parallel movement is selected for the adjacent block, the adjacent block is replaced with the motion vector of the representative pixel. The motion vector of the block is used as a prediction vector or used for evaluation. Specifically, instead of the motion vector of the fourth representative pixel d of the upper adjacent block B to be compared with the motion vector of the first representative pixel a of the target block, the motion vector of the upper adjacent block B is used and the left adjacent Instead of the motion vector of the fourth representative pixel d of the block A, the motion vector of the block A on the left is used. Further, instead of the motion vector of the fourth representative pixel d of the upper neighboring block B referred to when calculating the prediction vector or the motion vector of the second representative pixel b of the target block, the motion vector of the upper neighboring block B is used. Use. In addition, instead of the motion vector of the fourth representative pixel d of the block A on the left adjacent to which the prediction vector or motion vector of the third representative pixel c of the target block is calculated, the motion vector of the block A on the left adjacent to the block A Use.

  The prediction vector of the motion compensation prediction by geometric transformation has been described above. However, the second representative pixel b and the fourth representative pixel d of DMV2V, the third representative pixel c and the fourth representative pixel d of DMV2H, and the second representative pixel of DMV2A Except for the pixel b, the third representative pixel c, and the first representative pixel a and the fourth representative pixel d of the DMV 2B, the calculation is performed in the same manner as in the DMV4 mode. As long as the rules common to the decoding side are defined, the second representative pixel b and the fourth representative pixel d of DMV2V, the third representative pixel c and the fourth representative pixel d of DMV2H, and the second representative pixel b of DMV2A The third representative pixel c and the first representative pixel a and the fourth representative pixel d of the DMV 2B can be performed by the same calculation method as in the DMV4 mode.

  Returning to FIG. 1, the second encoded bit string generation unit 108 encodes the difference vector supplied from the difference vector calculation unit 114 by entropy encoding such as arithmetic encoding, and generates an encoded bit string.

  In motion compensation prediction by geometric transformation, 0 to 4 difference vectors are encoded according to the prediction mode. In the DMV4 mode, the difference vectors of the four representative pixels a, b, c, and d corresponding to the four vertices a, b, c, and d are encoded. In the DMV3 mode, the difference vectors of the three representative pixels a, b, and c corresponding to the three vertices a, b, and c are encoded. In the DMV2V mode, the difference vectors of the representative pixels a and c corresponding to the two vertices a and c are encoded. In the DMV2H mode, the difference vectors of the representative pixels a and b corresponding to the two vertices a and b are encoded. In the DMV2A mode, the difference vectors of the representative pixels a and d corresponding to the two vertices a and d are encoded. In the DMV2B mode, the difference vectors of the representative pixels b and c corresponding to the two vertices b and c are encoded. In the DMV1A mode, only the difference vector of the representative pixel a corresponding to the vertex a is encoded. In the DMV1B mode, only the difference vector of the representative pixel b corresponding to the vertex b is encoded. In the DMV1C mode, only the difference vector of the representative pixel c corresponding to the vertex c is encoded. In the DMV1D mode, only the difference vector of the representative pixel d corresponding to the vertex d is encoded. In the DMV0 mode, the difference vector of any representative pixel is not encoded.

  The prediction error signal generation unit 105 calculates a difference between the prediction signal generated by the prediction method adopted by the prediction method determination unit 104 and the image signal of the target block, and generates a prediction error signal. More specifically, the prediction error signal generation unit 105 subtracts the prediction signal supplied from the prediction method determination unit 104 from the image signal to be encoded supplied from the image buffer 101 to obtain a prediction error signal. Generated and supplied to the prediction error signal encoding unit 106.

  The prediction error signal encoding unit 106 performs compression encoding processing such as orthogonal transform and quantization on the prediction error signal supplied from the prediction error signal generation unit 105 to generate an encoded prediction error signal. The third encoded bit string generator 109 and the prediction error signal decoder 110 are supplied.

  The third encoded bit sequence generation unit 109 sequentially encodes the encoded prediction error signal supplied from the prediction error signal encoding unit 106 using entropy encoding such as arithmetic encoding to generate an encoded bit sequence To do.

  The encoded bit strings generated by the first encoded bit string generation unit 107, the second encoded bit string generation unit 108, and the third encoded bit string generation unit 109 are encoded by information other than prediction method information, motion vectors, and prediction error signals. The other encoded bit strings are multiplexed via the output switch 113 to generate an encoded stream.

  The prediction error signal decoding unit 110 performs decompression decoding processing such as inverse quantization and inverse orthogonal transform on the prediction error signal encoded by the prediction error signal encoding unit 106, and decodes the prediction error signal. The prediction error signal decoding unit 110 supplies the decoded prediction error signal to the decoded image signal generation unit 111. The decoded image signal generation unit 111 superimposes the prediction error signal supplied from the prediction error signal encoding unit 106 and the prediction signal supplied from the prediction method determination unit 104 to generate a decoded image signal. The decoded image signal generation unit 111 sequentially stores the decoded image signal in the decoded image buffer 112 in units of blocks. The decoded image stored in the decoded image buffer 112 is used as a reference image when performing motion compensation prediction on the subsequent image in the encoding order, if necessary.

  Hereinafter, as an example of the detection method of the geometric transformation motion vector detection unit 121 illustrated in FIG. 5 described above, the motion of the translation such as the motion vector of the macroblock or the macroblock / partition generated by the translation motion compensation prediction unit 102 is described below. A method for correcting a vector and detecting a motion vector of a representative pixel in geometric transformation will be described. FIG. 9 is a block diagram showing a configuration of the geometric transformation motion vector detection unit 121 according to Embodiment 1 of the present invention. The geometric transformation motion vector detection unit 121 includes a representative point motion vector setting unit 131, a second non-representative point motion vector calculation unit 132, a second prediction unit 133, a block matching unit 134, and an evaluation unit 135.

  When correcting the motion vector of the parallel movement of the macroblock and using it as the motion vector of the representative pixel of the geometric transformation, the geometric transformation motion vector detection unit 121 detects the motion of the macroblock generated by the translational motion compensation prediction unit 102. The vector value is set to the initial value of the motion vector of each representative pixel. Then, the motion vector values of those representative pixels are changed in the increasing direction and the decreasing direction, and block matching is performed by performing motion compensation while deforming the reference image by the same processing as the above-described geometric transformation motion compensation prediction. Thereby, the motion vector of the representative pixel of the geometric transformation is calculated.

  FIGS. 10A to 10H are diagrams illustrating specific examples of a method of detecting a motion vector of a representative pixel in geometric transformation by correcting a translation motion vector generated by the translation motion compensation prediction unit 102. FIG. It is. First, consider the case of DMV4 mode. The representative point motion vector setting unit 131 fixes the motion vectors of the representative points b, c, and d, and changes the motion vector of the representative point a in the horizontal and vertical directions within a predetermined motion vector detection range. The motion vector detection range may be, for example, a 7 × 7 pixel square region centered on the representative point a. The second non-representative point motion vector calculation unit 132 uses the same method as the first non-representative point motion vector calculation unit 122 every time the motion vector of the representative point a is changed. A motion vector is calculated. The second prediction unit 133 predicts the reference image while deforming it by interpolating the pixels of the reference image according to the motion vector of each pixel in the same manner as the first prediction unit 123. The block matching unit 134 performs block matching between the encoded image signal (more specifically, the target block signal) and the prediction signal, and calculates matching evaluation values such as a difference absolute value sum and a difference square sum. The evaluation unit 135 compares the matching evaluation value calculated this time with the matching evaluation value of the motion vector candidate of the representative point a (the minimum value of the matching evaluation values calculated so far), and the matching evaluation value is smaller As a candidate for a new representative point a. This series of processes is repeated to calculate a motion vector candidate for the representative point a (see FIG. 10A).

  The representative point motion vector setting unit 131 updates the motion vector of the representative point a, then fixes the motion vectors of the representative points a, c, and d, and moves the motion vector of the representative point b within a predetermined motion vector detection range. To change. The second non-representative point motion vector calculation unit 132 calculates the motion vector of each pixel other than the representative point every time the motion vector of the representative point b is changed. The second prediction unit 133 performs prediction while deforming the reference image according to the motion vector of each pixel. The block matching unit 134 performs block matching between the encoded image signal and the prediction signal (see FIG. 10B). For the representative points c and d, the motion vectors of the representative points c and d are detected by the same method (see FIGS. 10C and 10D). Further, if necessary, the motion vectors of the representative points a, b, c, and d are detected again by the same method (see FIGS. 10E to 10H). The detection of the motion vectors of the representative points a, b, c, and d is set as one cycle, and when the cycle is executed a predetermined number of times, the detection process is terminated. 10A to 10H, two cycles are executed. Alternatively, when each of the motion vectors of the representative points a, b, c, and d converges (that is, a state in which the motion vector is not changed), the detection process is terminated.

  In the above-described example, the motion vector of the macro block is used as the initial value of motion vector detection. However, the motion vector of the macro block partition or the sub macro block may be used. Also, a newly calculated motion vector may be used. When detecting the motion vector of the representative pixel of the geometric transformation by correcting the motion vector of the parallel movement of the macroblock / partition, the value of the motion vector of the macroblock / partition is the initial value of the motion vector of the corresponding representative pixel. The value of the motion vector of the representative pixels a, b, c, d is calculated by the above-described method.

  Next, modes other than the DMV4 mode will be considered. In modes other than the DMV4 mode, when detecting a motion vector of a representative pixel that encodes / decodes a difference vector, the motion vector of a representative pixel that does not encode / decode the difference vector corresponds to a prediction vector calculation rule of each prediction mode. Thus, the difference vector is moved in conjunction with the motion vector of the representative pixel that encodes and decodes the difference vector.

  In the DMV3 mode, the motion vectors of the representative pixels a, b, and c for encoding / decoding the difference vector are sequentially changed. At this time, the motion vector of the representative pixel d is also expressed by the above equation (1) or (2). Change according to the calculation. For example, when detecting while moving the motion vector of the representative pixel a, the motion vectors of the representative pixels b and c are fixed, but the motion vector of the representative pixel d that does not encode / decode the difference vector is expressed by the above equation (1). Or, since it is calculated according to (2) above, it is changed as needed according to the motion vector of the representative pixel a.

  In the DMV2V mode, the motion vectors of the representative pixels a and c for encoding / decoding the difference vector are sequentially changed. When the motion vector of the representative pixel a is detected while moving, the motion vectors of the representative pixels c and d are detected. Is fixed, but the motion vector of the representative pixel b that does not encode / decode the difference vector is changed to the same value as the motion vector of the representative pixel a as needed. When detection is performed while moving the motion vector of the representative pixel c, the motion vectors of the representative pixels a and b are fixed, but the motion vector of the representative pixel d that does not encode / decode the difference vector is the motion vector of the representative pixel b. Change to the same value from time to time.

  In the DMV2H mode, the motion vectors of the representative pixels a and b for encoding / decoding the difference vector are sequentially changed. When the motion vector of the representative pixel a is detected while moving, the motion vectors of the representative pixels b and d are detected. Is fixed, but the motion vector of the representative pixel c that does not encode / decode the difference vector is changed to the same value as the motion vector of the representative pixel a as needed. When detection is performed while moving the motion vector of the representative pixel c, the motion vectors of the representative pixels a and b are fixed, but the motion vector of the representative pixel d that does not encode / decode the difference vector is the motion vector of the representative pixel b. Change to the same value from time to time.

  In the DMV2A mode, DMV2B mode, DMV1A mode, DMV1B mode, DMV1C mode, and DMV1D mode, the motion vector of the representative pixel can be detected according to the above-described principle or the principle on the extension line. Note that those detection methods can be easily understood by those skilled in the art from the above description, and thus detailed description thereof is omitted. In the DMV0 mode in which no motion vector is encoded, a prediction vector predicted from a neighboring block or a target block is used as a motion vector.

  As described above, the prediction vector can be calculated from the decoded motion vectors of the surrounding blocks. It should be noted that if there are no motion vectors of neighboring blocks, they are not entered as candidates. When the peripheral block is in the parallel motion compensation mode, the motion vector value of the representative pixel a is set as the motion vector value of the representative pixel a of the peripheral block. In the representative pixel b of the peripheral block, the value of the motion vector of the block on the left is used as the value of the motion vector of the representative pixel b. In the representative pixel c of the peripheral block, the value of the motion vector of the upper adjacent block is set as the value of the motion vector of the representative pixel b. In the representative pixel d of the peripheral block, the value of the motion vector of the block on the upper left is set as the value of the motion vector of the representative pixel b.

  FIG. 11 is a flowchart showing a macroblock encoding processing procedure in image encoding apparatus 100 according to Embodiment 1 of the present invention. First, the translational motion compensation prediction unit 102 and the geometric transformation motion compensation prediction unit 103 extract a macroblock signal to be encoded from the image buffer 101 (S101).

  The translational motion compensation prediction unit 102 performs motion compensation prediction by translation between the macroblock signal to be encoded supplied from the image buffer 101 and the reference image signal supplied from the decoded image buffer 112 ( S102). The motion compensation prediction based on the parallel movement is performed for each mode. The geometric transformation motion compensation prediction unit 103 performs motion compensation prediction by geometric transformation between the encoding target macroblock signal supplied from the image buffer 101 and the reference image signal supplied from the decoded image buffer 112. Perform (S103). Motion compensation prediction by this geometric transformation is performed for each mode.

  The prediction method determination unit 104 determines which one of the motion compensation prediction method based on the parallel movement and the motion compensation prediction based on the geometric transformation is adopted (S104). At that time, it is also decided which mode is adopted.

  The first encoded bit string generation unit 107 encodes the prediction method information supplied from the prediction method determination unit 104, and generates an encoded bit string (S105). The difference vector calculation unit 114 calculates a prediction vector from the representative pixel, the peripheral block, or the peripheral pixel of the target block (S106), and calculates a difference vector between the prediction vector and the motion vector supplied from the prediction method determination unit 104. (S107).

  The second encoded bit string generation unit 108 encodes the difference vector supplied from the difference vector calculation unit 114 and generates an encoded bit string (S108). The prediction error signal generation unit 105 subtracts the prediction signal supplied from the prediction method determination unit 104 from the target image signal supplied from the image buffer 101 to generate a prediction error signal (S109). The prediction error signal encoding unit 106 encodes the prediction error signal (S110). The third encoded bit string generation unit 109 entropy-encodes the prediction error signal encoded by the prediction error signal encoding unit 106 using arithmetic encoding or the like to generate an encoded bit string (S111).

  The prediction error signal decoding unit 110 decodes the prediction error signal encoded by the prediction error signal encoding unit 106 (S112). The decoded image signal generation unit 111 generates a decoded image signal by superimposing the prediction error signal decoded by the prediction error signal decoding unit 110 and the prediction signal supplied from the prediction method determination unit 104 (S113). The decoded image signal generation unit 111 accumulates the generated decoded image signal in the decoded image buffer 112 (S114).

  FIG. 12 is a flowchart showing the procedure of the translational motion compensation process in step S102 of the flowchart of FIG. The translational motion compensation prediction unit 102 repeatedly executes the processing of step S1022 and step S1023 shown below until all candidate candidates (modes, reference images, etc.) for translational motion compensation are completed (S1021, S1024). The translational motion compensation prediction unit 102 detects a translational motion vector (S1022), and performs motion compensation by translation of the target block according to the detected motion vector (S1023).

  FIG. 13 is a flowchart showing the procedure of the geometric transformation motion compensation process in step S103 of the flowchart of FIG. The geometric transformation motion compensation prediction unit 103 repeatedly executes the processing of steps S1032 and S1033 shown below until all candidate objects (modes, reference images, etc.) of the geometric transformation motion compensation are completed (S1031, S1034). ). The geometric transformation motion compensation prediction unit 103 detects a motion vector of the geometric transformation (S1032), and performs motion compensation by geometric transformation of the target block according to the detected motion vector (S1033).

  FIG. 14 is a flowchart showing a procedure for detecting a translational motion vector. The translational motion compensation / prediction unit 102 repeatedly executes the processes of steps S122 and S127 described below until the full search within the predetermined detection range is completed (S121, S128). The translational motion compensation prediction unit 102 generates a temporary motion vector value (S122), and performs motion compensation by translation according to the generated temporary motion vector value (S123). The value of the temporary motion vector is sequentially updated in order to search the entire detection range.

  The translational motion compensation prediction unit 102 performs block matching between the target block after the motion compensation and the reference block, and calculates an evaluation value (S124). The currently calculated evaluation value is compared with the minimum evaluation value calculated so far (S125). If the former is greater than or equal to the latter (N in S125), the following steps S126 and S127 are skipped. If the former is less than the latter (Y in S125), the current temporary motion vector is set as the current candidate (S126), and the current evaluation value is set as the minimum evaluation value (S127).

  FIG. 15 is a flowchart showing a procedure for detecting a motion vector of geometric transformation. The flowchart shown in FIG. 15 is a flowchart showing a procedure for detecting a motion vector in a geometric transformation mode in which 1 to 4 difference vectors are encoded.

  The geometric transformation motion compensation prediction unit 103 acquires the translation motion vector of the target block generated by the translation motion compensation prediction unit 102 (S131), and sets it as the initial value of the motion vector of each representative point ( S132). The processes from step S134 to step S141 shown below are repeated a predetermined number of times (S133, S142). Further, the following processing from step S135 to step S140 is repeatedly executed until the full search within the predetermined detection range is completed (S134, S141).

  The geometric transformation motion compensation prediction unit 103 moves the value of the temporary motion vector one by one among the representative points for encoding the difference vector (S135). If necessary, the motion vector of the representative point that does not encode the difference vector is also moved in conjunction. According to a predetermined rule of each mode, a prediction vector of a representative point that does not encode a difference vector is calculated and used as a temporary motion vector. In accordance with the values of these temporary motion vectors, motion compensation is performed by geometric transformation of the target block (S136). The geometric transformation motion compensation prediction unit 103 performs block matching between the target block after the motion compensation and the reference block, and calculates the evaluation value (S137). The currently calculated evaluation value is compared with the minimum evaluation value calculated so far (S138). If the former is greater than or equal to the latter (N in S138), the following steps S139 and S140 are skipped. When the former is less than the latter (Y in S138), the current temporary motion vector is set as the current candidate (S139), and the current evaluation value is set as the minimum evaluation value (S140).

  FIG. 16 is a flowchart showing the procedure of the geometric transformation motion compensation process in step S136 of the flowchart of FIG. The representative point motion vector setting unit 131 sets a motion vector of each representative point of the target block (S1361). The second non-representative point motion vector calculation unit 132 calculates a motion vector of each pixel of the target block (S1362). The second prediction unit 133 acquires an interpolation signal corresponding to the motion vector for each pixel (S1363).

  As described above, according to the first embodiment, it is possible to improve the compression efficiency of the code amount by the image coding method using the motion compensated prediction based on the geometric transformation. That is, the amount of codes can be reduced by predictively encoding motion vectors that use motion compensated prediction by geometric transformation. Also, if the motion compensation prediction based on the parallel movement and the motion compensation prediction based on the geometric transformation are used in combination, the compression efficiency of the code amount can be further improved. At that time, by sharing a motion vector related to motion compensation prediction based on parallel movement and a motion vector encoding method related to motion compensation prediction based on geometric transformation, even if these two prediction methods are mixed, The motion vector predictive coding method can be used as it is.

  Further, even in a block in which a motion compensation prediction method by geometric transformation is adopted, a prediction vector is predicted from a motion vector of a representative pixel of a target block, a surrounding block, or a surrounding pixel, similarly to a motion compensation prediction by translation, A difference vector can be calculated. Thereby, even if the motion compensation prediction by parallel movement and the motion compensation prediction by geometric transformation are used together, it is possible to suppress an increase in the code amount of the motion vector. In the DMV4 mode, the prediction vector of the fourth representative pixel d is calculated from the motion vectors of the first representative pixel a, the second representative pixel b, and the third representative pixel c by the above formula (9) or the above formula (10). Thus, the value of the difference vector can be reduced, and an increase in the code amount of the motion vector can be suppressed. In the DMV3 mode, the motion vector of the fourth representative pixel d is calculated from the motion vectors of the first representative pixel a, the second representative pixel b, and the third representative pixel c by the above formula (1) or the above formula (2). By calculating, an increase in the code amount of the motion vector can be suppressed.

  FIG. 17 is a block diagram showing a configuration of image decoding apparatus 200 according to Embodiment 2 of the present invention. The image decoding apparatus 200 decodes the encoded stream generated by the image encoding apparatus 100 according to Embodiment 1. In the encoded stream, as described above, motion compensated prediction based on parallel movement and motion compensated prediction based on geometric transformation may be used together, or motion compensated prediction based on geometric transformation is used alone. (Intra coding is not considered).

  The image decoding apparatus 200 includes an input switch 209, a first encoded bit string decoding unit 201, a second encoded bit string decoding unit 202, a third encoded bit string decoding unit 203, a motion vector calculation unit 215, and a translational motion compensation prediction unit 204. , Geometric transformation motion compensation prediction unit 205, prediction error signal decoding unit 206, decoded image signal generation unit 207, decoded image buffer 208, switching control unit 214, first prediction unit switch 210, second prediction unit switch 211, third A prediction unit switch 212 and a fourth prediction unit switch 213 are provided.

  These configurations can be realized by an arbitrary processor, memory, or other LSI in terms of hardware, and can be realized by a program loaded in the memory in terms of software, but here by their cooperation. Draw functional blocks. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The first encoded bit string decoding unit 201, the second encoded bit string decoding unit 202, and the third encoded bit string decoding unit 203 decode the prediction method information, the difference vector, and the prediction error signal included in the encoded stream. As described above, the difference vector is encoded in the encoded stream. The difference vector is a difference between the motion vector of the target block and a prediction vector predicted from the motion vector of the representative pixel, adjacent block, or adjacent pixel of the target block. The motion vector calculation unit 215 calculates a prediction vector from the representative pixel of the target block, the adjacent block, or the motion vector of the adjacent pixel according to exactly the same rules as those on the encoding side, and adds the decoded difference vector to the prediction vector Thus, the motion vector of the target block or its representative pixel that has been predictively encoded is decoded.

  The translational motion compensation prediction unit 204 obtains a prediction signal from the motion vector between the target block in the target image and the reference block in the reference image that is translated with the target block, and the image signal of the reference block. Generate. The geometric transformation motion compensation prediction unit 205 performs prediction based on the motion vector between the target block in the target image, the reference block in the reference image having a geometric transformation relationship with the target block, and the image signal of the reference block. Generate a signal.

  As described above, the vertex of the target block is selected as the representative pixel, and the encoded stream includes the motion vector of each representative pixel. The geometric transformation motion compensation prediction unit 205 calculates motion vectors of pixels other than the plurality of representative pixels of the target block by interpolation using the motion vectors of the plurality of representative pixels. For example, the motion vectors of pixels other than the representative pixel are calculated by the arithmetic expressions shown in the above formulas (5) to (8).

  According to the prediction method information decoded by the first encoded bit string decoding unit 201, the switching control unit 214 performs the prediction method by the translation motion compensation prediction unit 204, the geometric transformation motion, for each target block in the target image. It is specified which of the prediction methods by the compensation prediction unit 205 is used.

  More specific description will be given below. The first encoded bit string decoding unit 201, the second encoded bit string decoding unit 202, and the third encoded bit string decoding unit 203 are included in the encoded stream generated by the image encoding device 100 according to Embodiment 1. The encoded bit string to be transmitted is selectively input via the input switch 209.

  The first encoded bit string decoding unit 201 decodes the encoded bit string supplied via the input switch 209 by entropy decoding such as arithmetic decoding, and acquires prediction method information. As described above, the prediction method information includes information indicating whether the translation motion compensation or the geometric transformation motion compensation is encoded, and in the case of translation motion compensation, 16 × 16, Information on whether to use 16 × 8, 8 × 16, or 8 × 8 mode is included. When coded by geometric conversion motion compensation, the coding is performed in any of DMV4 mode, DMV3 mode, DMV2V mode, DMV2H mode, DMV2A mode, DMV2B mode, DMV1A mode, DMV1B mode, DMV1C mode, DMV1D mode, and DMV0 mode. It also includes information on whether or not

  The second encoded bit string decoding unit 202 decodes the encoded bit string supplied via the input switch 209 by entropy decoding such as arithmetic decoding, and obtains a difference vector. As described above, the difference vector is the difference between the prediction vector calculated from the representative pixel of the target block, the adjacent block, or the motion vector of the adjacent pixel and the motion vector of the decoding target block. In addition, when the motion vector used in motion compensation prediction by parallel movement and the motion vector used in motion compensation prediction by geometric transformation are mixed and encoded, the difference mixed on the decoding side as well as the encoding side A motion vector is decoded from the vector according to prediction method information.

  Based on the prediction method information decoded by the first encoded bit stream decoding unit 201, which decoding image is used for which method of intra coding, motion compensation prediction by parallel movement, and motion compensation prediction by geometric transformation, and which reference image is used. You can see what block unit is selected and combined.

  The switching control unit 214 performs the first prediction unit switch 210, the second prediction unit switch 211, the third prediction unit switch 212, and the fourth prediction unit according to the prediction method information supplied from the first encoded bit string decoding unit 201. Switch 213 is switched. When the motion compensation prediction method by translation is selected as the prediction method of the target block, switching is performed so that the path of the translation motion compensation prediction unit 204 is selected, and the motion compensation prediction method by geometric transformation is selected. In this case, the path of the geometric transformation motion compensation prediction unit 205 is switched to be selected.

  The motion vector calculation unit 215 performs motion vectors of other representative pixels, adjacent blocks, or adjacent pixels of the target block that have already been encoded / decoded in accordance with the prediction method information supplied from the first encoded bit string decoding unit 201. The prediction vector is calculated by predicting the motion vector of the target block. Then, a motion vector is calculated by adding the difference vector supplied from the second encoded bit string decoding unit 202 to the prediction vector. The motion vector calculation unit 215 supplies the motion vector to the translational motion compensation prediction unit 204 or the geometric transformation motion compensation prediction unit 205 via the second prediction unit switch 211. The calculation of the prediction vector in the motion vector calculation unit 215 is performed by the same method as the calculation of the prediction vector in the difference vector calculation unit 114 of the image encoding device 100. In the case of parallel motion compensation, the motion vector of the target block can be acquired. In the case of geometric conversion motion compensation, the motion vector of the representative pixel of the target block is calculated.

  The translation motion compensated prediction unit 204 includes a decoded image to be a reference image supplied from the decoded image buffer 208 via the fourth prediction unit switch 213, and a second encoded bit string decoding unit 202 to the second prediction unit switch 211. Using the decoded motion vector supplied via, motion compensated prediction by translation is performed.

  The geometric transformation motion compensated prediction unit 205 receives a decoded image to be a reference image supplied from the decoded image buffer 208 via the fourth prediction unit switch 213 and a motion vector calculation unit 215 via the second prediction unit switch 211. The motion vectors of all the pixels are calculated by interpolation using the decoded motion vectors of the representative pixels. In that case, the same processing method as the geometric transformation motion compensation prediction part 103 of the image coding apparatus 100 demonstrated with reference to said Formula (1)-(11) can be used. The geometric transformation motion compensation prediction unit 205 performs motion compensation prediction by geometric transformation by performing motion compensation for each pixel according to the motion vector for each pixel.

  The third encoded bit string decoding unit 203 sequentially decodes the encoded bit string supplied via the input switch 209, and obtains an encoded prediction error signal. The prediction error signal decoding unit 206 performs decompression decoding processing such as inverse quantization and inverse orthogonal transform on the encoded prediction error signal supplied from the third encoded bit string decoding unit 203, and performs the decoded prediction Get the error signal.

  The decoded image signal generation unit 207 generates an image signal from the prediction signal and the prediction error signal. More specifically, the decoded image signal generation unit 207 switches from the translation motion compensation prediction unit 204 or the geometric transformation motion compensation prediction unit 205 to the third prediction unit switch according to the prediction method specified by the switching control unit 214. The prediction error signal supplied from the prediction error signal decoding unit 206 is superimposed on the prediction signal supplied via 212 to generate a decoded image signal. The decoded image signal generation unit 207 sequentially stores the decoded image signal in the decoded image buffer 208 in units of blocks.

  FIG. 18 is a flowchart showing a macroblock decoding process procedure in the image decoding apparatus 200 according to Embodiment 2 of the present invention. The first encoded bit string decoding unit 201 decodes the encoded bit string supplied via the input switch 209, and acquires prediction method information, information indicating a prediction vector calculation method, and the like (S201). It should be noted that a method for calculating a prediction vector for the first representative pixel a, a method for calculating a prediction vector for the second representative pixel b, a method for calculating a prediction vector for the third representative pixel c, and a method for calculating a prediction vector for the fourth representative pixel d. When switching by a flag, the first encoded bit string decoding unit 201 performs decoding.

  The second encoded bit string decoding unit 202 decodes the encoded bit string supplied via the input switch 209 and obtains a difference vector (S202). In addition, when prediction method information shows the motion compensation prediction by geometric transformation, 0-4 difference vectors are decoded according to the prediction method information. In the DMV4 mode, the difference vectors of the four representative pixels a, b, c, d corresponding to the four vertices a, b, c, d are decoded. In the DMV3 mode, the difference vectors of the three representative pixels a, b, and c corresponding to the three vertices a, b, and c are decoded. In the DMV2V mode, the difference vectors of the representative pixels a and c corresponding to the two vertices a and c are decoded. In the DMV2H mode, the difference vectors of the representative pixels a and b corresponding to the two vertices a and b are decoded. In the DMV2A mode, the difference vectors of the representative pixels a and d corresponding to the two vertices a and d are decoded. In the DMV2B mode, the difference vectors of the representative pixels b and c corresponding to the two vertices b and c are decoded. In the DMV1A mode, only the difference vector of the representative pixel a corresponding to the vertex a is decoded. In the DMV1B mode, only the difference vector of the representative pixel b corresponding to the vertex b is decoded. In the DMV1C mode, only the difference vector of the representative pixel c corresponding to the vertex c is decoded. In the DMV1D mode, only the difference vector of the representative pixel d corresponding to the vertex d is encoded. In the DMV0 mode, the difference vector of any representative pixel is not decoded.

  The motion vector calculation unit 215 calculates a prediction vector from the motion vector of the representative pixel, the peripheral block, or the peripheral pixel of the target block (S203). This is performed by the same method as the calculation of the prediction vector by the difference vector calculation unit 114 of the image encoding device 100. Further, the motion vector calculation unit 215 adds the difference vector supplied from the second encoded bit string decoding unit 202 to the prediction vector according to the prediction method information supplied from the first encoded bit string decoding unit 201. The motion vector of the target block or target pixel is calculated (S204). However, for representative points for which the difference vector has not been decoded by the second encoded bit string decoding unit 202, the prediction vector is used as the motion vector.

  The switching control unit 214 identifies the motion compensation prediction method for the target block according to the decoded prediction method information (S205). When the prediction method is a motion compensation prediction method by translation (parallel of S205), the translation motion compensation prediction unit 204 uses the decoded image signal to be the reference image signal supplied from the decoded image buffer 208 as the second image signal. Using the motion vector supplied from the encoded bit string decoding unit 202, motion compensation is performed by translation and a prediction signal is generated (S206).

  When the prediction method specified by the switching control unit 214 is a motion compensation prediction method based on geometric transformation (geometry in S205), the geometric transformation motion compensation prediction unit 205 uses the reference image signal supplied from the decoded image buffer 208 and The decoded image signal to be subjected to motion compensation by geometric transformation using the motion vector supplied from the second encoded bit string decoding unit 202 generates a prediction signal (S207).

  The third encoded bit string decoding unit 203 decodes the encoded bit string supplied via the input switch 209 and obtains an encoded prediction error signal (S208). The decoded image signal generation unit 207 decodes the acquired prediction error signal (S209). The decoded image signal generation unit 207 superimposes the prediction error signal decoded by the prediction error signal decoding unit 206 and the prediction signal generated by the translational motion compensation prediction unit 204 or the geometric transformation motion compensation prediction unit 205. Then, a decoded image signal is generated (S210). The decoded image signal generation unit 207 accumulates the generated decoded image signal in the decoded image buffer 208 (S211). The decoded image signal stored in the decoded image buffer 208 is used as a reference image signal in the translational motion compensation prediction unit 204 and the geometric transformation motion compensation prediction unit 205.

  FIG. 19 shows a procedure when a process of switching the calculation method of the prediction vector of the second representative pixel b occurs in S203 of the flowchart of FIG. 18 according to the value of the motion vector of the first representative pixel a of the target block. It is a flowchart to show. The motion vector calculation unit 215 compares the motion vector of the first representative pixel a of the target block with the motion vector of the fourth representative pixel d of the upper adjacent block or the motion vector of the upper adjacent block B (S2031). When the difference is equal to or smaller than the first set value (Y in S2032), the motion vector of the fourth representative pixel d of the upper adjacent block B or the motion vector of the upper adjacent block B is used as the second representative pixel b of the target block. Is adopted as the prediction vector (S2033). If it is larger than the first set value (N in S2032), the motion vector of the first representative pixel a of the target block is adopted as the prediction vector of the second representative pixel b of the target block (S2034).

  FIG. 20 shows a procedure when a process of switching the calculation method of the prediction vector of the third representative pixel c occurs in S203 of the flowchart of FIG. 18 according to the motion vector value of the first representative pixel a of the target block. It is a flowchart to show. The motion vector calculation unit 215 compares the motion vector of the first representative pixel a of the target block with the motion vector of the representative pixel d of the left adjacent block A or the motion vector of the upper adjacent block (S2035). When the difference is equal to or smaller than the second set value (Y in S2036), the motion vector of the fourth representative pixel d of the block A on the left side or the motion vector of the upper adjacent block is set to the third representative pixel c of the target block. The prediction vector is adopted (S2037). If it is larger than the second set value (N in S2036), the motion vector of the first representative pixel a of the target block is adopted as the prediction vector of the third representative pixel c of the target block (S2038).

  In addition, when switching the calculation method of the prediction vector of the 2nd representative pixel b and the calculation vector of the 3rd representative pixel c with a flag, it switches according to the flag decoded in the 1st encoding bit stream decoding part 201. FIG.

  As described above, according to the second embodiment, the encoded stream generated by the image encoding apparatus 100 according to the first embodiment can be efficiently decoded. Thereby, the effect implement | achieved by the image coding apparatus 100 which concerns on Embodiment 1 mentioned above is supported from the decoding side, and the effect can be ensured. That is, in the image coding method using motion compensation prediction by geometric transformation, the effect of improving the compression efficiency of the code amount can be supported from the decoding side, and the effect can be ensured. Also, in the image coding method using both motion compensation prediction by parallel movement and motion compensation prediction by geometric transformation, the effect of improving the compression efficiency of the code amount can be supported from the decoding side, and the effect can be ensured. it can. Moreover, the harmony and compatibility with the existing image decoding apparatus are high, and the introduction cost can be suppressed.

  In the above-described embodiment, the example in which the shape of the target block is a square has been described. In this regard, the shape of the target block may be other shapes such as a triangle, a parallelogram, and a trapezoid. In this case, it is preferable that the representative pixel is set at or near the vertex of the shape.

  DESCRIPTION OF SYMBOLS 100 Image coding apparatus, 101 Image buffer, 102 Translational motion compensation prediction part, 103 Geometric transformation motion compensation prediction part, 104 Prediction method determination part, 105 Prediction error signal generation part, 106 Prediction error signal coding part, 107 1 encoded bit string generation unit, 108 second encoded bit string generation unit, 109 third encoded bit string generation unit, 110 prediction error signal decoding unit, 111 decoded image signal generation unit, 112 decoded image buffer, 113 output switch, 121 geometry A first non-representative point motion vector calculation unit; a first prediction unit; a representative point motion vector setting unit; a second non-representative point motion vector calculation unit; Prediction unit, 134 Block matching unit, 135 Evaluation unit, 150 Encoding unit, 200 image decoding device, 201 first encoded bit string decoding unit, 202 second encoded bit string decoding unit, 203 third encoded bit string decoding unit, 204 translational motion compensation prediction unit, 205 geometric transformation motion compensation Prediction unit, 206 prediction error signal decoding unit, 207 decoded image signal generation unit, 208 decoded image buffer, 209 input switch, 210 first prediction unit switch, 211 second prediction unit switch, 212 third prediction unit switch, 213 fourth Prediction unit switch, 214 switching control unit, 215 motion vector calculation unit.

Claims (6)

Prediction method information for specifying a prediction mode, a differential motion vector of a representative pixel according to the prediction mode, and a prediction error signal included in an encoded stream encoded using motion compensated prediction by geometric transformation A decoding unit for decoding;
In accordance with the prediction mode specified by the prediction method information, a predicted motion vector of a representative pixel motion vector is calculated using motion vectors inside and outside the target block, and the difference motion vector of the representative pixel is added to the predicted motion vector. A motion vector generation unit for generating a motion vector of the representative pixel,
The motion vector of the representative pixel between the target block in the target image and the reference block in the reference image that is geometrically transformed with the target block, and the calculation using interpolation using the motion vector of the representative pixel A geometric transformation motion compensation prediction unit for generating a prediction signal from a motion vector other than a representative pixel and an image signal of the reference block;
An image signal generation unit for generating an image signal from the prediction signal and the prediction error signal decoded by the decoding unit;
As the representative pixel, a pixel located at the vertex constituting the target block, a pixel located near the vertex, or an interpolation pixel located near the vertex is selected,
The target block is a rectangular area;
The geometric transformation motion compensation prediction unit includes:
A first mode for decoding differential motion vectors of representative pixels in the upper left, upper right, lower left and lower right of the four vertices of the target block;
Of the four vertices of the target block, a second mode in which the differential motion vectors of the upper left, upper right and lower left representative pixels are decoded and the differential motion vector of the lower right representative pixel is not decoded;
Of the four vertices of the target block, a third mode that decodes the differential motion vectors of the upper left and lower left representative pixels and does not decode the differential motion vectors of the upper right and lower right representative pixels;
A fourth mode in which the differential motion vectors of the upper left and upper right representative pixels among the four vertices of the target block are decoded, and the differential motion vectors of the lower left and lower right representative pixels are not decoded;
Of the four vertices of the target block, a fifth mode that decodes the differential motion vectors of the upper left and lower right representative pixels and does not decode the differential motion vectors of the upper right and lower left representative pixels;
Among the four vertices of the target block, a sixth mode that decodes the differential motion vectors of the upper right and lower left representative pixels and does not decode the differential motion vectors of the upper left and lower right representative pixels;
A seventh mode in which the differential motion vector of the upper left representative pixel is decoded among the four vertices of the target block, and the differential motion vector of the upper right, lower left and lower right representative pixels is not decoded;
Among the four vertices of the target block, an eighth mode in which the differential motion vector of the upper right representative pixel is decoded and the differential motion vector of the upper left, lower left and lower right representative pixels is not decoded;
Of the four vertices of the target block, a ninth mode that decodes the differential motion vector of the lower left representative pixel and does not decode the differential motion vector of the upper left, upper right and lower right representative pixels;
A tenth mode in which the differential motion vector of the lower right representative pixel is decoded among the four vertices of the target block, and the differential motion vector of the upper left, upper right and lower left representative pixels is not decoded;
Among the four vertices of the target block, at least two prediction modes are included in the eleventh mode in which the differential motion vectors of the upper left, upper right, lower left, and lower right representative pixels are not decoded, and the at least one prediction mode includes , Including at least one of the fifth mode, the sixth mode, the seventh mode, the eighth mode, the ninth mode, the tenth mode, and the eleventh mode,
An image decoding apparatus characterized by that. In the first mode, the second mode, the fourth mode, the sixth mode, or the eighth mode, the motion vector generation unit calculates a predicted motion vector of a representative pixel corresponding to the upper right vertex in the square target block, Adaptively representing the motion vector value of the representative pixel corresponding to the lower right vertex of the adjacent block or the motion vector value of the upper adjacent block and the motion vector value of the representative pixel corresponding to the upper left vertex of the target block Selecting, calculating based on the value of the selected motion vector, adding the calculated predicted motion vector and the difference motion vector, and the motion vector of the representative pixel corresponding to the upper right vertex in the target block To calculate
In the third mode, the fifth mode, the seventh mode, the ninth mode, the tenth mode, or the eleventh mode, the predicted motion vector of the representative pixel corresponding to the upper right vertex in the square target block is Adaptively select the motion vector value of the representative pixel corresponding to the lower right vertex of the block or the motion vector value of the upper neighboring block and the motion vector value of the representative pixel corresponding to the upper left vertex of the target block The value of the selected motion vector is used as the motion vector of the representative pixel corresponding to the upper right vertex in the target block.
The image decoding apparatus according to claim 1 . In the first mode, the second mode, the third mode, the sixth mode, or the ninth mode, the motion vector generation unit generates a predicted motion vector of a representative pixel corresponding to a lower left vertex in the square target block, The value of the motion vector of the representative pixel corresponding to the lower right vertex of the adjacent block or the value of the motion vector of the left adjacent block and the value of the motion vector of the representative pixel corresponding to the upper left vertex of the target block are adaptively set. Select, calculate based on the value of the selected motion vector, add the calculated predicted motion vector and the difference motion vector, and the motion vector of the representative pixel corresponding to the lower left vertex in the target block To calculate
In the fourth mode, the fifth mode, the seventh mode, the eighth mode, the tenth mode, or the eleventh mode, the prediction motion vector of the representative pixel corresponding to the lower left vertex of the square target block is set to the adjacent left side. Adaptively select the motion vector value of the representative pixel corresponding to the lower right vertex of the block or the motion vector value of the left adjacent block and the motion vector value of the representative pixel corresponding to the upper left vertex of the target block The value of the selected motion vector is used as the motion vector of the representative pixel corresponding to the lower left vertex in the target block.
The image decoding apparatus according to claim 1 or 2 , wherein Translation motion compensation for generating a prediction signal from a motion vector between a target block in the target image and a reference block in a reference image that is translated with the target block, and the image signal of the reference block A predictor;
With reference to the prediction method information decoded by the decoding unit, for each target block in the target image, either the prediction method by the translation motion compensation prediction unit or the prediction method by the geometric transformation motion compensation prediction unit And a control unit for specifying whether to use
The data included in the encoded stream is encoded using a combination of motion compensated prediction based on the parallel movement and motion compensated prediction based on the geometric transformation,
The motion vector generation unit converts the predicted motion vector of the target block to a representative pixel corresponding to an adjacent vertex of the adjacent block according to the prediction mode and a motion compensation prediction method for the adjacent block adjacent to the target block. Calculate based on the value of the motion vector or the value of the motion vector of the adjacent block,
The image decoding device according to any one of claims 1 to 3 , wherein Prediction method information for specifying a prediction mode, a differential motion vector of a representative pixel according to the prediction mode, and a prediction error signal included in an encoded stream encoded using motion compensated prediction by geometric transformation A decoding step for decoding;
In accordance with the prediction mode specified by the prediction method information, a predicted motion vector of a representative pixel motion vector is calculated using motion vectors inside and outside the target block, and the difference motion vector of the representative pixel is added to the predicted motion vector. A motion vector generation step for generating a motion vector of the representative pixel;
The motion vector of the representative pixel between the target block in the target image and the reference block in the reference image that is geometrically transformed with the target block, and the calculation using interpolation using the motion vector of the representative pixel A geometric transformation motion compensated prediction step for generating a prediction signal from a motion vector other than the representative pixel and the image signal of the reference block;
An image signal generation step for generating an image signal from the prediction signal and the prediction error signal decoded by the decoding step;
As the representative pixel, a pixel located at the vertex constituting the target block, a pixel located near the vertex, or an interpolation pixel located near the vertex is selected,
The target block is a rectangular area;
The geometric transformation motion compensation prediction step includes:
A first mode for decoding differential motion vectors of representative pixels in the upper left, upper right, lower left and lower right of the four vertices of the target block;
Of the four vertices of the target block, a second mode in which the differential motion vectors of the upper left, upper right and lower left representative pixels are decoded and the differential motion vector of the lower right representative pixel is not decoded;
Of the four vertices of the target block, a third mode that decodes the differential motion vectors of the upper left and lower left representative pixels and does not decode the differential motion vectors of the upper right and lower right representative pixels;
A fourth mode in which the differential motion vectors of the upper left and upper right representative pixels among the four vertices of the target block are decoded, and the differential motion vectors of the lower left and lower right representative pixels are not decoded;
Of the four vertices of the target block, a fifth mode that decodes the differential motion vectors of the upper left and lower right representative pixels and does not decode the differential motion vectors of the upper right and lower left representative pixels;
Among the four vertices of the target block, a sixth mode that decodes the differential motion vectors of the upper right and lower left representative pixels and does not decode the differential motion vectors of the upper left and lower right representative pixels;
A seventh mode in which the differential motion vector of the upper left representative pixel is decoded among the four vertices of the target block, and the differential motion vector of the upper right, lower left and lower right representative pixels is not decoded;
Among the four vertices of the target block, an eighth mode in which the differential motion vector of the upper right representative pixel is decoded and the differential motion vector of the upper left, lower left and lower right representative pixels is not decoded;
Of the four vertices of the target block, a ninth mode that decodes the differential motion vector of the lower left representative pixel and does not decode the differential motion vector of the upper left, upper right and lower right representative pixels;
A tenth mode in which the differential motion vector of the lower right representative pixel is decoded among the four vertices of the target block, and the differential motion vector of the upper left, upper right and lower left representative pixels is not decoded;
Among the four vertices of the target block, at least two prediction modes are included in the eleventh mode in which the differential motion vectors of the upper left, upper right, lower left, and lower right representative pixels are not decoded, and the at least one prediction mode includes , Including at least one of the fifth mode, the sixth mode, the seventh mode, the eighth mode, the ninth mode, the tenth mode, and the eleventh mode,
An image decoding method characterized by the above. Prediction method information for specifying a prediction mode, a differential motion vector of a representative pixel according to the prediction mode, and a prediction error signal included in an encoded stream encoded using motion compensated prediction by geometric transformation Decryption processing for decryption;
In accordance with the prediction mode specified by the prediction method information, a predicted motion vector of a representative pixel motion vector is calculated using motion vectors inside and outside the target block, and the difference motion vector of the representative pixel is added to the predicted motion vector. Motion vector generation processing for generating motion vectors of representative pixels,
The motion vector of the representative pixel between the target block in the target image and the reference block in the reference image that is geometrically transformed with the target block, and the calculation using interpolation using the motion vector of the representative pixel A geometric transformation motion compensation prediction process for generating a prediction signal from a motion vector other than a representative pixel and an image signal of the reference block;
Causing the computer to execute the prediction signal and an image signal generation process for generating an image signal from the prediction error signal decoded by the decoding process;
As the representative pixel, a pixel located at the vertex constituting the target block, a pixel located near the vertex, or an interpolation pixel located near the vertex is selected,
The target block is a rectangular area;
The geometric transformation motion compensation prediction process is:
A first mode for decoding differential motion vectors of representative pixels in the upper left, upper right, lower left and lower right of the four vertices of the target block;
Of the four vertices of the target block, a second mode in which the differential motion vectors of the upper left, upper right and lower left representative pixels are decoded and the differential motion vector of the lower right representative pixel is not decoded;
Of the four vertices of the target block, a third mode that decodes the differential motion vectors of the upper left and lower left representative pixels and does not decode the differential motion vectors of the upper right and lower right representative pixels;
A fourth mode in which the differential motion vectors of the upper left and upper right representative pixels among the four vertices of the target block are decoded, and the differential motion vectors of the lower left and lower right representative pixels are not decoded;
Of the four vertices of the target block, a fifth mode that decodes the differential motion vectors of the upper left and lower right representative pixels and does not decode the differential motion vectors of the upper right and lower left representative pixels;
Among the four vertices of the target block, a sixth mode that decodes the differential motion vectors of the upper right and lower left representative pixels and does not decode the differential motion vectors of the upper left and lower right representative pixels;
A seventh mode in which the differential motion vector of the upper left representative pixel is decoded among the four vertices of the target block, and the differential motion vector of the upper right, lower left and lower right representative pixels is not decoded;
Among the four vertices of the target block, an eighth mode in which the differential motion vector of the upper right representative pixel is decoded and the differential motion vector of the upper left, lower left and lower right representative pixels is not decoded;
Of the four vertices of the target block, a ninth mode that decodes the differential motion vector of the lower left representative pixel and does not decode the differential motion vector of the upper left, upper right and lower right representative pixels;
A tenth mode in which the differential motion vector of the lower right representative pixel is decoded among the four vertices of the target block, and the differential motion vector of the upper left, upper right and lower left representative pixels is not decoded;
Among the four vertices of the target block, at least two prediction modes are included in the eleventh mode in which the differential motion vectors of the upper left, upper right, lower left, and lower right representative pixels are not decoded, and the at least one prediction mode includes , Including at least one of the fifth mode, the sixth mode, the seventh mode, the eighth mode, the ninth mode, the tenth mode, and the eleventh mode,
An image decoding program characterized by the above.

Images