JP2012080151A - Method and apparatus for moving image encoding and moving image decoding using geometry-transformed/motion-compensated prediction - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving image encoding apparatus, a moving image decoding apparatus, a moving image encoding method and a moving image decoding method that improve prediction efficiency without increasing the amount of code by reducing motion detection processing required to estimate a geometric transform parameter used for geometry-transformed/motion-compensated prediction.SOLUTION: The moving image encoding apparatus includes: a motion information acquisition section for acquiring motion information about one or more of adjacent blocks adjacent to one of divided pixel blocks of an image signal; a geometric transform information acquisition section for acquiring on the basis of the motion information a geometric transform parameter that is information on the form of mapping by a geometric transform of the pixel block in a reference image signal in the process of motion compensation of the pixel block; a geometry-transformed prediction section for performing a geometry-transformed motion prediction including a geometric transform between the reference image signal and the pixel block with the use of the reference image signal geometrically transformed with the geometric transform parameter; and an encoding section for encoding a prediction error value of the pixel block subjected to the geometry-transformed motion prediction.

Description

  The present invention relates to moving picture coding and moving picture decoding in which geometric transformation parameters are estimated using motion information of neighboring blocks and prediction target blocks, and geometric transformation prediction processing of the prediction target blocks is performed based on the estimated geometric transformation parameters. The present invention relates to a method, a program, and an apparatus.

  In recent years, a moving picture coding method having greatly improved coding efficiency has been jointly developed by ITU-T and ISO / IEC. H. H.264 and ISO / IEC 14496-10 (hereinafter referred to as “H.264”). H. In H.264, prediction processing, conversion processing, and entropy encoding processing are performed in units of rectangular blocks (for example, 16 × 16 pixels, 8 × 8 pixels, etc.). For this reason, H.C. In H.264, when an object that cannot be expressed by a rectangular block is predicted, the prediction efficiency is increased by selecting a smaller prediction block (4 × 4 pixels or the like). In order to predict such an object effectively, there are a method of preparing a plurality of prediction patterns in a rectangular block, a method of applying motion compensation using affine transformation to a deformed object, and the like.

  For example, in Japanese Patent Application Laid-Open No. 2007-312397 (Patent Document 1), an object movement model is an affine transformation model, and an optimal affine transformation parameter is calculated for each prediction target block, thereby enlarging / reducing the object. An invention such as a video frame transfer method using a prediction that takes rotation into consideration is disclosed.

  Further, Non-Patent Document 1 divides a block into triangular patches based on motion vector information calculated as a translation model as a motion model, and estimates an affine transformation parameter for each patch. A method of approximately performing motion compensation prediction of an affine transformation model is disclosed.

JP 2007-312397 A

R.C.Kordasiewicz, M.D.Galant, and S. Shirani, “Affine Motion Prediction Based on Translational Motion Vectors,” IEEE Trans. On Circuits and Systems for Video Technologies, Vol. 17, No. 10, October 2007.

  However, in the method described in Patent Document 1, six types of affine transformation parameters are transmitted for each pixel block, so that overhead increases. In addition, in order to calculate these parameters, it is necessary to perform block matching between a plurality of reference images and an input image corresponding to a prediction target block, which increases the amount of calculation.

  Further, the method described in Non-Patent Document 1 uses an affine transformation using motion vectors of eight types of adjacent blocks adjacent to a pixel block to be predicted, such as up, down, left, and right, and a motion vector calculated from the pixel block to be predicted. In order to estimate the parameters, it is necessary to re-encode the frame in order to obtain an optimal motion vector. On the other hand, when only the motion vector calculation is performed in units of frames, there is a problem in that it is not optimal in terms of code amount and encoding distortion, and encoding efficiency is reduced.

  The present invention has been invented in order to solve these problems in view of the above points. The present invention reduces the motion detection processing necessary for estimating the geometric transformation parameters used for the geometric transformation motion compensation prediction, and reduces the code amount. An object of the present invention is to provide a video encoding device, a video decoding device, a video encoding method, and a video decoding method that improve the prediction efficiency without increasing the image quality.

  In order to achieve the above object, the moving picture encoding apparatus of the present invention employs the following configuration.

  A moving image encoding apparatus according to the present invention includes a motion information acquisition unit that acquires motion information of one or more adjacent blocks among adjacent blocks adjacent to one of pixel blocks into which an image signal is divided, and motion for the pixel blocks A geometric transformation information acquisition unit that acquires, based on the motion information, a geometric transformation parameter that is information related to a shape of a mapping obtained by geometric transformation of the pixel block in a reference image signal when performing compensation, and the reference image signal A geometric transformation prediction unit that performs geometric transformation motion prediction including geometric transformation with the pixel block using the reference image signal subjected to geometric transformation by the geometric transformation parameter, and the geometric transformation motion prediction is performed. And an encoding unit that encodes the prediction error value of the pixel block.

  In order to achieve the above object, the moving picture coding method of the present invention provides motion information for acquiring motion information of one or more adjacent blocks among adjacent blocks adjacent to one of the pixel blocks into which the image signal is divided. Geometric transformation information for obtaining, based on the motion information, a geometric transformation parameter, which is information relating to the shape of a map obtained by geometric transformation of the pixel block, in an acquisition step and a reference image signal when performing motion compensation for the pixel block An obtaining step, and a geometric transformation prediction step including performing geometric transformation motion prediction including geometric transformation between the reference image signal and the pixel block using the reference image signal subjected to geometric transformation by the geometric transformation parameter. And a coding step for coding a prediction error value of the pixel block on which the geometric transformation motion prediction has been performed, and Rukoto can.

  In order to achieve the above object, a moving picture decoding apparatus according to the present invention includes a geometric transformation including a geometric transformation between a pixel block into which an image signal is divided and a reference image signal when motion compensation is performed on the pixel block. A decoding unit that decodes code data obtained by encoding the image signal, including a prediction error value obtained by conversion motion prediction, and a motion of one or more adjacent blocks among adjacent blocks adjacent to the pixel block A motion information acquisition unit that acquires information, and a geometric transformation information acquisition unit that acquires, based on the motion information, a geometric transformation parameter that is information related to a shape of a mapping by geometric transformation of the pixel block in the reference image signal; The geometric transformation motion prediction is performed using the reference image signal subjected to the geometric transformation by the geometric transformation parameter, and a prediction value is generated. When, can be configured to have, an adder for adding the said predicted value generated and decoded the prediction error value.

  In order to achieve the above object, the moving picture decoding method of the present invention includes a geometric conversion including a geometric transformation between a pixel block into which an image signal is divided and a reference image signal when motion compensation is performed on the pixel block. A decoding step for decoding code data obtained by encoding the image signal, including a prediction error value obtained by conversion motion prediction; and a motion of one or more adjacent blocks among adjacent blocks adjacent to the pixel block A motion information acquisition step for acquiring information, and a geometric conversion information acquisition step for acquiring, based on the motion information, a geometric transformation parameter that is information related to the shape of the mapping by geometric transformation of the pixel block in the previous reference image signal; , A geometric transformation motion prediction including a geometric transformation between the reference image signal and the pixel block is performed by the geometric transformation according to the geometric transformation parameter. And a geometric transformation prediction step for generating a prediction value, and an addition step for adding the decoded prediction error value and the generated prediction value. be able to.

  According to the moving image encoding device, the moving image decoding device, the moving image encoding method, and the moving image decoding method of the present invention, a motion detection process necessary for estimating a geometric transformation parameter used for geometric transformation motion compensation prediction It is possible to provide a moving picture encoding apparatus, a moving picture decoding apparatus, a moving picture encoding method, and a moving picture decoding method that improve the prediction efficiency without increasing the code amount.

FIG. 1 is a block diagram showing a configuration of a moving picture encoding apparatus according to the first embodiment. FIG. 2 is a block diagram showing a configuration of the inter prediction unit according to the first embodiment. FIG. 3 is a diagram illustrating a 16 × 16 pixel block. FIG. 4 is a diagram showing the flow of the encoding process. FIG. 5 is a diagram illustrating the relationship between the positional relationship between the reference image signal and the prediction target image and the motion vector. FIG. 6 is a diagram illustrating an example in which a pixel at a fractional position when performing motion compensation prediction is generated by interpolation processing. FIG. 7A is a diagram illustrating the size of a motion compensation block in units of macroblocks. FIG. 7B is a diagram illustrating the size of a motion compensation block in subblock units. FIG. 8 is a diagram illustrating a configuration of the geometric transformation parameter derivation unit. FIG. 9A is a diagram illustrating a positional relationship between a pixel block to be encoded or decoded and an adjacent block. FIG. 9B is a diagram illustrating the positional relationship between adjacent blocks when the pixel block to be encoded or decoded is at the upper left of the macroblock. FIG. 9C is a diagram illustrating a positional relationship between adjacent blocks when the pixel block to be encoded or decoded is at the upper right of the macro block. FIG. 9D is a diagram illustrating a positional relationship between adjacent blocks when a pixel block to be encoded or decoded is at the lower left of a macro block. FIG. 9E is a diagram illustrating a positional relationship between adjacent blocks when a pixel block to be encoded or decoded is located on the lower right side of a macroblock. FIG. 10A is a diagram illustrating a positional relationship when a block size of an adjacent block is small with respect to a pixel block to be encoded or decoded. FIG. 10B is a diagram illustrating a positional relationship when a block size of an adjacent block is small with respect to a pixel block to be encoded or decoded. FIG. 10C is a diagram illustrating a positional relationship when a block size of an adjacent block is small with respect to a pixel block to be encoded or decoded. FIG. 10D is a diagram illustrating a positional relationship when a block size of an adjacent block is small with respect to a pixel block to be encoded or decoded. FIG. 11A is a diagram showing the positional relationship between adjacent blocks in the case of the upper left of the pixel block, which is smaller than the adjacent block to be encoded or decoded. FIG. 11B is a diagram illustrating the positional relationship between adjacent blocks in the case of the upper right of the pixel block, which is smaller than the adjacent block to be encoded or decoded. FIG. 11C is a diagram illustrating the positional relationship between adjacent blocks in the case of the lower left of the pixel block that is smaller than the adjacent block to be encoded or decoded. FIG. 11D is a diagram illustrating the positional relationship between adjacent blocks in the case of the lower right of the pixel block that is smaller than the adjacent block to be encoded or decoded. FIG. 12 is a diagram illustrating a positional relationship when a block size of an adjacent block is small with respect to a pixel block to be encoded or decoded. FIG. 13A is a diagram illustrating division of a prediction target pixel when geometric conversion is performed on the prediction target pixel. FIG. 13B is a diagram illustrating division of a prediction target pixel when performing geometric transformation on the prediction target pixel. FIG. 14 is a diagram illustrating a configuration of the geometric transformation prediction unit. FIG. 15 is a diagram illustrating an example of prediction value generation for motion compensation prediction and geometric transformation prediction. FIG. 16 is a diagram illustrating an example in which the pixel value at the fractional position where the geometric transformation is performed is generated by the interpolation process. FIG. 17 is a diagram illustrating an example of deformation of an object when geometric transformation is performed. FIG. 18 is a flowchart showing a flow of processing of geometric transformation prediction shown in the first embodiment. FIG. 19 is a diagram illustrating a syntax structure. FIG. 20 is a diagram illustrating information included in the slice header syntax. FIG. 21 is a diagram illustrating information included in the slice data syntax according to the first embodiment. FIG. 22 is a diagram illustrating information included in the macroblock layer syntax according to the first embodiment. FIG. 23 is a diagram illustrating information included in the macroblock prediction syntax in the first embodiment. FIG. 24 is a diagram illustrating information included in the sub macroblock prediction syntax according to the first embodiment. FIG. 25 is a diagram illustrating information included in the macroblock layer syntax in the modification example of the first embodiment. FIG. 26 is a block diagram showing a video encoding apparatus according to the second embodiment. FIG. 27A is a diagram illustrating a 16 × 16 pixel block used for intra prediction. FIG. 27B is a diagram illustrating a 4 × 4 pixel block used for intra prediction. FIG. 27C is a diagram illustrating an 8 × 8 pixel block used for intra prediction. FIG. 28 is a block diagram illustrating an inter prediction unit according to the third embodiment. FIG. 29 is a diagram illustrating information included in the macroblock layer syntax in the fourth embodiment. FIG. 30 is a diagram illustrating information included in the sub macroblock layer syntax in the fourth embodiment. FIG. 31 is a block diagram showing a moving picture decoding apparatus according to the fifth embodiment. FIG. 32 is a block diagram showing a moving picture decoding apparatus according to the fifth embodiment. FIG. 33 is a block diagram showing a video decoding apparatus according to the sixth embodiment.

  The first to seventh embodiments will be described below with reference to the drawings. The first to fourth embodiments are embodiments using a moving image encoding device, and the fifth to seventh embodiments are performed using a moving image decoding device. It is a form. Note that “determination parameters” in the following embodiments correspond to “determination parameters”.

<Moving picture encoding apparatus>
A moving image encoding apparatus described in the following embodiments divides each frame constituting an input image signal into a plurality of pixel blocks, performs an encoding process on the divided pixel blocks, and performs compression encoding. , A device for outputting a code string.

[First Embodiment]
<Moving picture encoding apparatus 100>
FIG. 1 is a diagram illustrating a configuration of a moving image encoding apparatus 100 that realizes an encoding method using geometric transformation prediction. FIG. 2 is a block diagram of the inter prediction unit 130 included in the video encoding device 100.

  The moving image encoding apparatus 100 in FIG. 1 performs inter prediction (interframe prediction) encoding processing on the input image signal 114 based on the encoding parameter input from the encoding control unit 126, and generates the predicted image signal 123. Generate encoded data 124.

  In the moving image encoding apparatus 100, an input image signal 114 of a moving image or a still image is divided and input in units of pixel blocks, for example, macro blocks. The input image signal is one encoding processing unit including both a frame and a field. In this embodiment, an example in which a frame is used as one encoding processing unit will be described.

  The moving image encoding apparatus 100 performs encoding in a plurality of prediction modes having different block sizes and generation methods of the predicted image signal 123. Specifically, the prediction image signal 123 is generated using a plurality of reference frames that are roughly divided into intra prediction (intraframe prediction) in which a prediction image is generated only within a frame to be encoded and a temporally different reference frame. In this embodiment, an example in which a prediction image signal is generated using inter prediction will be described.

  In the first to seventh embodiments, the macro block is set to the basic processing block size of the encoding process. The macroblock is typically a 16 × 16 pixel block shown in FIG. 3, for example, but may be a 32 × 32 pixel block unit or an 8 × 8 pixel block unit. The shape of the macroblock does not necessarily need to be a square lattice. Hereinafter, the encoding target macroblock of the input image signal 114 is referred to as a “prediction target block”.

  In the first to seventh embodiments, it is assumed that the encoding process is performed from the upper left to the lower right as shown in FIG. 4 in order to simplify the description. In FIG. 4, in the encoded frame f subjected to the encoding process, the blocks located on the left and above the block c to be encoded are the encoded blocks p.

  The moving image coding apparatus 100 includes a subtractor 101, a transform / quantization unit 102, an inverse quantization / inverse transform unit 103, an adder 104, a reference image memory 105, a motion estimation unit 106, and an inter prediction unit 130. . The moving image encoding apparatus 100 is connected to the encoding control unit 126.

  Next, the flow of encoding in the moving image encoding apparatus 100 will be described. First, the input image signal 114 is input to the subtractor 101. The subtracter 101 further receives a predicted image signal 123 corresponding to each prediction mode output from the inter prediction unit 130. The subtractor 101 calculates a prediction error signal 115 obtained by subtracting the prediction image signal 123 from the input image signal 114. The prediction error signal 115 is input to the transform / quantization unit 102.

  The transform / quantization unit 102 performs orthogonal transform such as discrete cosine transform (DCT) on the prediction error signal 115 to generate transform coefficients. The transformation in the transformation / quantization unit 102 is H.264. In addition to the discrete cosine transform used in H.264, discrete sine transform, wavelet transform, component analysis, or the like may be used.

  The transform / quantization unit 102 quantizes the transform coefficient according to the quantization information represented by the quantization parameter, the quantization matrix, and the like given by the encoding control unit 126. The transform / quantization unit 102 outputs the quantized transform coefficient 116 to the entropy coding unit 112 and also outputs it to the inverse quantization / inverse transform unit 103.

  The entropy encoding unit 112 performs entropy encoding, for example, Huffman encoding or arithmetic encoding, on the quantized transform coefficient 116. The entropy encoding unit 112 further performs entropy encoding on various encoding parameters used when the target block is encoded, including the prediction information output from the encoding control unit 126 and the like. Thereby, encoded data is generated.

  Note that the encoding parameter is a parameter necessary for decoding prediction information, information on transform coefficients, information on quantization, and the like. Note that the encoding parameter of the prediction target block is held in an internal memory of the encoding control unit 126, and is used when the prediction target block is used as an adjacent block of another pixel block.

  The encoded data 124 generated by the entropy encoding unit 112 is output from the moving image encoding apparatus 100, and after being multiplexed and temporarily stored in the output buffer 113, according to the output timing managed by the encoding control unit 126. Output as encoded data 124. The encoded data 124 is sent to, for example, a storage system (storage medium) or a transmission system (communication line) (not shown).

  The inverse quantization / inverse transform unit 103 performs an inverse quantization process on the quantized transform coefficient 116 output from the transform / quantization unit 102. Here, the quantization information corresponding to the quantization information used in the transform / quantization unit 102 is loaded from the internal memory of the encoding control unit 126 and the inverse quantization process is performed. Note that the quantization information is, for example, a parameter represented by a quantization parameter, a quantization matrix, or the like.

  The inverse quantization / inverse transform unit 103 further reproduces the decoded prediction error signal 117 by performing inverse orthogonal transform such as inverse discrete cosine transform (IDCT) on the transform coefficient after inverse quantization. The

  The decoded prediction error signal 117 is input to the adder 104. The adder 104 adds the decoded prediction error signal 117 and the predicted image signal 123 output from the inter prediction unit 130, thereby generating a decoded image signal 118. The decoded image signal 118 is a local decoded image signal. The decoded image signal 118 is stored as the reference image signal 120 in the reference image memory 105. The reference image signal 120 accumulated in the reference image memory 105 is output to the motion estimation unit 106, the inter prediction unit 130, etc., and is referred to when performing prediction.

  The motion estimation unit 106 uses the input image signal 114 and the reference image signal 120 to calculate a motion vector 119 suitable for the prediction target block. The motion information may be represented by a motion vector, for example. For example, the motion information may be a predicted value when the motion vector is predicted by another motion vector or the like.

  The motion estimation unit 106 calculates the motion vector 119 by performing block matching between the prediction target block of the input image signal 114 and the interpolated image of the reference image signal 120. As an evaluation criterion for matching, for example, a value obtained by accumulating the difference between the input image signal 114 and the interpolated image after matching for each pixel is used.

  In addition to the method described above, the motion vector 119 may be determined by using a value obtained by converting the difference between the predicted image and the original image, taking into account the magnitude of the motion vector, the code amount of the motion vector, and the like. Or may be determined. Moreover, you may utilize costs, such as Formula (29) and Formula (30) mentioned later. Further, the matching may be performed through a search within the matching range based on search range information provided from the outside of the moving image encoding apparatus 100, or may be performed hierarchically for each pixel accuracy.

  The motion vectors 119 calculated for the plurality of reference image signals in this way are input to the inter prediction unit 130 and used to generate the predicted image signal 123. The plurality of reference image signals are locally decoded images having different display times.

  In addition, the calculated motion vector 119 is output to the entropy encoding unit 112, and after entropy encoding is performed, it is multiplexed into encoded data. Furthermore, the motion vector 119 obtained by encoding the target pixel block is stored in the internal memory of the encoding control unit 126 and is appropriately loaded from the inter prediction unit 130 and used.

<Inter prediction unit 130>
The inter prediction unit 130 in FIG. 2 includes a motion compensation prediction unit 107, a geometric transformation parameter derivation unit 108, a geometric transformation prediction unit 109, a determination parameter derivation unit 127, a prediction separation switch 110, a prediction switching unit 111, and an entropy coding unit. 112.

  The motion compensation prediction unit 107 generates a prediction image signal 123 using the input motion vector 119 and the reference image signal 120. FIG. 5 is a diagram illustrating an example of inter prediction performed by the motion compensation prediction unit 107. In FIG. 5, a motion vector is acquired by acquiring a position with a small prediction error in the frame (t−1) for the prediction target block included in the frame (t).

  In inter prediction, interpolation processing is performed using a plurality of reference image signals 120 stored in the reference image memory 105, and the generated interpolated image and the input image signal 114 are based on the amount of deviation from the pixel block at the same position. A predicted image signal 123 is generated. As the interpolation processing, for example, interpolation processing with 1/2 pixel accuracy, interpolation processing with 1/4 pixel accuracy, or the like is used. By performing interpolation processing such as filtering processing on the reference image signal 120, interpolation pixel processing is performed. Generate a value. For example, H.D. is allowed to perform interpolation processing up to 1/4 pixel accuracy for luminance signals. In H.264, the shift amount is expressed by four times the integer pixel accuracy. This amount of deviation is called a motion vector.

  The motion compensated prediction unit 107 calculates the position referenced by the motion vector 119 using the following equation (1) from the position of the prediction target block according to the information of the motion vector 119. Here, H. A description will be given taking an example of H.264 interpolation interpolation with 1/4 pixel accuracy. When each component of the motion vector is a multiple of 4, it means that it is an integer pixel position. In other cases, it is a predicted position corresponding to an interpolation position with fractional accuracy.

  Here, (x, y) is an index in the vertical and horizontal directions representing the head position of the prediction target block, and (x_pos, y_pos) represents the corresponding prediction position of the reference image signal. (Mv_x, mv_y) represents a motion vector having a 1/4 pixel accuracy. Next, a predicted pixel is generated by interpolation or interpolation processing of the corresponding pixel position of the reference image signal 120 with respect to the determined pixel position.

  FIG. It is a figure which shows the example of a H.264 prediction pixel production | generation. In the figure, alphabets indicated by capital letters indicate pixels at integer positions, and dot hatched squares indicate interpolation pixels at 1/2 pixel positions. In addition, hatched squares indicate interpolation pixels corresponding to 1/4 pixel positions. For example, in the drawing, the interpolation processing of 1/2 pixel corresponding to the positions of alphabets b and h is calculated by the following equation.

また、図中でアルファベットａ、ｄの位置に対応する１／４画素の補間処理は次式で算出される。

In the figure, the interpolation processing of 1/4 pixels corresponding to the positions of alphabets a and d is calculated by the following equation.

  In Expression (2), the interpolation pixel at the 1/2 pixel position is generated using a 6-tap FIR filter (tap coefficients: (1, -5, 20, 20, -5, 1) / 32). In Expression (3), the interpolation pixel at the 1/4 pixel position is calculated using a 2-tap average value filter (tap coefficient: (1, 1) / 2). The interpolation process of 1/2 pixel corresponding to the alphabet j existing in the middle of the four integer pixel positions is generated by performing both directions of 6 taps in the vertical direction and 6 taps in the horizontal direction. Interpolated values can be generated by the same rule for pixel positions other than those described.

  In inter prediction, a block size suitable for the current target pixel block can be selected from a plurality of prediction blocks. FIG. 7A shows the size of a motion compensation pixel block in units of macroblocks, and FIG. 7B shows the size of a motion compensation pixel block in units of sub-blocks (8 × 8 pixel blocks or less).

  FIG. 7A shows a 16 × 16 pixel MB1, an MB2 composed of two 16 × 8 pixel blocks, an MB3 composed of two 8 × 16 pixel blocks, and an MB4 composed of four 8 × 8 pixel blocks. It is. In FIG. 7B, an SB of 8 × 8 pixels, an SB2 composed of two 8 × 4 pixel blocks, an SB3 composed of two 4 × 8 pixel blocks, and an SB4 composed of four 4 × 4 pixel blocks. Is shown.

  Since it is possible to possess a motion vector for each size of these prediction target blocks, the optimal shape and motion vector of the prediction target block can be used according to the local nature of the input image signal 114. H. In H.264, information on which reference image signal the motion vector is calculated can be changed as a minimum for each 8 × 8 pixel block as Ref_idx.

  The prediction image signal 123 generated by the motion compensation prediction unit 107 is input to the prediction separation switch 110, and depends on the connection destination of the output end of the switch controlled according to the prediction switching information 122 output from the prediction switching unit 111 described later. Selected.

  On the other hand, the motion vector 119 output from the motion estimation unit 106 is input to the geometric transformation parameter derivation unit 108, and the geometric transformation parameter 121 is generated. Here, the geometric transformation parameter 121 is a parameter set for performing geometric transformation on the reference image signal 120. The geometric transformation parameter 121 output from the geometric transformation parameter derivation unit 108 is input to the geometric transformation prediction unit 109 and to the determination parameter derivation unit 127.

  The geometric transformation prediction unit 109 generates a predicted image signal 123 subjected to geometric transformation using the input geometric transformation parameter 121 and the reference image signal 120. The geometric transformation parameter 121 of the prediction target block is held in the internal memory of the encoding control unit 126, and is used when the prediction target block becomes an adjacent block of another pixel block.

  The determination parameter deriving unit 127 determines whether to use the predicted image signal generated by the motion compensation prediction unit 107 or the predicted image signal generated by the geometric conversion prediction unit 109 based on the input geometric conversion parameter 121. A determination parameter for determining is derived. The determination parameter 125 generated by the determination parameter deriving unit 127 is input to the prediction switching unit 111.

The prediction switching unit 111 outputs the prediction switching information 122 of the prediction target block according to the input determination parameter 125. The prediction separation switch 110 determines whether to connect the output terminal of the switch to the motion compensation prediction unit 107 or the geometric transformation prediction unit 109 according to the input prediction switching information 122. In accordance with this determination, one of the predicted image signals 123 is output to the subtracter 101 and the adder 104.
The above is the flow of the encoding process of the moving image encoding apparatus 100.

<Geometric Transformation Prediction Process in Inter Prediction Unit 130>
Hereinafter, details of the geometric transformation prediction processing according to the present embodiment will be described in more detail.

<Geometric transformation parameter derivation unit 108>
First, the process of the geometric transformation parameter derivation unit 108 will be specifically described. The geometric transformation parameter derivation unit 108 derives the geometric transformation parameter of the prediction target block using the motion vector 119 of the prediction target block output from the motion estimation unit 106 and the motion vector stored in the encoding control unit 126. . The motion vector stored in the encoding control unit 126 is a motion vector of an adjacent block, and is hereinafter referred to as “adjacent motion vector”.

  FIG. 8 is a block diagram showing a configuration of the geometric transformation parameter derivation unit 108. The geometric transformation parameter derivation unit 108 includes a motion vector acquisition unit 181 and a parameter derivation unit 182. The motion vector acquisition unit 181 determines an adjacent block from which motion information is acquired from among a plurality of adjacent blocks, and acquires motion information of the adjacent block, for example, a motion vector. The parameter deriving unit 182 derives a geometric transformation parameter from the motion vector of the adjacent block.

  Hereinafter, the process of deriving the adjacent motion vector by the motion vector acquisition unit 181 will be described with reference to FIGS. 9 to 11.

<< Derivation of Adjacent Block and Adjacent Motion Vector (Part 1)-When Block Size is Same >>
9A to 9E are diagrams illustrating the relationship between adjacent blocks with respect to a prediction target block. FIG. 9A shows an example in which the sizes of prediction target blocks and adjacent blocks (for example, 16 × 16 pixel blocks) match.

  In FIG. 9A, a pixel block p with hatching is a pixel block that has already been encoded or predicted (hereinafter referred to as a “predicted pixel block”). A block c with dot hatching indicates a prediction target block, and a pixel block n displayed in white is an uncoded pixel (unpredicted) block. In the figure, X represents an encoding (prediction) target pixel block.

  The adjacent block A is the adjacent block on the left of the prediction target block X, the adjacent block B is the adjacent block on the prediction target block X, the adjacent block C is the adjacent block on the upper right of the prediction target block X, and the adjacent block D is This is an adjacent block at the upper left of the prediction target block X.

  The adjacent motion vector held in the internal memory of the encoding control unit 126 is only the motion vector of the predicted pixel block. As shown in FIG. 4, since the pixel block is encoded and predicted from the upper left to the lower right, when the pixel block X is predicted, the right and lower pixel blocks are still encoded. Has not been made. Therefore, an adjacent motion vector cannot be derived from these adjacent blocks.

  9B to 9E are diagrams illustrating examples of adjacent blocks when the prediction target block is an 8 × 8 pixel block. In FIG. 9B to FIG. 9E, a bold line represents a macroblock boundary. 9B is a pixel block located at the upper left in the macro block, FIG. 9C is a pixel block located at the upper right in the macro block, FIG. 9D is a pixel block located at the lower left in the macro block, and FIG. 9E is a macro block. An example in which each pixel block located at the lower right in the block is a prediction target block is shown.

  Since the inside of the macroblock is similarly encoded from the upper left to the lower right, the position of the adjacent block changes according to the encoding order of the 8 × 8 pixel block. When the encoding process or the predicted image generation process for the corresponding 8 × 8 pixel block is completed, the pixel block becomes an encoded pixel block and is used as an adjacent block of the pixel block to be processed later. In FIG. 9E, since the upper right pixel block corresponding to the adjacent block C is an unencoded pixel block, the pixel block located at the upper right of the encoded pixel block is set as an adjacent block.

<< Derivation of Adjacent Block and Adjacent Motion Vector (Part 2)-When Block Sizes are Different >>
Next, the relationship between adjacent blocks when the block sizes of the adjacent block and the prediction target block are different will be described. When the block sizes of the prediction target block and the adjacent block are different, there are a plurality of adjacent pixel definitions.

  FIG. 10A to FIG. 10D are diagrams illustrating an example where the prediction target block is large and the adjacent block is small. FIG. 10A is an example in which a pixel block having a pixel closest to the upper left pixel of the prediction target block is set as an adjacent pixel. FIG. 10B is an example in which the pixel block located at the lower right of the pixel block adjacent to the prediction target block is set as the adjacent block.

  FIG. 10C is an example in which a pixel block existing at the center of a pixel block adjacent to the prediction target block is set as an adjacent block. In FIG. 10C, the center of the pixel block adjacent to the left of the prediction target block X is an 8 × 8 pixel block boundary. Thus, when the center position exists at the block boundary, the left pixel block is set as an adjacent block.

  FIG. 10D is an example in which the pixel block existing at the center of the pixel block adjacent to the prediction target block is an adjacent block. However, since the center point is located at the pixel block boundary as in FIG. 10C, This is an example in which a positioned pixel block is an adjacent block. When the center exists at the block boundary, any adjacent block may be defined as the central block, but the same definition is applied to all adjacent pixels. Thereby, when estimating the motion information of a prediction object block from the motion information of each adjacent block, the positional relationship with respect to an adjacent block can be expressed by the same phase.

  In the present embodiment, the definition of the adjacent block shown in FIG. 10A is used for simplicity.

  FIG. 11A to FIG. 11D are diagrams for explaining an example when the prediction target block is small and the block size of the adjacent block is large. Similarly to FIG. 9E, when the corresponding pixel block is an uncoded pixel block, it is replaced with a coded pixel block that can be used that is close in distance to the prediction target block.

  In the above description, the case of 16 × 16 pixels and 8 × 8 pixels has been described as an example, but a square pixel block such as 32 × 32 pixels, 4 × 4 pixels, or 16 × 8 pixels using the same framework. Adjacent blocks may also be determined for rectangular pixel blocks such as 8 × 16 pixels.

  In inter prediction, since encoding processing, that is, motion vector estimation, can be performed without depending on the encoding order in the macroblock, even in the case of an 8 × 8 pixel block, FIG. Adjacent blocks may be determined using any of 10D. Even when pixel blocks having different block sizes are mixed, adjacent blocks may be determined using any one of FIGS. 10A to 10D.

  In addition to using four pixel blocks A, B, C, and D as adjacent blocks, the adjacent blocks may be defined more widely. For example, a pixel block on the left of the adjacent block A may be used, or a pixel block further on the adjacent block B may be used. The definition of these adjacent blocks may be defined similarly to the definitions of FIGS. 9 to 11 described above.

≪Derivation of geometric transformation parameters≫
Next, a method for deriving the geometric transformation parameter 121 in the geometric transformation parameter deriving unit 108 will be described. The geometric transformation parameter 121 is executed by the parameter deriving unit 182. The adjacent motion vectors held by the adjacent blocks are defined by equations (4) to (7), respectively.

Also, the motion vector 119 provided from the motion estimation unit 106 is defined by equation (8). The motion vector 119 is a motion vector of the prediction target block X.

The geometric transformation parameter 121 is derived using the motion vector and the adjacent motion vector represented by the equations (4) to (8). When the geometric transformation is an affine transformation, the transformation formula is expressed by the following formula (9).

In equation (9), coordinates (x, y) are converted to coordinates (u, v) by affine transformation. Six parameters a, b, c, d, e, and f included in Expression (9) represent geometric transformation parameters. In the affine transformation, since these six types of parameters are estimated, six or more input values are required. When the motion vectors of the adjacent blocks A and B and the prediction target block X are used, geometric transformation parameters are derived by the following equation (10). Here, it is assumed that the motion vector is ¼ precision.

但し、ａｘ、ａｙは予測対象ブロックのサイズに基づく変数であり、次式で算出される。

However, ax and ay are variables based on the size of the prediction target block, and are calculated by the following equations.

Here, mb_size_x and mb_size_y indicate the horizontal and vertical sizes of the macroblock. In the case of a 16 × 16 pixel block, mb_size_x = 16 and mb_size_y = 16. Also, blk_size_x and blk_size_y represent the horizontal and vertical sizes of the prediction target block. In the case of FIG. 9B, blk_size_x = 8 and blk_size_y = 8.

  Here, an example in which the geometric transformation parameters are derived using the motion vectors of adjacent blocks A and B as input values is shown, but it is not always necessary to use the motion vectors of adjacent blocks A and B. Motion vectors calculated from D and other adjacent blocks may be used, or geometric transformation parameters may be obtained from the motion vectors of these adjacent blocks using parameter fitting. Further, in equation (10), a, b, d, and e are obtained as real numbers, respectively, but can be easily converted to integers by determining the calculation accuracy of these parameters in advance.

≪Derivation of adjacent motion vector-Edge consideration≫
When there is an edge of an object at the boundary with an adjacent block or when a different object exists, the geometric transformation parameter may not be correctly derived. Therefore, when the absolute difference value between the adjacent motion vector to be used and the motion vector of the prediction target block is greatly different, processing in which the motion vector of the adjacent block is not added to the input value may be performed. For _{example, |} mv a -mv _{x |} is calculated, and if greater than the threshold value D as defined previously, may not be utilized mv _a as input.

Further, when there are a plurality of adjacent block candidates located to the left of the prediction target block, a median value of motion vectors may be calculated from these candidates, and motion vectors having greatly different values may be excluded. FIG. 12 is a diagram illustrating an example in which a median value is calculated for each of four 8 × 8 pixel blocks. When the block size of the prediction target block X is large and the block size of adjacent pixel blocks is small, there are a plurality of pixel blocks that are candidates for adjacent pixels. Here, if the motion vectors of the four pixel blocks located on the left are mv _a , mv _b , mv _c , and mv _d , the motion vector is determined by the following equation (12).

式（１２）では、二次元ベクトルのメディアン値を用いる例を示したが、式（１３）に示すように、ベクトルの要素毎のメディアン値でもよい。

Although the example using the median value of the two-dimensional vector is shown in Expression (12), as shown in Expression (13), the median value for each vector element may be used.

In addition, an average value for each element, a random value of a two-dimensional vector, a random value for each element of the vector, or the like may be used. Also, the adjacent motion vectors may be obtained by performing the same processing in the adjacent blocks B, C, and D.

≪Division of prediction target pixel≫
Next, a region where geometric transformation is performed on the prediction target block will be described. The area from which the geometric transformation parameter is derived corresponds to the area where the geometric transformation is performed. Equation (10) shows an example in which a geometric transformation parameter is derived for a rectangular pixel block. However, it is not always necessary to derive the geometric transformation parameter by the rectangular pixel block, and the rectangular block may be divided by the triangular patch shown in FIG. 13A and 13B show an example in which the prediction target block is divided by a diagonal line and divided by two triangular patches. In FIG. 13A, each geometric transformation parameter is derived by the following equation. The prediction target triangular patch X1 is defined by the following equation (14) using the motion vectors of the adjacent blocks A and D and the prediction target block X1.

予測対象三角パッチＸ２は、隣接ブロックＢ、Ｄ及び予測対象ブロックＸ２の動きベクトルを用いて次式（１５）で定義される。

The prediction target triangular patch X2 is defined by the following equation (15) using the motion vectors of the adjacent blocks B and D and the prediction target block X2.

As in the examples shown in Expression (14) and Expression (15), the geometric transformation parameter is derived using the motion vector of the adjacent block having a spatial distance close to the prediction target triangular patch (or the prediction target block).

  In the case of FIG. 13B as well, geometric transformation parameters can be derived in the same manner. However, the prediction target triangular patch X2 has a spatial distance close to the uncoded pixel block side and a spatial distance from an adjacent block. In general, since the motion of an object has a high spatial correlation, it is preferable to define a division shape so that a large number of adjacent blocks can be used.

  In the present embodiment, an example in which a prediction target block is divided by two triangular patches has been described. However, the division shape may be further divided by a plurality of triangular patches, and may be rectangular, curved, trapezoidal, and parallel. You may divide | segment using a quadrilateral and these combination.

  In this embodiment, an example using affine transformation is shown as an example of geometric transformation, but any geometric transformation such as bilinear transformation, Helmart transformation, quadratic conformal transformation, projective transformation, three-dimensional projective transformation, etc. It may be used. For example, the projective transformation is expressed by the following equation (16).

In equation (16), if the numerator denominator is divided into scalars, there are eight parameters to be solved. Thus, by defining a large number of adjacent blocks that can be used, it is possible to derive geometric transformation parameters in the same framework as affine transformation.
The above is the outline of the processing of the geometric transformation parameter derivation unit 108.

<Geometric transformation prediction unit 109>
Next, the process of the geometric transformation prediction unit 109 will be specifically described. The geometric transformation prediction unit 109 performs geometric transformation on the reference image signal 120 based on the inputted geometric transformation parameter 121 to perform geometric transformation prediction. FIG. 14 is a block diagram illustrating an example of the configuration of the geometric transformation prediction unit 109. The geometric transformation prediction unit 109 includes a geometric transformation unit 191 and an interpolation unit 192. The geometric conversion unit 191 performs geometric conversion on the reference image signal 120 and calculates the position of the predicted pixel. The interpolation unit 192 calculates the predicted pixel value corresponding to the fractional position of the predicted pixel obtained by the geometric transformation by interpolation or the like.

  FIG. 15 is a diagram illustrating an example of geometric transformation prediction and motion compensation prediction for a prediction target block. FIG. 15 is an example of a 16 × 16 pixel block.

  In the figure, the prediction target block is a square pixel block CR composed of pixels indicated by triangles. The corresponding pixels of motion compensated prediction are indicated by black circles. A pixel block MER composed of pixels indicated by black circles is a square. On the other hand, pixels corresponding to the geometric transformation prediction are indicated by x, and a pixel block GTR composed of these pixels is a parallelogram.

  The region after motion compensation and the region after geometric transformation describe the corresponding region of the reference image signal according to the coordinates of the frame to be encoded. As described above, by using the geometric transformation prediction, it is possible to generate a predicted image signal in accordance with deformation such as rotation, enlargement / reduction, shearing, and mirror transformation of the rectangular pixel block.

  The geometric transformation unit 191 uses the geometric transformation parameter 121 calculated using the equations (10), (14), and (15), and uses the geometric transformation parameters (u, v) according to the equation (9). ) Is calculated. The calculated coordinates (u, v) after geometric transformation are real values. Therefore, the predicted value is generated by interpolating the luminance value corresponding to the coordinates (u, v) from the reference image signal.

  From Equation (10), Equation (14), and Equation (15), since the fractional position can be calculated without error by 6-bit calculation, the pixel accuracy at the time of interpolation is 6 bits. Thereby, the integer pixels are divided into 64 fractional pixels.

  FIG. 16 is a diagram illustrating an example of luminance value interpolation processing by bilinear interpolation. White circles cw0 to cw3 indicate luminance values at integer pixel positions, and black circles cb indicate interpolation pixel positions (u, v). In FIG. 16, the interpolated pixel value is generated from the ratio of the distances using the surrounding four integer pixel values adjacent to the fractional accuracy position. The bilinear interpolation method is expressed by the following equation (17).

Here, P (u, v) represents the predicted pixel value after the interpolation process, and R (x, y) represents the integer pixel value of the used reference image signal. Assuming that (x−u) = U / 64 and (y−v) = V / 64, Equation (17) can be transformed into an integer operation shown in Equation (18).

式（１８）において、ｆは丸めのオフセット（０≦ｆ＜２１２）を表している。本実施の形態ではｆ＝０としている。

In Expression (18), f represents a rounding offset (0 ≦ f <212). In this embodiment, f = 0.

  As described above, a new predicted image signal is generated by applying interpolation for each coordinate in the prediction target block subjected to geometric transformation.

  In the present embodiment, the bilinear interpolation method is used as the interpolation method. However, the closest interpolation method, the cubic convolution interpolation method, the linear filter interpolation method, the Lagrange interpolation method, Any interpolation method such as a spline interpolation method or a Lanczos interpolation method may be applied.

In this embodiment, an example of interpolation from the integer pixel position of the reference image signal has been described. However, when the motion estimation unit 106 has already held the interpolated image signal of the reference image signal 120. May reuse the interpolated image signal of fractional precision. For example, when the motion estimation unit 106 calculates a motion vector with ¼ pixel accuracy, if an enlarged reference image signal obtained by enlarging the reference image signal by four times is generated and held, the ¼ pixel An interpolation image with 1/64 pixel accuracy may be generated using an accuracy interpolation image. Thus, an interpolation image with 1/64 pixel accuracy can be generated by further performing interpolation interpolation processing with 1/16 accuracy from the 1/4 accuracy interpolation image. Note that the pixel accuracy of the interpolation process can be specified more finely. In this case, an interpolation process may be performed according to the designated interpolation accuracy.
The above is the outline of the processing of the geometric transformation prediction unit 109.

<Determination Parameter Deriving Unit 127>
Next, the determination parameter deriving unit 127 will be specifically described. The determination parameter deriving unit 127 uses the geometric transformation parameter 121 output from the geometric transformation parameter deriving unit 108 and the predicted image signal output from the motion compensation prediction unit 107 and the predicted image signal output from the geometric transformation prediction unit 109. The determination parameter 125 for determining which signal to output is generated and output to the prediction switching unit 111. When the geometric transformation is an affine transformation, the degree of geometric transformation can be evaluated using a parallel movement index, a rotation index, an enlargement / reduction index, a deformation index, and the like.

  A general moving image has a high correlation in the time direction. Since the motion vector is a value indicating the movement of an object between temporally different images, it is expected that the spatial correlation of motion is relatively high. Therefore, the prediction method is dynamically switched using these indexes as determination parameters. The parallel movement index is given by the following equation (19).

In Equation (19), cn represents the c component of the geometric transformation parameter of the adjacent block or the adjacent triangular patch obtained by dividing the prediction target block, that is, the triangular patch in the adjacent block of the same number in FIG. , Cx indicate the c component of the geometric transformation parameter of the adjacent block. When divided into triangular patches, the adjacent relationship with the divided triangular patches of the same shape is referred to. The same applies to the f component.

  FIG. 17 is a diagram illustrating an example of a pixel block change by affine transformation. In FIG. 17, coordinates (1, 0) and (0, 1) are converted into coordinates (a, d) and (b, c) by affine transformation, respectively, and rotated by θD from the center angle 45 ° of the vector before conversion. ing. Each of a, b, d, and e indicates a geometric transformation parameter obtained in the prediction target block. The rotation index is given by the following equation (20).

式（２０）において、ｓｇｎ（Ａ）は、Ａの符号を返す関数である。矩形ブロックの中心ベクトルがどの程度回転したかを表している。

In equation (20), sgn (A) is a function that returns the sign of A. This shows how much the center vector of the rectangular block has been rotated.

  The enlargement / reduction index corresponds to the area after the affine transformation shown in FIG. Therefore, an enlargement / reduction index is defined by the following equation.

式（２１）でＤｅｔ≒１となる場合は、更に次式を用いて回転指標を算出してもよい。

When Det≈1 in equation (21), the rotation index may be calculated using the following equation.

図１７のθCは次式（２３）の変形指標に対応しており、アフィン変換後の図形の変形角度を定義している。

17 corresponds to the deformation index of the following equation (23), and defines the deformation angle of the figure after the affine transformation.

  Further, the difference value between each component of the geometric transformation parameter calculated in the adjacent block and each component of the geometric transformation parameter calculated in the prediction target block shown in Expression (24) is also used as one parameter of the determination parameter.

  In addition, the following determination parameters may be used by using the prediction target block and the adjacent block index calculated using Expression (20), Expression (21), Expression (22), and Expression (23). .

For example, Detn is calculated from the geometric transformation parameter held by the adjacent block or the triangular patch of the adjacent block. A difference value between the index corresponding to the calculated geometric transformation parameter and the geometric transformation parameter is output to the prediction switching unit 111 as the determination parameter 125.
The above is the outline of the processing of the determination parameter deriving unit 127.

<Prediction switching unit 111-Dynamic switching of prediction>
Next, the prediction switching unit 111 will be specifically described. Based on the determination parameter 125 output from the determination parameter deriving unit 127, the prediction switching unit 111 generates prediction switching information 122 that describes which prediction image signal is used.

  First, a switching method in the case of using Expression (20) to Expression (23) will be described. The rotation index, the enlargement / reduction index, and the deformation index are indices that aim at the degree of rotation, enlargement, reduction, and deformation of the pixel block in addition to the parallel movement of only the motion vector. When the calculated geometric transformation parameter index is extremely large or small, it is expected that the estimation of the geometric transformation parameter is inappropriate, and a predicted image generated by geometric transformation prediction using this parameter The signal is expected to contain many errors. Therefore, when these indices exceed a predetermined threshold range, the prediction switching information 122 is generated so as not to use the prediction image signal of the geometric transformation prediction. As an example, the threshold range in the enlargement / reduction index of Expression (21) will be described. Det is an index indicating the degree of enlargement / reduction. The prediction switching information 122 is defined by the following formula (28).

Here, pred_flag represents the prediction switching information 122, and ThDet represents a threshold value. When Det exceeds the threshold range, pred_flag is 0, and motion compensation prediction is selected. On the other hand, in other cases, pred_flag is 1, and geometric transformation prediction is selected. Similar threshold determination is performed for other determination parameters, and prediction switching information 122 is generated.

  Next, a switching method when using Expression (19), Expression (24) to Expression (27) will be described. Since it is estimated that the geometric transformation parameters of the adjacent block and the prediction target block have a high spatial correlation, if each difference value exceeds the predetermined threshold value, the prediction image signal of the geometric transformation prediction is The prediction switching information 122 is generated so as not to be used.

  Next, a switching method when using the adjacent block and the encoding parameter of the prediction target block will be described. The quantization parameter included in the encoding parameter is a parameter that defines the quantization step size. When the quantization parameter is large, the transform coefficient is roughly quantized. As a result, the locally decoded reference image signal has a large error with respect to the input image signal, and the motion vector estimation accuracy decreases. In geometric transformation prediction, since a geometric transformation parameter is calculated using a motion vector, if the estimation accuracy of the motion vector is lowered, the calculation accuracy of the geometric transformation parameter is also lowered. Therefore, when the quantization parameter becomes large, processing for narrowing the threshold range of various determination parameters is performed. That is, the threshold value of each parameter is a decreasing function for the quantization parameter.

  In the present embodiment, an example in which the threshold range is controlled using the quantization parameter is shown, but other information included in the encoding parameter may be used. For example, information regarding the resolution of the encoded moving image, the type of the prediction target block, the Ref_idx of the reference image, the value of the motion vector, or the transform size may be used.

  Further, when all adjacent motion vectors of adjacent blocks and the motion vector of the prediction target block have the same value, the coordinates before and after geometric transformation do not change. In such a case, it is considered that the peripheral block is a parallel movement of the same object, so that it is not necessary to perform geometric transformation prediction. Therefore, when this condition is satisfied, the prediction switching information 122 has a value indicating that the geometric transformation prediction is not used.

Further, the threshold value used for each index may be a function that changes according to the value of the encoding parameter. For example, the threshold ThDet in Equation (28) is defined as a function that depends on the quantization parameter, and the function is such that the threshold decreases as the quantization parameter increases, thereby reducing the estimation accuracy. It is possible to effectively switch predictions.
The above is the outline of the processing of the prediction switching unit 111 and the prediction separation switch 110.

<Processing flow of inter prediction including geometric transformation prediction>
FIG. 18 is a flowchart showing the process of generating a predicted image signal by the inter prediction unit 130. When the processing is started (S501), the motion vector 119 of the prediction target block input from the outside of the inter prediction unit 130 is input to the geometric transformation parameter deriving unit 108. In response to this, the geometric transformation parameter derivation unit 108 determines an adjacent block corresponding to the prediction target block (S502).

  Next, the adjacent motion vector of the corresponding adjacent block is derived from the internal memory held by the encoding control unit 126 (S503). The geometric transformation parameter deriving unit 108 derives the geometric transformation parameter 121 using the derived adjacent motion vector and the motion vector of the prediction target block (S504).

  Using the derived geometric transformation parameter 121, the geometric transformation processing of the prediction target block is performed in the geometric transformation prediction unit 109 (S505). The geometric transformation prediction unit 109 generates a prediction image signal at a fractional pixel position newly calculated by geometric transformation by interpolation (S506).

  The geometric transformation parameter 121 calculated by the geometric transformation parameter deriving unit 108 is input to the determination parameter deriving unit 127, and the determination parameter 125 is calculated (S507). The calculated determination parameter 125 is input to the prediction switching unit 111, and the prediction switching information 122 is generated (S508).

  The geometric transformation parameter 121 calculated in the prediction target block is stored in an internal memory possessed in the encoding control unit 126 (S509). The motion vector 119 is stored in an internal memory possessed in the encoding control unit 126 (S510).

  According to the prediction switching information 122, the prediction separation switch 110 determines whether or not the prediction method of the prediction target block uses the predicted image signal generated by the geometric transformation prediction unit 109 (S512). When this determination is YES, the prediction separation switch 110 connects the output end of the geometric transformation prediction unit 109 to the switch, and outputs a predicted image signal (S513).

  On the other hand, when this determination is NO, the motion compensation prediction unit 107 performs motion compensation prediction processing according to the motion vector 119 (S514). The prediction separation switch 110 connects the switch to the output terminal of the motion compensation prediction unit 107 and outputs a prediction image signal (S515).

  Next, the encoding control unit 126 determines whether or not the prediction target block is the last block in the macroblock (S516). If this determination is YES, the prediction process for the macroblock is ended (S517). ). If this determination is NO, the process returns to the beginning, and a predicted image generation process for the pixel block next to the macroblock is performed. The above is the flow of the predicted image generation processing for inter prediction in the present embodiment of the present invention.

<Syntax structure>
Next, a syntax structure in the moving picture coding apparatus 100 will be described. FIG. 19 is a diagram illustrating a configuration of the syntax 1600. As shown in FIG. 19, the syntax 1600 mainly has three parts. The high-level syntax 1601 has higher layer syntax information that is equal to or higher than a slice. The slice level syntax 1602 has information necessary for decoding for each slice, and the macroblock level syntax 1603 has information necessary for decoding for each macroblock.

  Each part has a more detailed syntax. High level syntax 1601 includes sequence and picture level syntax, such as sequence parameter set syntax 1604 and picture parameter set syntax 1605. The slice level syntax 1602 includes a slice header syntax 1606, a slice data syntax 1607, and the like. The macroblock level syntax 1603 includes a macroblock layer syntax 1608, a macroblock prediction syntax 1609, and the like.

  FIG. 20 is a diagram illustrating an example of the slice header syntax 1605. The slice_affine_motion_prediction_flag shown in the figure is a syntax element indicating whether or not geometric transformation prediction is applied to the slice. When slice_affine_motion_prediction_flag is 0, the prediction switching unit 111 sets the prediction switching information 122 so as to always output the output terminal of the motion compensation prediction unit 107 in the slice, and switches the prediction separation switch 110. That is, this means that geometric transformation prediction is not applied to this slice. On the other hand, when slice_affine_motion_prediction_flag is 1, the prediction switching information 122 is set based on information indicated by the determination parameter 125 in the slice, and the prediction separation switch 110 dynamically switches the prediction image signal.

  FIG. 21 is a diagram illustrating an example of the slice data syntax 1606. Mb_skip_flag shown in the drawing is a flag indicating whether or not the macroblock is encoded in the skip mode. In the skip mode, transform coefficients, motion vectors, etc. are not encoded.

  AvailAffineMode is an internal parameter indicating whether geometric transformation prediction can be used in the macroblock. When AvailAffineMode is 0, it means that the prediction switching information 122 is set not to use the geometric transformation prediction by various values calculated by the determination parameter 125. Also, AvailAffineMode is 0 when the adjacent motion vector of the adjacent block and the motion vector of the prediction target block have the same value.

  On the other hand, when AvailAffineMode is 1, mb_affine_motion_skip_flag indicating whether to use geometric transformation prediction or motion compensation prediction is encoded. When mb_affine_motion_skip_flag is 1, it means that geometric transformation prediction is applied to the skip mode. When mb_affine_motion_skip_flag is 0, it means that motion compensation prediction is applied.

  FIG. 22 is a diagram illustrating an example of the macroblock layer syntax 1607. Mb_type shown in the figure indicates macroblock type information. That is, whether the current macroblock is intra-coded, inter-coded, what block shape is being predicted, whether the prediction direction is unidirectional prediction or bidirectional prediction, etc. Contains information. The mb_type is passed to the macroblock prediction syntax and the submacroblock prediction syntax indicating the syntax of the subblock in the macroblock.

  FIG. 23 is a diagram illustrating an example of macroblock prediction syntax. AvailAffineModeMb shown in the figure is an internal parameter indicating whether or not geometric transformation prediction can be used in the pixel block. When AvailAffineModeMb is 0, it means that the prediction switching information 122 is set not to use the geometric transformation prediction by various values calculated by the determination parameter 125. Also, AvailAffineModeMb is 0 when the adjacent motion vector of the adjacent block and the motion vector of the prediction target block have the same value.

  On the other hand, when AvailAffineModeMb is 1, mb_affine_pred_flag indicating whether to use geometric transformation prediction or motion compensation prediction is encoded. When mb_affine_pred_flag is 1, it means that geometric transformation prediction is applied to the pixel block. When mb_affine_pred_flag is 0, it means that motion compensation prediction is applied.

  NumMbPart () is an internal function that returns the number of block divisions specified in mb_type. It is 1 for a 16 × 16 pixel block, 2 for an 8 × 16 pixel block, and 2 for an 8 × 8 pixel block. In the case, 4 is output.

  FIG. 24 is a diagram illustrating an example of sub macroblock prediction syntax. AvailAffineModeSubMb shown in the figure is an internal parameter indicating whether or not geometric transformation prediction can be used in the pixel block. When AvailAffineModeSubMb is 0, it means that the prediction switching information 122 is set not to use the geometric transformation prediction by various values calculated by the determination parameter 125. The AvailAffineModeSubMb is also 0 when the adjacent motion vector of the adjacent block and the motion vector of the prediction target block have the same value.

  On the other hand, when AvailAffineModeSubMb is 1, mb_affine_pred_flag indicating whether to use geometric transformation prediction or motion compensation prediction is encoded. When mb_affine_pred_flag is 1, it means that geometric transformation prediction is applied to the pixel block. When mb_affine_pred_flag is 0, it means that motion compensation prediction is applied. NumSubMbPart () is an internal function that returns the number of block divisions defined in mb_type.

  FIG. 25 is a diagram illustrating an example of macroblock layer syntax as an example of an encoding parameter. Coded_block_pattern shown in the figure indicates whether a transform coefficient exists for each 8 × 8 pixel block. For example, when this value is 0, it means that there is no transform coefficient in the target block. Mb_qp_delta in the figure indicates information related to the quantization parameter. The difference value from the quantization parameter of the block encoded immediately before the target block is represented.

  Ref_idx_l0 and ref_idx_l1 in the figure indicate reference image indexes representing which reference image is used to predict the target block when inter prediction is selected. Mv_l0 and mv_l1 in the figure indicate motion vector information. In the figure, transform — 8 × 8_flag represents conversion information indicating whether or not the target block is 8 × 8 conversion.

It should be noted that syntax elements not defined in the present embodiment may be inserted between the rows in the syntax tables shown in FIGS. 19 to 25, and descriptions regarding other conditional branches may be included. Further, the syntax table may be divided into a plurality of tables, or a plurality of syntax tables may be integrated. Moreover, it is not always necessary to use the same term, and it may be arbitrarily changed depending on the form to be used. Further, each syntax element described in the macroblock layer syntax may be changed as specified in a macroblock data syntax described later.
The above is the description of the video encoding apparatus 100 according to the first embodiment.

(Second Embodiment: Addition of Intra Prediction)
FIG. 26 shows the structure of the moving picture coding apparatus 200 used in the second embodiment. In addition, the same code | symbol is attached | subjected to the block which has the same function as 1st Embodiment, and description is abbreviate | omitted here. In FIG. 26, in addition to the function of the moving picture coding apparatus 100, an intra prediction unit 201, a mode determination unit 202, and a prediction mode separation switch 203 are newly added.

  The intra prediction unit 201 generates a predicted image signal 123 using only the information in the screen based on the input reference image signal 120. As an example of the prediction mode in the intra prediction unit 201, H.264 H.264 intra prediction will be described. H. Pixel blocks used for H.264 intra prediction are shown in FIGS. 27A to 27C. FIG. 27A is a 16 × 16 pixel intra prediction, FIG. 27B is a 4 × 4 pixel intra prediction, and FIG. 27C is a pixel block used for 8 × 8 pixel intra prediction. H. H.264 defines these three intra predictions. In intra prediction, an interpolated pixel is created from a reference image signal 120 stored in the reference image memory 105, and a predicted value is generated by copying in the spatial direction.

<Mode determination unit 202>
Next, an outline of the mode determination unit 202 will be described. The mode determination unit 202 outputs the prediction mode switching information 204 to the prediction mode separation switch 203 according to the information of the slice that is currently encoded. The prediction mode switching information 204 describes information about which of the output terminal of the intra prediction unit 201 and the output terminal of the inter prediction unit 130 is connected to the switch.

  Next, the function of the mode determination unit 202 will be described. When the currently encoded slice is an intra-coded slice, the mode determination unit 202 connects the output terminal of the prediction mode separation switch 203 to the intra prediction unit 201. On the other hand, when the currently encoded slice is an inter encoded slice, the mode determination unit 202 determines whether the prediction mode separation switch 203 is connected to the output end of the intra prediction unit 201 or the output end of the inter prediction unit 130. judge.

  More specifically, in the above case, the mode determination unit 202 performs mode determination using the cost of the following equation (29), for example. The code amount related to the prediction information required when each prediction mode is selected, for example, the code amount of the motion vector 119 or the code amount of the block shape is OH, the absolute difference sum of the input image signal 114 and the predicted image signal 123, that is, When the absolute cumulative sum of the prediction error signal 115 is SAD, the following mode determination formula (29) is used.

  Here, K represents a cost, and λ represents a constant. λ is a Lagrangian undetermined multiplier determined based on the quantization scale and the value of the quantization parameter. Mode determination is performed based on the cost K obtained by the equation (29). That is, the mode that gives the smallest value of cost K is selected as the optimal prediction mode.

  The mode determination unit 202 may perform mode determination using (a) only the prediction information and (b) only SAD instead of the equation (29), and the Hadamard may be included in these (a) and (b). You may use the value which performed conversion, or the value approximated to it. Further, the mode determination unit 202 may create the cost by using the activity of the input image signal 114, that is, the variance of the signal value, or by creating a cost function using the quantization scale or the quantization parameter. Also good.

  As yet another example, a provisional encoding unit is prepared, and the amount of codes when the prediction error signal 115 actually generated by the provisional encoding unit in a certain prediction mode is encoded, the input image signal 114 and the decoded image signal. The mode determination may be performed using a square error with respect to 118. The mode judgment formula in this case is the following formula (30).

In Equation (30), J is the encoding cost, and D is the encoding distortion representing the square error between the input image signal 114 and the decoded image signal 118. On the other hand, R represents a code amount estimated by provisional encoding.

  When the encoding cost J of Expression (30) is used, provisional encoding and local decoding processing are required for each prediction mode, so that the circuit scale or the amount of calculation increases. On the other hand, since a more accurate code amount and encoding distortion are used, high encoding efficiency can be maintained. The cost may be calculated using only R or only D instead of Expression (30), and the cost function may be created using a value approximating R or D.

As described above, it is determined whether to select the prediction image signal generated by the intra prediction unit 201 or the prediction image signal generated by the inter prediction unit 130, and the output terminal of the prediction mode separation switch 203 is switched. . The prediction image signal 123 of the prediction mode selected here is output and input to the subtracter 101 and output to the adder 104.
The above is description of the process of the moving image encoder 200 which concerns on this Embodiment of this invention.
(Third embodiment: Re-search for motion vectors)

  FIG. 28 shows the structure of the inter prediction unit 300 used in the third embodiment. In addition, the same code | symbol is attached | subjected to the block which has the same function as 1st Embodiment, and description is abbreviate | omitted here. Further, since the input / output signals of the inter prediction unit 300 are the same as those of the inter prediction unit 130, the inter prediction unit 300 can be replaced with the inter prediction unit 130 included in the video encoding device 100 and the video encoding device 200.

  In FIG. 28, in addition to the function of the inter prediction unit 300, a motion vector re-search unit 301 is newly added. The motion vector re-search unit 301 further re-searches the motion vectors in the peripheral part with the input motion vector 119 as a reference.

  The motion vector 119 of the prediction target block inputted from the outside and the adjacent pixel vector held in the encoding control unit 126 are inputted to the geometric transformation parameter deriving unit 108, and the geometric transformation parameter 121 is calculated. The geometric transformation parameter 121 is input to the geometric transformation prediction unit 109 to perform geometric transformation prediction, and a predicted image signal is generated.

  The motion vector re-search unit 301 changes the motion vector of the block to be predicted within the search range given from the encoding control unit 126 with the input motion vector 119 as a reference, and re-derived the geometric transformation parameter. The re-derived geometric transformation parameters are input to the geometric transformation prediction unit 109, re-geometric transformation prediction is performed, and a predicted image signal is generated. In this way, using the motion vector of the prediction target block generated by the motion vector re-search unit 301, geometric conversion prediction for the re-search range is performed.

  In the re-search, the cost is calculated using the formula (29) or the formula (30), and only the motion vector and the prediction image signal with the low cost are left. When the geometric transformation prediction within the re-search range is completed, a motion vector and a predicted image signal that give the minimum cost are output to the prediction separation switch 110. Coding efficiency can be improved by changing the motion vector of the prediction target block so that the prediction error becomes small.

(Fourth embodiment: Signaling the difference of motion vectors)
The configuration of the moving picture encoding apparatus according to the fourth embodiment is the same as that of the third embodiment. In addition to the operation of the video encoding device in the third embodiment, the video encoding device according to the fourth embodiment adds the difference between the motion vector of the prediction target block and the motion vector calculated by re-searching. Is encoded. By using the motion vector re-searched by the motion vector re-search unit 301 for the geometric transformation motion compensated prediction, the original motion vector is changed to reduce the prediction error. If the motion vector after this re-search is used as an adjacent motion vector, the subsequent geometric transformation parameter may not be derived properly.

  Therefore, by encoding the original motion vector and the amount of deviation therefrom separately, the re-searched motion vector necessary for derivation of the geometric transformation parameter and the original motion vector required as the adjacent motion vector are obtained. Retained. This encoding process is preferably performed by the entropy encoding unit 112.

The syntax change according to the present embodiment is shown in FIGS. Of the characters included in the syntax element, L0 and l0 indicate motion vectors indicated on the reference image L0, and L1 and l1 indicate motion vectors indicated on the reference image L1. If the original motion vector before the re-search is mv _org = (mvx _org , mvy _org ) and the motion vector after the re-search is mv _refine = (mvx _refine , mvy _refine ), the motion vector difference mvd_affine [2] Is calculated by the following equation.

Note that mvd_affine [0] is a motion vector difference value corresponding to the vertical direction and mvd_affine [1] is corresponding to a syntax element. Note that here, the characters l0 and l1 included in the syntaxes shown in FIGS. 29 and 30 are omitted, but the motion vector difference shown in the equation (31) is different from the indexes L0 and L1. To calculate. Also, mvd_l0 and mvd_l1 are calculated by calculating the difference between the motion vector corresponding to each reference image signal and the value predicted from the motion vector, and other motion vector prediction techniques not explicitly described in this embodiment are used. Generated using.

When the encoding of the prediction target block is completed, mv _org = (mvx _org , mvy _org ), which is a motion vector before re-searching, is stored in the internal memory of the encoding control unit 126. In this way, by separately holding the motion vector used in the prediction target block and the motion vector used when it becomes an adjacent block, it is possible to prevent propagation of errors in geometric transformation parameters from the adjacent block Become.

  As described above, in the first to fourth embodiments, when predicting an object having motion that is not suitable for a rectangular block, excessive block division is performed and block division information is increased. prevent. The effect of improving coding efficiency and further improving subjective image quality by separating the motion area and background area in the block without applying additional information and applying the optimal prediction method to each. Play.

<Video decoding device>
Next, fifth to seventh embodiments relating to video decoding will be described.

(Fifth embodiment)
FIG. 31 shows a video decoding device according to the fourth embodiment. For example, the moving picture decoding apparatus 400 in FIG. 31 decodes encoded data generated by the moving picture encoding apparatus according to the first embodiment.

  31 decodes the encoded data 411 stored in the input buffer 401 and outputs a decoded image signal 420 to the output buffer 419. The encoded data 411 is encoded data that is transmitted from, for example, the moving image encoding apparatus 100, transmitted through a storage system or a transmission system, once stored in the input buffer 401, and multiplexed.

  The moving picture decoding apparatus 400 includes an encoded data decoding unit 402, an inverse quantization / inverse transform unit 403, an adder 404, a reference image memory 405, a motion compensation prediction unit 406, a geometric transformation parameter derivation unit 407, and a geometric transformation prediction unit. 408, a determination parameter deriving unit 422, a prediction switching unit 409, and a prediction separation switch 410. The video decoding device 400 is also connected to the input buffer 401, the output buffer 419, and the decoding control unit 421.

  The encoded data decoding unit 402 decodes the encoded data by syntax analysis based on the syntax for each frame or field. The encoded data decoding unit 402 sequentially entropy-decodes the code string of each syntax, and reproduces the motion vector 415, the encoding parameter of the target block, and the like. The encoding parameter is a parameter required for decoding prediction information, information on transform coefficients, information on quantization, and the like.

  The transform coefficient decoded by the encoded data decoding unit 402 is input to the inverse quantization / inverse transform unit 403. Various information relating to the quantization decoded by the encoded data decoding unit 402, that is, the quantization parameter and the quantization matrix are set in the internal memory of the decoding control unit 421 and used as an inverse quantization process. Loaded.

  The inverse quantization / inverse transform unit 403 first performs inverse quantization processing using the loaded information regarding quantization. Subsequently, the inverse quantized transform coefficient is subjected to inverse transform processing, for example, inverse discrete cosine transform. Here, the inverse orthogonal transform has been described. However, when wavelet transform or the like is performed in the encoding device, the inverse quantization / inverse transform unit 403 executes the corresponding inverse quantization and inverse wavelet transform. Good.

  The restored prediction error signal 412 is input to the adder 404 through the inverse quantization / inverse transform unit 403. The adder 404 adds a prediction error signal 412 and a predicted image signal 418 generated by a motion compensation prediction unit 406 or a geometric transformation prediction unit 408 described later to generate a decoded image signal 420.

  The generated decoded image signal 420 is output from the moving image decoding apparatus 400, temporarily stored in the output buffer 419, and then output according to the output timing managed by the decoding control unit 421. The decoded image signal 420 is stored in the reference image memory 405 and becomes a reference image signal 413.

  The reference image signal 413 is sequentially read from the reference image memory 405 for each frame or each field, and is input to the prediction motion compensation prediction unit 406 or the geometric transformation prediction unit 408. The motion vector 415 and the geometric transformation parameter 414 used in the target pixel block are stored in the decoding control unit 421 and are appropriately loaded and used by the geometric transformation parameter deriving unit 108 and the determination parameter deriving unit 422 described later.

  Note that the motion compensation prediction unit 406, the geometric transformation parameter derivation unit 407, the geometric transformation prediction unit 408, the determination parameter derivation unit 422, the prediction switching unit 409, and the prediction separation switch 410 in FIG. 31 have the same names shown in FIG. It has the same function and configuration as each part. More specifically, the motion vector input to the motion compensation prediction unit 107 is acquired by the motion estimation unit 106, whereas the motion vector input to the motion compensation prediction unit 406 is encoded data decoding. All are the same except that they are decrypted by the unit 402.

  32 includes the motion compensation prediction unit 406, the geometric transformation parameter derivation unit 407, the geometric transformation prediction unit 408, the determination parameter derivation unit 422, the prediction switching unit 409, and the prediction separation switch 410 in FIG. FIG. Each of these units has the same function and configuration as each unit of the inter prediction unit 130 shown in FIG. Therefore, when the moving picture decoding apparatus 400 holds the inter prediction unit 130 included in the moving picture encoding apparatus 100, the configuration illustrated in FIG. 31 can be realized.

<Geometric transformation parameter derivation unit 407>
In the geometric transformation parameter deriving unit 407, the motion vector 415 of the prediction target block decoded by the encoded data decoding unit 402 and the motion vector stored in the decoding control unit 421 (hereinafter referred to as “adjacent motion vector”). The geometric transformation parameter of the prediction target block is derived by using this.

  The relationship between adjacent blocks with respect to the prediction target block will be described with reference to FIGS. 4 and 9 to 13. In the description of FIG. 4 and FIGS. 9 to 13, “blocks to be encoded or predicted”, “pixel blocks for which encoding or prediction has been completed” and “uncoded” in the first to fourth embodiments. In the fifth to seventh embodiments, “decoded or unpredicted pixel block”, “decoded or predicted target block”, “decoded or predicted pixel block” and “undecoded or unpredicted pixel block”, respectively. It is read as “unpredicted pixel block”.

<< Derivation of Adjacent Block and Adjacent Motion Vector (Part 1)-When Block Size is Same >>
9A to 9E are diagrams for explaining the relationship between adjacent blocks with respect to a prediction target block. FIG. 9A shows an example in which the sizes of prediction target blocks and adjacent blocks (for example, 16 × 16 pixel blocks) match.

  In FIG. 9A, a pixel block p with hatched hatching is a pixel block that has already been decoded or predicted (hereinafter referred to as “predicted pixel block”), and has been hatched with dots. The block c is a prediction target block, and the pixel block n displayed in white is an undecoded pixel (unpredicted) block. In the figure, X represents a decoding (prediction) target pixel block.

  The adjacent block A is the adjacent block on the left of the prediction target block X, the adjacent block B is the adjacent block on the prediction target block X, the adjacent block C is the adjacent block on the upper right of the prediction target block X, and the adjacent block D is This is an adjacent block at the upper left of the prediction target block X.

  The adjacent motion vector held in the internal memory of the decoding control unit 421 is only the motion vector of the predicted pixel block. In FIG. 4, in the decoded frame f subjected to the decoding process, a block located on the left and above the block c to be decoded is a decoded block p. As shown in FIG. 4, since the pixel block is decoded and predicted from the upper left to the lower right, when the pixel block X is predicted, the right and lower pixel blocks are still decoded. Has not been made. Therefore, an adjacent motion vector cannot be derived from these adjacent blocks.

  9B to 9E are diagrams illustrating examples of adjacent blocks when the prediction target block is an 8 × 8 pixel block. In FIG. 9B to FIG. 9E, a bold line represents a macroblock boundary. 9B is a pixel block located at the upper left in the macro block, FIG. 9C is a pixel block located at the upper right in the macro block, FIG. 9D is a pixel block located at the lower left in the macro block, and FIG. 9E is a macro block. An example in which each pixel block located at the lower right in the block is a prediction target block is shown.

  Similarly, the decoding process is performed from the upper left to the lower right inside the macro block, so that the position of the adjacent block changes according to the decoding order of the 8 × 8 pixel block. When the decoding process or predicted image generation process for the corresponding 8 × 8 pixel block is completed, the pixel block becomes a decoded pixel block and is used as an adjacent block of the pixel block to be processed later. In FIG. 9E, since the upper right pixel block corresponding to the adjacent block C is an undecoded pixel block, the pixel block located at the upper right of the decoded pixel block is set as the adjacent block.

<< Derivation of Adjacent Block and Adjacent Motion Vector (Part 2)-When Block Sizes are Different >>
Next, the relationship between adjacent blocks when the block sizes of the adjacent block and the prediction target block are different will be described. When the block sizes of the prediction target block and the adjacent block are different, there are a plurality of adjacent pixel definitions.

  FIG. 10A to FIG. 10D are diagrams illustrating an example where the prediction target block is large and the adjacent block is small. FIG. 10A is an example in which a pixel block having a pixel closest to the upper left pixel of the prediction target block is set as an adjacent pixel. FIG. 10B is an example in which the pixel block located at the lower right of the pixel block adjacent to the prediction target block is set as the adjacent block.

  FIG. 10C is an example in which a pixel block existing at the center of a pixel block adjacent to the prediction target block is set as an adjacent block. In FIG. 10C, the center of the pixel block adjacent to the left of the prediction target block X is an 8 × 8 pixel block boundary. Thus, when the center position exists at the block boundary, the left pixel block is set as an adjacent block.

  FIG. 10D is an example in which the pixel block existing at the center of the pixel block adjacent to the prediction target block is an adjacent block. However, since the center point is located at the pixel block boundary as in FIG. 10C, This is an example in which a positioned pixel block is an adjacent block. When the center exists at the block boundary, any adjacent block may be defined as the central block, but the same definition is applied to all adjacent pixels. Thereby, when estimating the motion information of a prediction object block from the motion information of each adjacent block, the positional relationship with respect to an adjacent block can be expressed by the same phase.

  In the present embodiment, the definition of the adjacent block shown in FIG. 10A is used for simplicity.

  FIG. 11A to FIG. 11D are diagrams for explaining an example when the prediction target block is small and the block size of the adjacent block is large. Similarly to FIG. 9E, when the corresponding pixel block is an undecoded pixel block, it is replaced with an available decoded pixel block that is close in distance to the prediction target block.

  In the above description, the case of 16 × 16 pixels and 8 × 8 pixels has been described as an example, but a square pixel block such as 32 × 32 pixels, 4 × 4 pixels, or 16 × 8 pixels using the same framework. Adjacent blocks may also be determined for rectangular pixel blocks such as 8 × 16 pixels.

  In inter prediction, since it is possible to perform decoding processing, that is, motion vector estimation, without depending on the decoding order in the macroblock, even in the case of an 8 × 8 pixel block, FIG. Adjacent blocks may be determined using any of 10D. Even when pixel blocks having different block sizes are mixed, adjacent blocks may be determined using any one of FIGS. 10A to 10D.

  In addition to using four pixel blocks A, B, C, and D as adjacent blocks, the adjacent blocks may be defined more widely. For example, a pixel block on the left of the adjacent block A may be used, or a pixel block further on the adjacent block B may be used. The definition of these adjacent blocks may be defined similarly to the definitions of FIGS. 9 to 11 described above.

≪Derivation of geometric transformation parameters≫
Next, a method for deriving the geometric transformation parameter 414 in the geometric transformation parameter deriving unit 407 will be described. The adjacent motion vectors held by the adjacent blocks are defined by equations (4) to (7), respectively.

  Also, the motion vector 119 provided from the motion estimation unit 106 is defined by equation (8). The motion vector 119 is a motion vector of the prediction target block X.

  The geometric transformation parameter 121 is derived using the motion vector and the adjacent motion vector represented by the equations (4) to (8). When the geometric transformation is an affine transformation, the transformation formula is expressed by Equation (9). In equation (9), coordinates (x, y) are converted to coordinates (u, v) by affine transformation. Six parameters a, b, c, d, e, and f included in Expression (9) represent geometric transformation parameters. In the affine transformation, since these six types of parameters are estimated, six or more input values are required.

  When the motion vectors of the adjacent blocks A and B and the prediction target block X are used, a geometric transformation parameter is derived by Expression (10). Here, it is assumed that the motion vector is ¼ precision. However, ax and ay are variables based on the size of the prediction target block, and are calculated by Expression (11).

  In Expression (11), mb_size_x and mb_size_y indicate the size of the macroblock in the horizontal and vertical directions. In the case of a 16 × 16 pixel block, mb_size_x = 16 and mb_size_y = 16. Also, blk_size_x and blk_size_y represent the horizontal and vertical sizes of the prediction target block. In the case of FIG. 9B, blk_size_x = 8 and blk_size_y = 8.

  Here, an example in which the geometric transformation parameters are derived using the motion vectors of adjacent blocks A and B as input values is shown, but it is not always necessary to use the motion vectors of adjacent blocks A and B. Motion vectors calculated from D and other adjacent blocks may be used, or geometric transformation parameters may be obtained from the motion vectors of these adjacent blocks using parameter fitting. Further, in equation (10), a, b, d, and e are obtained as real numbers, respectively, but can be easily converted to integers by determining the calculation accuracy of these parameters in advance.

≪Derivation of adjacent motion vector-Edge consideration≫
When there is an edge of an object at the boundary with an adjacent block or when a different object exists, the geometric transformation parameter may not be correctly derived. Therefore, when the absolute difference value between the adjacent motion vector to be used and the motion vector of the prediction target block is greatly different, processing in which the motion vector of the adjacent block is not added to the input value may be performed. For _{example, |} mv a -mv _{x |} is calculated, and if greater than the threshold value D as defined previously, may not be utilized mv _a as input.

Further, when there are a plurality of adjacent block candidates located to the left of the prediction target block, a median value of motion vectors may be calculated from these candidates, and motion vectors having greatly different values may be excluded. FIG. 12 is a diagram illustrating an example in which a median value is calculated for each of four 8 × 8 pixel blocks. When the block size of the prediction target block X is large and the block size of adjacent pixel blocks is small, there are a plurality of pixel blocks that are candidates for adjacent pixels. Here, assuming that the motion vectors of the four pixel blocks located on the left are mv _a , mv _b , mv _c , and mv _d , the motion vector is determined by Expression (12).

  Although the example using the median value of the two-dimensional vector is shown in Expression (12), as shown in Expression (13), the median value for each vector element may be used. In addition, an average value for each element, a random value of a two-dimensional vector, a random value for each element of the vector, or the like may be used. Also, the adjacent motion vectors may be obtained by performing the same processing in the adjacent blocks B, C, and D.

≪Division of prediction target pixel≫
Next, a region where geometric transformation is performed on the prediction target block will be described. The area from which the geometric transformation parameter is derived corresponds to the area where the geometric transformation is performed. Equation (10) shows an example in which a geometric transformation parameter is derived for a rectangular pixel block. However, it is not always necessary to derive the geometric transformation parameter by the rectangular pixel block, and the rectangular block may be divided by the triangular patch shown in FIG. 13A and 13B show an example in which the prediction target block is divided by a diagonal line and divided by two triangular patches. In FIG. 13A, each geometric transformation parameter is derived by the following equation. The prediction target triangular patch X1 is defined by Expression (14) using the motion vectors of the adjacent blocks A and D and the prediction target block X1.
The prediction target triangular patch X2 is defined by Expression (15) using the motion vectors of the adjacent blocks B and D and the prediction target block X2.

  As in the examples shown in Expression (14) and Expression (15), the geometric transformation parameter is derived using the motion vector of the adjacent block having a spatial distance close to the prediction target triangular patch (or the prediction target block).

  In the case of FIG. 13B as well, geometric transformation parameters can be derived in the same manner. However, the prediction target triangular patch X2 has a spatial distance close to the undecoded pixel block side and a spatial distance from an adjacent block. In general, since the motion of an object has a high spatial correlation, it is preferable to define a division shape so that a large number of adjacent blocks can be used.

  In the present embodiment, an example in which a prediction target block is divided by two triangular patches has been described. However, the division shape may be further divided by a plurality of triangular patches, and may be rectangular, curved, trapezoidal, and parallel. You may divide | segment using a quadrilateral and these combination.

  In this embodiment, an example using affine transformation is shown as an example of geometric transformation, but any geometric transformation such as bilinear transformation, Helmart transformation, quadratic conformal transformation, projective transformation, three-dimensional projective transformation, etc. It may be used. For example, the projective transformation is expressed by Expression (16).

  In equation (16), if the numerator denominator is divided into scalars, there are eight parameters to be solved. Thus, by defining a large number of adjacent blocks that can be used, it is possible to derive geometric transformation parameters in the same framework as affine transformation.

  The above is the outline of the processing of the geometric transformation parameter deriving unit 407.

<Geometric transformation prediction unit 408>
Next, processing of the geometric transformation prediction unit 408 will be described with reference to FIGS. 15 to 17. In the description of FIGS. 15 to 17, the “reference image signal 120” in the first to fourth embodiments is a locally decoded image, whereas in the fifth to seventh embodiments, The “reference image signal 413” is a decoded image.

  The geometric transformation prediction unit 408 performs geometric transformation on the reference image signal 413 based on the input geometric transformation parameter 414. FIG. 15 is a diagram illustrating an example of geometric transformation prediction and motion compensation prediction for a prediction target block. FIG. 15 is an example of a 16 × 16 pixel block.

  In the figure, the prediction target block is a square pixel block CR composed of pixels indicated by triangles. The corresponding pixels of motion compensated prediction are indicated by black circles. A pixel block MER composed of pixels indicated by black circles is a square. On the other hand, pixels corresponding to the geometric transformation prediction are indicated by x, and a pixel block GTR composed of these pixels is a parallelogram.

  The region after motion compensation and the region after geometric transformation describe the corresponding region of the reference image signal according to the coordinates of the frame to be decoded. As described above, by using the geometric transformation prediction, it is possible to generate a predicted image signal in accordance with deformation such as rotation, enlargement / reduction, shearing, and mirror transformation of the rectangular pixel block.

  The geometric transformation prediction unit 109 uses the geometric transformation parameters 414 calculated using the equations (10), (14), and (15), and uses the geometric transformation parameters (u, v) is calculated. The calculated coordinates (u, v) after geometric transformation are real values. Therefore, the predicted value is generated by interpolating the luminance value corresponding to the coordinates (u, v) from the reference image signal.

  From Equation (10), Equation (14), and Equation (15), since the fractional position can be calculated without error by 6-bit calculation, the pixel accuracy at the time of interpolation is 6 bits. Thereby, the integer pixels are divided into 64 fractional pixels.

  FIG. 16 is a diagram illustrating an example of luminance value interpolation processing by bilinear interpolation. White circles cw0 to cw3 indicate luminance values at integer pixel positions, and black circles cb indicate interpolation pixel positions (u, v). In FIG. 16, the interpolated pixel value is generated from the ratio of the distances using the surrounding four integer pixel values adjacent to the fractional accuracy position. The bilinear interpolation method is expressed by Expression (17).

In Expression (17), P (u, v) represents the predicted pixel value after the interpolation process, and R (x, y) represents the integer pixel value of the used reference image signal. Assuming that (x−u) = U / 64 and (y−v) = V / 64, Equation (17) can be transformed into an integer operation shown in Equation (18). In Expression (18), f represents a rounding offset (0 ≦ f <212). In this embodiment, f = 0.
As described above, a new predicted image signal is generated by applying interpolation for each coordinate in the prediction target block subjected to geometric transformation.

  In the present embodiment, the bilinear interpolation method is used as the interpolation method. However, the closest interpolation method, the cubic convolution interpolation method, the linear filter interpolation method, the Lagrange interpolation method, Any interpolation method such as a spline interpolation method or a Lanczos interpolation method may be applied.

  In this embodiment, an example of interpolation from the integer pixel position of the reference image signal 413 has been described. However, the motion compensation prediction unit 406 has already generated an interpolated image signal of the reference image signal 413. In this case, the interpolated image signal with fractional accuracy may be reused. For example, the encoded data decoding unit 402 decodes a motion vector with ¼ pixel accuracy, and the motion compensation prediction unit 406 generates an enlarged reference image signal obtained by enlarging the reference image signal 413 corresponding to the motion vector by four times. When generated and held, an interpolation image with 1/64 pixel accuracy may be generated using an interpolation image with 1/4 pixel accuracy. Thus, an interpolation image with 1/64 pixel accuracy can be generated by further performing interpolation interpolation processing with 1/16 accuracy from the 1/4 accuracy interpolation image.

Note that the pixel accuracy of the interpolation process can be specified more finely. In this case, an interpolation process may be performed according to the designated interpolation accuracy.
The above is the outline of the process of the geometric transformation prediction unit 408.

<Determination Parameter Deriving Unit 422>
Next, the determination parameter derivation unit 422 will be specifically described. The determination parameter deriving unit 422 uses the geometric transformation parameter 414 output from the geometric transformation parameter deriving unit 407 and the predicted image signal output from the motion compensation prediction unit 406 and the predicted image signal output from the geometric transformation prediction unit 408. The determination parameter 416 for determining which signal to output is generated and output to the prediction switching unit 409. When the geometric transformation is an affine transformation, the degree of geometric transformation can be evaluated using a parallel movement index, a rotation index, an enlargement / reduction index, a deformation index, and the like.

  A general moving image has a high correlation in the time direction. Since the motion vector is a value indicating the movement of an object between temporally different images, it is expected that the spatial correlation of motion is relatively high. Therefore, the prediction method is dynamically switched using these indexes as determination parameters. The translation index is given by equation (19).

  In Equation (19), cn represents the c component of the geometric transformation parameter of the adjacent block or the adjacent triangular patch obtained by dividing the prediction target block, that is, the triangular patch in the adjacent block of the same number in FIG. , Cx indicate the c component of the geometric transformation parameter of the adjacent block. When divided into triangular patches, the adjacent relationship with the divided triangular patches of the same shape is referred to. The same applies to the f component.

  FIG. 17 is a diagram illustrating an example of a pixel block change by affine transformation. In FIG. 17, coordinates (1, 0) and (0, 1) are converted into coordinates (a, d) and (b, c) by affine transformation, respectively, and rotated by θD from the center angle 45 ° of the vector before conversion. ing. Each of a, b, d, and e indicates a geometric transformation parameter obtained in the prediction target block. The rotation index is given by equation (20). In equation (20), sgn (A) is a function that returns the sign of A. This shows how much the center vector of the rectangular block has been rotated.

  The enlargement / reduction index corresponds to the area after the affine transformation shown in FIG. Therefore, an enlargement / reduction index is defined by equation (21). When Det≈1 in equation (21), the rotation index may be calculated using equation (22). ΘC in FIG. 17 corresponds to the deformation index of equation (23), and defines the deformation angle of the figure after the affine transformation.

  Further, the difference value between each component of the geometric transformation parameter calculated in the adjacent block and each component of the geometric transformation parameter calculated in the prediction target block shown in Expression (24) is also used as one parameter of the determination parameter.

  Also, the determinations shown in Expression (25) to Expression (27) using the prediction target block and the adjacent block index calculated using Expression (20), Expression (21), Expression (22), and Expression (23). Parameters may be used.

For example, Detn is calculated from the geometric transformation parameter 414 held by the adjacent block or the triangular patch of the adjacent block. A difference value between the index corresponding to the calculated geometric transformation parameter and the geometric transformation parameter is output to the prediction switching unit 409 as the determination parameter 416.
The above is the outline of the processing of the determination parameter deriving unit 422.

<Prediction switching unit 409-Dynamic switching of prediction>
Next, the prediction switching unit 409 will be specifically described. The prediction switching unit 409 generates prediction switching information 417 that describes which prediction image signal is used based on the determination parameter 416 output from the determination parameter deriving unit 422.

  First, a switching method in the case of using Expression (20) to Expression (23) will be described. The rotation index, the enlargement / reduction index, and the deformation index are indices that aim at the degree of rotation, enlargement, reduction, and deformation of the pixel block in addition to the parallel movement of only the motion vector. When the calculated geometric transformation parameter index is extremely large or small, it is expected that the estimation of the geometric transformation parameter is inappropriate, and a predicted image generated by geometric transformation prediction using this parameter The signal is expected to contain many errors. Therefore, when these indices exceed a predetermined threshold range, the prediction switching information 417 is generated so as not to use the predicted image signal of the geometric transformation prediction.

  As an example, the threshold range in the enlargement / reduction index of Expression (21) will be described. Det is an index indicating the degree of enlargement / reduction. Prediction switching information 417 is defined by equation (28).

  In formula (28), pred_flag represents the prediction switching information 417, and ThDet represents a threshold value. When Det exceeds the threshold range, pred_flag is 0, and motion compensation prediction is selected. On the other hand, in other cases, pred_flag is 1, and geometric transformation prediction is selected. Similar threshold determination is performed for other determination parameters, and prediction switching information 417 is generated.

  Next, a switching method when using Expression (19), Expression (24) to Expression (27) will be described. Since it is estimated that the geometric transformation parameters of the adjacent block and the prediction target block have a high spatial correlation, if each difference value exceeds the predetermined threshold value, the prediction image signal of the geometric transformation prediction is Prediction switching information 417 is generated so as not to be used.

  Next, a switching method when using the adjacent block and the encoding parameter of the prediction target block will be described. The quantization parameter included in the encoding parameter is a parameter that defines the quantization step size. When the quantization parameter is large, the transform coefficient is roughly quantized. As a result, the locally decoded reference image signal has a large error with respect to the input image signal, and the motion vector estimation accuracy decreases. In geometric transformation prediction, since a geometric transformation parameter is calculated using a motion vector, if the estimation accuracy of the motion vector is lowered, the calculation accuracy of the geometric transformation parameter is also lowered. Therefore, when the quantization parameter becomes large, processing for narrowing the threshold range of various determination parameters is performed. That is, the threshold value of each parameter is a decreasing function for the quantization parameter.

  In the present embodiment, an example in which the threshold range is controlled using the quantization parameter is shown, but other information included in the encoding parameter may be used. For example, the resolution of the decoded moving image, the type of the prediction target block, the Ref_idx of the reference image, the value of the motion vector, or information on the transform size may be used.

  Further, when all adjacent motion vectors of adjacent blocks and the motion vector of the prediction target block have the same value, the coordinates before and after geometric transformation do not change. In such a case, it is considered that the peripheral block is a parallel movement of the same object, so that it is not necessary to perform geometric transformation prediction. Therefore, when this condition is satisfied, the prediction switching information 417 has a value indicating that the geometric transformation prediction is not used.

  Further, the threshold value used for each index may be a function that changes according to the value of the encoding parameter. For example, the threshold ThDet in Equation (28) is defined as a function that depends on the quantization parameter, and the function is such that the threshold decreases as the quantization parameter increases, thereby reducing the estimation accuracy. It is possible to effectively switch predictions.

  The above is the outline of the processing of the prediction switching unit 409 and the prediction separation switch 410.

<Syntax structure>
Next, the syntax structure of the encoded data decoded by the video decoding device 400 will be described. The encoded data 411 decoded by the video decoding device 400 may have the same syntax structure as that of the video encoding device 100. Therefore, here, description will be made with reference to FIGS.

  FIG. 19 is a diagram illustrating a configuration of the syntax 1600. As shown in FIG. 19, the syntax 1600 mainly has three parts. The high-level syntax 1601 has higher layer syntax information that is equal to or higher than a slice. The slice level syntax 1602 has information necessary for decoding for each slice, and the macroblock level syntax 1603 has information necessary for decoding for each macroblock.

  Each part has a more detailed syntax. High level syntax 1601 includes sequence and picture level syntax, such as sequence parameter set syntax 1604 and picture parameter set syntax 1605. The slice level syntax 1602 includes a slice header syntax 1605, a slice data syntax 1606, and the like. The macroblock level syntax 1603 includes a macroblock layer syntax 1607, a macroblock prediction syntax 1608, and the like.

  FIG. 20 is a diagram illustrating an example of the slice header syntax 1605. The slice_affine_motion_prediction_flag shown in the figure is a syntax element indicating whether or not geometric transformation prediction is applied to the slice. When slice_affine_motion_prediction_flag is 0, the prediction switching unit 409 sets the prediction switching information 417 so as to always output the output terminal of the motion compensation prediction unit 406 in the slice, and switches the prediction separation switch 410. That is, this means that geometric transformation prediction is not applied to this slice. On the other hand, when slice_affine_motion_prediction_flag is 1, the prediction switching information 417 is set based on the information indicated by the determination parameter 416 in the slice, and the prediction separation switch 410 dynamically switches the prediction image signal.

  FIG. 21 is a diagram illustrating an example of the slice data syntax 1606. Mb_skip_flag shown in the drawing is a flag indicating whether or not the macroblock is encoded in the skip mode. In the skip mode, conversion coefficients, motion vectors, and the like are not encoded.

  AvailAffineMode is an internal parameter indicating whether geometric transformation prediction can be used in the macroblock. When AvailAffineMode is 0, it means that the prediction switching information 417 is set not to use the geometric transformation prediction by various values calculated by the determination parameter 416. Also, AvailAffineMode is 0 when the adjacent motion vector of the adjacent block and the motion vector of the prediction target block have the same value.

  On the other hand, when AvailAffineMode is 1, mb_affine_motion_skip_flag indicating whether to use geometric transformation prediction or motion compensation prediction is encoded. When mb_affine_motion_skip_flag is 1, it means that geometric transformation prediction is applied to the skip mode. When mb_affine_motion_skip_flag is 0, it means that motion compensation prediction is applied.

  FIG. 22 is a diagram illustrating an example of the macroblock layer syntax 1607. Mb_type shown in the figure indicates macroblock type information. That is, whether the current macroblock is intra-coded, inter-coded, what block shape is being predicted, whether the prediction direction is unidirectional prediction or bidirectional prediction, etc. Contains information. The mb_type is passed to the macroblock prediction syntax and the submacroblock prediction syntax indicating the syntax of the subblock in the macroblock.

  FIG. 23 is a diagram illustrating an example of macroblock prediction syntax. AvailAffineModeMb shown in the figure is an internal parameter indicating whether or not geometric transformation prediction can be used in the pixel block. When AvailAffineModeMb is 0, it means that the prediction switching information 122 is set not to use the geometric transformation prediction by various values calculated by the determination parameter 125. Also, AvailAffineModeMb is 0 when the adjacent motion vector of the adjacent block and the motion vector of the prediction target block have the same value.

  On the other hand, when AvailAffineModeMb is 1, mb_affine_pred_flag indicating which of geometric transformation prediction and motion compensation prediction is used is encoded. When mb_affine_pred_flag is 1, it means that geometric transformation prediction is applied to the pixel block. When mb_affine_pred_flag is 0, it means that motion compensation prediction is applied.

  NumMbPart () is an internal function that returns the number of block divisions specified in mb_type. It is 1 for a 16 × 16 pixel block, 2 for an 8 × 16 pixel block, and 2 for an 8 × 8 pixel block. In the case, 4 is output.

  FIG. 24 is a diagram illustrating an example of sub macroblock prediction syntax. AvailAffineModeSubMb shown in the figure is an internal parameter indicating whether or not geometric transformation prediction can be used in the pixel block. When AvailAffineModeSubMb is 0, it means that the prediction switching information 417 is set not to use the geometric transformation prediction by various values calculated by the determination parameter 125. The AvailAffineModeSubMb is also 0 when the adjacent motion vector of the adjacent block and the motion vector of the prediction target block have the same value.

  On the other hand, when AvailAffineModeSubMb is 1, mb_affine_pred_flag indicating whether to use geometric transformation prediction or motion compensation prediction is encoded. When mb_affine_pred_flag is 1, it means that geometric transformation prediction is applied to the pixel block. When mb_affine_pred_flag is 0, it means that motion compensation prediction is applied. NumSubMbPart () is an internal function that returns the number of block divisions defined in mb_type.

  FIG. 25 is a diagram illustrating an example of macroblock layer syntax as an example of an encoding parameter. Coded_block_pattern shown in the figure indicates whether a transform coefficient exists for each 8 × 8 pixel block. For example, when this value is 0, it means that there is no transform coefficient in the target block. Mb_qp_delta in the figure indicates information related to the quantization parameter. The difference value from the quantization parameter of the block encoded immediately before the target block is represented.

  Ref_idx_l0 and ref_idx_l1 in the figure indicate reference image indexes representing which reference image is used to predict the target block when inter prediction is selected. Mv_l0 and mv_l1 in the figure indicate motion vector information. In the figure, transform — 8 × 8_flag represents conversion information indicating whether or not the target block is 8 × 8 conversion.

  It should be noted that syntax elements not defined in the present embodiment may be inserted between the rows in the syntax tables shown in FIGS. 19 to 25, and descriptions regarding other conditional branches may be included. Further, the syntax table may be divided into a plurality of tables, or a plurality of syntax tables may be integrated. Moreover, it is not always necessary to use the same term, and it may be arbitrarily changed depending on the form to be used. Further, each syntax element described in the macroblock layer syntax may be changed as specified in a macroblock data syntax described later.

  The above is the description of the moving picture decoding apparatus 400.

(Sixth embodiment: addition of intra prediction)
FIG. 33 is a diagram illustrating a structure of a moving picture decoding apparatus 500 used in the sixth embodiment. In FIG. 33, each unit having the same function as that of the moving picture decoding apparatus 400 in FIG. 32 is denoted by the same reference numeral, and description thereof is omitted here.

  In addition to the units included in the video decoding device 400, the video decoding device 500 includes an intra prediction unit 501 and a prediction changeover switch 502. The intra prediction unit 501 generates a predicted image signal 418 using only information in the screen based on the input reference image signal 413.

  Prediction switching information 503 is input to the prediction switching switch 502 from the prediction information decoded by the encoded data decoding unit 402. The prediction switching information 503 describes information about which of the output terminal of the intra prediction unit 501 and the output terminal of the inter prediction unit 130 is connected to the switch.

  When intra prediction is selected, the prediction changeover switch 502 connects the output terminal of the intra prediction unit 501 to the switch, and outputs the predicted image signal 418 obtained by the intra prediction unit 501 to the adder 404. On the other hand, when the inter prediction is selected, the output terminal of the inter prediction unit 130 is connected to the switch, and the prediction image signal 418 obtained by the inter prediction unit 130 is output to the adder 404.

  Thereby, it is determined whether to select the predicted image signal generated by the intra prediction unit 501 or the predicted image signal generated by the inter prediction unit 130, and the output terminal of the prediction mode separation switch 502 is switched.

  The above is the description of the processing of the video decoding device 500 according to the sixth embodiment.

(Seventh embodiment: signaling a difference of motion vectors)
The moving picture decoding apparatus according to the seventh embodiment of the present invention generates, for example, a decoded picture signal by decoding encoded data generated by the moving picture decoding apparatus according to the fourth embodiment. In addition to the function of the video decoding device 400 according to the fifth embodiment, the video coding device according to the seventh embodiment corrects the motion vector and geometric transformation parameter of the prediction target block. The difference between is decoded.

  The change of the syntax of the encoded data decoded by the video decoding device according to the present embodiment with respect to the syntax of the encoded data decoded by the video decoding device 400 according to the fifth embodiment is shown in FIG. And the same as FIG.

29 and 30, among the characters included in the syntax elements, L0 and l0 indicate motion vectors indicated on the reference image L0, and L1 and l1 indicate motion vectors indicated on the reference image L1. If the original motion vector before the re-search is mv _org = (mvx _org , mvy _org ) and the motion vector after the re-search is mv _refine = (mvx _refine , mvy _refine ), the motion vector difference mvd_affine [2] Is calculated by equation (31).

  In Equation (31), mvd_affine [0] is the difference value of the motion vector corresponding to the vertical direction, and mvd_affine [1] corresponds to the syntax element. Note that here, the characters l0 and l1 included in the syntaxes shown in FIGS. 29 and 30 are omitted, but the motion vector difference shown in the equation (31) is different from the indexes L0 and L1. To calculate. Also, mvd_l0 and mvd_l1 are calculated by calculating the difference between the motion vector corresponding to each reference image signal and the value predicted from the motion vector, and other motion vector prediction techniques not explicitly described in this embodiment are used. Generated using.

When the encoding of the prediction target block is completed, mv _org = (mvx _org , mvy _org ), which is a motion vector before re-searching, is stored in the internal memory of the encoding control unit 126. In this way, by separately holding the motion vector used in the prediction target block and the motion vector used when it becomes an adjacent block, it is possible to prevent propagation of errors in geometric transformation parameters from the adjacent block Become.

(Modifications of the first to seventh embodiments)
(1) In the first to seventh embodiments, the processing target frame is divided into short blocks of 16 × 16 pixel size and the like, and as shown in FIG. The case of encoding / decoding has been described, but the encoding order and decoding order are not limited to this. For example, encoding and decoding may be performed sequentially from the lower right to the upper left, or encoding and decoding may be performed sequentially from the center of the screen toward the spiral. Furthermore, encoding and decoding may be performed in order from the upper right to the lower left, or encoding and decoding may be performed in order from the peripheral part to the center part of the screen.

  (2) In the first to seventh embodiments, the block size has been described as a 4 × 4 pixel block and an 8 × 8 pixel block. However, the prediction target block does not need to have a uniform block shape, and 16 × Any block size such as an 8 pixel block, an 8 × 16 pixel block, an 8 × 4 pixel block, or a 4 × 8 pixel block may be used. Also, it is not necessary to make all blocks the same within one macroblock, and blocks of different sizes may be mixed. In this case, as the number of divisions increases, the amount of codes for encoding or decoding the division information increases. Therefore, the block size may be selected in consideration of the balance between the code amount of the transform coefficient and the locally decoded image or the decoded image.

  (3) In the first to seventh embodiments, the luminance signal and the color difference signal are not divided and described as an example limited to one color signal component. However, when the prediction processing is different between the luminance signal and the color difference signal, different prediction methods may be used, or the same prediction method may be used. When a different prediction method is used, the prediction method selected for the color difference signal is encoded or decoded by the same method as the luminance signal.

  (4) In the first to fourth embodiments, the example in which the determination parameter is not included in the encoded data has been described. However, the determination parameter for each pixel block may be included in the encoded data. In the fifth to seventh embodiments, the example in which the determination parameter is calculated for each pixel block based on the geometric transformation parameter has been described. However, when the determination parameter is included in the encoded data, the moving image decoding apparatus determines whether to perform motion compensation prediction by geometric transformation based on the decoded determination parameter without calculating the determination parameter. It is good to have a configuration to do.

  By using the proposed method, in order to predict a moving object that is not suitable for the parallel movement model, excessive block division is prevented and block division information is prevented from increasing. In other words, the effect of improving the coding efficiency and the subjective image quality by predicting the geometric deformation of the object in the block without applying additional information and applying a suitable geometric transformation parameter to each. Play.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  As described above, the moving image encoding device, the moving image decoding device, the moving image encoding method, and the moving image decoding method according to the present invention are useful for highly efficient moving image encoding. It is suitable for moving picture coding that reduces the motion detection process necessary for estimating the geometric transformation parameter used for the geometric transformation motion compensation prediction.

DESCRIPTION OF SYMBOLS 100 Moving image encoder 101 Subtractor 102 Transformer / quantizer 103 Dequantizer / Inverse transformer 104 Adder 105 Reference image memory 106 Motion estimator 107 Motion compensation predictor 108 Geometric transformation parameter derivation unit 109 Geometric transformation predictor 110 prediction separation switch 111 prediction switching unit 112 entropy encoding unit 113 output buffer 114 input image signal 115 prediction error signal 116 transform coefficient 117 decoded prediction error signal 118 decoded image signal 119 motion vector 120 reference image signal 121 geometric transformation parameter 122 prediction switching Information 123 Predictive image signal 124 Encoded data 125 Determination parameter 126 Encoding control unit 127 Determination parameter deriving unit 130 Inter prediction unit 181 Motion vector acquisition unit 182 Parameter deriving unit 191 Geometric conversion unit 192 Interpolation unit 00 video encoding device 201 intra prediction unit 202 mode determination unit 203 prediction mode separation switch 204 prediction mode switching information 300 inter prediction unit 301 vector re-search unit 400 video decoding device 401 input buffer 402 encoded data decoding unit 403 inverse Quantization / inverse transform unit 404 Adder 405 Reference image memory 406 Motion compensation prediction unit 407 Geometric transformation parameter derivation unit 408 Geometric transformation prediction unit 409 Prediction switching unit 410 Prediction separation switch 411 Encoded data 412 Prediction error signal 413 Reference image signal 414 Geometric transformation parameter 415 Motion vector 416 Determination parameter 417 Prediction switching information 418 Prediction image signal 419 Output buffer 420 Decoded image signal 421 Decoding control unit 422 Determination parameter derivation unit 500 Video decoding device 501 IN Tiger prediction unit 502 prediction mode separation switch 502 prediction changeover switch 503 prediction changeover information

Claims (35)

A motion information acquisition unit that acquires motion information of one or more adjacent blocks among adjacent blocks adjacent to one of the pixel blocks into which the image signal is divided;
A geometric transformation information acquisition unit that acquires, based on the motion information, a geometric transformation parameter that is information related to a shape of a mapping by geometric transformation of the pixel block in a reference image signal when performing motion compensation on the pixel block;
A geometric transformation prediction unit that performs geometric transformation motion prediction including geometric transformation between the reference image signal and the pixel block, using the reference image signal subjected to geometric transformation by the geometric transformation parameter;
An encoding unit that encodes a prediction error value of the pixel block on which the geometric transformation motion prediction has been performed;
A moving picture encoding apparatus having:   The moving image coding apparatus according to claim 1, further comprising a geometric transformation unit that performs geometric transformation on the reference image signal according to the geometric transformation parameter.   Information relating to the geometric transformation parameter, information relating to the rotation angle obtained from the geometric transformation parameter, information relating to the amount of enlargement or reduction obtained from the geometric transformation parameter, information relating to the amount of deformation obtained from the geometric transformation parameter And a determination unit that determines whether or not to perform the geometric transformation motion prediction based on a decision parameter that is one or more pieces of information regarding the amount of movement obtained from the geometric transformation parameter. The moving image encoding device according to 1 or 2.   The moving image encoding apparatus according to claim 3, wherein the encoding unit further encodes the determination parameter. The geometric transformation prediction unit further generates motion information of the pixel block,
The moving image encoding apparatus according to claim 1, wherein the encoding unit further encodes the motion information.   The moving image encoding apparatus according to claim 1, wherein the motion information acquisition unit acquires motion information of an adjacent block for which prediction has already been completed among the adjacent blocks.   The moving image encoding apparatus according to claim 1, wherein the geometric transformation information acquisition unit acquires the geometric transformation parameters by converting the motion information based on a relative position between the adjacent block and the pixel block.   The motion information acquisition unit includes motion information of pixels adjacent to the first pixel in the raster order in the pixel block, and pixels adjacent to other pixels excluding the first pixel in the raster order among the pixels at the vertex of the pixel block. 2. The moving picture encoding apparatus according to claim 1, wherein one or more pieces of movement information are acquired from the movement information of the first pixel and the movement information of a pixel located at a center of the pixel block.   The moving image according to claim 1, wherein the motion information acquisition unit acquires one piece of motion information based on a motion vector obtained by performing motion prediction on the reference image signal for each of the adjacent blocks of the plurality of adjacent blocks. Image encoding device. The motion information acquisition unit acquires a motion vector of the pixel block from the motion information,
The encoding unit includes the geometric transformation motion prediction based on the motion vector and the geometric transformation parameter, and the geometric transformation motion prediction based on a motion vector obtained by a motion search based on the motion vector and the geometric transformation parameter. The moving picture coding apparatus according to claim 1, wherein information relating to a motion vector of geometric transformation motion prediction having a small prediction error value is coded.   The moving image encoding apparatus according to claim 1, wherein the encoding unit further encodes the geometric transformation parameter.   The moving image encoding apparatus according to claim 1, wherein the encoding unit further encodes correction information used for correction for reducing the prediction error value with respect to the geometric transformation parameter.   The moving image encoding apparatus according to claim 3, wherein the determination unit determines to perform the geometric transformation motion prediction when a value of the determination parameter is within a predetermined range.   In addition to the determination parameter, the determination unit includes a predetermined parameter when the pixel block is encoded, or a predetermined parameter when an already encoded pixel block included in the image signal is encoded 4. The moving picture encoding apparatus according to claim 3, wherein whether to perform the geometric transformation motion prediction is determined based on.   The predetermined parameters include prediction type information, prediction mode information, quantization parameter information, transform coefficient information, coefficient existence information, motion of the pixel block or the already encoded pixel block of the image signal. The moving picture coding apparatus according to claim 14, wherein the moving picture coding apparatus is at least one of information and block division information.   The moving image code according to claim 1, wherein the geometric transformation is one or more of affine transformation, bilinear transformation, Helmat transformation, quadratic conformal transformation, projective transformation, and three-dimensional projective transformation. Device.   The geometric transformation unit is one of a bilinear interpolation method, a nearest interpolation method, and a cubic convolution interpolation method when pixels of the mapping obtained by the geometric transformation are located between integer pixels. The moving image encoding apparatus according to claim 2, wherein pixel values of the pixels of the mapping are acquired by interpolation. A motion information acquisition step of acquiring motion information of one or more adjacent blocks among adjacent blocks adjacent to one of the pixel blocks into which the image signal is divided;
A geometric transformation information acquisition step for acquiring a geometric transformation parameter, which is information related to a shape of a map obtained by geometric transformation of the pixel block, in a reference image signal when performing motion compensation for the pixel block, and based on the motion information;
A geometric transformation prediction step, wherein geometric transformation motion prediction including geometric transformation between the reference image signal and the pixel block is performed using the reference image signal subjected to geometric transformation by the geometric transformation parameter;
An encoding step for encoding a prediction error value of the pixel block on which the geometric transformation motion prediction has been performed;
A video encoding method comprising: The image signal including a prediction error value obtained by geometric transformation motion prediction including geometric transformation between a pixel block into which the image signal is divided and a reference image signal when performing motion compensation on the pixel block is encoded. A decoding unit for decoding the encoded data,
A motion information acquisition unit that acquires motion information of one or more adjacent blocks of adjacent blocks adjacent to one of the pixel blocks;
A geometric transformation information acquisition unit that acquires, based on the motion information, a geometric transformation parameter that is information related to a shape of a map obtained by geometric transformation of the pixel block in the reference image signal;
Performing the geometric transformation motion prediction using the reference image signal subjected to the geometric transformation by the geometric transformation parameter, and generating a prediction value; a geometric transformation prediction unit;
An adder that adds the decoded prediction error value and the generated prediction value;
A moving picture decoding apparatus comprising:   The moving picture decoding apparatus according to claim 19, further comprising a geometric transformation unit that performs geometric transformation on the reference image signal according to the geometric transformation parameter.   Information relating to the geometric transformation parameter, information relating to the rotation angle obtained from the geometric transformation parameter, information relating to the amount of enlargement or reduction obtained from the geometric transformation parameter, information relating to the amount of deformation obtained from the geometric transformation parameter And a determination unit that determines whether or not to perform the geometric transformation motion prediction based on a decision parameter that is one or more pieces of information related to the amount of movement obtained from the geometric transformation parameter. The moving picture decoding apparatus according to 19 or 20.   The moving picture decoding apparatus according to claim 21, wherein the encoding unit further encodes the determination parameter.   The video decoding device according to claim 19, wherein the decoding unit further decodes motion information of the pixel block.   The moving picture decoding apparatus according to claim 19, wherein the motion information acquisition unit acquires motion information of an adjacent block for which prediction has already been completed among the adjacent blocks.   The moving picture decoding apparatus according to claim 19, wherein the geometric transformation information acquisition unit acquires the geometric transformation parameters by converting the motion information based on a relative position between the adjacent block and the pixel block.   The motion information acquisition unit includes motion information of pixels adjacent to the first pixel in the raster order in the pixel block, and pixels adjacent to other pixels excluding the first pixel in the raster order among the pixels at the vertex of the pixel block. The moving picture decoding apparatus according to claim 19, wherein one or more pieces of movement information are acquired from the movement information of the first pixel and the movement information of a pixel located at a center of the pixel block.   The moving image according to claim 19, wherein the motion information acquisition unit acquires one piece of motion information based on a motion vector obtained by performing motion prediction on the reference image signal for each of the adjacent blocks of the plurality of adjacent blocks. Image decoding device. The code data includes information on the geometric transformation parameter,
The moving picture decoding apparatus according to claim 19, wherein the decoding unit further decodes the geometric transformation parameter. The code data further includes correction information used for correction to reduce the prediction error value for the geometric transformation parameter,
The video decoding device according to claim 19, wherein the decoding unit further decodes the correction information.   The moving picture decoding apparatus according to claim 21, wherein the determination unit determines to perform the geometric transformation motion prediction when the value of the determination parameter is within a predetermined range.   In addition to the determination parameter, the determination unit includes a predetermined parameter when the pixel block is encoded, or a predetermined parameter when an already encoded pixel block included in the image signal is encoded The moving picture decoding apparatus according to claim 21, wherein whether to perform the geometric transformation motion prediction is determined based on.   The predetermined parameters include prediction type information, prediction mode information, quantization parameter information, transform coefficient information, coefficient existence information, motion of the pixel block or the already encoded pixel block of the image signal. 32. The moving picture decoding apparatus according to claim 31, wherein the moving picture decoding apparatus is at least one of information and block division information.   The moving picture decoding according to claim 19, wherein the geometric transformation is one or more of an affine transformation, a bilinear transformation, a Helmat transformation, a quadratic conformal transformation, a projective transformation, and a three-dimensional projective transformation. Device.   The geometric transformation unit is one of a bilinear interpolation method, a nearest interpolation method, and a cubic convolution interpolation method when pixels of the mapping obtained by the geometric transformation are located between integer pixels. 21. The moving picture decoding apparatus according to claim 20, wherein pixel values of pixels of the mapping are acquired by interpolation. The image signal including a prediction error value obtained by a geometric transformation motion prediction including a geometric transformation between a pixel block into which the image signal is divided and a reference image signal when performing motion compensation on the pixel block is encoded. A decoding step of decoding the encoded data;
A motion information acquisition step of acquiring motion information of one or more adjacent blocks of adjacent blocks adjacent to one of the pixel blocks;
A geometric transformation information obtaining step for obtaining a geometric transformation parameter, which is information relating to a shape of a map obtained by geometric transformation of the pixel block in the reference image signal, based on the motion information;
Performing the geometric transformation motion prediction using the reference image signal subjected to the geometric transformation by the geometric transformation parameter, and generating a prediction value; a geometric transformation prediction step;
An adding step of adding the decoded prediction error value and the generated prediction value;
A video decoding method comprising:

Images