JP2012157061A - Method and device for decoding image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide image decoding with high encoding efficiency.SOLUTION: A decoding device includes: a decoding section decoding an encoded prediction error; a storage section 207 acquiring a first motion vector 308 corresponding to a first pixel region which is previously decoded; a first derivation section deriving a motion vector group including at least one first motion vector 308; a second derivation section 206 deriving a second motion vector 307 corresponding to each third pixel region obtained by dividing a second pixel region before decoding based on the motion vector group; a prediction section generating a first inter-prediction image of each third pixel region by using the second motion vector 307; an integration section integrating the first inter-prediction images and generating a second inter-prediction image corresponding to the second pixel region before decoding; and an addition section adding the second inter-prediction image and the prediction error and generating a decoding image.

Description

  The present invention relates to a method and apparatus for encoding / decoding moving images and still images.

  ITU-T Rec. H. 264 and ISO / IEC 14496-10 (hereinafter simply referred to as H.264) have been formulated by ITU-T and ISO / IEC as joint recommendations (standards) for video coding systems. Yes. In H.264, prediction processing, conversion processing, and entropy coding processing are performed for each rectangular block to be coded (coding target block). In the prediction process, for example, a frame (reference frame) that has already been encoded is referred to, and temporal prediction (motion compensation) is performed on the encoding target block. In the motion compensation, it is generally necessary to encode a motion vector that is spatial shift information between a block to be encoded and a block referenced in a reference frame. When a plurality of reference frames are used, it is necessary to encode a reference frame number in order to identify each reference frame.

  In the moving picture coding method described in Non-Patent Document 1 (ie, H.264), when a motion vector of an adjacent block of an encoding target block has already been encoded, a predicted motion is based on the motion vector of the adjacent block. A vector (for example, a median value of a motion vector of an adjacent block) is derived. The moving picture encoding method described in Non-Patent Document 1 encodes the difference between the predicted motion vector and the motion vector actually derived for the encoding target block.

  The moving picture coding method described in Non-Patent Document 2 prepares a plurality of variations of the predicted motion vector, switches the predicted motion vector used for each coding target block, and selects the selected predicted motion vector and the coding target block. On the other hand, the difference between the actually derived motion vector is encoded. Therefore, according to the moving picture coding method described in Non-Patent Document 2, the prediction motion vector can be selectively used so that the code amount is small.

  In the moving image encoding method described in Patent Document 1, a motion vector that has already been encoded is stored in units of frames, and a motion vector of a block at the same position as the encoding target block (Collocate position) in the frame is The motion vector of the encoding target block is obtained. Therefore, according to the moving image encoding method described in Patent Document 1, it is possible to omit encoding of a motion vector.

Japanese Patent No. 3977716

ITU-T Rec. H.264 ITU-T Q.6 / SG16, VCEG-AC06r1, "Competition-Based Scheme for Motion Vector Selection and Coding"

  The moving image encoding method described in Non-Patent Document 1 encodes a difference between a predicted motion vector and a motion vector actually derived with respect to the encoding target block. It is not always less than the code amount related to the derived motion vector.

  In the moving picture encoding method described in Non-Patent Document 2, it is necessary to encode switching information in order to notify the decoding side which prediction motion vector has been selected for each encoding target block. That is, in the moving picture coding method described in Non-Patent Document 2, there is a possibility that the code amount increases on the contrary due to the code amount of the switching information.

  According to the moving picture encoding method described in Patent Document 1, it is possible to omit the encoding of a motion vector, but the motion vector related to the block at the Collocate position is not necessarily appropriate as the motion vector of the encoding target block.

  Therefore, an object of the present invention is to provide an image encoding / decoding device with high encoding efficiency.

  An image decoding apparatus according to an aspect of the present invention includes a decoding unit that decodes an encoded prediction error, and a memory that acquires a first motion vector corresponding to a first pixel area that has already been decoded. , A first derivation unit for deriving a motion vector group including at least one of the first motion vectors, and a third pixel obtained by dividing the second pixel region before decoding based on the motion vector group A second derivation unit for deriving a second motion vector corresponding to each of the regions, and a prediction for generating a first inter prediction image for each of the third pixel regions using the second motion vector An integration unit that integrates the first inter-predicted image to generate a second inter-predicted image corresponding to the second pixel region before decoding, the second inter-predicted image, and the Add decoded error to generate decoded image ; And a calculation part.

  According to the present invention, an image encoding / decoding device with high encoding efficiency can be provided.

1 is a block diagram showing an image encoding device according to a first embodiment. The block diagram which shows the motion vector derivation | leading-out part of FIG. The block diagram which shows the motion vector process part of FIG. The block diagram which shows the inter estimation part of FIG. The figure which shows an example of the encoding process order with respect to the macroblock by the image coding apparatus of FIG. The figure which shows an example of the size of the encoding object block which the image coding apparatus of FIG. 1 uses. The figure which shows the other example of FIG. 6A. The figure which shows the other example of FIG. 6A and 6B. Explanatory drawing of an example of the inter prediction process. FIG. 7B is an explanatory diagram of another example of FIG. 7A. The figure which shows an example of the motion compensation block size of the macroblock unit in the inter prediction process. The figure which shows the other example of FIG. 8A. The figure which shows the other example of FIG. 8A and 8B. The figure which shows the other example of FIG. 8A, FIG. 8B, and FIG. 8C. The figure which shows an example of the motion compensation block size of the sub macroblock unit in the inter prediction process. The figure which shows the other example of FIG. 9A. The figure which shows the other example of FIG. 9A and 9B. The figure which shows the other example of FIG. 9A, FIG. 9B, and FIG. 9C. The figure which shows an example of the motion vector block produced | generated by the motion vector block production | generation part of FIG. The figure which shows an example of the motion vector frame memorize | stored in the reference motion vector memory of FIG. Explanatory drawing of the motion vector block information utilized by the motion vector production | generation part of FIG. The table which shows the relationship between the size of a motion vector block, and the number of the motion vectors produced | generated by the motion vector production | generation part of FIG. Explanatory drawing of the motion compensation process by the motion compensation part of FIG. Explanatory drawing of the interpolation process of the decimal pixel precision which can be utilized in the motion compensation process by the motion compensation part of FIG. The flowchart which shows the motion vector block prediction process by the inter estimation part of FIG. The figure which shows an example of the index in the macroblock divided | segmented by the process in step S402 of FIG. Explanatory drawing of the process in step S404 of FIG. The figure which shows notionally the example of application of the motion vector block prediction process by the inter prediction part of FIG. The figure which shows the other example of FIG. 19 notionally. Explanatory drawing of space SKIP mode. The block diagram which shows the image coding apparatus which concerns on 2nd Embodiment. The block diagram which shows the estimation part of FIG. Explanatory drawing of an example of the motion vector block prediction process by the inter estimation part of FIG. 1 in the case of using a some flame | frame. Explanatory drawing of the other example of FIG. 24A. The figure which shows an example of the syntax structure which the image coding apparatus of FIG. 1 uses. The figure which shows an example of the macroblock layer syntax of FIG. The figure which shows an example of the macroblock prediction syntax of FIG. The figure which shows an example regarding the submacroblock of the macroblock prediction syntax of FIG. The block diagram which shows the image coding apparatus which concerns on 3rd Embodiment. The block diagram which shows the estimation part of FIG. The block diagram which shows the 2nd inter estimation part of FIG. The figure which shows an example of the sequence parameter set syntax of FIG. The figure which shows an example of the picture parameter set syntax of FIG. The figure which shows an example of the slice header syntax of FIG. The figure which shows an example of the macroblock layer syntax of FIG. The figure which shows the other example of FIG. The figure which shows the other example of FIG. The flowchart which shows the encoding omission process of mv_block_in_mb_flag in motion vector block SKIP mode. The block diagram which shows the motion vector derivation | leading-out part in the image decoding apparatus which concerns on 4th Embodiment. FIG. 40 is a block diagram showing an image decoding apparatus including the motion vector deriving unit in FIG. 39. FIG. 40 is a block diagram showing a motion vector block processing unit in FIG. 39. The block diagram which shows the inter estimation part of FIG. The block diagram which shows the image decoding apparatus which concerns on 5th Embodiment. The block diagram which shows the estimation part of FIG. The block diagram which shows the image decoding apparatus which concerns on 6th Embodiment. The block diagram which shows the estimation part of FIG. The block diagram which shows the 2nd inter estimation part of FIG.

Embodiments of the present invention will be described below with reference to the drawings.
(First embodiment)
As shown in FIG. 1, the image coding apparatus according to the first embodiment of the present invention includes an image coding unit 1000, a coding control unit 130, and an output buffer 112. The image encoding device in FIG. 1 may be realized by hardware such as an LSI chip, or may be realized by causing a computer to execute an image encoding program. The image coding unit 1000 includes an inter prediction unit 101, a subtraction unit 102, a transform / quantization unit 103, an entropy coding unit 104, an inverse transform / inverse quantization unit 105, an addition unit 106, a reference image memory 107, and a motion vector derivation. Part 124.

  The encoding control unit 130 controls overall encoding processing by the image encoding unit 1000 such as feedback control of generated code amount, quantization control, prediction mode control, and entropy encoding control. Specifically, the encoding control unit 130 sets the encoding control information 30 in each unit of the image encoding unit 1000, and appropriately acquires the feedback information 31 from each unit of the image encoding unit 1000. The encoding control information 30 includes prediction information 22 (including prediction mode information and block size switching information), motion vector information 18, motion vector block information 21, and quantization parameters (quantization width (quantization step size) and quantum). And the like). The feedback information 31 includes the amount of code generated by the image encoding unit 1000 and is used, for example, for determining the quantization parameter.

  An original image 10 that is a moving image or a still image is input to the image encoding unit 1000 after being divided into predetermined processing units (for example, macroblocks, subblocks, or small pixel blocks such as one pixel). . The processing unit may be a frame or a field. In the following description, unless otherwise specified, the processing unit is referred to as a macro block, and the original image 10 of the macro block unit to be processed is particularly referred to as an encoding target block.

  The size of the encoding target block may be 16 × 16 pixels as shown in FIG. 6A, 8 × 8 pixels as shown in FIG. 6B, or 4 × 4 pixels as shown in FIG. 6C. Other sizes (for example, 32 × 32 pixels) may be used. Further, the shape of the macroblock is not limited to a square shape, but may be an arbitrary shape. However, in the following description, the shape of the macroblock is assumed to be a square shape.

  In addition, although the encoding processing order for each encoding target block constituting the encoding target frame of the original image 10 is arbitrary, in the following description, as shown in FIG.・ Processing shall be performed in scan order.

  The inter prediction unit 101 includes a reference image (reference image signal) 17 acquired from a reference image memory 107 described later, a plurality of motion vector information 18 input from a motion vector deriving unit 124 described later, and an encoding control unit 130. Inter prediction is performed based on the notified prediction information 22 to generate a predicted image (predicted image signal) 11 of the encoding target block. The inter prediction unit 101 inputs the predicted image 11 to the subtraction unit 102 and the addition unit 106. Note that the prediction processing by the inter prediction unit 101 is merely characteristic prediction processing performed by the image encoding device in FIG. 1, and the image encoding device in FIG. 1 selects so-called normal inter prediction processing and intra prediction processing. The prediction image 11 may be generated by performing the above. Details of the prediction processing by the inter prediction unit 101 will be described later.

  The subtraction unit 102 subtracts the prediction image 11 from the encoding target block, and inputs the prediction error (prediction error signal) 12 to the transform / quantization unit 103.

  The transform / quantization unit 103 performs transform processing / quantization processing on the prediction error 12 from the subtraction unit 102, and converts the quantized transform coefficient (hereinafter simply referred to as a quantized transform coefficient) 13 to an entropy code. To the conversion unit 104 and the inverse transform / inverse quantization unit 105. The conversion process is, for example, discrete cosine transform (DCT), wavelet transform, independent component analysis, or the like. The quantization process is performed according to a quantization parameter set by the encoding control unit 130.

  The entropy encoding unit 104 entropy is applied to the encoding parameters including the quantized transform coefficient 13 from the transform / quantization unit 103, the motion vector block information 21, the prediction information 22, and the quantization parameter from the encoding control unit 130. Encoding is performed, and the encoded data 14 is input to the output buffer 112. The entropy encoding process is, for example, isometric encoding, Huffman encoding, or arithmetic encoding. The conventional image encoding device needs to encode the motion vector information 18 when performing inter prediction, but the image encoding device in FIG. 1 encodes the motion vector information 18. First, the motion vector block information 21 is encoded. The encoded data 14 is temporarily stored in the output buffer 112 in a multiplexed state, and is output to the outside of the image encoding device in FIG. 1 according to the output timing managed by the encoding control unit 130. Here, the outside of the image encoding apparatus in FIG. 1 is a storage system (storage medium) or a transmission system (communication line) not shown.

  The inverse transform / inverse quantization unit 105 performs inverse quantization / inverse transform on the quantized transform coefficient 13 from the transform / quantization unit 103 to decode the prediction error 12, and adds it as a decoded prediction error 15. 106. The inverse quantization process is performed according to a quantization parameter set by the encoding control unit 130. The inverse transform process is an inverse transform process of the transform process described above, and is, for example, an inverse discrete cosine transform (IDCT) or an inverse wavelet transform.

  The adding unit 106 adds the decoded prediction error 15 from the inverse transform / inverse quantization unit 105 and the predicted image 11 from the inter prediction unit 101 to generate a local decoded image 16 of the encoding target block. The adding unit 106 stores the locally decoded image (local decoded image signal) 16 in the reference image memory 107 as a reference image (reference image signal) 17.

  The reference image memory 107 stores the reference image 17 in units of frames, for example, and is read out by the inter prediction unit 101 as necessary.

  The motion vector deriving unit 124 derives a plurality of motion vector information 18 based on the motion vector block information 21 and the prediction information 22 from the encoding control unit 130. Specifically, as shown in FIG. 2, a motion vector processing unit 108 and a reference motion vector memory 109 are included.

  The motion vector processing unit 108 acquires the reference motion vector information 19 from the reference motion vector memory 109 based on the motion vector block information 21 and the prediction information 22 from the encoding control unit 130 and derives a plurality of motion vector information 18. . Details of the motion vector derivation processing by the motion vector processing unit 108 will be described later.

  In the reference motion vector memory 109, already encoded motion vector information 18 is temporarily stored as reference motion vector information 19. More specifically, as shown in FIG. 11, the reference motion vector memory 109 stores the reference motion vector information 19 in units of frames to form a reference motion vector frame 114. Encoded motion vector information 18 is sequentially input to the reference motion vector memory 109 and stored as reference motion vector information 19 constituting a reference motion vector frame 114 corresponding to a temporal position. In FIG. 11, the motion compensation block size of the reference motion vector information 19 constituting the reference motion vector frame 114 is 4 × 4 pixels, but it may be 2 × 2 pixels or one pixel unit, and is not limited to a rectangular shape and is arbitrary. Shape may be sufficient. As shown in FIG. 10, the motion vector block 28 is a set of 16 reference motion vector information 19 of 4 × 4, but the specific mode of the motion vector block 28 is not limited to this.

  Hereinafter, the motion vector deriving process by the motion vector block processing unit 108 will be described with reference to FIG. As illustrated in FIG. 3, the motion vector block processing unit 108 includes a motion vector block generation unit 110 and a motion vector generation unit 111.

  The motion vector block generation unit 110 acquires the motion vector block information 21 and generates a motion vector block 28 that is a set of reference motion vector information 19 corresponding to the motion vector block information 21. The motion vector block generation unit 110 inputs the motion vector block 28 to the motion vector generation unit 111.

The motion vector block information 21 is information indicating the spatial position of the motion vector block 28 in the reference motion vector frame 114 as shown in FIG. In the following description, in the reference motion vector frame 114, the same position as the upper left vertex of the encoding target block is used as a reference point, and the spatial shift amount from the reference point to the upper left vertex of the motion vector block 28 is expressed as a motion vector block. Information 21 is assumed. In this embodiment, since the motion compensation block size of the reference motion vector information 19 is 4 × 4 pixels with respect to the luminance signal, the motion vector block information 21 is information of 4 × 4 pixel accuracy as shown in the following equation (1). It is.

  In Equation (1), (blk_x, blk_y) indicates the vertical and horizontal indexes (coordinates) in the frame at the left vertex position of the encoding target block, and (mvblk_x_pos, mvblk_y_pos) indicates the left vertex of the motion compensation block 28. The vertical and horizontal indexes in the reference motion vector frame 114 of the position are shown, and (mvblk_x, mvblk_y) show the vertical and horizontal components of the motion vector block information 21. The relationship between the size of the motion vector block 28 and the number of reference motion vector information 19 included in the motion vector block 28 is as shown in FIG. The motion vector block information 21 may be expressed by a difference with respect to other motion vector block vector information 21 that has already been encoded.

  The size of the motion vector block 28 may be the same size as the encoding target block or may be a size obtained by dividing the encoding target block. For example, if the encoding target block is a 16 × 16 pixel block, the motion vector block 28 is a 16 × 16 pixel block, a 16 × 8 pixel block, an 8 × 16 pixel block, an 8 × 8 pixel block, or an 8 × 4 pixel block. Either a 4 × 8 pixel block or a 4 × 4 pixel block may be used. When the size of the motion vector block 28 is a size obtained by dividing the encoding target block, it is desirable that the number of motion vector block information 21 corresponding to the number of divisions is acquired.

Here, the technical significance of using the motion vector block vector information 21 will be described.
For example, as shown in FIG. 10, the motion vector block 28 includes a total of 16 reference motion vector information 19 of 4 × 4, and the inter prediction unit 101 described later stores the 16 reference motion vector information 19. It can be applied to the encoding target block. For example, if the size of the encoding target block is 16 × 16 pixels, the inter prediction unit 101 can perform motion compensation in units of 4 × 4 pixel blocks obtained by further dividing the block, which improves the accuracy of inter prediction processing. I can expect. In H.264, it is necessary to encode the individual motion vector information 18, but as described above, the entropy encoding unit 104 converts the individual reference motion vector information 19 included in the motion vector block 28 into the motion vector information. It is not necessary to encode as 18, and only the motion vector block information 21 may be encoded. Therefore, according to the image coding apparatus according to the present embodiment, it is possible to reduce the amount of code (overhead) related to motion vector information. In addition, since the motion vector information in H.264 is information of 1/4 pixel accuracy, the motion vector block information 21 is information of 4 pixel accuracy, so the information amount can be suppressed to about 1/16.

  Further, the motion vector block information 21 may be given from the outside, or may be derived by actually searching the motion vector block 28 from the reference motion vector frame 114 using a so-called motion search process. In the motion search process, the cost shown in Equation (5) or (6) described later may be used. The spatial shift amount from the same position as the encoding target block in the reference motion vector frame 114 to the optimum motion vector block 28 obtained as a result of the motion search process is used as the motion vector block information 21.

  Based on the reference motion vector information 19 included in the motion vector block 28 from the motion vector block generation unit 110, the motion vector generation unit 111 includes a plurality of small pixel blocks (small pixel block signals) 23 obtained by dividing the encoding target block. The motion vector information 18 for each is generated and output. Specifically, the motion vector generation unit 111 substitutes appropriate reference motion vector information 19 as the motion vector information 18 regarding the small pixel block 23. For example, if the block size of the small pixel block 23 and the motion compensation block size of the reference motion vector information 19 match, each small pixel block 23 and each reference motion vector information 19 have a one-to-one correspondence. Therefore, the motion vector generation unit 111 may sequentially substitute the reference motion vector information 19 corresponding to the position of the small pixel block 23 as the motion vector information 18 corresponding to the small pixel block 23. When the block size of the small pixel block 23 and the motion compensation block size of the reference motion vector information 19 are different, the motion vector generation unit 111 corresponds to, for example, a motion compensation block whose spatial position overlaps with the small pixel block 23. The reference motion vector information 19 may be substituted.

  In addition, the motion vector generation unit 111 may add the following modifications instead of substituting the reference motion vector information 19 as the motion vector information 18 corresponding to the small pixel block 23 as it is.

(A) Inverted vector (negative value) of reference motion vector information 19 is substituted. (B) Weighted average value, median value, maximum of reference motion vector information 19 and reference motion vector information adjacent to the reference motion vector information 19 Substituting a value or minimum value (C) Substituting a value obtained by normalizing the reference motion vector according to the reference frame number or the temporal position of the motion vector block. Here, normalization is performed for the frame referred to by the reference motion vector information 19 This is preferably performed when the temporal distance is different from the temporal distance of the frame referenced by the encoding target block. Specifically, the temporal distance between the reference motion vector frame 114 to which the reference motion vector information 19 belongs and the reference image 17 to which the reference motion vector information 19 refers is TR, the frame to which the block to be encoded belongs, Assuming that the time distance between the encoding target block and the reference image 17 is TC, a value obtained by multiplying the reference motion vector information 19 by TC / TR may be substituted as the motion vector information 18.

  Note that the prediction information 22 specifies how to divide the encoding target block into a plurality of small pixel blocks 23. For example, if the encoding target block is a 16 × 16 pixel macroblock, a 4 × 4 pixel block, one pixel, and the like are examples of the small pixel block 23. Since the motion vector 18 related to the small pixel block 23 is generated based on the reference motion vector information 19 as described above, the motion compensation block size of the reference motion vector information 19 is equal to or smaller than the block size of the small pixel block 23. Is desirable. For example, if the block size of the small pixel block 23 is 4 × 4 pixels, the motion compensation block size of the reference motion vector information 19 may be 4 × 4 pixels or 2 × 2 pixels. One pixel may be sufficient.

Hereinafter, the inter prediction process by the inter prediction unit 101 will be described.
First, other prediction processes that can be used by the image coding apparatus in FIG. 1 will be described so that the inter prediction process by the inter prediction unit 101 can be easily understood. The image coding apparatus in FIG. 1 can use a plurality of prediction modes, and each prediction mode has a different prediction process and motion compensation block size for generating the predicted image 11. Note that the prediction mode is specified by the prediction information 22. The prediction process is roughly classified into a spatial direction prediction process and a temporal direction prediction process, and is referred to as intra prediction (intraframe prediction) and inter prediction (interframe prediction), respectively. In the intra prediction, the prediction image 11 is generated using the reference image 17 in the same frame or field as the encoding target block. In inter prediction, a predicted image 11 is generated using a reference image 17 of a frame or a field having a temporal position different from that of an encoding target block.

  In more detail regarding normal inter prediction, for example, as shown in FIG. 7A, a pixel block at a position spatially shifted according to the motion vector information 18 is predicted from the same position as the encoding target block in the reference image 17. Generated as an image 11. In the inter prediction, motion compensation with decimal pixel accuracy is possible. For example, with H.264, motion compensation with 1/2 pixel accuracy and 1/4 pixel accuracy can be performed for a luminance signal. When performing motion compensation with 1/4 pixel accuracy, the amount of information of the motion vector information 18 is 16 (= 4 × 4) times that when motion compensation with integer pixel accuracy is performed.

  In the inter prediction process, the number of frames that can be referred to is not limited to one. For example, as illustrated in FIG. 7B, a plurality of reference images 17 having different temporal positions may be used. When a plurality of reference images 17 are used, which reference image 17 is referenced is identified by the reference frame number. The reference frame number may be changed in units of each pixel area (such as a picture or a block). As the reference frame number, for example, “0” is given to the frame immediately before the frame to which the encoding target block belongs, and “1” is given to the second previous frame. When the number of reference frames is 1, the reference frame number of the reference frame is fixed to “0” or the like.

  Also, in the inter prediction process, it is possible to select a suitable one for the encoding target block from a plurality of motion compensation block sizes. Specifically, for the macroblock, 16 × 16 pixels shown in FIG. 8A, 16 × 8 pixels shown in FIG. 8B, 8 × 16 pixels shown in FIG. 8C, 8 × 8 pixels shown in FIG. For the sub-macroblock, 8 × 8 pixels shown in FIG. 9A, 8 × 4 pixels shown in FIG. 9B, 4 × 8 pixels shown in FIG. 9C, 4 × 4 pixels shown in FIG. 9D, and the like can be selected. Since a motion vector can be derived individually for each motion compensation block, an optimal motion compensation block shape (size) and motion vector are selected according to the local nature of the block to be encoded.

As illustrated in FIG. 4, the inter prediction unit 101 includes a predicted image signal integration unit 115 and a plurality of motion compensation units 113.
Each of the plurality of motion compensation units 113 generates a small predicted image 24 for each of the plurality of small pixel blocks 23 obtained by dividing the encoding target block. The motion compensation unit 113 inputs the small predicted image 24 to the predicted image signal integration unit 115.

As shown in FIG. 14, the motion compensation unit 113 subtracts a pixel block at a position spatially shifted from the same position as the small pixel block 23 in the reference image 17 according to the motion vector information 18 related to the small pixel block 23. (Small prediction image signal) 24 is generated. The motion compensation process performed by the motion compensation unit 113 on the small pixel block 23 can be realized in the same manner as the motion compensation process in H.264. Specifically, motion compensation processing with 1/2 pixel accuracy or 1/4 pixel accuracy can be used. When motion compensation is performed up to 1/4 pixel accuracy, the position of the small predicted image 24 is determined by the following formula (2).

  In Equation (2), (x, y) is the horizontal index and vertical index indicating the upper left vertex of the small pixel block 23, and (mv_x, mv_y) is the horizontal component and vertical direction of the motion vector information 18 The component (x_pos, y_pos) indicates a horizontal index and a vertical index indicating the upper left vertex of the small predicted image 24, respectively. The upper left vertex (x_pos, y_pos) of the small predicted image 24 is an integer pixel position if the horizontal direction component mv_x and the vertical direction component mv_y of the motion vector information 18 are multiples of 4, and is 1 if the multiple of 2 is not a multiple of 4. / 2 pixel position, if it is odd, it is a 1/4 pixel position. If the upper left vertex (x_pos, y_pos) of the small predicted image 24 is a 1/2 pixel position or a 1/4 pixel position, pixel value interpolation processing is required. Specifically, in FIG. 15, blocks with only the lower left oblique line are pixels corresponding to integer pixel positions, blocks with lower left oblique line and lower right oblique line are pixels corresponding to 1/2 pixel positions, and the like. Each block indicates a pixel corresponding to a 1/4 pixel position.

Pixel values b and h of the pixel corresponding to the 1/2 pixel position are derived by the following interpolation formula (3).

In Equation (3), >> indicates a right shift operation, and >> 5 corresponds to division by 32. That is, the pixel value of the pixel corresponding to the 1/2 pixel position is determined using a 6-tap FIR (Finite Impulse Response) filter (tap coefficient: (1, -5, 20, 20, -5, 1) / 32). Generated.

The pixel values a and d of the pixels corresponding to the 1/4 pixel position are derived by the following interpolation formula (4).

  That is, the pixel value of the pixel corresponding to the 1/4 pixel position is generated using a 2-tap average value filter (tap coefficient: (1/2, 1/2)). The pixel value j of the pixel existing in the middle of the four integer pixel positions is generated by appropriately combining a vertical 6-tap FIR filter and a horizontal 6-tap FIR filter. For example, pixel values cc, dd, h, m, ee, and ff are generated using a vertical 6-tap FIR filter, and a pixel value j is generated by applying a horizontal 6-tap FIR filter thereto. Also, pixel values aa, bb, b, s, gg and hh are generated using a horizontal 6-tap FIR filter, and a pixel value j is generated by applying a vertical 6-tap FIR filter to these. Pixel values of pixels corresponding to other positions can be generated according to the same interpolation rule.

  In addition, the pixel value of the pixel corresponding to the decimal precision pixel position may be generated by performing an interpolation process different from Expressions (3) and (4). In addition, the interpolation coefficient is not limited to that described above, and a fixed value or a variable value may be set by the encoding control unit 130. Further, when the interpolation coefficient is variable, the interpolation coefficient may be optimized for each frame from the viewpoint of encoding cost described later.

  The prediction image signal integration unit 115 integrates the small prediction images 24 from the plurality of motion compensation units 113 and generates the prediction image 11 corresponding to the encoding target block. That is, by integrating the small predicted images 24 corresponding to the individual small pixel blocks 23, the predicted image 11 corresponding to the encoding target block before being divided into the small pixel blocks 23 is obtained. Here, the integration means that the small predicted images 24 are connected to generate the predicted image 11.

  As described above, the inter prediction process performed by the inter prediction unit 101 is different from the normal inter prediction process in that the motion vector block 28 is used. Hereinafter, inter prediction processing using motion vector blocks is referred to as motion vector block prediction processing.

Hereinafter, the flow of motion vector block prediction processing by the inter prediction unit 101 will be described with reference to FIG. In FIG. 16, the encoding target block is a 16 × 16 pixel macroblock, and when a series of processing ends, processing for the next encoding target block is performed.
First, the motion vector block generation unit 110 generates a motion vector block 28 corresponding to the encoding target block based on the motion vector block information 21 (step S401). Specifically, the motion vector block generator 110 spatially shifts the reference motion vector frame 114 stored in the reference motion vector memory 109 from the same position as the encoding target block according to the motion vector block information 21. A set of reference motion vector information 19 at the determined position is generated as a motion vector block 28. In this example, each motion compensation block size of the reference motion vector information 19 is 4 × 4 pixels, and the motion vector block 28 includes 4 × 4 total 16 pieces of reference motion vector information 19.

  Next, the motion vector generation unit 111 divides the encoding target block into 4 × 4 pixel small pixel blocks 23 according to the prediction information 22 (step S402). Each small pixel block 23 is identified by being assigned an index from “0” to “15” as shown in FIG.

  Next, the motion vector generation unit 111 assigns “0” as an initial value to the variable BlkIdx for specifying the small pixel block 23 currently being processed (step S403), and the process proceeds to step S404.

  In step S404, appropriate reference motion vector information 19 is substituted as motion vector information Vc (BlkIdx) regarding the small pixel block 23 corresponding to the variable BlkIdx. In this example, since the block size (4 × 4 pixels) of the small pixel block 23 and the motion compensation block size (4 × 4 pixels) of the reference motion vector information 19 match, as shown in FIG. It is desirable to substitute the reference motion vector information Vr (BlkIdx) at the same position for the motion vector information Vc (BlkIdx).

  Next, any one of the plurality of motion compensation units 113 performs motion compensation using the motion vector information Vc (BlkIdx) regarding the small pixel block 23 corresponding to the variable BlkIdx, and performs a small compensation corresponding to the small pixel block 23. A predicted image 24 is generated (step S405).

  Next, the motion vector generation unit 111 stores the motion vector information Vc (BlkIdx) regarding the small pixel block 23 corresponding to the variable BlkIdx in the reference motion vector memory 109 (step S406). The motion vector generation unit 111 increments the variable BlkIdx by 1 (step S407), and compares the variable BlkIdx with the constant BLK_MAX (step S408). Here, the constant BLK_MAX is the maximum value of the variable BlkIdx, and “15” is set in this example.

  If the variable BlkIdx exceeds the constant BLK_MAX (step S408), the process proceeds to step S409. If the variable BlkIdx is equal to or less than the constant BLK_MAX (step S408), the process returns to step S404.

  In step S409, the prediction image signal integration unit 115 integrates all the small prediction images 24 obtained by the processing loop of steps S404 to S408, and generates the prediction image 11 corresponding to the encoding target block (step S409). ), The motion vector block prediction process for the encoding target block ends.

Hereinafter, an application example of the motion vector block prediction process will be conceptually described with reference to FIG.
In FIG. 19, the horizontal axis represents the time direction, and the vertical axis represents only the vertical direction of the spatial direction. More specifically, C0, C2, C8, and C10 in the frame at time t indicate the small pixel blocks 23 corresponding to the indexes 0, 2, 8, and 10 of the encoding target block, respectively. Based on the motion vector block information 21 corresponding to the block to be encoded, motion vector information Vc (0), Vc (2), Vc (8) and Vc (10) corresponding to the small pixel blocks C0, C2, C8 and C10. Reference motion vector information Vr (0), Vr (2), Vr (8) and Vr (10) are respectively substituted. When motion vector information Vc (0), Vc (2), Vc (8) and Vc (10) corresponding to the small pixel blocks C0, C2, C8 and C10 is generated, the small predicted images P0, P2, P8 and P10 is generated respectively. As described above, when motion vector block prediction processing is applied, if only motion vector block information 21 is encoded, individual motion vector information 18 can be used in generating small predicted images 24 for each small pixel block 23. It becomes possible. On the other hand, in the normal inter prediction process, in order to use the individual motion vector information 18 in generating the small predicted image 24, it is necessary to encode the individual motion vector information 18. In this regard, the motion vector block prediction process can reduce the amount of code related to motion vector information compared to the inter prediction process.

Hereinafter, an example different from FIG. 19 of the motion vector block prediction process will be conceptually described with reference to FIG.
In the example of FIG. 19, the reference frame is fixed. However, as described above, the reference frame may be arbitrarily selected. In the example shown in FIG. 20, it is assumed that which reference frame is referenced is identified by a reference frame number, and the reference frame number includes a temporal shift amount from the frame to which the encoding target block belongs to the reference frame. The corresponding value shall be set. For example, it is assumed that “0” is set as the reference frame number for the reference frame immediately before the frame to which the block to be encoded belongs, and “1” is set as the reference frame number for the reference frame two times before.

  In FIG. 20, the horizontal axis represents the time direction, and the vertical axis represents only the vertical direction of the spatial direction. More specifically, C0, C2, C8, and C10 in the frame at time t indicate the small pixel blocks 23 corresponding to the indexes 0, 2, 8, and 10 of the encoding target block, respectively. Based on the motion vector block information 21 corresponding to the block to be encoded, motion vector information Vc (0), Vc (2), Vc (8) and Vc (10) corresponding to the small pixel blocks C0, C2, C8 and C10. Reference motion vector information Vr (0), Vr (2), Vr (8) and Vr (10) are respectively substituted. Here, the reference motion vector information Vr (0), Vr (2), Vr (8) and Vr (10) include not only the reference motion vector which is a spatial shift amount in the reference frame but also the temporal shift amount. The reference frame number is also included, and the reference frame number is also substituted as motion vector information Vc (0), Vc (2), Vc (8), and Vc (10). When motion vector information Vc (0), Vc (2), Vc (8) and Vc (10) corresponding to the small pixel blocks C0, C2, C8 and C10 is generated, the small predicted images P0, P2, P8 and P10 is generated respectively. Thus, in the motion vector block prediction process, not only the reference motion vector but also the reference frame number is substituted as the motion vector information 18 corresponding to the small pixel block 23, so that the reference frame of the reference frame is generated in the generation of the small predicted image 24. Different individual motion vector information 18 can be used.

Hereinafter, the SKIP mode in the motion vector block prediction process will be described.
First, the spatial SKIP mode in H.264 will be described so that the SKIP mode in the motion vector block prediction process can be easily understood. In the spatial SKIP mode, the motion vector information 18 and the quantized transform coefficient 13 relating to the encoding target block are not encoded. Specifically, as shown in FIG. 21, a reference image (same as the encoding target block) referenced by the median value of the motion vector information 18 of the encoded macro blocks A, B, and C adjacent to the encoding target block. Size) is used as the decoded image and the local decoded image 16 as they are. Therefore, according to the spatial SKIP mode, it is not necessary to encode the motion vector information 18 of the encoding target block, the quantized transform coefficient 13, and the like.

  On the other hand, the SKIP mode in the motion vector block prediction process is as follows. In the following description, the SKIP mode is referred to as a motion vector block SKIP mode.

  In the motion vector block SKIP mode, the motion vector block information 21 corresponding to the encoding target block is not encoded. Specifically, the motion vector block 28 that has already been encoded and is spatially in the same position in the previous reference motion vector frame 114 is used as the motion vector block 28 corresponding to the encoding target block. It is done. That is, the motion vector block information 21 is “0 (same position)”. Also in the motion vector block SKIP mode, it is not necessary to encode the quantized transform coefficient 13 corresponding to the encoding target block, and each reference image referred to by the reference motion vector information 18 included in the motion vector block 28 (The same size as the small pixel block 23) are integrated and used as the decoded image and the local decoded image 16 as they are.

  As another variation of the motion vector block SKIP mode, the motion vector block information 21 corresponding to the encoding target block is derived using the motion vector information 18 or the motion vector block information 21 in the already encoded adjacent block. Also good. Specifically, median values, average values, maximum values, minimum values, and the like of the motion vector block information 21 corresponding to a plurality of adjacent blocks may be derived as the motion vector block information 21 corresponding to the encoding target block. .

  Hereinafter, an application example of the motion vector block prediction process to the bidirectional prediction process will be described. In the B slice in H.264, two pieces of motion vector information 18 are derived for the encoding target block, and the final predicted image 11 is obtained by weighted averaging in units of pixels of the predicted image based on the two motion vector information 18. The bi-directional prediction process to generate is performed. When the motion vector block prediction process is applied to the bidirectional prediction process, two motion vector blocks 28 are generated for one encoding target block, and two motion vector blocks are formed for each small pixel block 23 constituting the encoding target block. Motion vector information 18 is provided. Based on the two motion vector information 18, the motion compensation unit 113 performs a prediction process similar to the existing bidirectional prediction process on each small pixel block 23 to generate a small predicted image 24.

  For example, as shown in FIG. 24A, when two motion vector blocks are generated from two frames in the past direction, motion vector block information and motion vector blocks having a short temporal distance are represented as list 0 motion vector block information and list. The motion vector block information and the motion vector block having a long time distance can be identified as the list 1 motion vector block information and the list 1 motion vector block, respectively. List 0 and list 1 may be reversed.

  In addition, as shown in FIG. 24B, if one motion vector block is generated from one frame in the past direction and one frame in the future direction, a list of motion vector block information and motion vector blocks related to the frame in the past direction is listed. The motion vector block information and the motion vector block regarding the frame in the future direction can be identified as the list 1 motion vector block information and the list 1 motion vector block, respectively. List 0 and list 1 may be reversed.

In addition, the motion vector block prediction process can be similarly applied to the following cases.
(A) When two motion vector blocks exist in the same frame (B) When three or more motion vector blocks are generated for one encoding target block Hereinafter, the image encoding device of FIG. 1 uses A syntax structure to be described will be described.
As shown in FIG. 25, the syntax structure used by the image encoding apparatus of FIG. 1 is composed of three parts: a high level syntax 901, a slice level syntax 904, and a macroblock level syntax 907. The high level syntax 901 describes syntax information regarding a layer (sequence or picture) higher than the slice, and includes a sequence parameter set syntax 902 and a picture parameter set syntax 903. The slice level syntax 904 describes syntax information regarding a slice, and includes a slice header syntax 905 and a slice data syntax 906. The macro block level syntax 907 describes syntax information regarding a macro block, and includes a macro block layer syntax 908 and a macro block prediction syntax 909.

  As shown in FIG. 26, the macroblock layer syntax 908 describes macroblock type information mb_type. The macro block type information mb_type includes information such as a prediction process (intra prediction or inter prediction) performed on the macro block and a block shape.

  As shown in FIG. 27, the macroblock prediction syntax 909 includes a prediction mode in units of small pixel blocks (such as an 8 × 8 pixel block or a 16 × 16 pixel block) in a macroblock and a motion vector corresponding to the macroblock. Block information 21 is described. In FIG. 27, the list 0 motion vector block information (spatial shift amount in the reference motion vector frame 114) described above is expressed as mvblk_l0, and the list 1 motion vector block information is expressed as mvblk_l1. When only one motion vector block 28 is used for one encoding target block, the list 1 motion vector block information mvblk_1 is not described, and when three or more motion vector blocks 28 are used. List 2 motion vector block information mvblk_2 and the like are described.

  In the macroblock prediction syntax 909, a reference frame number ref_mvblk_idx_l0 representing a reference motion vector frame to which the list 0 motion vector block belongs and a reference frame number ref_mvblk_idx_l1 representing a reference motion vector frame to which the list 1 motion vector block belongs are described. .

  If the reference motion vector frame to which the list 0 motion vector block and the list 1 motion vector block belong is fixed to the previously encoded reference motion vector frame or the like, the reference frame numbers ref_mvblk_idx_l0 and ref_mvblk_idx_l1 Need not be encoded. When the reference frame numbers ref_mvblk_idx_l0 and ref_mvblk_idx_l1 are not encoded, for example, fixed values such as “0” are automatically substituted as the reference frame numbers ref_mvblk_idx_l0 and ref_mvblk_idx_l1.

  The macroblock prediction syntax 909 shown in FIG. 27 can be similarly described for sub-macroblocks (8 × 8 pixel blocks or less), and an example is shown in FIG.

  26 to 28 merely illustrate the syntax structure used by the image coding apparatus according to the present embodiment, and modifications may be made as appropriate. Specifically, in FIGS. 26 to 28, other syntax elements may be inserted, or other conditional branch descriptions may be included. 26 to 28, the syntax table may be divided or integrated into a plurality of tables. In addition, the terms described in FIGS. 26 to 28 may be changed. Furthermore, the syntax element described in the macroblock layer syntax 908 shown in FIG. 26 may be described in a macroblock data syntax described later.

  As described above, the image coding apparatus according to the present embodiment performs prediction processing using motion vector blocks in which motion vector information corresponding to each of small pixel blocks obtained by further subdividing the block to be coded is collected. The motion vector block information is encoded without individual encoding of the motion vector information. Therefore, the image coding apparatus according to the present embodiment can perform motion compensation prediction in units of small pixel blocks that are finer than the current block while reducing the amount of code related to motion vector information. Efficiency can be realized.

(Second Embodiment)
As shown in FIG. 22, the image coding apparatus according to the second embodiment of the present invention replaces the inter prediction unit 101 with a prediction unit 116 in the image coding apparatus shown in FIG. In the following description, the same parts in FIG. 22 as those in FIG. 1 are denoted by the same reference numerals, and different parts will be mainly described.

As illustrated in FIG. 23, the prediction unit 116 includes an inter prediction unit 101, an intra prediction unit 117, a mode determination unit 118, and a mode selection switch 119.
The inter prediction unit 101 performs the motion vector block prediction described in the first embodiment, and inputs the predicted image 11 to the mode selection switch 119.

  The intra prediction unit 117 performs intra prediction processing in so-called H.264. In H.264, the 16 × 16 pixel block unit intra prediction process shown in FIG. 6A, the 8 × 8 pixel block unit intra prediction process shown in FIG. 6B, and the 4 × 4 pixel block unit intra prediction process shown in FIG. It is prescribed. In any intra prediction process, the predicted image 11 is generated by copying the pixel value of an already encoded block in the same frame as the encoding target block. The intra prediction unit 117 inputs the predicted image 11 to the mode selection switch 119.

  The mode determination unit 118 determines a prediction mode to be selected by the mode selection switch 119 based on the prediction information 22 regarding the encoding target block (or slice), and inputs the determination result to the mode selection switch 119 as prediction mode determination information 26. To do.

  If the encoding target slice is a so-called I slice, the mode determination unit 118 connects the mode selection switch 119 to the intra prediction unit 117. On the other hand, if the encoding target slice is a so-called P slice or B slice, the mode determination unit 118 performs prediction mode determination based on the cost.

The mode determination unit 118 calculates the cost according to the following formula (5), for example.

  In Equation (5), K is the coding cost, SAD is the sum of absolute differences between the encoding target block and the predicted image 11 (that is, the sum of absolute values of the prediction error 12), and OH is the amount of code related to the prediction information 22. (For example, the code amount of the motion vector block information 21 and the code amount of the block shape, etc.) respectively. Further, λ is a Lagrange undetermined constant determined by the quantization width or the like. The mode determination unit 118 selects the prediction mode with the lowest cost K as the optimal prediction mode, and inputs the prediction mode determination information 26 to the mode selection switch 119.

  In addition, the cost that can be used by the mode determination unit 118 is not limited to the cost K in Equation (5), but a value calculated based only on the predicted image 11, a value calculated based only on the SAD, A value obtained by subjecting SAD to Hadamard transformation, a value approximated thereto, and the like may be used. The mode determination unit 118 may use a value calculated based on the activity (dispersion) of the pixel value of the encoding target block, or a value calculated based on the quantization width or the quantization parameter as the cost.

Further, the mode determination unit 118 temporarily encodes the prediction error 12 corresponding to the prediction image 11 from the inter prediction unit 101 and the prediction error 12 corresponding to the prediction image 11 from the intra prediction unit 117 by the temporary encoding unit. The encoding cost may be calculated using the generated code amount in this case and the sum of square errors between the encoding target block and the locally decoded image 16 and used for prediction mode determination. A specific formula for calculating the encoding cost is the following formula (6).

  In Equation (6), J represents the coding cost, D represents the coding distortion representing the sum of square errors between the block to be coded and the locally decoded image 16, and R represents the amount of code generated by provisional coding. λ is the same as Equation (5). The mode determination unit 118 selects the prediction mode with the lowest coding cost J as the optimal prediction mode, and inputs the prediction mode determination information 26 to the mode selection switch 119.

  When the encoding cost J in Equation (6) is used, provisional encoding processing and local decoding processing are required for each prediction mode. Therefore, the image encoding apparatus can be compared with the case where other costs described above are used. The circuit scale and the amount of calculation increase. However, the encoding cost J has higher reliability of prediction mode determination than the other costs described above, and an improvement in encoding efficiency can be expected. Further, the mode determination unit 118 may use the cost calculated based on only one of R and D without using the coding cost J in Equation (6) as it is, A cost calculated based on the approximate value may be used.

  The mode selection switch 119 acquires the predicted image 11 from either the inter prediction unit 101 or the intra prediction unit 117 according to the prediction mode determination information 26. The predicted image 11 acquired by the mode selection switch 119 is input to the subtraction unit 102 and the addition unit 106.

  As described above, the image coding apparatus according to the present embodiment performs the motion vector block prediction process and the so-called intra prediction process according to the first embodiment described above for each macroblock, each small pixel block, or each frame. Selective use. Therefore, according to the image coding apparatus according to the present embodiment, since a prediction process with higher coding efficiency is selected, an improvement in coding efficiency can be expected.

(Third embodiment)
As shown in FIG. 29, the image coding apparatus according to the third embodiment of the present invention replaces the prediction unit 116 with a prediction unit 125 and replaces the motion vector estimation unit 120 with the image coding apparatus shown in FIG. Furthermore, it is provided. In the following description, the same parts in FIG. 29 as those in FIG. 22 are denoted by the same reference numerals, and different parts will be mainly described.

  As illustrated in FIG. 30, the prediction unit 125 includes a first inter prediction unit 101, an intra prediction unit 117, a second inter prediction unit 121, a mode determination unit 122, and a mode selection switch 123.

  The first inter prediction unit 101 and the intra prediction unit 117 generate the prediction image 11 by performing the above-described motion vector block prediction processing and intra prediction processing, respectively, and input the prediction image 11 to the mode selection switch 123. In addition, in order to distinguish from the 2nd inter prediction part 121 mentioned later, the name of the inter prediction part 101 in FIG. 22 is changed into the 1st inter prediction part 101 in FIG. 29 for convenience.

  The second inter prediction unit 121 performs so-called inter prediction processing in H.264. The second inter prediction unit 121 generates the predicted image 11 based on motion vector information 25 from the motion vector estimation unit 120 described later and the reference image 17 from the reference image memory 107. The second inter prediction unit 121 includes one motion compensation unit 126 as illustrated in FIG.

  The motion compensation unit 126 performs motion compensation of the reference image 17 using the motion vector information 25. That is, the motion compensation unit 126 outputs the pixel block indicated by the motion vector information 25 in the reference image 17 as the predicted image 11.

  The basic functions of the mode determination unit 122 and the mode selection switch 123 are the same as those of the mode determination unit 118 and the mode selection switch 119 in FIG. 23, except that the prediction image 11 from the second inter prediction unit 121 can be selected. Different. When the prediction image 11 from the second inter prediction unit 121 is selected by the mode selection switch 119, the entropy encoding unit 104 performs entropy encoding processing on the motion vector information 25 instead of the motion vector block information 21. .

Hereinafter, the technical significance of selectively using normal inter prediction processing and motion vector block prediction processing will be described.
Since the normal inter prediction process uses the motion vector information 25 estimated by matching with the encoding target block as described later, the prediction accuracy is high. However, as the encoding target block is divided (that is, the more motion compensation target blocks are provided), the amount of code increases because the motion vector information 25 that needs to be encoded increases accordingly.

  On the other hand, the motion vector block prediction process does not need to encode the individual motion vector information 18, but uses a set of reference motion vector information 19 as the motion vector block 28. The degree of freedom in selecting the vector information 18 is low.

  In other words, the second inter prediction process is superior to the other in terms of suppressing the prediction error, and the motion vector prediction process is superior in terms of suppressing the code amount. Therefore, it is possible to further increase the encoding efficiency by selectively using both.

  The motion vector estimation unit 120 estimates the motion vector information 25 based on the encoding target block and the reference image 17. Specifically, the motion vector estimation unit 120 performs block matching between the encoding target block and the interpolated image of the reference image 17. In the block matching, for example, the motion vector estimation unit 120 calculates a value obtained by accumulating errors between the pixels for each pixel, and estimates motion vector information having the smallest accumulated value as the motion vector information 25. Further, the motion vector estimation unit 120 may use a value obtained by performing some conversion on the accumulated value as an evaluation value. Furthermore, the motion vector estimation unit 120 may use the size and the code amount of the motion vector information 25 and Equation (5) or (6) for estimation. In addition, the motion vector estimation unit 120 may perform a full search within the search range based on search range information specified from the outside of the encoding device, or may perform a hierarchical search according to pixel accuracy. The motion vector estimation unit 120 may output the motion vector information 25 input from the encoding control unit 130 without performing the search process.

  The image coding apparatus according to the first embodiment described above uses the macroblock prediction syntax 909 shown in FIG. 27, but the image coding apparatus according to the present embodiment uses the macroblock prediction syntax shown in FIG. ACTION syntax 909 is used.

  In FIG. 36, mv_block_in_mb_flag is a flag indicating whether or not motion vector block prediction processing is applied to a macroblock. If mv_block_in_mb_flag is “1 (TRUE)”, the motion vector block prediction process is applied to the macroblock, and if “0 (FALSE)”, the motion vector block prediction process is not applied to the macroblock. When mv_block_in_mb_flag is “1 (TRUE)”, the list 0 motion vector block information mvblk_l0, the list 1 motion vector block information is mvblk_l1, and the list 0 motion vector block is similar to the macroblock prediction syntax 909 shown in FIG. The reference frame number ref_mvblk_idx_l0, the reference frame number ref_mvblk_idx_l1 of the list 1 motion vector block, and the like are described. The macroblock prediction syntax 909 shown in FIG. 36 can be similarly described for sub-macroblocks (8 × 8 pixel blocks or less), and an example is shown in FIG.

  The image encoding apparatus according to the present embodiment uses a sequence parameter set syntax 902 shown in FIG. In FIG. 32, mv_block_in_seq_flag is a flag indicating whether or not motion vector block prediction processing is applicable in the sequence. If mv_block_in_seq_flag is “1 (TRUE)”, motion vector block prediction processing can be applied within the sequence, and if “0 (FALSE)”, motion vector block prediction processing cannot be applied within the sequence. .

  The image encoding apparatus according to the present embodiment uses a picture parameter set syntax 903 shown in FIG. In FIG. 33, mv_block_in_pic_flag is a flag indicating whether or not motion vector block prediction processing is applicable in a picture, and is described when mv_block_in_seq_flag described above is “1 (TRUE)”. If mv_block_in_pic_flag is “1 (TRUE)”, the motion vector block prediction process can be applied in the picture, and if “0 (FALSE)”, the motion vector block prediction process cannot be applied in the picture. .

  The image encoding apparatus according to the present embodiment uses a slider header syntax 905 shown in FIG. In FIG. 34, mv_block_in_slice_flag is a flag indicating whether or not motion vector block prediction processing is applicable in the slice, and is described when the above-described mv_block_in_pic_flag is “1 (TRUE)”. If mv_block_in_slice_flag is “1 (TRUE)”, the motion vector block prediction process can be applied within the slice, and if “0 (FALSE)”, the motion vector block prediction process cannot be applied within the slice. .

  The image encoding apparatus according to the present embodiment uses a macroblock layer syntax 908 shown in FIG. In FIG. 35, mv_block_in_mb_flag is described when mv_block_in_slice_flag described above is “1 (TRUE)”.

  Note that the initial values of the aforementioned flags (syntax elements) mv_block_in_seq_flag, mv_block_in_pic_flag, mv_block_in_slice_flag, and mv_block_in_mb_flag are “0 (FALSE)”.

  The information indicated by the syntax may be encoded and transmitted to the decoding side, or the information indicated by the syntax may be encoded and transmitted to the decoding side without being transmitted to the decoding side. Switching may be performed according to activity information such as distribution.

  Also, the image coding apparatus according to the present embodiment can use the above-described motion vector block SKIP mode. When the motion vector block SKIP mode is applied, encoding of syntax elements such as mv_block_in_mb_flag described above may be omitted under specific conditions. The specific condition is that all motion vector information constituting a motion vector block obtained by applying the motion vector block SKIP mode matches motion vector information obtained by applying the so-called spatial SKIP mode. It is.

Hereinafter, the process of omitting the encoding of mv_block_in_mb_flag will be described using the flowchart of FIG.
First, the motion vector generation unit 111 generates motion vector information MVblk1, MVblk2, MVblk3,..., MVblk16 included in the motion vector block 28 when the motion vector block SKIP mode is applied (step S501). In this example, the size of the motion vector block is 16 × 16 pixels, and the motion compensation block size of the motion vector information 18 is 4 × 4 pixels.

  If the motion vector information MVblk1, MVblk2, MVblk3,..., MVblk16 generated in step S501 are all equal, the process proceeds to step S503, and if not, the process proceeds to step S507 (step S502).

  In step S503, the motion vector generation unit 111 derives the motion vector information MVspa when the spatial SKIP mode is applied from the adjacent block of the encoding target block. If the motion vector information MVspa is equal to the motion vector information MVblk1 (= MVblk2 = MVblk3 =... = MVblk16), the process proceeds to step S505. Otherwise, the process proceeds to step S507 (step S507).

  In step S505, “0 (FALSE)” is set in mv_block_in_mb_flag, encoding of the mv_block_in_mb_flag is omitted (step S506), and the process ends. In step S507, mv_block_in_mb_flag is encoded, and the process ends.

  As described above, by performing the encoding omission processing of mv_block_in_mb_flag, unnecessary encoding of mv_block_in_mb_flag can be avoided and the code amount can be reduced. In addition to mv_block_in_mb_flag, the same encoding omission processing is also performed for the syntax elements indicating whether or not to apply the motion vector block prediction processing for the small pixel block obtained by dividing the macro block, and mv_block_in_seq_flag, mv_block_in_pic_flag, mv_block_in_slice_flag described above. Is possible. That is, in the above description, the size of the encoding target block may be replaced with a sequence, a picture, and a slice small pixel block, respectively. In addition, when motion vector block prediction processing other than the motion vector block SKIP mode is applied, if the list 0 and list 1 motion vector block information is encoded in advance and the reference position in the reference motion vector frame is specified, Similar processing can be performed.

  As described above, the image coding apparatus according to the present embodiment performs the motion vector block prediction process and the so-called intra prediction process and inter prediction process in the first embodiment described above for each macroblock, each small pixel block, or each pixel block. It is selectively used for each frame. Therefore, according to the image coding apparatus according to the present embodiment, since a prediction process with higher coding efficiency is selected, an improvement in coding efficiency can be expected.

(Fourth embodiment)
As shown in FIG. 40, an image decoding apparatus according to the fourth embodiment of the present invention is an image decoding apparatus corresponding to the image encoding apparatus according to the first embodiment described above, and includes an input buffer 200, The image decoding unit 2000 includes a decoding control unit 230 and an output buffer 208. The image decoding apparatus in FIG. 40 may be realized by hardware such as an LSI chip, or may be realized by causing a computer to execute an image decoding program. The image decoding unit 2000 includes an entropy decoding unit 201, an inverse transform / inverse quantization unit 202, an addition unit 203, a reference image memory 204, an inter prediction unit 205, and a motion vector derivation unit 218.

  Encoded data 300 input from a transmission system or storage system (not shown) is once stored in the input buffer 200 and input to the image decoding unit 2000 in a multiplexed state.

  The entropy decoding unit 201 performs syntax analysis based on the syntax for each frame or field, and decodes various data. Specifically, the entropy decoding unit 201 decodes the quantized transform coefficient 301, the prediction information 304, and the motion vector block information 309 corresponding to the decoding target block. The entropy decoding unit 201 inputs the quantized transform coefficient to the inverse transform / inverse quantization unit 202, the prediction information 304 to the motion vector deriving unit 218 and the inter prediction unit 205, and the motion vector block information 309 to the motion vector deriving unit 218, respectively. To do. In addition, the entropy decoding unit 201 also decodes information necessary for image decoding, such as a quantization parameter, and inputs the information as feedback information 331 to the decoding control unit 230.

  The inverse transform / inverse quantization unit 202 performs inverse quantization / inverse transform on the quantized transform coefficient 301 from the entropy decoding unit 201 to decode the prediction error, and outputs the decoded prediction error 302 to the addition unit 203. input. The inverse quantization process is performed according to the quantization parameter set by the decoding control unit 230. Moreover, the said inverse transformation process is an inverse transformation process of the transformation process performed by the encoding side, Comprising: For example, it is IDCT or inverse wavelet transformation.

The adding unit 203 adds the decoded prediction error 302 from the inverse transform / inverse quantization unit 202 and the predicted image 305 from the inter prediction unit 205 to generate a decoded image (decoded image signal) 303 of the decoding target block. To do. The adding unit 203 stores the decoded image 303 in the reference image memory 204 as a reference image (reference image signal) 306. The decoded image 303 is temporarily stored in the output buffer 208, and is output to the outside of the image encoding device in FIG. 40 according to the output timing managed by the decoding control unit 230. For example, the reference image 306 is stored in units of frames, and is read by the inter prediction unit 205 as necessary.

  The inter prediction unit 205 has the same function as the inter prediction unit 101 in FIG. Specifically, as illustrated in FIG. 42, the inter prediction unit 205 includes a prediction image signal integration unit 213 corresponding to the prediction image signal integration unit 115, and a plurality of motion compensation units 212 corresponding to the plurality of motion compensation units 113. Have

  Each of the motion compensation units 212 generates a small predicted image 312 for each of a plurality of small pixel blocks obtained by dividing the decoding target block. The motion compensation unit 212 inputs the small predicted image 312 to the predicted image signal integration unit 213.

  The motion compensation unit 212 generates, as a small predicted image 312, a pixel block at a position spatially shifted in accordance with the motion vector information 307 regarding the small pixel block from the same position as the small pixel block in the reference image 306. The motion compensation process performed by the motion compensation unit 212 on the small pixel block can be realized in the same manner as the motion compensation process in H.264.

  The prediction image signal integration unit 213 integrates the small prediction images from the plurality of motion compensation units 212 and generates a prediction image 305 corresponding to the decoding target block. That is, by integrating the small predicted images 312 corresponding to the individual small pixel blocks, a predicted image 305 corresponding to the decoding target block before being divided into the small pixel blocks is obtained.

  The motion vector deriving unit 218 has the same function as the motion vector deriving unit 124 in FIG. Specifically, as shown in FIG. 39, the motion vector deriving unit 218 includes a reference motion vector memory 207 corresponding to the reference motion vector memory 109 and a motion vector block processing unit 206 corresponding to the motion vector block processing unit 108. Have.

  In the reference motion vector memory 207, already decoded motion vector information 307 is temporarily stored as reference motion vector information 308.

  The motion vector processing unit 206 acquires the reference motion vector information 308 from the reference motion vector memory 207 based on the motion vector block information 309 and the prediction information 304 from the entropy decoding unit 201, and derives a plurality of motion vector information 307. . More specifically, as shown in FIG. 41, the motion vector processing unit 206 includes a motion vector block generation unit 210 corresponding to the motion vector block generation unit 110, a motion vector generation unit 211 corresponding to the motion vector generation unit 111, Have

  The motion vector block generation unit 210 acquires the motion vector block information 309 from the entropy decoding unit 201, and generates a motion vector block 311 that is a set of reference motion vector information 308 corresponding to the motion vector block information 309. The motion vector block generation unit 210 inputs the motion vector block 311 to the motion vector generation unit 211.

  Based on the reference motion vector information 308 included in the motion vector block 311 from the motion vector block generation unit 210, the motion vector generation unit 211 obtains motion vector information 307 regarding each of a plurality of small pixel blocks obtained by dividing the decoding target block. Generate and output. Specifically, the motion vector generation unit 211 substitutes appropriate reference motion vector information 308 as the motion vector information 307 regarding the small pixel block.

  As described above, the image decoding apparatus according to the present embodiment decodes the image encoded by the image encoding apparatus according to the first embodiment described above. Therefore, according to the image decoding apparatus according to the present embodiment, it is possible to reproduce a high-quality decoded image from relatively small encoded data.

(Fifth embodiment)
As shown in FIG. 43, an image decoding apparatus according to the fifth embodiment of the present invention is an image decoding apparatus corresponding to the image encoding apparatus according to the second embodiment described above. In the illustrated image decoding apparatus, the inter prediction unit 205 is replaced with a prediction unit 214. In the following description, the same parts in FIG. 43 as those in FIG. 40 are denoted by the same reference numerals, and different parts will be mainly described.

As shown in FIG. 44, the prediction unit 214 includes an inter prediction unit 205, an intra prediction unit 220, and a mode selection switch 221.
The inter prediction unit 205 performs the motion vector block prediction described in the first embodiment, and inputs the predicted image 305 to the mode selection switch 221. The intra prediction unit 220 has the same function as the intra prediction unit 117 in FIG. The intra prediction unit 220 inputs the predicted image 305 to the mode selection switch 221.

  The mode selection switch 221 acquires the predicted image 305 from either the inter prediction unit 205 or the intra prediction unit 220 according to the prediction information 304 from the entropy decoding unit 201. The predicted image 305 acquired by the mode selection switch 221 is input to the adding unit 203.

  As described above, the image decoding apparatus according to the present embodiment decodes the image encoded by the above-described image encoding apparatus according to the second embodiment. Therefore, according to the image decoding apparatus according to the present embodiment, it is possible to reproduce a high-quality decoded image from relatively small encoded data.

(Sixth embodiment)
As shown in FIG. 45, an image decoding apparatus according to the sixth embodiment of the present invention is an image decoding apparatus corresponding to the image encoding apparatus according to the third embodiment described above. In the illustrated image decoding apparatus, the prediction unit 214 is replaced with a prediction unit 223. In the following description, the same parts in FIG. 43 as those in FIG. 40 are denoted by the same reference numerals, and different parts will be mainly described.

As illustrated in FIG. 46, the prediction unit 223 includes a first inter prediction unit 205, an intra prediction unit 220, a second inter prediction unit 217, and a mode selection switch 216.
The first inter prediction unit 205 has the same function as the first inter prediction unit 205 in FIG. 44, performs a motion vector block prediction process, generates a prediction image 305, and inputs the prediction image 305 to the mode selection switch 216. The intra prediction unit 220 has the same function as the intra prediction unit 220 in FIG. 44, performs an intra prediction process, generates a prediction image 305, and inputs the prediction image 305 to the mode selection switch 216. In addition, in order to distinguish from the 2nd inter prediction part 217 mentioned later, the name of the inter prediction part 205 in FIG. 44 is changed into the 1st inter prediction part 205 in FIG. 44 for convenience.

  The second inter prediction unit 217 performs inter prediction processing in so-called H.264. The second inter prediction unit 217 generates a predicted image 305 based on the motion vector information 313 decoded by the entropy decoding unit 201 and the reference image 306 from the reference image memory 204. The second inter prediction unit 217 includes one motion compensation unit 222 as illustrated in FIG.

  The motion compensation unit 222 performs motion compensation of the reference image 306 using the motion vector information 313. That is, the motion compensation unit 222 outputs the pixel block indicated by the motion vector information 313 in the reference image 306 as the predicted image 305.

  The basic function of the mode selection switch 216 is the same as that of the mode selection switch 221 in FIG. 44 except that the prediction image 305 from the second inter prediction unit 217 can be selected.

  Moreover, although the encoding omission process of mv_block_in_mb_flag has been described in the third embodiment, the same decoding omission process can be performed also in the image decoding apparatus according to the present embodiment. By performing the above-described decoding omitting process, even if unnecessary mv_block_in_mb_flag is encoded on the encoding side, the image decoding apparatus in FIG. 45 can selectively decode only necessary mv_block_in_mb_flag. The amount can be reduced.

  As described above, the image decoding apparatus according to the present embodiment decodes the image encoded by the image encoding apparatus according to the third embodiment described above. Therefore, according to the image decoding apparatus according to the present embodiment, it is possible to reproduce a high-quality decoded image from relatively small encoded data.

  Note that the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the components without departing from the scope of the invention in the implementation stage. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. Further, for example, a configuration in which some components are deleted from all the components shown in each embodiment is also conceivable. Furthermore, you may combine suitably the component described in different embodiment.

For example, the same effect can be obtained by modifying the first to sixth embodiments as follows.
(1) The encoding / decoding order shown in FIG. 5 is merely an example, and may be, for example, a processing order from the lower right to the upper left of the screen or a processing order from the upper right to the lower left. Further, the processing order from the central part to the peripheral part may be a spiral, or the processing order from the peripheral part to the central part may be used.

  (2) Although the block sizes shown in FIGS. 6A to 6C are all square, the size of the block to be encoded is not limited to these. For example, 16 × 8 pixels, 8 × 16 pixels, 8 × 4 pixels, or 4 A rectangular shape such as × 8 pixels may be used. Also, different size encoding target blocks may be selectively used. In this case, encoding related to the size switching information is required, but it is desirable to realize encoding that balances encoding distortion and code amount by using the above-described cost or the like.

  (3) In the first to sixth embodiments, the example in which the luminance signal and the color difference signal are not divided and is limited to one color signal component has been described. However, the prediction process may divide the luminance signal and the color difference signal and apply the prediction process individually.

  In addition, it goes without saying that the present invention can be similarly implemented even if various modifications are made without departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 10 ... Original image 11 ... Predicted image 12 ... Prediction error 13 ... Quantization transform coefficient 14 ... Encoded data 15 ... Decoded prediction error 16 ... Local decoded image 17 ... Reference image 18 ... motion vector information 19 ... reference motion vector information 21 ... motion vector block information 22 ... prediction information 23 ... small pixel block 24 ... small prediction image 25 ... Motion vector information 26 ... Prediction mode determination information 28 ... Motion vector block 30 ... Coding control information 31 ... Feedback information 101 ... Inter prediction unit 102 ... Subtraction unit 103 ... Conversion Quantizer 104: Entropy encoder 105 ... Inverse transform / inverse quantizer 106 ... Adder 107 ... Reference image memory 108 ... Motion vector Lock processing unit 109 ... reference motion vector memory 110 ... motion vector block generation unit 111 ... motion vector generation unit 112 ... output buffer 113 ... motion compensation unit 114 ... reference motion vector frame 115 ... Predictive image signal integration part 116 ... Prediction part 117 ... Intra prediction part 118 ... Mode determination part 119 ... Mode selection switch 120 ... Motion vector estimation part 121 ... 2nd inter Prediction unit 122 ... mode selection switch 124 ... motion vector derivation unit 125 ... prediction unit 126 ... motion compensation unit 130 ... encoding control unit 200 ... input buffer 201 ... entropy decoding Conversion unit 202... Inverse transform / inverse quantization unit 203... Addition unit 204 .. reference image memory 205. Inter prediction unit 206 ... motion vector block processing unit 207 ... reference motion vector memory 208 ... output buffer 210 ... motion vector block generation unit 211 ... motion vector generation unit 212 ... motion compensation unit 213 ... Predictive image signal integration unit 214 ... Prediction unit 217 ... Second inter prediction unit 218 ... Motion vector derivation unit 220 ... Intra prediction unit 221 ... Mode selection switch 222 ... Motion compensation unit 223 ... Prediction unit 230 ... Decoding control unit 300 ... Encoded data 301 ... Quantized transform coefficient 302 ... Decoded prediction residual 303 ... Decoded image 304 ... Prediction information 305 ... Prediction image 306 ... Reference image 307 ... Motion vector information 308 ... Reference motion vector information 309 ... Motion vector block information 311 ... motion vector block 312 ... small prediction image 313 ... motion vector information 330 ... decoding control information 331 ... feedback information 901 ... high-level syntax 902 .. Sequence parameter set syntax 903 ... Picture parameter set syntax 904 ... Slice level syntax 905 ... Slice header syntax 906 ... Slice data syntax 907 ... Macroblock level syntax 908 ... Macroblock layer Syntax 909: Macroblock prediction syntax 1000, 1100, 1200 ... Image encoding unit 2000, 2100, 2200 ... Image decoding unit

Claims (2)

A decoding unit for decoding the encoded prediction error;
A storage unit for acquiring a first motion vector corresponding to a first pixel region that has already been decoded;
A first derivation unit for deriving a motion vector group including at least one of the first motion vectors;
A second deriving unit for deriving a second motion vector corresponding to each of the third pixel regions obtained by dividing the second pixel region before decoding based on the motion vector group;
A prediction unit that generates a first inter prediction image of each of the third pixel regions using the second motion vector;
An integration unit that integrates the first inter prediction image and generates a second inter prediction image corresponding to the second pixel region before decoding;
An image decoding apparatus comprising: an adder configured to add the second inter prediction image and the prediction error to generate a decoded image. Decoding the encoded prediction error;
Obtaining a first motion vector corresponding to a first pixel area that has already been decoded;
Deriving a group of motion vectors including at least one of the first motion vectors;
Deriving a second motion vector corresponding to each of the third pixel regions obtained by dividing the second pixel region before decoding based on the motion vector group;
Generating a first inter-predicted image for each of the third pixel regions using the second motion vector;
Integrating the first inter-predicted image to generate a second inter-predicted image corresponding to the second pixel region before decoding;
An image decoding method comprising: adding the second inter prediction image and the prediction error to generate a decoded image.

Images