JP2020074624A - Moving picture coding device and moving picture decoding method - Google Patents

Abstract

To provide a moving picture coding device including a motion information compression device capable of improving coding efficiency, and a moving picture decoding method.SOLUTION: A method of dividing an input image signal 151 into pixel blocks and performing inter prediction on the divided pixel blocks in an image coding device 100 includes a step of selecting the motion prediction information from a motion information memory 110 that holds motion information in an encoded area, and predicting motion information of an encoded target block using the predictive motion information, and a step of obtaining representative motion information according to first information indicating a method of selecting the motion vector predictor information from among a plurality of pieces of motion information in the area in which the encoding has been ended, and obtaining only the representative motion information.SELECTED DRAWING: Figure 1

Description

  Embodiments of the present invention relate to a motion information compression method, a moving image coding method, and a moving image decoding method in moving image coding and decoding.

  In recent years, an image coding method with greatly improved coding efficiency has been developed by ITU-T and ISO / IEC in cooperation with ITU-T Rec. H.264 and ISO / IEC 14496-10 (hereinafter H.264). Is said). In H.264, prediction processing, conversion processing, and entropy coding processing are performed in rectangular block units (for example, 16 × 16 pixel block units, 8 × 8 pixel block units, etc.). In the prediction process, motion compensation is performed on a rectangular block to be coded (block to be coded) by referring to a frame that has already been coded (reference frame) and performing prediction in the time direction. In such motion compensation, it is necessary to encode the motion information including the motion vector as the spatial shift information of the block to be coded and the block referred to in the reference frame and send it to the decoding side. Furthermore, when performing motion compensation using a plurality of reference frames, it is necessary to encode the reference frame number together with the motion information. For this reason, the code amount for the motion information and the reference frame number may increase. In addition, there is a motion information prediction method for deriving predicted motion information of an encoding target block by referring to motion information stored in a motion information memory of a reference frame (Patent Document 1 and Non-Patent Document 2). May increase the capacity of the motion information memory for storing.

  As an example of a method for reducing the capacity of the motion information memory, (Non-Patent Document 2) derives representative motion information within a predetermined block and stores only the representative motion information in the motion information memory.

Patent No. 4020789 J. Jung et al, “Temporal MV predictor modification for MV-Comp, Skip, Direct and Merge schemes”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 Document, JCTVC-D164, January 20110. Yeping Su et al, “CE9: Reduced resolution storage of motion vector data”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 Document, JCTVC-D072, January 20110.

  However, when the method of deriving the predicted motion information shown in Non-Patent Document 1 and the method of deriving the representative motion information shown in Non-Patent Document 2 are different, the time correlation of the predicted motion information is reduced, and therefore, the code related to the motion information There is a problem that the quantity is increased.

  The problems to be solved by the present invention have been made to solve the above problems, and provide a moving picture coding apparatus and a moving picture decoding apparatus including a motion information compression apparatus capable of improving coding efficiency. Is to provide.

  According to the embodiment, the moving image coding method is a method of dividing an input image signal into pixel blocks and performing inter prediction on the divided pixel blocks. This method includes selecting predicted motion information from a motion information buffer that holds motion information in a coded area, and predicting motion information of a current block using the predicted motion information. Furthermore, this method is to obtain representative motion information from a plurality of pieces of motion information in an area in which coding has been completed in accordance with first information indicating a method of selecting the predicted motion information, and obtain only the representative motion information. Including.

The block diagram which shows roughly the structure of the image coding apparatus which concerns on 1st Embodiment. Explanatory drawing of the predictive coding order of a pixel block. Explanatory drawing of an example of a pixel block size. Explanatory drawing of another example of a pixel block size. Explanatory drawing of another example of a pixel block size. Explanatory drawing of an example of the pixel block in a coding tree unit. Explanatory drawing of another example of the pixel block in a coding tree unit. Explanatory drawing of another example of the pixel block in a coding tree unit. Explanatory drawing of another example of the pixel block in a coding tree unit. FIG. 2 is a block diagram schematically showing a configuration of an entropy coding unit in FIG. 1. FIG. 3 is an explanatory diagram schematically showing the configuration of the motion information memory of FIG. 1. Explanatory drawing of an example of the inter prediction process which the inter prediction part of FIG. 1 performs. Explanatory drawing of another example of the inter prediction process which the inter prediction part of FIG. 1 performs. Explanatory drawing of an example of a prediction unit. Explanatory drawing of another example of a prediction unit. Explanatory drawing of another example of a prediction unit. Explanatory drawing of another example of a prediction unit. Explanatory drawing of another example of a prediction unit. Explanatory drawing of another example of a prediction unit. Explanatory drawing of another example of a prediction unit. Explanatory drawing which shows a skip mode, a merge mode, and an inter mode. FIG. 5 is a block diagram schematically showing the configuration of a motion information encoding unit in FIG. 4. Explanatory drawing which shows the example of the position of the prediction motion information candidate with respect to the encoding target prediction unit. Explanatory drawing which shows another example of the position of the prediction motion information candidate with respect to the encoding target prediction unit. Explanatory drawing which shows the example of the list | wrist which shows the relationship between the block position of several prediction motion information candidates, and index Mvpidx. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the center of a prediction unit in case the size of the encoding target prediction unit is 32x32. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the center of a prediction unit in case the size of the encoding target prediction unit is 32x16. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the center of a prediction unit when the size of the encoding target prediction unit is 16x32. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the center of a prediction unit, when the size of the encoding target prediction unit is 16x16. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the center of a prediction unit, when the size of the encoding target prediction unit is 16x8. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the center of a prediction unit when the size of the encoding target prediction unit is 8x16. Explanatory drawing which shows another example of the reference motion information acquisition position which shows the center of a prediction unit when the size of the encoding target prediction unit is 32x32. Explanatory drawing which shows another example of the reference motion information acquisition position which shows the center of a prediction unit in case the size of the encoding target prediction unit is 32x16. Explanatory drawing which shows another example of the reference motion information acquisition position which shows the center of a prediction unit, when the size of the encoding target prediction unit is 16x32. Explanatory drawing which shows another example of the reference motion information acquisition position which shows the center of a prediction unit when the size of the encoding target prediction unit is 16x16. Explanatory drawing which shows another example of the reference motion information acquisition position which shows the center of a prediction unit, when the size of the encoding target prediction unit is 16x8. Explanatory drawing which shows another example of the reference motion information acquisition position which shows the center of a prediction unit, when the size of the encoding target prediction unit is 8x16. Explanatory drawing regarding the space direction reference motion information memory 501 and the time direction reference motion information memory 502. 3 is a flowchart showing an example of the operation of the motion information compression unit in FIG. 1. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the upper left end of a prediction unit when the size of the encoding target prediction unit is 32x32. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the upper left end of a prediction unit when the size of the encoding target prediction unit is 32x16. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the upper left end of a prediction unit when the size of the encoding target prediction unit is 16x32. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the upper left end of a prediction unit when the size of the encoding target prediction unit is 16x16. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the upper left corner of a prediction unit in case the size of the encoding target prediction unit is 16x8. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the upper left end of a prediction unit in case the size of the encoding target prediction unit is 8x16. Explanatory drawing which shows the example of a representative motion information position. Explanatory drawing which shows another example of a representative motion information position. Explanatory drawing which shows the example of the center of the prediction unit in each prediction size. Explanatory drawing which shows the example of a representative motion information position when the gravity center of several reference motion information acquisition positions for every motion information compression block is set as a representative motion information position. FIG. 9 is an explanatory diagram showing another example of representative motion information positions when the center of gravity of a plurality of reference motion information acquisition positions for each motion information compression block is set as the representative motion information position. Explanatory drawing which shows the example of a representative motion information position. Explanatory drawing which shows another example of a representative motion information position. FIG. 6 illustrates a syntax structure according to one embodiment. FIG. 6 is a diagram showing an example of a sequence parameter set syntax according to one embodiment. It is a figure which shows another example of the sequence parameter set syntax according to one embodiment. It is a figure which shows an example of the prediction unit syntax according to one embodiment. The block diagram which shows the image decoding apparatus which concerns on 2nd Embodiment roughly. FIG. 26 is a block diagram schematically showing the entropy decoding unit of FIG. 25. FIG. 27 is a block diagram schematically showing the motion information decoding unit of FIG. 26.

Hereinafter, the moving image encoding device and the moving image decoding device according to each embodiment will be described in detail with reference to the drawings. Note that in the following description, the term "image" can be appropriately read as the terms "video", "pixel", "image signal", "image data", and the like. Further, in the following embodiments, the same operation is performed for the parts to which the same numbers are assigned, and the overlapping description will be omitted.
(First embodiment)
The first embodiment relates to an image encoding device. A moving picture decoding apparatus corresponding to the image coding apparatus according to this embodiment will be described in the second embodiment. This image encoding device can be realized by hardware such as an LSI (Large-Scale Integration) chip, a DSP (Digital Signal Processor), and an FPGA (Field Programmable Gate Array). The image coding apparatus can also be realized by causing a computer to execute the image coding program.

  As shown in FIG. 1, the image encoding device 100 according to the present embodiment refers to a subtraction unit 101, an orthogonal transformation unit 102, a quantization unit 103, an inverse quantization unit 104, an inverse orthogonal transformation unit 105, an addition unit 106. An image memory 107, an inter prediction unit 108, a motion information compression unit 109, a motion information memory 110, and an entropy coding unit 112 are included. The coding control unit 114 and the output buffer 113 are usually installed outside the image coding apparatus 100.

  The image coding apparatus 100 of FIG. 1 divides each frame, each field, or each slice forming the input image signal 151 into a plurality of pixel blocks, performs predictive coding on these divided pixel blocks, and codes the pixel blocks. The converted data 163 is output. In the following description, for simplification, it is assumed that pixel blocks are predictively encoded from upper left to lower right as shown in FIG. 2A. In FIG. 2A, the encoded pixel block p is located on the left side and the upper side of the encoding target pixel block c in the encoding target frame f.

  Here, the pixel block refers to a unit for processing an image, such as an M × N size block (N and M are natural numbers), a coding unit, a macro block, a sub block, and one pixel. In the following description, the pixel block is basically used in the meaning of a coding unit, but the pixel block can be interpreted in the above meaning by appropriately reading the description. The coding unit is typically, for example, a 16 × 16 pixel block shown in FIG. 2B, but may be a 32 × 32 pixel block shown in FIG. 2C or a 64 × 64 pixel block shown in FIG. 2D. It may be an 8 × 8 pixel block or a 4 × 4 pixel block. Also, the coding unit does not necessarily have to be square. Hereinafter, the coding target block or coding unit of the input image signal 151 may be referred to as a “prediction target block”. The coding unit is not limited to the pixel block like the coding unit, but a frame, a field, a slice, or a combination thereof can be used.

  3A to 3D are diagrams showing specific examples of the coding unit. FIG. 3A shows an example in which the size of the coding unit is 64 × 64 (N = 32). Here, N represents the size of the reference coding unit, and the size when divided is defined as N, and the case where it is not divided is defined as 2N. The coding tree unit has a quad tree structure, and when divided, four pixel blocks are indexed in the Z scan order. FIG. 3B shows an example in which the 64 × 64 pixel block of FIG. 3A is divided into quadtrees. The numbers shown in the figure represent the order of Z scanning. Further, it is possible to perform further quadtree partitioning within the index of one quadtree of the coding unit. The depth of division is defined by Depth. That is, FIG. 3A shows an example where Depth = 0. FIG. 3C shows an example of a 32 × 32 (N = 16) size coding tree unit when Depth = 1. The largest unit of such a coding tree unit is called a large coding tree unit or a tree block, and as shown in FIG. 2A, an input image signal is encoded in this unit in raster scan order.

  The image coding apparatus 100 in FIG. 1 is based on a coding parameter input from the coding control unit 114, and is inter prediction (also referred to as inter-frame prediction, inter-frame prediction, motion compensation prediction, etc.) for a pixel block, or Intra prediction (not shown) (also referred to as intra-frame prediction, intra-frame prediction, etc.) is performed to generate a predicted image signal 159. The image coding apparatus 100 orthogonally transforms and quantizes a prediction error signal 152 between a pixel block (input image signal 151) and a predicted image signal 159, performs entropy coding, and generates coded data 163. Output.

  The image coding apparatus 100 in FIG. 1 performs coding by selectively applying a plurality of prediction modes having different block sizes and different generation methods of the predicted image signal 159. The method of generating the predicted image signal 159 is roughly classified into two types, that is, intra prediction that performs prediction within a frame to be coded and inter prediction that performs prediction using one or more temporally different reference frames. is there.

Hereinafter, each element included in the image encoding device 100 of FIG. 1 will be described.
The subtraction unit 101 subtracts the corresponding prediction image signal 159 from the encoding target block of the input image signal 151 to obtain the prediction error signal 152. The subtraction unit 101 inputs the prediction error signal 152 to the orthogonal transformation unit 102.

  The orthogonal transform unit 102 performs an orthogonal transform such as a discrete cosine transform (DCT) on the prediction error signal 152 from the subtraction unit 101 to obtain a transform coefficient 153. The orthogonal transformation unit 102 outputs the transformation coefficient 153 to the quantization unit 103.

  The quantization unit 103 quantizes the transform coefficient 153 from the orthogonal transform unit 102 to obtain a quantized transform coefficient 154. Specifically, the quantization unit 103 performs quantization according to quantization information such as a quantization parameter and a quantization matrix designated by the encoding control unit 114. The quantization parameter indicates the fineness of quantization. The quantization matrix is used for weighting the fineness of the quantization for each component of the transform coefficient, but the use / non-use of the quantization matrix is not an essential part of the embodiment of the present invention. The quantization unit 103 outputs the quantized transform coefficient 154 to the entropy coding unit 112 and the inverse quantization unit 104.

  The entropy coding unit 112 has quantized transform coefficients 154 from the quantization unit 103, motion information 160 from the inter prediction unit 108, prediction information 165 designated by the coding control unit 114, and reference from the coding control unit 114. Entropy coding (for example, Huffman coding and arithmetic coding) is performed on various coding parameters such as position information 164 and quantization information to generate coded data 163. Note that the coding parameters are parameters necessary for decoding the prediction information 165, information about transform coefficients, information about quantization, and the like. For example, the encoding control unit 114 has an internal memory (not shown), the encoding parameter is held in this memory, and the encoding parameter of the already encoded pixel block adjacent to the prediction target block is encoded. To use.

  Specifically, the entropy coding unit 112 includes a parameter coding unit 401, a transform coefficient coding unit 402, a motion information coding unit 403, and a multiplexing unit 404, as shown in FIG. The parameter coding unit 401 codes the coding parameter such as the prediction information 165 received from the coding control unit 114 to generate the coded data 451A. The transform coefficient coding unit 402 codes the quantized transform coefficient 154 received from the quantizing unit 103 to generate coded data 451B.

  The motion information coding unit 403 codes the motion information 160 received from the inter prediction unit 108 with reference to the reference motion information 166 received from the motion information memory 110 and the reference position information 164 received from the coding control unit 114. To generate encoded data 451C. Details of the motion information encoding unit 403 will be described later.

  The multiplexing unit 404 multiplexes the coded data 451A, 451B, and 451C to generate coded data 163. The generated encoded data 163 includes the motion information 160 and the prediction information 165 as well as all parameters necessary for decoding such as information about transform coefficients and information about quantization.

  The encoded data 163 generated by the entropy encoding unit 112 is temporarily stored in the output buffer 113 through, for example, multiplexing, and is output as the encoded data 163 according to an appropriate output timing managed by the encoding control unit 114. It The encoded data 163 is output to, for example, a storage system (storage medium) or a transmission system (communication line) not shown.

  The inverse quantization unit 104 inversely quantizes the quantized transform coefficient 154 from the quantizer 103 to obtain a restored transform coefficient 155. Specifically, the inverse quantization unit 104 performs inverse quantization according to the quantization information used by the quantization unit 103. The quantization information used in the quantization unit 103 is loaded from the internal memory of the encoding control unit 114. The inverse quantization unit 104 outputs the restoration transform coefficient 155 to the inverse orthogonal transform unit 105.

  The inverse orthogonal transform unit 105 performs an inverse orthogonal transform corresponding to the orthogonal transform performed in the orthogonal transform unit 102, such as an inverse discrete cosine transform, on the restored transform coefficient 155 from the inverse quantization unit 104, A reconstruction prediction error signal 156 is obtained. The inverse orthogonal transformation unit 105 outputs the restoration prediction error signal 156 to the addition unit 106.

  The addition unit 106 adds the restored prediction error signal 156 and the corresponding predicted image signal 159 to generate a local decoded image signal 157. The decoded image signal 157 is input to the reference image memory 107 after being subjected to a deblocking filter, a Wiener filter or the like (not shown).

  The reference image memory 107 stores the locally decoded filtered image signal 158 in the memory and is referred to as the reference image signal 158 when the inter prediction unit 108 generates a predicted image as necessary.

  The inter prediction unit 108 uses the reference image signal 158 stored in the reference image memory 107 to perform inter prediction. Specifically, the inter prediction unit 108 performs block matching processing between the prediction target block and the reference image signal 158 to derive a motion shift amount (motion vector). The inter prediction unit 108 performs motion compensation (interpolation processing in the case of decimal precision motion) based on this motion vector to generate an inter prediction image. H. With H.264, interpolation processing up to ¼ pixel precision is possible. The derived motion vector is entropy coded as a part of the motion information 160.

  The motion information memory 110 includes a motion information compression unit 109, appropriately compresses the motion information 160 to reduce the amount of information, and temporarily stores it as reference motion information 166. As shown in FIG. 5, the motion information memory 110 is held in units of frames (or slices), and the spatial direction reference motion information memory 501 that stores the motion information 160 on the same frame as reference motion information 166, and It further has a temporal direction reference motion information memory 502 which stores the motion information 160 of the encoded frame as reference motion information 166. A plurality of temporal direction reference motion information memories 502 may be provided according to the number of reference frames used for prediction by the encoding target frame.

  Further, the spatial direction reference motion information memory 501 and the temporal direction reference motion information memory 502 may logically partition the physically same memory. Further, the spatial-direction reference motion information memory 501 holds only the spatial-direction motion information necessary for the frame currently being encoded, sequentially compresses the spatial-direction motion information for which reference is no longer necessary, and temporally references the reference motion. It may be stored in the information memory 502.

  The reference motion information 166 is held in the spatial direction reference motion information memory 501 and the time direction reference motion information memory 502 in units of a predetermined area (for example, in units of 4 × 4 pixel blocks). The reference motion information 166 further includes information indicating whether the area is coded by inter prediction described below or coded by intra prediction described below. In addition, the coding unit (or prediction unit) is an H.264 standard. Like the skip mode, the direct mode, or the merge mode described later, which is defined by H.264, the value of the motion vector in the motion information 160 is not coded, and the motion information 160 predicted from the coded area is used to interleave. Even in the case of prediction, the motion information of the coding unit (or the prediction unit) is held as the reference motion information 166.

  When the encoding process of the frame or slice to be encoded is completed, the spatial direction reference motion information memory 501 of the frame is changed in its treatment as the temporal direction reference motion information memory 502 used for the frame to be encoded next. It At this time, in order to reduce the memory capacity of the time-direction reference motion information memory 502, the motion information 160 compressed by the motion information compression unit 109 described later is stored in the time-direction reference motion information memory 502.

  The prediction information 165 is in accordance with the prediction mode controlled by the encoding control unit 114, and as described above, inter prediction or intra prediction (not shown) or inter prediction (not shown) for generating the predicted image signal 159 can be selected. Further, a plurality of modes can be further selected for each of the inter prediction and the inter prediction. The encoding control unit 114 determines one of a plurality of prediction modes of intra prediction and inter prediction as the optimum prediction mode, and sets the prediction information 165.

  For example, the encoding control unit 114 determines the optimal prediction mode using the cost function shown in the following mathematical expression (1).

  In Expression (1) (hereinafter, referred to as a simple coding cost), OH indicates the code amount regarding the prediction information 160 (for example, motion vector information, prediction block size information), and SAD indicates the prediction target block and the prediction image signal 159. The sum of absolute differences (i.e., cumulative sum of absolute values of the prediction error signal 152) is shown. Further, λ indicates a Lagrange undetermined multiplier determined based on the value of the quantization information (quantization parameter), and K indicates the coding cost. When Equation (1) is used, the prediction mode that minimizes the coding cost K is determined as the optimum prediction mode from the viewpoint of the generated code amount and prediction error. As a modification of Expression (1), the coding cost may be estimated from only OH or only SAD, or the coding cost may be estimated using a value obtained by subjecting SAD to Hadamard transform or its approximate value.

  It is also possible to determine the optimum prediction mode by using a temporary coding unit (not shown). For example, the encoding control unit 114 determines the optimum prediction mode using the cost function shown in the following mathematical expression (2).

  In Expression (2), D indicates the sum of squared errors (that is, coding distortion) between the prediction target block and the locally decoded image, and R indicates the prediction between the prediction target block and the prediction image signal 159 in the prediction mode. The code amount estimated by tentative coding for the error is shown, and J shows the coding cost. When deriving the coding cost J (hereinafter, referred to as detailed coding cost) of the mathematical expression (2), provisional coding processing and local decoding processing are required for each prediction mode, so that the circuit scale or the amount of calculation increases. .. On the other hand, since the coding cost J is derived based on more accurate coding distortion and code amount, it is easy to determine the optimum prediction mode with high accuracy and maintain high coding efficiency. As a modification of Equation (2), the coding cost may be estimated from only R or D, or the coding cost may be estimated using an approximate value of R or D. Also, these costs may be used hierarchically. The encoding control unit 114 makes a determination using Expression (1) or Expression (2) based on information obtained in advance regarding the prediction target block (a prediction mode of surrounding pixel blocks, a result of image analysis, etc.). The number of prediction mode candidates may be narrowed down in advance.

  As a modified example of the present embodiment, by performing a two-step mode determination that combines Equation (1) and Equation (2), it is possible to further reduce the number of prediction mode candidates while maintaining the coding performance. Becomes Here, unlike the mathematical expression (2), the simple coding cost represented by the mathematical expression (1) does not require local decoding processing, and thus can be calculated at high speed. In the moving image encoding device according to the present embodiment, the Even if compared with H.264, the number of prediction modes is large, and therefore mode determination using the detailed coding cost is not realistic. Therefore, as the first step, the mode determination using the simple coding cost is performed for the prediction modes available in the pixel block, and the prediction mode candidates are derived.

  Here, the number of prediction mode candidates is changed by utilizing the property that the correlation between the simple coding cost and the detailed coding cost increases as the value of the quantization parameter that defines the roughness of quantization increases.

Next, the prediction process of the image encoding device 100 will be described.
Although not shown, the image coding apparatus 100 of FIG. 1 is provided with a plurality of prediction modes, and the prediction image signal 159 generation method and the motion compensation block size are different in each prediction mode. The method for the prediction unit 108 to generate the predicted image signal 159 is specifically broadly divided into intra prediction (intra-frame) for generating a predicted image using the reference image signal 158 of the encoding target frame (or field). Prediction) and inter-prediction (inter-frame prediction) in which a predicted image is generated using the reference image signal 158 of one or more coded reference frames (or reference fields). The prediction unit 108 selectively switches between intra prediction and inter prediction to generate the predicted image signal 159 of the current block.

  FIG. 6A shows an example of inter prediction. The inter prediction is typically performed in units of prediction units, and it is possible to have different motion information 160 in units of prediction units. In inter prediction, as shown in FIG. 6A, a pixel block in a reference frame that has already been encoded (for example, an encoded frame that is one frame before), and a prediction unit that is an encoding target. The predicted image signal 159 is generated from the block 601 at the same position using the reference image signal 158 of the block 602 at the position spatially shifted according to the motion vector included in the motion information 160. That is, in the generation of the predicted image signal 159, the reference image signal 158 of the block 602 in the reference frame, which is specified by the position (coordinates) of the encoding target block and the motion vector included in the motion information 160, is used.

  In inter prediction, motion compensation with a small number of pixels (for example, 1/2 pixel accuracy or 1/4 pixel accuracy) is possible, and the value of the interpolated pixel is generated by performing filtering processing on the reference image signal 158. To be done. For example, H.264. In H.264, interpolation processing up to ¼ pixel accuracy is possible for luminance signals. The interpolation processing is based on H.264. This can be performed by using any filtering other than the filtering defined by H.264.

  It should be noted that in inter prediction, not only the example of using the reference frame one frame before as shown in FIG. 6A, but any coded reference frame may be used as shown in FIG. 6B. When the reference image signals 158 of a plurality of reference frames having different time positions are held, information indicating which time position of the reference image signal 158 the predicted image signal 159 is generated is represented by a reference frame number. The reference frame number is included in the motion information 160. The reference frame number can be changed in area units (pictures, slices, block units, etc.). That is, a different reference frame may be used for each prediction unit. As an example, when the coded reference frame one frame before is used for prediction, the reference frame number of this area is set to 0, and when the coded reference frame two frames before is used for prediction, The reference frame number of this area is set to 1. As another example, when the reference image signal 158 for only one frame is held in the reference image memory 107 (the number of held reference frames is only one), the reference frame number is always 0. Is set.

  Furthermore, in the inter prediction, a size suitable for the encoding target block can be selected and used from the sizes of a plurality of prediction units prepared in advance. For example, it is possible to perform motion compensation for each prediction unit obtained by dividing a coding tree unit as shown in FIGS. 7A to 7G. Further, it is possible to perform motion compensation for each prediction unit obtained by dividing into a rectangle other than the rectangle as shown in FIGS. 7F and 7G.

  As described above, since the motion information 160 of the coded pixel block (for example, 4 × 4 pixel block) in the coding target frame used for inter prediction is held as the reference motion information 166, the input image signal According to the local property of 151, the optimum motion compensation block shape and motion vector, and reference frame number can be used. Further, the coding unit and the prediction unit can be arbitrarily combined. When the coding tree unit is a 64 × 64 pixel block, for each of the four coding tree units (32 × 32 pixel blocks) obtained by dividing the 64 × 64 pixel block, further dividing the coding tree unit into four. Thus, it is possible to hierarchically use a 16 × 16 pixel block from a 64 × 64 pixel block. Similarly, it is possible to hierarchically use a block of 64 × 64 pixels to a block of 8 × 8 pixels. Here, if the prediction unit is a coding tree unit divided into four, it is possible to perform hierarchical motion compensation processing from a 64 × 64 pixel block to a 4 × 4 pixel block. ..

  In addition, in inter prediction, bidirectional prediction using two types of motion compensation can be performed on a pixel block to be encoded. H. In H.264, two types of motion compensation are performed on the pixel block to be encoded, and two types of predicted image signals are weighted and averaged to obtain a new predicted image signal (not shown). In bidirectional prediction, two types of motion compensation are called list 0 prediction and list 1 prediction, respectively.

<Explanation of skip mode, merge mode, and inter mode>
The image coding apparatus 100 according to the present embodiment uses a plurality of prediction modes having different coding processes shown in FIG. The skip mode in the drawing is a mode in which only the syntax relating to the motion vector predictor information position 954, which will be described later, is coded and the other syntaxes are not coded. The merge mode is a mode in which only the syntax related to the motion vector predictor position 954 and the transform coefficient information 153 are coded, and the other syntaxes are not coded. The inter mode is a mode in which the syntax related to the predicted motion information position 954, differential motion information 953, and transform coefficient information 153, which will be described later, are encoded. These modes are switched by the prediction information 165 controlled by the encoding control unit 114.

<Motion information coding unit 403>
The motion information encoding unit 403 will be described below with reference to FIG.

  The motion information coding unit 403 includes a reference motion vector acquisition unit 901, a motion vector predictor selection switch (also referred to as motion vector predictor selection switch) 902, a subtraction unit 903, a differential motion information coding unit 904, and motion vector predictor position coding. It has a unit 905 and a multiplexing unit 906.

  The reference motion vector acquisition unit 901 receives the reference motion information 166 and the reference position information 164 and generates at least one or more predicted motion information candidates (also referred to as predicted motion vector candidates) 951 (951A, 951B, ...). .. 10 and 11 show an example of the position of the motion vector predictor candidate 951 with respect to the target prediction unit. FIG. 10 shows the position of the prediction unit spatially adjacent to the target prediction unit. AX (X = 0 to nA-1) is a prediction unit adjacent to the left of the target prediction unit, and BY (Y = 0 to nB-1) is a prediction unit adjacent to the top of the target prediction unit. Function units C, D, and E indicate prediction units that are adjacent to the target prediction unit at the upper right, upper left, and lower left, respectively. Further, FIG. 11 shows the position of the prediction unit in the already encoded reference frame with respect to the encoding target prediction unit. Col in FIG. 11 indicates a prediction unit located at the same position as the encoding target prediction unit in the reference frame. FIG. 12 shows an example of a list showing the relationship between the block positions of the plurality of motion vector predictor information candidates 951 and the index Mvpidx. Mvpidx 0 to 2 indicates a motion vector predictor candidate 951 located in the spatial direction, and Mvpidx 3 indicates a motion vector predictor candidate 951 located in the time direction. The prediction unit position A is a prediction unit having inter-prediction, that is, the reference motion information 166 in AX shown in FIG. 10, and the position where the value of X is the smallest is the prediction unit position A. Further, the prediction unit position B is an inter prediction among BY shown in FIG. 10, that is, a prediction unit having reference motion information 166, and a position having the smallest Y value is referred to as a prediction unit position A. To do. When the prediction unit position C is not inter prediction, the reference motion information 166 of the prediction unit position D is replaced with the reference motion information 166 of the prediction unit position C. When the prediction unit positions C and D are not inter prediction, the reference motion information 166 of the prediction unit position E is replaced with the reference motion information 166 of the prediction unit position C.

  When the size of the prediction target prediction unit is larger than the minimum prediction unit, the prediction unit position Col may hold a plurality of reference motion information 166 in the temporal direction reference motion information memory 502. In this case, the reference motion information 166 in the prediction unit at the position Col is acquired according to the reference position information 164. Hereinafter, the acquisition position of the reference motion information 166 in the prediction unit at the position Col will be referred to as the reference motion information acquisition position. 13A to 13F show examples of reference motion information acquisition positions when the reference position information 164 indicates the center of the prediction unit at the position Col, for each size (32x32 to 16x16) of the prediction target prediction unit. Each block in the drawing indicates a 4x4 prediction unit, and a circle indicates the position of the 4x4 prediction unit acquired as the motion vector predictor candidate 951. Another example of the reference motion information acquisition position is shown in FIGS. In FIGS. 14A to F, since the 4x4 prediction unit does not exist at the position of the circle, a predetermined method such as the average value or the median value of the reference motion information 166 in the four 4x4 prediction units adjacent to the circle, The motion vector predictor candidate 951 is generated. As still another example of the reference motion information acquisition position, the reference motion information 166 of the 4 × 4 prediction unit located at the upper left end of the prediction unit at the position Col may be the predicted motion information candidate 951. Even if the method is not limited to the above example, any position and method may be used to generate the motion vector predictor candidate 951 as long as it is a predetermined method.

  When the reference motion information 166 does not exist, the motion information 160 having a zero vector is output as the motion vector predictor candidate 951.

  By the above, at least one or more prediction motion information candidates 951 are output from the reference motion block. When the reference frame number of the motion vector predictor candidate 951 and the reference frame number of the prediction target prediction unit are different from each other, the motion vector predictor candidate 951 includes the reference frame number of the motion vector predictor candidate 951 and the prediction target prediction target. The scaling may be performed according to the reference frame number of the unit.

  The predictive motion information selection switch 902 selects one of the plurality of predictive motion information candidates 951 according to a command from the encoding control unit 114, and outputs the predictive motion information 952. Further, the motion vector predictor selection switch 902 may output motion vector predictor position information 954, which will be described later. The selection may be performed using an evaluation function such as Equations (1) and (2). The subtraction unit 903 subtracts the predicted motion vector information 952 from the motion information 160, and outputs the difference motion information 953 to the difference motion information encoding unit 904. The differential motion information encoding unit 904 encodes the differential motion information 953 and outputs encoded data 960A. In the skip mode and the merge mode, the differential motion information coding unit 904 does not need to code the differential motion information 953.

  The motion vector predictor position encoding unit 905 encodes motion vector predictor position information 954 (Mvpidx) indicating which motion vector predictor candidate 951 has been selected from the list shown in FIG. 12, and outputs coded data 960B. To do. The motion vector predictor information position information 954 is coded using isometric coding or variable length coding generated from the total number of motion vector predictor candidate 951. Variable length coding may be performed using the correlation with adjacent blocks. Further, when the plurality of motion vector predictor candidates 951 have overlapping information, a code table is created from the total number of motion vector predictor information candidates 951 in which the redundant motion vector predictor information candidates 951 are deleted, and the motion vector predictor position information 954 is encoded. It doesn't matter. In addition, when the total number of motion vector predictor candidates 951 is one, the motion vector predictor information candidate 951 is determined as motion vector predictor motion information 952, and thus it is not necessary to encode the motion vector predictor position information 954.

  Further, the derivation method of the motion vector predictor candidate 951 need not be the same in each of the skip mode, the merge mode, and the inter mode, and the derivation method of the motion vector predictor information candidate 951 may be set independently. In the present embodiment, the method of deriving the motion vector predictor information candidate 951 in the skip mode and the inter mode is the same, and the method of deriving the motion vector predictor information candidate 951 in the merge mode is different.

<Details of the motion information compression unit 109>
First, the motion information compression process will be described with reference to FIG. In FIG. 15, the reference motion information 166 in the spatial direction reference motion information memory 501 is compressed and stored in the temporal direction reference motion information memory 502. In the spatial direction reference motion information memory 501, the reference motion information 166 held at the representative motion information position is stored in the temporal direction reference motion information memory 502 for each motion information compression block (16 × 16 pixel block in the figure). When performing the above-described motion information encoding process, the reference motion information 166 held at the reference motion information acquisition position is set as the motion vector predictor candidate 951. At this time, the reference motion information 166 held at the reference motion information acquisition position described above may be set as the predicted motion information candidate 951 by assuming that the motion information compression block virtually has the same reference motion information 166. No (the same predicted motion information candidate 951 is derived).
Next, the motion information compression unit 109 will be described using the flowchart shown in FIG. The motion information compressing unit 109 compresses the motion information 160 and stores the motion information 160 in the temporal direction reference motion information memory 502 when the encoding process of a frame (or an arbitrary unit such as a slice or a coding unit) is completed. ..

  First, the reference position information 164 is acquired from the encoding control unit 114 (step S1601), and the frame is divided into motion information compressed blocks that are compression units of the motion information 160 (step S1602). The motion information compression block is a pixel block larger than a unit (typically a 4x4 pixel block) in which the motion information 160 is held by the motion compensation process, and is typically a 16x16 pixel block. The motion information compression block may be a 64x64 pixel block, a 32x32 pixel block, an 8x8 pixel block, a rectangular pixel block, or a pixel area of any shape.

  Next, the representative motion information position is generated according to the reference position information 164 (step S1603). As an example of generating the representative motion information position, when the motion information compression block is a 16 × 16 pixel block, the reference motion information acquisition position when the size of the prediction unit shown in FIGS. 13D, 14D, and 17D is 16 × 16 is representative. This is the motion information position. Next, the reference motion information 166 at the generated representative motion information position is set as the representative motion information (step S1604), and the representative motion information is stored in the temporal direction reference motion information memory (step S1605). The above steps S1604 to S1605 are executed for all motion information compressed blocks.

  When the unit for holding the motion information 160 is MxM block and the size of the motion information compression block is NxN (N is a multiple of M), the motion information compression process is executed to increase the capacity of the reference motion information memory by (MxM ) / (NxN).

<Another Embodiment of Representative Motion Information Position>
As another example of generating the representative motion information position, the center position of the plurality of reference motion information acquisition positions may be used as the representative motion information position. 18A and 18B show representative motion information positions for each motion compression block having a size of 16 × 16. FIG. 18A is a representative motion information position when the reference motion information acquisition position is the position shown in FIG. 13D. Similarly, FIG. 18B is a representative motion information position when the reference motion information acquisition position is the position shown in FIG. 17D. Each position is shown. Circles in FIG. 18A and FIG. 18B indicate reference motion information acquisition positions when the prediction unit is a 16 × 16 block, and are set to the center position (also referred to as the center of gravity position) of the four reference motion information acquisition positions. Representative motion information positions indicated by crosses are arranged.

  As still another example of generating the representative motion information position, the reference motion information acquisition position for each size of the plurality of prediction units is provided as the reference position information 164, and the representative motion information position is generated from the plurality of reference motion information acquisition positions. It doesn't matter.

  As an example of generating the representative motion information position, even if the reference motion information acquisition position for each size of the plurality of prediction units is provided as the reference position information 164 and the representative motion information position is generated from the plurality of reference motion information acquisition positions. I do not care. FIG. 19 shows the center (reference motion information acquisition position) of the prediction unit in each size of 16 × 16 or more when the tree block is a 64 × 64 pixel block.

  As another example of generating the representative motion information position, the representative motion information position may be set using the reference motion information acquisition position arranged for each motion information compression block. FIG. 20A shows an example in which the center of gravity of a plurality of reference motion information acquisition positions for each motion information compression block is set as the representative motion information position. When the position of the center of gravity does not match the position of the 4x4 block, the nearest 4x4 block may be used as the representative motion information position, or the reference motion vector 166 of the position of the center of gravity may be determined by using an interpolation method such as bilinear interpolation. You can generate it.

  Also, FIG. 20B shows an example in which one of the plurality of reference motion information acquisition positions is selected for each motion information compressed block and set as the representative motion information position.

  Further, FIGS. 21A and 21B further show an example in which the reference motion information acquisition positions are made the same in each motion information compression block in the tree block. Since the representative motion information position is the same in all motion information compression blocks, it is not necessary to switch the representative motion information position according to the position in the tree block. 21A and 21B, the representative motion information position may be at any position such as the upper left end or the upper right end in the motion information compression block.

  As an example of generating the representative motion information position, BlkIdx indicating the 4x4 block position in the motion information compression block in Z scan order may be used to indicate the representative motion information position. When the size of the motion information compression block is 16 × 16, the representative motion information position shown in FIG. 21A corresponds to the position of BlkIdx = 12. The representative motion information position shown in FIG. 21B corresponds to the position of BlkIdx = 15.

  As another example of the motion information compression process, the reference frame number may be included in the motion information compression process in order to reduce the memory capacity related to the reference frame number. In this case, the reference frame number held at the representative motion information position is stored in the memory capacity related to the reference frame number. Therefore, the spatial direction reference motion information memory 501 and the spatial direction reference motion information memory 502 shown in FIG. 5 store the reference frame number in addition to the motion vector information.

  As still another example of the motion information compression processing, when the reference frame number is not included in the motion information compression processing, the motion vector information in the motion information at the representative motion information position is subjected to scaling processing using the reference frame number. Then, it may be stored in the motion information memory 110. A typical example of the scaling process is a linear scaling process with reference frame number zero as a reference. This is to perform linear scaling processing so that when the reference frame number is a value other than zero, the motion vector information refers to the reference frame corresponding to the reference frame number zero. The reference of the reference frame number may be a value other than zero as the standard for the above-described scaling processing. When division occurs when performing the above-described linear scaling processing, the division processing may be made into a table in advance and the table may be drawn each time to realize the division.

  If the size of the motion information compression block is other than 16 × 16 blocks, the representative motion information position is generated using the same processing as described above. In one example, when the size of the motion information compression block is 64x64, the reference motion information acquisition position when the size of the prediction unit is 64x64 is set as the representative motion information position. In yet another example, the representative motion information position in which the size of the motion information compression block shown in FIG. 21A, FIG. 21B, etc. is 16 × 16 blocks is scaled in the horizontal direction and the vertical direction according to the size of the motion information compression block. It may be an information position.

  If the reference motion information does not exist because the representative motion information position is outside the picture or slice, the position at which the reference motion information can be acquired in the motion information compression block, such as the upper left end of the motion information compression block, is set as the new representative motion. The information position may be replaced. Also, when the representative motion information position is an area to which intra prediction is applied and reference motion information does not exist, the same process may be executed to replace it with a new representative motion information position.

<Syntax configuration>
The syntax used by the image coding apparatus 100 in FIG. 1 will be described below.
The syntax indicates the structure of the coded data (for example, the coded data 163 in FIG. 1) when the image coding apparatus codes the moving image data. When decoding this encoded data, the moving image decoding apparatus performs syntax interpretation by referring to the same syntax structure. A syntax 2200 used by the moving picture coding apparatus of FIG. 1 is illustrated in FIG.

  The syntax 2200 includes three parts of a high level syntax 2201, a slice level syntax 2202 and a coding tree level syntax 2203. The high level syntax 2201 includes syntax information of a layer higher than the slice. A slice refers to a rectangular area or a continuous area included in a frame or field. The slice level syntax 2202 contains the information needed to decode each slice. The coding tree level syntax 2203 includes information necessary for decoding each coding tree (that is, each coding tree unit). Each of these parts contains more detailed syntax.

  High-level syntax 2201 includes sequence and picture-level syntax, such as sequence parameter set syntax 2204 and picture parameter set syntax 2205. The slice level syntax 2202 includes a slice header syntax 2206 and a slice data syntax 2207. The coding tree level syntax 2203 includes a coding tree unit syntax 2208, a transform unit syntax 2209, a prediction unit syntax 2210, and the like.

  The coding tree unit syntax 2208 may have a quadtree structure. Specifically, the coding tree unit syntax 2208 can be recursively called as a syntax element of the coding tree unit syntax 2208. That is, one coding tree unit can be subdivided into quadtrees. Further, the coding tree unit syntax 2208 includes a transform unit syntax 2209 and a pre-cushion unit syntax 2210. The transform unit syntax 2209 and the pre-cushion unit syntax 2210 are called in each coding tree unit syntax 2208 at the end of the quadtree. The predicate cushion unit syntax 2210 describes information related to prediction, and the transform unit syntax 2209 describes information related to inverse orthogonal transform and quantization.

FIG. 23 illustrates a sequence parameter set syntax 2204 according to this embodiment. The motion_vector_buffer_comp_flag shown in FIG. 23A and FIG. 23B is a syntax indicating whether the motion information compression according to the present embodiment is valid or invalid for the sequence. When the motion_vector_buffer_comp_flag is 0, the motion information compression according to the present embodiment is invalid for the sequence. Therefore, the processing of the motion information compression unit shown in FIG. 1 is skipped. As an example, when the motion_vector_buffer_comp_flag is 1, the motion information compression according to the present embodiment is effective for the sequence. The motion_vector_buffer_comp_ratio_log2 shown in FIGS. 23 and 23B is information indicating a unit of the motion information compression process, and is indicated when the motion_vector_buffer_comp_flag is 1. The motion_vector_buffer_comp_ratio_log2 indicates, for example, information on the size of the motion information compression block according to the present embodiment, and the motion_vector_buffer_comp_ratio_log2 is motion_compress_block2_motion_complex_value 2 which is the minimum unit of the motion compensation and the motion compensation ^{vector_buffer_buffer_comp_r_buffer2_motion_buffer_buffer_comp_value} . An example in which the minimum unit of motion compensation is a 4x4 pixel block, that is, the reference motion information memory is held in a 4x4 pixel block unit, is shown below. When motion_vector_buffer_comp_ratio_log2 is 1, the size of the motion information compression block according to the present embodiment is 8 × 8 pixel block. Similarly, when motion_vector_buffer_comp_ratio_log2 is 2, the size of the motion information compression block according to this embodiment is 16 × 16 pixel blocks. The motion_vector_buffer_comp_position shown in FIG. 23B is information indicating the representative motion information position in the motion information compression block, and is indicated when the motion_vector_buffer_comp_flag is 1. The motion_vector_buffer_comp_position indicates the reference motion information position in the motion information compression block as shown in FIGS. 21A and 21B, or the reference motion information position for each motion information compression block as shown in FIGS. 20A and 20B. It doesn't matter. It may also be located at the center of a plurality of blocks.

  Further, as another example, a layer lower than the motion_vector_buffer_comp_flag_motion, motion_vector_buffer_comp_ratio_log2, motion_vector_buffer_comp_position_localization region, a layer lower than the motion element vector syntax, slice level syntax, or a unit of a slice unit syntax of the motion parameter vector buffer_compress_position local slice unit, a coding unit tree, a slice tree syntax unit, a coding unit tree, and the like. The validity / invalidity of such prediction may be defined.

  FIG. 24 shows an example of the prediction unit syntax. Skip_flag in the figure is a flag indicating whether or not the prediction mode of the coding unit to which the prediction unit syntax belongs is the skip mode. When the skip_flag is 1, it indicates that the syntax (coding unit syntax, prediction unit syntax, transform unit syntax) other than the predicted motion information position information 954 is not coded. NumMVPCand (L0) and NumMVPCand (L1) indicate the numbers of motion vector predictor information candidates 951 in the list 0 prediction and the list 1 prediction, respectively. When the motion vector predictor candidate 951 exists (NumMVPCand (LX)> 0, X = 0 or 1), mvp_idx_lX indicating the motion vector predictor position information 954 is encoded.

  When skip_flag is 0, it indicates that the prediction mode of the coding unit to which the prediction unit syntax belongs is not the skip mode. NumMergeCandidates indicates the number of motion vector predictor information candidates 951 derived in FIG. When the motion vector predictor candidate 951 exists (NumMergeCandidates> 0), merge_flag, which is a flag indicating whether or not the prediction unit is in the merge mode, is encoded. The merge_flag indicates that the prediction unit is in the merge mode when the value is 1, and indicates that the prediction unit uses the inter mode when the value is 0. When the merge_flag is 1 and there are two or more prediction motion information candidates 951 (NumMergeCandidates> 1), the prediction motion information 952, ie, the merge_idx, indicating which block is to be merged from among the prediction motion information candidates 951 is coded. ..

  When the merge_flag is 1, it is not necessary to encode the prediction unit syntax other than the merge_flag and the merge_idx.

  When the merge_flag is 0, it indicates that the prediction unit is in the inter mode. In the inter mode, in the case of mvd_lX (X = 0 or 1) indicating the differential motion vector information included in the differential motion information 953, the reference frame number ref_idx_lX, or the B slice, the prediction unit is unidirectional prediction (list 0 or list 1). Or inter_pred_idc indicating whether the prediction is bi-directional prediction is encoded. Also, similar to the skip mode, NumMVPCand (L0) and NumMVPCand (L1) are acquired, and when the motion prediction information candidate 951 exists (NumMVPCand (LX)> 0, X = 0 or 1), the motion prediction information position information 954. Is coded.

  The above is the syntax configuration according to the present embodiment.

(Second embodiment)
The second embodiment relates to a moving picture decoding device. The moving picture coding apparatus corresponding to the moving picture decoding apparatus according to this embodiment is as described in the first embodiment. That is, the moving picture decoding apparatus according to the present embodiment decodes the coded data generated by the moving picture coding apparatus according to the first embodiment, for example.

  As shown in FIG. 25, the moving image decoding apparatus according to this embodiment includes an entropy decoding unit 2501, an inverse quantization unit 2502, an inverse orthogonal transformation unit 2503, an addition unit 2504, a reference image memory 2505, and an inter prediction unit 2506. , A reference motion information memory 2507, a reference motion information compression unit 2508, and a decoding control unit 2510.

  The moving picture decoding apparatus in FIG. 25 decodes the encoded data 2550, accumulates the decoded image signal 2554 in the output buffer 2511, and outputs it as an output image. The coded data 2550 is output from, for example, the moving picture coding apparatus in FIG. 1, and is input to the moving picture decoding apparatus 2500 via a storage system or a transmission system (not shown).

  The entropy decoding unit 2501 decodes the encoded data 2550 based on the syntax for decoding. The entropy decoding unit 2501 sequentially entropy-decodes the code string of each syntax, and reproduces the coding parameters of the coding target block such as the motion information 2559 and the quantized transform coefficient 2551. The coding parameters are parameters necessary for decoding prediction information, transform coefficient information, quantization information, and the like.

  Specifically, the entropy decoding unit 2501 includes a separation unit 2601, a parameter decoding unit 2602, a transform coefficient decoding unit 2603, and a motion information decoding unit 2604, as shown in FIG. The separation unit 2601 separates the encoded data 2550, the encoded data 2651A regarding the parameter into the parameter decoding unit 2602, the encoded data 2651B regarding the transform coefficient into the transform coefficient decoding unit 2603, and the encoded data 2651C regarding the motion information into the motion information. It outputs to each of the decoding units 2604. The parameter decoding unit 2602 decodes the coding parameter 2570 such as prediction information, outputs the coding parameter 2570, and outputs the coding parameter 2570 to the decoding control unit 2510. The transform coefficient decoding unit 2603 receives the encoded data 2651B as input, decodes the transform coefficient information 2551 and outputs it to the inverse quantization unit 2502.

  The motion information decoding unit 2604 receives the encoded data 2651C from the separation unit 2601, the reference position information 2560 from the decoding control unit 2510, the reference motion information 2558 from the reference motion information memory 2507, and outputs the motion information 2559. The output motion information 2559 is input to the inter prediction unit 2506.

  As shown in FIG. 27, the motion information decoding unit 2604 includes a separation unit 2701, a differential motion information decoding unit 2702, a motion vector predictor position decoding unit 2503, a reference motion information acquisition unit 2704, a motion vector selection switch 2705, and The adding unit 2706 is included.

  The encoded data 2651C regarding the motion information is input to the separation unit 2701, and is separated into the encoded data 2751 regarding the differential motion information and the encoded data 2752 regarding the predicted motion information position. The differential motion information coding unit 2702 inputs the coded data 2751 related to the differential motion information, and decodes the differential motion information 2753. The differential motion information 2753 is added to the prediction motion information 2756 described later by the adding unit 2706, and the motion information 2759 is output. The motion vector predictor position decoding unit 2703 receives the encoded data 2752 relating to the motion vector predictor position and decodes the motion vector predictor position 2754.

  The predicted motion information position 2754 is input to the predicted motion information selection switch 2705, and the predicted motion information 2756 is selected from the predicted motion information candidates 2755. The predicted motion information position information 2560 is decoded using isometric decoding or variable length decoding generated from the number of predicted motion information candidates 2755. Variable length decoding may be performed using the correlation with adjacent blocks. Furthermore, when a plurality of motion vector predictor information candidates 2755 overlap, the motion vector predictor information position information 2560 may be decoded from the code table generated from the total number of motion vector predictor information candidates 2755 with the duplication removed. Also, when the total number of motion vector predictor candidates 2755 is one, the motion vector predictor information candidate 2755 is determined as motion vector predictor information 2556, and thus it is not necessary to decode the motion vector predictor position information 2754.

  The reference motion information acquisition unit 2704 has the same configuration and processing content as the reference motion information acquisition unit 901 described in the first embodiment.

  The reference motion information acquisition unit 2704 receives the reference motion information 2558 and the reference position information 2560 and generates at least one or more predicted motion information candidates 2755 (2755A, 2755B, ...). FIG. 10 and FIG. 11 show an example of the positions of the motion vector predictor candidates 2755 with respect to the decoding target prediction unit. FIG. 10 shows the position of the prediction unit spatially adjacent to the decoding target prediction unit. AX (X = 0 to nA-1) is a prediction unit that is adjacent to the left of the target prediction unit, and BY (Y = 0 to nB-1) is a prediction unit that is adjacent to the target prediction unit above. Function units, C, D, and E, indicate prediction units that are adjacent to the decoding target prediction unit at the upper right, upper left, and lower left, respectively. Further, FIG. 11 shows the position of the prediction unit in the reference frame that has already been decoded with respect to the decoding target prediction unit. Col in the figure indicates a prediction unit located at the same position as the decoding target prediction unit in the reference frame. FIG. 12 shows an example of a list showing the relationship between the block positions of the plurality of motion vector predictor information candidates 2755 and the index Mvpidx. Mvpidx 0 to 2 indicate the predicted motion information candidate 2755 located in the spatial direction, and Mvpidx 3 indicates the motion vector predictor candidate 2755 located in the time direction. The prediction unit position A is a prediction unit having inter-prediction, that is, the reference motion information 2558 in AX shown in FIG. 10, and the position where the value of X is the smallest is the prediction unit position A. Further, the prediction unit position B is an inter prediction among BY shown in FIG. 10, that is, a prediction unit having reference motion information 2558, and a position having the smallest Y value is referred to as a prediction unit position A. To do. When the prediction unit position C is not inter prediction, the reference motion information 2558 of the prediction unit position D is replaced with the reference motion information 2558 of the prediction unit position C. When the prediction unit positions C and D are not inter prediction, the reference motion information 2558 of the prediction unit position E is replaced with the reference motion information 2558 of the prediction unit position C.

  When the size of the decoding target prediction unit is larger than the minimum prediction unit, the prediction unit position Col may hold a plurality of reference motion information 2558 in the temporal reference motion information memory 2507. In this case, the reference motion information 2558 in the prediction unit at the position Col is acquired according to the reference position information 2560. Hereinafter, the acquisition position of the reference motion information 2558 in the prediction unit at the position Col will be referred to as the reference motion information acquisition position. 13A to 13F show an example of the reference motion information acquisition position when the reference position information 2560 indicates the center of the prediction unit at the position Col, for each size (32x32 to 16x16) of the prediction target prediction unit. Each block in the figure indicates a 4x4 prediction unit, and a circle indicates the position of the 4x4 prediction unit acquired as the motion vector predictor candidate 2755. Another example of the reference motion information acquisition position is shown in FIGS. In FIGS. 14A to 14F, since the 4x4 prediction unit does not exist at the position of the circle, in a predetermined method such as the average value or the median value of the reference motion information 2558 in the four x4 prediction units adjacent to the circle, The predicted motion information candidate 2755 is generated. As still another example of the reference motion information acquisition position, the reference motion information 2558 of the 4 × 4 prediction unit located at the upper left end of the prediction unit at the position Col may be used as the predicted motion information candidate 2755. Even if it puts besides the above-mentioned example, if it is a predetermined system, you may generate prediction motion information candidate 2755 using any position and a system.

  If the reference motion information 2558 does not exist, the motion information 2559 having the zero vector is output as the motion vector predictor candidate 2755.

  By the above, at least one or more prediction motion information candidates 2755 are output from the reference motion block. When the reference frame number of the motion vector predictor candidate 2755 and the reference frame number of the decoding target prediction unit are different, the motion vector predictor candidate 2755 is set to the reference frame number of the motion vector predictor candidate 2755 and the prediction target prediction unit. The scaling may be performed according to the reference frame number of. The motion vector predictor selection switch 2705 selects one of the motion vector predictor information candidates 2755 according to the motion vector predictor information position 2754 and outputs motion vector predictor information 952.

  The dequantization unit 2502 dequantizes the quantized transform coefficient 2551 from the entropy decoding unit 2501 to obtain a restored transform coefficient 2552. Specifically, the dequantization unit 2502 performs dequantization in accordance with the quantization information decoded by the entropy decoding unit 2501. The inverse quantization unit 2502 outputs the restoration transform coefficient 2552 to the inverse orthogonal transform unit 2503.

  The inverse orthogonal transform unit 2503 performs an inverse orthogonal transform corresponding to the orthogonal transform performed on the encoding side on the restored transform coefficient 2552 from the inverse quantization unit 2502, and obtains a restored prediction error signal 2553. The inverse orthogonal transformation unit 2503 inputs the restoration prediction error signal 2553 to the addition unit 2504.

  The addition unit 2504 adds the restored prediction error signal 2553 and the corresponding predicted image signal 2556 to generate a decoded image signal 2554. The decoded image signal 2554 is subjected to a deblocking filter or a Wiener filter (not shown), is temporarily stored in the output buffer 2511 for the output image, and is also stored in the reference image memory 2505 for the reference image signal 2555. It The decoded image signal 2554 stored in the reference image memory 2505 is referred to as a reference image signal 2555 by the inter prediction unit 2506 in a frame unit or a field unit as needed. The decoded image signal 2554 temporarily stored in the output buffer 2511 is output according to the output timing managed by the decoding control unit 2510.

  The inter prediction unit 2506 performs inter prediction using the reference image signal 2555 stored in the reference image memory 2505. Specifically, the inter prediction unit 2506 acquires, from the entropy decoding unit 2501, motion information 2559 including the amount of motion deviation (motion vector) between the prediction target block and the reference image signal 2555, and uses this motion vector as the motion vector. Interpolation processing (motion compensation) is performed based on this to generate an inter prediction image. The generation of the inter-predicted image is the same as that in the first embodiment, so the description thereof will be omitted.

  The decoding control unit 2510 controls each element of the moving picture decoding apparatus in FIG. Specifically, the decoding control unit 2510 outputs reference position information 2560 described below to the entropy decoding unit 2501 and performs various controls for the decoding process including the above-described operation.

<Explanation of skip mode, merge mode, and inter mode>
The image decoding apparatus 2500 according to this embodiment uses a plurality of prediction modes having different decoding processes shown in FIG. The skip mode in the drawing is a mode in which only the syntax related to the motion vector predictor position 2754 described below is decoded and the other syntaxes are not decoded. The merge mode is a mode in which only the syntax related to the motion vector predictor position 2754 and the transform coefficient information 2551 are decoded, and the other syntaxes are not decoded. The inter mode is a mode for decoding the syntax regarding the predicted motion information position 2754, differential motion information 2753, and transform coefficient information 2551 described later. These modes are switched by the prediction information 2571 controlled by the decoding control unit 2510.

  Further, since the moving picture decoding apparatus in FIG. 25 uses the same or similar syntax as the syntax described in FIG. 28, detailed description thereof will be omitted.

<Details of the motion information compression unit 2508>
Next, the motion information compression unit 2508 will be described using the flowchart shown in FIG. The motion information compressing unit 2508 compresses the motion information 2559 and stores the motion information 2559 in the temporal direction reference motion information memory 502 when the decoding process of a frame (or an arbitrary unit such as a slice or a coding unit) is completed. ..

  First, the reference position information 2560 is acquired from the decoding control unit 2510 (step S1601), and the frame is divided into motion information compressed blocks that are compression units of the motion information 2559 (step S1602). The motion information compression block is a pixel block larger than a unit (typically a 4x4 pixel block) in which the motion information 2559 is held by the motion compensation process, and is typically a 16x16 pixel block. The motion information compression block may be a 32x32 pixel block, an 8x8 pixel block, a rectangular pixel block, or a pixel area of any shape.

  Next, the representative motion information position is generated according to the reference position information 2560 (step S1603). As an example of generating the representative motion information position, when the motion information compression block is a 16 × 16 pixel block, the reference motion information acquisition position when the size of the prediction unit shown in FIGS. 13D, 14D, and 17D is 16 × 16 is representative. This is the motion information position. Next, the reference motion information 2558 at the generated representative motion information position is set as the representative motion information (step S1605), and the representative motion information is stored in the temporal direction reference motion information memory (step S1606). The above steps S1604 to S1605 are executed for all motion information compressed blocks.

  If the unit in which the motion information 2559 is held is MxM blocks and the size of the motion information compression block is NxN (N is a multiple of M), the above-mentioned motion information compression processing is executed to increase the capacity of the reference motion information memory by (MxM ) / (NxN).

<Another Embodiment of Representative Motion Information Position>
As another example of generating the representative motion information position, the center position of the plurality of reference motion information acquisition positions may be used as the representative motion information position. 18A and 18B show representative motion information positions for each motion compression block having a size of 16 × 16. FIG. 18A is a representative motion information position when the reference motion information acquisition position is the position shown in FIG. 13D. Similarly, FIG. 18B is a representative motion information position when the reference motion information acquisition position is the position shown in FIG. 17D. Each position is shown. Circles in FIG. 18A and FIG. 18B indicate reference motion information acquisition positions when the prediction unit is 16 × 16, and representative motions indicated by crosses at the center positions of the four reference motion information acquisition positions. The information position is arranged.

  As still another example of generating the representative motion information position, reference motion information acquisition positions for each size of a plurality of prediction units are provided as reference position information 2560, and the representative motion information position is generated from the plurality of reference motion information acquisition positions. It doesn't matter. FIG. 19 shows the center (reference motion information acquisition position) of the prediction unit in each size of 16 × 16 or more when the tree block is a 64 × 64 pixel block.

  As another example of generating the representative motion information position, the representative motion information position may be set using the reference motion information acquisition position arranged for each motion information compression block. FIG. 20A shows an example in which the center of gravity of a plurality of reference motion information acquisition positions for each motion information compression block is set as the representative motion information position. When the position of the center of gravity does not match the position of the 4x4 block, the nearest 4x4 block may be used as the representative motion information position, or the reference motion vector 166 of the position of the center of gravity may be determined by using an interpolation method such as bilinear interpolation. You can generate it.

  Also, FIG. 20B shows an example in which one of the plurality of reference motion information acquisition positions is selected for each motion information compressed block and set as the representative motion information position.

  Further, FIGS. 21A and 21B further show an example in which the reference motion information acquisition positions are made the same in each motion information compression block in the tree block. Since the representative motion information position is the same in all motion information compression blocks, it is not necessary to switch the representative motion information position according to the position in the tree block. Further, the representative motion information position may be at any position such as the upper left end or the upper right end in the motion information compression block in addition to FIGS. 21A and 21B.

  As an example of generating the representative motion information position, BlkIdx indicating the 4x4 block position in the motion information compression block in Z scan order may be used to indicate the representative motion information position. When the size of the motion information compression block is 16 × 16, the representative motion information position shown in FIG. 21A corresponds to the position of BlkIdx = 12. The representative motion information position shown in FIG. 21B corresponds to the position of BlkIdx = 15.

  As another example of the motion information compression process, the reference frame number may be included in the motion information compression process in order to reduce the memory capacity related to the reference frame number. In this case, the reference frame number held at the representative motion information position is stored in the memory capacity related to the reference frame number. Therefore, the spatial direction reference motion information memory 501 and the spatial direction reference motion information memory 502 shown in FIG. 5 store the reference frame number in addition to the motion vector information.

  As still another example of the motion information compression processing, when the reference frame number is not included in the motion information compression processing, the motion vector information in the motion information at the representative motion information position is subjected to scaling processing using the reference frame number. Then, it may be stored in the motion information memory 110. A typical example of the scaling process is a linear scaling process with reference frame number zero as a reference. This is to perform linear scaling processing so that when the reference frame number is a value other than zero, the motion vector information refers to the reference frame corresponding to the reference frame number zero. The reference of the reference frame number may be a value other than zero as the standard for the above-described scaling processing. When division occurs when performing the above-described linear scaling processing, the division processing may be made into a table in advance and the table may be drawn each time to realize the division.

  If the size of the motion information compression block is other than 16 × 16 blocks, the representative motion information position is generated using the same processing as described above. In one example, when the size of the motion information compression block is 64x64, the reference motion information acquisition position when the size of the prediction unit is 64x64 is set as the representative motion information position. In yet another example, the representative motion information position in which the size of the motion information compression block shown in FIG. 21A, FIG. 21B, etc. is 16 × 16 blocks is scaled in the horizontal direction and the vertical direction according to the size of the motion information compression block. It may be an information position.

  If the reference motion information does not exist because the representative motion information position is outside the picture or slice, the position at which the reference motion information can be acquired in the motion information compression block, such as the upper left end of the motion information compression block, is set as the new representative motion. The information position may be replaced. Also, when the representative motion information position is an area to which intra prediction is applied and reference motion information does not exist, the same process may be executed to replace it with a new representative motion information position.

Hereinafter, modifications of each embodiment will be listed and introduced.
In the first and second embodiments, an example in which a frame is divided into rectangular blocks of 16 × 16 pixel size and the encoding / decoding is performed in order from the block at the upper left of the screen to the lower right ( See FIG. 2A). However, the encoding order and the decoding order are not limited to this example. For example, the encoding and decoding may be performed in order from the lower right to the upper left, or the encoding and decoding may be performed so as to draw a spiral from the center of the screen to the end of the screen. Further, the encoding and decoding may be performed in order from the upper right to the lower left, or the encoding and decoding may be performed so as to draw a spiral from the screen edge toward the screen center.

  In the first and second embodiments, the prediction target block size such as the 4 × 4 pixel block, the 8 × 8 pixel block, and the 16 × 16 pixel block has been described as an example, but the prediction target block is a uniform block. It does not have to be a shape. For example, the prediction target block (prediction unit) size may be a 16 × 8 pixel block, an 8 × 16 pixel block, an 8 × 4 pixel block, a 4 × 8 pixel block, or the like. Further, it is not necessary to unify all block sizes in one coding tree unit, and a plurality of different block sizes may be mixed. When a plurality of different block sizes are mixed in one coding tree unit, the code amount for encoding or decoding the division information increases as the number of divisions increases. Therefore, it is desirable to select the block size in consideration of the balance between the code amount of the division information and the locally decoded image or the quality of the decoded image.

  In the first and second embodiments, for simplification, the luminance signal and the color difference signal are not distinguished from each other, and the comprehensive description is given regarding the color signal component. However, if the prediction process differs between the luminance signal and the color difference signal, the same or different prediction method may be used. If different prediction methods are used between the luminance signal and the chrominance signal, the prediction method selected for the chrominance signal can be encoded or decoded in the same manner as the luminance signal.

  In the first and second embodiments, for simplification, the luminance signal and the color difference signal are not distinguished from each other, and the comprehensive description is given regarding the color signal component. However, if the orthogonal transform processing differs between the luminance signal and the color difference signal, the same or different orthogonal transform method may be used. If different orthogonal transform methods are used between the luminance signal and the color difference signal, the orthogonal transform method selected for the color difference signal can be encoded or decoded in the same manner as the luminance signal.

  In the first and second embodiments, syntax elements not specified in the embodiment may be inserted between the rows of the table shown in the syntax configuration, and a description regarding other conditional branching is included. It doesn't matter. Alternatively, the syntax table can be divided and integrated into a plurality of tables. Further, it is not always necessary to use the same term, and the term may be arbitrarily changed depending on the form used.

  As described above, each of the embodiments can realize highly efficient orthogonal transform and inverse orthogonal transform while alleviating the difficulty in hardware implementation and software implementation. Therefore, according to each embodiment, the coding efficiency is improved and the subjective image quality is also improved.

Further, the instructions shown in the processing procedure shown in the above-described embodiment can be executed based on a program that is software. A general-purpose computer system may store this program in advance, and by reading this program, it is possible to obtain the same effects as the effects of the moving picture coding apparatus and the moving picture decoding apparatus of the above-described embodiments. is there. The instructions described in the above embodiments are magnetic disks (flexible disks, hard disks, etc.), optical disks (CD-ROM, CD-R, CD-RW, DVD-ROM, DVD) as programs that can be executed by a computer. ± R, DVD ± RW, etc.), a semiconductor memory, or a recording medium similar thereto. The storage format may be any form as long as it is a recording medium readable by a computer or an embedded system. If the computer reads the program from this recording medium and causes the CPU to execute the instructions described in the program based on this program, the computer will be similar to the moving picture coding apparatus and the moving picture decoding apparatus of the above-described embodiment. The operation can be realized. Of course, when the computer acquires or loads the program, it may be acquired or loaded via a network.
In addition, an OS (operating system) running on a computer, database management software, MW (middleware) such as a network, etc., which realizes the present embodiment, based on instructions of a program installed in a computer or an embedded system from a recording medium. You may perform a part of each process for doing.
Further, the recording medium in the embodiments of the present invention is not limited to a medium independent of a computer or an embedded system, but includes a recording medium in which a program transmitted via a LAN, the Internet or the like is downloaded and stored or temporarily stored. Further, the program that implements the processing of each of the above embodiments may be stored on a computer (server) connected to a network such as the Internet and may be downloaded by a computer (client) via the network.
Further, the number of recording media is not limited to one, and even when the processing in this embodiment is executed from a plurality of media, it is included in the recording media in the embodiments of the present invention, and the structure of the medium may be any one. Good.

The computer or the embedded system in the embodiment of the present invention is for executing each processing in the present embodiment based on the program stored in the recording medium, and is an apparatus including one such as a personal computer and a microcomputer. It may have any configuration such as a system in which a plurality of devices are network-connected.
Further, the computer in the embodiment of the present invention is not limited to a personal computer, and includes an arithmetic processing unit, a microcomputer, etc. included in an information processing device, and a device capable of realizing the function in the embodiment of the present invention by a program, It is a generic term for devices.

  Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the scope equivalent thereto.

100 ... Image coding device, 101 ... Subtraction unit, 102 ... Orthogonal transformation unit, 103 ... Quantization unit, 104, 2502 ... Inverse quantization unit, 105, 2503 ... Inverse orthogonal transformation unit, 106, 2504, 2706 ... Addition unit , 107, 2505 ... Reference image memory, 108, 2506 ... Inter prediction unit, 109 ... Motion information compression unit, 110 ... Motion information memory, 112 ... Entropy coding unit, 113 ... Output buffer, 114 ... Coding control unit, 401 ... Parameter coding unit, 402 ... Transform coefficient coding unit, 403 ... Motion information coding unit, 404 ... Multiplexing unit, 901 ... Reference motion vector acquisition unit, 902 ... Prediction motion vector selection switch, 903 ... Subtraction unit, 904 ... Differential motion information coding unit, 905 ... Predictive motion information position coding unit, 906 ... Multiplexing unit, 2500 ... Moving image decoding device, 250 Entropy decoding unit, 2507 ... Reference motion information memory, 2508 ... Reference motion information compression unit, 2510 ... Decoding control unit, 2601, 2701 ... Separation unit, 2602 ... Parameter decoding unit, 2603 ... Transform coefficient decoding unit, 2604 ... Motion information decoding unit, 2702 ... Differential motion information decoding unit, 2503 ... Prediction motion information position decoding unit, 2704 ... Reference motion information acquisition unit, 2705 ... Prediction motion information selection switch.

Claims (2)

In a moving image coding method of dividing an input image signal into pixel blocks and performing inter prediction on these divided pixel blocks,
From the motion information in the coded region, based on the information obtained according to the information indicating the method of selecting the motion vector predictor, selecting the motion vector predictor,
Predicting motion information of a current block using the predicted motion information;
Representative motion information is acquired from a plurality of pieces of motion information in the region where encoding has been completed, in accordance with first information indicating a method of selecting the predicted motion information.
Equipped with
The first information is
A moving picture coding method including second information for specifying a position within the coding target block for selecting the motion information. In the moving image decoding method of dividing the input image signal into pixel blocks and performing inter prediction on these divided pixel blocks,
From the motion information in the decoded area, based on the information obtained according to the information indicating the method of selecting the motion vector predictor, select the motion vector predictor,
Predicting motion information of a block to be decoded using the predicted motion information,
Representative motion information is obtained from a plurality of pieces of motion information in the region where decoding has been completed, according to first information indicating a method of selecting the predicted motion information.
Equipped with
The first information is
A moving picture decoding method, comprising: second information for specifying a position within the decoding target block for selecting the motion information.