JP6370977B2 - Image encoding method and image decoding method - Google Patents

Description

  The present invention relates to an encoding and decoding method for moving images and still images.

  In recent years, video coding methods that have greatly improved coding efficiency have been jointly developed by ITU-T and ISO / IEC in accordance with ITU-T Rec. H.264 and ISO / IEC 14496-10 (hereinafter referred to as H.264). 264)). In H.264, prediction processing, conversion processing, and entropy encoding processing are performed in units of rectangular blocks (for example, 16 × 16 pixel block units, 8 × 8 pixel block units, etc.). In the prediction process, motion compensation is performed on a rectangular block to be encoded (encoding target block) with reference to an already encoded frame (reference frame) to perform prediction in the time direction. In such motion compensation, it is necessary to encode motion information including a motion vector as spatial shift information between an encoding target block and a block referred to in a reference frame and send the encoded motion information to the decoding side. Furthermore, when motion compensation is performed using a plurality of reference frames, it is necessary to encode a reference frame number together with motion information. For this reason, the amount of codes related to motion information and reference frame numbers may increase.

  As an example of a method for obtaining a motion vector in motion compensated prediction, a motion vector to be assigned to an encoding target block is derived from a motion vector assigned to an already encoded block, and prediction is performed based on the derived motion vector. There is a direct mode for generating an image (see Patent Document 1 and Patent Document 2). In the direct mode, since the motion vector is not encoded, the code amount of the motion information can be reduced. The direct mode is adopted in H.264 / AVC, for example.

Patent No. 4020789 US Pat. No. 7,233,621

  In the direct mode, the motion vector of the encoding target block is predicted and generated by a fixed method of calculating the motion vector from the median value of the motion vector of the encoded block adjacent to the encoding target block. For this reason, the freedom degree of motion vector calculation is low.

  In order to increase the degree of freedom of motion vector calculation, a method has been proposed in which one of a plurality of encoded blocks is selected and a motion vector is assigned to the encoding target block. In this method, selection information for identifying the selected block must always be transmitted so that the decoding side can identify the selected encoded block. Accordingly, when one of a plurality of encoded blocks is selected and a motion vector to be assigned to the encoding target block is determined, there is a problem that the amount of code related to the selection information is increased.

  The present invention has been made to solve the above problems, and an object thereof is to provide an image encoding and image decoding method with high encoding efficiency.

  An image encoding method according to an embodiment of the present invention includes a first step of selecting at least one motion reference block from among encoded pixel blocks having motion information, and motion to be applied to an encoding target block. A second step of selecting, from among the motion reference blocks, at least one usable block having pixel information and having different motion information, and selecting one of the available blocks A third step of selecting a block, a fourth step of generating a predicted image of the target block using the motion information of the selected block, and encoding a prediction error between the predicted image and the original image 5th step to be converted, and a selection that identifies the selected block with reference to a code table that is predetermined according to the number of available blocks Comprising a sixth step of encoding the broadcast, the.

  An image decoding method according to another embodiment of the present invention is applied to a decoding target block and a first step of selecting at least one motion reference block from among decoded pixel blocks having motion information. A second step of selecting at least one usable block having motion information candidates and having different motion information from among the motion reference blocks, and in advance according to the number of usable blocks A third step of obtaining selection information for specifying the selected block by decoding the input encoded data with reference to a predetermined code table, and according to the selection information, 4th step of selecting one selected block from the above, and using the motion information of the selected block, A fifth step of generating a measured image; a sixth step of decoding a prediction residual of the decoding target block from the encoded data; a seventh step of obtaining a decoded image from the prediction image and the prediction residual; Are provided.

  According to the present invention, encoding efficiency can be improved.

It is a block diagram which shows roughly the structure of the image coding apparatus which concerns on 1st Embodiment. It is a figure which shows an example of the size of the macroblock which is a process unit of the encoding of the image decoding part shown in FIG. It is a figure which shows the other example of the size of the macroblock which is a process unit of the encoding of the image decoding part shown in FIG. It is a figure which shows the order in which the image encoding part shown in FIG. 1 encodes the pixel block in an encoding object frame. It is a figure which shows an example of the motion information frame which the motion information memory shown in FIG. 1 hold | maintains. It is a flowchart which shows an example of the procedure which processes the input image signal of FIG. It is a figure which shows an example of the inter prediction process which the motion compensation part of FIG. 1 performs. It is a figure which shows the other example of the inter prediction process which the motion compensation part of FIG. 1 performs. It is a figure which shows an example of the size of the motion compensation block used for the inter prediction process. It is a figure which shows the other example of the size of the motion compensation block used for the inter prediction process. It is a figure which shows the further another example of the size of the motion compensation block used for the inter prediction process. It is a figure which shows the other example of the size of the motion compensation block used for the inter prediction process. It is a figure which shows an example of arrangement | positioning of a space direction and a time direction motion reference block. It is a figure which shows the other example of arrangement | positioning of a spatial direction motion reference block. It is a figure which shows the relative position of the spatial direction motion reference block with respect to the encoding object block shown to FIG. 8B. It is a figure which shows the other example of arrangement | positioning of a time direction motion reference block. It is a figure which shows the further another example of arrangement | positioning of a time direction motion reference block. It is a figure which shows the further another example of arrangement | positioning of a time direction motion reference block. It is a flowchart which shows an example of the method in which the available block acquisition part of FIG. 1 selects an available block from a motion reference block. FIG. 10 is a diagram illustrating an example of an available block selected from the motion reference blocks illustrated in FIG. 8 according to the method of FIG. 9. It is a figure which shows an example of the available block information which the available block acquisition part of FIG. 1 outputs. It is a figure which shows an example of the identity determination of the motion information between blocks by the available block acquisition part of FIG. It is a figure which shows the other example of the identity determination of the motion information between the blocks by the available block acquisition part of FIG. It is a figure which shows the further another example of the identity determination of the motion information between blocks by the available block acquisition part of FIG. It is a figure which shows the other example of the identity determination of the motion information between the blocks by the available block acquisition part of FIG. It is a figure which shows the further another example of the identity determination of the motion information between blocks by the available block acquisition part of FIG. It is a figure which shows the other example of the identity determination of the motion information between the blocks by the available block acquisition part of FIG. It is a block diagram which shows schematically the structure of the estimation part of FIG. It is a figure which shows the group of the motion information which the time direction motion information acquisition part of FIG. 13 outputs. It is explanatory drawing explaining the interpolation process of the minority pixel precision which can be utilized in the motion compensation process by the motion compensation part of FIG. It is a flowchart which shows an example of operation | movement of the estimation part of FIG. It is a figure which shows a mode that the motion compensation part of FIG. 13 copies the motion information of a time direction motion reference block to an encoding object block. FIG. 2 is a block diagram schematically showing a configuration of a variable length coding unit in FIG. 1. It is a figure which shows the example which produces | generates syntax according to usable block information. It is a figure which shows the example of the binarization of the selection block information syntax corresponding to usable block information. It is explanatory drawing explaining scaling of motion information. It is a figure which shows the syntax structure according to embodiment. It is a figure which shows an example of the macroblock layer synchronization according to 1st Embodiment. It is a figure which shows the other example of the macroblock layer synchronization according to 1st Embodiment. H. 2 is a diagram illustrating a code table corresponding to mb_type and mb_type at the time of B slice in H.264. It is a figure which shows an example of the code table which concerns on embodiment. H. 2 is a diagram illustrating a code table corresponding to mb_type and mb_type at the time of P slice in H.264. It is a figure which shows the other example of the code table which concerns on embodiment. It is a figure which shows an example of the code table corresponding to mb_type and mb_type in B slice according to embodiment. It is a figure which shows the other example of the code table corresponding to mb_type and mb_type in P slice according to embodiment. It is a block diagram which shows roughly the structure of the image coding apparatus which concerns on 2nd Embodiment. FIG. 27 is a block diagram schematically showing a configuration of a prediction unit in FIG. 26. It is a block diagram which shows roughly the structure of the 2nd estimation part of FIG. FIG. 27 is a block diagram schematically showing a configuration of a variable length coding unit in FIG. 26. It is a figure which shows an example of the macroblock layer syntax according to 2nd Embodiment. It is a figure which shows the other example of the macroblock layer syntax according to 2nd Embodiment. It is a block diagram which shows roughly the image decoding apparatus which concerns on 3rd Embodiment. FIG. 32 is a block diagram showing in more detail the encoded sequence decoding unit shown in FIG. 31. FIG. 32 is a block diagram showing the prediction unit shown in FIG. 31 in more detail. It is a block diagram which shows roughly the image decoding apparatus which concerns on 4th Embodiment. FIG. 34 is a block diagram illustrating in more detail the encoded string decoding unit illustrated in FIG. 33. It is a block diagram which shows the prediction part shown in FIG. 33 in detail.

  Hereinafter, an image encoding and image decoding method and apparatus according to an embodiment of the present invention will be described with reference to the drawings as necessary. Note that, in the following embodiments, the same numbered portions are assumed to perform the same operation, and repeated description is omitted.

(First embodiment)
FIG. 1 schematically shows the configuration of an image coding apparatus according to the first embodiment of the present invention. As shown in FIG. 1, the image encoding device includes an image encoding unit 100, an encoding control unit 150, and an output buffer 120. This image encoding apparatus may be realized by hardware such as an LSI chip, or may be realized by causing a computer to execute an image encoding program.

  An original image (input image signal) 10 that is a moving image or a still image is input to the image encoding unit 100, for example, in units of pixel blocks obtained by dividing the original image. As will be described in detail later, the image encoding unit 100 compresses and encodes the input image signal 10 to generate encoded data 14. The generated encoded data 14 is temporarily stored in the output buffer 120 and sent to a storage system (storage medium) or a transmission system (communication line) (not shown) at an output timing managed by the encoding control unit 150. .

  The encoding control unit 150 controls overall encoding processing of the image encoding unit 100 such as feedback control of generated code amount, quantization control, prediction mode control, and entropy encoding control. Specifically, the encoding control unit 150 provides the encoding control information 50 to the image encoding unit 100 and receives the feedback information 51 from the image encoding unit 100 as appropriate. The encoding control information 50 includes prediction information, motion information 18 and quantization parameter information. The prediction information includes prediction mode information and block size information. The motion information 18 includes a motion vector, a reference frame number, and a prediction direction (unidirectional prediction, bidirectional prediction). The quantization parameter information includes a quantization parameter such as a quantization width (quantization step size) and a quantization matrix. The feedback information 51 includes the amount of code generated by the image encoding unit 100, and is used, for example, to determine a quantization parameter.

  The image encoding unit 100 encodes the input image signal 10 in units of pixel blocks obtained by dividing the original image (for example, macroblocks, sub-blocks, 1 pixel, etc.). For this reason, the input image signal 10 is sequentially input to the image encoding unit 100 in units of pixel blocks obtained by dividing the original image. In this embodiment, the encoding processing unit is a macro block, and a pixel block (macro block) that is an encoding target and corresponds to the input image signal 10 is simply referred to as an encoding target block. Also, an image frame including an encoding target block, that is, an encoding target image frame is referred to as an encoding target frame.

  Such an encoding target block may be, for example, a 16 × 16 pixel block as shown in FIG. 2A or a 64 × 64 pixel block as shown in FIG. 2B. The encoding target block may be a 32 × 32 pixel block, an 8 × 8 pixel block, or the like. The shape of the macroblock is not limited to the square shape as shown in FIGS. 2A and 2B, and may be set to an arbitrary shape such as a rectangular shape. Furthermore, the processing unit is not limited to a pixel block such as a macroblock, and may be a frame or a field.

  The encoding process for each pixel block in the encoding target frame may be executed in any order. In the present embodiment, in order to simplify the description, as shown in FIG. 3, the pixel blocks are arranged row by row from the upper left pixel block of the encoding target frame to the lower right pixel block, that is, in raster scan order. It is assumed that the encoding process is performed on

  1 includes a prediction unit 101, a subtracter 102, a transform / quantization unit 103, a variable length coding unit 104, an inverse quantization / inverse transform unit 105, an adder 106, and a frame memory 107. A motion information memory 108 and an available block acquisition unit 109.

  In the image encoding unit 100, the input image signal 10 is input to the prediction unit 101 and the subtracter 102. The subtracter 102 receives the input image signal 10 and also receives the predicted image signal 11 from the prediction unit 101 described later. The subtractor 102 calculates a difference between the input image signal 10 and the predicted image signal 11 and generates a prediction error image signal 12.

  The transform / quantization unit 103 receives the prediction error image signal 12 from the subtractor 102, performs a conversion process on the received prediction error image signal 12, and generates a transform coefficient. The transform process is, for example, an orthogonal transform such as a discrete cosine transform (DCT). In another embodiment, the transform / quantization unit 103 may generate transform coefficients using a technique such as wavelet transform and independent component analysis instead of the discrete cosine transform. Further, the transform / quantization unit 103 quantizes the generated transform coefficient based on the quantization parameter given by the encoding control unit 150. The quantized transform coefficient (transform coefficient information) 13 is output to the variable length coding unit 104 and the inverse quantization / inverse transform unit 105.

  The inverse quantization / inverse transform unit 105 inversely quantizes the quantized transform coefficient 13 according to the quantization parameter given by the encoding control unit 150, that is, the same quantization parameter as the transform / quantization unit 103. . Subsequently, the inverse quantization / inverse transform unit 105 performs inverse transform on the inversely quantized transform coefficient to generate a decoded prediction error signal 15. The inverse transformation process by the inverse quantization / inverse transformation unit 105 matches the inverse transformation process of the transformation process by the transformation / quantization unit 103. For example, the inverse transform process is an inverse discrete cosine transform (IDCT) or an inverse wavelet transform.

  The adder 106 receives the decoded prediction error signal 15 from the inverse quantization / inverse transform unit 105, and further receives the predicted image signal 11 from the prediction unit 101. The adder 106 adds the decoded prediction error signal 15 and the predicted image signal 11 to generate a local decoded image signal 16. The generated local decoded image signal 16 is stored as a reference image signal 17 in the frame memory 107. The reference image signal 17 stored in the frame memory 107 is read and referred to by the prediction unit 101 when a subsequent encoding target block is encoded.

  The prediction unit 101 receives the reference image signal 17 from the frame memory 107 and also receives available block information 30 from an available block acquisition unit 109 described later. Further, the prediction unit 101 receives reference motion information 19 from a motion information memory 108 described later. The prediction unit 101 generates the prediction image signal 11, the motion information 18, and the selected block information 31 of the encoding target block based on the reference image signal 17, the reference motion information 19, and the available block information 30. Specifically, the prediction unit 101 generates motion information 18 and selected block information 31 based on the available block information 30 and the reference motion information 19, and based on the motion information 18. A motion compensation unit 113 that generates the predicted image signal 11 is provided. The predicted image signal 11 is sent to the subtracter 102 and the adder 106. The motion information 18 is stored in the motion information memory 108 for subsequent prediction processing on the current block. The selected block information 31 is sent to the variable length coding unit 104. The prediction unit 101 will be described in detail later.

  In the motion information memory 108, motion information 18 is temporarily stored as reference motion information 19. FIG. 4 shows an example of the configuration of the motion information memory 108. As shown in FIG. 4, the motion information memory 108 stores reference motion information 19 in units of frames, and the reference motion information 19 forms a motion information frame 25. The motion information memory 108 sequentially receives motion information 18 relating to encoded blocks, and as a result, the motion information memory 108 holds a plurality of motion information frames 25 having different encoding times.

  The reference motion information 19 is held in the motion information frame 25 in predetermined block units (for example, 4 × 4 pixel block units). The motion vector block 28 illustrated in FIG. 4 indicates a pixel block having the same size as the encoding target block, the usable block, the selection block, and the like, and is, for example, a 16 × 16 pixel block. For example, a motion vector is assigned to the motion vector block 28 for each 4 × 4 pixel block. Inter prediction processing using motion vector blocks is referred to as motion vector block prediction processing. The reference motion information 19 held in the motion information memory 108 is read by the prediction unit 101 when the motion information 18 is generated. The motion information 18 included in the available block as described later refers to the reference motion information 19 held in the area where the available block in the motion information memory 108 is located.

  Note that the motion information memory 108 is not limited to the example in which the reference motion information 19 is stored in units of 4 × 4 pixel blocks, and the motion information memory 108 may store the reference motion information 19 in units of other pixel blocks. For example, the pixel block unit related to the reference motion information 19 may be one pixel or a 2 × 2 pixel block. In addition, the shape of the pixel block related to the reference motion information 19 is not limited to a square shape, and may be an arbitrary shape.

  The available block acquisition unit 109 in FIG. 1 acquires the reference motion information 19 from the motion information memory 108, and based on the acquired reference motion information 19, from the plurality of blocks that have already been encoded, the prediction unit 101 An available block that can be used for the prediction process is selected. The selected available block is sent to the prediction unit 101 and the variable length coding unit 104 as available block information 30. A coded block that is a candidate for selecting an available block is referred to as a motion reference block. A method for selecting a motion reference block and an available block will be described in detail later.

  In addition to the transform coefficient information 13, the variable length coding unit 104 uses the selected block information 31 from the prediction unit 101, and the coding parameters such as the prediction information and the quantization parameter from the coding control unit 150. The available block information 30 is received from 109. The variable length coding unit 104 performs entropy coding (for example, isometric coding, Huffman coding, arithmetic coding, etc.) on the quantized transform coefficient 13, selected block information 31, usable block information 30, and coding parameters. ) To generate encoded data 14. The coding parameters include all parameters necessary for decoding such as information on transform coefficients and information on quantization, as well as selected block information 31 and prediction information. The generated encoded data 14 is temporarily stored in the output buffer 120 and sent to a storage system or transmission system (not shown).

  FIG. 5 shows the processing procedure of the input image signal 10. As shown in FIG. 5, first, the prediction image signal 11 is generated by the prediction unit 101 (step S501). In the generation of the predicted image signal 11 in step S501, one of the available blocks described later is selected as a selected block, and the predicted image is selected using the selected block information 31, the motion information of the selected block, and the reference image signal 17. A signal 11 is created. The difference between the predicted image signal 11 and the input image signal 10 is calculated by the subtracter 102, and the predicted error image signal 12 is generated (step S502).

  Subsequently, orthogonal transform and quantization are performed on the prediction error image signal 12 by the transform / quantization unit 103, and transform coefficient information 13 is generated (step S503). The transform coefficient information 13 and the selected block information 31 are sent to the variable length encoding unit 104, subjected to variable length encoding, and encoded data 14 is generated (step S504). In step S504, the code table is switched so that the code table has the same number of entries as the number of available blocks according to the selected block information 31, and the selected block information 31 is variable-length encoded. The encoded data bit stream 20 is sent to a storage system (not shown) or a transmission path.

  The transform coefficient information 13 generated in step S503 is inversely quantized by the inverse quantization / inverse transform unit 105, subjected to inverse transform processing, and becomes the decoded prediction error signal 15 (step S505). The decoded prediction error signal 15 is added to the reference image signal 17 used in step S501 to become a local decoded image signal 16 (step S506), and stored as a reference image signal in the frame memory 107 (step S507).

Next, each configuration of the above-described image encoding unit 100 will be described in more detail.
A plurality of prediction modes are prepared in the image encoding unit 100 of FIG. 1, and each prediction mode differs in the generation method of the predicted image signal 11 and the motion compensation block size. Specifically, the prediction unit 101 generates the predicted image signal 11 broadly, and can be broadly divided into intra prediction (intraframe) that generates a predicted image using the reference image signal 17 related to the encoding target frame (or field). Prediction) and inter prediction (interframe prediction) for generating a predicted image using the reference image signal 17 related to one or more encoded reference frames (reference fields). The prediction unit 101 selectively switches between intra prediction and inter prediction, and generates the predicted image signal 11 of the encoding target block.

  FIG. 6A shows an example of inter prediction by the motion compensation unit 113. In inter prediction, as shown in FIG. 6A, from a block (also referred to as a prediction block) 23 in a reference frame one frame before encoding that is the same position as the current block to be encoded. The predicted image signal 11 is generated using the reference image signal 17 related to the block 24 at the position spatially shifted according to the motion vector 18 a included in the motion information 18. That is, in the generation of the predicted image signal 11, the reference image signal 17 related to the block 24 in the reference frame specified by the position (coordinates) of the encoding target block and the motion vector 18a included in the motion information 18 is used. . In inter prediction, motion compensation with a small pixel accuracy (for example, a 1/2 pixel accuracy or a 1/4 pixel accuracy) is possible, and a value of an interpolation pixel is generated by performing a filtering process on the reference image signal 17. Is done. For example, H.M. In H.264, it is possible to perform interpolation processing up to 1/4 pixel accuracy on the luminance signal. When motion compensation with ¼ pixel accuracy is performed, the information amount of the motion information 18 is four times the integer pixel accuracy.

  In inter prediction, not only an example in which a reference frame one frame before as shown in FIG. 6A is used, but any encoded reference frame may be used as shown in FIG. 6B. . When reference image signals 17 relating to a plurality of reference frames having different time positions are held, information indicating which time position the reference image signal 17 is generated from the reference image signal 17 is represented by a reference frame number. The reference frame number is included in the motion information 18. The reference frame number can be changed on an area basis (picture, block basis, etc.). That is, a different reference frame can be used for each pixel block. As an example, when the encoded reference frame one frame before is used for prediction, the reference frame number of this region is set to 0, and when the encoded reference frame two frames before is used for prediction, The reference frame number of this area is set to 1. As another example, when the reference image signal 17 for only one frame is held in the frame memory 107 (the number of reference frames is 1), the reference frame number is always set to 0.

  Further, in inter prediction, a block size suitable for a coding target block can be selected from a plurality of motion compensation blocks. That is, the encoding target block may be divided into a plurality of small pixel blocks, and motion compensation may be performed for each small pixel block. 7A to 7C show the size of the motion compensation block in units of macroblocks, and FIG. 7D shows the size of the motion compensation block in units of sub-blocks (pixel blocks of 8 × 8 pixels or less). As shown in FIG. 7A, when the encoding target block is 64 × 64 pixels, the motion compensation block is a 64 × 64 pixel block, a 64 × 32 pixel block, a 32 × 64 pixel block, a 32 × 32 pixel block, or the like. Can be selected. As shown in FIG. 7B, when the encoding target block is 32 × 32 pixels, the motion compensation block is a 32 × 32 pixel block, a 32 × 16 pixel block, a 16 × 32 pixel block, or a 16 × 16 pixel. A block or the like can be selected. Further, as shown in FIG. 7C, when the encoding target block is 16 × 16 pixels, the motion compensation block is a 16 × 16 pixel block, a 16 × 8 pixel block, an 8 × 16 pixel block, or an 8 × 8 pixel. It can be set to a block or the like. Furthermore, as shown in FIG. 7D, when the encoding target block is 8 × 8 pixels, the motion compensation block is an 8 × 8 pixel block, an 8 × 4 pixel block, a 4 × 8 pixel block, or 4 × 4. A pixel block or the like can be selected.

  As described above, since the small pixel block (for example, 4 × 4 pixel block) in the reference frame used for the inter prediction has the motion information 18, the optimal value is determined according to the local property of the input image signal 10. The shape and motion vector of the motion compensation block can be used. Also, the macroblocks and sub-macroblocks in FIGS. 7A to 7D can be arbitrarily combined. When the encoding target block is a 64 × 64 pixel block as shown in FIG. 7A, each block size shown in FIG. 7B is set for each of the four 32 × 32 pixel blocks obtained by dividing the 64 × 64 pixel block. By selecting, a block of 64 × 64 to 16 × 16 pixels can be used hierarchically. Similarly, when the block size shown in FIG. 7D can be selected, a block size of 64 × 64 to 4 × 4 can be used hierarchically.

Next, the motion reference block will be described with reference to FIGS. 8A to 8F.
The motion reference block is selected from the encoding target frame and the encoded region (block) in the reference frame according to a method determined by both the image encoding device of FIG. 1 and an image decoding device described later. The FIG. 8A shows an example of the arrangement of motion reference blocks selected according to the position of the encoding target block. In the example of FIG. 8A, nine motion reference blocks A to D and TA to TE are selected from the encoding target frame and the encoded region in the reference frame. Specifically, four blocks A, B, C, and D adjacent to the left, upper, upper right, and upper left of the encoding target block are selected as motion reference blocks from the encoding target frame, and from the reference frame, A block TA at the same position as the encoding target block and four pixel blocks TB, TC, TD, TE adjacent to the right, bottom, left and top of the block TA are selected as motion reference blocks. In the present embodiment, the motion reference block selected from the encoding target frame is referred to as a spatial direction motion reference block, and the motion reference block selected from the reference frame is referred to as a temporal direction motion reference block. A symbol p assigned to each motion reference block in FIG. 8A indicates an index of the motion reference block. The indexes are numbered in the order of motion reference blocks in the temporal direction and the spatial direction. However, the order is not limited to this, and the order is not necessarily limited as long as the indexes do not overlap. For example, the motion reference blocks in the temporal direction and the spatial direction may be numbered out of order.

  Note that the spatial direction motion reference block is not limited to the example illustrated in FIG. 8A, and as illustrated in FIG. Macroblock etc.). In this case, the relative position (dx, dy) from the upper left pixel e in the encoding target block to each of the pixels a, b, c, d is set as shown in FIG. 8C. Here, in the example shown in FIGS. 8A and 8B, the macroblock is shown as being an N × N pixel block.

  Further, as shown in FIG. 8D, all the blocks A1 to A4, B1, B2, C, and D adjacent to the encoding target block may be selected as the spatial direction motion reference block. In the example of FIG. 8D, the number of spatial direction motion reference blocks is 8.

  Further, as shown in FIG. 8E, the temporal motion reference block may have a part of each of the blocks TA to TE overlapping, and the blocks TA to TE are arranged apart from each other as shown in FIG. 8F. It does not matter. In FIG. 8E, a portion where the temporal direction motion reference blocks TA and TB overlap is indicated by hatching. Furthermore, the temporal direction motion reference block is not limited to the example of the block at the position corresponding to the encoding target block (Collocate position) and the block positioned around it, and is arranged at any position in the reference frame. It may be a block. For example, the block in the reference frame specified by the position of the reference block and the motion information 18 included in any of the encoded blocks adjacent to the encoding target block is set as the central block (for example, the block TA). The central block and its surrounding blocks may be selected as the temporal direction motion reference block. Furthermore, the time direction reference blocks may not be arranged at equal intervals from the central block.

  In any of the above cases, if the number and position of the spatial and temporal motion reference blocks are determined in advance in the encoding device and the decoding device, how are the number and position of the motion reference blocks set? It doesn't matter. In addition, the size of the motion reference block is not necessarily the same size as the encoding target block. For example, as illustrated in FIG. 8D, the size of the motion reference block may be larger or smaller than the encoding target block. Furthermore, the motion reference block is not limited to a square shape, and may be set to an arbitrary shape such as a rectangular shape. In addition, the motion reference block may be set to any size.

  Further, the motion reference block and the available block may be arranged only in one of the temporal direction and the spatial direction. Further, the motion reference block and the usable block in the time direction may be arranged according to the type of slice such as the P slice and the B slice, and the motion reference block and the usable block in the spatial direction may be arranged.

  FIG. 9 shows a method in which the available block acquisition unit 109 selects an available block from the motion reference blocks. An available block is a block in which motion information can be applied to an encoding target block, and has different motion information. The available block acquisition unit 109 refers to the reference motion information 19 and determines whether or not each motion reference block is an available block according to the method shown in FIG. 9, and outputs the available block information 30.

  As shown in FIG. 9, first, a motion reference block whose index p is zero is selected (S800). In the description of FIG. 9, it is assumed that the motion reference blocks are processed in order from the index p of 0 to M−1 (M indicates the number of motion reference blocks). Further, the description will be made assuming that the availability determination process for the motion reference blocks with the index p of 0 to p−1 is completed and the index of the motion reference block that is the target of determining whether or not the index p is p. .

  The available block acquisition unit 109 determines whether or not the motion reference block p has the motion information 18, that is, whether or not at least one motion vector is assigned (S801). If the motion reference block p does not have a motion vector, that is, the time direction motion reference block p is a block in an I slice that does not have motion information, or in the time direction motion reference block p When all the small pixel blocks are intra prediction encoded, the process proceeds to step S805. In step S805, the motion reference block p is determined as an unusable block.

  If the motion reference block p has motion information in step S801, the process proceeds to step S802. The available block acquisition unit 109 selects a motion reference block q (available block q) that has already been selected as an available block. Here, q is a value smaller than p. Subsequently, the available block acquisition unit 109 compares the motion information 18 of the motion reference block p with the motion information 18 of the available block q to determine whether or not the same motion information is included (S803). . When it is determined that the motion information 18 of the motion reference block p and the motion information 18 of the motion reference block q selected as an available block are the same, the process proceeds to step S805, and the motion reference block p is not available. It is determined.

  If it is determined in step S803 that the motion information 18 of the motion reference block p and the motion information 18 of the available block q are not the same for all available blocks q satisfying q <p, the process proceeds to step S804. . In step S804, the available block acquisition unit 109 determines the motion reference block p as an available block.

  When it is determined that the motion reference block p is an available block or an unusable block, the available block acquisition unit 109 determines whether the availability determination has been performed for all the motion reference blocks ( S806). If there is a motion reference block for which the availability determination has not been performed, for example, if p <M−1, the process proceeds to step S807. Subsequently, the available block acquisition unit 109 increments the index p by 1 (step S807), and executes steps S801 to S806 again. When the availability determination is executed for all the motion reference blocks in step S806, the availability determination process ends.

  By executing the availability determination process described above, it is determined whether each motion reference block is an available block or an unavailable block. The available block acquisition unit 109 generates usable block information 30 including information on available blocks. Thus, by selecting an available block from the motion reference blocks, the amount of information related to the available block information 30 is reduced, and as a result, the amount of encoded data 14 can be reduced.

  FIG. 10 shows an example of the result of executing the availability determination process on the motion reference block shown in FIG. 8A. In FIG. 10, it is determined that two spatial direction motion reference blocks (p = 0, 1) and two temporal direction motion reference blocks (p = 5, 8) are usable blocks. An example of the available block information 30 relating to the example of FIG. 10 is shown in FIG. As shown in FIG. 11, the available block information 30 includes an index of a motion reference block, availability, and a motion reference block name. In the example of FIG. 11, indexes p = 0, 1, 5, and 8 are usable blocks, and the number of usable blocks is four. The prediction unit 101 selects one optimum usable block from these usable blocks as a selected block, and outputs information (selected block information) 31 regarding the selected block. The selected block information 31 includes the number of available blocks and the index value of the selected available block. For example, when the number of available blocks is 4, the corresponding selected block information 31 is encoded by the variable-length encoding unit 104 using a code table whose maximum entry is 4.

  In step S801 in FIG. 9, when at least one of the blocks in the temporal direction motion reference block p is an intra prediction encoded block, the available block acquisition unit 109 uses the motion reference block p. It may be determined as an impossible block. In other words, the process may proceed to step S802 only when all the blocks in the temporal direction motion reference block p are encoded by inter prediction.

  12A to 12E show an example in which it is determined that the motion information 18 of the motion reference block p and the motion information 18 of the available block q are the same in the comparison of the motion information 18 in step S803. 12A to 12E each show a plurality of hatched blocks and two white blocks. In FIG. 12A to FIG. 12E, to simplify the explanation, it is assumed that the motion information 18 of these two white blocks is compared without considering the shaded blocks. One of the two white blocks is a motion reference block p, and the other is a motion reference block q (usable block q) that has already been determined to be usable. Unless otherwise specified, any of the two white blocks may be the motion reference block p.

  FIG. 12A shows an example in which both the motion reference block p and the available block q are spatial blocks. In the example of FIG. 12A, if the motion information 18 of the blocks A and B is the same, it is determined that the motion information 18 is the same. At this time, the sizes of the blocks A and B do not have to be the same.

  FIG. 12B illustrates an example in which one of the motion reference block p and the available block q is the block A in the spatial direction, and the other is the block TB in the temporal direction. In FIG. 12B, there is one block having motion information in the block TB in the time direction. If the motion information 18 of the block TB in the time direction and the motion information 18 of the block A in the spatial direction are the same, it is determined that the motion information 18 is the same. At this time, the sizes of the blocks A and TB do not have to be the same.

  FIG. 12C shows another example in which one of the motion reference block p and the available block q is a block A in the spatial direction and the other is a block TB in the temporal direction. FIG. 12C shows a case where the block TB in the time direction is divided into a plurality of small blocks, and there are a plurality of small blocks having the motion information 18. In the example of FIG. 12C, if all the blocks having the motion information 18 have the same motion information 18, and the motion information 18 is the same as the motion information 18 of the block A in the spatial direction, the motion information 18 is the same. It is determined. At this time, the sizes of the blocks A and TB do not have to be the same.

  FIG. 12D illustrates an example in which both the motion reference block p and the available block q are temporal blocks. In this case, if the motion information 18 of the blocks TB and TE is the same, it is determined that the motion information 18 is the same.

  FIG. 12E shows another example in which the motion reference block p and the available block q are both temporal blocks. FIG. 12E shows a case where the blocks TB and TE in the time direction are each divided into a plurality of small blocks, and there are a plurality of small blocks each having the motion information 18. In this case, the motion information 18 is compared for each small block in the block, and if the motion information 18 is the same for all the small blocks, the motion information 18 of the block TB and the motion information 18 of the block TE are the same. It is determined that there is.

  FIG. 12F illustrates still another example in which the motion reference block p and the available block q are both temporal blocks. FIG. 12F shows a case where the block TE in the time direction is divided into a plurality of small blocks, and there are a plurality of small blocks having the motion information 18 in the block TE. When all the motion information 18 of the block TE is the same motion information 18 and the same as the motion information 18 included in the block TD, it is determined that the motion information 18 of the blocks TD and TE is the same.

  In this way, in step S803, it is determined whether or not the motion information 18 of the motion reference block p and the motion information 18 of the available block q are the same. In the example of FIGS. 12A to 12F, the number of available blocks q to be compared with the motion reference block p has been described as 1. However, when the number of available blocks q is 2 or more, the motion information of the motion reference block p 18 may be compared with the motion information 18 of each available block q. Further, when scaling described later is applied, the motion information 18 after scaling becomes the motion information 18 described above.

  Note that the determination that the motion information of the motion reference block p and the motion information of the available block q are the same is not limited to the case where the motion vectors included in the motion information completely match. For example, if the norm of the difference between two motion vectors is within a predetermined range, the motion information of the motion reference block p and the motion information of the available block q may be regarded as substantially the same.

  FIG. 13 shows a more detailed configuration of the prediction unit 101. As described above, the prediction unit 101 receives the available block information 30, the reference motion information 19, and the reference image signal 17, and outputs the prediction image signal 11, the motion information 18, and the selected block information 31. As illustrated in FIG. 13, the motion information selection unit 118 includes a spatial direction motion information acquisition unit 110, a time direction motion information acquisition unit 111, and a motion information changeover switch 112.

  The spatial direction motion information acquisition unit 110 receives the usable block information 30 and the reference motion information 19 related to the spatial direction motion reference block. The spatial direction motion information acquisition unit 110 outputs motion information 18A including motion information included in each available block located in the spatial direction and an index value of the available block. When the information shown in FIG. 11 is input as the usable block information 30, the spatial direction motion information acquisition unit 110 generates two motion information outputs 18A, and each motion information output 18A includes an available block and this usage. The motion information 19 included in the possible block is included.

  The temporal direction motion information acquisition unit 111 receives the usable block information 30 and the reference motion information 19 related to the temporal direction motion reference block. The time direction motion information acquisition unit 111 outputs the motion information 19 included in the available time direction motion reference block specified by the available block information 30 and the index value of the available block as motion information 18B. The temporal direction motion reference block is divided into a plurality of small pixel blocks, and each small pixel block has motion information 19. The motion information 18B output from the temporal direction motion information acquisition unit 111 includes a group of motion information 19 included in each small pixel block in the available block, as illustrated in FIG. When the motion information 18B includes a group of motion information 19, motion compensation prediction can be executed on the encoding target block in units of small pixel blocks obtained by dividing the encoding target block. When the information shown in FIG. 11 is input as the usable block information 30, the time direction motion information acquisition unit 111 generates two motion information outputs 18B, and each motion information output includes an available block and this usable information. It includes a group of motion information 19 that the block has.

  The time direction motion information acquisition unit 111 obtains an average value or representative value of motion vectors included in the motion information 19 of each pixel small block, and outputs the average value or representative value of the motion vectors as motion information 18B. It doesn't matter.

  The motion information changeover switch 112 in FIG. 13 selects an appropriate usable block as a selected block based on the motion information 18A and 18B output from the spatial direction motion information acquisition unit 110 and the temporal direction motion information acquisition unit 111. Then, the motion information 18 (or group of motion information 18) corresponding to the selected block is output to the motion compensation unit 113. The motion information changeover switch 112 outputs selected block information 31 related to the selected block. The selected block information 31 includes the name of the index p or the motion reference block, and is also simply referred to as selection information. The selected block information 31 is not limited to the name of the index p and the motion reference block, and may be any information as long as the position of the selected block can be specified.

For example, the motion information changeover switch 112 selects an available block that minimizes the coding cost derived by the cost equation shown in the following Equation 1 as a selected block.

  Here, J represents the coding cost, and D represents the coding distortion that represents the sum of square errors between the input image signal 10 and the reference image signal 17. R indicates the code amount estimated by provisional encoding, and λ. The Lagrange undetermined coefficient determined by the quantization width or the like is shown. The coding cost J may be calculated using only the code amount R or the encoding distortion D instead of the equation 1, and the value obtained by approximating the code amount R or the encoding distortion D is used. A cost function may be created. Furthermore, the coding distortion D is not limited to the sum of square errors, and may be a sum of absolute values (SAD) of prediction errors. As the code amount R, only the code amount related to the motion information 18 may be used. In addition, it is not limited to an example in which an available block with the lowest coding cost is selected as the selected block, and one available block having a value within a certain range that is equal to or larger than the smallest value of the coding cost is selected as the selected block. It does not matter if it is selected.

  The motion compensation unit 113 derives the position of the pixel block from which the reference image signal 17 is extracted as the predicted image signal 11 based on the motion information (or a group of motion information) included in the selected block selected by the motion information selection unit 118. To do. When a group of motion information is input to the motion compensation unit 113, the motion compensation unit 113 divides a pixel block from which the reference image signal 17 is extracted as the predicted image signal 11 into small pixel blocks (for example, 4 × 4 pixel blocks). And the prediction image signal 11 is acquired from the reference image signal 17 by applying the corresponding motion information 18 to each of these small pixel blocks. The position of the block from which the predicted image signal 11 is acquired is, for example, a position shifted from the small pixel block in the spatial direction according to the motion vector 18a included in the motion information 18, as shown in FIG. 4A.

The motion compensation process for the encoding target block is H.264. Similar to the H.264 motion compensation process can be used. Here, as an example, an interpolation method with 1/4 pixel accuracy will be specifically described. In 1/4 pixel precision interpolation, if each component of the motion vector is a multiple of 4, the motion vector points to an integer pixel position. In other cases, the motion vector indicates a predicted position corresponding to the interpolation position with fractional accuracy.

Here, x and y indicate vertical and horizontal indexes indicating the head position (for example, the upper left vertex) of the prediction target block, and x_pos and y_pos indicate the corresponding prediction position of the reference image signal 17. (mv_x, mv_y) indicates a motion vector having a 1/4 pixel accuracy. Next, a predicted pixel is generated by interpolation or interpolation of the corresponding pixel position of the reference image signal 17 for the determined pixel position. In FIG. An example of H.264 prediction pixel generation is shown. In FIG. 15, squares indicated by capital letters (hatched squares) indicate pixels at integer positions, and squares indicated by shading indicate interpolation pixels at 1/2 pixel positions. Yes. A square displayed in white indicates an interpolation pixel corresponding to a 1/4 pixel position. For example, in FIG. 15, the 1/2 pixel interpolation process corresponding to the positions of the alphabets b and h is calculated by the following Equation 3.

Here, alphabets (for example, b, h, C1, etc.) shown in Formula 3 and Formula 4 below indicate pixel values of pixels to which the same alphabet is assigned in FIG. “>>” indicates a right shift operation, and “>> 5” corresponds to division by 32. That is, the interpolated pixel at the 1/2 pixel position is calculated using a 6-tap FIR (Finite Impulse Response) filter (tap coefficient: (1, -5, 20, 20, -5, 1) / 32).

Further, the interpolation processing of ¼ pixels corresponding to the positions of the alphabets a and d in FIG.

  In this way, the interpolation pixel at the 1/4 pixel position is calculated using a 2-tap average value filter (tap coefficients: (1/2, 1/2)). The interpolation process of 1/2 pixel corresponding to the alphabet j existing in the middle of the four integer pixel positions is generated by using both directions of 6 taps in the vertical direction and 6 taps in the horizontal direction. Interpolated pixel values are generated in the same manner for pixel positions other than those described.

  Note that the interpolation processing is not limited to the examples of Equation 3 and Equation 4, and may be generated using other interpolation coefficients. The interpolation coefficient may be a fixed value given from the encoding control unit 150, or the interpolation coefficient is optimized for each frame based on the encoding cost described above, and the optimized interpolation coefficient is calculated. May be used.

  In the present embodiment, the processing related to motion vector block prediction processing in which a motion reference block is a macroblock (for example, a 16 × 16 pixel block) is described. However, the present invention is not limited to a macroblock, and a 16 × 8 pixel block unit. The prediction process may be executed in units of 8 × 16 pixel blocks, 8 × 8 pixel blocks, 8 × 4 pixel blocks, 4 × 8 pixel blocks, or 4 × 4 pixel blocks. In this case, the information regarding the motion vector block is derived for each pixel block. Further, the above prediction processing may be performed in units larger than 16 × 16 pixel blocks, such as 32 × 32 pixel block units, 32 × 16 pixel block units, and 64 × 64 pixel block units.

  When substituting the reference motion vector in the motion vector block as the motion vector of the small pixel block in the encoding target block, (A) a negative value (inversion vector) of the reference motion vector may be substituted, or (B) A weighted average value or median value, maximum value, and minimum value using a reference motion vector corresponding to a small block and a reference motion vector adjacent to the reference motion vector may be substituted.

  FIG. 16 schematically shows the operation of the prediction unit 101. As shown in FIG. 16, first, a reference frame (motion reference frame) including a temporal direction reference motion block is acquired (step S1501). The motion reference frame is typically a reference frame having the smallest temporal distance from the encoding target frame and is a temporally past reference frame. For example, the motion reference frame is a frame encoded immediately before the encoding target frame. In another example, any reference frame in which the motion information 18 is stored in the motion information memory 108 may be acquired as a motion reference frame. Next, the spatial direction motion information acquisition unit 110 and the temporal direction motion information acquisition unit 111 each acquire the available block information 30 output from the available block acquisition unit 109 (step S1502). Next, the motion information changeover switch 112 selects one of the available blocks as a selected block according to Equation 1, for example (step S1503). Subsequently, the motion compensation unit 113 copies the motion information included in the selected block to be encoded (step S1504). At this time, if the selected block is a spatial direction reference block, as shown in FIG. 17, the motion information 18 included in this selected block is copied to the encoded reference block. If the selected block is a temporal direction reference block, the group of motion information 18 included in the selected block is copied to the encoding target block together with the position information. Next, motion compensation is executed using the motion information 18 or the group of motion information 18 copied by the motion compensation unit 113, and the predicted image signal 11 and the motion information 18 used for motion compensation prediction are output.

  FIG. 18 shows a more detailed configuration of the variable length coding unit 104. As shown in FIG. 18, the variable length coding unit 104 includes a parameter coding unit 114, a transform coefficient coding unit 115, a selected block coding unit 116, and a multiplexing unit 117. The parameter encoding unit 114 encodes parameters necessary for decoding, such as prediction mode information, block size information, and quantization parameter information, excluding the transform coefficient information 13 and the selected block information 31, and generates encoded data 14A. . The transform coefficient encoding unit 115 encodes the transform coefficient information 13 to generate encoded data 14B. In addition, the selected block encoding unit 116 encodes the selected block information 31 with reference to the available block information 30, and generates encoded data 14C.

  As shown in FIG. 19, when the available block information 30 includes an index and availability of a motion reference block corresponding to the index, a motion that cannot be used among a plurality of preset motion reference blocks. Excluding the reference block, only the available motion reference block is converted into syntax (stds_idx). In FIG. 19, five motion reference blocks out of nine motion reference blocks cannot be used, and syntax stds_idx is assigned in order from 0 to four motion reference blocks excluding these. In this example, since the selected block information to be encoded is not selected from nine, but is selected from four available blocks, the allocated code amount (bin number) can be reduced on average.

  FIG. 20 illustrates an example of a code table indicating binary information (bin) of syntax stds_idx and syntax stds_idx. As shown in FIG. 18, the smaller the number of available motion reference blocks, the lower the average number of bins required for encoding the syntax stds_idx. For example, when the number of available blocks is 4, the syntax stds_idx can be represented by 3 bits or less. The binary information (bin) of the syntax stds_idx may be binarized so that every stds_idx has the same bin number for every number of usable blocks, and binarized according to a binarization method determined by prior learning. It doesn't matter. Also, a plurality of binarization methods may be prepared and switched appropriately for each encoding target block.

  Entropy coding (for example, isometric coding, Huffman coding, arithmetic coding, etc.) can be applied to these coding units 114, 115, 116, and the generated coded data 14A, 14B, 14C is multiplexed by the multiplexing unit 117 and output.

  In the present embodiment, the description is given assuming an example in which a frame encoded one frame before the encoding target frame is referred to as a reference frame. However, the motion vector and reference in the reference motion information 19 included in the selected block are described. The motion vector may be scaled (or normalized) using the frame number, and the reference motion information 19 may be applied to the encoding target block.

This scaling processing will be specifically described with reference to FIG. Tc shown in FIG. 21 indicates a time distance (POC (number indicating the display order) distance) between the encoding target frame and the motion reference frame, and is calculated by the following Equation 5. Tr [i] shown in FIG. 21 indicates a time distance between the motion reference frame and the frame i referred to by the selected block, and is calculated by the following Equation 6.

  Here, curPOC indicates the POC (Picture Order Count) of the encoding target frame, colPOC indicates the POC of the motion reference frame, and refPOC indicates the POC of the frame i referenced by the selected block. Clip (min, max, target) outputs min if target is less than min, outputs max if it is greater than max, and outputs target otherwise. This is a clip function. DiffPicOrderCnt (x, y) is a function for calculating the difference between two POCs.

If the motion vector of the selected block is MVr = (MVr_x, MVr_y) and the motion vector to be applied to the encoding target block is MV = (MV_x, MV_y), the motion vector MV is calculated by Equation 7 below.

  Here, Abs (x) represents a function for extracting the absolute value of x. In this manner, in motion vector scaling, the motion vector MVr assigned to the selected block is converted into a motion vector MV between the encoding target frame and the motion first reference frame.

In addition, another example relating to scaling of motion vectors will be described below.
First, for each slice or frame, a scaling coefficient (DistScaleFactor [i]) is obtained for all time distances tr that the motion reference frame can take according to the following Equation 8. The number of scaling factors is equal to the number of frames referenced by the selected block, that is, the number of reference frames.

  The calculation of tx shown in Equation 8 may be tabulated in advance.

When scaling for each block to be encoded, the motion vector MV can be calculated only by multiplication, addition, and shift operation by using the following formula 9.

  When such scaling processing is performed, the motion information 18 after scaling is applied to the processing of the usable block acquisition unit 109 together with the prediction unit 101. When the scaling process is performed, the reference frame referred to by the encoding target block is a motion reference frame.

  FIG. 22 shows a syntax structure in the image encoding unit 100. As shown in FIG. 22, the syntax mainly includes three parts, namely, a high level syntax 901, a slice level syntax 904, and a macroblock level syntax 907. The high-level syntax 901 holds higher layer syntax information above the slice. The slice level syntax 904 holds information necessary for each slice, and the macroblock level syntax 907 holds data required for each macroblock shown in FIGS. 7A to 7D.

  Each part includes more detailed syntax. High level syntax 901 includes sequence and picture level syntax such as sequence parameter set syntax 902 and picture parameter set syntax 903. The slice level syntax 904 includes a slice header syntax 905, a slice data syntax 906, and the like. Further, the macroblock level syntax 907 includes a macroblock layer syntax 908, a macroblock prediction syntax 909, and the like.

  23A and 23B show examples of macroblock layer syntax. The available_block_num shown in FIGS. 23A and 23B indicates the number of available blocks. If this is a value larger than 1, the selected block information needs to be encoded. Furthermore, stds_idx indicates selected block information, and stds_idx is encoded using the above-described code table corresponding to the number of available blocks.

  FIG. 23A shows the syntax when the selected block information is encoded after mb_type. When the mode indicated by mb_type is a defined size or a defined mode (TARGET_MODE), and available_block_num is a value greater than 1, stds_idx is encoded. For example, stds_idx is encoded when the motion information of the selected block is available when the block size is 64 × 64 pixels, 32 × 32 pixels, 16 × 16 pixels, or in the direct mode.

  FIG. 23B shows the syntax when the selected block information is encoded before mb_type. When available_block_num is a value larger than 1, stds_idx is encoded. If available_block_num is 0, H. Since conventional motion compensation represented by H.264 is performed, mb_type is encoded.

  It should be noted that syntax elements not defined in the present invention can be inserted between the rows of the tables shown in FIGS. 23A and 23B, and other conditional branch descriptions may be included. . Alternatively, the syntax table can be divided and integrated into a plurality of tables. Moreover, it is not always necessary to use the same term, and it may be arbitrarily changed depending on the form to be used. Further, each syntax element described in the macroblock layer syntax may be changed as specified in a macroblock data syntax described later.

  Further, it is possible to reduce mb_type information by using stds_idx information. FIG. 2 is a code table corresponding to mb_type and mb_type at the time of B slice in H.264. N shown in FIG. 24A is a value representing the size of the block to be encoded, such as 16, 32, 64, and M is half the value of N. Therefore, when mb_type is 4 to 21, it indicates that the encoding target block is a rectangular block. Also, L0, L1, and Bi in FIG. 24A indicate unidirectional prediction (List0 direction only), unidirectional prediction (List1 direction only), and bidirectional prediction, respectively. When the encoding target block is a rectangular block, mb_type includes information indicating which of L0, L1, and Bi is predicted for each of the two rectangular blocks in the encoding target block. B_Sub means that the above process is performed on each pixel block obtained by dividing a macroblock into four. For example, when the encoding target block is a 64 × 64 pixel macroblock, the encoding target block is further encoded with mb_type assigned to each of four 32 × 32 pixel blocks obtained by dividing the macroblock into four. It becomes.

  If the selected block indicated by stds_idx is Spatial Left (a pixel block adjacent to the left side of the encoding target block), the motion information of the pixel block adjacent to the left side of the encoding target block is Since it is motion information, stds_idx is predicted for the encoding target block using horizontally long rectangular blocks indicated by mb_type = 4, 6, 8, 10, 12, 14, 16, 18, 20 in FIG. 24A. Is equivalent to executing When the selected block indicated by stds_idx is Spatial Up, the motion information adjacent to the upper side of the block to be encoded is used as the motion information of the block to be encoded, so stds_idx is mb_type = 5,7, This has the same meaning as executing the prediction on the vertically long rectangular block indicated by 9,11,13,15,17,19,21. Therefore, by using stds_idx, it is possible to create a code table in which the columns of mb_type = 4 to 21 in FIG. 24A as shown in FIG. 24B are reduced. Similarly, H.264 shown in FIG. As for the code table corresponding to mb_type and mb_type at the time of P slice in H.264, a code table with a reduced number of mb_types as shown in FIG. 24D can be created.

  Also, stds_idx information may be included in mb_type information for encoding. FIG. 25A is a code table when stds_idx information is included in mb_type information, and shows an example of a code table corresponding to mb_type and mb_type in a B slice. B_STDS_X (X = 0, 1, 2) in FIG. 25A indicates a mode corresponding to stds_idx, and B_STDS_X is added by the number of available blocks (in FIG. 25A, the number of available blocks is 3). Similarly, FIG. 25B shows another example of mb_type related to the P slice. The description of FIG. 25B is omitted because it is similar to the B slice.

  The order of mb_type and the binarization method (binization) are not limited to the examples shown in FIGS. 25A and 25B, and mb_type may be encoded according to another order and binarization method. B_STDS_X and P_STDS_X do not need to be continuous and may be arranged between each mb_type. Also, the binarization method (binization) may be designed based on the selection frequency learned in advance.

  In the present embodiment, the present invention can also be applied to an extended macroblock that performs motion compensation prediction by collecting a plurality of macroblocks. In the present embodiment, any order may be used for the scan order of encoding. For example, the present invention can be applied to a line scan or a Z scan.

  As described above, the image coding apparatus according to the present embodiment selects an available block from a plurality of motion reference blocks, and applies a motion reference block to an encoding target block according to the selected number of available blocks. The information for specifying is generated, and this information is encoded. Therefore, according to the image coding apparatus according to the present embodiment, it is possible to perform motion compensation in units of small pixel blocks that are finer than the current block while reducing the amount of code related to motion vector information. Can be realized.

(Second Embodiment)
FIG. 26 shows an image encoding device according to the second embodiment of the present invention. In the second embodiment, parts and operations different from those in the first embodiment will be mainly described. As shown in FIG. 26, the image coding unit 200 according to the present embodiment is different from the first embodiment in the configuration of a prediction unit 201 and a variable length coding unit 204. As illustrated in FIG. 27, the prediction unit 201 includes a first prediction unit 101 and a second prediction unit 202, and selectively generates a predicted image signal 11 by selectively switching the first and second prediction units 101 and 202. . The first prediction unit 101 has the same configuration as the prediction unit 101 (FIG. 1) according to the first embodiment, and follows a prediction method (first prediction method) that performs motion compensation using the motion information 18 included in the selected block. The prediction image signal 11 is generated. The second prediction unit 202 performs motion compensation on the current block using one motion vector. The predicted image signal 11 is generated according to a prediction method (second prediction method) such as H.264. The second prediction unit 202 uses the input image signal 10 and the reference image signal 17 from the frame memory to generate a predicted image signal 11B.

  FIG. 28 schematically shows the configuration of the second prediction unit 202. As shown in FIG. 28, the second prediction unit 202 includes a motion information acquisition unit 205 that generates the motion information 21 using the input image signal 10 and the reference image signal 17, and the reference image signal 17 and the motion information 21. Is used to generate a predicted image signal 11A. The motion information acquisition unit 205 obtains a motion vector to be assigned to the encoding target block based on the input image signal 10 and the reference image signal 17 by, for example, block matching. As an evaluation criterion for matching, a value obtained by accumulating the difference between the input image signal 10 and the interpolated image after matching for each pixel is used.

  Note that the motion information acquisition unit 205 may determine an optimal motion vector using a value obtained by converting the difference between the predicted image signal 11 and the input image signal 10. Further, an optimal motion vector may be determined in consideration of the size of the motion vector and the amount of code of the motion vector and the reference frame number, or using Equation 1. The matching method may be executed based on search range information provided from the outside of the image encoding device, or may be executed hierarchically for each pixel accuracy. Further, the motion information given by the encoding control unit 150 may be used as the output 21 of the motion information acquisition unit 205 without performing the search process.

  The prediction unit 101 in FIG. 27 further includes a prediction method changeover switch 203 that selects and outputs one of the prediction image signal 11A from the first prediction unit 101 and the prediction image signal 11B from the second prediction unit 202. Yes. For example, the prediction method changeover switch 203 uses the input image signal 10 for each of the predicted image signals 11A and 11B to obtain an encoding cost according to, for example, Equation 1 so that the prediction image becomes smaller. One of the signals 11A and 11B is selected and output as the predicted image signal 11. Further, the prediction method changeover switch 203, together with the motion information 18 and the selected block information 31, predictive switching indicating whether the output predicted image signal 11 is output from either the first prediction unit 101 or the second prediction unit 202. Information 32 is also output. The output motion information 18 is encoded by the variable length encoding unit 204 and then multiplexed on the encoded data 14.

  FIG. 29 schematically shows the configuration of the variable length coding unit 204. The variable length encoding unit 204 illustrated in FIG. 29 includes a motion information encoding unit 217 in addition to the configuration of the variable length encoding unit 104 illustrated in FIG. 29, unlike the selected block encoding unit 116 in FIG. 18, the selected block encoding unit 216 encodes the prediction switching information 32 to generate encoded data 14D. When the first prediction unit 101 executes the prediction process, the selected block encoding unit 216 further encodes the available block information 30 and the selected block information 31. The encoded usable block information 30 and the selected block information 31 are included in the encoded data 14D. When the second prediction unit 202 executes the prediction process, the motion information encoding unit 217 encodes the motion information 18 and generates encoded data 14E. Each of the selected block encoding unit 216 and the motion information encoding unit 217 performs a prediction process based on prediction switching information 32 indicating whether or not a predicted image has been generated by motion compensated prediction using the motion information of the selected block. It is determined which one of the first prediction unit 101 and the second prediction unit 202 has been executed.

  The multiplexing unit 117 receives the encoded data 14A, B, D, E from the parameter encoding unit 114, the transform coefficient encoding unit 115, the selected block encoding unit 216, and the motion information encoding unit, and receives the encoded data 14A, B, D, E are multiplexed.

  FIG. 30A and FIG. 30B each show an example of the macroblock layer syntax according to the present embodiment. The available_block_num shown in FIG. 30A indicates the number of available blocks. When this is a value greater than 1, the selected block encoding unit 216 encodes the selected block information 31. Also, stds_flag is a flag indicating whether or not the motion information of the selected block is used as motion information of the encoding target block in motion compensation prediction, that is, the prediction method changeover switch 203 is the first prediction unit 101 and the second prediction unit. A flag indicating which of 202 is selected. When the number of available blocks is larger than 1 and stds_flag is 1, it indicates that the motion information of the selected block is used for motion compensation prediction. If stds_flag is 0, the motion information of the selected block is not used and Similarly to H.264, a difference value obtained by directly or predicting the information of motion information 18 is encoded. Furthermore, stds_idx indicates selected block information, and the code table corresponding to the number of usable blocks is as described above.

  FIG. 30A shows the syntax when the selected block information is encoded after mb_type. Only when the mode indicated by mb_type is a predetermined size or a predetermined mode, stds_flag and stds_idx are encoded. For example, stds_flag and stds_idx are encoded when the motion information of the selected block is available when the block size is 64 × 64, 32 × 32, 16 × 16, or in the direct mode.

  FIG. 30B shows the syntax when the selected block information is encoded before mb_type. For example, when stds_flag is 1, mb_type does not need to be encoded. If stds_flag is 0, mb_type is encoded.

  As described above, the image coding apparatus according to the second embodiment uses the first prediction unit 101 according to the first embodiment and the prediction scheme such as H.264 so that the coding cost is reduced. The input image signal is compressed and encoded by selectively switching between the second prediction unit 202 and the second prediction unit 202. Therefore, in the image coding apparatus according to the second embodiment, the coding efficiency is improved as compared with the image coding apparatus according to the first embodiment.

(Third embodiment)
FIG. 31 schematically shows an image decoding apparatus according to the third embodiment. As shown in FIG. 31, the image decoding apparatus includes an image decoding unit 300, a decoding control unit 350, and an output buffer 308. The image decoding unit 300 is controlled by the decoding control unit 350. An image decoding apparatus according to the third embodiment corresponds to the image encoding apparatus according to the first embodiment. That is, the decoding process by the image decoding apparatus in FIG. 31 has a complementary relationship with the encoding process by the image encoding process in FIG. The image decoding device in FIG. 31 may be realized by hardware such as an LSI chip, or may be realized by causing a computer to execute an image decoding program.

  The image decoding apparatus in FIG. 31 includes an encoded sequence decoding unit 301, an inverse quantization / inverse conversion unit 302, an adder 303, a frame memory 304, a prediction unit 305, a motion information memory 306, and an available block acquisition unit 307. I have. In the image decoding unit 300, encoded data 80 from a storage system or transmission system (not shown) is input to the encoded string decoding unit 301. The encoded data 80 corresponds to, for example, the encoded data 14 transmitted in a multiplexed state from the image encoding device in FIG.

  In the present embodiment, a pixel block (for example, a macro block) that is a decoding target is simply referred to as a decoding target block. An image frame including a decoding target block is referred to as a decoding target frame.

  The encoded sequence decoding unit 301 performs decoding by syntax analysis based on the syntax for each frame or field. Specifically, the coded sequence decoding unit 301 sequentially performs variable length decoding on the code sequence of each syntax, and provides prediction information such as transform coefficient information 33, selected block information 61, block size information, and prediction mode information. Including the encoding parameters relating to the decoding target block.

  In the present embodiment, the decoding parameters include the transform coefficient 33, the selected block information 61, and the prediction information, and include all parameters necessary for decoding such as information about the transform coefficient and information about the quantization. Prediction information, information on transform coefficients, and information on quantization are input to the decoding control unit 350 as control information 71. The decoding control unit 350 gives the decoding control information 70 including parameters necessary for decoding such as prediction information and quantization parameters to each unit of the image decoding unit 300.

  Further, the encoded sequence decoding unit 301 simultaneously decodes the encoded data 80 to obtain prediction information and selected block information 61, as will be described later. The motion information 38 including the motion vector and the reference frame number may not be decoded.

  The transform coefficient 33 decoded by the encoded sequence decoding unit 301 is sent to the inverse quantization / inverse transform unit 302. Various information relating to the quantization decoded by the coded sequence decoding unit 301, that is, the quantization parameter and the quantization matrix are given to the decoding control unit 350, and when the inverse quantization is performed, the inverse quantization / inverse transformation is performed. Loaded into the unit 302. The inverse quantization / inverse transform unit 302 inversely quantizes the transform coefficient 33 according to the loaded information related to quantization, and then performs inverse transform processing (for example, inverse discrete cosine transform) to obtain the prediction error signal 34. obtain. The inverse transform process by the inverse quantization / inverse transform unit 302 in FIG. 31 is an inverse transform of the transform process by the transform / quantization unit in FIG. For example, when the wavelet transform is performed by the image encoding device (FIG. 1), the inverse quantization / inverse transform unit 302 executes the corresponding inverse quantization and inverse wavelet transform.

  The prediction error signal 34 restored by the inverse quantization / inverse transform unit 302 is input to the adder 303. The adder 303 adds the prediction error signal 34 and the prediction image signal 35 generated by the prediction unit 305 described later to generate a decoded image signal 36. The generated decoded image signal 36 is output from the image decoding unit 300, temporarily stored in the output buffer 308, and then output according to the output timing managed by the decoding control unit 350. The decoded image signal 36 is stored as a reference image signal 37 in the frame memory 304. The reference image signal 37 is sequentially read from the frame memory 304 for each frame or field and is input to the prediction unit 305.

  The available block acquisition unit 307 receives reference motion information 39 from a motion information memory 306 described later, and outputs usable block information 60. The operation of the available block acquisition unit 307 is the same as that of the available block acquisition unit 109 (FIG. 1) described in the first embodiment.

  The motion information memory 306 receives the motion information 38 from the prediction unit 305 and temporarily stores it as reference motion information 39. The motion information memory 306 temporarily stores the motion information 38 output from the prediction unit 305 as reference motion information 39. FIG. 4 shows an example of the motion information memory 306. The motion information memory 306 holds a plurality of motion information frames 26 having different encoding times. The motion information 38 or the group of motion information 38 that has been decoded is stored as reference motion information 39 in the motion information frame 26 corresponding to the decoding time. In the motion information frame 26, the reference motion information 39 is stored, for example, in units of 4 × 4 pixel blocks. The reference motion information 39 held in the motion information memory 306 is read and referenced by the prediction unit 305 when generating the motion information 38 of the decoding target block.

  Next, the motion reference block and the available block according to the present embodiment will be described. The motion reference block is a candidate block that is selected from the already decoded regions according to a predetermined method by the above-described image encoding device and image decoding device. FIG. 8A shows an example regarding an available block. In FIG. 8A, a total of nine motion reference blocks of four motion reference blocks in the decoding target frame and five motion reference blocks in the reference frame are arranged. The motion reference blocks A, B, C, and D in the decoding target frame in FIG. 8A are blocks adjacent to the decoding target block on the left, upper, upper right, and upper left. In the present embodiment, a motion reference block selected from decoding target frames including a decoding target block is referred to as a spatial direction motion reference block. The motion reference block TA in the reference frame is a pixel block at the same position as the decoding target block in the reference frame, and the pixel blocks TB, TC, TD, TE in contact with the motion reference block TA are moved. Selected as a reference block. A motion reference block selected from among the pixel blocks in the reference frame is referred to as a temporal direction motion reference block. A frame in which the temporal direction motion reference block is located is referred to as a motion reference frame.

  The spatial direction motion reference block is not limited to the example shown in FIG. 8A. As shown in FIG. 8B, the pixel block to which the pixels a, b, c, and d adjacent to the decoding target block belong is used as the spatial direction motion reference block. It does not matter if it is selected. In this case, the relative positions (dx, dy) of the pixels a, b, c, d with respect to the upper left pixel in the decoding target block are shown in FIG. 8C.

  Further, as illustrated in FIG. 8D, all the pixel blocks A1 to A4, B1, B2, C, and D adjacent to the decoding target block may be selected as the spatial direction motion reference block. In FIG. 8D, the number of spatial direction motion reference blocks is eight.

  Further, the temporal direction motion reference blocks TA to TE may partially overlap each other as shown in FIG. 8E, or may be separated from each other as shown in FIG. 8F. In addition, the temporal direction motion reference block does not necessarily need to be located around the block at the Collocate position, and may be a pixel block at any position within the motion reference frame. For example, using the motion information of an already decoded block adjacent to the decoding target block, the reference block indicated by the motion vector included in the motion information is selected as the center of the motion reference block (for example, block TA). It doesn't matter. Furthermore, the reference blocks in the time direction may not be arranged at equal intervals.

  In the method for selecting a motion reference block as described above, if both the image decoding apparatus and the image decoding apparatus share information on the number and position of spatial direction and temporal direction motion reference blocks, the motion reference block May be selected from any number and position. The size of the motion reference block is not necessarily the same size as the decoding target block. For example, as illustrated in FIG. 8D, the size of the motion reference block may be larger than the size of the decoding target block, may be smaller, or may be an arbitrary size. The shape of the motion reference block is not limited to a square shape, and may be a rectangular shape.

  Next, available blocks will be described. The available block is a pixel block selected from the motion reference blocks, and is a pixel block to which motion information can be applied to the decoding target block. The available blocks have different motion information. The available blocks are selected by executing the available block determination process shown in FIG. 9 for a total of nine motion reference blocks in the decoding target frame and the reference frame as shown in FIG. 8A, for example. . FIG. 10 shows the result of executing the available block determination process shown in FIG. In FIG. 10, a shaded pixel block indicates an unusable block, and a white block indicates an available block. That is, two of the spatial direction motion reference blocks and two of the temporal direction motion reference blocks are determined as usable blocks. The motion information selection unit 314 in the prediction unit 305 can use one of the available blocks arranged in the temporal and spatial directions according to the selected block information 61 received from the selected block decoding unit 323. Select a block as the selected block.

  Next, the available block acquisition unit 307 will be described. The available block acquisition unit 307 has the same function as the available block acquisition unit 109 of the first embodiment, acquires the reference motion information 39 from the motion information memory 306, and can be used or used for each motion reference block. The available block information 60, which is information indicating an impossible block, is output.

  The operation of the available block acquisition unit 307 will be described with reference to the flowchart of FIG. First, the available block acquisition unit 307 determines whether or not the motion reference block (index p) has motion information (step S801). That is, in step S801, it is determined whether or not at least one small pixel block in the motion reference block p has motion information. When it is determined that the motion reference block p does not have motion information, that is, the temporal motion reference block is a block in the I slice without motion information, or all the small blocks in the temporal motion reference block If the pixel block has been subjected to intra prediction decoding, the process proceeds to step S805. In step S805, the motion reference block p is determined as an unusable block.

  When it is determined in step S801 that the motion reference block p has motion information, the available block acquisition unit 307 selects a motion reference block q (referred to as an available block q) that has already been determined as an available block. (Step S802). Here, q is a value smaller than p. Subsequently, the available block acquisition unit 307 compares the motion information of the motion reference block p with the motion information of the available block q for all q, and the motion reference block p can use the available block q. It is determined whether or not the same motion information is included (S803). If the motion reference block p has the same motion vector as that of the available block q, the process proceeds to step S805. In step S805, the motion reference block p is converted into an unavailable block by the available block acquisition unit 307. Determined. When the motion reference block p has motion information different from all the available blocks q, the motion reference block p is determined as an available block by the available block acquisition unit 307 in step S804.

  By executing the above-described usable block determination process for all the motion reference blocks, it is determined for each motion reference block whether the block is usable or unavailable, and usable block information 60 is generated. An example of the usable block information 60 is shown in FIG. The available block information 60 includes the index p and availability of the motion reference block, as shown in FIG. In FIG. 11, the usable block information 60 indicates that motion reference blocks whose indexes p are 0, 1, 5 and 8 are selected as usable blocks, and the number of usable blocks is four.

  Note that, in step S801 in FIG. 9, when at least one of the blocks in the temporal direction motion reference block p is an intra prediction encoded block, the available block acquisition unit 307 uses the motion reference block p. It may be determined as an impossible block. In other words, the process may proceed to step S802 only when all the blocks in the temporal direction motion reference block p are encoded by inter prediction.

  12A to 12E show an example in which it is determined that the motion information 38 of the motion reference block p and the motion information 38 of the available block q are the same in the comparison of the motion information 38 in step S803. 12A to 12E each show a plurality of hatched blocks and two white blocks. In FIG. 12A to FIG. 12E, in order to simplify the description, it is assumed that the motion information 38 of these two white blocks is compared without considering the shaded blocks. One of the two white blocks is a motion reference block p, and the other is a motion reference block q (usable block q) that has already been determined to be usable. Unless otherwise specified, any of the two white blocks may be the motion reference block p.

  FIG. 12A shows an example in which both the motion reference block p and the available block q are spatial blocks. In the example of FIG. 12A, if the motion information 38 of the blocks A and B is the same, it is determined that the motion information 38 is the same. At this time, the sizes of the blocks A and B do not have to be the same.

  FIG. 12B illustrates an example in which one of the motion reference block p and the available block q is the block A in the spatial direction, and the other is the block TB in the temporal direction. In FIG. 12B, there is one block having motion information in the block TB in the time direction. If the motion information 38 of the block TB in the time direction and the motion information 38 of the block A in the spatial direction are the same, it is determined that the motion information 38 is the same. At this time, the sizes of the blocks A and TB do not have to be the same.

  FIG. 12C shows another example in which one of the motion reference block p and the available block q is a block A in the spatial direction and the other is a block TB in the temporal direction. FIG. 12C shows a case where the block TB in the time direction is divided into a plurality of small blocks and there are a plurality of small blocks having the motion information 38. In the example of FIG. 12C, if all the blocks having the motion information 38 have the same motion information 38 and the motion information 38 is the same as the motion information 38 of the block A in the spatial direction, the motion information 38 is the same. It is determined. At this time, the sizes of the blocks A and TB do not have to be the same.

  FIG. 12D illustrates an example in which both the motion reference block p and the available block q are temporal blocks. In this case, if the motion information 38 of the blocks TB and TE is the same, it is determined that the motion information 38 is the same.

  FIG. 12E shows another example in which the motion reference block p and the available block q are both temporal blocks. FIG. 12E shows a case where the blocks TB and TE in the time direction are each divided into a plurality of small blocks, and there are a plurality of small blocks each having motion information 38. In this case, the motion information 38 is compared for each small block in the block, and if the motion information 38 is the same for all the small blocks, the motion information 38 of the block TB and the motion information 38 of the block TE are the same. It is determined that there is.

  FIG. 12F illustrates still another example in which the motion reference block p and the available block q are both temporal blocks. FIG. 12F shows a case where the block TE in the time direction is divided into a plurality of small blocks, and there are a plurality of small blocks having the motion information 38 in the block TE. When all the motion information 38 of the block TE is the same motion information 38 and is the same as the motion information 38 included in the block TD, it is determined that the motion information 38 of the blocks TD and TE is the same.

  In this way, in step S803, it is determined whether or not the motion information 38 of the motion reference block p and the motion information 38 of the available block q are the same. In the example of FIGS. 12A to 12F, the number of available blocks q to be compared with the motion reference block p has been described as 1. However, when the number of available blocks q is 2 or more, the motion information of the motion reference block p 38 may be compared with the motion information 38 of each available block q. When scaling described later is applied, the motion information 38 after scaling becomes the motion information 38 described above.

  Note that the determination that the motion information of the motion reference block p and the motion information of the available block q are the same is not limited to the case where the motion vectors included in the motion information completely match. For example, if the norm of the difference between two motion vectors is within a predetermined range, the motion information of the motion reference block p and the motion information of the available block q may be regarded as substantially the same.

  FIG. 32 is a block diagram showing the encoded sequence decoding unit 301 in more detail. As shown in FIG. 32, the encoded sequence decoding unit 301 separates the encoded data 80 into syntax units, a separating unit 320, a transform coefficient decoding unit 322 for decoding transform coefficients, and decoding selected block information. And a parameter decoding unit 321 that decodes a prediction block size, a parameter relating to quantization, and the like.

  The parameter decoding unit 321 receives the encoded data 80A including the prediction block size and the parameters related to quantization from the demultiplexing unit, and generates the control information 71 by decoding the encoded data 80A. The transform coefficient decoding unit 322 receives the encoded transform coefficient 80B from the separation unit 320, decodes the encoded transform coefficient 80B, and obtains transform coefficient information 33. The selected block decoding unit 323 receives the encoded data 80C relating to the selected block and the available block information 60 as inputs, and outputs selected block information 61. The input available block information 60 indicates the availability for each motion reference block, as shown in FIG.

Next, the prediction unit 305 will be described in detail with reference to FIG.
As illustrated in FIG. 33, the prediction unit 305 includes a motion information selection unit 314 and a motion compensation unit 313. The motion information selection unit 314 includes a spatial direction motion information acquisition unit 310, a time direction motion information acquisition unit 311, and motion information. A changeover switch 312 is provided. The prediction unit 305 basically has the same configuration and function as the prediction unit 101 described in the first embodiment.

  The prediction unit 305 receives the available block information 60, the selected block information 61, the reference motion information 39, and the reference image signal 37, and outputs a predicted image signal 35 and motion information 38. The spatial direction motion information acquisition unit 310 and the temporal direction motion information acquisition unit 311 have the same functions as the spatial direction motion information acquisition unit 110 and the temporal direction motion information acquisition unit 111 described in the first embodiment, respectively. The spatial direction motion information acquisition unit 310 uses the available block information 60 and the reference motion information 39 to generate motion information 38A including motion information and an index for each available block located in the spatial direction. The time direction motion information acquisition unit 311 uses the available block information 60 and the reference motion information 39, and motion information (or a group of motion information) including motion information and an index of each available block located in the time direction. 38B is generated.

  In the motion information changeover switch 312, one of the motion information 38 A from the spatial direction motion information acquisition unit 310 and the motion information (or group of motion information) 38 B from the time direction motion information acquisition unit 311 is selected according to the selected block information 61. The motion information 38 is obtained. The selected motion information 38 is sent to the motion compensation unit 313 and the motion information memory 306. The motion compensation unit 313 performs motion compensation prediction according to the selected motion information 38 in the same manner as the motion compensation unit 113 described in the first embodiment, and generates a predicted image signal 35.

  The motion vector scaling function of the motion compensation unit 313 is the same as that described in the first embodiment, and a description thereof will be omitted.

  FIG. 22 shows a syntax structure in the image decoding unit 300. As shown in FIG. 22, the syntax mainly includes three parts, namely, a high level syntax 901, a slice level syntax 904, and a macroblock level syntax 907. The high-level syntax 901 holds higher layer syntax information above the slice. The slice level syntax 904 holds information necessary for each slice, and the macroblock level syntax 907 holds data required for each macroblock shown in FIGS. 7A to 7D.

  Each part includes more detailed syntax. High level syntax 901 includes sequence and picture level syntax such as sequence parameter set syntax 902 and picture parameter set syntax 903. The slice level syntax 904 includes a slice header syntax 905, a slice data syntax 906, and the like. Further, the macroblock level syntax 907 includes a macroblock layer syntax 908, a macroblock prediction syntax 909, and the like.

  23A and 23B show examples of macroblock layer syntax. The available_block_num shown in FIGS. 23A and 23B indicates the number of available blocks. When this is a value larger than 1, the selected block information needs to be decoded. Furthermore, stds_idx indicates selected block information, and stds_idx is encoded using the above-described code table corresponding to the number of available blocks.

  FIG. 23A shows syntax when decoding selected block information after mb_type. When the prediction mode indicated by mb_type is a predetermined size or a predetermined mode (TARGET_MODE), and available_block_num is a value larger than 1, stds_idx is decoded. For example, stds_idx is encoded when the motion information of the selected block is available when the block size is 64 × 64 pixels, 32 × 32 pixels, 16 × 16 pixels, or in the direct mode.

  FIG. 23B shows the syntax when the selected block information is decoded before mb_type. When available_block_num is greater than 1, stds_idx is decoded. If available_block_num is 0, H. Since conventional motion compensation represented by H.264 is performed, mb_type is encoded.

  A syntax element not defined in the present invention can be inserted between the rows of the tables shown in FIGS. 23A and 23B, or other conditional branch descriptions can be included. Alternatively, the syntax table can be divided and integrated into a plurality of tables. Moreover, it is not always necessary to use the same term, and it may be arbitrarily changed depending on the form to be used. Further, each syntax element described in the macroblock layer syntax may be changed as specified in a macroblock data syntax described later.

  As described above, the image decoding apparatus according to the present embodiment decodes the image encoded by the image encoding apparatus according to the first embodiment described above. Therefore, the image decoding according to the present embodiment can reproduce a high-quality decoded image from relatively small encoded data.

(Fourth embodiment)
FIG. 34 schematically shows an image decoding apparatus according to the fourth embodiment. As shown in FIG. 34, the image decoding apparatus includes an image decoding unit 400, a decoding control unit 350, and an output buffer 308. An image decoding apparatus according to the fourth embodiment corresponds to the image encoding apparatus according to the second embodiment. In the fourth embodiment, parts and operations different from those of the third embodiment will be mainly described. As shown in FIG. 34, the image decoding unit 400 according to the present embodiment is different from the third embodiment in an encoded sequence decoding unit 401 and a prediction unit 405.

  The prediction unit 405 of the present embodiment includes a prediction method (first prediction method) that performs motion compensation using motion information included in the selected block, The prediction image signal 35 is generated by selectively switching a prediction method (second prediction method) such as H.264 that performs motion compensation on a decoding target block using one motion vector.

  FIG. 35 is a block diagram showing the encoded sequence decoding unit 401 in more detail. The encoded sequence decoding unit 401 illustrated in FIG. 35 includes a motion information decoding unit 424 in addition to the configuration of the encoded sequence decoding unit 301 illustrated in FIG. 35, unlike the selected block decoding unit 323 shown in FIG. 32, the selected block decoding unit 423 decodes the encoded data 80C related to the selected block to obtain the prediction switching information 62. The prediction switching information 62 indicates which one of the first and second prediction methods is used by the prediction unit 101 in the image encoding device in FIG. When the prediction switching information 62 indicates that the prediction unit 101 has used the first prediction method, that is, when the decoding target block is encoded by the first prediction method, the selected block decoding unit 423 performs encoding. The selected block information in the data 80C is decoded to obtain selected block information 61. When the prediction switching information 62 indicates that the prediction unit 101 uses the second prediction method, that is, when the decoding target block is encoded by the second prediction method, the selected block decoding unit 423 selects the selected block information. Without decoding the motion information, the motion information decoding unit 424 decodes the encoded motion information 80D to obtain the motion information 40.

  FIG. 36 is a block diagram showing the prediction unit 405 in more detail. A prediction unit 405 illustrated in FIG. 34 includes a first prediction unit 305, a second prediction unit 410, and a prediction method changeover switch 411. The second prediction unit 410 performs motion compensation prediction similar to the motion compensation unit 313 in FIG. 33 using the motion information 40 and the reference image signal 37 decoded by the coded sequence decoding unit 401, and performs the prediction image signal. 35B is generated. The first prediction unit 305 is the same as the prediction unit 305 described in the third embodiment, and generates a predicted image signal 35B. In addition, the prediction method changeover switch 411 selects one of the prediction image signal 35B from the second prediction unit 410 and the prediction image signal 35A from the first prediction unit 305 based on the prediction switching information 62. The prediction image signal 35 of the prediction unit 405 is output. At the same time, the prediction method changeover switch 411 sends the motion information used by the selected first prediction unit 305 or second prediction unit 410 to the motion information memory 306 as motion information 38.

  Next, regarding the syntax structure relating to the present embodiment, differences from the third embodiment will be mainly described.

  FIG. 30A and FIG. 30B each show an example of the macroblock layer syntax according to the present embodiment. The available_block_num shown in FIG. 30A indicates the number of available blocks. When this is a value larger than 1, the selected block decoding unit 423 decodes the selected block information in the encoded data 80C. Also, stds_flag is a flag indicating whether or not the motion information of the selected block is used as motion information of the decoding target block in motion compensation prediction, that is, the prediction method changeover switch 411 has the first prediction unit 305 and the second prediction unit. This is a flag indicating which of 410 has been selected. When the number of available blocks is larger than 1 and stds_flag is 1, it indicates that the motion information of the selected block is used for motion compensation prediction. If stds_flag is 0, the motion information of the selected block is not used and Similarly to H.264, a difference value obtained by directly or predicting motion information is encoded. Furthermore, stds_idx indicates selected block information, and the code table corresponding to the number of usable blocks is as described above.

  FIG. 30A shows syntax when decoding selected block information after mb_type. Only when the prediction mode indicated by mb_type is a predetermined block size or a predetermined mode, stds_flag and stds_idx are decoded. For example, when the block size is 64 × 64, 32 × 32, 16 × 16, or in the direct mode, stds_flag and stds_idx are decoded.

  FIG. 30B shows the syntax when the selected block information is decoded before mb_type. For example, when stds_flag is 1, mb_type does not need to be decoded. If stds_flag is 0, mb_type is decoded.

  As described above, the image decoding apparatus according to the present embodiment decodes the image encoded by the above-described image encoding apparatus according to the second embodiment. Therefore, the image decoding according to the present embodiment can reproduce a high-quality decoded image from relatively small encoded data.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine the component covering different embodiment suitably.

  As an example of this, the same effect can be obtained even if the above-described first to fourth embodiments are modified as follows.

  (1) In the first to fourth embodiments, the processing target frame is divided into rectangular blocks such as 16 × 16 pixel blocks, and the pixel blocks from the upper left pixel block to the lower right pixel block as shown in FIG. Although the case where encoding or decoding is performed in order has been described as an example, the encoding or decoding order is not limited to this example. For example, the encoding or decoding order may be an order from the lower right to the upper left of the screen or an order from the upper right to the lower left. Further, the encoding or decoding order may be a spiral order from the central part of the screen to the peripheral part, or an order from the peripheral part to the central part of the screen.

(2) In the first to fourth embodiments, the case where the luminance signal and the color difference signal are not divided but limited to one color signal component is described as an example. However, different prediction processes may be used for the luminance signal and the color difference signal, or the same prediction process may be used. When different prediction processes are used, the prediction method selected for the color difference signal is encoded / decoded in the same manner as the luminance signal.

  In addition, it goes without saying that the present invention can be similarly implemented even if various prejudices are given without departing from the gist of the present invention.

  Since the image encoding / decoding method according to the present invention can improve the encoding efficiency, it has industrial applicability.

DESCRIPTION OF SYMBOLS 10 ... Input image signal, 11 ... Predictive image signal, 12 ... Prediction error image signal, 13 ... Quantization transform coefficient, 14 ... Encoded data, 15 ... Decoded prediction error signal, 16 ... Local decoded image signal, 17 ... Reference image Signal, 18 ... motion information, 20 ... bit stream, 21 ... motion information, 25, 26 ... information frame, 30 ... usable block information, 31 ... selected block information, 32 ... prediction switching information, 33 ... transform coefficient information, 34 ... Prediction error signal, 35 ... Predictive image signal, 36 ... Decoded image signal, 37 ... Reference image signal, 38 ... Motion information, 39 ... Reference motion information, 40 ... Motion information, 50 ... Coding control information, 51 ... Feedback information , 60 ... Available block information, 61 ... Selected block information, 62 ... Prediction switching information, 70 ... Decoding control information, 71 ... Control information, 80 ... Encoded data, 100 ... Image encoding unit 101 ... Prediction unit 102 ... Subtractor 103 ... Transform / quantization unit 104 ... Variable length encoding unit 105 ... Inverse quantization / inverse transform unit 106 ... Adder 107 ... Frame memory , 108 ... Information memory, 109 ... Available block acquisition unit, 110 ... Spatial direction motion information acquisition unit, 111 ... Time direction motion information acquisition unit, 112 ... Information changeover switch, 113 ... Motion compensation unit, 114 ... Parameter encoding unit , 115 ... transform coefficient encoding unit, 116 ... selected block encoding unit, 117 ... multiplexing unit, 118 ... motion information selection unit, 120 ... output buffer, 150 ... encoding control unit, 200 ... image encoding unit, 201 ... Prediction unit, 202 ... Second prediction unit, 203 ... Prediction method changeover switch, 204 ... Variable length coding unit, 205 ... Motion information acquisition unit, 216 ... Selected block coding unit, 217 Motion information encoding unit, 300 ... image decoding unit, 301 ... encoded sequence decoding unit, 301 ... code sequence decoding unit, 302 ... inverse quantization / inverse transform unit, 303 ... adder, 304 ... frame memory, 305 ... Prediction unit, 306 ... Information memory, 307 ... Available block acquisition unit, 308 ... Output buffer, 310 ... Spatial direction motion information acquisition unit, 311 ... Time direction motion information acquisition unit, 312 ... Motion information changeover switch, 313 ... Motion Compensation unit, 314 ... information selection unit, 320 ... separation unit, 321 ... parameter decoding unit, 322 ... transform coefficient decoding unit, 323 ... selected block decoding unit, 350 ... decoding control unit, 400 ... image decoding unit , 401 ... Coding sequence decoding unit, 405 ... Prediction unit, 410 ... Second prediction unit, 411 ... Prediction method changeover switch, 423 ... Selected block decoding unit, 424 ... Information decoding unit 901 ... High level syntax 902 ... Sequence parameter set syntax 903 ... Picture parameter set syntax 904 ... Slice level syntax 905 ... Slice header syntax 906 ... Slice data syntax 907 ... Macro block level syntax 908 ... Macro block Layer syntax, 909 ... Macroblock prediction syntax.

Claims (3)

An input unit for receiving input of mode information related to the prediction mode of the target block;
A determination unit that determines whether or not a plurality of candidate blocks having a predetermined positional relationship with the target block can be used according to a predetermined order according to the positional relationship;
When the mode information indicates predetermined information and the number of candidate blocks determined to be usable is 2 or more, identification information indicating one of the candidate blocks determined to be usable is encoded. A decryption unit obtained from the data;
When the mode information indicates the predetermined information and the number of candidate blocks determined to be usable is two or more, a selected block is selected from the candidate blocks determined to be usable according to the identification information A selection section to
A prediction unit that generates a predicted image of the target block based on motion information corresponding to the selected block;
Comprising
The determination unit determines that a candidate block is usable when the candidate block has motion information and the motion information does not match motion information corresponding to a candidate block that has already been determined to be usable. Decryption device. Receiving mode information regarding the prediction mode of the target block;
Determining whether or not a plurality of candidate blocks having a predetermined positional relationship with respect to the target block can be used according to a predetermined order according to the positional relationship;
When the mode information indicates predetermined information and the number of candidate blocks determined to be usable is 2 or more, identification information indicating one of the candidate blocks determined to be usable is encoded. Retrieving from the data;
When the mode information indicates the predetermined information and the number of candidate blocks determined to be usable is two or more, a selected block is selected from the candidate blocks determined to be usable according to the identification information And steps to
Generating a predicted image of the target block based on motion information corresponding to the selected block;
Comprising
The determining step determines a candidate block that can be used when the candidate block has motion information and the motion information does not match motion information corresponding to a candidate block that has already been determined to be usable, Image decoding method. Means for receiving input of mode information related to the prediction mode of the target block;
Means for determining whether or not a plurality of candidate blocks having a predetermined positional relationship with the target block can be used according to a predetermined order according to the positional relationship;
When the mode information indicates predetermined information and the number of candidate blocks determined to be usable is 2 or more, identification information indicating one of the candidate blocks determined to be usable is encoded. Means to obtain from data,
When the mode information indicates the predetermined information and the number of candidate blocks determined to be usable is two or more, a selected block is selected from the candidate blocks determined to be usable according to the identification information And an image decoding program for causing a computer to function as a prediction unit that generates a predicted image of the target block based on motion information corresponding to the selected block,
The determining means determines that the candidate block has motion information, and the candidate block is usable when the motion information does not match the motion information corresponding to the candidate block that has already been determined to be usable, Image decoding program.