JP2024023525A - Image encoding method and image decoding method - Google Patents

Abstract

The present invention provides an image encoding and image decoding method with high encoding efficiency. An image encoding method includes a first step of selecting at least one motion reference block from encoded pixel blocks having motion information, and selecting motion information candidates to be applied to a block to be encoded. a second step of selecting at least one available pixel block having mutually different motion information from among the motion reference blocks; and selecting one selection block from the available blocks. a third step; a fourth step of generating a predicted image of the block to be encoded using the motion information of the selected block; and a fifth step of encoding a prediction error between the predicted image and the original image. and a sixth step of encoding selection information specifying the selected block with reference to a code table predetermined according to the number of available blocks. [Selection diagram] None

Description

The present invention relates to encoding and decoding methods for moving and still images.

In recent years, a video coding method that has significantly improved coding efficiency has been developed jointly by ITU-T and ISO/IEC, and has been published in ITU-T Rec. H.264 and ISO/IEC 14496-10 (hereinafter referred to as H.264). 264). In H.264, prediction processing, conversion processing, and entropy encoding processing are performed in rectangular block units (eg, 16 x 16 pixel block units, 8 x 8 pixel block units, etc.). In the prediction process, motion compensation is performed on a rectangular block to be encoded (block to be encoded) to perform temporal prediction with reference to an already encoded frame (reference frame). In such motion compensation, it is necessary to encode motion information including a motion vector as spatial shift information between a block to be encoded and a block referenced in a reference frame and send it to the decoding side. Furthermore, when performing motion compensation using a plurality of reference frames, it is necessary to encode the reference frame number along with the motion information. Therefore, the amount of code related to motion information and reference frame numbers may increase.

An example of how to obtain a motion vector in motion compensated prediction is to derive a motion vector to be assigned to a block to be encoded from motion vectors assigned to blocks that have already been encoded, and then make predictions based on the derived motion vector. There is a direct mode for generating images (see Patent Document 1 and Patent Document 2). In the direct mode, since motion vectors are not encoded, the code amount of motion information can be reduced. Direct mode is used, for example, in H.264/AVC.

Patent No. 4020789 US Patent No. 7233621

In the direct mode, a motion vector of a block to be coded is predicted and generated using a fixed method of calculating a motion vector from the median value of motion vectors of encoded blocks adjacent to the block to be coded. Therefore, the degree of freedom in calculating motion vectors is low.

In order to increase the degree of freedom in calculating motion vectors, a method has been proposed in which one is selected from a plurality of encoded blocks and a motion vector is assigned to the block to be encoded. In this method, selection information specifying the selected block must always be transmitted so that the decoding side can specify the selected encoded block. Therefore, when one of a plurality of coded blocks is selected and a motion vector to be assigned to the block to be coded is determined, there is a problem in that the amount of codes related to the selection information is increased.

The present invention was made to solve the above problems, and an object of the present invention is to provide an image encoding and image decoding method with high encoding efficiency.

An image encoding method according to an embodiment of the present invention includes a first step of selecting at least one motion reference block from encoded pixel blocks having motion information, and a motion applied to the block to be encoded. a second step of selecting at least one available pixel block having information candidates and having mutually different motion information from among the motion reference blocks; and selecting one from the available blocks. a third step of selecting a block; a fourth step of generating a predicted image of the block to be coded using the motion information of the selected block; and a step of coding a prediction error between the predicted image and the original image. and a sixth step of encoding selection information specifying the selected block with reference to a code table predetermined according to the number of available blocks.

An image decoding method according to another embodiment of the present invention includes a first step of selecting at least one motion reference block from decoded pixel blocks having motion information, and applying the method to a block to be decoded. a second step of selecting at least one available pixel block having motion information candidates and having mutually different motion information from among the motion reference blocks; a third step of obtaining selection information for identifying a selected block by decoding the input coded data with reference to a predetermined code table; a fourth step of selecting one selected block from the selected block; a fifth step of generating a predicted image of the block to be decoded using the motion information of the selected block; and a seventh step of obtaining a decoded image from the predicted image and the prediction residual.

According to the present invention, encoding efficiency can be improved.

FIG. 1 is a block diagram schematically showing the configuration of an image encoding device according to a first embodiment. FIG. 2 is a diagram illustrating an example of the size of a macroblock, which is a processing unit of encoding by the image decoding unit illustrated in FIG. 1; FIG. 2 is a diagram showing another example of the size of a macroblock, which is a processing unit of encoding by the image decoding unit shown in FIG. 1; FIG. 2 is a diagram showing the order in which the image encoding unit shown in FIG. 1 encodes pixel blocks in a frame to be encoded. 2 is a diagram showing an example of a motion information frame held by the motion information memory shown in FIG. 1. FIG. 2 is a flowchart illustrating an example of a procedure for processing the input image signal of FIG. 1. FIG. 2 is a diagram illustrating an example of inter prediction processing executed by the motion compensation unit in FIG. 1. FIG. 2 is a diagram showing another example of inter prediction processing executed by the motion compensation unit in FIG. 1. FIG. FIG. 3 is a diagram illustrating an example of the size of a motion compensation block used in inter prediction processing. FIG. 7 is a diagram illustrating another example of the size of a motion compensation block used in inter prediction processing. FIG. 7 is a diagram illustrating still another example of the size of a motion compensation block used in inter prediction processing. FIG. 7 is a diagram illustrating another example of the size of a motion compensation block used in inter prediction processing. FIG. 3 is a diagram illustrating an example of the arrangement of spatial and temporal motion reference blocks. FIG. 7 is a diagram illustrating another example of the arrangement of spatial direction motion reference blocks. 8B is a diagram showing the relative position of a spatial direction motion reference block with respect to the encoding target block shown in FIG. 8B. FIG. FIG. 7 is a diagram showing another example of the arrangement of temporal motion reference blocks. FIG. 7 is a diagram showing still another example of the arrangement of temporal motion reference blocks. FIG. 7 is a diagram showing still another example of the arrangement of temporal motion reference blocks. 2 is a flowchart illustrating an example of a method by which the available block acquisition unit of FIG. 1 selects an available block from motion reference blocks. 9 is a diagram showing an example of available blocks selected from among the motion reference blocks shown in FIG. 8 according to the method of FIG. 9; FIG. FIG. 2 is a diagram illustrating an example of available block information output by the available block acquisition unit of FIG. 1. FIG. FIG. 2 is a diagram illustrating an example of determination of the identity of motion information between blocks by the available block acquisition unit of FIG. 1; FIG. 3 is a diagram illustrating another example of determining the identity of motion information between blocks by the available block acquisition unit of FIG. 1; FIG. 3 is a diagram illustrating still another example of determining the identity of motion information between blocks by the available block acquisition unit of FIG. 1; FIG. 3 is a diagram illustrating another example of determining the identity of motion information between blocks by the available block acquisition unit of FIG. 1; FIG. 3 is a diagram illustrating still another example of determining the identity of motion information between blocks by the available block acquisition unit of FIG. 1; FIG. 3 is a diagram illustrating another example of determining the identity of motion information between blocks by the available block acquisition unit of FIG. 1; 2 is a block diagram schematically showing the configuration of a prediction unit in FIG. 1. FIG. 14 is a diagram showing a group of motion information output by the temporal motion information acquisition unit of FIG. 13. FIG. FIG. 14 is an explanatory diagram illustrating interpolation processing with small-pixel accuracy that can be used in motion compensation processing by the motion compensation unit in FIG. 13; 14 is a flowchart illustrating an example of the operation of the prediction unit in FIG. 13. 14 is a diagram illustrating how the motion compensation unit in FIG. 13 copies motion information of a temporal motion reference block to a block to be encoded. FIG. FIG. 2 is a block diagram schematically showing the configuration of a variable length encoder in FIG. 1. FIG. FIG. 6 is a diagram illustrating an example of generating syntax according to available block information. FIG. 6 is a diagram illustrating an example of binarization of selected block information syntax corresponding to available block information. FIG. 3 is an explanatory diagram illustrating scaling of motion information. FIG. 3 is a diagram illustrating a syntax structure according to an embodiment. FIG. 3 is a diagram illustrating an example of a macroblock layer syntax according to the first embodiment. FIG. 7 is a diagram illustrating another example of macroblock layer syntax according to the first embodiment. H. 3 is a diagram showing mb_type and a code table corresponding to mb_type at the time of B slice in H.264. FIG. FIG. 3 is a diagram showing an example of a code table according to the embodiment. H. FIG. 2 is a diagram showing mb_type and a code table corresponding to mb_type at the time of P slice in H.264. FIG. 7 is a diagram showing another example of the code table according to the embodiment. FIG. 6 is a diagram illustrating an example of mb_type and a code table corresponding to mb_type in a B slice according to an embodiment. FIG. 7 is a diagram illustrating another example of mb_type in a P slice and a code table corresponding to mb_type according to the embodiment. FIG. 2 is a block diagram schematically showing the configuration of an image encoding device according to a second embodiment. 27 is a block diagram schematically showing the configuration of the prediction unit in FIG. 26. FIG. 28 is a block diagram schematically showing the configuration of a second prediction unit in FIG. 27. FIG. FIG. 27 is a block diagram schematically showing the configuration of a variable length encoding section in FIG. 26. FIG. FIG. 7 is a diagram illustrating an example of macroblock layer syntax according to the second embodiment. FIG. 7 is a diagram illustrating another example of macroblock layer syntax according to the second embodiment. FIG. 3 is a block diagram schematically showing an image decoding device according to a third embodiment. FIG. 32 is a block diagram showing the encoded sequence decoding section shown in FIG. 31 in more detail. FIG. 32 is a block diagram showing the prediction unit shown in FIG. 31 in more detail. FIG. 3 is a block diagram schematically showing an image decoding device according to a fourth embodiment. FIG. 34 is a block diagram showing the encoded sequence decoding section shown in FIG. 33 in more detail. FIG. 34 is a block diagram showing the prediction unit shown in FIG. 33 in more detail.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an image encoding and image decoding method and apparatus according to an embodiment of the present invention will be described with reference to the drawings as necessary. Note that in the following embodiments, portions with the same numbers perform similar operations, and redundant explanation will be omitted.

(First embodiment)
FIG. 1 schematically shows the configuration of an image encoding device according to a first embodiment of the present invention. This image encoding device includes an image encoding section 100, an encoding control section 150, and an output buffer 120, as shown in FIG. This image encoding device may be realized by hardware such as an LSI chip, or may be realized by causing a computer to execute an image encoding program.

An original image (input image signal) 10, which is a moving image or a still image, is input to the image encoding unit 100, for example, in units of pixel blocks obtained by dividing the original image. The image encoding unit 100 compresses and encodes the input image signal 10 to generate encoded data 14, as will be described in detail later. The generated encoded data 14 is temporarily stored in the output buffer 120, and is sent to a storage system (storage medium) or a transmission system (communication line) (not shown) at an output timing managed by the encoding control unit 150. .

The encoding control unit 150 controls overall encoding processing of the image encoding unit 100, such as feedback control of the amount of generated code, quantization control, prediction mode control, and entropy encoding control. Specifically, the encoding control unit 150 provides encoding control information 50 to the image encoding unit 100 and receives feedback information 51 from the image encoding unit 100 as appropriate. The encoding control information 50 includes prediction information, motion information 18, quantization parameter information, and the like. Prediction information includes prediction mode information and block size information. The motion information 18 includes a motion vector, a reference frame number, and a prediction direction (unidirectional prediction, bidirectional prediction). The quantization parameter information includes a quantization parameter such as a quantization width (quantization step size) and a quantization matrix. The feedback information 51 includes the amount of code generated by the image encoding unit 100, and is used, for example, to determine a quantization parameter.

The image encoding unit 100 encodes the input image signal 10 in units of pixel blocks (for example, macroblocks, subblocks, one pixel, etc.) obtained by dividing the original image. Therefore, the input image signal 10 is sequentially input to the image encoding unit 100 in units of pixel blocks obtained by dividing the original image. In this embodiment, the encoding processing unit is a macroblock, and the pixel block (macroblock) that is the encoding target and corresponds to the input image signal 10 is simply referred to as the encoding target block. Further, an image frame including a block to be encoded, that is, an image frame to be encoded is referred to as a frame to be encoded.

Such a block to be encoded may be, for example, a 16×16 pixel block as shown in FIG. 2A, or a 64×64 pixel block as shown in FIG. 2B. Further, the encoding target block may be a 32x32 pixel block, an 8x8 pixel block, or the like. Further, the shape of the macroblock is not limited to the square example shown in FIGS. 2A and 2B, but may be set to any shape such as a rectangle. Further, the processing unit is not limited to a pixel block such as a macroblock, but may be a frame or a field.

Note that the encoding process for each pixel block within the encoding target frame may be performed in any order. In this embodiment, in order to simplify the explanation, as shown in FIG. It is assumed that encoding processing is performed on .

The image encoding unit 100 shown in FIG. , a motion information memory 108 and an available block acquisition unit 109.

In the image encoding unit 100, the input image signal 10 is input to the prediction unit 101 and the subtracter 102. Subtractor 102 receives input image signal 10 and also receives predicted image signal 11 from prediction unit 101, which will be described later. The subtracter 102 calculates the difference between the input image signal 10 and the predicted image signal 11 to generate a predicted error image signal 12.

The conversion/quantization unit 103 receives the prediction error image signal 12 from the subtracter 102, performs a conversion process on the received prediction error image signal 12, and generates conversion coefficients. The transformation process is, for example, orthogonal transformation such as discrete cosine transform (DCT). In other embodiments, the transform/quantization unit 103 may generate transform coefficients using techniques such as wavelet transform and independent component analysis instead of discrete cosine transform. Further, the transformation/quantization unit 103 quantizes the generated transformation coefficients based on the quantization parameter given by the encoding control unit 150. The quantized transform coefficients (transform coefficient information) 13 are output to variable length encoding section 104 and inverse quantization/inverse transform section 105 .

The dequantization/inverse transform unit 105 dequantizes the quantized transform coefficients 13 according to the quantization parameter given by the encoding control unit 150, that is, the same quantization parameter as the transform/quantization unit 103. . Subsequently, the inverse quantization/inverse transform unit 105 performs inverse transform on the dequantized transform coefficients to generate a decoded prediction error signal 15. The inverse transformation process by the inverse quantization/inverse transformation unit 105 corresponds to the inverse transformation process of the transformation process by the transformation/quantization unit 103. For example, the inverse transform process is an inverse discrete cosine transform (IDCT), an inverse wavelet transform, or the like.

Adder 106 receives decoded prediction error signal 15 from inverse quantization/inverse transform section 105 and further receives predicted image signal 11 from prediction section 101 . Adder 106 adds decoded prediction error signal 15 and predicted image signal 11 to generate locally decoded image signal 16. The generated locally decoded image signal 16 is stored in the frame memory 107 as a reference image signal 17. The reference image signal 17 stored in the frame memory 107 is read out and referenced by the prediction unit 101 when encoding a subsequent block to be encoded.

The prediction unit 101 receives the reference image signal 17 from the frame memory 107, and also receives available block information 30 from the available block acquisition unit 109, which will be described later. Further, the prediction unit 101 receives reference motion information 19 from a motion information memory 108, which will be described later. The prediction unit 101 generates a predicted image signal 11, motion information 18, and selected block information 31 of the block to be encoded based on the reference image signal 17, reference motion information 19, and available block information 30. Specifically, the prediction unit 101 includes a motion information selection unit 118 that generates motion information 18 and selected block information 31 based on the available block information 30 and reference motion information 19; A motion compensation unit 113 that generates a predicted image signal 11 is provided. Predicted image signal 11 is sent to subtracter 102 and adder 106. The motion information 18 is stored in the motion information memory 108 for subsequent prediction processing on the block to be encoded. Further, the selected block information 31 is sent to the variable length encoding section 104. The prediction unit 101 will be explained in detail later.

The motion information 18 is temporarily stored in the motion information memory 108 as reference motion information 19. FIG. 4 shows an example of the configuration of the motion information memory 108. As shown in FIG. 4, the motion information memory 108 holds reference motion information 19 in units of frames, and the reference motion information 19 forms a motion information frame 25. Motion information 18 regarding encoded blocks is sequentially input to the motion information memory 108, and as a result, the motion information memory 108 holds a plurality of motion information frames 25 having different encoding times.

The reference motion information 19 is held in the motion information frame 25 in predetermined block units (for example, 4×4 pixel block units). The motion vector block 28 shown in FIG. 4 is a pixel block of the same size as the encoding target block, the available block, the selected block, etc., and is, for example, a 16×16 pixel block. A motion vector is assigned to the motion vector block 28, for example, for each 4×4 pixel block. Inter prediction processing using motion vector blocks is referred to as motion vector block prediction processing. The reference motion information 19 held in the motion information memory 108 is read by the prediction unit 101 when generating the motion information 18. Motion information 18 possessed by an available block as described later refers to reference motion information 19 held in an area in the motion information memory 108 where the available block is located.

Note that the motion information memory 108 is not limited to the example in which the reference motion information 19 is held in units of 4×4 pixel blocks, and may be held in units of other pixel blocks. For example, the pixel block unit regarding the reference motion information 19 may be one pixel or may be a 2×2 pixel block. Furthermore, the shape of the pixel block related to the reference motion information 19 is not limited to the square example, but can be any shape.

The available block acquisition unit 109 in FIG. 1 acquires reference motion information 19 from the motion information memory 108, and based on the acquired reference motion information 19, the prediction unit 109 selects blocks that have already been encoded. Select available blocks that can be used for prediction processing. The selected available blocks are sent as available block information 30 to the prediction unit 101 and the variable length encoding unit 104. Encoded blocks that are candidates for selecting available blocks are referred to as motion reference blocks. A method for selecting motion reference blocks and available blocks will be described in detail later.

In addition to the transform coefficient information 13, the variable length encoding unit 104 receives selected block information 31 from the prediction unit 101, prediction information and encoding parameters such as quantization parameters from the encoding control unit 150, and receives the encoding parameters such as prediction information and quantization parameters from the available block acquisition unit. The available block information 30 is received from 109. The variable length encoding unit 104 performs entropy encoding (for example, equal length encoding, Huffman encoding, arithmetic encoding, etc.) on the quantized transform coefficients 13, selected block information 31, available block information 30, and encoding parameters. ) to generate encoded data 14. The encoding parameters include selected block information 31 and prediction information, as well as all parameters necessary for decoding, such as information regarding transform coefficients and information regarding quantization. The generated encoded data 14 is temporarily stored in an output buffer 120 and sent to a storage system or a transmission system (not shown).

FIG. 5 shows a processing procedure for the input image signal 10. As shown in FIG. 5, first, the predicted image signal 11 is generated by the prediction unit 101 (step S501). In the generation of the predicted image signal 11 in step S501, one of the available blocks to be described later is selected as a selected block, and the predicted image signal is A signal 11 is created. The difference between the predicted image signal 11 and the input image signal 10 is calculated by the subtracter 102, and a predicted error image signal 12 is generated (step S502).

Subsequently, the prediction error image signal 12 is subjected to orthogonal transformation and quantization by the transformation/quantization unit 103 to generate transformation coefficient information 13 (step S503). The transform coefficient information 13 and the selected block information 31 are sent to the variable length encoding unit 104, where they are subjected to variable length encoding to generate encoded data 14 (step S504). Further, in step S504, the code table is switched according to the selected block information 31 so that the code table has the same number of entries as the number of available blocks, and the selected block information 31 is variable-length coded. The encoded data bit stream 20 is sent to a storage system or transmission path (not shown).

The transform coefficient information 13 generated in step S503 is dequantized by the dequantization/inverse transform unit 105 and subjected to an inverse transform process to become a decoded prediction error signal 15 (step S505). The decoded prediction error signal 15 is added to the reference image signal 17 used in step S501 to become the locally decoded image signal 16 (step S506), and is stored in the frame memory 107 as a reference image signal (step S507).

Next, each configuration of the image encoding section 100 described above will be explained in more detail.
The image encoding unit 100 in FIG. 1 is provided with a plurality of prediction modes, and each prediction mode differs in the generation method of the predicted image signal 11 and the motion compensation block size. Specifically, the method by which the prediction unit 101 generates the predicted image signal 11 can be broadly divided into intra prediction (intra-frame prediction) in which a predicted image is generated using the reference image signal 17 regarding the frame (or field) to be encoded. prediction) and inter prediction (interframe prediction) in which a predicted image is generated using a reference image signal 17 regarding one or more encoded reference frames (reference fields). The prediction unit 101 selectively switches between intra prediction and inter prediction to generate a predicted image signal 11 of the current block to be encoded.

FIG. 6A shows an example of inter prediction by the motion compensation unit 113. In inter prediction, as shown in FIG. 6A, a block (also referred to as a prediction block) 23, which is a block in the previous reference frame that has already been encoded and is located at the same position as the block to be encoded, is used. , the predicted image signal 11 is generated using the reference image signal 17 regarding the block 24 at a position spatially shifted according to the motion vector 18a included in the motion information 18. That is, in generating the predicted image signal 11, the reference image signal 17 regarding the block 24 in the reference frame, which is specified by the position (coordinates) of the block to be encoded and the motion vector 18a included in the motion information 18, is used. . In inter prediction, motion compensation with small pixel precision (for example, 1/2 pixel precision or 1/4 pixel precision) is possible, and interpolated pixel values are generated by filtering the reference image signal 17. be done. For example, H. In H.264, interpolation processing of up to 1/4 pixel accuracy is possible for luminance signals. When performing motion compensation with 1/4 pixel precision, the amount of information of the motion information 18 is four times the integer pixel precision.

Note that inter prediction is not limited to the example of using the reference frame one frame before as shown in FIG. 6A, and any encoded reference frame may be used as shown in FIG. 6B. . When reference image signals 17 relating to a plurality of reference frames at different temporal positions are held, information indicating from which temporal position the reference image signal 17 is used to generate the predicted image signal 11 is represented by a reference frame number. The reference frame number is included in the motion information 18. The reference frame number can be changed in area units (picture, block units, etc.). That is, different reference frames can be used for each pixel block. As an example, if the coded reference frame one frame before is used for prediction, the reference frame number of this area is set to 0, and if the coded reference frame two frames before is used for prediction, The reference frame number for this area is set to 1. As another example, when only one frame worth of reference image signal 17 is held in frame memory 107 (the number of reference frames is 1), the reference frame number is always set to 0.

Furthermore, in inter prediction, a block size suitable for a block to be coded can be selected from among a plurality of motion compensation blocks. That is, the encoding target block may be divided into a plurality of small pixel blocks, and motion compensation may be performed for each small pixel block. 7A to 7C show the size of the motion compensation block in units of macroblocks, and FIG. 7D shows the size of the motion compensation block in units of subblocks (pixel blocks of 8×8 pixels or less). As shown in FIG. 7A, when the encoding target block is 64 x 64 pixels, the motion compensation block may be a 64 x 64 pixel block, a 64 x 32 pixel block, a 32 x 64 pixel block, or a 32 x 32 pixel block. can be selected. Further, as shown in FIG. 7B, when the encoding target block is 32 x 32 pixels, the motion compensation block is a 32 x 32 pixel block, a 32 x 16 pixel block, a 16 x 32 pixel block, or a 16 x 16 pixel block. Blocks etc. can be selected. Furthermore, as shown in FIG. 7C, when the encoding target block is 16 x 16 pixels, the motion compensation block is a 16 x 16 pixel block, a 16 x 8 pixel block, an 8 x 16 pixel block, or an 8 x 8 pixel block. It can be set as a block, etc. Furthermore, as shown in FIG. 7D, when the encoding target block is 8x8 pixels, the motion compensation block is an 8x8 pixel block, an 8x4 pixel block, a 4x8 pixel block, or a 4x4 pixel block. Pixel blocks, etc. can be selected.

As mentioned above, since the small pixel block (for example, 4x4 pixel block) in the reference frame used for inter prediction has motion information 18, the optimal The shape and motion vector of the motion compensation block can be used. Furthermore, the macroblocks and submacroblocks in FIGS. 7A to 7D can be combined arbitrarily. If the block to be encoded is a 64 x 64 pixel block as shown in Fig. 7A, each block size shown in Fig. 7B is set for each of the four 32 x 32 pixel blocks obtained by dividing the 64 x 64 pixel block. By selecting, blocks of 64×64 to 16×16 pixels can be used hierarchically. Similarly, if it is possible to select up to the block size shown in FIG. 7D, block sizes from 64×64 to 4×4 can be used hierarchically.

Next, motion reference blocks will be described with reference to FIGS. 8A to 8F.
The motion reference block is selected from encoded areas (blocks) in the encoding target frame and the reference frame according to a method agreed upon by both the image encoding device in FIG. 1 and the image decoding device described later. Ru. FIG. 8A shows an example of the arrangement of motion reference blocks selected according to the position of the encoding target block. In the example of FIG. 8A, nine motion reference blocks A to D and TA to TE are selected from encoded regions in the current frame and the reference frame. Specifically, from the encoding target frame, four blocks A, B, C, and D adjacent to the left, above, upper right, and upper left of the encoding target block are selected as motion reference blocks, and from the reference frame, A block TA at the same position as the block to be encoded and four pixel blocks TB, TC, TD, and TE adjacent to the right, bottom, left, and top of this block TA are selected as motion reference blocks. In this embodiment, the motion reference block selected from the encoding target frame is referred to as a spatial motion reference block, and the motion reference block selected from the reference frame is referred to as a temporal motion reference block. The symbol p given to each motion reference block in FIG. 8A indicates the index of the motion reference block. The indexes are numbered in the order of the motion reference blocks in the temporal direction and the spatial direction, but the order is not limited to this and does not necessarily have to be in this order as long as the indexes do not overlap. For example, the motion reference blocks in the temporal and spatial directions may be numbered out of order.

Note that the spatial direction motion reference block is not limited to the example shown in FIG. 8A, but as shown in FIG. macroblock, etc.). In this case, the relative positions (dx, dy) from the upper left pixel e to each pixel a, b, c, d in the encoding target block are set as shown in FIG. 8C. Here, in the examples shown in FIGS. 8A and 8B, the macroblock is shown as being an N×N pixel block.

Further, as shown in FIG. 8D, all blocks A1 to A4, B1, B2, C, and D adjacent to the current block to be encoded may be selected as spatial motion reference blocks. In the example of FIG. 8D, the number of spatial motion reference blocks is eight.

Further, in the temporal motion reference blocks, as shown in FIG. 8E, the blocks TA to TE may partially overlap, or as shown in FIG. 8F, the blocks TA to TE may be arranged apart from each other. It doesn't matter if you stay there. In FIG. 8E, the portion where the temporal motion reference blocks TA and TB overlap is indicated by diagonal lines. Furthermore, the temporal motion reference block is not limited to the block at the position corresponding to the block to be encoded (Collocate position) and the blocks located around it, but may be located at any position within the reference frame. It doesn't matter if it's a block with For example, if a block in the reference frame, which is specified by the position of the reference block and the motion information 18 of any encoded block adjacent to the block to be encoded, is set as the center block (for example, block TA), The center block and the blocks around it may be selected as the temporal motion reference blocks. Furthermore, the temporal reference blocks do not have to be arranged at equal intervals from the center block.

In any of the above cases, if the number and position of motion reference blocks in the spatial and temporal directions are determined in advance by the encoding device and the decoding device, the number and position of the motion reference blocks can be set in any way. I don't mind. Furthermore, the size of the motion reference block does not necessarily have to be the same size as the current block to be encoded. For example, as shown in FIG. 8D, the size of the motion reference block may be larger or smaller than the encoding target block. Further, the motion reference block is not limited to a square shape, and may be set to any shape such as a rectangular shape. Further, the motion reference block may be set to any size.

Further, the motion reference block and the available block may be arranged only in either the temporal direction or the spatial direction. Further, motion reference blocks and available blocks in the temporal direction may be arranged according to slice types such as P slices and B slices, or motion reference blocks and available blocks in the spatial direction may be arranged.

FIG. 9 shows a method in which the available block acquisition unit 109 selects available blocks from among the motion reference blocks. The available blocks are blocks to which motion information can be applied to the encoding target block, and have mutually different motion information. The available block acquisition unit 109 refers to the reference motion information 19, determines whether each motion reference block is an available block according to the method shown in FIG. 9, and outputs the available block information 30.

As shown in FIG. 9, first, a motion reference block whose index p is zero is selected (S800). In the explanation of FIG. 9, it is assumed that motion reference blocks are processed in order from index p 0 to M-1 (M indicates the number of motion reference blocks). Furthermore, the description will be made assuming that the usability determination process for motion reference blocks with index p from 0 to p-1 has been completed, and that the index of the motion reference block whose usability is to be determined is p. .

The available block acquisition unit 109 determines whether the motion reference block p has motion information 18, that is, whether at least one motion vector is assigned (S801). If the motion reference block p does not have a motion vector, that is, the temporal motion reference block p is a block within an I slice that does not have motion information, or If all small pixel blocks have been intra-predictively encoded, the process advances to step S805. In step S805, the motion reference block p is determined to be an unavailable block.

If the motion reference block p has motion information in step S801, the process advances to step S802. The available block acquisition unit 109 selects a motion reference block q (available block q) that has already been selected as an available block. Here, q is a value smaller than p. Next, the available block acquisition unit 109 compares the motion information 18 of the motion reference block p and the motion information 18 of the available block q, and determines whether they have the same motion information (S803). . If it is determined that the motion information 18 of the motion reference block p and the motion information 18 of the motion reference block q selected as an available block are the same, the process advances to step S805, and the motion reference block p is an unavailable block. It is determined that

If it is determined in step S803 that the motion information 18 of the motion reference block p and the motion information 18 of the available block q are not the same for all available blocks q satisfying q<p, the process advances to step S804. . In step S804, the available block acquisition unit 109 determines the motion reference block p as an available block.

When it is determined that the motion reference block p is an available block or an unusable block, the available block acquisition unit 109 determines whether the availability determination has been performed for all motion reference blocks ( S806). If there is a motion reference block for which the availability determination has not been performed, for example, if p<M-1, the process advances to step S807. Subsequently, the available block acquisition unit 109 increments the index p by 1 (step S807), and executes steps S801 to S806 again. When the usability determination is executed for all motion reference blocks in step S806, the usability determination process ends.

By executing the availability determination process described above, it is determined whether each motion reference block is an available block or an unavailable block. The available block acquisition unit 109 generates available block information 30 including information regarding available blocks. In this way, by selecting an available block from among the motion reference blocks, the amount of information regarding the available block information 30 is reduced, and as a result, the amount of encoded data 14 can be reduced.

FIG. 10 shows an example of the result of performing the usability determination process on the motion reference block shown in FIG. 8A. In FIG. 10, two spatial motion reference blocks (p=0, 1) and two temporal motion reference blocks (p=5, 8) are determined to be available blocks. An example of the available block information 30 regarding the example of FIG. 10 is shown in FIG. As shown in FIG. 11, the available block information 30 includes motion reference block index, availability, and motion reference block name. In the example of FIG. 11, indexes p=0, 1, 5, and 8 are available blocks, and the number of available blocks is 4. The prediction unit 101 selects an optimal available block from among these available blocks as a selected block, and outputs information 31 regarding the selected block (selected block information). The selected block information 31 includes the number of available blocks and the index value of the selected available block. For example, when the number of available blocks is 4, the corresponding selected block information 31 is encoded by the variable length encoding unit 104 using a code table with a maximum of 4 entries.

Note that in step S801 in FIG. 9, if at least one of the blocks in the temporal motion reference block p is an intra-prediction encoded block, the available block acquisition unit 109 uses the motion reference block p. It does not matter if it is determined to be an impossible block. That is, the process may proceed to step S802 only when all blocks in the temporal motion reference block p have been encoded by inter prediction.

12A to 12E show an example in which the motion information 18 of the motion reference block p and the motion information 18 of the available block q are determined to be the same in the comparison of the motion information 18 in step S803. 12A to 12E each show a plurality of blocks shaded with diagonal lines and two blocks painted white. In FIGS. 12A to 12E, in order to simplify the explanation, it is assumed that the motion information 18 of these two white blocks is compared without considering the shaded block. It is assumed that one of the two white blocks is a motion reference block p, and the other is a motion reference block q (available block q) that has already been determined to be available. Unless otherwise specified, it does not matter which of the two white blocks may be the motion reference block p.

FIG. 12A shows an example where both the motion reference block p and the available block q are spatial blocks. In the example of FIG. 12A, if the motion information 18 of blocks A and B are the same, it is determined that the motion information 18 is the same. At this time, the sizes of blocks A and B do not need to be the same.

FIG. 12B shows an example in which one of the motion reference block p and the available block q is a block A in the spatial direction, and the other is a block TB in the temporal direction. In FIG. 12B, there is one block having motion information in the block TB in the time direction. If the motion information 18 of the block TB in the temporal direction and the motion information 18 of the block A in the spatial direction are the same, it is determined that the motion information 18 is the same. At this time, the sizes of blocks A and TB do not need to be the same.

FIG. 12C shows another example in which one of the motion reference block p and the available block q is a block A in the spatial direction, and the other is a block TB in the temporal direction. FIG. 12C shows a case where the block TB in the time direction is divided into a plurality of small blocks, and there are a plurality of small blocks having motion information 18. In the example of FIG. 12C, if all blocks having motion information 18 have the same motion information 18 and the motion information 18 is the same as the motion information 18 of block A in the spatial direction, then the motion information 18 is the same. It is determined that At this time, the sizes of blocks A and TB do not need to be the same.

FIG. 12D shows an example in which the motion reference block p and the available block q are both blocks in the time direction. In this case, if the motion information 18 of blocks TB and TE is the same, it is determined that the motion information 18 is the same.

FIG. 12E shows another example in which the motion reference block p and the available block q are both blocks in the time direction. FIG. 12E shows a case where blocks TB and TE in the time direction are each divided into a plurality of small blocks, and there are a plurality of small blocks each having motion information 18. In this case, the motion information 18 is compared for each small block within the block, and if the motion information 18 is the same for all small blocks, the motion information 18 of the block TB and the motion information 18 of the block TE are the same. It is determined that there is.

FIG. 12F shows yet another example in which the motion reference block p and the available block q are both blocks in the time direction. FIG. 12F shows a case where the block TE in the time direction is divided into a plurality of small blocks, and the block TE includes a plurality of small blocks having motion information 18. If all the motion information 18 of the block TE is the same motion information 18 and the same as the motion information 18 of the block TD, it is determined that the motion information 18 of the blocks TD and TE are the same.

In this way, in step S803, it is determined whether the motion information 18 of the motion reference block p and the motion information 18 of the available block q are the same. In the examples of FIGS. 12A to 12F, the number of available blocks q to be compared with the motion reference block p is 1. However, in the case where the number of available blocks q is 2 or more, the motion information of the motion reference block p 18 may be compared with the motion information 18 of each available block q. Furthermore, when scaling described later is applied, the motion information 18 after scaling becomes the motion information 18 described above.

Note that the determination that the motion information of the motion reference block p and the motion information of the available block q are the same is not limited to the case where each motion vector included in the motion information completely matches. For example, if the norm of the difference between two motion vectors is within a predetermined range, the motion information of the motion reference block p and the motion information of the available block q may be considered to be substantially the same.

FIG. 13 shows a more detailed configuration of the prediction unit 101. As described above, the prediction unit 101 receives the available block information 30, reference motion information 19, and reference image signal 17 as input, and outputs the predicted image signal 11, motion information 18, and selected block information 31. The motion information selection section 118 includes a spatial motion information acquisition section 110, a temporal motion information acquisition section 111, and a motion information changeover switch 112, as shown in FIG.

The available block information 30 and the reference motion information 19 regarding the spatial motion reference block are input to the spatial motion information acquisition unit 110 . The spatial direction motion information acquisition unit 110 outputs motion information 18A including motion information of each available block located in the spatial direction and an index value of the available block. When the information shown in FIG. 11 is input as the available block information 30, the spatial direction motion information acquisition unit 110 generates two motion information outputs 18A, and each motion information output 18A includes information about available blocks and their usage. Contains motion information 19 possessed by possible blocks.

The available block information 30 and the reference motion information 19 regarding the temporal motion reference block are input to the temporal motion information acquisition unit 111 . The temporal motion information acquisition unit 111 outputs the motion information 19 of the available temporal motion reference block specified by the available block information 30 and the index value of the available block as motion information 18B. The temporal motion reference block is divided into a plurality of small pixel blocks, and each small pixel block has motion information 19. As shown in FIG. 14, the motion information 18B output by the temporal motion information acquisition unit 111 includes a group of motion information 19 possessed by each small pixel block within the available block. When the motion information 18B includes a group of motion information 19, motion compensation prediction can be performed on the current block to be encoded in units of small pixel blocks obtained by dividing the current block to be encoded. When the information shown in FIG. 11 is input as the available block information 30, the temporal motion information acquisition unit 111 generates two motion information outputs 18B, and each motion information output includes an available block and this available block information. Contains a group of motion information 19 that the block has.

Note that the temporal motion information acquisition unit 111 calculates the average value or representative value of the motion vectors included in the motion information 19 that each pixel small block has, and outputs the average value or representative value of the motion vectors as the motion information 18B. I don't mind.

The motion information changeover switch 112 in FIG. 13 selects one appropriate available block as the selected block based on the motion information 18A and 18B output from the spatial motion information acquisition section 110 and the temporal motion information acquisition section 111. Then, motion information 18 (or a group of motion information 18) corresponding to the selected block is output to the motion compensation unit 113. Further, the motion information changeover switch 112 outputs selected block information 31 regarding the selected block. The selected block information 31 includes the index p or the name of the motion reference block, and is also simply referred to as selection information. The selected block information 31 is not limited to the index p and the name of the motion reference block, and may be any information as long as it can specify the position of the selected block.

The motion information changeover switch 112 selects, for example, an available block whose encoding cost, which is derived from the cost formula shown in Equation 1 below, is the minimum as the selected block.

Here, J represents the encoding cost, and D represents the encoding distortion representing the sum of squared errors between the input image signal 10 and the reference image signal 17. Further, R indicates the code amount estimated by temporary encoding, and λ is. It shows the Lagrangian undetermined coefficient determined by the quantization width, etc. Instead of formula 1, the encoding cost J may be calculated using only the code amount R or encoding distortion D, or by using a value that approximates the code amount R or encoding distortion D. You may also create a cost function. Furthermore, the encoding distortion D is not limited to the sum of square errors, but may be the sum of absolute differences (SAD) of prediction errors. As the code amount R, only the code amount related to the motion information 18 may be used. Furthermore, the selection block is not limited to the example in which the available block with the minimum encoding cost is selected as the selected block, but one available block having a value within a certain range that is greater than or equal to the minimum encoding cost is selected as the selected block. It doesn't matter if it is selected.

The motion compensation unit 113 derives the position of the pixel block from which the reference image signal 17 is extracted as the predicted image signal 11 based on the motion information (or group of motion information) possessed by the selected block selected by the motion information selection unit 118. do. When a group of motion information is input to the motion compensation unit 113, the motion compensation unit 113 divides the pixel block from which the reference image signal 17 is extracted as the predicted image signal 11 into small pixel blocks (for example, 4×4 pixel blocks). Then, the predicted image signal 11 is obtained from the reference image signal 17 by applying the corresponding motion information 18 to each of these small pixel blocks. The position of the block from which the predicted image signal 11 is acquired is, for example, a position shifted in the spatial direction from the small pixel block according to the motion vector 18a included in the motion information 18, as shown in FIG. 4A.

Motion compensation processing for the current block to be encoded is performed using H. A motion compensation process similar to that of H.264 can be used. Here, as an example, a 1/4 pixel precision interpolation method will be specifically described. For quarter-pixel precision interpolation, if each component of the motion vector is a multiple of four, the motion vector points to an integer pixel location. Otherwise, the motion vector points to a predicted position that corresponds to an interpolated position with fractional precision.

Here, x and y indicate vertical and horizontal indexes indicating the starting position (for example, the upper left vertex) of the prediction target block, and x_pos and y_pos indicate the corresponding predicted position of the reference image signal 17. (mv_x, mv_y) indicates a motion vector with 1/4 pixel precision. Next, a predicted pixel is generated by supplementing or interpolating the corresponding pixel position of the reference image signal 17 with respect to the determined pixel position. In FIG. 15, H. An example of H.264 predicted pixel generation is shown. In FIG. 15, squares indicated by capital letters (shaded squares) indicate pixels at integer positions, and squares indicated by shading indicate interpolated pixels at 1/2 pixel position. There is. Moreover, the square displayed in white indicates the interpolation pixel corresponding to the 1/4 pixel position. For example, in FIG. 15, the interpolation process for 1/2 pixels corresponding to the positions of alphabets b and h is calculated using Equation 3 below.

Here, the alphabets (for example, b, h, C1, etc.) shown in Equation 3 and Equation 4 below indicate the pixel values of pixels assigned the same alphabet in FIG. Further, ">>" indicates a right shift operation, and ">> 5" corresponds to division by 32. That is, the interpolation pixel at the 1/2 pixel position is calculated using a 6-tap FIR (Finite Impulse Response) filter (tap coefficients: (1, -5, 20, 20, -5, 1)/32).

Further, the interpolation process for 1/4 pixels corresponding to the positions of alphabets a and d in FIG. 15 is calculated using the following formula 4.

In this way, the interpolated pixel at the 1/4 pixel position is calculated using a 2-tap average value filter (tap coefficients: (1/2, 1/2)). The interpolation process for the 1/2 pixel corresponding to the alphabet j existing in the middle of the four integer pixel positions is generated using both 6 taps in the vertical direction and 6 taps in the horizontal direction. Interpolated pixel values are generated for pixel positions other than those described in the same manner.

Note that the interpolation process is not limited to the examples of Equations 3 and 4, and may be generated using other interpolation coefficients. Further, the interpolation coefficient may be a fixed value given from the encoding control unit 150, or the interpolation coefficient may be optimized for each frame based on the aforementioned encoding cost, and the optimized interpolation coefficient may be used. It may be generated using

In addition, in this embodiment, the motion reference block is a process related to motion vector block prediction processing in units of macroblocks (for example, 16 x 16 pixel blocks), but is not limited to macroblocks, but in units of 16 x 8 pixel blocks, The prediction process may be performed in units of 8×16 pixel blocks, in units of 8×8 pixel blocks, in units of 8×4 pixel blocks, in units of 4×8 pixel blocks, or in units of 4×4 pixel blocks. In this case, information regarding motion vector blocks is derived for each pixel block. Further, the above prediction process may be performed in units larger than 16 x 16 pixel blocks, such as in units of 32 x 32 pixel blocks, units of 32 x 16 pixel blocks, and units of 64 x 64 pixel blocks.

When substituting a reference motion vector in a motion vector block as a motion vector of a small pixel block in a block to be encoded, (A) a negative value (inverted vector) of the reference motion vector may be substituted, or (B) A reference motion vector corresponding to a small block, a weighted average value or a median value, a maximum value, and a minimum value using reference motion vectors adjacent to this reference motion vector may be substituted.

FIG. 16 schematically shows the operation of the prediction unit 101. As shown in FIG. 16, first, a reference frame (motion reference frame) including a temporal reference motion block is obtained (step S1501). The motion reference frame is typically a reference frame having the smallest temporal distance from the encoding target frame, and is a reference frame in the temporal past. For example, the motion reference frame is a frame encoded immediately before the frame to be encoded. In other examples, any reference frame whose motion information 18 is stored in the motion information memory 108 may be acquired as the motion reference frame. Next, the spatial direction motion information acquisition unit 110 and the temporal direction motion information acquisition unit 111 each acquire the available block information 30 output from the available block acquisition unit 109 (step S1502). Next, the motion information changeover switch 112 selects one of the available blocks as a selection block, for example, according to Equation 1 (step S1503). Subsequently, the motion compensation unit 113 copies the motion information of the selected block to the encoding target block (step S1504). At this time, if the selected block is a spatial reference block, as shown in FIG. 17, the motion information 18 of this selected block is copied to the encoded reference block. Further, when the selected block is a temporal reference block, the group of motion information 18 included in this selected block is copied to the encoding target block together with the position information. Next, motion compensation is performed using the motion information 18 or a group of motion information 18 copied by the motion compensation unit 113, and the predicted image signal 11 and the motion information 18 used for motion compensation prediction are output.

FIG. 18 shows a more detailed configuration of variable length encoding section 104. The variable length encoding section 104 includes a parameter encoding section 114, a transform coefficient encoding section 115, a selected block encoding section 116, and a multiplexing section 117, as shown in FIG. The parameter encoding unit 114 encodes parameters necessary for decoding, such as prediction mode information, block size information, and quantization parameter information, excluding the transform coefficient information 13 and selected block information 31, and generates encoded data 14A. . The transform coefficient encoding unit 115 encodes the transform coefficient information 13 to generate encoded data 14B. Furthermore, the selected block encoding unit 116 refers to the available block information 30, encodes the selected block information 31, and generates encoded data 14C.

When the available block information 30 includes an index and the availability of a motion reference block corresponding to the index, as shown in FIG. Exclude reference blocks and convert only available motion reference blocks to syntax (stds_idx). In FIG. 19, five of the nine motion reference blocks are unavailable, so the syntax stds_idx is sequentially assigned from 0 to the four motion reference blocks excluding these blocks. In this example, the selected block information to be encoded is not selected from nine blocks but from four available blocks, so the amount of code (number of bins) to be allocated is small on average.

FIG. 20 shows an example of a code table showing syntax stds_idx and binary information (bin) of syntax stds_idx. As shown in FIG. 18, the fewer the number of available motion reference blocks, the lower the average number of bins required for encoding the syntax stds_idx. For example, if the number of available blocks is 4, the syntax stds_idx can be represented with 3 bits or less. Binary information (bin) of syntax stds_idx may be binarized so that all stds_idx have the same number of bins for each number of available blocks, and may be binarized according to a binarization method determined by prior learning. I don't mind. Further, a plurality of binarization methods may be prepared and switched appropriately for each block to be encoded.

Entropy encoding (for example, equal-length encoding, Huffman encoding, or arithmetic encoding) can be applied to these encoding units 114, 115, and 116, and the generated encoded data 14A, 14B, 14C is multiplexed by the multiplexer 117 and output.

In this embodiment, an example is described in which a frame encoded one frame before the encoding target frame is referred to as a reference frame. The reference motion information 19 may be applied to the block to be encoded by scaling (or normalizing) the motion vector using the frame number.

This scaling process will be specifically explained with reference to FIG. 21. tc shown in FIG. 21 indicates the time distance (POC (number indicating display order) distance) between the encoding target frame and the motion reference frame, and is calculated by the following equation 5. tr[i] shown in FIG. 21 indicates the time distance between the motion reference frame and the frame i referred to by the selected block, and is calculated by the following equation 6.

Here, curPOC indicates the POC (Picture Order Count) of the encoding target frame, colPOC indicates the POC of the motion reference frame, and refPOC indicates the POC of the frame i referenced by the selected block. Also, Clip (min, max, target) outputs min if target is smaller than min, max if it is larger than max, and outputs target otherwise. This is the clip function that does this. Furthermore, DiffPicOrderCnt(x,y) is a function that calculates the difference between two POCs.

Assuming that the motion vector of the selected block is MVr=(MVr_x, MVr_y) and the motion vector applied to the block to be encoded is MV=(MV_x, MV_y), the motion vector MV is calculated using Equation 7 below.

Here, Abs(x) indicates a function that extracts the absolute value of x. In this way, in motion vector scaling, the motion vector MVr assigned to the selected block is converted into a motion vector MV between the encoding target frame and the first motion reference frame.

Further, other examples regarding scaling of motion vectors will be described below.
First, for each slice or frame, a scaling factor (DistScaleFactor[i]) is determined for all possible temporal distances tr of the motion reference frame according to Equation 8 below. The number of scaling factors is equal to the number of frames referenced by the selected block, ie, the number of reference frames.

The calculation of tx shown in Equation 8 may be made into a table in advance.

When scaling each block to be encoded, by using Equation 9 below, the motion vector MV can be calculated using only multiplication, addition, and shift operations.

When such scaling processing is performed, the motion information 18 after scaling is applied to the processing of the available block acquisition unit 109 together with the prediction unit 101 . When scaling processing is performed, the reference frame referenced by the encoding target block becomes a motion reference frame.

FIG. 22 shows the syntax structure in the image encoding section 100. As shown in FIG. 22, the syntax mainly includes three parts: high-level syntax 901, slice-level syntax 904, and macroblock-level syntax 907. The high-level syntax 901 holds syntax information of layers higher than slices. The slice level syntax 904 holds information necessary for each slice, and the macroblock level syntax 907 holds data required for each macroblock shown in FIGS. 7A to 7D.

Each part includes further detailed syntax. High-level syntax 901 includes sequence and picture level syntax such as sequence parameter set syntax 902 and picture parameter set syntax 903. Slice level syntax 904 includes slice header syntax 905, slice data syntax 906, and the like. Furthermore, macroblock level syntax 907 includes macroblock layer syntax 908, macroblock prediction syntax 909, and the like.

23A and 23B show examples of macroblock layer syntax. available_block_num shown in FIGS. 23A and 23B indicates the number of available blocks, and if this is a value greater than 1, encoding of selected block information is required. Furthermore, stds_idx indicates selected block information, and stds_idx is encoded using the code table according to the number of available blocks described above.

FIG. 23A shows the syntax when the selected block information is encoded after mb_type. If the mode indicated by mb_type is a defined size or mode (TARGET_MODE), and available_block_num is a value greater than 1, stds_idx is encoded. For example, stds_idx is encoded when the motion information of the selected block becomes available when the block size is 64 x 64 pixels, 32 x 32 pixels, or 16 x 16 pixels, or when the direct mode is used.

FIG. 23B shows the syntax when the selected block information is encoded before mb_type. If available_block_num is greater than 1, encode stds_idx. Also, if available_block_num is 0, H. Since conventional motion compensation such as H.264 is performed, mb_type is encoded.

Note that syntax elements not specified in the present invention may be inserted between the rows of the tables shown in FIGS. 23A and 23B, and descriptions regarding other conditional branches may also be included. . Alternatively, it is also possible to divide and integrate the syntax table into multiple tables. Furthermore, it is not necessary to use the same terms, and they may be arbitrarily changed depending on the form of use. Furthermore, each syntax element described in the macroblock layer syntax may be changed as specified in the macroblock data syntax described later.

Furthermore, by using the stds_idx information, it is possible to reduce the mb_type information. FIG. 24A shows H. This is a code table corresponding to mb_type and mb_type at the time of B slice in H.264. N shown in FIG. 24A is a value representing the size of the block to be encoded, such as 16, 32, or 64, and M is a half value of N. Therefore, when mb_type is 4 to 21, it indicates that the block to be encoded is a rectangular block. Further, L0, L1, and Bi in FIG. 24A indicate unidirectional prediction (only in the List0 direction), unidirectional prediction (only in the List1 direction), and bidirectional prediction, respectively. When the block to be encoded is a rectangular block, mb_type includes information indicating which of L0, L1, and Bi prediction has been performed for each of the two rectangular blocks in the block to be encoded. Further, B_Sub means that the above processing is performed on each of the pixel blocks obtained by dividing the macroblock into four. For example, if the block to be encoded is a 64x64 pixel macroblock, the block to be encoded is divided into four 32x32 pixel blocks, each of which is further assigned an mb_type and coded. be converted into

Here, if the selected block indicated by stds_idx is Spatial Left (pixel block adjacent to the left side of the block to be encoded), the motion information of the pixel block adjacent to the left side of the block to be encoded is Since it is motion information, stds_idx is predicted for the block to be encoded using the horizontally long rectangular blocks indicated by mb_type=4,6,8,10,12,14,16,18,20 in Figure 24A. It has the same meaning as executing. Furthermore, when the selected block indicated by stds_idx is Spatial Up, the motion information adjacent to the upper side of the block to be encoded is used as the motion information of the block to be encoded, so stds_idx is set to mb_type=5,7, This has the same meaning as performing prediction on the vertically long rectangular blocks indicated by 9, 11, 13, 15, 17, 19, and 21. Therefore, by using stds_idx, it is possible to create a code table such as shown in FIG. 24B in which the columns of mb_type=4 to 21 in FIG. 24A are removed. Similarly, H. Regarding the code table corresponding to mb_type and mb_type at the time of P slice in H.264, it is possible to create a code table with a reduced number of mb_types as shown in FIG. 24D.

Furthermore, stds_idx information may be included in mb_type information and encoded. FIG. 25A is a code table when stds_idx information is included in mb_type information, and shows an example of a code table corresponding to mb_type and mb_type in a B slice. B_STDS_X (X=0, 1, 2) in FIG. 25A indicates a mode corresponding to stds_idx, and B_STDS_X is added by the number of available blocks (in FIG. 25A, the number of available blocks is 3). Similarly, another example of mb_type regarding P slice is shown in FIG. 25B. The description of FIG. 25B is omitted because it is the same as the B slice.

The order and binarization method (binization) of mb_type are not limited to the examples shown in FIGS. 25A and 25B, and mb_type may be encoded according to other orders and binarization methods. B_STDS_X and P_STDS_X do not need to be consecutive, and may be placed between each mb_type. Further, the binarization method (binization) may be designed based on the selection frequency learned in advance.

In this embodiment, the present invention is also applicable to extended macroblocks in which motion compensation prediction is performed on a plurality of macroblocks. Furthermore, in this embodiment, the encoding scan order may be in any order. For example, the present invention is also applicable to line scan or Z scan.

As described above, the image encoding device according to the present embodiment selects an available block from a plurality of motion reference blocks, and applies a motion reference block to the encoding target block according to the number of selected available blocks. The system generates information to identify the object and encodes this information. Therefore, according to the image encoding device according to the present embodiment, motion compensation can be performed in units of small pixel blocks, which are smaller than the target block to be encoded, while reducing the amount of code related to motion vector information, resulting in high encoding efficiency. can be realized.

(Second embodiment)
FIG. 26 shows an image encoding device according to a second embodiment of the present invention. In the second embodiment, parts and operations that are different from the first embodiment will be mainly described. As shown in FIG. 26, the image encoding unit 200 according to this embodiment differs from the first embodiment in the configurations of a prediction unit 201 and a variable length encoding unit 204. As shown in FIG. 27, the prediction unit 201 includes a first prediction unit 101 and a second prediction unit 202, and generates the predicted image signal 11 by selectively switching between the first and second prediction units 101 and 202. . The first prediction unit 101 has the same configuration as the prediction unit 101 (FIG. 1) according to the first embodiment, and follows a prediction method (first prediction method) that performs motion compensation using motion information 18 of the selected block. , generates a predicted image signal 11. The second prediction unit 202 performs motion compensation on the block to be encoded using one motion vector. The predicted image signal 11 is generated according to a prediction method such as H.264 (second prediction method). The second prediction unit 202 generates a predicted image signal 11B using the input image signal 10 and the reference image signal 17 from the frame memory.

FIG. 28 schematically shows the configuration of the second prediction unit 202. As shown in FIG. 28, the second prediction unit 202 includes a motion information acquisition unit 205 that generates motion information 21 using the input image signal 10 and the reference image signal 17, and a motion information acquisition unit 205 that generates the motion information 21 using the input image signal 10 and the reference image signal 17; The motion compensation unit 113 (FIG. 1) generates the predicted image signal 11A using the motion compensation unit 113 (FIG. 1). The motion information acquisition unit 205 obtains a motion vector to be assigned to a block to be encoded based on the input image signal 10 and the reference image signal 17, for example, by block matching. As a matching evaluation criterion, a value obtained by accumulating the differences between the input image signal 10 and the interpolated image after matching for each pixel is used.

Note that the motion information acquisition unit 205 may determine the optimal motion vector using a value obtained by converting the difference between the predicted image signal 11 and the input image signal 10. Further, the optimal motion vector may be determined by taking into consideration the size of the motion vector and the amount of code of the motion vector and the reference frame number, or by using Equation 1. The matching method may be executed based on search range information provided from outside the image encoding device, or may be executed hierarchically for each pixel precision. Alternatively, the motion information provided by the encoding control section 150 may be used as the output 21 of the motion information acquisition section 205 without performing the search process.

The prediction unit 101 in FIG. 27 further includes a prediction method changeover switch 203 that selects and outputs either the predicted image signal 11A from the first prediction unit 101 or the predicted image signal 11B from the second prediction unit 202. There is. For example, the prediction method changeover switch 203 uses the input image signal 10 for each of the predicted image signals 11A and 11B to determine the encoding cost according to Equation 1, and selects the predicted image so that the encoding cost is smaller. Either one of the signals 11A and 11B is selected and output as the predicted image signal 11. Further, the prediction method changeover switch 203 is a prediction switch that indicates whether the outputted predicted image signal 11 is output from either the first prediction unit 101 or the second prediction unit 202 together with the motion information 18 and the selected block information 31. Information 32 is also output. The output motion information 18 is encoded by the variable length encoding section 204 and then multiplexed into the encoded data 14.

FIG. 29 schematically shows the configuration of variable length encoding section 204. The variable length encoding section 204 shown in FIG. 29 includes a motion information encoding section 217 in addition to the configuration of the variable length encoding section 104 shown in FIG. Further, the selected block encoding unit 216 in FIG. 29, unlike the selected block encoding unit 116 in FIG. 18, encodes the prediction switching information 32 to generate encoded data 14D. When the first prediction unit 101 executes the prediction process, the selected block encoding unit 216 further encodes the available block information 30 and the selected block information 31. The encoded available block information 30 and selected block information 31 are included in the encoded data 14D. When the second prediction unit 202 executes the prediction process, the motion information encoding unit 217 encodes the motion information 18 and generates encoded data 14E. The selected block encoding unit 216 and the motion information encoding unit 217 each perform a prediction process based on prediction switching information 32 indicating whether the predicted image is generated by motion compensated prediction using motion information of the selected block. It is determined which of the first prediction unit 101 and the second prediction unit 202 has been executed.

The multiplexing unit 117 receives encoded data 14A, B, D, and E from the parameter encoding unit 114, the transform coefficient encoding unit 115, the selected block encoding unit 216, and the motion information encoding unit, and converts the received encoded data 14A, B, D, and E are multiplexed.

30A and 30B each show an example of macroblock layer syntax according to this embodiment. available_block_num shown in FIG. 30A indicates the number of available blocks, and if this is a value larger than 1, the selected block encoding unit 216 encodes the selected block information 31. In addition, stds_flag is a flag indicating whether or not the motion information of the selected block is used as the motion information of the encoding target block in motion compensation prediction, that is, the prediction method changeover switch 203 is This is a flag indicating which one of 202 has been selected. If the number of available blocks is greater than 1 and stds_flag is 1, this indicates that motion information possessed by the selected block is used for motion compensated prediction. Furthermore, when stds_flag is 0, H. Similar to H.264, the difference value obtained by directly or predicting the information of the motion information 18 is encoded. Furthermore, stds_idx indicates selected block information, and the code table corresponding to the number of available blocks is as described above.

FIG. 30A shows the syntax when the selected block information is encoded after mb_type. stds_flag and stds_idx are encoded only when the mode indicated by mb_type is a specified size or a specified mode. For example, stds_flag and stds_idx are encoded when the motion information of the selected block becomes available when the block size is 64x64, 32x32, or 16x16, or when the direct mode is used.

FIG. 30B shows the syntax when the selected block information is encoded before mb_type. For example, if stds_flag is 1, mb_type does not need to be encoded. If stds_flag is 0, mb_type is encoded.

As described above, the image encoding device according to the second embodiment uses the first prediction unit 101 according to the first embodiment and a prediction method such as H.264 so that the encoding cost is reduced. The input image signal is compressed and encoded by selectively switching the second prediction unit 202 to compress and encode the input image signal. Therefore, in the image encoding device according to the second embodiment, the encoding efficiency is improved compared to the image encoding device according to the first embodiment.

(Third embodiment)
FIG. 31 schematically shows an image decoding device according to the third embodiment. This image decoding device includes an image decoding section 300, a decoding control section 350, and an output buffer 308, as shown in FIG. Image decoding section 300 is controlled by decoding control section 350. The image decoding device according to the third embodiment corresponds to the image encoding device according to the first embodiment. That is, the decoding process by the image decoding device in FIG. 31 has a complementary relationship with the encoding process by the image encoding process in FIG. 1. The image decoding device shown in FIG. 31 may be realized by hardware such as an LSI chip, or by causing a computer to execute an image decoding program.

The image decoding device in FIG. 31 includes a coded sequence decoding section 301, an inverse quantization/inverse transformation section 302, an adder 303, a frame memory 304, a prediction section 305, a motion information memory 306, and an available block acquisition section 307. We are prepared. In the image decoding section 300, encoded data 80 from a storage system or a transmission system (not shown) is input to a coded string decoding section 301. This encoded data 80 corresponds to, for example, the encoded data 14 sent out in a multiplexed state from the image encoding device of FIG. 1.

In this embodiment, a pixel block (for example, a macroblock) that is a decoding target is simply referred to as a decoding target block. Further, an image frame including a block to be decoded is referred to as a frame to be decoded.

The coded string decoding unit 301 performs decoding by syntax analysis for each frame or field based on the syntax. Specifically, the coded string decoding unit 301 sequentially performs variable length decoding on the coded strings of each syntax, and obtains prediction information such as transform coefficient information 33, selected block information 61, block size information, and prediction mode information. The encoding parameters related to the block to be decoded, including the decoding target block, are decoded.

In this embodiment, the decoding parameters include the transform coefficients 33, selected block information 61, and prediction information, and include all parameters necessary for decoding, such as information regarding transform coefficients and information regarding quantization. The prediction information, information regarding transform coefficients, and information regarding quantization are input to the decoding control unit 350 as control information 71. The decoding control unit 350 provides each unit of the image decoding unit 300 with decoding control information 70 including parameters necessary for decoding such as prediction information and quantization parameters.

Furthermore, the coded stream decoding unit 301 simultaneously decodes the coded data 80 to obtain prediction information and selected block information 61, as will be described later. The motion information 38 including the motion vector and reference frame number does not need to be decoded.

The transform coefficients 33 decoded by the coded string decoding section 301 are sent to the inverse quantization/inverse transform section 302 . Various information related to quantization, that is, quantization parameters and quantization matrices decoded by the encoded sequence decoding unit 301, are given to the decoding control unit 350, and are dequantized and inversely transformed when dequantizing. 302. The inverse quantization/inverse transform unit 302 inversely quantizes the transform coefficients 33 according to the loaded quantization information, and then performs inverse transform processing (for example, inverse discrete cosine transform) to generate a prediction error signal 34. obtain. The inverse transformation process by the inverse quantization/inverse transformation unit 302 in FIG. 31 is the inverse transformation of the transformation process by the transformation/quantization unit in FIG. 1. For example, when a wavelet transform is performed by the image encoding device (FIG. 1), the inverse quantization/inverse transform unit 302 performs the corresponding inverse quantization and inverse wavelet transform.

The prediction error signal 34 restored by the inverse quantization/inverse transform section 302 is input to the adder 303. The adder 303 adds the prediction error signal 34 and a predicted image signal 35 generated by a prediction unit 305, which will be described later, to generate a decoded image signal 36. The generated decoded image signal 36 is outputted from the image decoding section 300, temporarily stored in the output buffer 308, and then outputted according to the output timing managed by the decoding control section 350. Further, this decoded image signal 36 is stored in the frame memory 304 as a reference image signal 37. The reference image signal 37 is sequentially read out frame by frame or field by field from the frame memory 304 and input to the prediction unit 305 .

The available block acquisition unit 307 receives reference motion information 39 from a motion information memory 306, which will be described later, and outputs available block information 60. The operation of the available block acquisition unit 307 is similar to that of the available block acquisition unit 109 (FIG. 1) described in the first embodiment.

The motion information memory 306 receives the motion information 38 from the prediction unit 305 and temporarily stores it as reference motion information 39. The motion information memory 306 temporarily stores the motion information 38 output from the prediction unit 305 as reference motion information 39. FIG. 4 shows an example of the motion information memory 306. The motion information memory 306 holds a plurality of motion information frames 26 having different encoding times. The motion information 38 or a group of motion information 38 that has been decoded is stored as reference motion information 39 in the motion information frame 26 corresponding to the decoding time. In the motion information frame 26, the reference motion information 39 is stored, for example, in units of 4×4 pixel blocks. The reference motion information 39 held in the motion information memory 306 is read out and referenced by the prediction unit 305 when generating the motion information 38 of the block to be decoded.

Next, motion reference blocks and available blocks according to this embodiment will be explained. The motion reference block is a candidate block selected from the already decoded area by the above-described image encoding device and image decoding device according to a predetermined method. FIG. 8A shows an example regarding available blocks. In FIG. 8A, a total of nine motion reference blocks are arranged, including four motion reference blocks in the decoding target frame and five motion reference blocks in the reference frame. Motion reference blocks A, B, C, and D in the frame to be decoded in FIG. 8A are blocks adjacent to the block to be decoded on the left, above, upper right, and upper left. In this embodiment, a motion reference block selected from a frame to be decoded that includes a block to be decoded is referred to as a spatial direction motion reference block. Further, the motion reference block TA in the reference frame is a pixel block in the same position as the decoding target block in the reference frame, and the pixel blocks TB, TC, TD, and TE that are in contact with this motion reference block TA move. Selected as reference block. A motion reference block selected from among the pixel blocks in the reference frame is referred to as a temporal motion reference block. Further, a frame in which a temporal motion reference block is located is referred to as a motion reference frame.

The spatial direction motion reference block is not limited to the example shown in FIG. 8A, but as shown in FIG. 8B, the pixel block to which pixels a, b, c, and d adjacent to the decoding target block belong is used as the spatial direction motion reference block. It doesn't matter if it is selected. In this case, the relative positions (dx, dy) of pixels a, b, c, and d with respect to the upper left pixel in the block to be decoded are shown in FIG. 8C.

Furthermore, as shown in FIG. 8D, all pixel blocks A1 to A4, B1, B2, C, and D adjacent to the block to be decoded may be selected as spatial motion reference blocks. In FIG. 8D, the number of spatial motion reference blocks is eight.

Furthermore, the temporal motion reference blocks TA to TE may partially overlap each other, as shown in FIG. 8E, or may be separated from each other, as shown in FIG. 8F. Further, the temporal motion reference block does not necessarily have to be located at the block at the collocate position and its surroundings, but may be a pixel block at any position within the motion reference frame. For example, by using motion information of already decoded blocks adjacent to the block to be decoded, a reference block pointed to by a motion vector included in the motion information is selected as the center of the motion reference block (for example, block TA). I don't mind. Furthermore, the reference blocks in the time direction do not have to be arranged at equal intervals.

In the method for selecting motion reference blocks as described above, if both the image decoding device and the image decoding device share information regarding the number and position of motion reference blocks in the spatial and temporal directions, the motion reference blocks can be selected. may be selected from any number and position. Further, the size of the motion reference block does not necessarily have to be the same size as the block to be decoded. For example, as shown in FIG. 8D, the size of the motion reference block may be larger or smaller than the size of the block to be decoded, or may be any size. Further, the shape of the motion reference block is not limited to a square shape, but may be a rectangular shape.

Next, available blocks will be explained. The available block is a pixel block selected from among the motion reference blocks, and is a pixel block for which motion information can be applied to the block to be decoded. The available blocks have different motion information. The available blocks are selected, for example, by executing the available block determination process shown in FIG. 9 on a total of nine motion reference blocks in the decoding target frame and reference frame as shown in FIG. 8A. . FIG. 10 shows the result of executing the available block determination process shown in FIG. 9. In FIG. 10, hatched pixel blocks indicate unavailable blocks, and white blocks indicate usable blocks. That is, two out of the spatial motion reference blocks and two out of the temporal motion reference blocks, a total of four blocks, are determined to be available blocks. The motion information selection unit 314 in the prediction unit 305 selects an optimal one available from among these available blocks arranged in the temporal and spatial directions, according to the selected block information 61 received from the selected block decoding unit 323. Select the block as the selected block.

Next, the available block acquisition unit 307 will be explained. The available block acquisition unit 307 has the same function as the available block acquisition unit 109 of the first embodiment, and acquires the reference motion information 39 from the motion information memory 306, and acquires the available block or usage information for each motion reference block. Available block information 60, which is information indicating impossible blocks, is output.

The operation of the available block acquisition unit 307 will be explained with reference to the flowchart of FIG. First, the available block acquisition unit 307 determines whether the motion reference block (index p) has motion information (step S801). That is, in step S801, it is determined whether at least one small pixel block within the motion reference block p has motion information. If it is determined that the motion reference block p does not have motion information, that is, the temporal motion reference block is a block in an I slice without motion information, or all the small temporal motion reference blocks in the temporal motion reference block are If the pixel block has been intra-predictively decoded, the process advances to step S805. In step S805, this motion reference block p is determined to be an unavailable block.

If it is determined in step S801 that the motion reference block p has motion information, the available block acquisition unit 307 selects a motion reference block q (referred to as available block q) that has already been determined to be an available block. (Step S802). Here, q is a value smaller than p. Next, the available block acquisition unit 307 compares the motion information of the motion reference block p with the motion information of the available block q for all q, and determines that the motion reference block p is the available block q. It is determined whether it has the same motion information as (S803). If the motion reference block p has the same motion vector as the available block q, the process advances to step S805, and in step S805, the motion reference block p is classified as an unavailable block by the available block acquisition unit 307. It will be judged. If the motion reference block p has different motion information from all available blocks q, in step S804, the motion reference block p is determined to be an available block by the available block acquisition unit 307.

By executing the above-described available block determination process on all motion reference blocks, it is determined whether each motion reference block is an available block or an unusable block, and available block information 60 is generated. An example of the available block information 60 is shown in FIG. The available block information 60 includes the index p and availability of the motion reference block, as shown in FIG. In FIG. 11, the available block information 60 indicates that motion reference blocks with indexes p of 0, 1, 5, and 8 are selected as available blocks, and the number of available blocks is 4.

Note that in step S801 in FIG. 9, if at least one of the blocks in the temporal motion reference block p is an intra-prediction encoded block, the available block acquisition unit 307 uses the motion reference block p. It does not matter if it is determined to be an impossible block. That is, the process may proceed to step S802 only when all blocks in the temporal motion reference block p have been encoded by inter prediction.

12A to 12E show an example in which the motion information 38 of the motion reference block p and the motion information 38 of the available block q are determined to be the same in the comparison of the motion information 38 in step S803. 12A to 12E each show a plurality of blocks shaded with diagonal lines and two blocks painted white. In FIGS. 12A to 12E, in order to simplify the explanation, it is assumed that the motion information 38 of these two white blocks is compared without considering the shaded block. It is assumed that one of the two white blocks is a motion reference block p, and the other is a motion reference block q (available block q) that has already been determined to be available. Unless otherwise specified, it does not matter which of the two white blocks may be the motion reference block p.

FIG. 12A shows an example where both the motion reference block p and the available block q are spatial blocks. In the example of FIG. 12A, if the motion information 38 of blocks A and B is the same, it is determined that the motion information 38 is the same. At this time, the sizes of blocks A and B do not need to be the same.

FIG. 12B shows an example in which one of the motion reference block p and the available block q is a block A in the spatial direction, and the other is a block TB in the temporal direction. In FIG. 12B, there is one block having motion information in the block TB in the time direction. If the motion information 38 of the block TB in the temporal direction and the motion information 38 of the block A in the spatial direction are the same, it is determined that the motion information 38 is the same. At this time, the sizes of blocks A and TB do not need to be the same.

FIG. 12C shows another example in which one of the motion reference block p and the available block q is a block A in the spatial direction, and the other is a block TB in the temporal direction. FIG. 12C shows a case where the block TB in the time direction is divided into a plurality of small blocks, and there are a plurality of small blocks having motion information 38. In the example of FIG. 12C, if all blocks having motion information 38 have the same motion information 38 and the motion information 38 is the same as the motion information 38 of block A in the spatial direction, then the motion information 38 is the same. It is determined that At this time, the sizes of blocks A and TB do not need to be the same.

FIG. 12D shows an example in which the motion reference block p and the available block q are both blocks in the time direction. In this case, if the motion information 38 of blocks TB and TE is the same, it is determined that the motion information 38 is the same.

FIG. 12E shows another example in which the motion reference block p and the available block q are both blocks in the time direction. FIG. 12E shows a case where blocks TB and TE in the time direction are each divided into a plurality of small blocks, and there are a plurality of small blocks each having motion information 38. In this case, the motion information 38 is compared for each small block within the block, and if the motion information 38 is the same for all small blocks, the motion information 38 of the block TB and the motion information 38 of the block TE are the same. It is determined that there is.

FIG. 12F shows yet another example in which the motion reference block p and the available block q are both blocks in the time direction. FIG. 12F shows a case where the block TE in the time direction is divided into a plurality of small blocks, and the block TE includes a plurality of small blocks having motion information 38. If all the motion information 38 of the block TE is the same motion information 38 and the same as the motion information 38 of the block TD, it is determined that the motion information 38 of the blocks TD and TE are the same.

In this way, in step S803, it is determined whether the motion information 38 of the motion reference block p and the motion information 38 of the available block q are the same. In the examples of FIGS. 12A to 12F, the number of available blocks q to be compared with the motion reference block p is 1. However, in the case where the number of available blocks q is 2 or more, the motion information of the motion reference block p 38 may be compared with the motion information 38 of each available block q. Furthermore, when scaling described later is applied, the motion information 38 after scaling becomes the motion information 38 described above.

Note that the determination that the motion information of the motion reference block p and the motion information of the available block q are the same is not limited to the case where each motion vector included in the motion information completely matches. For example, if the norm of the difference between two motion vectors is within a predetermined range, the motion information of the motion reference block p and the motion information of the available block q may be considered to be substantially the same.

FIG. 32 is a block diagram showing encoded sequence decoding section 301 in more detail. As shown in FIG. 32, the coded stream decoding unit 301 includes a separating unit 320 that separates coded data 80 into syntax units, a transform coefficient decoding unit 322 that decodes transform coefficients, and a decodes selected block information. and a parameter decoding unit 321 that decodes parameters related to predicted block size and quantization.

The parameter decoding unit 321 receives encoded data 80A including parameters regarding the predicted block size and quantization from the separation unit, decodes the encoded data 80A, and generates control information 71. The transform coefficient decoding unit 322 receives the encoded transform coefficient 80B from the separating unit 320, decodes the encoded transform coefficient 80B, and obtains the transform coefficient information 33. The selected block decoding unit 323 receives the encoded data 80C regarding the selected block and the available block information 60, and outputs selected block information 61. The input available block information 60 indicates the availability of each motion reference block, as shown in FIG.

Next, the prediction unit 305 will be explained in detail with reference to FIG. 33.
As shown in FIG. 33, the prediction unit 305 includes a motion information selection unit 314 and a motion compensation unit 313, and the motion information selection unit 314 includes a spatial motion information acquisition unit 310, a temporal motion information acquisition unit 311, and a motion information A changeover switch 312 is provided. The prediction unit 305 basically has the same configuration and function as the prediction unit 101 described in the first embodiment.

The prediction unit 305 receives the available block information 60, selected block information 61, reference motion information 39, and reference image signal 37 as input, and outputs a predicted image signal 35 and motion information 38. The spatial direction motion information acquisition section 310 and the temporal direction motion information acquisition section 311 have the same functions as the spatial direction motion information acquisition section 110 and the temporal direction motion information acquisition section 111, respectively, described in the first embodiment. The spatial direction motion information acquisition unit 310 uses the available block information 60 and the reference motion information 39 to generate motion information 38A including motion information and an index of each available block located in the spatial direction. The temporal motion information acquisition unit 311 uses the available block information 60 and the reference motion information 39 to obtain motion information (or a group of motion information) including motion information and an index of each available block located in the temporal direction. Generate 38B.

The motion information changeover switch 312 selects one of the motion information 38A from the spatial motion information acquisition section 310 and the motion information (or group of motion information) 38B from the temporal motion information acquisition section 311 according to the selection block information 61. Select one to obtain motion information 38. The selected motion information 38 is sent to the motion compensation section 313 and the motion information memory 306. The motion compensation unit 313 performs motion compensation prediction in accordance with the selected motion information 38 in the same manner as the motion compensation unit 113 described in the first embodiment, and generates a predicted image signal 35.

The motion vector scaling function of the motion compensation unit 313 is the same as that described in the first embodiment, so the description thereof will be omitted.

FIG. 22 shows the syntax structure in the image decoding section 300. As shown in FIG. 22, the syntax mainly includes three parts: high-level syntax 901, slice-level syntax 904, and macroblock-level syntax 907. The high-level syntax 901 holds syntax information of layers higher than slices. The slice level syntax 904 holds information necessary for each slice, and the macroblock level syntax 907 holds data required for each macroblock shown in FIGS. 7A to 7D.

Each part includes further detailed syntax. High-level syntax 901 includes sequence and picture level syntax such as sequence parameter set syntax 902 and picture parameter set syntax 903. Slice level syntax 904 includes slice header syntax 905, slice data syntax 906, and the like. Furthermore, macroblock level syntax 907 includes macroblock layer syntax 908, macroblock prediction syntax 909, and the like.

23A and 23B show examples of macroblock layer syntax. available_block_num shown in FIGS. 23A and 23B indicates the number of available blocks, and if this is a value greater than 1, it is necessary to decode the selected block information. Furthermore, stds_idx indicates selected block information, and stds_idx is encoded using the code table according to the number of available blocks described above.

FIG. 23A shows the syntax when the selected block information is decoded after mb_type. When the prediction mode indicated by mb_type is a defined size or a defined mode (TARGET_MODE), and available_block_num is a value greater than 1, stds_idx is decoded. For example, stds_idx is encoded when the motion information of the selected block becomes available when the block size is 64 x 64 pixels, 32 x 32 pixels, or 16 x 16 pixels, or when the direct mode is used.

FIG. 23B shows the syntax when the selected block information is decoded before mb_type. If available_block_num is greater than 1, decode stds_idx. Also, if available_block_num is 0, H. Since conventional motion compensation such as H.264 is performed, mb_type is encoded.

Syntax elements not defined by the present invention may be inserted between the rows of the tables shown in FIGS. 23A and 23B, and descriptions regarding other conditional branches may also be included. Alternatively, it is also possible to divide and integrate the syntax table into multiple tables. Furthermore, it is not necessary to use the same terms, and they may be arbitrarily changed depending on the form of use. Furthermore, each syntax element described in the macroblock layer syntax may be changed as specified in the macroblock data syntax described later.

As described above, the image decoding device according to this embodiment decodes the image encoded by the image encoding device according to the first embodiment described above. Therefore, the image decoding according to this embodiment can reproduce a high-quality decoded image from relatively small encoded data.

(Fourth embodiment)
FIG. 34 schematically shows an image decoding device according to the fourth embodiment. The image decoding device includes an image decoding section 400, a decoding control section 350, and an output buffer 308, as shown in FIG. The image decoding device according to the fourth embodiment corresponds to the image encoding device according to the second embodiment. In the fourth embodiment, parts and operations that are different from the third embodiment will be mainly described. As shown in FIG. 34, the image decoding unit 400 according to this embodiment differs from the third embodiment in a coded sequence decoding unit 401 and a prediction unit 405.

The prediction unit 405 of this embodiment uses a prediction method (first prediction method) that performs motion compensation using motion information possessed by a selected block, and an H. The predicted image signal 35 is generated by selectively switching between a prediction method (second prediction method) such as H.264 that performs motion compensation using one motion vector for a block to be decoded.

FIG. 35 is a block diagram showing encoded sequence decoding section 401 in more detail. Coded stream decoding section 401 shown in FIG. 35 includes a motion information decoding section 424 in addition to the configuration of coded stream decoding section 301 shown in FIG. Further, the selected block decoding unit 423 shown in FIG. 35 differs from the selected block decoding unit 323 shown in FIG. 32 in that it decodes the encoded data 80C regarding the selected block to obtain prediction switching information 62. The prediction switching information 62 indicates which of the first and second prediction methods is used by the prediction unit 101 in the image encoding device of FIG. When the prediction switching information 62 indicates that the prediction unit 101 used the first prediction method, that is, when the block to be decoded is encoded using the first prediction method, the selected block decoding unit 423 uses the first prediction method. The selected block information in the data 80C is decoded to obtain the selected block information 61. When the prediction switching information 62 indicates that the prediction unit 101 has used the second prediction method, that is, when the block to be decoded is encoded using the second prediction method, the selected block decoding unit 423 uses the selected block information. The motion information decoding unit 424 decodes the encoded motion information 80D without decoding the encoded motion information 80D to obtain the motion information 40.

FIG. 36 is a block diagram showing the prediction unit 405 in more detail. The prediction unit 405 shown in FIG. 34 includes a first prediction unit 305, a second prediction unit 410, and a prediction method changeover switch 411. The second prediction unit 410 uses the motion information 40 decoded by the coded sequence decoding unit 401 and the reference image signal 37 to perform motion compensation prediction similar to the motion compensation unit 313 in FIG. Generate 35B. The first prediction unit 305 is the same as the prediction unit 305 described in the third embodiment, and generates a predicted image signal 35B. Further, the prediction method changeover switch 411 selects either one of the predicted image signal 35B from the second prediction unit 410 and the predicted image signal 35A from the first prediction unit 305 based on the prediction switching information 62. The prediction unit 405 outputs the predicted image signal 35. At the same time, the prediction method changeover switch 411 sends the motion information used in the selected first prediction section 305 or second prediction section 410 to the motion information memory 306 as motion information 38.

Next, regarding the syntax structure of this embodiment, differences from the third embodiment will be mainly explained.

30A and 30B each show an example of macroblock layer syntax according to this embodiment. available_block_num shown in FIG. 30A indicates the number of available blocks, and if this is a value greater than 1, the selected block decoding unit 423 decodes the selected block information in the encoded data 80C. In addition, stds_flag is a flag indicating whether or not the motion information of the selected block is used as the motion information of the decoding target block in motion compensation prediction, that is, the prediction method changeover switch 411 is 410 is selected. If the number of available blocks is greater than 1 and stds_flag is 1, this indicates that motion information possessed by the selected block is used for motion compensated prediction. Furthermore, when stds_flag is 0, H. Similar to H.264, the difference value obtained by directly or predicting motion information is encoded. Furthermore, stds_idx indicates selected block information, and the code table corresponding to the number of available blocks is as described above.

FIG. 30A shows the syntax when the selected block information is decoded after mb_type. stds_flag and stds_idx are decoded only when the prediction mode indicated by mb_type has a defined block size or a defined mode. For example, if the block size is 64x64, 32x32, or 16x16, or in direct mode, stds_flag and stds_idx are decoded.

FIG. 30B shows the syntax when the selected block information is decoded before mb_type. For example, if stds_flag is 1, mb_type does not need to be decoded. If stds_flag is 0, mb_type is decoded.

As described above, the image decoding device according to the present embodiment decodes the image encoded by the image encoding device according to the second embodiment described above. Therefore, the image decoding according to this embodiment can reproduce a high-quality decoded image from relatively small encoded data.

It should be noted that the present invention is not limited to the above-described embodiments as they are, but can be implemented by modifying the constituent elements within the scope of the invention at the implementation stage. Moreover, various inventions can be formed by appropriately combining the plurality of components disclosed in the above embodiments. For example, some components may be deleted from all the components shown in the embodiments. Furthermore, components of different embodiments may be combined as appropriate.

As an example of this, similar effects can be obtained even if the first to fourth embodiments described above are modified as follows.

(1) In the first to fourth embodiments, the processing target frame is divided into rectangular blocks such as 16 x 16 pixel blocks, and the processing starts from the upper left pixel block of the screen to the lower right pixel block as shown in FIG. Although the case where encoding or decoding is performed in this order is described as an example, the encoding or decoding order is not limited to this example. For example, the encoding or decoding order may be from the lower right to the upper left of the screen, or from the upper right to the lower left. Further, the encoding or decoding order may be a spiral order from the center of the screen to the periphery, or may be an order from the periphery to the center of the screen.

(2) In the first to fourth embodiments, the case where the luminance signal and the color difference signal are not divided but limited to one color signal component is explained as an example. However, different prediction processing may be used for the luminance signal and the color difference signal, or the same prediction processing may be used. When using different prediction processes, the prediction method selected for the chrominance signal is encoded/decoded in the same manner as for the luminance signal.

It goes without saying that the present invention can also be implemented in the same manner by applying various biases without departing from the gist of the present invention.

Since the image encoding/decoding method according to the present invention can improve encoding efficiency, it has industrial applicability.

10... Input image signal, 11... Predicted image signal, 12... Predicted error image signal, 13... Quantized transform coefficient, 14... Encoded data, 15... Decoded predicted error signal, 16... Locally decoded image signal, 17... Reference image Signal, 18... Motion information, 20... Bit stream, 21... Motion information, 25, 26... Information frame, 30... Available block information, 31... Selected block information, 32... Prediction switching information, 33... Transform coefficient information, 34 ...Prediction error signal, 35...Prediction image signal, 36...Decoded image signal, 37...Reference image signal, 38...Motion information, 39...Reference motion information, 40...Motion information, 50...Encoding control information, 51...Feedback information , 60... Available block information, 61... Selected block information, 62... Prediction switching information, 70... Decoding control information, 71... Control information, 80... Encoded data, 100... Image encoding section, 101... Prediction section, 102...Subtractor, 103...Transformation/quantization section, 104...Variable length encoding section, 105...Dequantization/inverse transformation section, 106...Adder, 107...Frame memory, 108...Information memory, 109...Available Block acquisition unit, 110... Spatial motion information acquisition unit, 111... Temporal motion information acquisition unit, 112... Information changeover switch, 113... Motion compensation unit, 114... Parameter encoding unit, 115... Transform coefficient encoding unit, 116 ...Selected block encoding unit, 117…Multiplexing unit, 118…Motion information selection unit, 120…Output buffer, 150…Encoding control unit, 200…Image encoding unit, 201…Prediction unit, 202…Second prediction unit , 203... Prediction method changeover switch, 204... Variable length encoding section, 205... Motion information acquisition section, 216... Selected block encoding section, 217... Motion information encoding section, 300... Image decoding section, 301... Encoding Sequence decoding unit, 301... Code stream decoding unit, 302... Inverse quantization/inverse transformation unit, 303... Adder, 304... Frame memory, 305... Prediction unit, 306... Information memory, 307... Available block acquisition unit, 308... Output buffer, 310... Spatial direction motion information acquisition section, 311... Temporal direction motion information acquisition section, 312... Motion information changeover switch, 313... Motion compensation section, 314... Information selection section, 320... Separation section, 321... Parameter Decoding unit, 322... Transform coefficient decoding unit, 323... Selected block decoding unit, 350... Decoding control unit, 400... Image decoding unit, 401... Encoded string decoding unit, 405... Prediction unit, 410... Second prediction unit, 411... Prediction method changeover switch, 423... Selected block decoding unit, 424... Information decoding unit, 901... High level syntax, 902... Sequence parameter set syntax, 903... Picture parameter set syntax, 904... Slice Level syntax, 905...Slice header syntax, 906...Slice data syntax, 907...Macroblock level syntax, 908...Macroblock layer syntax, 909...Macroblock prediction syntax.

Claims (1)

Equipped with a circuit for transmitting encoded data,
The encoded data includes mode information regarding the prediction mode of the target block and identification information that specifies the selected block,
When the mode information indicates predetermined information, the identification information determines whether or not a plurality of candidate blocks having a predetermined positional relationship with respect to the target block are available, which is predetermined according to the positional relationship. and selecting one block whose motion information is used as motion information of the target block as the selected block from among the candidate blocks determined to be available. generated,
The plurality of candidate blocks include a block adjacent to the upper left of the target block and a block adjacent to the upper left of the target block,
The determining includes determining the block adjacent to the upper left of the target block after determining the block adjacent to the top of the target block, so that the candidate block corresponds to a candidate block that has already been determined to be available. determining that the candidate block is usable if the candidate block has motion information that does not match motion information that is
Transmitting device.