JP6323543B2 - Moving picture decoding apparatus, moving picture decoding method, and moving picture decoding program - Google Patents

Description

The present invention relates to a moving picture decoding technique using motion compensated prediction, and more particularly to a moving picture decoding apparatus, a moving picture decoding method, and a moving picture decoding program used for motion compensated prediction.

MPEG-4 AVC / H. In moving picture coding represented by H.264 (hereinafter AVC), motion compensated prediction is used in which a picture is divided into rectangular blocks, and motion estimation and compensation are performed in units of blocks between pictures. A motion vector generated in each block in motion compensation prediction is subjected to a prediction process in order to reduce the code amount.

In AVC, taking advantage of the strong correlation between the motion vectors of adjacent blocks,
A prediction value is calculated from an adjacent block, and a code amount is reduced by encoding a difference vector with the prediction value. However, in these prediction methods, since the positions of adjacent blocks to be referred to are limited, there is a problem that if the prediction is not successful, the difference between the motion vectors becomes large and the generated code amount increases. In addition, although the amount of motion vector coding is reduced, other motion information such as the prediction direction and the reference image index is encoded for each block to be processed, so that there is a problem that efficient encoding has not been achieved. It was.

In order to solve these problems, as in Patent Document 1, by encoding additional information for specifying an adjacent block to be referred to from among a plurality of adjacent blocks, the motion information of the block to be processed is encoded. Instead, a merge encoding technique is used in which the motion information of adjacent blocks is used for encoding to reduce the code amount.

JP-A-10-276439

Merge encoding reduces the amount of code and reduces the coding efficiency by encoding using the motion information of spatially and temporally adjacent blocks without encoding the motion information of the block to be processed. Has improved. Also, a plurality of adjacent block candidates to be referred to are prepared, and additional information specifying the adjacent block to be referenced is encoded, so that an adjacent block closer to the motion information of the encoding target block can be referred to.

Therefore, if the reference adjacent block candidate includes a plurality of adjacent blocks having the same motion information, the types of motion information that can be expressed by the reference adjacent block candidate are reduced, and as a result, the encoding efficiency is not improved. Therefore, when the motion information of the reference adjacent block candidates is compared in detail and the reference adjacent block candidates having the same motion information are reduced, the processing amount increases.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technique for reducing the processing load and improving the encoding efficiency of motion information when encoding is performed using the motion information of adjacent blocks. It is to provide.

In order to solve the above problems, a moving picture decoding apparatus according to an aspect of the present invention is a moving picture decoding apparatus that decodes a coded code string in units of blocks obtained by dividing each picture of moving picture data. Deriving a plurality of spatial motion information candidates from motion information of a plurality of decoded neighboring blocks at predetermined positions spatially close to the decoding target block, and temporally comparing the picture having the decoding target block to be decoded Deriving motion information of a decoded block included in a different picture, deriving a temporal motion information candidate of the decoding target block from the derived motion information of the decoded block, and deriving the spatial motion information candidate derived A candidate list generation unit for generating a combined motion information candidate list including the temporal motion information candidates; and motion information candidates included in the combined motion information candidate list. A decoding unit that decodes the index to be determined and decodes the decoding target block based on the motion information candidate specified by the index, and the candidate list generation unit compares the spatial motion information candidates in all combinations Without comparing the candidates for the first candidate, the second candidate, and the last candidate in the order of arrangement of the combined motion information candidate list, if the motion information is the same, the motion information is One spatial motion information candidate is derived from the same candidate.
Yet another aspect of the present invention is a video decoding method. This method is a moving picture decoding method for decoding a coded code string in units of blocks into which each picture of moving picture data is divided, and has been decoded at a predetermined position spatially close to the decoding target block. Deriving a plurality of spatial motion information candidates from motion information of a plurality of neighboring blocks, deriving motion information of a decoded block included in a picture temporally different from a picture having a decoding target block to be decoded, A temporal motion information candidate of the decoding target block is derived from the derived motion information of the decoded block, and a combined motion information candidate list including the derived spatial motion information candidate and the temporal motion information candidate is generated. A candidate list generating step; and decoding an index for designating motion information candidates included in the combined motion information candidate list; A decoding step of decoding the decoding target block based on the motion information candidate specified by the candidate motion information candidate list, wherein the candidate list generation step includes the combination motion information candidate list without comparing the spatial motion information candidates in all combinations. For the first candidate, the second candidate and the last candidate of the arrangement order of each, the candidates are compared, and if the motion information is the same, the motion information is the same from the candidates with the same motion information Spatial motion information candidates are derived.
Yet another aspect of the present invention is a moving image decoding program. This program is a moving picture decoding program for decoding a coded code sequence in units of blocks obtained by dividing each picture of moving picture data, and has been decoded at a predetermined position spatially close to the decoding target block Deriving a plurality of spatial motion information candidates from motion information of a plurality of neighboring blocks, deriving motion information of a decoded block included in a picture temporally different from a picture having a decoding target block to be decoded, A temporal motion information candidate of the decoding target block is derived from the derived motion information of the decoded block, and a combined motion information candidate list including the derived spatial motion information candidate and the temporal motion information candidate is generated. A candidate list generation step;
Decoding an index specifying motion information candidates included in the combined motion information candidate list;
The computer executes a decoding step of decoding the decoding target block based on the motion information candidate specified by the index, and the candidate list generation step includes the combination without comparing the spatial motion information candidates in all combinations. For the first candidate, the second candidate, and the last candidate in the order of arrangement in the motion information candidate list, the respective candidates are compared with each other, and if the motion information is the same, the motion information is from the same candidate Derives one spatial motion information candidate.

It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, and the like are also effective as an aspect of the present invention.

ADVANTAGE OF THE INVENTION According to this invention, the encoding efficiency of motion information can be improved, suppressing the load at the time of processing motion information.

It is a figure which shows the structure of the moving image encoder which concerns on Embodiment 1 of this invention. It is a figure which shows an example of an encoding target image. It is a figure which shows the detailed definition of prediction block size. 4A to 4D are diagrams for explaining the prediction direction of motion compensation prediction. It is a flowchart which shows the flow of the operation | movement of the encoding process in the moving image encoder which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the moving image decoding apparatus which concerns on Embodiment 1 of this invention. It is a flowchart which shows the flow of operation | movement of the decoding process in the moving image decoding apparatus which concerns on Embodiment 1 of this invention. FIGS. 8A and 8B are diagrams illustrating two prediction modes for encoding motion information used in motion compensation prediction in Embodiment 1 of the present invention. FIG. 5 is a diagram illustrating a detailed configuration of a prediction mode determination unit in the video encoding device according to the first embodiment. FIG. 10 is a diagram showing a configuration of a combined motion information calculation unit in FIG. 9 in the first embodiment. It is a flowchart explaining the detailed operation | movement of the motion compensation prediction mode / predicted signal generation process of step S502 of FIG. 12 is a flowchart for explaining a detailed operation of the combined motion information candidate list generation of FIG. 11 in the first embodiment. It is a figure which shows the space candidate block group used for a space joint motion information candidate list generation. It is a flowchart explaining the detailed operation | movement of a space joint motion information candidate list generation. 5 is a flowchart for explaining detailed operation of combined motion information candidate deletion in the first embodiment. FIG. 10 is a diagram illustrating a comparison relationship of motion information candidates in combined motion information candidate deletion in the first embodiment. FIG. 10 is a diagram illustrating a comparison relationship of motion information candidates in combined motion information candidate deletion in the first embodiment. It is a figure which shows the time candidate block group used for a time joint movement information candidate list generation. It is a flowchart explaining the detailed operation | movement of time coupling | bonding motion information candidate list generation. It is a figure explaining the calculation method of the motion vector value mvL0t and mvL1t registered with respect to L0 prediction and L1 prediction with respect to the reference | standard motion vector value ColMv with respect to time joint motion information. It is a flowchart explaining the detailed operation | movement of a 1st joint motion information candidate list addition part. It is a figure explaining the relationship between the frequency | count of combination inspection, combined motion information candidate M, and combined motion information candidate N. It is a flowchart explaining the detailed operation | movement of a 2nd joint motion information candidate list addition part. It is a flowchart explaining the detailed operation | movement of the joint prediction mode evaluation value production | generation process of FIG. It is a figure which shows the Truncated Unary code sequence when the number of combined motion information candidates is 5. It is a flowchart explaining the detailed operation | movement of the prediction mode evaluation value production | generation process of FIG. 5 is a diagram illustrating a detailed configuration of a motion information decoding unit in the video decoding device in Embodiment 1. FIG. It is a flowchart explaining the detailed operation | movement of the motion information decoding process of FIG. It is a flowchart explaining the detailed operation | movement of the joint prediction motion information decoding process of FIG. It is a flowchart explaining the detailed operation | movement of the prediction motion information decoding process of FIG. 10 is a flowchart for explaining a detailed operation of combined motion information candidate deletion in the second embodiment. FIG. 10 is a diagram illustrating comparison contents of motion information candidates in combined motion information candidate deletion in the second embodiment. FIG. 10 is a diagram showing a comparison relationship of motion information candidates in combined motion information candidate deletion in the second embodiment. FIG. 10 is a diagram showing a comparison relationship of motion information candidates in combined motion information candidate deletion in the second embodiment. 18 is a flowchart for explaining a detailed operation of combined motion information candidate deletion in the third embodiment. FIG. 10 is a diagram showing a comparison relationship of motion information candidates in combined motion information candidate deletion in the third embodiment. FIG. 16 is a diagram showing comparison contents of motion information candidates in combined motion information candidate deletion in the third embodiment. 18 is a flowchart for explaining a detailed operation of combined motion information candidate deletion in the fourth embodiment. FIG. 20 is a diagram illustrating a comparison relationship of motion information candidates in combined motion information candidate deletion in the fourth embodiment. It is a figure which shows the comparison content of the motion information candidate in combined motion information candidate deletion in Embodiment 4. FIG. 29 is a flowchart for explaining detailed operation of combined motion information candidate deletion in the fifth embodiment. FIG. 25 is a diagram showing a comparison relationship of motion information candidates in combined motion information candidate deletion in the fifth embodiment. FIG. 25 is a diagram showing comparison contents of motion information candidates in combined motion information candidate deletion in the fifth embodiment. FIG. 25 is a diagram showing a comparison relationship of motion information candidates in combined motion information candidate deletion in the fifth embodiment. It is a figure explaining the usable condition of block A0. FIG. 10 is a diagram illustrating a configuration of a combined motion information calculation unit in FIG. 9 according to Embodiment 6. 29 is a flowchart for describing detailed operation of combined motion information candidate list generation in the sixth embodiment. FIG. 38 is a diagram illustrating comparison contents of motion information candidates in combined motion information candidate deletion in the sixth embodiment. FIG. 38 is a diagram showing a comparison relationship of motion information candidates in combined motion information candidate deletion in the sixth embodiment.

Hereinafter, a moving picture coding apparatus, a moving picture coding method according to an embodiment of the present invention together with the drawings,
Preferred embodiments of a moving image encoding program, a moving image decoding apparatus, a moving image decoding method, and a moving image decoding program will be described in detail. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted.

(Embodiment 1)
[Overall configuration of video encoding apparatus]
FIG. 1 is a diagram showing a configuration of a moving picture coding apparatus according to Embodiment 1 of the present invention. Less than,
The operation of each part will be described. The video encoding apparatus according to Embodiment 1 includes an input terminal 100.
, Subtraction unit 101, orthogonal transform / quantization unit 102, prediction error encoding unit 103, inverse quantization / inverse transform unit 104, addition unit 105, decoded image memory 106, motion vector detection unit 107, motion compensated prediction unit 108, Prediction mode determination unit 109, motion information encoding unit 110, motion information memory 1
11, a multiplexing unit 112, and an output terminal 113.

The image signal of the prediction block to be encoded is extracted from the image signal input from the input terminal 100 based on the position information and the prediction block size of the prediction block, and the image signal of the prediction block is subtracted by the subtractor 101, the motion vector. This is supplied to the detection unit 107 and the prediction mode determination unit 109.

FIG. 2 is a diagram illustrating an example of an encoding target image. As for the prediction block size according to Embodiment 1, as shown in FIG.
U) Encoding processing is performed in units, and the prediction block is configured by a unit obtained by further dividing the encoding block. The maximum prediction block size is 64 × 64 pixels, which is the same as the encoded block, and the minimum prediction block size is 4 × 4 pixels. The coding block is divided into prediction blocks: non-division (2N × 2N), horizontal / vertical division (N × N), horizontal division only (2N × N)
Dividing only in the vertical direction (N × 2N) is possible. Only in the case of horizontal and vertical division, the further divided prediction block can be divided into prediction blocks hierarchically as encoded blocks, and the hierarchy is expressed by the number of CU divisions.

FIG. 3 is a diagram showing a detailed definition of the predicted block size. There are 13 predicted block sizes from 64 pixels × 64 pixels, which is the maximum predicted block size with 0 CU divisions, to 4 pixels × 4 pixels, which is the minimum predicted block size with 3 CU partitions. Will exist.

Regarding the division | segmentation structure of the prediction block which concerns on Embodiment 1 of this invention, it is not limited to this combination. In addition, the selection of the prediction block size in the moving image encoding apparatus can adaptively select a structure with better encoding efficiency in units of encoding blocks, but the first embodiment is in units of prediction blocks. Since attention is paid to inter-frame prediction and inter-frame motion information encoding, the components and description relating to selection of the optimal prediction block size are omitted. Regarding the subsequent operation of the video encoding apparatus, the operation performed in units of the selected prediction block size will be described.

Returning to FIG. 1, the subtraction unit 101 subtracts the image signal supplied from the input terminal 100 and the prediction signal supplied from the prediction mode determination unit 109 to calculate a prediction error signal, and orthogonally transforms / quantizes the prediction error signal. To the conversion unit 102.

The orthogonal transform / quantization unit 102 performs orthogonal transform and quantization on the prediction error signal supplied from the subtraction unit 101, and the quantized prediction error signal is subjected to a prediction error encoding unit 103 and an inverse quantization / inverse conversion unit. 104 is supplied.

The prediction error encoding unit 103 entropy-encodes the quantized prediction error signal supplied from the orthogonal transform / quantization unit 102, generates a code string for the prediction error signal, and supplies the code sequence to the multiplexing unit 112 .

The inverse quantization / inverse transform unit 104 performs processing such as inverse quantization and inverse orthogonal transform on the quantized prediction error signal supplied from the orthogonal transform / quantization unit 102, and outputs the decoded prediction error signal. Generated and supplied to the adding unit 105.

The addition unit 105 adds the decoded prediction error signal supplied from the inverse quantization / inverse conversion unit 104 and the prediction signal supplied from the prediction mode determination unit 109, generates a decoded image signal, and outputs the decoded image signal. The decoded image memory 116 is supplied.

The decoded image memory 106 stores the decoded image signal supplied from the adding unit 105. In addition, a decoded image for which decoding of the entire image has been completed is stored as a reference image by a predetermined number of images, and the reference image signal is supplied to the motion vector detection unit 107 and the motion compensation prediction unit 108.

The motion vector detection unit 107 receives the prediction block image signal supplied from the input terminal 100 and the reference image signal stored in the decoded image memory 106, detects the motion vector for each reference image, and detects the motion vector. The value is supplied to the prediction mode determination unit 109.

A general motion vector detection method calculates an error evaluation value for an image signal corresponding to a reference image moved by a predetermined movement amount from the same position as the image signal, and moves the movement amount that minimizes the error evaluation value. Let it be a vector. As the error evaluation value, the sum of the absolute differences SAD for each pixel SAD (
Sum of Absolute Difference), the sum of square error values SSE (Sum of Square Error) for each pixel, and the like are used. Furthermore, the code amount related to the coding of the motion vector can also be included in the error evaluation value.

The motion compensated prediction unit 108 predicts the reference image indicated by the reference image designation information in the decoded image memory 106 by a motion vector value according to the reference image designation information designated by the prediction mode determination unit 109 and the motion vector value. An image signal at a position moved from the same position as the image signal is acquired to generate a prediction signal.

When the prediction mode specified by the prediction mode determination unit 109 is uni-prediction, a prediction signal acquired from one reference image is used as a motion-compensated prediction signal. When the prediction mode is bi-prediction, two reference images are used. A weighted average of the obtained prediction signals is used as a motion compensation prediction signal, and the motion compensation prediction signal is supplied to the prediction mode determination unit 109. Here, the ratio of the weighted average of bi-prediction is set to 1: 1.

4A to 4D are diagrams for explaining the prediction type of motion compensation prediction. A process for performing prediction from a single reference image is defined as single prediction. In the case of single prediction, L0 prediction or L
One of the reference images registered in the two reference image management lists of 1 prediction is used.

FIG. 4A shows a case of single prediction and the reference image (RefL0Pic) for L0 prediction is at a time before the encoding target image (CurPic). FIG. 4B shows a case in which the prediction image is a single prediction and the reference image of the L0 prediction is at a time after the encoding target image. Similarly, the L0 prediction reference image of FIG. 4A and FIG. 4B is replaced with the L1 prediction reference image (RefL1Pi).
It can replace with c) and can perform single prediction.

The process of performing prediction from two reference images is defined as bi-prediction, and in the case of bi-prediction, L0 prediction and L
This is expressed as BI prediction by using both of one prediction. FIG. 4C illustrates a case where bi-prediction is performed, and the reference image for L0 prediction is at a time before the encoding target image and the reference image for L1 prediction is at a time after the encoding target image. . FIG. 4D shows a case of bi-prediction, where the reference image for L0 prediction and the reference image for L1 prediction are at a time before the encoding target image. As described above, the relationship between the prediction type of L0 / L1 and time can be used without being limited to L0 being the past direction and L1 being the future direction.

Returning to FIG. 1, the prediction mode determination unit 109 detects the motion vector value detected for each reference image input from the motion vector detection unit 107 and the motion information stored in the motion information memory 111 (prediction type, motion Based on the vector value and the reference image designation information), the reference image designation information and the motion vector value used for each of the motion compensation prediction modes defined in the first embodiment are set in the motion compensation prediction unit 108. . Based on the set value, an optimal motion compensation prediction mode is determined using the motion compensation prediction signal supplied from the motion compensation prediction unit 108 and the image signal of the prediction block supplied from the input terminal 100.

The prediction mode determination unit 109 includes the determined prediction mode and the prediction type according to the prediction mode,
Information for specifying the motion vector and the reference image designation information is supplied to the motion information encoding unit 110, and the prediction type, the motion vector value, and the reference image designation information for the determined prediction mode and the prediction mode are stored in the motion information memory. 111 and the subtracting unit 101 and the adding unit 1
A prediction signal corresponding to the prediction mode determined in 05 is supplied.

In the moving image encoding apparatus, intra-frame prediction is performed in which prediction is performed using an encoded image in the same screen in order to encode a reference image as a reference. Embodiment 1 focuses on inter-screen prediction. Therefore, the components related to the intra prediction are omitted. Prediction mode determination unit 10
The detailed configuration of 9 will be described later.

The motion information encoding unit 110 encodes the prediction mode supplied from the prediction mode determination unit 109 and information specifying the prediction type, the motion vector, and the reference image designation information according to the prediction mode according to a predetermined syntax structure. Thus, a code string of motion information is generated and supplied to the multiplexing unit 112.

The motion information memory 111 stores motion information (prediction type, motion vector, and reference image index) supplied from the prediction mode determination unit 109 for a predetermined image on the basis of the minimum prediction block size unit. The motion information of the adjacent block of the prediction block to be processed is set as a spatial candidate block group, and the motion information of the block on the ColPic at the same position as the prediction block to be processed and its neighboring blocks is set as the time candidate block group.

ColPic is a decoded image different from the prediction block to be processed, and is stored in the decoded image memory 106 as a reference image. In Embodiment 1, ColPic is a reference image decoded immediately before. In the first embodiment, ColPic is the reference image decoded immediately before, but the reference image immediately before in display order or the reference image immediately after in display order may be used, and the reference image used for ColPic is included in the encoded stream. Direct specification is also possible.

The motion information memory 111 supplies the motion information of the spatial candidate block group and the temporal candidate block group to the prediction mode determination unit 109 as motion information of the candidate block group. The multiplexing unit 112
The prediction error encoding sequence supplied from the prediction error encoding unit 103 and the motion information encoding unit 110
The encoded bit stream is generated by multiplexing the encoded sequence of motion information supplied from the output terminal 113, and the encoded bit stream is output to the recording medium / transmission path via the output terminal 113.

The configuration of the moving image encoding apparatus shown in FIG. 1 is a CPU (Central Processi
ng Unit), a frame memory, a hard disk, and the like.

FIG. 5 is a flowchart showing the flow of the encoding process in the video encoding apparatus according to Embodiment 1 of the present invention. A prediction block image to be processed is acquired from the input terminal 100 for each prediction block unit (S500). The motion vector detection unit 107 calculates a motion vector value for each reference image from the prediction block image to be processed and a plurality of reference images stored in the decoded image memory 106 (S501).

Subsequently, the prediction mode determination unit 109 uses each of the motion vectors supplied from the motion vector detection unit 107 and the motion information stored in the motion information memory 111 to each of the motion compensation prediction modes defined in the first embodiment. Is obtained using the motion compensation prediction unit 108, an optimal prediction mode is selected, and a prediction signal is generated (S502). Step S
Details of the process 502 will be described later.

Subsequently, the subtraction unit 101 calculates a difference between the prediction block image to be processed and the prediction signal supplied from the prediction mode determination unit 109 as a prediction error signal (S503). The motion information encoding unit 110 encodes the prediction mode supplied from the prediction mode determination unit 109 and information specifying the prediction type, the motion vector, and the reference image designation information according to the prediction mode according to a predetermined syntax structure. Then, encoded data of motion information is generated (S504).

Subsequently, the prediction error encoding unit 103 entropy encodes the quantized prediction error signal generated by the orthogonal transform / quantization unit 102 to generate encoded data of the prediction error (S5).
05). The multiplexing unit 112 multiplexes the motion information encoded data supplied from the motion information encoding unit 110 and the prediction error encoded data supplied from the prediction error encoding unit 103 to generate an encoded bitstream. (S506).

The addition unit 105 adds the decoded prediction error signal supplied from the inverse quantization / inverse conversion unit 104 and the prediction signal supplied from the prediction mode determination unit 109 to generate a decoded image signal (S5).
07). The generated decoded image signal is supplied to and stored in the decoded image memory 106 by the adding unit 105, and is used for motion compensation prediction processing of an encoded image to be encoded later (S508).
. The motion information memory 111 stores the motion information (prediction type, motion vector, and reference image designation information) supplied from the motion vector detection unit 107 in units of the minimum predicted block size (S509).

[Overall configuration of video decoding apparatus]
FIG. 6 is a diagram showing a configuration of the moving picture decoding apparatus according to Embodiment 1 of the present invention. Hereinafter, the operation of each unit will be described. The video decoding apparatus according to Embodiment 1 includes an input terminal 600, a demultiplexing unit 601, a prediction difference information decoding unit 602, an inverse quantization / inverse transform unit 603, and an addition unit 604.
A decoded image memory 605, a motion information decoding unit 606, a motion information memory 607, a motion compensation prediction unit 608, and an output terminal 609.

The encoded bit stream is supplied from the input terminal 600 to the demultiplexing unit 601. The demultiplexing unit 601 specifies the code sequence of the supplied encoded bitstream, the encoded sequence of prediction error information, the prediction mode, the prediction type according to the prediction mode, the motion vector, and the reference image designation information Are separated into motion information encoded sequences. The encoded sequence of the prediction error information is supplied to the prediction difference information decoding unit 602, and the encoded sequence of the motion information is converted to the motion information decoding unit 6
06.

The prediction difference information decoding unit 602 decodes the encoded sequence of the prediction error information supplied from the demultiplexing unit 601 and generates a quantized prediction error signal. The prediction difference information decoding unit 602 supplies the generated quantized prediction error signal to the inverse quantization / inverse transform unit 603.

The inverse quantization / inverse transform unit 603 performs a process such as inverse quantization or inverse orthogonal transform on the quantized prediction error signal supplied from the prediction difference information decoding unit 602 to generate a prediction error signal, and performs decoding prediction. The error signal is supplied to the adding unit 604.

The adder 604 adds the decoded prediction error signal supplied from the inverse quantization / inverse transform unit 603 and the prediction signal supplied from the motion compensation prediction unit 608 to generate a decoded image signal, and decodes the decoded image signal This is supplied to the image memory 605.

The decoded image memory 605 has the same function as the decoded image memory 106 in the moving image encoding apparatus in FIG. 1, stores the decoded image signal supplied from the addition unit 604, and stores the reference image signal in the motion compensation prediction unit 608. Supply. Also, the decoded image memory 605 supplies the stored decoded image signal to the output terminal 609 in accordance with the image display order in accordance with the reproduction time.

The motion information decoding unit 606 is based on the encoded sequence of motion information supplied from the demultiplexing unit 601.
Information specifying the prediction mode and the prediction type, motion vector, and reference image designation information corresponding to the prediction mode is decoded as motion information. Decoded motion information and motion information memory 607
The prediction type, the motion vector, and the reference image designation information used for the motion compensation prediction are reproduced from the motion information of the candidate block group supplied more and supplied to the motion compensation prediction unit 608. In addition, the motion information decoding unit 606 supplies the reproduced motion information to the motion information memory 607. A detailed configuration of the motion information decoding unit 606 will be described later.

The motion information memory 607 has the same function as that of the motion information memory 111 in the video encoding apparatus of FIG. 1, and reproduces the reproduced motion information supplied from the motion information decoding unit 606 for a predetermined image on the basis of the minimum prediction block size unit. Remember. Also, the motion information memory 607 supplies the motion information of the space candidate block group and the time candidate block group to the motion information decoding unit 606 as motion information of the candidate block group.

The motion compensation prediction unit 608 has the same function as that of the motion compensation prediction unit 108 in the video encoding apparatus in FIG. 1, and based on the motion information supplied from the motion information decoding unit 606, a reference image in the decoded image memory 605. A prediction signal is generated by acquiring an image signal at a position obtained by moving the reference image indicated by the designation information from the same position as the image signal of the prediction block by the motion vector value. If the prediction type of motion compensation prediction is bi-prediction, an average of the prediction signals of each prediction type is generated as a prediction signal, and the prediction signal is supplied to the adding unit 604.

The output terminal 609 reproduces a decoded image signal by outputting the decoded image signal supplied from the decoded image memory 605 to a display medium such as a display.

The configuration of the video decoding device shown in FIG. 6 can also be realized by hardware such as an information processing device including a CPU, a frame memory, a hard disk, etc., similarly to the configuration of the video encoding device shown in FIG. is there.

FIG. 7 is a flowchart showing a flow of decoding processing in the video decoding apparatus according to Embodiment 1 of the present invention. The demultiplexing unit 601 separates the encoded bit stream supplied from the input terminal 600 into an encoded sequence of prediction error information and an encoded sequence of motion information (S70).
0). The separated coded sequence of motion information is supplied to the motion information decoding unit 606, and the motion information of the decoding target block is decoded using the motion information of the candidate block group supplied from the motion information memory 607 (S701). Details of the processing in step S701 will be described later.

The separated coded sequence of prediction error information is supplied to the prediction difference information decoding unit 602, decoded as a quantized prediction error signal, and dequantized or inverse orthogonal transformed by an inverse quantization / inverse transformation unit 603. By performing the above process, a decoded prediction error signal is generated (S702).

The motion information decoding unit 606 supplies the motion information of the decoding target block to the motion compensation prediction unit 608, and the motion compensation prediction unit 608 performs motion compensation prediction according to the motion information to calculate a prediction signal (S703). The adding unit 604 adds the decoded prediction error signal supplied from the inverse quantization / inverse transform unit 603 and the prediction signal supplied from the motion compensation prediction unit 608 to generate a decoded image signal (S704).

The decoded image signal supplied from the adding unit 604 is stored in the decoded image memory 605 (S
705), the motion information of the decoding target block supplied from the motion information decoding unit 606 is stored in the motion information memory 607 (S706). This completes the decoding process in units of prediction blocks.

[Detailed Function Description of Embodiment 1]
FIG. 5 shows the operation of the prediction mode determination unit 109 of the video encoding apparatus according to Embodiment 1 of the present invention.
The following describes the processing in step S502 in the flowchart, the operation of the motion information decoding unit 606 in the video decoding apparatus according to Embodiment 1 of the present invention, and the detailed operation in step S701 in the flowchart in FIG.

[Definition of Motion Compensation Prediction Mode in Embodiment 1]
FIGS. 8A and 8B are diagrams for explaining two prediction modes for encoding motion information used in motion compensated prediction according to Embodiment 1 of the present invention. In the first prediction mode, the prediction target block directly encodes its own motion information using the continuity of motion in the temporal direction and the spatial direction in the prediction target block and the encoded block adjacent to the prediction target block. In this method, the motion information of spatially and temporally adjacent blocks is used for encoding, which is called a joint prediction mode (merge mode).

In the first prediction mode, the prediction target block directly encodes its own motion information using the continuity of motion in the temporal direction and the spatial direction in the prediction target block and the encoded block adjacent to the prediction target block. In this method, the motion information of spatially and temporally adjacent blocks is used for encoding, which is called a joint prediction mode (merge mode).

Here, spatially adjacent blocks indicate blocks adjacent to the prediction target block among the encoded blocks belonging to the same image as the prediction target block. Here, temporally adjacent blocks are blocks that belong to an encoded image different from the prediction target block.
A block in the same spatial position as the prediction target block and its surroundings.

In the combined prediction mode, motion information that is selectively combined from a plurality of adjacent block candidates can be defined, and the motion information is encoded with information (index) that specifies the adjacent block to be used. The obtained motion information is used as it is for motion compensation prediction. Furthermore,
In the joint prediction mode, a skip mode is defined in which the prediction signal predicted in the joint prediction mode is a decoded picture without encoding and transmitting the prediction difference information, and the decoded image is reproduced with the information having only the combined motion information. It has a configuration that can. The motion information transmitted in the Skip mode is designation information that defines adjacent blocks, as in the combined prediction mode.

The second prediction mode is a technique for individually coding all the components of motion information and transmitting motion information with little prediction error to the prediction block, and is called a motion detection prediction mode. In the motion detection prediction mode, information for specifying a reference image (reference image index) and information for specifying a motion vector are encoded separately, as in the case of encoding motion information in conventional motion compensation prediction. The

The motion detection prediction mode indicates whether to use single prediction or bi-prediction in the prediction mode,
In the case of single prediction / single prediction, a difference vector between information specifying a reference image for one reference image and a motion vector prediction vector is encoded. In the case of bi-prediction, information for specifying reference images for two reference images and a motion vector are individually encoded. The prediction vector for the motion vector is generated from the motion information of the adjacent block similarly to the AVC. However, similarly to the combined prediction mode, the motion vector used for the prediction vector can be selected from a plurality of adjacent block candidates, and the motion vector is the prediction vector. Information that specifies the adjacent block used for (
It is transmitted by encoding the index) and the difference vector.

[Detailed Operation Description of Prediction Mode Determination Unit in Moving Picture Encoding Device in Embodiment 1]
FIG. 9 is a diagram illustrating a detailed configuration of the prediction mode determination unit 109 in the video encoding apparatus according to Embodiment 1. The prediction mode determination unit 109 has a function of determining an optimal motion compensation prediction mode.

The prediction mode determination unit 109 includes a motion compensation prediction generation unit 900, a prediction error calculation unit 901, a prediction vector calculation unit 902, a difference vector calculation unit 903, a motion information code amount calculation unit 904, a prediction mode evaluation unit 905, and a combined motion information calculation. A combined motion compensation prediction generation unit 907.

1, the motion vector value input from the motion vector detection unit 107 is supplied to the motion compensation prediction generation unit 900, and the motion information input from the motion information memory 111 is the prediction vector. The calculation unit 902 and the combined motion information calculation unit 906 are supplied.

Also, the motion compensation prediction generation unit 900 and the combined motion compensation prediction generation unit 907 output reference image designation information and motion vectors used for motion compensation prediction to the motion compensation prediction unit 108, and the motion compensation prediction unit 108. Thus, the generated motion compensated prediction image is converted into a prediction error calculation unit 901.
To be supplied. The prediction error calculation unit 901 is further supplied with an image signal of a prediction block to be encoded from the input terminal 100.

Also, the motion information encoding unit 110 is supplied with motion information to be encoded and the determined prediction mode information from the prediction mode evaluation unit 905, the motion information is supplied to the motion information memory 111, and the motion compensated prediction signal is subtracted. To the unit 101 and the addition unit 105.

The motion compensation prediction generation unit 900 receives the motion vector value calculated for each reference image that can be used for prediction, supplies reference image designation information to the prediction vector calculation unit 902, and the reference image designation information and the motion vector Is output.

The prediction error calculation unit 901 calculates a prediction error evaluation value from the input motion compensated prediction image and the prediction block image to be processed. As the calculation for calculating the error evaluation value, the sum SAD of the absolute difference value for each pixel, the sum SSE of the square error value for each pixel, and the like can be used as in the error evaluation value in motion vector detection. Furthermore, a more accurate error evaluation value can be calculated by taking into account the amount of distortion components generated in the decoded image by performing orthogonal transform / quantization performed when encoding the prediction residual. . In this case, the prediction error calculation unit 901 includes a subtraction unit 101, an orthogonal transformation / quantization unit 102, an inverse quantization / inverse transformation unit 104, and an addition unit 1 in FIG.
This can be realized by having the function 05.

The prediction error calculation unit 901 supplies the prediction error evaluation value calculated in each prediction mode and the motion compensation prediction signal to the prediction mode evaluation unit 905.

The prediction vector calculation unit 902 is supplied with the reference image designation information from the motion compensation prediction generation unit 900, and the motion vector value for the designated reference image from the candidate block group in the motion information of the adjacent block supplied from the motion information memory 111. Are generated together with the prediction vector candidate list, and supplied to the difference vector calculation unit 903 together with the reference image designation information. The prediction vector calculation unit 902 creates prediction vector candidates and registers them as prediction vector candidates.

The difference vector calculation unit 903 calculates the difference between each of the prediction vector candidates supplied from the prediction vector calculation unit 902 and the motion vector value supplied from the motion compensated prediction generation unit 900, and calculates the difference vector value. calculate. When the prediction vector index which is the designation information for the calculated difference vector value and the prediction vector candidate is encoded, the code amount is the smallest. The difference vector calculation unit 903 supplies the prediction vector index and the difference vector value for the prediction vector having the smallest information amount, together with the reference image designation information, to the motion information code amount calculation unit 904.

The motion information code amount calculation unit 904 calculates the code amount required for motion information in each prediction mode from the difference vector value, reference image designation information, prediction vector index, and prediction mode supplied from the difference vector calculation unit 903. . Also, the motion information code amount calculation unit 904
Needs to be transmitted in the combined prediction mode from the combined motion compensation prediction generation unit 907.
The combined motion information index and information indicating the prediction mode are received, and the code amount required for the motion information in the combined prediction mode is calculated.

The motion information code amount calculation unit 904 supplies the motion information calculated in each prediction mode and the code amount required for the motion information to the prediction mode evaluation unit 905.

The prediction mode evaluation unit 905 uses the prediction error evaluation value of each prediction mode supplied from the prediction error calculation unit 901 and the motion information code amount of each prediction mode supplied from the motion information code amount calculation unit 904. An overall motion compensation prediction error evaluation value for the prediction mode is calculated, a prediction mode having the smallest evaluation value is selected, motion information for the selected prediction mode and the selected prediction mode, a motion information encoding unit 110, a motion information memory To 111. Similarly, the prediction mode evaluation unit 905 selects a prediction signal in the selected prediction mode for the motion compensated prediction signal supplied from the prediction error calculation unit 901 and outputs it to the subtraction unit 101 and the addition unit 105.

The combined motion information calculation unit 906 uses a candidate block group in the motion information of the adjacent blocks supplied from the motion information memory 111, a prediction type indicating whether it is uni-prediction or bi-prediction, reference image designation information, motion A plurality of pieces of motion information are generated together with the combined motion information candidate list as motion information composed of vector values, and supplied to the combined motion compensation prediction generation unit 907.

FIG. 10 is a diagram illustrating a configuration of the combined motion information calculation unit 906. Combined motion information calculation unit 90
6 includes a combined spatial motion information candidate list generation unit 1000 and a combined motion information candidate list deletion unit 10.
01, a temporally combined motion information candidate list generating unit 1002, a first combined motion information candidate list adding unit 1003, and a second combined motion information candidate list adding unit 1004. The combined motion information calculation unit 906 creates motion information candidates in a predetermined order from spatially adjacent candidate block groups,
After deleting candidates with the same motion information from them, by adding motion information candidates created from temporally adjacent candidate block groups, only valid motion information is registered as combined motion information candidates. . The point that this temporally combined motion information candidate list generation unit is arranged after the combined motion information candidate list deletion unit is a characteristic configuration of the present embodiment, and deletes the same motion information from temporally combined motion information candidates. By eliminating the processing target, it is possible to reduce the amount of calculation without reducing the encoding efficiency. The detailed operation of the combined motion information calculation unit 906 will be described later.

Returning to FIG. 9, the combined motion compensation prediction generation unit 907 determines, based on the motion information, the prediction type for each registered combined motion information candidate from the combined motion information candidate list supplied from the combined motion information calculation unit 906. In accordance with the reference image designation information and motion vector value of one reference image (uni-prediction) or two reference images (bi-prediction), the motion compensated prediction unit 108 is designated to generate a motion compensated prediction image, The combined motion information index is supplied to the motion information code amount calculation unit 904.

In the configuration of FIG. 9, the prediction mode evaluation for each combined motion information index is performed by the prediction mode evaluation unit 905. The prediction error evaluation value and the motion information code amount are calculated by the prediction error calculation unit 901 and the motion information code amount calculation. Unit motion compensation prediction generation unit 907
It is also possible to take a configuration in which the optimal prediction mode including other prediction modes is evaluated after the combined motion index of the optimal combined motion compensation prediction is determined.

FIG. 11 is a flowchart for explaining detailed operations of the motion compensation prediction mode / prediction signal generation processing in step S502 of FIG. This operation is performed by the prediction mode determination unit 109 in FIG.
The detailed operation in FIG.

First, a combined motion information candidate list is generated (S1100), and a combined prediction mode evaluation value is generated (S1101). Subsequently, a prediction mode evaluation value is generated (S1102), and the optimal evaluation mode is selected by comparing the generated evaluation values (S1103). However, the order of evaluation value generation in steps S1101 and S1102 is not limited to this.

A prediction signal is output in accordance with the selected prediction mode (S1104), and motion information is output in accordance with the selected prediction mode (S1105), thereby completing the motion compensation prediction mode / prediction signal generation process for each prediction block. Detailed operations in steps S1100, S1101, and S1102 will be described later.

FIG. 12 is a flowchart for explaining the detailed operation of generating the combined motion information candidate list in step S1100 of FIG. This operation shows the detailed operation of the configuration in the combined motion information calculation unit 906 in FIG.

The spatially coupled motion information candidate list generation unit 1000 in FIG. 10 performs spatially coupled motion from candidate blocks excluding candidate blocks outside the region or candidate blocks in the intra mode from the spatial candidate block group supplied from the motion information memory 111. An information candidate list is generated (S120
0). Detailed operations for generating the spatially coupled motion information candidate list will be described later.

Subsequently, the combined motion information candidate list deletion unit 1001 deletes the combined motion information candidate having the same motion information from the generated spatial combined motion information candidate list and updates the motion information candidate list (S1201). Detailed operation of the combined motion information candidate deletion will be described later.

The temporally combined motion information candidate list generation unit 1002 subsequently extracts temporally combined motion information from candidate blocks excluding candidate blocks outside the region from the temporal candidate block group supplied from the motion information memory 111 and candidate blocks that are in the intra mode. A candidate list is generated (S1202)
The combined motion information candidate list is combined with the time combined motion information candidate list. Detailed operation of the time combination motion information candidate list generation will be described later.

Next, the first combined motion information candidate list adding unit 1003 adds 0 to the combined motion information candidates registered in the combined motion information candidate list generated by the temporal combined motion information candidate list generating unit 1002.
1 to 4 first supplemental combined motion information candidates are generated and added to the combined motion information candidate list (S
1203), the combined motion information candidate list is added to the second combined motion information candidate list adding unit 1004.
To supply. The detailed operation of adding the first combined motion information candidate list will be described later.

Next, the second combined motion information candidate list adding unit 1004 includes 0 to 2 second supplemental combined motion information candidates that do not depend on the combined motion information candidate list supplied from the first combined motion information candidate list adding unit 1003. Is added to the combined motion information candidate list supplied from the first combined motion information candidate list adding unit 1003 (S1204), and the process ends. Detailed operations for adding the second combined motion information candidate list will be described later.

The candidate block group of motion information supplied from the motion information memory 111 to the combined motion information calculation unit 906 includes a spatial candidate block group and a temporal candidate block group. First, generation of a spatially coupled motion information candidate list will be described.

FIG. 13 is a diagram illustrating a spatial candidate block group used for generating a spatially coupled motion information candidate list. The spatial candidate block group indicates a block of the same image adjacent to the prediction target block of the encoding target image. The block group is managed in units of the minimum prediction block size, and the position of the candidate block is managed in units of the minimum prediction block size, but when the prediction block size of the adjacent block is larger than the minimum prediction block size The same motion information is stored in all candidate blocks within the predicted block size. In the first embodiment, among adjacent block groups, block A0, block A1, block B0, as shown in FIG.
Five blocks of block B1 and block B2 are set as a space candidate block group.

FIG. 14 is a flowchart for explaining the detailed operation of generating the spatially coupled motion information candidate list. Of the five candidate blocks included in the spatial candidate block group, the following processing is repeated for block A0, block A1, block B0, block B1, and block B2 in the order of block A1, block B1, block B0, and block A0. (S1400
~ S1403).

First, the validity of the candidate block is checked (S1401). If the candidate block is not out of the region and not in the intra mode, the candidate block is valid. If the candidate block is valid (
S1401: YES), the motion information of the candidate block is added to the spatially coupled motion information candidate list (S1402).

Following the repetitive processing from step S1400 to S1403, when the number of candidates added to the spatial combination motion information candidate list is less than 4 (S1404: YES), the validity of the candidate block B2 is checked (S1405). When the block B2 is not out of the region and is not in the intra mode (S1405: YES), the motion information of the block B2 is added to the spatially coupled motion information candidate list (S1406).

Here, it is assumed that the spatial combination motion information candidate list includes motion information of four or less candidate blocks, but the spatial candidate block group is at least one or more processed blocks adjacent to the prediction block to be processed. The number of spatially coupled motion information candidate lists may be changed depending on the effectiveness of the candidate block, and the present invention is not limited to this.

FIG. 15 is a flowchart for explaining the detailed operation of combined motion information candidate deletion.
The maximum number of combined motion information candidates generated by the spatial combined motion information candidate list creation process is M
If axSpatialCand, then i = MaxSpatialCand-1 to i>
The following processing is repeated for the combined motion information candidates up to 0 (candidate (i)) (S150).
0-S1506).

If the candidate (i) exists (YES in S1501), the following processing is repeated for the combined motion information candidates (candidate (ii)) from ii = i−1 to ii> = 0 (S1502).
To S1505), if there is no candidate (i) (NO in S1501), step S150
The repetition process for the candidate (ii) from 2 to S1505 is skipped.

First, whether the motion information (motion information (i)) of the candidate (i) and the motion information (motion information (ii)) of the candidate (ii) are the same is checked (S1503), and if they are the same (YES in S1503)
), Candidate (i) is deleted from the combined motion information candidate list (S1504), and the iterative process for candidate (ii) ends.

When the motion information (i) and the motion information (ii) are not the same (NO in S1503), 1 is subtracted from ii, and the process for the candidate (ii) is repeated (S1502 to S1505).

Subsequent to the repetitive processing from step S1500 to S1505, 1 is subtracted from i,
The process for candidate (i) is repeated (S1500 to S1506).

FIG. 16 shows a comparison relationship of candidates in the list when there are four combined motion information candidates. That is, four spatially combined motion information candidates that do not include temporally combined motion information candidates are compared by brute force to determine identity, and duplicate candidates are deleted.

Here, the joint prediction mode uses temporal and spatial continuity of motion, and the prediction target block encodes motion information of spatially and temporally adjacent blocks without directly encoding its own motion information. Although the spatially coupled motion information candidate is based on continuity in the spatial direction, the temporally coupled motion information candidate is generated by the method described later based on the temporal direction continuity. These properties are different. Therefore, it is rare that the same motion information is included in the temporally combined motion information candidate and the spatially combined motion information candidate, and the temporally combined motion information candidate is removed from the target of the combined motion information candidate deletion process for deleting the same motion information. Even if they are excluded, it is rare that the same motion information is included in the finally obtained combined motion information candidate list.

Also, as will be described later, temporally combined motion information candidate blocks are managed in units of minimum spatial prediction blocks having a size larger than that of the minimum prediction block, so that the size of the temporally adjacent prediction blocks is larger than that of the minimum spatial prediction block. If it is small, motion information at a position deviating from the original position is used, and as a result, the motion information often includes an error. Therefore, the motion information is often different from the motion information of the spatially coupled motion information candidate, and there is little influence even if the motion information is excluded from the target of the combined motion information candidate deletion process for deleting the same motion information.

FIG. 17 is an example of comparison contents of candidates in deletion of combined motion information candidates when the maximum number of spatially combined motion information candidates is four. FIG. 17A shows the comparison contents when only the spatially coupled motion information candidate is the target of the coupled motion information candidate deletion process, and FIG. 17B is the target of processing the spatially coupled motion information candidate and the temporally coupled motion information. It is a comparison content in the case of. Since only the spatially combined motion information candidate is the target of the combined motion information candidate deletion process, the number of comparisons of motion information is 1.
Decrease from 0 to 6 times.

Thus, by not deleting the temporally combined motion information candidate as the target of the combined motion information candidate deletion process, the number of motion information comparisons is reduced from 10 to 6 while appropriately deleting the same motion information. Is possible.

Subsequently, generation of a time combination motion information candidate list will be described. FIG. 18 is a diagram illustrating the definition of the temporal direction peripheral prediction block used for generating the temporally combined motion information candidate list. The temporal candidate block group is a decoded image ColPic different from the image to which the prediction target block belongs.
Among the blocks belonging to, blocks at the same position as the prediction target block and in the vicinity thereof are shown. The block group is managed in units of the minimum spatial prediction block size, and the position of the candidate block is managed in units of the minimum spatial prediction block size. In Embodiment 1 of the present invention, the minimum spatial prediction block size is set to a size obtained by doubling the minimum prediction block size in the vertical and horizontal directions. When the size of the prediction block of temporally adjacent blocks is larger than the minimum spatial prediction block size, the same motion information is stored in all candidate blocks within the prediction block size. On the other hand, when the size of the prediction block is smaller than the minimum spatial prediction block size, the motion information of the prediction block located at the upper left of the temporal direction peripheral prediction block is used as the temporal direction peripheral prediction block information. FIG. 18B shows motion information of the temporal direction neighboring prediction block when the prediction block size is smaller than the minimum spatial prediction block size.

A1-A4, B1-B4, C, D, E, F1-F4, G1-G4 in FIG.
, H, and I1 to I16 are blocks adjacent in time. In the first embodiment, among these temporally adjacent block groups, the temporal candidate block group is assumed to be two blocks, block H and block I6.

FIG. 19 is a flowchart for explaining the detailed operation of generating the time combination motion information candidate list. Regarding the block H and the block I6 which are two candidate blocks included in the time candidate block group (S1800, S1805), the validity of the candidate block is checked in the order of the block H and the block I6 (S1801). If the candidate block is valid (S1801)
: YES), the processing of steps S1802 to S1804 is performed, the generated motion information is registered in the temporally combined motion information candidate list, and the processing ends. When the candidate block indicates a position outside the screen area, or when the candidate block is an intra prediction block (S1801)
: NO), the candidate block is not valid, and valid / invalid determination of the next candidate block is performed.
If the candidate block is valid (S1801: YES), the reference image selection candidate to be registered in the combined motion information candidate is determined based on the motion information of the candidate block (S1802). In the first embodiment, the L0 prediction reference image is the reference image that is the closest to the processing target image among the L0 prediction reference images, and the L1 prediction reference image is the processing target image among the L1 prediction reference images. The reference image is the closest distance.

The determination method of the reference image selection candidate here is not limited to this as long as the reference image for L0 prediction and the reference image for L1 prediction can be determined. The reference image intended at the time of encoding can be determined by determining the reference image by the same method in the encoding process and the decoding process. Other deterministic methods include
For example, a method of selecting a reference image having a reference image index of 0 for a reference image for L0 prediction and a reference image for L1 prediction, and a L0 reference image and an L1 reference image used by spatial adjacent blocks are most frequently used. It is possible to use a method of selecting a reference image as a reference image in a prediction target block, or a method of specifying a reference image of each prediction type in an encoded stream.

Next, a motion vector value to be registered in the combined motion information candidate is determined based on the motion information of the candidate block (S1803). In the first embodiment, the temporally coupled motion information calculates bi-prediction motion information based on motion vector values that are effective prediction types in motion information of candidate blocks. When the prediction type of the candidate block is L0 prediction or L1 prediction single prediction, motion information of the prediction type (L0 prediction or L1 prediction) used for prediction is selected, and its reference image designation information and motion vector value are selected. Is a reference value for generating bi-predictive motion information.

If the prediction type of the candidate block is bi-prediction, motion information of either L0 prediction or L1 prediction is selected as a reference value. The selection method of the reference value is, for example, selecting motion information that exists in the same prediction type as ColPic, selecting the reference image of the candidate block that has a shorter inter-image distance from ColPic in the L0 prediction and L1 prediction, or For example, it is possible to select a direction in which the motion vectors of the L0 prediction and the L1 prediction of the candidate block intersect with the encoding process target image.

When the motion vector value used as a reference for bi-predictive motion information generation is determined, a motion vector value to be registered in the combined motion information candidate is calculated.

FIG. 20 is a diagram for explaining a calculation method of motion vector values mvL0t and mvL1t registered for L0 prediction and L1 prediction with respect to the reference motion vector value ColMv for temporally coupled motion information.

The distance between images between ColPic for the reference motion vector value ColMv and the reference image that is the target of the motion vector used as a reference for the candidate block is defined as ColDist. L0 prediction, L1
The inter-image distance between each prediction reference image and the processing target image is set to CurrL0Dist, CurrL1.
Let it be Dist. ColMv, ColDist, CurrL0Dist, CurrL1
The motion vector scaled by the distance ratio of Dist is set as a motion vector to be registered for each. Specifically, the motion vector values mvL0t and mvL1t to be registered are expressed by the following formula 1,
2 is calculated.
mvL0t = mvCol × CurrL0Dist / ColDist (Formula 1)
mvL1t = mvCol × CurrL1Dist / ColDist (Formula 2)
It becomes.

Returning to FIG. 19, the bi-predicted reference image selection information (index) and the motion vector value generated in this way are added to the combined motion information candidates (S1804), and the temporal combined motion information candidate list creation process ends. To do.

Next, a detailed operation of the first combined motion information candidate list adding unit 1003 will be described.
FIG. 21 is a flowchart for explaining the operation of the first combined motion information candidate list adding unit 1003. First, from the number of combined motion information candidates (NumCandList) and the maximum number of combined motion information candidates (MaxNumMergeCand) registered in the combined motion information candidate list supplied from the temporally combined motion information candidate list generation unit 1002, the first additional combination is performed. MaxNumGenCand, which is the maximum number of motion information candidate generations, is calculated from Equation 3 (S200).
0).
MaxNumGenCand = MaxNumMergeCand-NumCandList; (NumCandList> 1)
MaxNumGenCand = 0; (NumCandList <= 1) (Formula 3)

Next, it is inspected whether MaxNumGenCand is larger than 0 (S2001). Max
If NumGenCand is not greater than 0 (NO in S2001), the process ends.
If MaxNumGenCand is greater than 0 (YES in S2001), the following processing is performed. First, loopTimes that is the number of combination inspections is determined. loopTim
es is set to NumCandList × NumCandList. However, loop
If Times exceeds 8, loopTimes is limited to 8 (S2002). Here, loopTimes is an integer from 0 to 7. The following processing is repeatedly performed for loopTimes (S2002 to S2009).

A combination of the combined motion information candidate M and the combined motion information candidate N is determined (S2003). Here, the relationship between the number of combination inspections, the combined motion information candidate M, and the combined motion information candidate N will be described.

FIG. 22 is a diagram for explaining the relationship among the number of combination inspections, the combined motion information candidate M, and the combined motion information candidate N. As shown in FIG. 22, M and N are different values, and are set in order of decreasing total value of M and N.

It is checked whether the L0 prediction of the combined motion information candidate M is valid and the L1 prediction of the combined motion information candidate N is valid (S2004). If the L0 prediction of the combined motion information candidate M is valid and the L1 prediction of the combined motion information candidate N is valid (YES in S2004), the reference image and the motion vector of the combined motion information candidate M for the L0 prediction are combined motion information candidates. It is checked whether the reference image of the N L1 prediction is different from the motion vector (S2005). If the L0 prediction of the combined motion information candidate M is not valid and the L1 prediction of the combined motion information candidate N is not valid (NO in S2004), the next combination is processed. If the L0 prediction reference image of the combined motion information candidate M and the L1 prediction reference image of the combined motion information candidate N are different (YES in S2005), the combined motion information of the L0 prediction motion vector of the combined motion information candidate M and the reference image is combined. A combined motion information candidate is generated by combining the motion vector of L1 prediction of candidate N and the reference image (S2006). Here, as the first additional combined motion information candidate, bi-coupled motion information is generated by combining L0 prediction of a certain combined motion information candidate and L1 prediction motion information of a different combined motion information candidate. If the reference image for L0 prediction of the combined motion information candidate M and the reference image for L1 prediction of the combined motion information candidate N are the same (NO in S2005),
Process the following combinations: Subsequent to step S2006, bi-join motion information candidates are added to the joint motion information candidate list (S2007). Subsequent to step S2007, it is checked whether the number of generated double-coupled motion information is MaxNumGenCand (S2008). If the number of generated double coupled motion information is MaxNumGenCand (YES in S2008)
), The process is terminated. If the number of generated double coupled motion information is not MaxNumGenCand (NO in S2008), the next combination is processed.

Here, the first additional combined motion information candidate is set as the L0 prediction motion vector of a certain combined motion information candidate registered in the combined motion information candidate list and the reference image, and the L1 prediction motion vector of another combined motion information candidate. In combination with the reference image, a bi-coupled motion information candidate whose prediction type of motion compensation prediction is bi-prediction is used, but the present invention is not limited to this. For example, the motion vector for the L0 prediction and the motion vector for the L1 prediction of a certain combined motion information candidate registered in the combined motion information candidate list are +
A motion vector of L0 prediction or a motion vector of L1 prediction of a combined motion information candidate registered in the combined motion information candidate list in which the prediction type of motion compensation prediction to which an offset value such as 1 is added is bi-prediction The motion compensation prediction type of the motion compensation prediction obtained by adding an offset value such as +1 to the combined motion information candidate may be a single prediction, or may be arbitrarily combined.

Here, the first additional combined motion information candidate is a combined motion information candidate when there is a slight difference between the motion information of the combined motion information candidate registered in the combined motion information candidate list and the motion information candidate motion to be processed. Coding efficiency can be improved by correcting the motion information of the combined motion information candidates registered in the list to generate effective combined motion information candidates.

Next, the detailed operation of the second combined motion information candidate list adding unit 1004 will be described.
FIG. 23 is a flowchart for explaining the operation of the second combined motion information candidate list adding unit 1004. First, the first addition from the number of combined motion information candidates (NumCandList) and the maximum number of combined motion information candidates (MaxNumMergeCand) registered in the combined motion information candidate list supplied from the first combined motion information candidate list adding unit 1003. MaxNumGenCand, which is the maximum number for generating combined motion information candidates, is calculated from Equation 4 (S220).
0).
MaxNumGenCand = MaxNumMergeCand-NumCandList; (Formula 4)

Next, the following process is repeated MaxNumCand times for i (S220).
1 to S2205). Here, i is an integer from 0 to MaxNumGenCand-1. The second additional combined motion in which the motion vector for L0 prediction is (0,0), the reference index is i, the motion vector for L1 prediction is (0,0), and the prediction type is i for the reference index is bi-prediction. Information candidates are generated (S2202). The second additional combined motion information candidate is added to the combined motion information candidate list (S2203). The next i is processed (S2204).

Here, the second additional combined motion information candidate has a motion vector for L0 prediction of (0, 0), a reference index of i, a motion vector of L1 prediction of (0, 0), and a reference index of i. The combined motion information candidate whose prediction type is bi-prediction was used. This is because, in a general moving image, the frequency of occurrence of combined motion information candidates in which the motion vector for L0 prediction and the motion vector for L1 prediction are (0, 0) is statistically high. The present invention is not limited to this as long as it is a combined motion information candidate that is statistically frequently used without depending on the motion information of the combined motion information candidate registered in the combined motion information candidate list. For example, the motion vectors of L0 prediction and L1 prediction are (0, 0
) Vector values other than () may be used, and the L0 prediction and the L1 prediction may have different reference indices. Alternatively, the second additional combined motion information candidate can be set as an encoded image or motion information with a high occurrence frequency of a part of the encoded image, encoded in an encoded stream, and transmitted. Although the B picture (B slice) has been described here, in the case of the P picture (P slice), the second additional combined motion in which the motion vector for L0 prediction is (0, 0) and the prediction type is L0 prediction. Generate information candidates.

Here, by setting a combined motion information candidate that does not depend on the combined motion information candidate registered in the combined motion information candidate list as the second additional combined motion information candidate, the combined motion information candidate registered in the combined motion information candidate list When the number is zero, it is possible to use the joint prediction mode and improve the encoding efficiency. In addition, when the motion information of the combined motion information candidate registered in the combined motion information candidate list and the motion information candidate motion to be processed are different, by generating a new combined motion information candidate and expanding the range of options, Encoding efficiency can be improved.

FIG. 24 is a flowchart for explaining the detailed operation of the combined prediction mode evaluation value generation process in step S1101 of FIG. This operation shows the detailed operation of the configuration using the combined motion compensation prediction generation unit 908 of FIG.

First, the prediction error evaluation value is set to the maximum value, and the combined motion information index that minimizes the prediction error is initialized (for example, a value outside the list such as −1) (S2300). The number of combined motion information candidate lists generated by the combined motion information candidate list generation process is set to num_of_in.
If dex is assumed, the following processing is repeated for the combined motion information candidates from i = 0 to num_of_index-1 (S2301 to S2309).

First, the motion information stored in the index i is acquired from the combined motion information candidate list (
S2302). Subsequently, a motion information code amount is calculated (S2303). In the joint prediction mode, since only the joint motion information index is encoded, only the joint motion information index becomes the motion information code amount.

In the first embodiment, the code sequence of the combined motion information index is Truncated.
An Unary code string is used. FIG. 25 shows a case where the number of combined motion information candidates is 5
It is a figure which shows a ted Unary code sequence. When the value of the combined motion information index is encoded using the Truncated Unary code string, the smaller the combined motion information index, the smaller the code bit assigned to the combined motion information index. For example, when the number of combined motion information candidates is 5, if the combined motion information index is 1, '1
It is expressed by 2 bits of 0, but if the combined motion information index is 3, 4 of '1110'
Expressed in bits. Here, as described above, the Truncated Unary code string is used for encoding the combined motion information index, but other code string generation methods can be used, and the present invention is not limited to this.

Subsequently, when the prediction type of the motion information is single prediction (S2304: YES), the reference image designation information and the motion vector for one reference image are set in the motion compensation prediction unit 108 in FIG. A prediction block is generated (S2305). When the motion information is not uni-prediction (S2304: NO), reference image designation information and motion vectors for two reference images are set in the motion compensated prediction unit 108 to generate a motion compensated bi-prediction block (S2306).

Subsequently, from the prediction error and the motion information code amount of the motion compensated prediction block and the prediction target block,
A prediction error evaluation value is calculated (S2307). When the prediction error evaluation value is the minimum value, the evaluation value is updated and the prediction error minimum index is updated (S2308).

As a result of comparison of the prediction error evaluation values for all the combined motion information candidates, the selected prediction error minimum index is output together with the prediction error minimum value and the motion compensated prediction block as a combined motion information index used in the combined prediction mode. (S2310) The combined prediction mode evaluation value generation process ends.

FIG. 26 is a flowchart for explaining detailed operation of the prediction mode evaluation value generation processing in step S1102 of FIG.

First, it is determined whether or not the prediction mode is single prediction (S2500). If it is single prediction, the reference image list (LX) to be processed is set as the reference image list used for prediction (S2501). If it is not uni-prediction, it is bi-prediction, so in this case LX is set to L0 (
S2502).

Next, reference image designation information (index) and motion vector values for LX prediction are acquired (S2503). Subsequently, a prediction vector candidate list is generated (S2504), an optimal prediction vector is selected from the prediction vectors, and a difference vector is generated (S2505). It is desirable to select the optimal prediction vector with the least amount of code when the difference vector between the prediction vector and the motion vector to be transmitted is actually encoded. However, the horizontal and vertical components of the difference vector are simply selected. The calculation may be simplified by a method such as selecting one having a small absolute sum.

Subsequently, it is determined again whether or not the prediction mode is single prediction (S2506). If the prediction mode is single prediction, the process proceeds to step S2509. If it is not uni-prediction, that is, if it is bi-prediction, it is determined whether or not the reference list LX to be processed is L1 (S2507). Reference list LX is L1
If so, the process proceeds to step S2509, and if it is not L1, that is, if L0, LX is changed to L1
(S2508), the same processing as the processing from step S2503 to step S2506 is performed.

Subsequently, a motion information code amount is calculated (S2509). In the case of the uni-prediction mode, the motion information to be encoded includes three elements of reference image designation information, a difference vector value, and a prediction vector index for one reference image. In the case of the bi-prediction mode, L0 and L1 The reference image designation information, the difference vector value, and the prediction vector index for the two reference images are a total of six elements, and the total amount of the encoded amount is calculated as the motion information code amount. As a prediction vector index code string generation method according to the present embodiment, a Trunked Unary code string is used in the same manner as the combined motion information index code string.

Subsequently, the reference image designation information and the motion vector for the reference image are set in the motion compensated prediction unit 108 in FIG. 1 to generate a motion compensated prediction block (S2510).

Furthermore, from the prediction error and the motion information code amount of the motion compensated prediction block and the prediction target block,
The prediction error evaluation value is calculated (S2511), the prediction error evaluation value, the reference image designation information, the difference vector value, and the prediction vector index, which are motion information for the reference image, are output together with the motion compensated prediction block (S2512). The mode evaluation value generation process is terminated.

The above process is performed by the prediction mode determination unit 10 in the video encoding apparatus in the first embodiment.
9 detailed operations.

[Detailed Operation Description of Motion Information Decoding Unit in Moving Picture Decoding Device in Embodiment 1]
FIG. 27 shows a motion information decoding unit 606 in the video decoding apparatus according to Embodiment 1 shown in FIG.
It is a figure which shows the detailed structure of these. The motion information decoding unit 606 includes a motion information bitstream decoding unit 2600, a prediction vector calculation unit 2601, a vector addition unit 2602, a motion compensation prediction decoding unit 2603, a combined motion information calculation unit 2604, and a combined motion compensation prediction decoding unit 2605. .

For the motion information decoding unit 606 in FIG. 6, the motion information bitstream input from the demultiplexing unit 601 is supplied to the motion information bitstream decoding unit 2600, and the motion information input from the motion information memory 607 is predicted. This is supplied to the vector calculation unit 2601 and the combined motion information calculation unit 2604.

In addition, the motion compensation prediction prediction unit 608 outputs reference image designation information and motion vectors used for motion compensation prediction from the motion compensation prediction decoding unit 2603 and the combined motion compensation prediction decoding unit 2605, and indicates information indicating the prediction type. The included decoded motion information is stored in the motion information memory 607.

The motion information bitstream decoding unit 2600 generates the motion information corresponding to the transmitted prediction mode and the prediction mode by decoding the motion information bitstream input from the demultiplexing unit 601 according to the encoding syntax. To do. Among the generated motion information, the combined motion information index is supplied to the combined motion compensation prediction decoding unit 2605, the reference image designation information is supplied to the prediction vector calculation unit 2601, and the prediction vector index is the vector addition unit 2.
The difference vector value is supplied to the vector addition unit 2602.

The prediction vector calculation unit 2601 predicts a reference image to be subjected to motion compensation prediction from the motion information of the adjacent block supplied from the motion information memory 607 and the reference image designation information supplied from the motion information bitstream decoding unit 2600. A vector candidate list is generated and supplied to the vector addition unit 2602 together with the reference image designation information. Prediction vector calculation unit 2601
With respect to the above operation, the same operation as the prediction vector calculation unit 902 in FIG. 9 in the moving image encoding apparatus is performed, and the same candidate list as the prediction vector candidate list at the time of encoding is generated.

The vector addition unit 2602 indicates a prediction vector index from the prediction vector candidate list and reference image designation information supplied from the prediction vector calculation unit 2601 and the prediction vector index and difference vector supplied from the motion information bitstream decoding unit 2600. By adding the prediction vector value and the difference vector value registered at the set position, the motion vector value for the reference image to be motion compensated prediction is reproduced. The reproduced motion vector value is supplied to the motion compensated prediction decoding unit 2603 together with the reference image designation information.

The motion compensated prediction decoding unit 2603 is supplied with the reproduced motion vector value and reference image designation information for the reference image from the vector addition unit 2602, and sets the motion vector value and the reference image designation information in the motion compensated prediction unit 608. Thus, a motion compensated prediction signal is generated.

The combined motion information calculation unit 2604 generates a combined motion information candidate list from the motion information of adjacent blocks supplied from the motion information memory 607, and combines the combined motion information candidate list and the combined motion information candidate that is a component in the list. The reference image designation information and the motion vector value are supplied to the combined motion compensation prediction decoding unit 2605.

Regarding the operation of the combined motion information calculation unit 2604, the same operation as that of the combined motion information calculation unit 906 in FIG. 9 in the moving image encoding apparatus is performed, and the same candidate list as the combined motion information candidate list at the time of encoding is generated. Is done.

The combined motion compensated prediction decoding unit 2605 supplies a combined motion information candidate list supplied from the combined motion information calculation unit 2604, reference image designation information of a combined motion information candidate that is a component in the list, a motion vector value, and a motion information bit. The reference image designation information and the motion vector value in the combined motion information candidate list indicated by the combined motion information index are reproduced from the combined motion information index supplied from the stream decoding unit 2600 and set in the motion compensation prediction unit 608. Generate a motion compensated prediction signal.

FIG. 28 is a flowchart for explaining the detailed operation of the motion information decoding process in step S701 of FIG. Motion information bitstream decoding unit 2600 and prediction vector calculation unit 2
The motion information decoding process in step S701 in FIG. 7 is performed by the 601 and the combined motion information calculation unit 2604.

The motion information decoding process is a process of decoding motion information from an encoded bit stream encoded with a specific syntax structure. First, the Skip flag is decoded in a predetermined unit of the encoded block (S2700). Thereafter, processing is performed in units of prediction blocks.

When the Skip flag indicates the Skip mode (S2701: YES), joint prediction motion information decoding is performed (S2702). Detailed processing in step S2702 will be described later.

When it is not the Skip mode (S2701: NO), the merge flag is decoded (S270).
3). If the merge flag indicates 1 (S2704: YES), step S27
The process proceeds to 02 joint prediction motion information decoding.

When the merge flag is not 1 (S2704: NO), the motion prediction flag is decoded (S27).
05), predictive motion information decoding is performed (S2706). Detailed operation of step S2706 will be described later.

FIG. 29 is a flowchart for explaining detailed operation of the joint prediction motion information decoding process in step S2702 of FIG.

First, the combined prediction mode is set as the prediction mode (S2800), and a combined motion information candidate list is generated (S2801). The processing in step S2801 is the same processing as the combined motion information candidate list generation processing in step S1100 of FIG. 11 in the video encoding device.

Next, the combined motion information index is decoded (S2802), and then the motion information stored at the position indicated by the combined motion information index is acquired from the combined motion information candidate list (S2803). The motion information to be acquired includes a prediction type indicating single prediction / bi-prediction, reference image designation information, and a motion vector value.

The generated motion information is stored as motion information in the joint prediction mode (S2804), and is supplied to the joint motion compensation prediction decoding unit 2606.

FIG. 30 is a flowchart for explaining the detailed operation of the prediction motion information decoding process in step S2706 of FIG.

First, it is determined whether or not the prediction type is single prediction (S2900). If it is simple prediction,
A reference image list (LX) to be processed is set as a reference image list used for prediction (
S2901). If it is not uni-prediction, it is bi-prediction. In this case, LX is set to L0 (S
2902).

Next, the reference image designation information is decoded (S2903), and the difference vector value is decoded (S29).
04). Next, a prediction vector candidate list is generated (S2905), and when the prediction vector candidate list is larger than 1 (S2906: YES), the prediction vector index is decoded (S2906).
2907), when the prediction vector candidate list is 1 (S2906: NO), 0 is set to the prediction vector index (S2908).

Here, in step S2905, processing similar to that in step S2504 in the flowchart of FIG. 26 in the video encoding device is performed.

Next, the motion vector value stored at the position indicated by the prediction vector index is acquired from the prediction vector candidate list (S2909). A motion vector is reproduced by adding the decoded difference vector value and motion vector value (S2910).

Subsequently, it is determined again whether or not the prediction type is single prediction (S2911). If the prediction type is single prediction, the process proceeds to step S2914. If it is not uni-prediction, that is, if it is bi-prediction, it is determined whether or not the reference list LX to be processed is L1 (S2912). If the reference list LX is L1, the process proceeds to step S2914, and if it is not L1, that is, if it is L0, LX is set to L1 (S2913), and the same processing as the processing from step S2903 to step S2911 is performed.

Subsequently, as the generated motion information, in the case of single prediction, reference image designation information and motion vector values for one reference image, and in the case of bi-prediction, reference image designation information and motion for two reference images. The vector value is stored as motion information (S2914) and supplied to the motion compensated prediction decoding unit 2603.

In the moving image encoding device and the moving image decoding device according to the first embodiment, in the combined motion information calculation unit, the temporally combined motion information candidate list generation unit is arranged after the combined motion information candidate list deletion unit, and the space By using only combined motion information candidates as targets for combined motion information candidate deletion processing, it becomes possible to appropriately delete combined motion information candidates having the same motion information from the combined motion information candidate list with a small amount of computation. As a result, by adding new combined motion information candidates to the combined motion information candidate list, the types of motion information that can be used in the combined prediction mode are increased, and the coding efficiency can be improved.

Further, in the moving image encoding device and the moving image decoding device according to the first embodiment, the number of times of comparison of motion information is reduced in the motion information candidate list deletion unit, so that the effect of reducing the circuit scale and processing time is also obtained. Have.

(Embodiment 2)
Next, a second embodiment of the present invention will be described. The second embodiment differs from the first embodiment in the operation of deleting combined prediction candidates. Since the configuration and processing other than the combined prediction candidate deletion operation are the same as those in the first embodiment, here, the difference from the first embodiment of the combined motion information list deletion process in the combined motion information candidate list deletion unit 1001 in FIG. Only explained.

FIG. 31 is a flowchart for explaining a detailed operation of combined motion information candidate deletion in the second embodiment. Assuming that the maximum number of combined motion information candidates generated by the spatially combined motion information candidate list creation process (S1200) in the spatially combined motion information candidate list generation unit 1000 is maxSpatialCand, i = maxSpatialCand-1 to i
The following processing is repeated for candidate combined motion information (candidate (i)) up to> 0 (S30).
00-S3004).

The candidate (i) exists (YES in S3001), and the motion information (motion information (i)) of the candidate (i) and the motion information (motion information (i-1)) of the candidate (i-1) are the same. (S30
02), the candidate (i) is deleted from the combined motion information candidate list (S3003).

When the candidate (i) does not exist (NO in S3001), or even if it exists, the motion information (
If i) and motion information (i-1) are not identical (NO in S3002), the process is repeated (S
3004).

Subsequently, a candidate (max SpatialCand-1) exists (Y in S3005).
ES), candidate (maxSpatialCand-1) motion information (motion information (maxSp
If (firstCand-1)) and the first combined motion information candidate (motion information (0)) in the combined motion information candidate list are the same (YES in S3006), the candidate (max SpatialC)
and-1) is deleted (S3007), and the process ends.

When there is no candidate (max SpatialCand-1) (NO in S3005),
Or even if it exists, the motion information (maxSpatialCand-1) and the motion information (0)
Are not identical (NO in S3006), the process is terminated as it is.

FIG. 32 is an example of comparison contents of candidates in combined motion information candidate deletion in Embodiment 2 when the maximum number of spatially combined motion information candidates is 4. As shown in FIG. 32, when the maximum number of spatially coupled motion information candidates is 4, only four comparisons are performed.

FIG. 33 shows a comparison relationship of candidates in the list when there are four combined motion information candidates. That is, for four spatially combined motion information candidates that do not include temporally combined motion information candidates, the candidates that are adjacent in the sequence order of the combined motion information candidate list are compared, and the highest candidate and the lowest candidate are compared. As a result, identity is determined, and duplicate candidates are deleted.

FIG. 34 is a diagram for explaining comparison of neighboring blocks in the combined motion information candidate deletion process when the maximum number of spatially combined motion information candidates is four. FIG. 34 (a) shows blocks A1, B1,
An example in which B0 and A0 are usable is shown. When the blocks A1, B1, B0, and A0 are usable, the combined motion information candidate list created by the above-described spatially coupled motion information candidate list creation process is {A1, B1, B0, A0}. The comparison of the motion information in the combined motion information candidate deletion is performed in the relationship of the arrows in FIG.

Similarly, FIG. 34 (b) shows a case where blocks B1, B0, A0, and B2 are usable.
4 (c) shows a relationship of motion information comparison in combined motion information candidate deletion when the blocks A1, B1, B0, and B2 are usable.

Here, since the motion information is strongly continuous in the spatial direction, the probability that the motion information is the same in adjacent blocks is high, while the probability that the motion information is the same in a non-adjacent block is low. Therefore, in the combined motion information candidate deletion process, it is possible to appropriately delete the same motion information even if only comparison between adjacent blocks is performed.

In addition, in order to compare only the motion information of adjacent blocks, a method of comparing motion information at predetermined block positions in the peripheral candidate block group is also conceivable, but in this case, each motion in the combined motion information candidate list is considered. It is necessary to record all the block positions of information candidates, which leads to an increase in circuit scale and complexity of processing.

Therefore, in the combined motion information candidate deletion in Embodiment 2 of the present invention, the combined motion information candidate list is compared in the order added to the combined motion information candidate list regardless of the block position of the combined motion information candidate. By comparing the last candidate and the first candidate in FIG.
As shown in FIG. 4, motion information between adjacent blocks can be compared, and the same motion information can be appropriately deleted with a small number of comparisons.

(Embodiment 3)
Next, a third embodiment of the present invention will be described. The third embodiment differs from the first embodiment in the operation of deleting combined prediction candidates. Since the configuration and processing other than the combined prediction candidate deletion operation are the same as those in the first embodiment, here, the difference from the first embodiment of the combined motion information list deletion process in the combined motion information candidate list deletion unit 1001 in FIG. Only explained.

FIG. 35 is a flowchart for explaining the detailed operation of combined motion information candidate deletion in the third embodiment. The number of combined motion information candidates generated by the spatially combined motion information candidate list creation processing (S1200) in the spatially combined motion information candidate list generation unit 1000 is expressed as nu.
mCand, the combined motion information candidate comparison operation count in the combined motion information candidate deletion operation is n
When umCompare, i = numCompare [numCand] −1 to i
The following processing is repeated until> = 0 (S3300 to S3303).

Here, the number of comparison operations numCompare [i] depends on the value of numCand.
It shall be determined as in 7 (a). Further, the comparison candidate index cur [
i] and reference candidate index ref [i] are determined as shown in FIG.

The combined motion information candidate (candidate (cu
r [i])) motion information (motion information (cur [i])) and candidate (ref [i]) motion information (motion information (ref [i])) are the same (YES in S3301) , Candidate (cu
r [i]) is deleted from the combined motion information candidate list (S3302).

When the motion information (cur [i]) and the motion information (ref [i]) are not the same (S3301)
NO) repeats the process (S3303).

FIG. 36 shows a comparison relationship of candidates in the list when there are four combined motion information candidates. That is, for the four spatially combined motion information candidates that do not include the temporally combined motion information candidates, the candidates included from the top to the third in the order of arrangement in the combined motion information candidate list are compared brute force, and if necessary By comparing the highest candidate with the lowest candidate, the identity is determined, and duplicate candidates are deleted.

As shown in FIG. 37, in the combined prediction candidate deletion operation according to Embodiment 3 of the present invention, the number of motion information comparisons is the combination of combined motion information candidate lists generated by the spatially combined motion information candidate list generation unit. Depending on the number of motion information candidates, the number of times is 0 to 4, and the number of comparisons can be greatly reduced as compared to the case where all motion information is compared.

In addition, since all combinations are compared for the first three combined motion candidates in the combined motion information candidate list, the combined motion information candidates having the same motion information cannot be deleted due to the reduction of the number of comparisons. The same motion information exists only in the second half of the combined motion information candidate list.

As described above, the code sequence of the combined motion information index is Truncated Unar.
Since the y code string is used, if redundant motion information exists in a short small index value of the code string, it leads to a decrease in coding efficiency, but according to the combined motion information candidate deletion method in Embodiment 3 of the present invention, Since the same motion information in the combined motion information candidate list having a small index value can be surely deleted, the coding efficiency of motion information can be improved while reducing the processing load.

(Embodiment 4)
Next, a fourth embodiment of the present invention will be described. The fourth embodiment differs from the second embodiment in the operation of deleting combined prediction candidates. Since the configuration and processing other than the combined prediction candidate deletion operation are the same as those in the second embodiment, here, the difference from the first embodiment of the combined motion information list deletion process in the combined motion information candidate list deletion unit 1001 in FIG. Only explained.

FIG. 38 is a flowchart for explaining detailed operation of combined motion information candidate deletion in the fourth embodiment. First, when the number of combined motion information candidates generated by the spatially combined motion information candidate list creation process (S1200) in the spatially combined motion information candidate list generation unit 1000 is numCand, and the maximum number of combined motion information candidates is maxSpatialCand, If numCand> 0 (YES in S3500), i = maxSpati
The value of the combined motion information candidate (candidate (i)) from alCand-1 to i> = numCand is set as the last combined motion information candidate (candidate (numCand-1)) in the combined motion information candidate list (S3501 to S3503). .

If numCand is 0 or less (NO in S3500), the process is terminated without doing anything.

Next, the following processing is repeated for the combined motion information candidates (candidate (i)) from i = maxSpatialCand-1 to i> 0 (S3504 to S3507).

Candidate (i) motion information (motion information (i)) and candidate (i-1) motion information (motion information (i
-1)) is the same (YES in S3505), the candidate (i) is deleted from the combined motion information candidate list (S3506).

If the motion information (i) and the motion information (i-1) are not the same (NO in S3505), the process is repeated (S3507).

Subsequently, a candidate (maxSpatialCand-1) exists (Y in S3508).
ES), candidate (maxSpatialCand-1) motion information (motion information (maxSp
If (firstCand-1)) and the first combined motion information candidate (motion information (0)) in the combined motion information candidate list are the same (YES in S3509), the candidate (max SpatialC)
and-1) is deleted (S3510), and the process is terminated.

When there is no candidate (max SpatialCand-1) (NO in S3508),
Or even if it exists, the motion information (maxSpatialCand-1) and the motion information (0)
Are not the same (NO in S3509), the process is terminated as it is.

The combined motion information candidate deletion operation according to Embodiment 4 of the present invention is the addition of S3500 to S3503 to the combined motion information candidate deletion operation according to Embodiment 2, and S300.
The difference is that 01 is deleted. By adding S3500 to S3503, the spatially coupled motion information candidate list creation process (S12) in the spatially coupled motion information candidate list generation unit 1000 is performed.
00), the combined motion information candidates at the end of the combined motion information candidates are added up to the maximum number of combined motion information candidates when the number of generated combined motion information candidates is less than the maximum number of spatially combined motion information candidates. ing.

FIG. 39 shows a comparison relationship of candidates in the list when there are four combined motion information candidates. That is, when the number of candidates registered in the combined motion information candidate list is less than the predetermined maximum number of candidates, the lowest candidate in the combined motion information candidate list is copied to fill in the empty candidates, and then the combined motion information candidate By comparing adjacent candidates in the arrangement order of the list, and comparing the highest candidate with the lowest candidate, the identity is determined, and duplicate candidates are deleted. For example, when one candidate is missing, the third candidate is copied fourth, the adjacent candidates are compared, and the top candidate and the bottom candidate are compared, resulting in three This is essentially the same as a brute force comparison of candidates.

FIG. 40 is a diagram for explaining the operation of deleting combined motion information candidates in the fourth embodiment of the present invention. FIG. 40A shows the combined motion information candidates after addition before adding the leading candidate when the number of generated combined motion information candidates is 3 and the maximum number of spatially combined motion information candidates is 4. in this way,
Since the last combined motion information candidate in the list is added at the position of index 3, S3504
The comparison of combined motion information candidates in S3507 is as shown in FIG. 40 (b). When the number of combined motion information candidates generated is less than the maximum number of spatially combined motion information candidates, all combinations are compared. It is possible to delete the combined motion information candidates of the same motion information without fail.

(Embodiment 5)
Next, a fifth embodiment of the present invention will be described. Embodiment 5 differs from Embodiment 1 in the operation of deleting combined prediction candidates. Since the configuration and processing other than the combined prediction candidate deletion operation are the same as those in the first embodiment, here, the difference from the first embodiment of the combined motion information list deletion process in the combined motion information candidate list deletion unit 1001 in FIG. Only explained.

FIG. 41 is a flowchart for explaining detailed operation of combined motion information candidate deletion in the third embodiment. Assuming that the maximum number of combined motion information candidates generated by the spatially combined motion information candidate list creation process (S1200) in the spatially combined motion information candidate list generation unit 1000 is maxSpatialCand, i = maxSpatialCand-1 to i
The following processing is repeated until> = 0 (S3700 to S3704).

Here, according to the value of i, the comparison candidate index cur [i] and the reference candidate index r
The value of ef [i] is determined as shown in FIG.

The combined motion information candidate (candidate (cu
r [i])) exists (YES in S3701), the motion information (motion information (cur [i])) of the candidate (cur [i]) and the motion information (motion of the candidate (ref [i])) Information (ref [
i])) is the same (YES in S3702), the candidate (cur [i]) is deleted from the combined motion information candidate list (S3703).

When there is no candidate (cur [i]) (NO in S3701), or even if it exists, the motion information (cur [i]) and the motion information (ref [i]) are not the same (NO in S3702). The process is repeated (S3004).

FIG. 42 shows a comparison relationship of candidates in the list when there are four combined motion information candidates. That is, for the four spatially combined motion information candidates that do not include the temporally combined motion information candidate, adjacent candidates are compared for the top to third in the order of arrangement in the combined motion information candidate list, and the lowest candidate and the lowest candidate are compared. By comparing the upper candidates and further comparing the lowest candidate with the second candidate, the identity is determined, and duplicate candidates are deleted.

As shown in FIG. 43, in the combined prediction candidate deletion operation according to Embodiment 5 of the present invention, the number of comparisons of motion information is 4, and the number of comparisons is smaller than when all motion information is compared. It can be greatly reduced.

FIG. 44 shows neighboring blocks in the combined prediction candidate deletion process according to Embodiment 5 of the present invention when the maximum number of spatially combined motion information candidates is 4 and block A0 is unusable among spatially combined motion information candidate blocks. It is a figure explaining the comparison of. As shown in FIG. 44, according to the operation of spatially coupled motion information candidate deletion in the fifth embodiment of the present invention, when block A0 is unusable, an appropriate comparison is performed between adjacent blocks. Yes.

FIG. 45 is a diagram showing a usable state of the block A0 in various prediction block sizes and the intra-CU division mode. In FIG. 45, a prediction block that can be used by the block A0 is indicated by ◯, and a prediction block that cannot be used is indicated by ×. As shown in FIG.
Since the frequency at which the block A0 becomes unusable is high, the same motion information can be appropriately deleted at a high frequency by appropriately performing a comparison when the block A0 becomes unusable. It is possible to improve the encoding efficiency of motion information while reducing the load.

(Embodiment 6)
Next, a sixth embodiment of the present invention will be described. The sixth embodiment is different from the third embodiment in the number of comparison operations numCompar in the configuration of the combined motion information calculation unit 906, the detailed operation of the combined motion information candidate list generation in step S1100 in FIG.
e [i], comparison candidate index cur [i], and reference candidate index ref [i]
The value of is different. Since other configurations and processing are the same as those in the third embodiment, here, the configuration of the combined motion information calculation unit 906, the detailed operation of combined motion information candidate list generation in step S1100 of FIG. 11, and the comparison in combined prediction candidate deletion Number of operations numCompare [i
], Only differences between the values of the comparison candidate index cur [i] and the reference candidate index ref [i] from the third embodiment will be described.

FIG. 46 is a diagram showing a configuration of the combined motion information calculation unit 906 according to Embodiment 6 of the present invention. The combined motion information calculation unit 906 includes a spatially combined motion information candidate list generation unit 4000, a temporally combined motion information candidate list generation unit 4002, a combined motion information candidate list deletion unit 4001, a first
Combined motion information candidate list adding unit 4003 and second combined motion information candidate list adding unit 400
4 is included. The combined motion information calculation unit 906 creates motion information candidates in a predetermined order from spatially adjacent candidate block groups, and deletes candidates having the same motion information from the candidates,
By adding motion information candidates created from temporally adjacent candidate block groups, only valid motion information is registered as combined motion information candidates.

FIG. 47 is a flowchart for explaining the detailed operation of generating the combined motion information candidate list in step S1100 of FIG. 11 in the sixth embodiment of the present invention. This operation shows the detailed operation in the sixth embodiment of the present invention having the configuration in the combined motion information calculation unit 906 in FIG.

Operations of the spatially combined motion information candidate list generation (S4200), the temporally combined motion information candidate list generation (S4202), the first combined motion information candidate list addition (S4203), and the second combined motion information candidate list addition (S4204) of FIG. FIG. 12 in Embodiment 3 of the present invention.
Since this is the same as the step assigned with the same number, the description is omitted.

The combined motion information candidate list deletion unit 4001 is a time combined motion information candidate list generation unit 400.
The combined motion information candidate having the same motion information is deleted from the combined motion information candidate list generated in 2 to update the motion information candidate list (S4201). This combined motion information candidate list deletion unit 4001 is arranged after the temporal combined motion information candidate list generation unit 4002 and includes the combined temporal motion information candidate in the combined prediction candidate deletion process in the combined prediction candidate deletion operation described later. It is the characteristic in the actual form 6 of this invention.

Next, detailed operation of combined prediction candidate deletion according to Embodiment 6 of the present invention will be described with reference to FIGS.
8 will be used for explanation. FIG. 35 is a flowchart for explaining the detailed operation of joint prediction candidate deletion in the sixth embodiment of the present invention, which is the same as the detailed operation of joint prediction candidate deletion in the third embodiment of the present invention.

The number of combined prediction candidates included in the combined motion information candidate list generated by the temporal combined motion information candidate list generation unit 4002 is numCand, and the number of comparison operations numCompa.
As shown in FIG. 48A, re [i] is changed according to the value of numCand, and the values of the comparison candidate index cur [i] and the reference candidate index ref [i] are changed according to the value of i as shown in FIG. Determine as follows.

In Embodiment 6 of the present invention, the maximum number of spatially coupled prediction candidates generated by spatially coupled motion information candidate list generating section 4000 is 4, and temporal coupled motion information candidate list generating section 4002 is temporally coupled. Since motion information candidates are generated, the maximum number of combined prediction candidates included in the combined prediction candidate list is 5, but the number of comparison operations numCompare [i] is the value of numCand as shown in FIG. Is 4 when is 5.

That is, when the number of spatial motion information candidates generated by the spatial combination motion information candidate list generation unit 4000 is 4, which is the maximum value, the temporal combination generated by the time combination motion information candidate list generation unit 4002 Since the motion information candidate is not a target of the comparison operation in the combined prediction candidate deletion operation, the combined motion information candidate deletion (S4201) is generated as a time combined motion information candidate list (
The operation performed in the previous stage of S4202), that is, the operation similar to the operation of the combined motion information candidate list generation in the third embodiment of the present invention is performed.

On the other hand, when the number of spatial motion information candidates generated by the spatial combination motion information candidate list generation unit 4000 is less than 4, which is the maximum value, the time generated by the time combination motion information candidate list generation unit 4002 The combined motion information candidate is also subjected to a comparison operation in the combined prediction candidate deletion operation.

FIG. 49 shows a comparison relationship between candidates in the list when there are four spatially coupled motion information candidates and when there are three spatially coupled motion information candidates. That is, when there are four spatially coupled motion information candidates (FIG. 49A), four spatially coupled motion information candidates that do not include temporally coupled motion information candidates are as follows:
For the candidates included in the combined motion information candidate list from the top to the third in the order of arrangement, a brute force comparison is made, and if necessary, the highest candidate and the lowest candidate are compared, thereby identifying the identity. Determine and remove duplicate candidates. When there are three spatially coupled motion information candidates (
FIG. 49 (b)), for the four combined motion information candidates including the temporal combined motion information candidates, the candidates included from the top to the third in the arrangement order of the combined motion information candidate list are compared brute force, Furthermore, the identity is determined by comparing the highest candidate and the lowest candidate as necessary, and duplicate candidates are deleted.

Therefore, when the number of effective spatial motion information candidates is the maximum value, the temporally combined motion information candidates having different characteristics are excluded from the target of the combined motion information candidate deletion process, and the processing load is reduced. If the number of the motion information candidates is less than the maximum value, the combined motion information candidate deletion processing accuracy can be further increased with the same processing load by using the temporally combined motion information candidates as the target of the combined motion information candidate deletion processing. Therefore, it is possible to improve the encoding efficiency of motion information while reducing the processing load.

In the above description, an example in which the method of Embodiment 3 is used as the combined prediction candidate comparison method has been described in Embodiment 6, but in Embodiment 6, the combined prediction candidate comparison method is implemented. Any of the methods of forms 1, 2, 4, and 5 may be applied.

The moving picture encoding apparatus and moving picture decoding apparatus according to the embodiments described above have the following technical features.

[1] In the candidate deletion process in the merge coding technique, the candidate deletion process is performed prior to the generation of the temporal combination motion information candidate, and the temporal combination prediction candidates are excluded from the candidates for the candidate deletion process (FIGS. 10 and 12). Steps S1201 and S1202). As a result, the number of candidate comparison operations in the candidate deletion process can be reduced while maintaining encoding efficiency.

[2] In the candidate deletion process in the merge coding technique, the combined motion information candidates added to the combined motion information candidate list are compared in the order of addition to the list, and the last candidate and the first candidate are further compared. Comparison is made (from steps S3000 to S301 in FIG. 31).
8).

[3] In the candidate deletion process in the merge coding technique, the comparison calculation process for determining the identity of the combined motion information candidates included in the combined motion information candidate list is performed a predetermined number of times for a predetermined index ( 35, steps S3300 to S3303, FIG. 37, steps S3700 to S3704 of FIG. 41, and steps S420 of FIG. 43 and FIG.
0 to S4204, see FIG. 48).

[4] In the candidate deletion process in the merge coding technique, when the number of combined motion information candidates generated in the spatial combined information candidate generation process is less than the maximum value, the combined motion of a predetermined index in the combined motion information candidate list The information candidates are added until the number of combined motion information candidates reaches the maximum value, and then the combined motion information is subjected to a candidate comparison operation (see steps S3501 to S3503 in FIG. 38).

The moving image encoded stream output from the moving image encoding apparatus of the embodiment described above has a specific data format so that it can be decoded according to the encoding method used in the embodiment. Therefore, the moving picture decoding apparatus corresponding to the moving picture encoding apparatus can decode the encoded stream of this specific data format.

When a wired or wireless network is used to exchange an encoded stream between a moving image encoding device and a moving image decoding device, the encoded stream is converted into a data format suitable for the transmission form of the communication path. It may be transmitted. In that case, a video transmission apparatus that converts the encoded stream output from the video encoding apparatus into encoded data in a data format suitable for the transmission form of the communication channel and transmits the encoded data to the network, and receives the encoded data from the network Then, a moving image receiving apparatus that restores the encoded stream and supplies the encoded stream to the moving image decoding apparatus is provided.

The moving image transmitting apparatus is a memory that buffers the encoded stream output from the moving image encoding apparatus, a packet processing unit that packetizes the encoded stream, and transmission that transmits the packetized encoded data via the network. Part. The moving image receiving apparatus generates a coded stream by packetizing the received data, a receiving unit that receives the packetized coded data via a network, a memory that buffers the received coded data, and packet processing. And a packet processing unit provided to the video decoding device.

Further, the above-described processing relating to encoding and decoding can be realized as a transmission, storage, and reception device using hardware, as well as a ROM (Read Only Mem).
ory), firmware stored in a flash memory or the like, or software such as a computer. Recording the firmware program and software program in a computer-readable recording medium and providing the same
It can be provided from a server through a wired or wireless network, or can be provided as terrestrial or satellite digital broadcast data broadcast.

The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

100 input terminals, 101 subtraction unit, 102 orthogonal transform / quantization unit, 103 prediction error coding unit, 104 inverse quantization / inverse transform unit, 105 addition unit, 106 decoded image memory, 107 motion vector detection unit, 108 motion compensation Prediction unit, 109 prediction mode determination unit, 110 motion information encoding unit, 111 motion information memory, 112 multiplexing unit, 113 output terminal, 600 input terminal, 601 demultiplexing unit, 602
Prediction difference information decoding unit, 603 inverse quantization / inverse conversion unit, 604 addition unit, 605 decoded image memory, 606 motion information decoding unit, 607 motion information memory, 608 motion compensation prediction unit, 609 output terminal, 900 motion compensation prediction generation 901 prediction error calculation unit, 902 prediction vector calculation unit, 903 difference vector calculation unit, 904 motion information code amount calculation unit, 905 prediction mode evaluation unit, 906 combined motion information calculation unit,
907 combined motion compensation prediction generation unit 1000 spatial combination motion information candidate list generation unit,
1001 combined motion information candidate list deletion unit, 1002 temporal combined motion information candidate list generation unit, 1003 first combined motion information candidate list addition unit, 1004 second combined motion information candidate list addition unit, 2600 motion information bitstream decoding unit, 2601 Prediction vector calculation unit, 2602 vector addition unit, 2603 motion compensation prediction decoding unit, 26
04 joint motion information calculation unit, 2605 joint motion compensation prediction decoding unit, 4000 spatial joint motion information candidate list generation unit, 4001 joint motion information candidate list deletion unit, 4002
A temporally combined motion information candidate list generating unit; 4003 a first combined motion information candidate list adding unit; and 4004 a second combined motion information candidate list adding unit.

Claims (3)

A moving image decoding apparatus that decodes an encoded code string in units of blocks obtained by dividing each picture of moving image data,
A plurality of spatial motion information candidates are derived from motion information of a plurality of decoded neighboring blocks at predetermined positions spatially adjacent to the decoding target block, and temporally compared with a picture having the decoding target block to be decoded Deriving motion information of a decoded block that a different picture has, deriving temporal motion information candidates of the decoding target block from the derived motion information of the decoded block, and deriving the spatial motion information candidate derived and the A candidate list generation unit for generating a combined motion information candidate list including temporal motion information candidates;
A decoding unit that decodes an index that specifies a motion information candidate included in the combined motion information candidate list, and that decodes the decoding target block based on the motion information candidate specified by the index;
The candidate list generation unit compares the candidates for the first candidate, the second candidate, and the last candidate in the combined order of the combined motion information candidate list without comparing the spatial motion information candidates in all combinations. When the motion information is the same, the moving picture decoding apparatus is characterized in that one spatial motion information candidate is derived from candidates having the same motion information. A moving image decoding method for decoding an encoded code string in units of blocks obtained by dividing each picture of moving image data,
A plurality of spatial motion information candidates are derived from motion information of a plurality of decoded neighboring blocks at predetermined positions spatially adjacent to the decoding target block, and temporally compared with a picture having the decoding target block to be decoded Deriving motion information of a decoded block that a different picture has, deriving temporal motion information candidates of the decoding target block from the derived motion information of the decoded block, and deriving the spatial motion information candidate derived and the A candidate list generating step for generating a combined motion information candidate list including temporal motion information candidates;
A decoding step of decoding an index specifying a motion information candidate included in the combined motion information candidate list, and decoding the decoding target block based on the motion information candidate specified by the index;
In the candidate list generation step, the spatial motion information candidates are not compared in all combinations, and the order of the combined motion information candidate list is the first candidate, the second candidate, and the last candidate. When the motion information is the same, a single spatial motion information candidate is derived from candidates having the same motion information. A moving picture decoding program for decoding an encoded code string in units of blocks obtained by dividing each picture of moving picture data,
A plurality of spatial motion information candidates are derived from motion information of a plurality of decoded neighboring blocks at predetermined positions spatially adjacent to the decoding target block, and temporally compared with a picture having the decoding target block to be decoded Deriving motion information of a decoded block that a different picture has, deriving temporal motion information candidates of the decoding target block from the derived motion information of the decoded block, and deriving the spatial motion information candidate derived and the A candidate list generating step for generating a combined motion information candidate list including temporal motion information candidates;
Decoding an index specifying a motion information candidate included in the combined motion information candidate list, and causing the computer to execute a decoding step of decoding the decoding target block based on the motion information candidate specified by the index;
In the candidate list generation step, the spatial motion information candidates are not compared in all combinations, and the order of the combined motion information candidate list is the first candidate, the second candidate, and the last candidate. When the motion information is the same, a moving picture decoding program characterized in that one spatial motion information candidate is derived from candidates having the same motion information.

Images