JP2013106312A - Image encoder, image encoding method and image encoding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve encoding efficiency when motion compensation prediction is used.SOLUTION: A prediction mode determination section 109 can select a mode for using an index of a reference block adjacent to a block of an encoding object and a difference vector between motion vector of the block of the encoding object and that of the reference block as motion information of the block of the encoding object. A prediction error encoding section 103 encodes a prediction error signal being a difference between a prediction signal generated based on the motion information supplied from the prediction mode determination section 109 and an image signal of the block of the encoding object. A motion information encoding section 110 encodes the motion information. When a prediction type acquired from the motion information of the reference block is single prediction, the prediction mode determination section 109 sets the motion vector of the single prediction to be the motion vector of first prediction and performs integral multiplication on the motion vector of the first prediction so as to generate the motion vector of second prediction for generating the motion vector of bi-prediction.

Description

  The present invention relates to a moving image encoding technique using motion compensated prediction, and more particularly to an image encoding device, an image encoding method, and an image encoding program used for motion compensated prediction.

  MPEG-4 AVC / H. In moving picture coding represented by H.264 (hereinafter referred to as AVC) and the like, as information compression using correlation in the time direction, a picture to be coded which is a picture signal to be coded is already coded and decoded. Motion-compensated prediction that uses the local decoded signal as a reference picture, detects a motion amount (hereinafter referred to as a motion vector) between the target picture and the reference picture in a predetermined coding processing unit, and generates a prediction signal. Used.

  In AVC, single prediction for generating a single prediction signal using one motion vector from one reference picture in motion compensated prediction, and two predictions using two motion vectors from two reference pictures. Bi-prediction that generates a signal is used. A method of changing the size of a block to be processed (hereinafter referred to as a prediction target block) within a 16 × 16 pixel two-dimensional block as a predetermined encoding processing unit, or among a plurality of reference pictures By applying this method to the method of selecting a reference picture used for prediction, the information amount of the prediction signal is reduced. On the encoding side, information specifying the prediction mode information and the reference image is selected and transmitted, and on the decoding side, motion compensation prediction processing is performed according to the transmitted prediction mode information and information specifying the reference image.

  For the motion vector, the motion vector of the encoded block adjacent to the processing target block is set as the prediction motion vector (hereinafter referred to as prediction vector), the difference between the motion vector of the processing target block and the prediction vector is obtained, and the difference vector is encoded. The compression efficiency is improved by transmitting as a vector.

  Further, in AVC, a direct motion compensated prediction that realizes motion compensated prediction without transmitting an encoded vector by using a motion vector used for encoding a block of a reference picture at the same position as the prediction target block. It has been known.

  In addition, as a method of not transmitting other encoded vectors, motion compensation prediction that realizes motion compensation prediction using motion information of a block adjacent to a processing target block is known (see, for example, Patent Document 1). .

JP-A-10-276439

  The direct motion compensated prediction described above pays attention to the continuity of motion in the temporal direction in the block of the reference picture located at the same position as the prediction target block, and the motion compensated prediction that does not transmit the coded vector described above is predicted as the prediction target block. By focusing on the continuity of motion in the spatial direction in the encoded block adjacent to the target block, the motion information of other blocks is used as it is. As a result, encoding efficiency is improved without encoding the difference vector as an encoded vector.

  However, when the continuity of motion is not sufficiently maintained or when the motion vector in the motion information of other blocks does not indicate an accurate motion, etc. In this case, a predicted image using the motion information in which the shift has occurred is generated. In that case, there is a difficult aspect that a motion compensated prediction image with high accuracy cannot be generated and the encoding efficiency is not improved.

  In addition, when motion compensation is not performed without encoding a differential motion vector, conventional motion compensated prediction that encodes a differential motion vector is used, but when performing conventional motion compensated prediction, a prediction mode is used. It is necessary to encode information, designation information of the reference image, and a motion vector used for the designated reference image as a difference vector from the prediction vector. In this case, there is a difficult aspect that the amount of codes for specifying the motion compensation prediction method increases and the encoding efficiency is not sufficiently improved.

  The present invention has been made in view of such circumstances, and an object thereof is to provide a technique for improving the coding efficiency when using motion compensation prediction.

  In order to solve the above problems, an image encoding device according to an aspect of the present invention is an image encoding device that detects and encodes a motion vector in units of blocks obtained by dividing each picture of a moving image. Prediction mode capable of selecting a mode using an index of a reference block adjacent to the coding target block and a difference vector between the motion vector of the coding target block and the motion vector of the reference block as motion information of the target block A prediction error signal that is a difference between a prediction signal generated based on motion information supplied from the determination unit (109) and the prediction mode determination unit (109) and an image signal of the encoding target block is encoded. A prediction error signal encoding unit (103) to be encoded and motion information encoding to encode the motion information supplied from the prediction mode determination unit (109) (110) and a multiplexing unit (112) that multiplexes the encoded data supplied from the prediction error signal encoding unit (103) and the encoded data supplied from the motion information encoding unit (110). ). When the prediction type acquired from the motion information of the reference block is uni-prediction, the prediction mode determination unit (109) uses the uni-prediction motion vector as a first prediction in order to generate a bi-prediction motion vector. A motion vector for the second prediction is generated by multiplying the motion vector for the first prediction by an integer.

  Another aspect of the present invention is an image encoding method. This method is an image coding method in which a motion vector is detected and coded in units of blocks obtained by dividing each picture of a moving image, and reference information adjacent to the coding target block is used as motion information of the coding target block. A prediction mode determination step capable of selecting a mode using a block index and a difference vector between the motion vector of the encoding target block and the motion vector of the reference block; and motion information supplied from the prediction mode determination step. A prediction error signal encoding step that encodes a prediction error signal that is a difference between a prediction signal that is originally generated and an image signal of the encoding target block; and motion information supplied from the prediction mode determination step. A motion information encoding step to be encoded, and encoded data supplied from the prediction error signal encoding step. When, and a multiplexing step of multiplexing the encoded data supplied from the motion information encoding step. When the prediction type acquired from the motion information of the reference block is uni-prediction, the prediction mode determination step uses the uni-prediction motion vector as a first-prediction motion vector to generate a bi-prediction motion vector. Then, a motion vector for the second prediction is generated by multiplying the motion vector for the first prediction by an integer.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, and the like are also effective as an aspect of the present invention.

  According to the present invention, it is possible to improve the coding efficiency when using motion compensated prediction.

It is a figure which shows the structure of the moving image encoder which concerns on Embodiment 1 of this invention. It is a figure which shows an example of an encoding target image. It is a figure which shows the detailed definition of prediction block size. 4A to 4D are diagrams for explaining the prediction type of motion compensation prediction. It is a flowchart which shows the flow of the operation | movement of the encoding process in the moving image encoder which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the moving image decoding apparatus which concerns on Embodiment 1 of this invention. It is a flowchart which shows the flow of operation | movement of the decoding process in the moving image decoding apparatus which concerns on Embodiment 1 of this invention. FIGS. 8A and 8B are diagrams for explaining three prediction modes for encoding motion information used in motion compensated prediction according to Embodiment 1 of the present invention. It is a figure for demonstrating the relationship between the number of the predictions of motion compensation, the information which defines a reference image, and the number of the motion vectors to encode in the prediction in motion which does not implement motion compensation prediction and motion compensation prediction. FIG. 3 is a diagram illustrating a detailed configuration of a mode determination unit in the moving image encoding device according to the first embodiment. It is a figure which shows the structure of a prediction vector calculation part. It is a figure which shows the structure of a joint motion information calculation part. It is a figure which shows the structure of a joint motion correction motion compensation prediction production | generation part. FIG. 11 is a flowchart for explaining an operation of motion compensation prediction mode / prediction signal generation, which is step S502 of FIG. 5, which operates via the prediction mode determination unit of FIG. It is a flowchart for demonstrating the detailed operation | movement of combined motion information candidate list generation in step S1400 of FIG. It is a figure which shows the space candidate block group used for a space joint motion information candidate list generation. It is a flowchart for demonstrating the detailed operation | movement of a space joint motion information candidate list generation. It is a figure which shows the time candidate block group used for a time joint movement information candidate list generation. It is a flowchart for demonstrating the detailed operation | movement of time coupling | bonding motion information candidate list generation. It is a figure for demonstrating the calculation method of the motion vector value mvL0t and mvL1t registered with respect to the reference | standard motion vector value ColMv with respect to time coupling | bonding motion information with respect to L0 prediction and L1 prediction. It is a flowchart for demonstrating the detailed operation | movement of combined prediction mode evaluation value generation in step S1401 of FIG. It is a flowchart for demonstrating the detailed operation | movement of combined motion correction prediction mode evaluation value production | generation in step S1402 of FIG. FIGS. 23A to 23C are diagrams illustrating the concept of scaling processing for combined motion information in the combined motion modified prediction mode. It is a figure which shows the number of the motion vectors generated by scaling which exist with respect to each joint motion information. FIGS. 25A to 25C are conceptual diagrams showing the differential motion vector transmission reference image determination algorithm in the first embodiment. It is a flowchart for demonstrating the detailed operation | movement of step S2203 of FIG. It is a flowchart for demonstrating the detailed operation | movement of step S2601 of FIG. It is a flowchart for demonstrating the detailed operation | movement of step S2702 of FIG. It is a flowchart for demonstrating the detailed operation | movement of step S1403 of FIG. It is a flowchart for demonstrating the detailed operation | movement of step S2901 of FIG. It is a flowchart for demonstrating the detailed operation | movement of spatial motion information utilization prediction vector candidate list generation. It is a flowchart for demonstrating the detailed operation | movement of temporal motion information utilization prediction vector candidate list generation. It is a flowchart for demonstrating the detailed operation | movement of step S1404 of FIG. It is a flowchart for demonstrating the detailed operation | movement of step S504 of FIG. 6 is a diagram illustrating an example of a motion prediction flag according to Embodiment 1. FIG. FIG. 35 is a diagram illustrating an example of syntax of an encoded stream generated by motion information encoded data generation processing illustrated in the flowchart of FIG. 34. It is a figure which shows the detailed structure of the motion information decoding part in the moving image decoding apparatus of Embodiment 1 shown in FIG. It is a flowchart for demonstrating the detailed operation | movement of step S701 of FIG. FIG. 39 is a flowchart for explaining detailed operation of step S3802 of FIG. 38. FIG. FIG. 39 is a flowchart for explaining detailed operation of step S3807 of FIG. 38. FIG. FIG. 39 is a flowchart for explaining detailed operation of step S3811 of FIG. 38. FIG. It is a flowchart for demonstrating the detailed operation | movement of step S3810 of FIG. 43 (a) to 43 (d) are diagrams showing the concept of scaling processing for combined motion information in the combined motion modified prediction mode. FIG. 10 is a diagram showing an example of a reference image list in the second embodiment. It is a figure which shows the structure of a joint motion correction motion compensation prediction production | generation part. It is a flowchart for demonstrating the detailed operation | movement of joint motion correction prediction mode evaluation value production | generation using the structure of a joint motion correction motion compensation prediction production | generation part. It is a flowchart for demonstrating the detailed operation | movement of step S701 of FIG. It is a flowchart for demonstrating the detailed operation | movement of step S4711 of FIG. It is a table which obtains a division result from the absolute value of CurrL0Dist and CurrL1Dist.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Preferred embodiments of a moving image encoding apparatus, a moving image encoding method, a moving image encoding program, a moving image decoding apparatus, a moving image decoding method, and a moving image decoding program according to an embodiment of the present invention will be described below with reference to the drawings. A form is demonstrated in detail. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted.

(Embodiment 1)
[Overall configuration of video encoding apparatus]
FIG. 1 is a diagram showing a configuration of a moving picture coding apparatus according to Embodiment 1 of the present invention. Hereinafter, the operation of each unit will be described. The moving picture coding apparatus according to Embodiment 1 includes an input terminal 100, a subtraction unit 101, an orthogonal transformation / quantization unit 102, a prediction error coding unit 103, an inverse quantization / inverse transformation unit 104, an addition unit 105, and a decoding An image memory 106, a motion vector detection unit 107, a motion compensation prediction unit 108, a prediction mode determination unit 109, a motion information encoding unit 110, a motion information memory 111, a multiplexing unit 112, and an output terminal 113 are provided.

  The image signal of the prediction block to be encoded is extracted from the image signal input from the input terminal 100 based on the position information and the prediction block size of the prediction block, and the image signal of the prediction block is subtracted by the subtractor 101, the motion vector. This is supplied to the detection unit 107 and the prediction mode determination unit 109.

  FIG. 2 is a diagram illustrating an example of an encoding target image. Regarding the prediction block size according to the first embodiment, as shown in FIG. 2, the encoding target image is encoded in units of 64 × 64 pixel encoding blocks (CU), and the prediction block is further divided into encoding blocks. Unit. The maximum prediction block size is 64 × 64 pixels, which is the same as the encoded block, and the minimum prediction block size is 4 × 4 pixels. The coding block is divided into prediction blocks: non-division (2N × 2N), horizontal / vertical division (N × N), horizontal only division (2N × N), and vertical division only (N × 2N) is possible. Only in the case of horizontal and vertical division, the further divided prediction block can be divided into prediction blocks hierarchically as encoded blocks, and the hierarchy is expressed by the number of CU divisions.

  FIG. 3 is a diagram showing a detailed definition of the predicted block size. There are 13 predicted block sizes from 64 pixels × 64 pixels, which is the maximum predicted block size with 0 CU divisions, to 4 pixels × 4 pixels, which is the minimum predicted block size with 3 CU partitions. Will exist.

  Regarding the division | segmentation structure of the prediction block which concerns on Embodiment 1 of this invention, it is not limited to this combination. In addition, the selection of the prediction block size in the moving image encoding apparatus can adaptively select a structure with better encoding efficiency in units of encoding blocks, but the first embodiment is in units of prediction blocks. Since attention is paid to inter-frame prediction and inter-frame motion information encoding, the components and description relating to selection of the optimal prediction block size are omitted. Regarding the subsequent operation of the video encoding apparatus, the operation performed in units of the selected prediction block size will be described.

  Returning to FIG. 1, the subtraction unit 101 subtracts the image signal supplied from the input terminal 100 and the prediction signal supplied from the prediction mode determination unit 109 to calculate a prediction error signal, and orthogonally transforms / quantizes the prediction error signal. To the conversion unit 102.

  The orthogonal transform / quantization unit 102 performs orthogonal transform and quantization on the prediction error signal supplied from the subtraction unit 101, and the quantized prediction error signal is subjected to a prediction error encoding unit 103 and an inverse quantization / inverse conversion unit. 104 is supplied.

  The prediction error encoding unit 103 entropy-encodes the quantized prediction error signal supplied from the orthogonal transform / quantization unit 102, generates a code string for the prediction error signal, and supplies the code sequence to the multiplexing unit 112 .

  The inverse quantization / inverse transform unit 104 performs processing such as inverse quantization and inverse orthogonal transform on the quantized prediction error signal supplied from the orthogonal transform / quantization unit 102, and outputs the decoded prediction error signal. Generated and supplied to the adding unit 105.

  The addition unit 105 adds the decoded prediction error signal supplied from the inverse quantization / inverse conversion unit 104 and the prediction signal supplied from the prediction mode determination unit 109, generates a decoded image signal, and outputs the decoded image signal. The decoded image memory 116 is supplied.

  The decoded image memory 106 stores the decoded image signal supplied from the adding unit 105. In addition, a decoded image for which decoding of the entire image has been completed is stored as a reference image by a predetermined number of images, and the reference image signal is supplied to the motion vector detection unit 107 and the motion compensation prediction unit 108.

  The motion vector detection unit 107 receives the prediction block image signal supplied from the input terminal 100 and the reference image signal stored in the decoded image memory 106, detects the motion vector for each reference image, and detects the motion vector. The value is supplied to the prediction mode determination unit 109.

  A general motion vector detection method calculates an error evaluation value for an image signal corresponding to a reference image moved by a predetermined movement amount from the same position as the image signal, and moves the movement amount that minimizes the error evaluation value. Let it be a vector. As the error evaluation value, a sum of absolute differences SAD (Sum of Absolute Difference) for each pixel, a sum of square error values SSE (Sum of Square Error) for each pixel, or the like is used. Furthermore, the code amount related to the coding of the motion vector can also be included in the error evaluation value.

  The motion compensated prediction unit 108 predicts the reference image indicated by the reference image designation information in the decoded image memory 106 by a motion vector value according to the reference image designation information designated by the prediction mode determination unit 109 and the motion vector value. An image signal at a position moved from the same position as the image signal is acquired to generate a prediction signal.

  When the prediction mode specified by the prediction mode determination unit 109 is uni-prediction, a prediction signal acquired from one reference image is used as a motion-compensated prediction signal. When the prediction mode is bi-prediction, two reference images are used. A weighted average of the obtained prediction signals is used as a motion compensation prediction signal, and the motion compensation prediction signal is supplied to the prediction mode determination unit 109. Here, the ratio of the weighted average of bi-prediction is set to 1: 1.

  4A to 4D are diagrams for explaining the prediction type of motion compensation prediction. The process of performing prediction from a single reference image is defined as single prediction. In the case of single prediction, the reference image registered in the two reference image management lists of the reference image list for L0 prediction or the reference image list for L1 prediction is used. Use either one.

  FIG. 4A shows a case of single prediction and the reference image (RefL0Pic) for L0 prediction is at a time before the encoding target image (CurPic). FIG. 4B shows a case in which the prediction image is a single prediction and the reference image of the L0 prediction is at a time after the encoding target image. Similarly, the L0 prediction reference image shown in FIGS. 4A and 4B can be replaced with the L1 prediction reference image (RefL1Pic) to perform single prediction.

  The process of performing prediction from two reference images is defined as bi-prediction. In the case of bi-prediction, both L0 prediction and L1 prediction are used to represent BI prediction. FIG. 4C illustrates a case where bi-prediction is performed, and the reference image for L0 prediction is at a time before the encoding target image and the reference image for L1 prediction is at a time after the encoding target image. . FIG. 4D shows a case of bi-prediction, where the reference image for L0 prediction and the reference image for L1 prediction are at a time before the encoding target image. As described above, the relationship between the prediction of L0 / L1 and time can be used without being limited to L0 being the past direction and L1 being the future direction.

  In the first embodiment, POC (Picture Order Count) is used as information indicating the time of the reference image and the encoding target image. POC is a counter indicating the display order of images defined by AVC. When the image display order is increased by 1, the POC is also increased by 1. Therefore, the time difference (distance) between images can be acquired from the POC difference between images.

  Returning to FIG. 1, the prediction mode determination unit 109 detects the motion vector value detected for each reference image input from the motion vector detection unit 107 and the motion information stored in the motion information memory 111 (prediction type, motion Based on the vector value and the reference image designation information), the reference image designation information and the motion vector value used for each of the motion compensation prediction modes defined in the first embodiment are set in the motion compensation prediction unit 108. . Based on the set value, an optimal motion compensation prediction mode is determined using the motion compensation prediction signal supplied from the motion compensation prediction unit 108 and the image signal of the prediction block supplied from the input terminal 100.

  The prediction mode determination unit 109 supplies information that identifies the determined prediction mode and the prediction type, motion vector, and reference image designation information corresponding to the prediction mode to the motion information encoding unit 110, and determines the determined prediction mode and A prediction type, a motion vector value, and reference image designation information for the prediction mode are supplied to the motion information memory 111, and a prediction signal corresponding to the determined prediction mode is supplied to the subtraction unit 101 and the addition unit 105.

  In the moving image encoding apparatus, intra-frame prediction is performed in which prediction is performed using an encoded image in the same screen in order to encode a reference image as a reference. Embodiment 1 focuses on inter-screen prediction. Therefore, the components related to the intra prediction are omitted. A detailed configuration of the prediction mode determination unit 109 will be described later.

  The motion information encoding unit 110 encodes the prediction mode supplied from the prediction mode determination unit 109 and information specifying the prediction type, the motion vector, and the reference image designation information according to the prediction mode according to a predetermined syntax structure. By doing so, a code string of motion information is generated and supplied to the multiplexing unit 112. A detailed configuration of the motion information encoding unit 110 will be described later.

  The motion information memory 111 stores motion information (prediction type, motion vector, and reference image index) supplied from the prediction mode determination unit 109 for a predetermined image on the basis of the minimum prediction block size unit. The motion information of the adjacent block of the prediction block to be processed is set as a spatial candidate block group, and the motion information of the block on the ColPic at the same position as the prediction block to be processed and its neighboring blocks is set as the time candidate block group.

  ColPic is a decoded image different from the prediction block to be processed, and is stored in the decoded image memory 106 as a reference image. In Embodiment 1, ColPic is a reference image decoded immediately before. In the first embodiment, ColPic is the reference image decoded immediately before, but the reference image immediately before in display order or the reference image immediately after in display order may be used, and the reference image used for ColPic is included in the encoded stream. Direct specification is also possible.

  The motion information memory 111 supplies the motion information of the spatial candidate block group and the temporal candidate block group to the prediction mode determination unit 109 as motion information of the candidate block group. The multiplexing unit 112 multiplexes the prediction error encoded sequence supplied from the prediction error encoding unit 103 and the motion information encoded sequence supplied from the motion information encoding unit 110 to encode the encoded bit stream. And the encoded bit stream is output to the recording medium / transmission path via the output terminal 113.

  The configuration of the moving picture encoding apparatus shown in FIG. 1 can also be realized by hardware such as an information processing apparatus including a CPU (Central Processing Unit), a frame memory, and a hard disk.

  FIG. 5 is a flowchart showing the flow of the encoding process in the video encoding apparatus according to Embodiment 1 of the present invention. A prediction block image to be processed is acquired from the input terminal 100 for each prediction block unit (S500). The motion vector detection unit 107 calculates a motion vector value for each reference image from the prediction block image to be processed and a plurality of reference images stored in the decoded image memory 106 (S501).

  Subsequently, the prediction mode determination unit 109 uses each of the motion vectors supplied from the motion vector detection unit 107 and the motion information stored in the motion information memory 111 to each of the motion compensation prediction modes defined in the first embodiment. Is obtained using the motion compensation prediction unit 108, an optimal prediction mode is selected, and a prediction signal is generated (S502). Details of the processing in step S502 will be described later.

  Subsequently, the subtraction unit 101 calculates a difference between the prediction block image to be processed and the prediction signal supplied from the prediction mode determination unit 109 as a prediction error signal (S503). The motion information encoding unit 110 encodes the prediction mode supplied from the prediction mode determination unit 109 and information specifying the prediction type, the motion vector, and the reference image designation information according to the prediction mode according to a predetermined syntax structure. Then, encoded data of motion information is generated (S504). Details of the processing in step S504 will be described later.

  Subsequently, the prediction error encoding unit 103 entropy encodes the quantized prediction error signal generated by the orthogonal transform / quantization unit 102 to generate encoded data of the prediction error (S505). The multiplexing unit 112 multiplexes the motion information encoded data supplied from the motion information encoding unit 110 and the prediction error encoded data supplied from the prediction error encoding unit 103 to generate an encoded bitstream. (S506).

  The adding unit 105 adds the decoded prediction error signal supplied from the inverse quantization / inverse transform unit 104 and the prediction signal supplied from the prediction mode determination unit 109 to generate a decoded image signal (S507). The generated decoded image signal is supplied to and stored in the decoded image memory 106 by the adding unit 105, and is used for motion compensation prediction processing of an encoded image to be encoded later (S508). The motion information memory 111 stores the motion information (prediction type, motion vector, and reference image designation information) supplied from the motion vector detection unit 107 in units of the minimum predicted block size (S509).

[Overall configuration of video decoding apparatus]
FIG. 6 is a diagram showing a configuration of the moving picture decoding apparatus according to Embodiment 1 of the present invention. Hereinafter, the operation of each unit will be described. The video decoding apparatus according to Embodiment 1 includes an input terminal 600, a demultiplexing unit 601, a prediction difference information decoding unit 602, an inverse quantization / inverse transformation unit 603, an addition unit 604, a decoded image memory 605, and a motion information decoding unit. 606, a motion information memory 607, a motion compensation prediction unit 608, and an output terminal 609.

  The encoded bit stream is supplied from the input terminal 600 to the demultiplexing unit 601. The demultiplexing unit 601 specifies the code sequence of the supplied encoded bitstream, the encoded sequence of the prediction error information, the prediction mode, the prediction type according to the prediction mode, the motion vector, and the reference image designation information. It separates into motion information encoded sequences composed of information. The encoded sequence of the prediction error information is supplied to the prediction difference information decoding unit 602, and the encoded sequence of the motion information is supplied to the motion information decoding unit 606.

  The prediction difference information decoding unit 602 decodes the encoded sequence of the prediction error information supplied from the demultiplexing unit 601 and generates a quantized prediction error signal. The prediction difference information decoding unit 602 supplies the generated quantized prediction error signal to the inverse quantization / inverse transform unit 603.

  The inverse quantization / inverse transform unit 603 performs a process such as inverse quantization or inverse orthogonal transform on the quantized prediction error signal supplied from the prediction difference information decoding unit 602 to generate a prediction error signal, and performs decoding prediction. The error signal is supplied to the adding unit 604.

  The adder 604 adds the decoded prediction error signal supplied from the inverse quantization / inverse transform unit 603 and the prediction signal supplied from the motion compensation prediction unit 608 to generate a decoded image signal, and decodes the decoded image signal This is supplied to the image memory 605.

  The decoded image memory 605 has the same function as the decoded image memory 106 in the moving image encoding apparatus in FIG. 1, stores the decoded image signal supplied from the addition unit 604, and stores the reference image signal in the motion compensation prediction unit 608. Supply. Also, the decoded image memory 605 supplies the stored decoded image signal to the output terminal 609 in accordance with the image display order in accordance with the reproduction time.

  The motion information decoding unit 606 obtains information specifying the prediction mode, the prediction type corresponding to the prediction mode, the motion vector, and the reference image designation information from the motion information encoded sequence supplied from the demultiplexing unit 601. Decrypt as Based on the decoded motion information and the motion information of the candidate block group supplied from the motion information memory 607, the prediction type, motion vector, and reference image designation information used for motion compensation prediction are reproduced and supplied to the motion compensation prediction unit 608. In addition, the motion information decoding unit 606 supplies the reproduced motion information to the motion information memory 607. A detailed configuration of the motion information decoding unit 606 will be described later.

  The motion information memory 607 has the same function as that of the motion information memory 111 in the video encoding apparatus of FIG. 1, and reproduces the reproduced motion information supplied from the motion information decoding unit 606 for a predetermined image on the basis of the minimum prediction block size unit. Remember. Also, the motion information memory 607 supplies the motion information of the space candidate block group and the time candidate block group to the motion information decoding unit 606 as motion information of the candidate block group.

  The motion compensation prediction unit 608 has the same function as that of the motion compensation prediction unit 108 in the video encoding apparatus in FIG. 1, and based on the motion information supplied from the motion information decoding unit 606, a reference image in the decoded image memory 605. A prediction signal is generated by acquiring an image signal at a position obtained by moving the reference image indicated by the designation information from the same position as the image signal of the prediction block by the motion vector value. If the motion compensation prediction is bi-prediction, an average of the prediction signals of the predictions is generated as a prediction signal, and the prediction signal is supplied to the adding unit 604.

  The output terminal 609 reproduces a decoded image signal by outputting the decoded image signal supplied from the decoded image memory 605 to a display medium such as a display.

  The configuration of the video decoding device shown in FIG. 6 can also be realized by hardware such as an information processing device including a CPU, a frame memory, a hard disk, etc., similarly to the configuration of the video encoding device shown in FIG. is there.

  FIG. 7 is a flowchart showing a flow of decoding processing in the video decoding apparatus according to Embodiment 1 of the present invention. The demultiplexing unit 601 separates the coded bit stream supplied from the input terminal 600 into a coded sequence of prediction error information and a coded sequence of motion information (S700). The separated coded sequence of motion information is supplied to the motion information decoding unit 606, and the motion information of the decoding target block is decoded using the motion information of the candidate block group supplied from the motion information memory 607 (S701).

  The separated coded sequence of prediction error information is supplied to the prediction difference information decoding unit 602, decoded as a quantized prediction error signal, and dequantized or inverse orthogonal transformed by an inverse quantization / inverse transformation unit 603. By performing the above process, a decoded prediction error signal is generated (S702). Details of the processing in step S702 will be described later.

  The motion information decoding unit 606 supplies the motion information of the decoding target block to the motion compensation prediction unit 608, and the motion compensation prediction unit 608 performs motion compensation prediction according to the motion information to calculate a prediction signal (S703). The adding unit 604 adds the decoded prediction error signal supplied from the inverse quantization / inverse transform unit 603 and the prediction signal supplied from the motion compensation prediction unit 608 to generate a decoded image signal (S704).

  The decoded image signal supplied from the adding unit 604 is stored in the decoded image memory 605 (S705), and the motion information of the decoding target block supplied from the motion information decoding unit 606 is stored in the motion information memory 607 ( S706). This completes the decoding process in units of prediction blocks.

[Detailed Function Description of Embodiment 1]
Operation of prediction mode determination unit 109 and motion information encoding unit 110 of the video encoding apparatus according to Embodiment 1 of the present invention, processing in steps S502 and S504 in the flowchart of FIG. 5, and embodiment of the present invention The operation of the motion information decoding unit 606 in the moving image decoding apparatus according to 1 and the detailed operation of the processing in step S702 in the flowchart of FIG. 7 will be described below.

[Definition of Motion Compensation Prediction Mode in Embodiment 1]
FIGS. 8A and 8B are diagrams for explaining three prediction modes for encoding motion information used in motion compensated prediction according to Embodiment 1 of the present invention. In the first prediction mode, the prediction target block directly encodes its own motion information using the continuity of motion in the temporal direction and the spatial direction in the prediction target block and the encoded block adjacent to the prediction target block. In this method, the motion information of spatially and temporally adjacent blocks is used for encoding, which is called a joint prediction mode (merge mode).

  In the combined prediction mode, motion information that is selectively combined from a plurality of adjacent block candidates can be defined, and the motion information is encoded with information (index) that specifies the adjacent block to be used. The obtained motion information is used as it is for motion compensation prediction. Further, in the joint prediction mode, a Skip mode is defined in which the prediction signal predicted in the joint prediction mode is a decoded picture without encoding prediction transmission of the prediction difference information, and a decoded image is obtained with information having only the combined motion information. Can be reproduced. The motion information transmitted in the Skip mode is designation information that defines adjacent blocks, as in the combined prediction mode.

  The second prediction mode is a technique for individually coding all the components of motion information and transmitting motion information with little prediction error to the prediction block, and is called a motion detection prediction mode. In the motion detection prediction mode, information for specifying a reference image (reference image index) and information for specifying a motion vector are encoded separately, as in the case of encoding motion information in conventional motion compensation prediction. The

  In the motion detection prediction mode, the prediction mode indicates whether to use single prediction or bi-prediction. In the case of single prediction, information for specifying a reference image for one reference image and a prediction vector of a motion vector Encode the difference vector. In the case of bi-prediction, information for specifying reference images for two reference images and a motion vector are individually encoded. The prediction vector for the motion vector is generated from the motion information of the adjacent block similarly to the AVC. However, similarly to the combined prediction mode, the motion vector used for the prediction vector can be selected from a plurality of adjacent block candidates, and the motion vector is the prediction vector. The information is transmitted by encoding two pieces of information (index) for designating adjacent blocks to be used for and a difference vector.

  In the third prediction mode, a part of the motion information generated from the adjacent block is encoded by additionally encoding a difference vector value for correcting the motion vector value with respect to the motion information of the adjacent block. This is a technique of correcting motion vectors and encoding them as motion information, and is called a combined motion correction prediction mode (merge MVD mode). Similar to the combined prediction mode, the motion information transmitted in the combined motion correction prediction mode is a difference vector value for the motion vector to be corrected in addition to the information specifying the adjacent block. In the combined motion modified prediction mode in the first embodiment, one difference vector value is always transmitted, and the motion compensated prediction is bi-prediction.

  The configuration having the combined motion correction prediction mode is a factor that improves the coding efficiency over the conventional configuration that does not have the combined motion correction prediction mode. That is, in the combined motion correction prediction mode, a difference vector for correcting a motion vector is encoded and added to the information encoded in the combined prediction mode. As a result, when the continuity of motion is not sufficiently maintained, or when the motion vector in the motion information of other blocks does not indicate an accurate motion, the motion vector is generated for the motion information generated from the adjacent block. By correcting only the motion information, motion information for generating a motion compensated prediction signal with a small prediction residual can be expressed with a small amount of information.

  FIG. 9 is a diagram for explaining the relationship between the type of motion compensation, information defining a reference image, and the number of motion vectors to be encoded in motion compensation prediction and intra prediction without performing motion compensation prediction.

  Of the motion detection prediction modes, when the prediction is a single prediction mode (UniPred), one encoded vector, one reference image index, and one prediction vector index are transmitted as motion information to be encoded. .

  Similarly, in the case of the bi-prediction mode (BiPred) in which the prediction is bi-prediction, two encoded vectors, two reference image indexes, and two prediction vector indexes are transmitted as motion information to be encoded.

  In the joint prediction mode (Merge), only one joint motion information index is transmitted as motion information to be encoded, and single prediction or bi-prediction motion compensation prediction is performed based on the motion information.

  In the combined motion correction prediction mode (MergeMvd), one combined motion information index and one encoded vector are transmitted as motion information to be encoded, and bi-prediction is performed according to the motion information and the motion vector correction information. The

  In the intra mode (Intra), which is a predictive coding mode in which motion compensation prediction is not performed, motion information is not coded.

  When temporal and spatial correlation with adjacent motion information is very high, the combined prediction mode is selected as an effective motion compensation prediction mode. When the correlation is low or only a part of the motion information can be effectively used as the motion information of the prediction block, the motion detection prediction mode is selected as an effective motion compensation prediction mode. If the correlation is high but there is a slight shift in the motion, or if the motion information of the adjacent block does not show accurate motion, or most of the motion information is valid, but only some motion information When the correlation is not sufficient, the combined motion correction prediction mode is selected as an effective motion compensation prediction mode. As described above, by providing the combined motion correction prediction mode in addition to the combined prediction mode and the motion detection prediction mode, it is possible to select motion compensated prediction with high coding efficiency according to various motion information correlation situations. .

[Detailed Operation Description of Prediction Mode Determination Unit in Moving Picture Encoding Device in Embodiment 1]
FIG. 10 is a diagram illustrating a detailed configuration of the prediction mode determination unit 109 in the video encoding device according to the first embodiment. The prediction mode determination unit 109 has a function of determining an optimal motion compensation prediction mode.

  The prediction mode determination unit 109 includes a motion compensation single prediction generation unit 1000, a motion compensation bi-prediction generation unit 1001, a prediction error calculation unit 1002, a prediction vector calculation unit 1003, a difference vector calculation unit 1004, a motion information code amount calculation unit 1005, a prediction A mode evaluation unit 1006, a combined motion information calculation unit 1007, a combined motion compensation prediction generation unit 1008, and a combined motion modified motion compensation prediction generation unit 1009 are included.

  The motion vector values input from the motion vector detection unit 107 to the prediction mode determination unit 109 in FIG. 1 are the motion compensation single prediction generation unit 1000, the motion compensation bi-prediction generation unit 1001, and the combined motion correction motion compensation prediction generation. The motion information supplied to the unit 1009 and input from the motion information memory 111 is supplied to the prediction vector calculation unit 1003 and the combined motion information calculation unit 1007.

  In addition, the motion compensation prediction unit 108 receives motion compensation prediction from the motion compensation single prediction generation unit 1000, the motion compensation bi-prediction generation unit 1001, the combined motion compensation prediction generation unit 1008, and the combined motion correction motion compensation prediction generation unit 1009. Reference image designation information and motion vectors used for the are output. Further, the motion compensation prediction image generated by the motion compensation prediction unit 108 is supplied to the prediction error calculation unit 1002. The prediction error calculation unit 1002 is further supplied with an image signal of a prediction block to be encoded from the input terminal 100.

  Also, the motion information encoding unit 110 is supplied with motion information to be encoded and the determined prediction mode information from the prediction mode evaluation unit 1006, the motion information is supplied to the motion information memory 111, and the motion compensated prediction signal is subtracted. To the unit 101 and the addition unit 105.

  The motion compensation single prediction generation unit 1000 receives a motion vector value calculated for each reference image that can be used for single prediction, supplies reference image designation information to the prediction vector calculation unit 1003, and the reference image designation information and The motion vector is output to the motion compensation prediction unit 108.

  Similarly, the motion-compensated bi-prediction generation unit 1001 receives the motion vector value calculated for each reference image that can be used for bi-prediction, and supplies reference image designation information for each reference image to the prediction vector calculation unit 1003. Then, reference image designation information and motion vectors for each reference image are output to the motion compensation prediction unit 108.

  The prediction error calculation unit 1002 calculates a prediction error evaluation value from the input motion compensated prediction image and the prediction block image to be processed. As the calculation for calculating the error evaluation value, the sum SAD of the absolute difference value for each pixel, the sum SSE of the square error value for each pixel, and the like can be used as in the error evaluation value in motion vector detection. Furthermore, a more accurate error evaluation value can be calculated by taking into account the amount of distortion components generated in the decoded image by performing orthogonal transform / quantization performed when encoding the prediction residual. . In this case, the prediction error calculation unit 1002 can be realized by having the functions of the subtraction unit 101, the orthogonal transformation / quantization unit 102, the inverse quantization / inverse transformation unit 104, and the addition unit 105 in FIG. The prediction error calculation unit 1002 supplies the prediction error evaluation value calculated in each prediction mode and the motion compensation prediction signal to the prediction mode evaluation unit 1006.

  The prediction vector calculation unit 1003 is supplied with the reference image designation information from the motion compensation uni-prediction generation unit 1000 and the motion compensation bi-prediction generation unit 1001, and from the candidate block group in the motion information of adjacent blocks supplied from the motion information memory 111, The motion vector value for the specified reference image is specified. Then, a plurality of prediction vectors are generated together with the prediction vector candidate list, and supplied to the difference vector calculation unit 1004 together with the reference image designation information.

  FIG. 11 is a diagram illustrating a configuration of the prediction vector calculation unit 1003. The prediction vector calculation unit 1003 includes a prediction vector candidate list generation unit 1100 and a prediction vector candidate list deletion unit 1101, creates prediction vector candidates in a predetermined order from the candidate block group, and uses the same motion vector value from among them. By deleting the candidates to be shown, only valid prediction vectors are registered as prediction vector candidates. The detailed operation of the prediction vector calculation unit 1003 will be described later.

  Returning to FIG. 10, the difference vector calculation unit 1004 is supplied from the motion compensation single prediction generation unit 1000 and the motion compensation bi-prediction generation unit 1001 for each prediction vector candidate supplied from the prediction vector calculation unit 1003. The difference with the motion vector value is calculated, and the difference vector value is calculated. The difference vector value and the prediction vector index, which is the designation information for the prediction vector candidate, are the difference vectors that can represent the motion vector value with the least amount of information when encoding the prediction vector index. Therefore, the prediction vector index and the difference vector value for the prediction vector having the smallest information amount are supplied to the motion information code amount calculation unit 1005 together with the reference image designation information.

  The motion information code amount calculation unit 1005 supplies motion information in each prediction mode based on the difference vector value, reference image designation information, prediction vector index, and prediction mode supplied from the difference vector calculation unit 1004. The amount of code required for the calculation is calculated. In addition, the motion information code amount calculation unit 1005 receives information indicating a combined motion information index and a prediction mode that needs to be transmitted in the combined prediction mode from the combined motion compensation prediction generation unit 1008, and performs motion in the combined prediction mode. The amount of code required for information is calculated.

  Similarly, the motion information code amount calculation unit 1005 indicates a combined motion information index, a difference vector value, and a prediction mode that need to be transmitted in the combined motion correction prediction prediction mode from the combined motion correction motion compensation prediction generation unit 1009. Information is received, and the amount of code required for motion information in the combined motion correction prediction mode is calculated. The motion information code amount calculation unit 1005 supplies the motion information calculated in each prediction mode and the code amount required for the motion information to the prediction mode evaluation unit 1006.

  The prediction mode evaluation unit 1006 uses the prediction error evaluation value of each prediction mode supplied from the prediction error calculation unit 1002 and the motion information code amount of each prediction mode supplied from the motion information code amount calculation unit 1005. An overall motion compensation prediction error evaluation value of the prediction mode is calculated, a prediction mode having the smallest evaluation value is selected, and the motion information encoding unit 110 and the motion information memory store the selected prediction mode and motion information for the selected prediction mode. To 111. Similarly, the prediction mode evaluation unit 1006 selects a prediction signal in the selected prediction mode for the motion compensated prediction signal supplied from the prediction error calculation unit 1002 and outputs it to the subtraction unit 101 and the addition unit 105.

  The combined motion information calculation unit 1007 uses a candidate block group in the motion information of the adjacent blocks supplied from the motion information memory 111, a prediction type indicating whether the prediction is uni-prediction or bi-prediction, reference image designation information, motion A plurality of pieces of motion information are generated together with the combined motion information candidate list as motion information composed of vector values, and supplied to the combined motion compensation prediction generation unit 1008 and the combined motion correction motion compensation prediction generation unit 1009.

  FIG. 12 is a diagram illustrating a configuration of the combined motion information calculation unit 1007. The combined motion information calculation unit 1007 includes a combined motion information candidate list generation unit 1200 and a combined motion information candidate list deletion unit 1201. The combined motion information calculation unit 1007 creates motion information candidates from the candidate block group in a predetermined order, and from among them, each element of motion information (prediction type, reference image designation information, and motion vector value) is the same. By deleting the candidate indicating the value of, only valid motion information is registered as a combined motion information candidate. The detailed operation of the combined motion information calculation unit 1007 will be described later.

  Returning to FIG. 10, the combined motion compensation prediction generation unit 1008 determines, based on the motion information, the prediction type for each registered combined motion information candidate from the combined motion information candidate list supplied from the combined motion information calculation unit 1007. In accordance with the reference image designation information and motion vector value of one reference image (uni-prediction) or two reference images (bi-prediction), the motion compensated prediction unit 108 is designated to generate a motion compensated prediction image, Are provided to the motion information code amount calculation unit 1005. The detailed operation of the combined motion compensation prediction generation unit 1008 will be described later.

  In the configuration of FIG. 10, the prediction mode evaluation in each combined motion information index is performed by the prediction mode evaluation unit 1006. The prediction error evaluation value and the motion information code amount are calculated by the prediction error calculation unit 1002 and the motion information code amount calculation. In other words, the combined motion compensation prediction generation unit 1008 receives the information from the unit 1005 and, after the combined motion index of the optimal combined motion compensated prediction is determined, evaluates the optimal prediction mode including other prediction modes. Is possible.

  The combined motion corrected motion compensated prediction generation unit 1009 calculates bi-prediction motion information for the combined motion information candidate list and the registered combined motion information candidates supplied from the combined motion information calculation unit 1007. Then, the reference image whose motion vector value is to be corrected is identified, the motion vector value detected by the motion vector detection unit 107 is input to the reference image designation information of the identified reference image, and the difference vector value is obtained. calculate. Then, the motion compensated prediction unit 108 is designated with motion vector values used for motion compensated prediction including the bi-predicted reference image designation information and the corrected motion vector value, and a motion compensated predicted image is generated. The information index and the difference vector value to be transmitted are supplied to the motion information code amount calculation unit 1005.

  FIG. 13 is a diagram illustrating a configuration of the combined motion corrected motion compensation prediction generation unit 1009. The combined motion corrected motion compensation prediction generation unit 1009 includes a reference reference image / motion correction reference image selection unit 1300, a motion correction reference image motion vector acquisition unit 1301, a combined motion information correction motion compensation prediction generation unit 1302, and a difference vector calculation unit 1303. including.

  The reference reference image / motion correction reference image selection unit 1300 has a function of calculating a bi-prediction motion information for a registered combined motion information candidate and specifying a reference image for correcting a motion vector value. The motion correction reference image motion vector acquisition unit 1301 has a function of inputting a motion vector value for the reference image designation information of the identified reference image. The combined motion information modified motion compensation prediction generation unit 1302 has a function of outputting a motion vector value used for motion compensation prediction and reference image designation information to the motion compensation prediction unit 108. The difference vector calculation unit 1303 has a function of calculating a difference vector value between the motion vector value of the motion information calculated by the standard reference image / motion correction reference image selection unit 1300 and the motion vector value input from the motion vector detection unit 107. Have. The detailed operation of the combined motion corrected motion compensation prediction generation unit 1009 will be described later.

  FIG. 14 is a flowchart for explaining the operation of motion compensation prediction mode / prediction signal generation, which is step S502 of FIG. 5, which operates via the prediction mode determination unit 109 of FIG. First, a combined motion information candidate list is generated (S1400), and a combined prediction mode evaluation value is generated (S1401).

  Subsequently, using the combined motion information candidate list generated in step S1400, a combined motion correction prediction mode evaluation value is generated (S1402), a single prediction mode evaluation value is generated (S1403), and a bi-prediction mode evaluation value is calculated. It generates (S1404) and compares the generated evaluation values to select an optimal prediction mode (S1405). However, the order of evaluation value generation in step S1401, step S1402, step S1403, and step S1404 is not limited to this order.

  A prediction signal is output in accordance with the selected prediction mode (S1406), and motion information is output in accordance with the selected prediction mode (S1407), thereby completing the motion compensation prediction mode / prediction signal generation process for each prediction block. Detailed operations of step S1400, step S1401, step S1402, step S1404, and step S1404 will be described later.

  FIG. 15 is a flowchart for explaining the detailed operation of generating the combined motion information candidate list in step S1400 of FIG. This operation shows the detailed operation of the configuration in the combined motion information calculation unit 1007 in FIG. The combined motion information candidate list generation unit 1200 in FIG. 12 performs spatial combined motion from a candidate block group obtained by excluding candidate blocks outside the region and candidate blocks in the intra mode from the spatial candidate block group supplied from the motion information memory 111. An information candidate list is generated (S1500). Detailed operations for generating the spatially coupled motion information candidate list will be described later.

  The combined motion information candidate list generation unit 1200 subsequently generates temporal combined motion information from a candidate block group obtained by excluding candidate blocks outside the region and candidate blocks in the intra mode from the temporal candidate block group supplied from the motion information memory 111. A candidate list is generated (S1501). Detailed operation of the time combination motion information candidate list generation will be described later.

  Subsequently, in the combined motion information candidate list deleting unit 1201, combined motion information having overlapping motion information from the combined motion information candidate list obtained by integrating the generated spatial combined motion information candidate list and the temporal combined motion information candidate list. If there are a plurality of candidates, the motion information candidate list is updated by deleting one of the combined motion information candidates (S1502).

  Finally, when there is no valid motion information in the adjacent block in the above process and there is no combined motion information candidate (S1503: YES), fixed motion information is added to the combined motion information candidate list (S1504), The process ends. As the fixed motion information in the first embodiment, the prediction type is bi-prediction, the reference image designation information (index) is 0, and the two motion vector values are both (0, 0). The fixed motion information in the first embodiment is not limited to the above setting, and the same motion information can be reproduced in the moving picture decoding apparatus by being set by a method that can be implicitly specified. In the above process, when valid motion information exists in adjacent blocks and there is a combined motion information candidate (S1503: NO), step S1504 is skipped and the process ends.

  The candidate block group of motion information supplied from the motion information memory 111 to the combined motion information calculation unit 1007 includes a spatial candidate block group and a temporal candidate block group. First, generation of a spatially coupled motion information candidate list will be described.

  FIG. 16 is a diagram illustrating a spatial candidate block group used for generating a spatially coupled motion information candidate list. The spatial candidate block group indicates a block of the same image adjacent to the prediction target block of the encoding target image. The block group is managed in units of the minimum predicted block size, and the blocks at positions A1 to A4, B1 to B4, C, D, and E as shown in FIG. 16 are adjacent block groups. The position of the candidate block is managed in units of the minimum prediction block size, but if the prediction block size of the adjacent block is larger than the minimum prediction block size, the same motion information for all candidate blocks within the prediction block size Is stored. In the first embodiment, among these adjacent block groups, the spatial candidate block group is assumed to be four blocks of block A1, block B1, block C, and block D.

  FIG. 17 is a flowchart for explaining the detailed operation of generating the spatially coupled motion information candidate list. The following processing is repeated for the four candidate blocks included in the space candidate block group, block A1, block B1, block C, and block D (S1700 to S1703).

  First, the validity of the candidate block is checked (S1701). If the candidate block is not outside the area and is not in the intra mode (S1701: YES), the candidate block is valid. If the candidate block is valid, the motion information of the candidate block is added to the spatially combined motion information candidate list (S1702).

  Here, it is assumed that the spatial combination motion information candidate list includes motion information of four or less candidate blocks, but the spatial candidate block group is at least one or more processed blocks adjacent to the prediction block to be processed. The number of spatially coupled motion information candidate lists may be changed depending on the effectiveness of the candidate block, and the present invention is not limited to this. If the candidate block is out of the region or in the intra mode (S1701: NO), step S1702 is skipped, and the validity / invalidity determination of the next candidate block is performed.

  Subsequently, generation of a time combination motion information candidate list will be described. FIG. 18 is a diagram illustrating a time candidate block group used for generating a time combination motion information candidate list. The temporal candidate block group indicates blocks at the same position and the periphery of the prediction target block in the decoded image ColPic different from the prediction target block. The blocks at positions A1 to A4, B1 to B4, C, D, E, F1 to F4, G1 to G4, H, and I1 to I16 in FIG. 18 are temporally adjacent block groups. In the first embodiment, among these temporally adjacent block groups, the temporal candidate block group is assumed to be two blocks, block H and block I6.

  FIG. 19 is a flowchart for explaining the detailed operation of generating the time combination motion information candidate list. Regarding the block H and block I6 which are two candidate blocks included in the time candidate block group (step 1900, step S1905), the validity of the candidate block is checked (S1901). If the candidate block is valid (S1901: YES), the processing of step S1902 to step S1904 is performed, the generated motion information is registered in the time combination motion information candidate list, and the processing ends. When the candidate block indicates a position outside the screen area, or when the candidate block is an intra prediction block (S1901: NO), the candidate block is not valid and the next candidate block is determined to be valid / invalid.

  If the candidate block is valid (S1901: YES), the reference image selection candidate to be registered in the combined motion information candidate is determined based on the motion information of the candidate block (S1902). In the first embodiment, the L0 prediction reference image is the reference image that is the closest to the processing target image among the L0 prediction reference images, and the L1 prediction reference image is the processing target image among the L1 prediction reference images. The reference image is the closest distance.

  The determination method of the reference image selection candidate here is not limited to this as long as the reference image for L0 prediction and the reference image for L1 prediction can be determined. The reference image intended at the time of encoding can be determined by determining the reference image by the same method in the encoding process and the decoding process. As another determination method, for example, a method of selecting a reference image having a reference image index of 0 for a reference image for L0 prediction and a reference image for L1 prediction, or a L0 reference image and a L1 reference image used by spatially neighboring blocks. Thus, it is possible to use a method of selecting the most frequently used reference image as a reference image in the prediction target block, or a method of designating reference images for L0 prediction and L1 prediction in the encoded stream.

  Next, the motion vector value to be registered in the combined motion information candidate is determined based on the motion information of the candidate block (S1903). In the first embodiment, the temporally coupled motion information calculates bi-prediction motion information based on motion vector values that are effective prediction types in motion information of candidate blocks. When the prediction type of the candidate block is L0 prediction or L1 prediction single prediction, the motion information for the reference image list (L0 prediction reference image list or L1 prediction reference image list) used for prediction is selected. The reference image designation information and the motion vector value are set as a reference value for generating bi-predictive motion information.

  If the prediction type of the candidate block is bi-prediction, motion information of either L0 prediction or L1 prediction is selected as a reference value. The selection method of the reference value is, for example, selecting motion information that exists in the same prediction type as ColPic, selecting the reference image of the candidate block that has a shorter inter-image distance from ColPic in the L0 prediction and L1 prediction, or For example, it is possible to select a direction in which the motion vectors of the L0 prediction and the L1 prediction of the candidate block intersect with the encoding process target image.

  When the motion vector value used as a reference for bi-predictive motion information generation is determined, a motion vector value to be registered in the combined motion information candidate is calculated.

FIG. 20 is a diagram for explaining a calculation method of motion vector values mvL0t and mvL1t registered for L0 prediction and L1 prediction with respect to the reference motion vector value ColMv for temporally coupled motion information. The distance between images between ColPic for the reference motion vector value ColMv and the reference image that is the target of the motion vector used as a reference for the candidate block is referred to as ColDist. The inter-image distances between the L0 prediction and L1 prediction reference images and the processing target image are CurrL0Dist and CururL1Dist. A motion vector obtained by scaling ColMv with a distance ratio of ColDist to CurrL0Dist and CurrL1Dist is set as a motion vector to be registered. Specifically, the motion vector values mvL0t and mvL1t to be registered are calculated by the following formulas 1 and 2.
mvL0t = mvCol × CurrL0Dist / ColDist (Formula 1)
mvL1t = mvCol × CurrL1Dist / ColDist (Formula 2)
It becomes.

  Returning to FIG. 19, the bi-predicted reference image selection information (index) and the motion vector value generated in this way are added to the combined motion information candidates (S1904), and the temporally combined motion information candidate list creation process ends. To do.

  FIG. 21 is a flowchart for explaining the detailed operation of the combined prediction mode evaluation value generation in step S1401 of FIG. This operation is a detailed operation of the configuration using the combined motion compensation prediction generation unit 1008 of FIG.

  First, the prediction error evaluation value is set to the maximum value, and the combined motion information index that minimizes the prediction error is initialized (for example, a value outside the list such as −1) (S2100). If the number of combined motion information candidate lists generated by the combined motion information candidate list generation process is num_of_index, the following processing is repeated for combined motion information candidates from i = 0 to num_of_index-1 (S2101 to S2109).

  First, the motion information stored in the index i is acquired from the combined motion information candidate list (S2102). Subsequently, a motion information code amount is calculated (S2103). In the joint prediction mode, since only the joint motion information index is encoded, only the joint motion information index becomes the motion information code amount. In the first embodiment, a Truncated Unary code string is used as the code string of the combined motion information index.

  Subsequently, when the motion information prediction type is single prediction (S2104: YES), the reference image designation information and the motion vector for one reference image are set in the motion compensation prediction unit 108 in FIG. A prediction block is generated (S2105). When the motion information is not uni-prediction (S2104: NO), reference image designation information and motion vectors for two reference images are set in the motion compensated prediction unit 108, and a motion compensated bi-prediction block is generated (S2106).

  Subsequently, a prediction error evaluation value is calculated from the prediction error and the motion information code amount of the motion compensated prediction block and the prediction target block (S2107), and when the prediction error evaluation value is the minimum value, the evaluation value is updated. The prediction error minimum index is updated (S2108).

  As a result of comparison of the prediction error evaluation values for all the combined motion information candidates, the selected prediction error minimum index is output together with the prediction error minimum value and the motion compensated prediction block as a combined motion information index used in the combined prediction mode. (S2110), the combined prediction mode evaluation value generation process is terminated.

  FIG. 22 is a flowchart for explaining the detailed operation of the combined motion correction prediction mode evaluation value generation in step S1402 of FIG. This operation shows a detailed operation of the configuration using the combined motion corrected motion compensation prediction generation unit 1009 of FIG.

  First, the prediction error evaluation value is set to the maximum value, and the combined motion information index that minimizes the prediction error is initialized (for example, a value outside the list such as −1) (S2200). If the number of combined motion information candidate lists generated by the combined motion information candidate list generation process is num_of_index, the following processing is repeated for combined motion information candidates from i = 0 to num_of_index-1 (S2201 to S2210).

  First, the motion information stored in the index i is acquired from the combined motion information candidate list (S2202). Subsequently, a motion-corrected reference image serving as a reference image for transmitting a bi-predicted motion vector value, reference image designation information, and a difference vector is determined from the acquired motion information (S2203). The detailed operation of step S2203 will be described later.

  Next, the motion vector value detected for the motion correction reference image is input from the motion vector detection unit 107 of FIG. 1 to the motion correction reference image determined in step S2203 (S2204). In order to use the input motion vector value for motion compensation prediction, a difference vector with the motion vector value of the motion correction reference image in the motion information calculated from the combined motion information candidate is generated (S2205).

  Subsequently, a motion information code amount is calculated (S2206). In the combined motion correction prediction mode, necessary motion information is transmitted by encoding the combined motion information index and the difference vector value. For this reason, the code amount of the combined motion information index and the code amount of the difference vector value are added to obtain the motion information code amount in the combined motion correction prediction mode.

  In the combined corrected motion prediction mode, the motion information obtained by replacing only the motion vector corrected for the motion corrected reference image with respect to the bi-predicted motion vector generated from the combined motion information and the reference image selection information is shown in FIG. A motion compensation bi-prediction block is generated by setting the motion compensation prediction unit 108 (S2207).

  A prediction error evaluation value is calculated from the prediction error and the motion information code amount of the motion compensated prediction block and the prediction target block (S2208). When the prediction error evaluation value is the minimum value, the evaluation value is updated and the prediction error is calculated. The minimum index is updated (S2209).

  As a result of comparison of the prediction error evaluation values for all the combined motion information candidates, the selected prediction error minimum index is output together with the prediction error minimum value and the motion compensated prediction block as a combined motion information index used in the combined prediction mode. (S2211) The combined motion correction prediction mode evaluation value generation process ends.

  Next, before describing the detailed operation of step S2203, the scaling process for the combined motion information in the combined motion correction prediction mode will be described. 23A to 23D are diagrams illustrating the concept of scaling processing for combined motion information in the combined motion modified prediction mode. As the combined motion information, the prediction type calculated from the space BiPred whose prediction type is the bi-prediction calculated from the space adjacent block shown in FIG. 23A and the space adjacent block shown in FIGS. There exists a space UniPred whose type is single prediction and a time BiPred calculated from the time concatenated block shown in FIG.

  The scaling process for the combined motion information in the first embodiment uses the motion vector applied at the time of calculating the combined motion information as it is for the space BiPred and the time BiPred where bi-prediction motion information can be acquired. It was not used by multiplying the motion vector value defined in the prediction by 1 or -1 according to the temporal positional relationship between the encoding target image and the L0 reference image and the L1 reference image. A motion vector of the reference image list is generated.

That is, as shown in FIG. 43 (b), when the L0 reference image and the L1 reference image are in the opposite directions in the temporal direction with respect to the encoding target image, the motion defined by the single prediction is performed. By inverting the vector value by −1, that is, the sign of the motion vector value, a motion vector in the opposite direction to the motion vector of the reference image list already used is generated. Note that the following condition determination is used for the condition determination in which the L0 reference image and the L1 reference image exist in opposite directions with respect to the encoding target image.
The POC value of the L0 reference image <the POC value of the encoding target image, and the POC value of the encoding target image <the POC value of the reference image of L1

On the other hand, as shown in FIG. 43 (c), when the L0 reference image and the L1 reference image are in the same direction in the time direction with respect to the encoding target image, the motion vector defined by single prediction is used. Is multiplied by 1, that is, the same motion vector value is used as the other motion vector value. The following condition determination is used for the condition determination in which the L0 reference image and the L1 reference image exist in the same direction with respect to the encoding target image.
PO value of the reference image of L0 <POC value of the encoding target image, and POC value of the reference image of L1 <POC value of the encoding target image or POC value of the encoding target image <reference of L1 The POC value of the image, and the POC value of the encoding target image <the value of the POC of the reference image of L0

  By performing the above processing on the motion vector defined by the single prediction, the reference image motion vector of the other reference image list that has not been used can be generated from the motion vector defined by the single prediction. it can. This is a characteristic configuration that improves the encoding efficiency in the present embodiment, and a simpler method without performing complicated processing even when the motion information of the block adjacent to the processing target block is uni-prediction It is possible to perform bi-prediction by generating the other vector.

In the encoding device of the present invention, there is a possibility that the motion vector of the other reference image list that is not used by the maximum number of combined prediction mode candidates may be generated. The amount of processing can be reduced by this method that can generate a motion vector of the other reference image list without performing an operation on a vector value.
In the decoding apparatus of the present invention, when the modified prediction mode is selected, it is possible to reduce the calculation of motion vector values that may be necessary after the combined prediction list is generated, and the modified prediction mode with almost the same computational load as when the combined prediction mode is selected. Motion information calculation processing can be performed.

  FIG. 44 shows an example of reference images registered in the reference image list for L0 prediction and L1 prediction. FIG. 44 shows an example of a reference image when the POC of the encoding target image is 10. Usually, reference images are registered in order from an image that is temporally close to the encoding target image. In addition, the L0 prediction reference image is preferentially registered with respect to the temporally preceding image relative to the encoding target image, and the L1 prediction reference image is temporally later than the encoding target image. In this case, the first image in each reference image list is often set at the same distance with respect to the encoding target image. In addition, since an image close to the encoding target image has a high correlation with the encoding target image, the probability of being used as a reference image is high, that is, the first image in the reference image list is frequently used as a reference image. Therefore, even if the motion vector of the single prediction is multiplied by 1 or −1 according to the position of the reference image and the other motion vector is generated, it is compared with the case where the scaling operation according to the normal inter-image distance is performed. Therefore, a combined motion modified prediction mode that performs bi-prediction with a smaller amount of processing can be used.

  FIG. 24 is a diagram illustrating the number of scaled motion vectors that exist for each piece of combined motion information. The space UniPred is 1 in the above scaling process, the space BiPred is 0 because no scaling process is performed when calculating the combined motion information, and the time BiPred is subjected to the scaling process when calculating two motion vector values when calculating the combined motion information. Because it is, it becomes two.

  When the motion vector value subjected to the above scaling processing is compared with the combined motion information in which the motion vector value used in the adjacent block is directly adopted as the motion vector value, the influence of the calculation error due to scaling and the temporal change of the motion amount There is an influence when it is not constant, and there is a high possibility that the accuracy of the motion information has shifted. In addition, since the motion vector value used for the adjacent block has a larger screen change as the reference image is temporally separated from the encoding target, the accuracy of the value decreases.

  In the first embodiment, using the characteristics at the time of calculation of the motion vector registered in the combined motion information, a motion vector that needs to be corrected by sending a difference vector is transmitted as additional information. And specify only the difference vector. Thereby, motion information with high prediction accuracy of motion compensation prediction can be realized by transmission of a small amount of motion information.

  In the first embodiment, a reference image having a motion vector value calculated by scaling is preferentially set as a motion correction reference image for transmitting a difference vector. When the two motion vectors are both scaled or are not scaled together, a reference image that is separated in time from the prediction target image is set as a motion-corrected reference image that transmits the difference vector.

  FIGS. 25A to 25C are conceptual diagrams showing the differential motion vector transmission reference image determination algorithm in the first embodiment. In the space BiPred shown in FIG. 25A, since the scaled motion vector is zero, a difference vector is added to the motion vector of the reference image far in time. In the space UniPred shown in FIG. 25B, since there is one scaled motion vector, a difference vector is added to the scaled motion vector. At time BiPred shown in FIG. 25C, since there are two scaled motion vectors, a difference vector is added to the motion vector of the reference image far in time.

  FIG. 26 is a flowchart for explaining detailed operation of step S2203 of FIG. In step S2203 of FIG. 22, a reference image transmission process for transmitting the difference vector is performed.

  First, when the prediction type of the acquired combined motion information is uni-prediction (S2600: YES), the second reference for performing bi-prediction motion compensation prediction is combined motion information derived in the space UniPred. Processing for determining the motion vector values of the image and the second reference image is performed (S2601). The detailed operation of step S2601 will be described later.

  Subsequently, when the scaled motion vector is only one motion vector of uni-prediction among the motion vector values of the bi-predictive reference image (S2602: YES), the reference image having the scaled motion vector value is corrected for motion. The reference image is set (S2603).

  When the scaled motion vector is not a single motion vector for uni-prediction (S2602: NO), POC values indicating temporal information of the encoding target image, the L0 reference image, and the L1 reference image are acquired, and POCcur, POC_L0, and POC_L1 are obtained. (S2604).

The temporal distance between the encoding target image and the two reference images is compared by the following expression 3.
abs (POCcur-POC_L0)> abs (POCcur-POC_L1)
... (Formula 3)

  When Expression 3 is satisfied (S2605: YES), the L0 reference image is determined to be a distant reference image, and the L0 reference image is set as a motion corrected reference image (S2606). When Expression 3 is not satisfied (S2605: NO), the L1 reference image is set as a motion-corrected reference image (S2607). This completes the process of determining the reference image for transmitting the bi-prediction motion vector value and the difference vector.

  FIG. 27 is a flowchart for explaining the detailed operation of step S2601 of FIG. First, when the motion information stored in the combined motion information is L0 prediction (S2700: YES), a reference image index for L1 prediction is determined as the second reference image (S2701). When it is not L0 prediction (S2700: NO), a reference image index for L0 prediction is determined as the second reference image (S2702). The reference image index determination process in step S2702 will be described later.

  Subsequently, POC values of the encoding target image, the reference image stored in the combined motion information, and the second reference image are acquired and set to POCcur, POC_Lx, POC_Ly (S2703). Using the POC value, the motion vector value stored in the combined motion information is scaled by the distance ratio between the two reference images and the encoding target image (S2704).

  When the motion information stored in the combined motion information is L0 prediction (S2705: YES), the motion vector value generated by scaling is set as the reference motion vector value for the reference image of L1 prediction (S2706). If it is not the L0 prediction (S2705: NO), the motion vector value generated by scaling is set as the reference motion vector value for the reference image for the L0 prediction (S2707), and the process is terminated.

  FIG. 28 is a flowchart for explaining the detailed operation of step S2702 of FIG. In step S2702 of FIG. 27, a process of determining a reference image index is performed.

  First, the reference image stored in the combined motion information is acquired as a standard reference image, and the reference image list in the prediction type for setting the second reference image is acquired as a second reference image candidate list (S2800).

  Next, it is checked whether or not a reference image that exists at a time position in the opposite direction to the standard reference image across the encoding target image exists in the second reference image candidate list (S2801). If it exists (S2801: YES), the reference image existing at the time position closest to the encoding target image is determined as the reference image index for the second reference image (S2802). If it does not exist (S2801: NO), the reference image existing at the time position closest to the encoding target is set as the second reference image candidate (S2803), and the reference reference image and the second reference image candidate are the same image. (S2804).

  If the reference reference image and the second reference image candidate are not the same image (S2804: NO), the second reference image candidate is determined as a reference image index for the second reference image (S2805). When the reference reference image and the second reference image candidate are the same image (S2804: YES), it is checked whether another reference image exists in the second reference image candidate list (S2806). If it exists (S2806: YES), a reference image closest to the next encoding target image at the time position is acquired and determined as a reference image index for the second reference image (S2807). If no other reference image exists (S2806: NO), the process proceeds to step S2805, and the second reference image candidate is determined as a reference image index for the second reference image. As described above, the reference image index for the second reference image in all cases is determined, and the process ends.

  The determination algorithm for the second reference image in the first embodiment is an interpolation type (interpolation type) bi-prediction in which the motion-compensated prediction process is composed of reference images that exist in opposite directions across the encoding target image in bi-prediction. Compared to interpolation (extrapolation type) bi-prediction consisting of reference images that exist in the same direction without interposing the encoding target image, it is possible to average the effects of motion and temporal changes in objects, so that the prediction accuracy The reference image closer to the encoding target image is less affected by the motion and the time change of the object and the accuracy of motion compensation prediction is better, and the prediction image generated by superimposing the same reference image The bi-predictive image generated from different reference images is a deterministic algorithm based on three findings that the prediction image has a higher prediction accuracy when the motion is correctly captured. This method has an advantage that a reference image suitable for bi-prediction can be set as the second reference image.

  However, regarding the above-described determination process of the reference image index, it is only necessary to determine the reference image index of the second reference image, and it is possible to use a different determination algorithm. The reference image intended at the time of encoding can be determined by determining the reference image by the same method in the encoding process and the decoding process. As another determination method, when a motion vector in the motion information of another block does not indicate an accurate motion, a minute change is added to the reference image by using a prediction image generated by overlapping the same reference image. Emphasizing the effect that a motion compensated prediction signal can be added as a new candidate, step S2804, which is a process for checking whether the reference reference image and the second reference image candidate are the same image in the flowchart of FIG. 28, is eliminated. Even if the images are the same, a method of determining them as the second reference image candidates can be used.

As another determination method, among the reference images of the prediction type determined as the second reference image, a method of simply selecting a reference image that is closest to the encoding target image, or a reference image index that is always 0 is used. Time, such as a method for obtaining the indicated reference image, or a method for selecting the reference image used most frequently as the reference image in the prediction target block in the L0 reference image and the L1 reference image used by the spatially adjacent block It is also possible to use determination means used in the reference image determination process in BiPred. In addition, the algorithm shown in FIG. 28 can be used for reference image determination processing at time BiPred.

  FIG. 29 is a flowchart for explaining the detailed operation of step S1403 of FIG. In step S1403 of FIG. 14, a single prediction mode evaluation value generation process is performed.

  First, reference image designation information (index) and motion vector values for single prediction are acquired (S2900). Subsequently, a prediction vector candidate list is generated (S2901), an optimal prediction vector is selected from the prediction vectors, and a difference vector is generated (S2902). It is desirable to select the optimal prediction vector with the least amount of code when the difference vector between the prediction vector and the motion vector to be transmitted is actually encoded. However, the horizontal and vertical components of the difference vector are simply selected. The calculation may be simplified by a method such as selecting one having a small absolute sum. Detailed operation of step S2901 will be described later.

  Subsequently, a motion information code amount is calculated (S2903). In the single prediction mode, as motion information to be encoded, there are three elements of reference image designation information, a difference vector value, and a prediction vector index for one reference image, and the total amount of the encoded amount of motion is the motion. It is calculated as the information code amount. As a predictive vector index code string generation method according to Embodiment 1, a Trunked Unary code string is used in the same manner as the combined motion information index code string.

  Subsequently, reference image designation information and a motion vector for one reference image are set in the motion compensated prediction unit 108 in FIG. 1 to generate a motion compensated single prediction block (S2904). Further, a prediction error evaluation value is calculated from the prediction error and the motion information code amount of the motion compensated prediction block and the prediction target block (S2905), and a reference image designation that is the prediction error evaluation value and motion information for one reference image. Information, a difference vector value, and a prediction vector index are output with a motion compensation prediction block (S2906), and a single prediction mode evaluation value generation process is complete | finished.

  FIG. 30 is a flowchart for explaining the detailed operation of step S2901 of FIG. In step S2901 in FIG. 29, a prediction vector candidate list is generated. This operation shows the detailed operation of the configuration in the prediction vector calculation unit 1003 of FIG.

  The prediction vector candidate list generation unit 1100 in FIG. 11 uses the spatial motion information from the candidate block group excluding the candidate block outside the region from the spatial candidate block group supplied from the motion information memory 111 and the candidate block in the intra mode. A prediction vector candidate list is generated (S3000). The detailed operation of generating the spatial motion information use prediction vector candidate list will be described later.

  The prediction vector candidate list generation unit 1100 subsequently uses the temporal motion information use prediction vector from the candidate block group obtained by excluding the candidate block outside the region and the candidate block in the intra mode from the temporal candidate block group supplied from the motion information memory 111. A candidate list is generated (S3001). The detailed operation of generating the temporal motion information use prediction vector candidate list will be described later.

  Subsequently, the prediction vector candidate list deletion unit 1101 performs prediction with overlapping motion vectors from the prediction vector candidate list obtained by integrating the generated spatial motion information use prediction vector candidate list and temporal motion information use prediction vector candidate list. If there are a plurality of vector candidates, the prediction vector candidate list is updated by deleting one of the prediction vector candidates (S3002).

  Finally, when there is no valid motion information in the adjacent block and no prediction vector candidate exists in the above processing (S3003: YES), the fixed vector value (0, 0) is added to the prediction vector candidate list (S3004). ), The process is terminated.

  FIG. 31 is a flowchart for explaining a detailed operation of generating a spatial motion information use prediction vector candidate list. The candidate block group of motion information supplied from the motion information memory 111 to the prediction vector calculation unit 1003 includes a spatial candidate block group and a temporal candidate block group. The concepts of the spatial candidate block group and the temporal candidate block group are the same as those of the motion information candidate blocks for the combined motion information shown in FIGS.

  In the first embodiment, the space candidate block group is assumed to be four blocks of block A1, block B1, block C, and block D in FIG. In Embodiment 1, the number and position of candidate blocks are not limited, and the number and position of candidate blocks used in the combined motion information and the number and position of candidate blocks used in the prediction vector match even if they match. It doesn't have to be.

  First, the following processing is repeated for the four candidate blocks included in the space candidate block group, block A1, block B1, block C, and block D (S3100 to S3103).

  First, the validity of the motion vector value in the reference image designated in the candidate block is checked (S3101). The motion vector value becomes invalid not only when the candidate block is not out of the region and not in the intra mode, but also when the reference image designated by the single prediction is not used or when the reference index is different. However, in the first embodiment, when the reference image specified in the single prediction is not used or when the reference index is different, additional processing such as obtaining a motion vector by scaling the motion vector value of another reference image And the motion vector value valid / invalid determination method is not limited to the method of the first embodiment.

  Subsequently, if the motion vector value is valid (S3101: YES), the motion vector value of the candidate block is added to the prediction vector candidate list (S3102). If the motion vector value is invalid (S3101: NO), step S3102 is skipped, and valid / invalid determination of the motion vector value of the next candidate block is performed.

  FIG. 32 is a flowchart for explaining a detailed operation of generating a temporal motion information use prediction vector candidate list. In the first embodiment, among the temporally adjacent block groups in FIG. 18, the temporal candidate block group is assumed to be two blocks, block H and block I6. Also in the generation of a temporal motion information prediction vector candidate list, the number and position of candidate blocks are not limited, and can be set separately from the number and position of candidate blocks used in combined motion information.

  Regarding the block H and the block I6 that are two candidate blocks included in the time candidate block group (step 3200, step S3204), the validity of the candidate block is checked (S3201). If the candidate block is valid (S3201: YES), the processes of steps S3202 to S3203 are performed, the generated motion vector values are registered in the prediction vector candidate list, and the process ends. When the candidate block indicates a position outside the screen area, or when the candidate block is an intra prediction block, the candidate block is not valid (S3201: NO), and the validity / invalidity determination of the next candidate block is performed.

  If the candidate block is valid (S3201: YES), the motion vector value to be registered in the prediction vector candidate is determined based on the motion information of the candidate block (S3202). The confirmed motion vector value is added to the prediction vector candidate list (S3203), and generation of the temporal motion information use prediction vector candidate list ends.

  FIG. 33 is a flowchart for explaining the detailed operation of step S1404 of FIG. In step S1404 of FIG. 14, bi-prediction mode evaluation value generation processing is performed.

  First, reference image designation information (index) and motion vector values for L0 prediction are acquired (S3300). Subsequently, an L0 prediction vector candidate list is generated (S3301), an optimal prediction vector in L0 prediction is selected from the prediction vectors, and a difference vector is generated (S3302). These processes are the same as the processes from step 2900 to step S2902 in the single prediction mode evaluation value generation process in the flowchart of FIG.

  Subsequently, reference image designation information (index) and motion vector value for L1 prediction are acquired (S3303). Subsequently, an L1 prediction vector candidate list is generated (S3304), an optimal prediction vector in L1 prediction is selected from the prediction vectors, and a difference vector is generated (S3305). These processes are also the same as the processes from step 2900 to step S2902 in the single prediction mode evaluation value generation process in the flowchart of FIG.

  Subsequently, a motion information code amount is calculated (S3306). In the case of the bi-prediction mode, the motion information to be encoded is a total of six elements of reference image designation information, difference vector value, and prediction vector index for the two reference images of L0 and L1, each encoded. The total amount of code is calculated as the motion information code amount. For this reason, the amount of code of motion information to be encoded is larger than in the combined prediction mode and the combined motion correction prediction mode.

  Subsequently, reference image designation information and motion vectors for the two reference images are set in the motion compensated prediction unit 108 in FIG. 1 to generate a motion compensated bi-prediction block (S3307). Further, a prediction error evaluation value is calculated from the prediction error and the motion information code amount of the motion compensated prediction block and the prediction target block (S3308), and the prediction error evaluation value and motion information for two reference images L0 and L1. Two pieces of reference image designation information, two difference vector values, and two prediction vector indexes are output together with the motion compensated prediction block (S3309), and the bi-prediction mode evaluation value generation process is terminated.

  The above processing is the detailed operation of the prediction mode determination unit 109 in the video encoding apparatus in the first embodiment.

[Detailed Description of Operation of Motion Information Encoding Unit in Moving Picture Encoding Device in Embodiment 1]
FIG. 34 is a flowchart for explaining the detailed operation of step S504 of FIG. The process of step S504 in FIG. 5 shows a process of encoding motion information for each motion compensation prediction mode in the motion information encoding unit 110 of the video encoding apparatus of the first embodiment.

  The motion information encoding unit 110 is supplied with information indicating the prediction mode in the motion compensated prediction mode determined by the prediction mode determination unit 109 and information necessary for expressing the motion information of each prediction mode. The information encoding unit 110 starts a motion information encoded data generation process.

  First, when the prediction mode is the combined prediction mode (S3400: YES), the process proceeds to step S3401, and when the prediction error signal is not encoded in the target prediction block (Skip mode) (S3401: YES), The Skip flag is encoded (S3402). Otherwise (S3401: NO), the merge flag is encoded as 1 (S3403).

  After the Skip flag or merge flag is encoded, if the combined motion information candidate list is greater than 1 (S3404: YES), the combined motion information index is encoded (S3405), and the process ends. When the combined motion information candidate list is 1 (S3404: NO), since combined motion information can be specified, the combined motion information index is not transmitted.

  Next, when the prediction mode is not the combined prediction mode (S3400: NO), the merge flag is encoded as 0 (S3406), and the motion prediction flag is encoded (S3407). The motion prediction flag is encoded as information indicating whether the prediction type is single prediction or bi-prediction.

  FIG. 35 is a diagram illustrating an example of a motion prediction flag in the first embodiment. As a motion prediction flag (inter_pred_flag), a value (Pred_LC) indicating uni-prediction is 0, and a value (Pred_BI) indicating bi-prediction is 1. Regarding the combined motion correction prediction mode, the structure of motion compensation prediction is bi-prediction, and a motion prediction flag is sent as Pred_BI.

  Subsequently, when the prediction type indicated by the motion prediction flag is single prediction (S3408: YES), motion information transmission in the single prediction mode is performed. First, reference image designation information (index) is encoded (S3409), and then a difference vector is encoded (S3410). If the predicted vector candidate list is greater than 1 (S3411: YES), the predicted vector index is encoded (S3412), and the process is terminated. When the prediction vector candidate list is 1 (S3411: NO), since the prediction vector can be specified, the index is not transmitted.

  On the other hand, when the prediction type indicated by the motion prediction flag is not single prediction (S3408: NO), when the prediction mode is the combined motion correction prediction mode (S3413: YES), motion information transmission in the combined motion correction prediction mode is performed. Done.

  First, the merge Mvd flag for specifying the combined motion correction prediction mode is encoded with 1 (S3414). Subsequently, when the combined motion information candidate list is larger than 1 (S3415: YES), the combined motion information index is encoded (S3416). When the combined motion information candidate list is 1 (S3415: NO), the combined motion information index is not transmitted. Finally, the difference vector value between the motion vector value of the motion correction reference image specified in the combined motion correction prediction mode and the motion vector before correction of the motion correction reference image of the combined motion information is encoded (S3417), and processing Exit.

  When the prediction mode is not the combined motion correction prediction mode (S3413: NO), the prediction mode becomes the bi-prediction mode, and motion information transmission in the bi-prediction mode is performed. First, the merge Mvd flag is encoded with 0 to indicate that it is not the combined motion correction prediction mode (S3418). Subsequently, L0 reference image designation information, which is motion information of L0 prediction, is encoded (S3419), and an L0 difference vector value is encoded (S3420). When the prediction vector candidate list for the L0 prediction is larger than 1 (S3421: YES), the L0 prediction vector index is encoded (S3422). When the prediction vector candidate list for L0 prediction is 1 (S3421: NO), the L0 prediction vector index is not transmitted.

  Subsequently, L1 reference image designation information, which is motion information of L1 prediction, is encoded (S3423), and an L1 difference vector value is encoded (S3424). When the prediction vector candidate list for L1 prediction is larger than 1 (S3425: YES), the L1 prediction vector index is encoded (S3426). When the prediction vector candidate list for L1 prediction is 1 (S3425: NO), the L1 prediction vector index is not transmitted. With the above processing, the generation of motion information encoded data is completed.

  FIG. 36 shows an example of the syntax of the encoded stream generated by the motion information encoded data generation process shown in the flowchart of FIG. In the coding syntax of FIG. 36, the transmission of the Skip flag is selected in units of coding blocks in a layer higher than the coding of motion information in units of prediction blocks. Therefore, it is determined (S3401) whether or not the prediction error signal in FIG. 34 is not encoded (Skip mode) in units of higher-order encoded blocks.

[Detailed Operation Description of Motion Information Decoding Unit in Moving Picture Decoding Device in Embodiment 1]
FIG. 37 is a diagram showing a detailed configuration of the motion information decoding unit 606 in the moving picture decoding apparatus according to Embodiment 1 shown in FIG. The motion information decoding unit 606 includes a motion information bitstream decoding unit 3700, a prediction vector calculation unit 3701, a vector addition unit 3702, a motion compensation uni-prediction decoding unit 3703, a motion compensation bi-prediction decoding unit 3704, a combined motion information calculation unit 3705, a combination A motion compensation prediction decoding unit 3706 and a combined motion modified motion compensation prediction decoding unit 3707 are included.

  For the motion information decoding unit 606 in FIG. 6, the motion information bitstream input from the demultiplexing unit 601 is supplied to the motion information bitstream decoding unit 3700, and the motion information input from the motion information memory 607 is predicted. The vector calculation unit 3701 and the combined motion information calculation unit 3705 are supplied.

  In addition, the motion compensated prediction prediction unit 608 includes a motion compensated prediction prediction unit 3703, a motion compensated bi-prediction decoding unit 3704, a joint motion compensation prediction decoding unit 3706, and a joint motion corrected motion compensation prediction decoding unit 3707. The reference image designation information and the motion vector used for are output, and the decoded motion information including information indicating the prediction type is stored in the motion information memory 607.

  The motion information bitstream decoding unit 3700 generates motion information corresponding to the transmitted prediction mode and the prediction mode by decoding the motion information bitstream input from the demultiplexing unit 601 according to the encoding syntax. To do. Among the generated motion information, the combined motion information index is supplied to the combined motion compensation prediction decoding unit 3706 or the combined motion modified motion compensation prediction decoding unit 3707 according to the prediction mode, and the reference image designation information is the prediction vector calculation unit. 3701, the prediction vector index is supplied to the vector addition unit 3702, and the difference vector value is supplied to the vector addition unit 3702 or the combined motion modified motion compensated prediction decoding unit 3707 according to the prediction mode.

  The prediction vector calculation unit 3701 predicts a reference image that is a target of motion compensation prediction from the motion information of the adjacent block supplied from the motion information memory 607 and the reference image designation information supplied from the motion information bitstream decoding unit 3700. A vector candidate list is generated and supplied to the vector addition unit 3702 together with the reference image designation information. Regarding the operation of the prediction vector calculation unit 3701, the same operation as that of the prediction vector calculation unit 1003 in the moving picture encoding apparatus in FIG. 10 is performed, and the same candidate list as the prediction vector candidate list at the time of encoding is generated.

  The vector addition unit 3702 indicates the prediction vector index from the prediction vector candidate list and reference image designation information supplied from the prediction vector calculation unit 3701 and the prediction vector index and difference vector supplied from the motion information bitstream decoding unit 3700. By adding the prediction vector value and the difference vector value registered at the set position, the motion vector value for the reference image to be motion compensated prediction is reproduced. The reproduced motion vector value is supplied to the motion compensation uni-prediction decoding unit 3703 or the motion compensation bi-prediction decoding unit 3704 according to the prediction mode together with the reference image designation information.

  The motion compensated uni-prediction decoding unit 3703 is supplied with the reconstructed motion vector value and reference image designation information for one reference image from the vector addition unit 3702, and sends the motion vector value and the reference image designation information to the motion compensation prediction unit 608. By setting, a motion compensation prediction signal is generated.

  The motion compensated bi-prediction decoding unit 3704 is supplied with the reconstructed motion vector values and reference image designation information for the two reference images for bi-prediction from the vector addition unit 3702, and performs motion compensation prediction on the motion vector values and the reference image designation information. By setting in the unit 608, a motion compensated prediction signal is generated.

  The combined motion information calculation unit 3705 generates a combined motion information candidate list from the motion information of adjacent blocks supplied from the motion information memory 607, and combines the combined motion information candidate list and the combined motion information candidate that is a component in the list. The reference image designation information and the motion vector value are supplied to the joint motion compensation prediction decoding unit 3706 or the joint motion corrected motion compensation prediction decoding unit 3707 according to the prediction mode.

  Regarding the operation of the combined motion information calculation unit 3705, the same operation as that of the combined motion information calculation unit 1007 in the moving image encoding apparatus of FIG. 10 is performed, and the same candidate list as the combined motion information candidate list at the time of encoding is generated. Is done.

  The combined motion compensated prediction decoding unit 3706 includes a combined motion information candidate list supplied from the combined motion information calculation unit 3705, reference image designation information of a combined motion information candidate that is a component in the list, a motion vector value, and a motion information bit. The reference image designation information and the motion vector value in the combined motion information candidate list indicated by the combined motion information index are reproduced from the combined motion information index supplied from the stream decoding unit 3700, and set in the motion compensation prediction unit 608. Generate a motion compensated prediction signal.

  The combined motion corrected motion compensated prediction decoding unit 3707 includes a combined motion information candidate list supplied from the combined motion information calculation unit 3705, reference image designation information of a combined motion information candidate that is a component in the list, a motion vector value, and a motion. The motion information for the combined motion correction prediction mode is calculated from the combined motion information index supplied from the information bitstream decoding unit 3700, and the motion correction reference image is determined. Then, by adding the difference vector value to the motion vector value of the motion-corrected reference image, the bi-predictive reference image designation information and the motion vector value are reproduced and set in the motion compensation prediction unit 608, so that motion compensation is performed. Generate a prediction signal.

  FIG. 38 is a flowchart for explaining the detailed operation of step S701 in FIG. The motion information decoding process in step S701 in FIG. 7 is performed by the motion information bitstream decoding unit 3700, the prediction vector calculation unit 3701, the combined motion information calculation unit 3705, and the combined motion correction motion compensation prediction decoding unit 3707.

  The motion information decoding process is a process of decoding motion information from the encoded bit stream encoded with the syntax structure of FIG. First, the Skip flag is decoded in a predetermined unit of the encoded block (S3800). Thereafter, processing is performed in units of prediction blocks. When the Skip flag indicates the Skip mode (S3801: YES), joint prediction motion information decoding is performed (S3802). Detailed processing in step S3802 will be described later.

  If it is not the Skip mode (S3801: NO), the merge flag is decoded (S3803). When the merge flag indicates 1 (S3804: YES), the process proceeds to joint prediction motion information decoding in step S3802.

  If the merge flag is not 1 (S3804: NO), the motion prediction flag is decoded (S3805). When the motion prediction flag is in the single prediction mode (S3806: YES), that is, when inter_pred_flag in FIG. 35 indicates Pred_LC, single prediction motion information decoding is performed (S3807). Detailed operation of step S3807 will be described later.

  When the motion prediction flag does not indicate the single prediction mode (S3806: NO), the merge Mvd flag is decoded (S3808). When the merge Mvd flag is 1 (S3809: YES), joint motion modified prediction motion information decoding is performed (S3810), and when the merge Mvd flag is not 1 (S3809: NO), bi-prediction motion information decoding is performed ( S3811). Detailed operations of step S3810 and step S3811 will be described later.

  FIG. 39 is a flowchart for explaining the detailed operation of step S3802 of FIG. In step S3802 of FIG. 38, joint prediction motion information decoding processing is performed.

  First, the combined prediction mode is set as the prediction mode (S3900), and a combined motion information candidate list is generated (S3901). The process of step S3901 is the same process as step S1401 which is the combined motion information candidate list generation in the moving image encoding device in the flowchart of FIG.

  When the combined motion information candidate list is larger than 1 (S3902: YES), that is, when there are a plurality of combined motion information candidates, the combined motion information index is decoded (S3903), and the combined motion information candidate list is one. (S3902: NO), 0 is set to the combined motion information index (S3904).

  Subsequently, the motion information stored at the position indicated by the combined motion information index is acquired from the combined motion information candidate list (S3905). The motion information to be acquired includes a prediction type indicating single prediction / bi-prediction, reference image designation information, and a motion vector value. The generated motion information is stored as motion information in the joint prediction mode (S3906), and is supplied to the joint motion compensation prediction decoding unit 3706.

  FIG. 40 is a flowchart for explaining the detailed operation of step S3807 of FIG. In step S3807 of FIG. 38, single prediction motion information decoding processing is performed.

  First, the reference image designation information is decoded (S4000), and the difference vector value is decoded (S4001). Next, a prediction vector candidate list is generated (S4002). When the prediction vector candidate list is larger than 1 (S4003: YES), the prediction vector index is decoded (S4004), and when the prediction vector candidate list is 1 (S4003: NO), 0 is set to the prediction vector index (S4005).

  Here, in step S4002, processing similar to that in step S2901 in the flowchart of FIG. 29 in the video encoding device is performed. Next, the motion vector value stored at the position indicated by the prediction vector index is acquired from the prediction vector candidate list (S40006). The motion vector is reproduced by adding the decoded difference vector value and the motion vector value (S4007). As generated motion information, reference image designation information and motion vector values for one reference image are stored as motion information in the single prediction mode (S4008) and supplied to the motion compensation single prediction decoding unit 3703.

  FIG. 41 is a flowchart for explaining the detailed operation of step S3811 of FIG. In step S3811 of FIG. 38, bi-predictive motion information decoding processing is performed.

  First, reference image designation information for L0 prediction is decoded (S4100), and a difference vector value is decoded (S4101). Next, a prediction vector candidate list for L0 prediction is generated (S4102). When the prediction vector candidate list is larger than 1 (S4103: YES), a prediction vector index for L0 prediction is decoded (S4104). When the prediction vector candidate list is 1 (S4103: NO), 0 is set to the prediction vector index of L0 prediction (S4105). In step S4102, processing similar to that in step S3301 in the flowchart of FIG. 33 in the video encoding device is performed.

  Next, the motion vector value stored at the position indicated by the prediction vector index of the L0 prediction is acquired from the prediction vector candidate list (S4106). A motion vector is reproduced by adding the decoded difference vector value and motion vector value (S4107).

  Subsequently, a similar motion information decoding process is performed on the reference image for L1 prediction. The reference image designation information for L1 prediction is decoded (S4108), and the difference vector value is decoded (S4109). A prediction vector candidate list for L1 prediction is generated (S4110), and if the prediction vector candidate list is larger than 1 (S4111: YES), a prediction vector index for L1 prediction is decoded (S4112). When the prediction vector candidate list is 1 (S4111: NO), 0 is set to the prediction vector index of L1 prediction (S4113). In step S4113, processing similar to that in step S3304 in the flowchart of FIG. 33 in the video encoding device is performed.

  Next, the motion vector value stored at the position indicated by the prediction vector index of the L1 prediction is acquired from the prediction vector candidate list (S4114). A motion vector is reproduced by adding the decoded difference vector value and the motion vector value (S4115). As the generated motion information, reference image designation information and motion vector values for two reference images are stored as motion information in bi-prediction mode (S4116) and supplied to the motion-compensated bi-prediction decoding unit 3704.

  FIG. 42 is a flowchart for explaining the detailed operation of step S3810 of FIG. In step S3810 of FIG. 38, a combined motion corrected predicted motion information decoding process is performed.

  First, the combined motion correction prediction mode is set as the prediction mode (S4200), and a combined motion information candidate list is generated (S4201). In step S4201, the same processing as in step S1401 of the flowchart of FIG. 14 in the moving image encoding apparatus is performed.

  When the combined motion information candidate list is larger than 1 (S4202: YES), that is, when there are a plurality of combined motion information candidates, the combined motion information index is decoded (S4203), and the combined motion information candidate list is one. (S4202: NO), 0 is set to the combined motion information index (S4204).

  Subsequently, the motion information stored at the position indicated by the combined motion information index is acquired from the combined motion information candidate list (S4205). The motion information to be acquired includes a prediction type indicating single prediction / bi-prediction, reference image designation information, and a motion vector value.

  Next, based on the acquired combined motion information index and motion information, the combined motion corrected motion compensated prediction decoding unit 3707 performs reference image determination processing for transmitting the bi-predicted motion vector value and the difference vector (S4206). In step S4206, the same processing as in step S2203 in the flowchart of FIG. 22 in the moving image encoding apparatus is performed. For more detailed processing contents, the processing of the flowcharts shown in FIGS. 26, 27, and 28 is performed, and the same bi-predicted motion vector values and difference vectors as those of the moving image coding apparatus are determined. Is called.

  Next, with respect to the motion correction reference image that is the reference image for transmitting the determined difference vector, the difference vector for the motion vector value before correction stored as motion information is decoded (S4207), and the motion vector before correction is obtained. The motion vector value of the motion correction reference image is calculated by adding the difference vector to the value (S4208).

  For the motion information stored at the position indicated by the combined motion information index, the prediction type is set to bi-prediction, and reference image designation information for designating the second reference image when the motion information is uni-prediction The updated motion vector value in the motion correction reference image is stored as motion information (S4209), and is output from the combined motion correction motion compensation prediction decoding unit 3707 to the motion compensation prediction unit 608 and motion information memory 607 in FIG. The

  In the moving picture coding apparatus and the moving picture decoding apparatus in Embodiment 1, the motion information of more adjacent blocks is used as selection candidates for the motion compensated prediction in AVC which is the conventional moving picture coding method. Thus, the combined motion correction is a new prediction mode in which the surrounding motion information is registered as a candidate that can be used more effectively and the motion vector value correction information is encoded as a difference vector with respect to the surrounding motion information. Prediction mode was used. As a result, the motion information is highly correlated but the continuity of the motion is not sufficiently maintained, or the motion vector in the motion information of other blocks is not accurate, due to factors such as distortion when the prediction residual is encoded. For example, a motion compensated prediction signal that can correct an appropriate motion vector with a small amount of information and has a small prediction residual with a small amount of additional information with respect to motion information generated from adjacent blocks. An encoding device and a decoding device to be generated can be realized.

  In the video encoding device and video decoding device according to Embodiment 1, the setting of the reference image for correcting the motion vector is combined from the prediction mode stored in the surrounding motion information and the surrounding motion information. Based on the acquisition method when acquiring the motion information, the reliability of the motion vector in the combined motion information is evaluated, and the reference image that needs to be corrected is specified. Accordingly, there is an effect that it is possible to correct an appropriate motion vector without adding designation information to be transmitted at the time of encoding.

  In the video encoding device and video decoding device according to Embodiment 1, the reference reference image and the reference motion vector of the combined motion modified prediction mode are determined and calculated in common with the combined prediction mode. Accordingly, it is possible to provide a combined motion correction prediction mode that can encode a small amount of additional information while minimizing new processing for the combined motion correction prediction mode.

  Further, in the moving image encoding device and the moving image decoding device according to the first embodiment, even when the motion vector in the surrounding motion information is only a uni-prediction motion vector, it is multiplied by 1 or according to the position of the reference image. By multiplying by −1, it is possible to use a combined motion modified prediction mode in which bi-prediction is performed with a smaller load and code amount, and it is possible to increase motion information of adjacent blocks to be referred to in blocks to be encoded / decoded thereafter. Therefore, encoding efficiency can be improved.

(Embodiment 2)
Next, the second embodiment of the present invention will be described. The second embodiment is different from the first embodiment in scaling processing for combined motion information in the combined motion modified prediction mode. Since the configuration and processing other than the scaling processing for the combined motion information are the same as those in the first embodiment, only the differences from the first embodiment of the scaling processing for the combined motion information in the prediction mode determination unit 109 in FIG. 10 will be described here. .

  FIGS. 43A to 43C are diagrams showing the concept of scaling processing for combined motion information in the second embodiment. As the combined motion information, the prediction type calculated from the spatial adjacent block shown in FIG. 43A is bi-prediction, and the prediction type calculated from the spatial adjacent block shown in FIG. There is a space UniPred and a time BiPred calculated from the time concatenation block shown in FIG.

  The scaling processing for the combined motion information in the second embodiment uses the motion vector applied when calculating the combined motion information as it is in the space BiPred and the time BiPred in which bi-prediction motion information can be acquired. The motion vector value defined in the prediction is scaled by using the distance ratio between the encoding target image and the L0 reference image, the L1 reference image, the L0 reference image, and the L1 reference image as an integer. Thus, a motion vector of a reference image list that has not been used is generated.

That is, as shown in FIG. 23B, when only the L0 prediction motion vector value mvL0 is defined, the POC value of the L0 reference image is subtracted from the POC value of the encoding target image. When the inter-image distance is CurrL0Dist, the inter-image distance obtained by subtracting the POC value of the L1 reference image from the POC value of the encoding target image is CurrL1Dist, the scale factor is ScaleFactor, the scale factor ScaleFactor and the other motion vector The value mvL1 is calculated by the following formulas 3 and 4.
ScaleFactor = Int (CurrL1Dist / CurrL0Dist)
... (Formula 3)
mvL1 = ScaleFactor × mvL0
... (Formula 4)
Here, Int () in Equation 3 represents an integer value.

  Here, the derivation of the scale factor ScaleFactor is derived from the table of FIG. 49 that has been calculated and recorded in advance instead of performing the operation of Equation 3, and no division is performed. FIG. 49 is a table for obtaining a division result from the absolute values of CurrL0Dist and CurrL1Dist. When the signs of CurrL0Dist and CurrL1Dist are different, the sign of the value obtained by referring to the table of FIG. 49 is inverted. If the values of CurrL0Dist and CurrL1Dist are values outside the range of the table of FIG. 49, the scale factor ScaleFactor is set to 1.

  By performing the above processing on a motion vector defined by single prediction, the other vector can be generated from the motion vector defined by single prediction. This is a characteristic configuration that improves the coding efficiency in the present embodiment. Even when the motion information of the block adjacent to the processing target block is uni-predicted, the other vector is used without performing complicated processing. It is possible to generate and perform bi-prediction.

  In particular, the scaling process for the combined motion information in the second embodiment derives a scale coefficient that is an integer value using the distance ratio between the reference image of L0 and the reference image of L1, and thus a normal inter-image distance. A large error does not occur compared to the case of performing the scaling operation according to the above, and the scale factor corresponding to the distance between the encoding target image and the reference image of L0 and the reference image of L1 is tabulated in advance. Therefore, it is possible to use the joint motion modified prediction mode that performs bi-prediction with a small amount of processing, and it is possible to increase the motion information of adjacent blocks that are referred to in the blocks to be encoded / decoded later. Can be improved.

(Embodiment 3)
Next, the third embodiment of the present invention will be described. The third embodiment is different from the first embodiment in the configuration of the combined motion corrected motion compensated prediction generation unit 1009. The configuration and processing other than the configuration of the combined motion corrected motion compensated prediction generation unit 1009, the combined motion corrected motion compensation prediction generation unit 1009, and the motion information decoding processing in step S701 are the same as those in the first embodiment. Since they are the same, only the difference from the first embodiment of the combined motion corrected prediction mode evaluation value generation process in the combined motion corrected motion compensation prediction generation unit 1009 will be described here.

  FIG. 45 is a diagram illustrating a configuration of the combined motion corrected motion compensation prediction generation unit 1009. The combined motion correction motion compensation prediction generation unit 1009 includes a combined motion information determination unit 4504, a reference reference image / motion correction reference image selection unit 4500, a motion correction reference image motion vector acquisition unit 4501, and a combined motion information correction motion compensation prediction generation unit 4502. , And a difference vector calculation unit 4503.

  The combined motion information determination unit 4504 has a function of determining one combined motion information for the registered combined motion information candidate, and the reference reference image / motion correction reference image selection unit 4500 has determined one combined motion information. On the other hand, it has a function of specifying a reference image for correcting motion vector values by calculating bi-prediction motion information. The configuration provided with the combined motion information determination unit 4504 is a characteristic configuration according to the third embodiment of the present invention, and the additional processing is reduced while minimizing the processing load required for selecting motion information candidates. It is possible to perform encoding.

  FIG. 46 is a flowchart for explaining detailed operation of combined motion correction prediction mode evaluation value generation using the configuration of the combined motion correction motion compensation prediction generation unit 1009 of FIG.

  First, an index i using motion information is determined from the combined motion information list (S4600). In Embodiment 3, when the number of combined motion information candidate lists generated by the combined motion information candidate list generation process is num_of_index, the motion information in which the combined motion information candidate i = 0 at the head of the list is fixed Used as The operation in step S4600 is the detailed operation of the combined motion information determination unit 4504 in FIG. 45, and the operations in the subsequent steps are the same as the operations in the corresponding steps in FIG. 22 in the first embodiment. Description is omitted.

  FIG. 47 is a flowchart for explaining the detailed operation of step S701 in FIG. The motion information decoding process in step S701 in FIG. 7 is performed by the motion information bitstream decoding unit 3700, the prediction vector calculation unit 3701, the combined motion information calculation unit 3705, and the combined motion correction motion compensation prediction decoding unit 3707.

  The motion information decoding process is a process of decoding motion information from the encoded bit stream encoded with the syntax structure of FIG. First, the Skip flag is decoded in a predetermined unit of the encoded block (S4700). Thereafter, processing is performed in units of prediction blocks. When the Skip flag indicates the Skip mode (S4701: YES), joint prediction motion information decoding is performed (S4702).

  When the mode is not the Skip mode (S4701: NO), the merge flag is decoded (S4703). When the merge flag indicates 1 (S4704: YES), the process proceeds to joint prediction motion information decoding in step S4702.

  When the merge flag is not 1 (S4704: NO), the merge Mvd flag is decoded (S4705). When the merge Mvd flag is not 1 (S4706: NO), the motion prediction flag is decoded (S4707). When the motion prediction flag is single prediction (S4708: YES), that is, when inter_pred_flag in FIG. 35 indicates Pred_LC, single prediction motion information decoding is performed (S4709).

  When the motion prediction flag does not indicate single prediction (S4708: NO), bi-predictive motion information decoding is performed (S4710). If the merge Mvd flag is 1 (S4706: YES), combined motion corrected predicted motion information decoding is performed (S4711). The combined motion modified prediction motion information decoding process in step S4711 is the difference between the third embodiment and the first embodiment.

  FIG. 48 is a flowchart for explaining the detailed operation of step S4711 of FIG. In step S4711 of FIG. 47, a combined motion corrected predicted motion information decoding process is performed.

  First, the combined motion correction prediction mode is set as the prediction mode (S4800), and a combined motion information candidate list is generated (S4801). In step S4801, the same processing as step S1401 in the flowchart of FIG. 14 in the moving image encoding apparatus is performed.

  Subsequently, an index i using motion information is implicitly specified from the combined motion information candidate list (S4802). With respect to the index i, the same designation as in step S4600 of the flowchart of FIG. 46 in the moving picture coding apparatus of the third embodiment is performed, and the number of combined motion information candidate lists generated by the combined motion information candidate list generation process is set to num_of_index. In this case, the combined motion information candidate i = 0 at the top of the list is used as the confirmed motion information. Since the operation of each subsequent step is the same as the operation of each corresponding step in FIG. 42 of the first embodiment, the description thereof is omitted.

  With the above configuration and operation, one piece of motion information is specified according to a predetermined rule from the combined motion information candidate list used in the combined prediction mode, and a reference reference image and a reference motion vector of the combined motion correction prediction mode are determined and calculated. . Therefore, in the combined motion correction prediction mode, the combined motion information candidate list deletion unit 1201 in FIG. 12 and the processing of S1502 in FIG. Can be encoded with a small amount of additional information while minimizing the new processing for generating the image and minimizing the processing load required to select motion information candidates.

  In the combined motion information candidate list, at least one combined motion information candidate is registered in descending order of priority. For example, among the plurality of combined motion information candidates, combined motion information candidates are selected in descending order of the degree of adjacency with respect to the encoding target block, and are registered in the combined motion information candidate list. Further, the space candidate block is selected before the time candidate block. In the combined motion modified prediction mode of Embodiment 3, the process of implicitly specifying the index i using motion information (moving image encoding device: S4600 in FIG. 46, moving image decoding device: S4802 in FIG. 48) When the number of combined motion information candidate lists generated by the combined motion information candidate list generation process is num_of_index, the combined motion information candidate with i = 0 at the head of the list is determined as an index using motion information However, it is only necessary to have a function for specifying one index, and another deterministic algorithm may be used.

  For example, an algorithm for determining the combined motion information candidate of i = num_of_index-1, which is the last combined motion information candidate in the combined motion information candidate list, as an index, or the contents of combined motion information in ascending order of the index of the combined motion information candidate list. First, an index in which motion information (that is, bi-prediction motion information) in space BiPred or time BiPRed is first stored may be determined as an index in the combined motion correction prediction mode.

  When the index i = 0 is specified as an implicit index, the use of motion information with high priority in the joint prediction mode increases the possibility that valid motion information can be used, and the first motion information in the list. Since the motion information can be acquired in a fixed manner, there is an advantage that the motion information acquisition process can be configured simply.

  When the index i = num_of_index-1 is specified as an implicit index, by using motion information with low priority in the joint prediction mode, a difference vector is added to motion information that is difficult to select in the joint prediction mode, and effective motion is achieved. It is possible to transmit as information, and there is an advantage that the degree of duplication in the amount of information in selection with the combined prediction mode can be reduced.

  When the content of the combined motion information is confirmed in ascending order of the index of the combined motion information candidate list, and the index in which the motion information of space BiPred or time BiPRed is first stored is specified as an implicit index, effective motion information By preferentially using BiPred motion information that is likely to be stored more, there is an advantage that it is possible to increase the possibility of generating a motion compensated prediction block with good quality in the combined motion correction prediction mode. .

  Regarding the algorithm for specifying these indexes, in addition to the configuration in which one method is fixedly applied, the information for specifying the algorithm is encoded in a predetermined unit to selectively switch the index acquisition algorithm. Is also possible.

  The moving image encoded stream output from the moving image encoding apparatus of the embodiment described above has a specific data format so that it can be decoded according to the encoding method used in the embodiment. Therefore, the moving picture decoding apparatus corresponding to the moving picture encoding apparatus can decode the encoded stream of this specific data format.

  When a wired or wireless network is used to exchange an encoded stream between a moving image encoding device and a moving image decoding device, the encoded stream is converted into a data format suitable for the transmission form of the communication path. It may be transmitted. In that case, a video transmission apparatus that converts the encoded stream output from the video encoding apparatus into encoded data in a data format suitable for the transmission form of the communication channel and transmits the encoded data to the network, and receives the encoded data from the network Then, a moving image receiving apparatus that restores the encoded stream and supplies the encoded stream to the moving image decoding apparatus is provided.

  The moving image transmitting apparatus is a memory that buffers the encoded stream output from the moving image encoding apparatus, a packet processing unit that packetizes the encoded stream, and transmission that transmits the packetized encoded data via the network. Part. The moving image receiving apparatus generates a coded stream by packetizing the received data, a receiving unit that receives the packetized coded data via a network, a memory that buffers the received coded data, and packet processing. And a packet processing unit provided to the video decoding device.

  Further, the above-described processing relating to encoding and decoding can be realized as a transmission, storage, and reception device using hardware, and is stored in a ROM (Read Only Memory), a flash memory, or the like. It can also be realized by firmware or software such as a computer. The firmware program and software program can be recorded on a computer-readable recording medium, provided from a server through a wired or wireless network, or provided as a data broadcast of terrestrial or satellite digital broadcasting Is also possible.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  100 input terminals, 101 subtraction unit, 102 orthogonal transform / quantization unit, 103 prediction error coding unit, 104 inverse quantization / inverse transform unit, 105 addition unit, 106 decoded image memory, 107 motion vector detection unit, 108 motion compensation Prediction unit, 109 prediction mode determination unit, 110 motion information encoding unit, 111 motion information memory, 112 multiplexing unit, 113 output terminal, 600 input terminal, 601 demultiplexing unit, 602 prediction difference information decoding unit, 603 inverse quantization Inverse transformation unit, 604 addition unit, 605 decoded image memory, 606 motion information decoding unit, 607 motion information memory, 608 motion compensation prediction unit, 609 output terminal, 1000 motion compensation single prediction generation unit, 1001 motion compensation bi-prediction generation unit , 1002 prediction error calculation unit, 1003 prediction 1004 Coefficient calculation unit, 1004 Difference vector calculation unit, 1005 Motion information code amount calculation unit, 1006 Prediction mode evaluation unit, 1007 Coupling motion information calculation unit, 1008 Coupling motion compensation prediction generation unit, 1009 Coupling motion correction motion compensation prediction generation unit, 1100 Prediction vector candidate list generation unit 1101 Prediction vector candidate list deletion unit 1200 Combined motion information candidate list generation unit 1201 Combined motion information candidate list deletion unit 1300 Reference reference image / motion correction reference image selection unit 1301 Motion correction reference image Motion vector acquisition unit, 1302 combined motion information modified motion compensation prediction generation unit, 1303 difference vector calculation unit, 3700 motion information bitstream decoding unit, 3701 prediction vector calculation unit, 3702 vector addition unit, 3703 motion compensation Compensated uni-prediction decoding unit, 3704 motion compensation bi-prediction decoding unit, 3705 joint motion information calculation unit, 3706 joint motion compensation prediction decoding unit, 3707 joint motion correction motion compensation prediction decoding unit, 4500 reference reference image / motion correction reference image selection unit 4501 Modified reference image motion vector acquisition unit 4502 Information modified motion compensation prediction generation unit 4503 Difference vector calculation unit 4504 Combined motion information determination unit

Claims (5)

An image encoding device that detects and encodes a motion vector in units of blocks obtained by dividing each picture of a moving image,
A mode that uses an index of a reference block adjacent to the encoding target block and a difference vector between the motion vector of the encoding target block and the motion vector of the reference block can be selected as motion information of the encoding target block. A prediction mode determination unit;
A prediction error signal encoding unit that encodes a prediction error signal that is a difference between a prediction signal generated based on motion information supplied from the prediction mode determination unit and an image signal of the encoding target block;
A motion information encoding unit that encodes the motion information supplied from the prediction mode determination unit;
A multiplexing unit that multiplexes the encoded data supplied from the prediction error signal encoding unit and the encoded data supplied from the motion information encoding unit;
When the prediction type acquired from the motion information of the reference block is uni-prediction, the prediction mode determination unit uses the uni-prediction motion vector as a first-prediction motion vector in order to generate a bi-prediction motion vector. An image encoding apparatus for generating a motion vector for the second prediction by multiplying the motion vector for the first prediction by an integer.   The image encoding apparatus according to claim 1, wherein the integer multiple is −1 or 1 times.   The said motion information encoding part encodes the difference vector with the motion vector of the said encoding object block as said motion information about the motion vector of said 2nd prediction, The motion information of Claim 1 or 2 characterized by the above-mentioned. Image encoding device. An image encoding method for detecting and encoding a motion vector in units of blocks obtained by dividing each picture of a moving image,
A mode that uses an index of a reference block adjacent to the encoding target block and a difference vector between the motion vector of the encoding target block and the motion vector of the reference block can be selected as motion information of the encoding target block. A prediction mode determination step;
A prediction error signal encoding step for encoding a prediction error signal that is a difference between a prediction signal generated based on motion information supplied from the prediction mode determination step and an image signal of the encoding target block;
A motion information encoding step for encoding the motion information supplied from the prediction mode determination step;
A multiplexing step for multiplexing the encoded data supplied from the prediction error signal encoding step and the encoded data supplied from the motion information encoding step;
When the prediction type acquired from the motion information of the reference block is uni-prediction, the prediction mode determination step uses the uni-prediction motion vector as a first-prediction motion vector to generate a bi-prediction motion vector. An image encoding method comprising generating a motion vector for the second prediction by multiplying the motion vector for the first prediction by an integer. An image encoding program for detecting and encoding a motion vector in units of blocks obtained by dividing each picture of a moving image,
A mode that uses an index of a reference block adjacent to the encoding target block and a difference vector between the motion vector of the encoding target block and the motion vector of the reference block can be selected as motion information of the encoding target block. A prediction mode determination step;
A prediction error signal encoding step for encoding a prediction error signal that is a difference between a prediction signal generated based on motion information supplied from the prediction mode determination step and an image signal of the encoding target block;
A motion information encoding step for encoding the motion information supplied from the prediction mode determination step;
Causing the computer to execute a multiplexing step of multiplexing the encoded data supplied from the prediction error signal encoding step and the encoded data supplied from the motion information encoding step;
When the prediction type acquired from the motion information of the reference block is uni-prediction, the prediction mode determination step uses the uni-prediction motion vector as a first-prediction motion vector to generate a bi-prediction motion vector. An image encoding program for generating a motion vector for the second prediction by multiplying the motion vector for the first prediction by an integer.

Images