JP2013021573A - Image decoder, image decoding method, and image decoding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To further improve efficiency for encoding motion information which includes a motion vector.SOLUTION: A candidate list generating section generates a candidate list by selecting a plurality of blocks which include motion information, including at least motion vector information and reference image information, from a plurality of decoded blocks adjacent to a decoding target block. A motion information acquisition section obtains the motion information of the blocks included in the candidate list. A scaling section generates a second motion vector by scaling a first motion vector included in the motion information obtained by the motion information acquisition section. A selection-candidate generating section generates a new selection candidate comprising the motion information which includes the first motion vector and motion information which includes the second motion vector. A code row analysis section decodes candidate-specifying information for specifying a selected candidate which is selected from the candidate list at an encoding side in the candidate list. A selection section selects one candidate from selection candidates included in the candidate list by using the decoded candidate-specifying information.

Description

  The present invention relates to a moving image encoding technique using motion compensated prediction, and more particularly to an image decoding apparatus, an image decoding method, and an image decoding program for decoding motion information used in motion compensated prediction.

  In general video compression coding, motion compensation prediction is used. Motion compensation prediction divides the target image into fine blocks, uses the decoded image as a reference image, and based on the amount of motion indicated by the motion vector, the position of the position moved from the target block of the target image to the reference block of the reference image This is a technique for generating a signal as a prediction signal. Some motion compensation predictions are performed unidirectionally using one motion vector, and others are performed bidirectionally using two motion vectors.

  As for the motion vector, the motion vector of the encoded block adjacent to the processing target block is used as a prediction motion vector (also simply referred to as “prediction vector”), and the difference between the motion vector of the processing target block and the prediction vector is obtained. The compression efficiency is improved by transmitting the difference vector as an encoded vector.

  MPEG-4AVC improves the efficiency of motion compensation prediction by making the block size of motion compensation prediction finer and more diversified than MPEG-2. On the other hand, since the number of motion vectors increases by making the block size finer, the code amount of the encoded vector becomes a problem.

  Therefore, in MPEG-2, the left motion vector of the processing target block is simply used as the prediction vector, but in MPEG-4 AVC, the accuracy of the prediction vector is improved by using the median value of the motion vectors of a plurality of adjacent blocks as the prediction vector. Therefore, an increase in the code amount of the encoded vector is suppressed. Further, direct motion compensation prediction is known in MPEG-4 AVC. Direct motion compensated prediction generates a new motion vector by scaling the motion vector of a block at the same position as the processing target block of another encoded image by the distance between the target image and two reference images. The motion compensation prediction is realized without transmitting the quantization vector.

  In addition, motion compensation prediction that realizes motion compensation prediction without transmitting an encoded vector using motion information of a block adjacent to a processing target block is known (see, for example, Patent Document 1).

JP-A-10-276439

  As described above, direct motion compensated prediction that does not transmit an encoded vector focuses on the continuity of motion of a block that is in the same position as a processing target block and a processing target block of another encoded image. Further, Patent Document 1 focuses on the continuity of movement of a processing target block and a block adjacent to the processing target block. As a result, by using the motion information of other blocks, the encoding efficiency is improved without encoding the motion information including the difference vector as the encoded vector.

  However, in the conventional motion compensated prediction, the motion of the processing target block is shifted to the motion of the processing target block and the adjacent block, or the motion of a block around the same position as the processing target block of another encoded image. In such a case, the motion information including the difference vector must be encoded, and there is a difficult aspect that the improvement of the encoding efficiency is not sufficiently exhibited.

  The present invention has been made in view of such circumstances, and an object thereof is to provide a technique for further improving the efficiency of encoding motion information including a motion vector.

  In order to solve the above problems, an image decoding apparatus according to an aspect of the present invention is an image decoding apparatus that performs motion compensation prediction, and includes information on motion vectors from a plurality of decoded blocks adjacent to a decoding target block. A candidate list generation unit (230) that generates a candidate list by selecting a plurality of blocks having motion information including at least reference image information, and a motion information acquisition unit that acquires motion information of blocks included in the candidate list (150), a scaling unit (162) that generates a second motion vector by scaling a first motion vector included in the motion information acquired by the motion information acquisition unit (150), and the first information A selection candidate generating unit (151) for generating a new selection candidate having motion information including a motion vector and motion information including the second motion vector The candidate list is decoded using a code string analyzing unit (201) for decoding candidate specifying information for specifying a candidate selected from the candidate list on the encoding side in the candidate list, and the decoded candidate specifying information. A selection unit (231) that selects one candidate from the selection candidates included in the candidate list generated by the generation unit (230). “The plurality of decoded blocks adjacent to the decoding target block” may include a block of an image that is temporally different from the image including the decoding target block.

  When the number of blocks included in the candidate list is less than the set maximum number, a new selection candidate generated by the selection candidate generation unit (151) may be added to the candidate list.

  The plurality of decoded blocks adjacent to the decoding target block may include a first type having unidirectional motion information and a second type having bidirectional information. The motion information acquisition unit (150) may preferentially acquire the first type block among the plurality of blocks included in the candidate list, or may preferentially acquire the second type block. You may get it.

  The motion information acquisition unit (150) may preferentially acquire a block having a relatively high degree of adjacency with the decoding target block among a plurality of blocks included in the candidate list.

  When the motion information acquired by the motion information acquisition unit (150) is motion information of a block having a bidirectional motion vector, the scaling unit (162) is longer than the two motion vectors of the block. The first motion vector may be scaled as the first motion vector, or the shorter motion vector may be scaled as the first motion vector.

  The image processing apparatus may further include a reference image setting unit (161) that sets a reference index value that specifies a reference image of the second motion vector generated by the scaling unit (162) to zero.

  The reference image setting unit (161), when the value of the reference index that identifies the reference image of the first motion vector is the same as the value of the reference index that identifies the reference image of the second motion vector, A reference index value that specifies a reference image of a motion vector of a block of the temporally different image may be set to a reference index value that specifies a reference image of the second motion vector. 1 may be set instead of the value of the reference index for specifying the reference image of the motion vector of the block of the temporally different image.

  The motion information acquisition unit (150) includes motion information in the first prediction direction from the first block included in the candidate list, and motion information in the second prediction direction from the second block included in the candidate list. May be obtained. The selection candidate generation unit (151, 154) is based on the motion information in the first prediction direction and the motion information in the second prediction direction acquired by the motion information acquisition unit (150). A new selection candidate having one piece of motion information may be generated.

  The scaling unit (162), when the motion information in the first prediction direction or the motion information in the second prediction direction acquired by the motion information acquisition unit (150) is invalid, the effective prediction The second motion vector may be generated by scaling a motion vector included in the direction motion information as the first motion vector.

  Another aspect of the present invention is also an image decoding method. This apparatus is an image decoding method for performing motion compensation prediction, and a plurality of blocks having motion information including at least motion vector information and reference image information from a plurality of decoded blocks adjacent to a decoding target block. Generating a candidate list by selecting, a step of acquiring motion information of blocks included in the candidate list, and a second motion vector by scaling the first motion vector included in the acquired motion information , Generating a new selection candidate having motion information including the first motion vector and motion information including the second motion vector, and selecting from a candidate list on the encoding side Decoding candidate specifying information for specifying a candidate that is made in the candidate list, and using the decoded candidate specifying information, And a step of selecting one candidate from among the selection candidates included in the serial candidate list.

  Another aspect of the present invention is an image encoding device. This apparatus is an image encoding apparatus that performs motion compensation prediction, and has motion information including at least motion vector information and reference image information from a plurality of encoded blocks adjacent to an encoding target block. A candidate list generation unit (140) that generates a candidate list by selecting a plurality of blocks, a motion information acquisition unit (150) that acquires motion information of blocks included in the candidate list, and the motion information acquisition unit (150) ), A scaling unit (162) that generates a second motion vector by scaling the first motion vector included in the motion information acquired by (2), motion information including the first motion vector, the second motion vector, A selection candidate generation unit (151) for generating a new selection candidate having motion information including a motion vector, and a plurality of selection candidates included in the candidate list A selection unit (141) for selecting one from among, a code string generation unit (104) for encoding candidate specifying information for specifying a candidate selected by the selection unit (141) in the candidate list, . “The plurality of encoded blocks adjacent to the encoding target block” may include a block of an image that is temporally different from an image including the encoding target block.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, and the like are also effective as an aspect of the present invention.

  According to the present invention, it is possible to further improve the encoding efficiency of motion information including motion vectors.

It is a figure for demonstrating the example which divides | segments an image into the largest encoding block. 2A and 2B are diagrams for explaining an encoded block. 3A to 3D are diagrams for explaining a prediction block. It is a figure for demonstrating a prediction block size. It is a figure for demonstrating prediction encoding mode. FIGS. 6A to 6D are diagrams for explaining the prediction direction of motion compensation prediction. It is a figure for demonstrating an example of the syntax of a prediction block. FIGS. 8A to 8C are diagrams for explaining the truncated unary code string of the merge index. It is a figure for demonstrating the structure of the moving image encoder which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the management method of the motion information in the motion information memory of FIG. It is a figure for demonstrating the structure of the motion information generation part of FIG. It is a figure for demonstrating the structure of the difference vector calculation part of FIG. It is a figure for demonstrating a space candidate block group. It is a figure for demonstrating a time candidate block group. It is a figure for demonstrating the structure of the joint motion information determination part of FIG. It is a figure for demonstrating the structure of the joint motion information candidate production | generation part of FIG. It is a figure for demonstrating the structure of the scaling joint motion information candidate production | generation part of FIG. It is a figure for demonstrating a candidate number management table. FIGS. 19A and 19B are diagrams for explaining conversion from a merge candidate number to a merge index. It is a flowchart for demonstrating the operation | movement of the encoding of the moving image encoder which concerns on Embodiment 1 of this invention. 10 is a flowchart for explaining an operation of a motion information generation unit in FIG. 9. 12 is a flowchart for explaining an operation of a difference vector calculation unit in FIG. 11. It is a flowchart for demonstrating operation | movement of the joint motion information determination part of FIG. 16 is a flowchart for explaining an operation of a combined motion information candidate generation unit in FIG. 15. It is a flowchart for demonstrating operation | movement of the production | generation of a space joint motion information candidate list. It is a flowchart for demonstrating the operation | movement of the production | generation of a time combination motion information candidate list. It is a figure for demonstrating the calculation method of the motion vector mvL0t of a time joint motion information candidate, and mvL1t. It is a flowchart for demonstrating operation | movement of the production | generation of a scaling joint motion information candidate. It is a figure for demonstrating the calculation method of the motion vector of the LY direction of a scaling joint motion information candidate. It is a figure for demonstrating the structure of the moving image decoding apparatus which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the structure of the motion information reproduction | regeneration part of FIG. It is a figure for demonstrating the structure of the motion vector reproduction | regeneration part of FIG. FIG. 32 is a diagram for describing a configuration of a combined motion information reproduction unit in FIG. 31. It is a flowchart for demonstrating the decoding operation | movement of the moving image decoding apparatus which concerns on Embodiment 1 of this invention. FIG. 31 is a flowchart for explaining an operation of a motion information reproducing unit in FIG. 30. FIG. FIG. 32 is a flowchart for explaining an operation of a motion vector reproducing unit in FIG. 31. FIG. FIG. 32 is a flowchart for explaining an operation of a combined motion information reproducing unit in FIG. 31. FIG. FIGS. 38A and 38B are diagrams for explaining a candidate number management table according to the first modification of the first embodiment. FIG. 10 is a diagram for explaining another candidate number management table according to the first modification of the first embodiment. 12 is a flowchart for explaining an operation of generating scaling combined motion information candidates according to the third modification of the first embodiment. 10 is a flowchart for explaining an operation of generating scaling combined motion information candidates according to the fifth modification of the first embodiment. 18 is a flowchart for explaining an operation of generating another scaling combined motion information candidate according to the fifth modification of the first embodiment. It is a figure for demonstrating the relationship between a process target image and a reference image. FIG. 6 is a diagram for explaining an effect of the first embodiment. FIG. 10 is a diagram for explaining a candidate number management table according to the second embodiment. 10 is a diagram for explaining a configuration of a combined motion information candidate generation unit according to Embodiment 2. FIG. FIG. 47 is a diagram for describing a configuration of a dual coupled motion information candidate generation unit in FIG. 46. 10 is a flowchart for explaining an operation of a combined motion information candidate generation unit according to the second embodiment. 47 is a flowchart for explaining detailed operations of a dual-coupled motion information candidate generation unit in FIG. 46. It is a flowchart for demonstrating operation | movement of the reference direction movement information determination part of FIG. It is a flowchart for demonstrating operation | movement of the reverse direction motion information determination part of FIG. It is a figure for demonstrating determination of the prediction direction of a bi-joint motion information candidate. 10 is a flowchart for explaining an operation of generating scaling combined motion information candidates according to the second embodiment. 18 is a flowchart for explaining an operation of a combined motion information candidate generation unit according to Modification 4 of Embodiment 2. FIG. 10 is a diagram for explaining an effect of the second embodiment.

  First, a technique that is a premise of the embodiment of the present invention will be described.

  Currently, apparatuses and systems that comply with an encoding method such as MPEG (Moving Picture Experts Group) are widely used. In such an encoding method, a plurality of images that are continuous on the time axis are handled as digital signal information. At that time, for the purpose of broadcasting, transmitting or storing highly efficient information, compression using motion compensation prediction using temporal redundancy and orthogonal transform such as discrete cosine transform using spatial redundancy Encode.

  In 1995, the MPEG-2 video (ISO / IEC 13818-2) encoding method was established as a general-purpose video compression encoding method, such as a magnetic tape based on DVD and D-VHS (registered trademark) digital VTR. It is widely used as an application for storage media and digital broadcasting.

  Furthermore, in 2003, joint work of the International Technical Organization (ISO) and the International Electrotechnical Commission (IEC) Joint Technical Committee (ISO / IEC) and the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) -4 AVC / H. An encoding method called H.264 (the ISO / IEC has a standard number of 14496-10 and ITU-T has an H.264 standard number, hereinafter referred to as MPEG-4AVC) has been established as an international standard.

  Coding currently called HEVC by the joint work of the International Technical Organization (ISO) and the International Electrotechnical Commission (IEC) Joint Technical Committee (ISO / IEC) and the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) Standardization of the method is being studied.

(Encoding block)
In the embodiment of the present invention, an input image signal is divided into maximum coding block units as shown in FIG. 1, and the divided coding blocks are processed in a raster scan order. FIG. 1 is a diagram for explaining an example in which an image is divided into maximum coding blocks. The encoded block has a hierarchical structure, and can be made smaller encoded blocks by sequentially equally dividing into 4 in consideration of the encoding efficiency. Note that the encoded blocks divided into four are encoded in the zigzag scan order. An encoded block that cannot be further reduced is called a minimum encoded block. An encoded block is a unit of encoding, and the maximum encoded block is also an encoded block when the number of divisions is zero. In this embodiment, the maximum coding block is 64 pixels × 64 pixels, and the minimum coding block is 8 pixels × 8 pixels.

  2A and 2B are diagrams for explaining an encoded block. In the example of FIG. 2A, the encoded block is divided into ten. CU0, CU1 and CU9 are 32 × 32 pixel coding blocks, CU2, CU3 and CU8 are 16 × 16 pixel coding blocks, and CU4, CU5, CU6 and CU7 are 8 × 8 pixel coding blocks. It has become. In the example of FIG. 2B, the encoded block is divided into one.

(Prediction block)
In the embodiment of the present invention, the encoded block is further divided into prediction blocks. 3A to 3D are diagrams for explaining a prediction block. 3A is 2N × 2N that does not divide the encoded block, FIG. 3B is 2N × N that is horizontally divided, FIG. 3C is N × 2N that is vertically divided, and FIG. 3D. Indicates N × N divided horizontally and vertically. FIG. 4 is a diagram for explaining the prediction block size. In other words, as shown in FIG. 4, the prediction block size includes a CU division number of 0 and a maximum prediction block size of 64 pixels × 64 pixels to a CU division number of 3 and a minimum prediction block size. There are 13 predicted block sizes up to 4 pixels x 4 pixels.

  In the embodiment of the present invention, the maximum coding block is 64 pixels × 64 pixels and the minimum coding block is 8 pixels × 8 pixels, but the present invention is not limited to this combination. Further, although the prediction block division patterns are shown in FIGS. 3A to 3D, the division is not limited to this as long as it is divided into one or more.

(Predictive coding mode)
In the embodiment of the present invention, the prediction direction of motion compensation prediction and the number of encoded vectors can be switched by the block size of the prediction block. Here, an example of a predictive coding mode in which the prediction direction of motion compensation prediction and the number of coding vectors are associated will be briefly described with reference to FIG. FIG. 5 is a diagram for explaining the predictive coding mode.

  The prediction coding mode shown in FIG. 5 includes a unidirectional mode (UniPred) in which the prediction direction of motion compensation prediction is unidirectional and the number of coding vectors is 1, and the prediction direction of motion compensation prediction is bidirectional. There is a bidirectional mode (BiPred) in which the number of encoded vectors is 2, and a merge mode (MERGE) in which the prediction direction of motion compensation prediction is unidirectional or bidirectional and the number of encoded vectors is 0. There is also an intra mode (Intra) which is a predictive coding mode in which motion compensation prediction is not performed.

(Reference image index)
In the embodiment of the present invention, it is possible to select an optimal reference image from a plurality of reference images in motion compensation prediction in order to improve the accuracy of motion compensation prediction. Therefore, the reference image used in motion compensation prediction is encoded as a reference image index together with the encoded vector. The reference image index used in motion compensation prediction is a numerical value of 0 or more.

  A plurality of reference images that can be selected by the reference image index are managed in a reference index list. If the prediction direction of motion compensation prediction is unidirectional, one reference image index is encoded. If the prediction direction of motion compensation prediction is bidirectional, a reference image index indicating a reference image in each prediction direction is encoded. (FIG. 5).

(Predicted vector index)
In HEVC, in order to improve the accuracy of a prediction vector, it is considered to select an optimal prediction vector from among a plurality of prediction vector candidates and to encode a prediction vector index for indicating the selected prediction vector. Yes. In the embodiment of the present invention, the prediction vector index is introduced. If the prediction direction of motion compensation prediction is unidirectional, one prediction vector index is encoded. If the prediction direction of motion compensation prediction is bidirectional, a prediction vector index indicating a prediction vector in each prediction direction is encoded. (FIG. 5).

(Merge index)
In HEVC, in order to further improve the coding efficiency, an optimum block is selected from among a plurality of adjacent block candidates and a block located at the same position as a processing target block of another coded image, and the selected block is selected. Encoding and decoding a merge index indicating This is a motion compensation prediction technique (merge technique) in which motion information including a prediction direction, motion vector information, and reference image information of a motion compensation prediction of a block indicated by a selected merge index is used in a processing target block. In the embodiment of the present invention, the above-described merge index (merge technique) is introduced. As shown in FIG. 5, one merge index is encoded when the prediction encoding mode is the merge mode. If the motion information is bidirectional, the motion information includes motion vector information and reference image information in each prediction direction.

  Hereinafter, the motion information of a block that may be indicated by the merge index is referred to as a combined motion information candidate, and the aggregate of combined motion information candidates is referred to as a combined motion information candidate list.

(Forecast direction)
In the embodiment of the present invention, two directions of the L0 direction and the L1 direction are set as the prediction directions of the motion compensation prediction. FIGS. 6A to 6D are diagrams for explaining the prediction direction of motion compensation prediction. The prediction direction of motion compensation prediction will be briefly described with reference to FIGS. When the prediction direction of motion compensation prediction is unidirectional, either the L0 direction or the L1 direction is used. FIG. 6A shows a case where the reference image (RefL0Pic) in the unidirectional direction and the L0 direction is at a time before the encoding target image (CurPic). FIG. 6B shows a case where the reference image in the unidirectional direction and the L0 direction is at a time after the encoding target image. The reference image in the L0 direction in FIGS. 6A and 6B may be replaced with a reference image in the L1 direction (RefL1Pic).

  In the case of bidirectional, the BI direction is expressed using two of the L0 direction and the L1 direction. FIG. 6C shows a case where the reference image in the L0 direction is at a time before the encoding target image and the reference image in the L1 direction is at a time after the encoding target image. . FIG. 6D shows a case in which the reference image in the L0 direction and the reference image in the L1 direction are at a time before the encoding target image. The reference image in the L0 direction in FIGS. 6C and 6D may be replaced with the reference image in the L1 direction (RefL1Pic), and the reference image in the L1 direction may be replaced with the reference image in the L0 direction (RefL0Pic). As described above, the L0 direction and the L1 direction, which are prediction directions of motion compensation prediction, can be indicated in either the forward direction or the backward direction in terms of time.

  The L0 direction and the L1 direction can be predicted from a plurality of reference images, respectively. The reference image in the L0 direction is registered in the reference image list L0, and the reference image in the L1 direction is registered in the reference image list L1, The position of the reference image in the reference image list is designated by the reference image index in each prediction direction, and the reference image is determined. Hereinafter, the prediction direction is the L0 direction is a prediction direction that uses motion information associated with the reference image registered in the reference image list L0, and the prediction direction is the L1 direction is registered in the reference image list L1. The prediction direction uses motion information associated with the reference image.

(Syntax)
An example of the syntax of the prediction block according to the embodiment of the present invention will be described with reference to FIG. Whether the prediction block is intra or inter is specified by the higher-order encoding block, and FIG. 7 shows the syntax when the prediction block is inter. The prediction block includes a merge flag (merge_flag), a merge index (merge_idx), an inter prediction mode (inter_pred_type), a reference index (ref_idx_l0 and ref_idx_l1), and a difference vector (mvd_l0 [0], mvd_l0 [1], mvd_l mvd_l1 [1]) and prediction vector indexes (mvp_idx_l0 and mvp_idx_l1) are installed. [0] of the difference vector indicates a horizontal component and [1] indicates a vertical component.

  Here, ref_idx_l0, mvd_l0 [0], mvd_l0 [1], and mvp_idx_l0 are information regarding the L0 direction, and ref_idx_l1, mvd_l1 [0], mvd_l1 [1], and mvp_idx_l1 are information regarding the L1 direction. There are three types of inter_pred_type: Pred_L0 (unidirectional in the L0 direction), Pred_L1 (unidirectional in the L1 direction), and Pred_BI (bidirectional in the BI).

(Code amount of motion information)
As can be seen from the syntax of FIG. 7, the merge mode can transmit motion information with one merge index. Therefore, if the prediction errors in the merge mode (merge flag is 1) and non-merge mode (merge flag is 0) are about the same, the merge mode can more efficiently encode motion information. In other words, the efficiency of motion information encoding can be improved by increasing the selection rate of the merge mode.

  The syntax of the prediction block according to the embodiment of the present invention is set as shown in FIG. 7, but according to the embodiment of the present invention, the motion information is encoded with less information in the merge mode than in the non-merge mode. What is necessary is not limited to this.

(Characteristics of merge index)
In FIG. 7, NumMergeCands (), which is a function for calculating the number of merge candidates before the merge index decoding (encoding), calculates the number of prediction vector candidates before the prediction vector index decoding (encoding). NumMvpCands () is installed. These are functions necessary for obtaining the number of candidates because the number of merge candidates and the number of prediction vector candidates change for each prediction block depending on the validity of motion information of adjacent blocks. Note that the motion information of an adjacent block is valid means that the adjacent block is not a block outside the area or the intra mode, and the motion information of the adjacent block is invalid means that the adjacent block is out of the area. It is a block or intra mode.

  If the number of merge candidates is 1, the merge index is not decoded (encoded). This is because when the number of merge candidates is 1, it can be uniquely determined without specifying. The same applies to the prediction vector index.

  The merge index code string will be described with reference to FIGS. In the embodiment of the present invention, a Trunked Unary code string is used as the code string of the merge index. 8A shows a merge index code string based on a truncated unary code string when the number of merge candidates is two, and FIG. 8B shows a merge based on a truncated unary code string when the number of merge candidates is three. FIG. 8C shows a code sequence of the index, and FIG. 8C shows a code sequence of the merge index by the Truncated Unary code sequence when the number of merge candidates is four.

  8A to 8C that even when the same merge index value is encoded, the smaller the number of merge candidates, the smaller the number of code bits assigned to the merge index. For example, if the merge index is 1, if the number of merge candidates is two, it is 1 bit of “1”, but if the number of merge candidates is 3 or 4, it is 2 bits of “10”. Become.

  As described above, the encoding efficiency of the merge index improves as the number of merge candidates decreases. In other words, it is possible to improve the merge index encoding efficiency by leaving candidates having a relatively high selectivity and reducing candidates having a relatively low selectivity. In addition, when the number of candidates is the same, the code amount of the smaller merge index is smaller, so that the encoding efficiency can be improved by assigning a small merge index to a candidate having a relatively high selectivity. .

(POC)
In the embodiment of the present invention, POC (Picture Order Count) is used as time information (distance information) of an image. POC is a counter indicating the display order of images defined by MPEG-4 AVC. When the image display order is increased by 1, the POC is also increased by 1. Therefore, the time difference (distance) between images can be acquired from the POC difference between images.

(Characteristics of motion information of adjacent blocks)
In general, the degree of correlation between the motion information of the processing target block and the motion information of the block adjacent to the processing target block (hereinafter referred to as an adjacent block) is high when the processing target block and the adjacent block have the same motion. For example, this is a case where the region including the processing target block and the adjacent block is translated. In general, the degree of correlation between the motion information of the processing target block and the motion information of the adjacent block also depends on the length of contact between the processing target block and the adjacent block.

(Characteristics of motion information of another image)
On the other hand, a block in the same position as the processing target block (hereinafter referred to as the same position block) on another decoded image generally used in the temporal direct mode or the spatial direct mode, and the processing target block The case where the degree of correlation is high is a case where the same position block and the processing target block are in the same motion information, that is, in a uniform motion.

  Hereinafter, with reference to the drawings, details of a moving image encoding apparatus, a moving image encoding method, a moving image encoding program, a moving image decoding apparatus, a moving image decoding method, and a moving image decoding program according to a preferred embodiment of the present invention will be described. explain. In the description of the drawings, the same elements are denoted by the same reference numerals and redundant description is omitted.

[Embodiment 1]
(Configuration of moving picture coding apparatus 100)
FIG. 9 shows a configuration of moving picture coding apparatus 100 according to Embodiment 1 of the present invention. The moving image encoding apparatus 100 is an apparatus that encodes a moving image signal in units of prediction blocks for performing motion compensated prediction. It is assumed that the encoding block division, the prediction block size determination, and the prediction encoding mode determination are determined by a higher-level encoding control unit (not shown).

  The moving image encoding apparatus 100 is realized by hardware such as an information processing apparatus including a CPU (Central Processing Unit), a frame memory, and a hard disk. The moving image encoding apparatus 100 realizes functional components described below by operating the above components. Note that the position information of the prediction block to be processed, the prediction block size, and the prediction direction of motion compensated prediction are assumed to be shared in the video encoding device 100 and are not shown.

  The moving image encoding apparatus 100 according to Embodiment 1 includes a prediction block image acquisition unit 101, a subtraction unit 102, a prediction error encoding unit 103, a code string generation unit 104, a prediction error decoding unit 105, a motion compensation unit 106, and an addition unit. 107, a motion vector detection unit 108, a motion information generation unit 109, a frame memory 110, and a motion information memory 111.

(Function of moving picture coding apparatus 100)
Hereinafter, functions of each unit will be described. The prediction block image acquisition unit 101 acquires the image signal of the prediction block to be processed from the image signal supplied from the terminal 10 based on the position information and the prediction block size of the prediction block, and subtracts the image signal of the prediction block 102, and supplied to the motion vector detection unit 108 and the motion information generation unit 109.

  The subtraction unit 102 subtracts the image signal supplied from the prediction block image acquisition unit 101 and the prediction signal supplied from the motion compensation unit 106 to calculate a prediction error signal, and calculates the prediction error signal to the prediction error encoding unit 103. To supply.

  The prediction error encoding unit 103 performs processing such as quantization and orthogonal transformation on the prediction error signal supplied from the subtraction unit 102 to generate prediction error encoded data, and encodes the prediction error encoded data. The data is supplied to the column generation unit 104 and the prediction error decoding unit 105.

  The code string generation unit 104 includes the prediction error encoded data supplied from the prediction error encoding unit 103, the merge flag supplied from the motion information generation unit 109, the merge candidate number, the prediction direction of motion compensation prediction, and the reference image index. The difference vector and the prediction vector index are entropy-encoded according to the syntax together with the prediction direction of motion compensation prediction to generate a code string, and the code string is supplied to the terminal 11.

  Here, the merge candidate number is converted into a merge index to generate a code string. Here, the merge candidate number is a number indicating the selected combined motion information candidate. The conversion from the merge candidate number to the merge index will be described later. In the first embodiment, a truncated unity code string is used for encoding a merge index and a prediction vector index as described above. However, the code string is not limited to this as long as the number of candidates is small and the code string can be encoded with fewer bits. .

  The prediction error decoding unit 105 performs a process such as inverse quantization or inverse orthogonal transform on the prediction error encoded data supplied from the prediction error encoding unit 103 to generate a prediction error signal, and the prediction error signal Is supplied to the adder 107.

  The motion compensation unit 106 performs motion compensation on the reference image in the frame memory 110 indicated by the reference image index supplied from the motion information generation unit 109 based on the motion vector supplied from the motion information generation unit 109, and generates a prediction signal. Generate. If the prediction direction is bidirectional, the prediction signal is obtained by averaging the prediction signals in the L0 direction and the L1 direction.

  The addition unit 107 adds the prediction error signal supplied from the prediction error decoding unit 105 and the prediction signal supplied from the motion compensation unit 106 to generate a decoded image signal, and supplies the decoded image signal to the frame memory 110. To do.

  The motion vector detection unit 108 detects a motion vector and a reference image index indicating the reference image from the image signal supplied from the prediction block image acquisition unit 101 and an image signal corresponding to a plurality of reference images, and the motion vector and the motion vector The reference image index is supplied to the motion information generation unit 109. If the prediction direction is bidirectional, motion vectors and reference image indexes in the L0 direction and the L1 direction are detected.

  A general motion vector detection method calculates an error evaluation value for an image signal corresponding to a reference image moved by a predetermined movement amount from the same position as the image signal of the target image, and moves to minimize the error evaluation value. Let the amount be a motion vector. As the error evaluation value, SAD (Sum of Absolute Difference) indicating the sum of absolute differences, MSE (Mean Square Error) indicating the mean square error, or the like can be used.

  The motion information generation unit 109 uses the motion vector and reference image index supplied from the motion vector detection unit 108, the candidate block group supplied from the motion information memory 111, and the reference image indicated by the reference image index in the frame memory 110. , A merge candidate number, or a difference vector and a prediction vector index, and a code flag generation unit 104, a motion compensation unit 106, and a merge flag, a merge candidate number, a reference image index, a difference vector, and a prediction vector index The motion information memory 111 is supplied. A detailed configuration of the motion information generation unit 109 will be described later.

  The frame memory 110 stores the decoded image signal supplied from the adding unit 107. In addition, for a decoded image in which decoding of the entire image is completed, a predetermined number of images of 1 or more is stored as a reference image. The frame memory 110 supplies the stored reference image signal to the motion compensation unit 106 and the motion information generation unit 109. A storage area for storing the reference image is controlled by a FIFO (First In First Out) method.

  The motion information memory 111 stores the motion information supplied from the motion information generation unit 109 for a predetermined number of images in units of the minimum predicted block size. The motion information of the adjacent block of the prediction block to be processed is set as a space candidate block group.

  In addition, the motion information memory 111 sets the motion information of the block on the ColPic located in the same position as the prediction block to be processed and its neighboring blocks as a time candidate block group. The motion information memory 111 supplies the spatial candidate block group and the temporal candidate block group to the motion information generation unit 109 as candidate block groups. The motion information memory 111 is synchronized with the frame memory 110 and is controlled by a FIFO (First In First Out) method.

  Here, ColPic refers to a decoded image different from the prediction block to be processed and stored in the frame memory 110 as a reference image. In Embodiment 1, ColPic is a reference image decoded immediately before. In Embodiment 1, ColPic is a reference image decoded immediately before. However, it may be an encoded image, for example, a reference image immediately before in display order or a reference image immediately after in display order may be used. It can also be specified in the encoded stream.

  Here, a method of managing motion information in the motion information memory 111 will be described with reference to FIG. The motion information is stored in each memory area in units of the smallest prediction block. FIG. 10 shows a state where the predicted block size to be processed is 16 pixels × 16 pixels. In this case, the motion information of the prediction block is stored in 16 memory areas indicated by hatching in FIG.

  When the predictive coding mode is the intra mode, (0, 0) is stored as the motion vector in the L0 direction and the L1 direction, and “−1” is stored as the reference image index in the L0 direction and the L1 direction. The reference image index “−1” may be any value as long as it can be determined that the motion compensation prediction is not performed. From this point onward, unless expressed otherwise, the term “block” refers to the smallest predicted block unit when expressed simply as a block. Also, in the case of a block outside the area, as in the intra mode, (0, 0) is stored as the motion vector in the L0 direction and the L1 direction, and “−1” is stored as the reference image index in the L0 direction and the L1 direction. . The LX direction (X is 0 or 1) is valid when the reference image index in the LX direction is 0 or more, and the LX direction is invalid (not valid). The reference image index in the LX direction is “− 1 ”.

  Next, a detailed configuration of the motion information generation unit 109 will be described with reference to FIG. FIG. 11 shows the configuration of the motion information generation unit 109. The motion information generation unit 109 includes a difference vector calculation unit 120, a combined motion information determination unit 121, and a predictive coding mode determination unit 122. The terminal 12 is in the motion information memory 111, the terminal 13 is in the motion vector detection unit 108, the terminal 14 is in the frame memory 110, the terminal 15 is in the prediction block image acquisition unit 101, the terminal 16 is in the code string generation unit 104, and the terminal 50 Are connected to the motion compensation unit 106, and the terminal 51 is connected to the motion information memory 111, respectively.

  Hereinafter, functions of each unit will be described. The difference vector calculation unit 120 predicts from the candidate block group supplied from the terminal 12, the motion vector and reference image index supplied from the terminal 13, the reference image supplied from the terminal 14, and the image signal supplied from the terminal 15. A vector index is determined, and a difference vector and a rate distortion evaluation value are calculated. Then, the reference image index, the motion vector, the difference vector, the prediction vector index, and the rate distortion evaluation value are supplied to the prediction encoding mode determination unit 122. The detailed configuration of the difference vector calculation unit 120 will be described later.

  The combined motion information determination unit 121 generates a combined motion information candidate list from the candidate block group supplied from the terminal 12, the reference image supplied from the terminal 14, and the image signal supplied from the terminal 15. Then, the combined motion information determination unit 121 selects a combined motion information candidate from the generated combined motion information candidate list, determines a merge candidate number, calculates a rate distortion evaluation value, and combines the combined motion information candidate. Motion information, the merge candidate number, and the rate distortion evaluation value are supplied to the predictive coding mode determination unit 122. A detailed configuration of the combined motion information determination unit 121 will be described later.

  The predictive coding mode determination unit 122 compares the rate distortion evaluation value supplied from the difference vector calculation unit 120 with the rate distortion evaluation value supplied from the combined motion information determination unit 121. If the former is less than the latter, the merge flag is set to “0”. The predictive coding mode determination unit 122 supplies the merge flag, the reference image index, the difference vector, and the prediction vector index supplied from the difference vector calculation unit 120 to the terminal 16, and the motion vector supplied from the difference vector calculation unit 120. The reference image index is supplied to the terminal 50 and the terminal 51.

  If the latter is less than or equal to the former, the merge flag is set to 1. The predictive coding mode determination unit 122 supplies the merge flag and the merge candidate number supplied from the combined motion information determination unit 121 to the terminal 16, and the motion vector of the motion information supplied from the combined motion information determination unit 121 and the reference image The index is supplied to the terminal 50 and the terminal 51. Although a specific calculation method of the rate distortion evaluation value is not the main point of the present invention, the details thereof will be omitted. However, the evaluation value has a characteristic that the encoding efficiency increases as the rate distortion evaluation value decreases.

  Next, a detailed configuration of the difference vector calculation unit 120 will be described with reference to FIG. FIG. 12 shows the configuration of the difference vector calculation unit 120. The difference vector calculation unit 120 includes a prediction vector candidate list generation unit 130, a prediction vector determination unit 131, and a subtraction unit 132. The terminal 17 is connected to the predictive coding mode determination unit 122.

  The prediction vector candidate list generation unit 130 is also installed in the same manner in the video decoding device 200 (FIG. 30) that decodes the code sequence generated by the video encoding device 100 according to the first embodiment. The apparatus 100 and the moving picture decoding apparatus 200 generate a prediction vector candidate list having no contradiction.

  Hereinafter, functions of each unit will be described. The prediction vector candidate list generation unit 130 deletes candidate blocks outside the region and candidate blocks in the intra mode from the candidate block group supplied from the terminal 12. Further, when there are a plurality of candidate blocks having overlapping motion vectors, one candidate block is left and the rest is deleted. The prediction vector candidate list generation unit 130 generates a prediction vector candidate list from these candidate blocks after deletion, and supplies the prediction vector candidate list to the prediction vector determination unit 131. It is assumed that the prediction vector candidate list generated in this way includes one or more prediction vector candidates without duplication. For example, when there is no candidate block having a motion vector, the vector (0, 0) is added to the prediction vector candidate list. If the prediction direction is bidirectional, a prediction vector candidate list is generated and supplied for the L0 direction and the L1 direction.

  The prediction vector determination unit 131 selects an optimal prediction vector for the motion vector supplied from the terminal 13 from the prediction vector candidate list supplied from the prediction vector candidate list generation unit 130. The prediction vector determination unit 131 supplies the selected prediction vector to the subtraction unit 132, and supplies a reference vector index and a prediction vector index that is information indicating the selected prediction vector to the terminal 17. If the prediction direction is bidirectional, an optimal prediction vector is selected and supplied for the L0 direction and the L1 direction.

  Here, based on the motion vector of the prediction vector candidate, the prediction error amount is calculated from the reference image supplied from the terminal 14 and the image signal supplied from the terminal 15. Then, the rate distortion evaluation value is calculated from the code amount of the reference image index, the difference vector, and the prediction vector index, and the above-described prediction error amount, and the prediction vector candidate that minimizes the rate distortion evaluation value is determined as the optimal prediction vector. Selected.

  The subtraction unit 132 subtracts the prediction vector supplied from the prediction vector determination unit 131 from the motion vector supplied from the terminal 13 to calculate a difference vector, and supplies the difference vector to the terminal 17. If the prediction direction is bidirectional, a difference vector is calculated and supplied for the L0 direction and the L1 direction.

(Candidate block group supplied to prediction vector candidate list generation unit 130)
Here, the candidate block group supplied to the prediction vector candidate list production | generation part 130 is demonstrated using FIG. 13 and FIG. The candidate block group includes a spatial candidate block group and a temporal candidate block group.

  FIG. 13 shows adjacent blocks of a prediction block to be processed when the prediction block size to be processed is 16 pixels × 16 pixels. In the first embodiment, the space candidate block group is assumed to be five blocks of block A1, block C, block D, block B1, and block E shown in FIG. Here, the spatial candidate block group is five blocks of block A1, block C, block D, block B1, and block E, but the spatial candidate block group is at least one or more processed adjacent to the prediction block to be processed. Any block may be used, and the present invention is not limited to these. For example, all of block A1, block A2, block A3, block A4, block B1, block B2, block B3, block B4, block C, block D, and block E may be spatial candidate blocks.

  Next, the time candidate block group will be described with reference to FIG. FIG. 14 shows a block in a prediction block on ColPic and its peripheral blocks at the same position as the prediction block to be processed when the prediction block size to be processed is 16 pixels × 16 pixels. In the first embodiment, the time candidate block group includes two blocks, block H and block I6 shown in FIG.

  Here, the time candidate block group is two blocks of block H and block I6 on ColPic, but the time candidate block group is at least one block on a decoded image different from the prediction block to be processed. There is no limitation to these. For example, all of block I1 to block I16, block A1 to block A4, block B1 to block B4, block C, block D, block E, block F1 to block F4, block G1 to block G4 and block H on ColPic It may be a candidate block. Hereinafter, unless otherwise specified, block A4 is referred to as block A, and block B4 is referred to as block B. Hereinafter, unless otherwise specified, blocks H and I6 are referred to as time blocks.

  Next, a detailed configuration of the combined motion information determination unit 121 will be described with reference to FIG. FIG. 15 shows the configuration of the combined motion information determination unit 121. The combined motion information determination unit 121 includes a combined motion information candidate generation unit 140 and a combined motion information selection unit 141. The combined motion information candidate generation unit 140 is also installed in the moving image decoding apparatus 200 that decodes the code sequence generated by the moving image encoding apparatus 100 according to Embodiment 1, and is combined with the moving image encoding apparatus 100 and the moving image. The image decoding apparatus 200 generates a combined motion information list that is consistent.

  Hereinafter, functions of each unit will be described. The combined motion information candidate generation unit 140 generates a combined motion information candidate list from the candidate block group supplied from the terminal 12, and supplies the combined motion information candidate list to the combined motion information selection unit 141. A detailed configuration of the combined motion information candidate generation unit 140 will be described later.

  The combined motion information selection unit 141 is information that selects an optimal combined motion information candidate from the combined motion information candidate list supplied from the combined motion information candidate generation unit 140 and indicates the selected combined motion information candidate. The merge candidate number is supplied to the terminal 17.

  Here, the prediction error amount is calculated from the reference image supplied from the terminal 14 obtained based on the prediction direction, the motion vector, and the reference image index of the combined motion information candidate, and the image signal supplied from the terminal 15. A rate distortion evaluation value is calculated from the code amount of the merge candidate number and the prediction error amount, and the combined motion information candidate that minimizes the rate distortion evaluation value is selected as the optimal combined motion information candidate.

  Here, the candidate block group supplied to the combined motion information candidate generation unit 140 will be described with reference to FIGS. 13 and 14. The candidate block group includes a spatial candidate block group and a temporal candidate block group. In the first embodiment, the space candidate block group is assumed to be four blocks of block A4, block B4, block C, and block E shown in FIG. Here, the spatial candidate block group is four blocks of block A4, block B4, block C, and block E, but the spatial candidate block group may be at least one or more processed blocks adjacent to the prediction block to be processed. The number and position of blocks are not limited to these. For example, the space candidate block group may be five blocks of block A1, block B1, block C, block E, and block D.

  Next, the time candidate block group will be described with reference to FIG. In the first embodiment, the time candidate block group includes two blocks, block H and block I6 shown in FIG. Here, the time candidate block group is the same as the time candidate block group supplied to the prediction vector candidate list generation unit 130, but the time candidate block group is on a decoded image different from the prediction block to be processed. Any block may be used, and the number and position of the blocks are not limited to these.

  Next, a detailed configuration of the combined motion information candidate generation unit 140 will be described with reference to FIG. FIG. 16 shows a configuration of the combined motion information candidate generation unit 140. The terminal 18 is connected to the combined motion information selection unit 141. The combined motion information candidate generation unit 140 includes a single combined motion information candidate list generation unit 150, a scaling combined motion information candidate generation unit 151, and a combined motion information candidate list reduction unit 152.

  Hereinafter, functions of each unit will be described. The single combined motion information candidate list generation unit 150 generates a first combined motion information candidate list from the candidate block group supplied from the terminal 12, and sends the first combined motion information candidate list to the scaled combined motion information candidate generation unit 151. Supply.

  The scaling combined motion information candidate generation unit 151 generates a scaling combined motion information candidate from the first combined motion information candidate list supplied from the single combined motion information candidate list generation unit 150, and the scaling combined motion information candidate is first combined. A second combined motion information candidate list is generated by adding to the motion information candidate list, and the second combined motion information candidate list is supplied to the combined motion information candidate list reduction unit 152.

  The combined motion information candidate list reduction unit 152 is 1 when there are a plurality of combined motion information candidates having motion information overlapping from the second combined motion information candidate list supplied from the scaling combined motion information candidate generation unit 151. The combined motion information candidate list is generated by leaving two combined motion information candidates and deleting the rest, and the combined motion information candidate list is supplied to the terminal 18.

  Here, the single combined motion information candidate is a motion information candidate of a candidate block used in a so-called merge technique, and is motion information obtained from one candidate block. On the other hand, the scaled combined motion information is motion information obtained by scaling one or two pieces of motion information of one candidate block. In the present embodiment, one piece of motion information of a candidate block is scaled and used.

  Next, a detailed configuration of the scaled combined motion information candidate generation unit 151 will be described with reference to FIG. FIG. 17 shows a configuration of the scaling combined motion information candidate generation unit 151. The terminal 19 is connected to the single combined motion information candidate list generating unit 150, and the terminal 20 is connected to the combined motion information candidate list reducing unit 152. The scaling combined motion information candidate generation unit 151 includes a target candidate selection unit 160, a reference image setting unit 161, a scaling unit 162, and a scaling combined motion information candidate addition unit 163.

  Hereinafter, functions of each unit will be described. The target candidate selection unit 160 selects a combined motion information candidate that is a target of the scaling combined motion information candidate from the first combined motion information candidate list supplied from the terminal 19, and information on the combined motion information candidate that is the target and The first combined motion information candidate list is supplied to the reference image setting unit 161.

  The reference image setting unit 161 refers to the L0 direction and the L1 direction of the scaling combined motion information candidates based on the combined motion information candidate information and the first combined motion information candidate list supplied from the target candidate selecting unit 160. Set the image. The reference image setting unit 161 supplies the reference image information in the L0 direction and the L1 direction, information on the combined motion information candidates to be described above, and the first combined motion information candidate list to the scaling unit 162.

  The scaling unit 162 determines the scaled combined motion information candidate from the reference image information in the L0 direction and the L1 direction supplied from the reference image setting unit 161, the combined motion information candidate information described above and the first combined motion information candidate list. A motion vector is calculated by scaling to determine a scaled combined motion information candidate. The scaling unit 162 supplies the scaling combined motion information candidate and the first combined motion information candidate list to the scaling combined motion information candidate adding unit 163.

  The scaling combined motion information candidate addition unit 163 adds the scaling combined motion information candidate supplied from the scaling unit 162 to the first combined motion information candidate list to generate a second combined motion information candidate list, and generates the second combined motion information candidate list. The list is supplied to terminal 20.

(Candidate number management table)
Here, a candidate number management table indicating the relationship between merge candidate numbers and combined motion information candidates used in Embodiment 1 will be described with reference to FIG. Merge candidate numbers 0 to 6 are combined motion information candidates (A) of block A, combined motion information candidates (B) of block B, and combined motion information candidates (COL) of time blocks included in the combined motion information candidate list, respectively. The combined motion information candidate (C) of block C, the combined motion information candidate (E) of block E, the first scaled combined motion information candidate (BP0), and the second scaled combined motion information candidate (BP1) are shown. The maximum number of combined motion information candidates included in the combined motion information candidate list is 7 (the maximum value of the merge index is 6). Although the candidate number management table used in the first embodiment is shown in FIG. 18, the merged motion information candidate having a relatively high selection rate may be assigned with a smaller merge candidate number, and is not limited to this. In addition, although the maximum number of scaling combined motion information candidates used in Embodiment 1 is two, the number is not limited to this as long as it is one or more.

  Here, the maximum number of combined motion information candidates included in the candidate number management table and the combined motion information candidate list is assumed to be shared in the moving image encoding apparatus 100 and is not illustrated. Hereinafter, conversion from the merge candidate number to the merge index will be described with reference to FIGS.

  FIG. 19A shows a combined motion information candidate for block A, a combined motion information candidate for block B, a combined motion information candidate for time block, a combined motion information candidate for block C, a combined motion information candidate for block E, the first When the scaling combined motion information candidate and the second scaling combined motion information candidate are all valid, it indicates that the merge candidate number becomes the merge index as it is.

  FIG. 19B shows a case where merge indexes are assigned in ascending order of merge candidate numbers after filling invalid merge candidate numbers when the combined motion information candidates include invalid blocks. As shown in FIG. 19B, when the combined motion information candidate of the block B with the merge candidate number 1 and the block E with the merge candidate number 4 is invalid, the merge index 0 is set to the merge candidate number 0 and the merge index 1 is converted to merge candidate number 2, merge index 2 is converted to merge candidate number 3, merge index 3 is converted to merge candidate number 5, and merge index 4 is converted to merge candidate number 6.

  In the moving picture decoding apparatus 200 that decodes the code sequence generated by the moving picture encoding apparatus 100 according to the first embodiment, the reverse conversion from the merge index to the merge candidate number is performed, and the moving picture encoding apparatus 100 And the moving picture decoding apparatus 200 generates a candidate number management table with no contradiction.

(Operation of moving picture coding apparatus 100)
Next, an encoding operation in the moving image encoding apparatus 100 according to Embodiment 1 will be described with reference to the flowchart of FIG. The prediction block image acquisition unit 101 acquires the image signal of the prediction block to be processed from the image signal supplied from the terminal 10 based on the position information of the prediction block and the prediction block size (S100).

  The motion vector detection unit 108 detects a motion vector and a reference image index indicating a reference image from the image signal supplied from the predicted block image acquisition unit 101 and image signals corresponding to a plurality of reference images (S101).

  The motion information generation unit 109 is supplied with the motion vector and reference image index supplied from the motion vector detection unit 108, the merge candidate number or the difference vector of the candidate block group supplied from the motion information memory 111, and the prediction vector index. Motion information for generating an image prediction signal is generated from the inside (S102).

  The motion compensation unit 106 performs motion compensation on the reference image indicated by the reference image index in the frame memory 110 based on the motion vector supplied from the motion vector detection unit 108 to generate a prediction signal. If the prediction direction is bidirectional, an average of the prediction signals in the L0 direction and the L1 direction is generated as a prediction signal (S103).

  The subtraction unit 102 calculates a difference between the image signal supplied from the prediction block image acquisition unit 101 and the prediction signal supplied from the motion compensation unit 106 to calculate a prediction error signal (S104). The prediction error encoding unit 103 performs processing such as quantization and orthogonal transformation on the prediction error signal supplied from the subtraction unit 102 to generate prediction error encoded data (S105).

  The code string generation unit 104 includes the prediction error encoded data supplied from the prediction error encoding unit 103, and the merge flag, merge candidate number, reference image index, difference vector, and prediction vector index supplied from the motion information generation unit 109. Is entropy-coded according to the syntax along with the prediction direction to generate a code string (S106).

  The adding unit 107 adds the prediction error signal supplied from the prediction error decoding unit 105 and the prediction signal supplied from the motion compensation unit 106 to generate a decoded image signal (S107). The frame memory 110 stores the decoded image signal supplied from the adding unit 107 (S108). The motion information memory 111 stores one image of the motion vector supplied from the motion vector detection unit 108 in units of the minimum predicted block size (S109).

  Subsequently, the operation of the motion information generation unit 109 will be described using the flowchart of FIG. The difference vector calculation unit 120 includes a candidate block group supplied from the terminal 12, a motion vector and reference image index supplied from the terminal 13, a reference image supplied from the terminal 14, and an image signal supplied from the terminal 15. A prediction vector candidate list is created, a prediction vector index is determined, and a difference vector and a rate distortion evaluation value are calculated (S110).

  The combined motion information determination unit 121 creates a combined motion information candidate list from the candidate block group supplied from the terminal 12, the reference image supplied from the terminal 14, and the image signal supplied from the terminal 15, and merge candidate numbers And the rate distortion evaluation value is calculated (S111).

  The predictive coding mode determination unit 122 compares the rate distortion evaluation value supplied from the difference vector calculation unit 120 with the rate distortion evaluation value supplied from the combined motion information determination unit 121, and merges if the former is smaller than the latter The flag is set to “0”, otherwise, the merge flag is set to “1” (S112).

  Subsequently, the operation of the difference vector calculation unit 120 will be described using the flowchart of FIG. The prediction vector candidate list generation unit 130 excludes candidate blocks that are out of the region, candidate blocks that are in the intra mode, and candidate blocks that have overlapping motion vectors from the candidate block group supplied from the terminal 12. A prediction vector candidate list is generated. If the prediction direction is bidirectional, a prediction vector candidate list is generated for the L0 direction and the L1 direction (S120).

  The prediction vector determination unit 131 selects an optimal prediction vector for the motion vector supplied from the terminal 13 from the prediction vector candidate list supplied from the prediction vector candidate list generation unit 130. If the prediction direction is bidirectional, an optimal prediction vector is selected for the L0 direction and the L1 direction (S121). The subtraction unit 132 subtracts the prediction vector supplied from the prediction vector determination unit 131 from the motion vector supplied from the terminal 13 to calculate a difference vector. If the prediction direction is bidirectional, a difference vector is calculated for the L0 direction and the L1 direction (S122).

  Subsequently, the operation of the combined motion information determination unit 121 will be described in detail using the flowchart of FIG. The combined motion information candidate generation unit 140 generates a combined motion information candidate list from the candidate block group supplied from the terminal 12 (S130). The combined motion information selection unit 141 selects from the combined motion information candidate list supplied from the combined motion information candidate generation unit 140, the motion vector and reference image index supplied from the terminal 13, and the combined motion information optimal for the prediction direction. Is determined (S131).

  Subsequently, the operation of the combined motion information candidate generation unit 140 will be described in detail using the flowchart of FIG. The single combined motion information candidate list generation unit 150 generates a spatial combined motion information candidate list from candidate blocks obtained by excluding candidate blocks outside the region and candidate blocks in the intra mode from the spatial candidate block group supplied from the terminal 12. (S140). Detailed operations for generating the spatially coupled motion information candidate list will be described later.

  The single combined motion information candidate list generation unit 150 generates a time combined motion information candidate list from candidate blocks obtained by excluding candidate blocks that are out of the region and candidate blocks that are in the intra mode from the time candidate block group supplied from the terminal 12. (S141). The detailed operation of generating the time combination motion information candidate list will be described later.

  The single combined motion information candidate list generation unit 150 generates the first combined motion information candidate list by combining the spatially combined motion information candidate list and the temporally combined motion information candidate list in the order of merge candidate numbers (S142).

  The scaling combined motion information candidate generation unit 151 selects a combined motion information candidate to be a scaling combined motion information candidate from the first combined motion information candidate list supplied from the single combined motion information candidate list generation unit 150, and performs scaling combining. Motion information candidates are generated (S143). The detailed operation of generating the scaling combined motion information candidate will be described later.

  The scaling combined motion information candidate generation unit 151 adds the scaling combined motion information candidates to the first combined motion information candidate list in the order of merge candidate numbers, and generates the second combined motion information candidate list (S144).

  When there are a plurality of combined motion information candidates having motion information overlapping from the second combined motion information candidate list supplied from the scaling combined motion information candidate generation unit 151, the combined motion information candidate list reduction unit 152 One combined motion information candidate is left, and the remaining is deleted to generate a combined motion information candidate list (S145).

  Subsequently, the detailed operation of generating the spatially coupled motion information candidate list will be described using the flowchart of FIG. FIG. 25 is a flowchart for explaining the detailed operation of generating the spatially coupled motion information candidate list. In Embodiment 1, it is assumed that the spatial combination motion information candidate list includes motion information of four or less candidate blocks.

  The following processing is repeated for block A, block B, block C, and block E, which are the four candidate blocks included in the space candidate block group (S150 to S153). The validity of the candidate block is checked (S151). A candidate block is valid when the candidate block is not out of the region and is not in intra mode. If the candidate block is valid (Y in S151), the motion information of the candidate block is added to the spatially combined motion information candidate list (S152). If the candidate block is not valid (N in S151), step S152 is skipped.

  In the first embodiment, it is assumed that the spatial combination motion information candidate list includes motion information of four or less candidate blocks. However, the number of spatial combination motion information candidates may be changed depending on the validity of the candidate block, and the present invention is not limited thereto. Not.

  Subsequently, a detailed operation of generating a time combination motion information candidate list will be described using the flowchart of FIG. FIG. 26 is a flowchart for explaining the detailed operation of generating the time combination motion information candidate list. In Embodiment 1, it is assumed that the temporal combination motion information candidate list includes motion information of one or less candidate blocks.

  The following processing is repeated for time blocks that are two candidate blocks included in the time candidate block group (S160 to S166). The validity of the candidate block is checked (S161). A candidate block is valid when the candidate block is not out of the region and is not in intra mode. If the candidate block is valid (Y in S161), a temporally combined motion information candidate is generated, the temporally combined motion information candidate is added to the temporally combined motion information candidate list (from step S162 to step S165), and the process ends. If the candidate block is not valid (N in S161), the next candidate block is inspected (S166).

  If the candidate block is valid, the prediction direction of the temporally combined motion information candidate is determined (S162). In the first embodiment, the prediction direction of the combined motion information candidate is bidirectional. Next, the reference images in the L0 direction and the L1 direction of the time combination motion information candidate are determined (S163). In the first embodiment, the reference image in the L0 direction is the reference image that is the closest to the processing target image among the reference images in the L0 direction, and the reference image in the L1 direction is the processing target image among the reference images in the L1 direction. The reference image is the closest distance. Here, the reference image in the L0 direction is the reference image that is the closest to the processing target image among the reference images in the L0 direction, and the reference image in the L1 direction is the closest to the processing target image among the reference images in the L1 direction. Although the reference image is at a distance, it is only necessary to determine the reference image in the L0 direction and the reference image in the L1 direction, and the present invention is not limited to this. For example, the reference images in the L0 direction and the L1 direction may be encoded in the encoded stream, the reference image indexes in the L0 direction and the L1 direction may be set to 0, and the L0 used by the adjacent block of the processing target block. Of the reference image in the direction and the reference image in the L1 direction, the most frequently used reference image may be used as a reference image for referring to the L0 direction and the L1 direction.

  Next, the motion vector of the time combination motion information candidate is calculated (S164). In this embodiment, the temporally combined motion information candidate calculates bidirectional motion information based on the reference image ColRefPic and the motion vector mvCol, which are effective prediction directions in the motion information of the candidate block.

  When the prediction direction of the candidate block is the unidirectional direction of the L0 direction or the L1 direction, it is selected based on the reference image and the motion vector in the prediction direction.

  When the prediction direction of the candidate block is bidirectional, the selection is made based on either the reference image in the L0 direction or the L1 direction and the motion vector. For example, a reference image that is present in the same time direction as ColPic and a motion vector is selected as a reference, and a candidate image that is closer to the ColPic is selected based on the reference image in the L0 direction or L1 direction of the candidate block. For example, the candidate block may be selected on the basis of which one of the motion vectors in the L0 direction or the L1 direction intersects the processing target image.

  When a reference image and a motion vector as a reference for generating temporally coupled motion information are selected, a motion vector of temporally coupled motion information candidates is calculated.

  Here, a method of calculating motion vectors mvL0t and mvL1t of temporally coupled motion information candidates from the motion vector ColMv as a reference for generating temporally coupled motion information and the reference image ColRefPic will be described with reference to FIG. FIG. 27 is a diagram for explaining a method of calculating motion vectors mvL0t and mvL1t of time-coupled motion information candidates.

The distance between the images of ColPic and ColRefPic is ColDist, the distance between the reference images ColL0Pic in the L0 direction of the temporally coupled motion information candidate and the image to be processed CurPic is CurL0Dist, the reference image ColL1Pic of the temporally coupled motion information candidate in the L1 direction and the target of processing Assuming that the distance between images of the image CurPic is CurL1Dist, a motion vector of the following equation 1 obtained by scaling ColMv with a distance ratio of ColDist, CurL0Dist, and CurL1Dist is set as a motion vector of a temporally combined motion information candidate. Here, the calculation of the inter-image distance is performed using the POC, and has a positive and negative sign. Division is rounded off.
mvL0t = mvCol × CurrL0Dist / ColDist
mvL1t = mvCol × CurrL1Dist / ColDist (Formula 1)
Note that ColPic, ColRefPic, ColL0Pic, and ColL1Pic in FIG. 27 are examples, and other relationships may be used.

  Here, the time combination motion information candidates are generated as described above. However, bidirectional motion information may be determined using motion information of another encoded image, and the present invention is not limited to this. Returning to FIG. 26, the temporally combined motion information candidates calculated in step S164 are added to the temporally combined motion information candidate list (S165). If the candidate block is invalid (N in S161), the next candidate block is inspected (S166).

  Here, it is assumed that the time combination motion information candidate list includes motion information of one or less candidate blocks, but the number of time combination motion information candidates may be changed depending on the validity of the candidate blocks, and is not limited thereto. Similarly, the prediction direction, reference image, and motion vector determination method are not limited to these.

(Generation of scaling combined motion information candidates)
Subsequently, the detailed operation of generating the scaling combined motion information candidate will be described using the flowchart of FIG. FIG. 28 is a flowchart for explaining the detailed operation of generating the scaling combined motion information candidate. The scaling combined motion information candidate generation unit 151 repeats the following processing for the number of times corresponding to the number of combined motion information candidates (NCands) included in the first combined motion information candidate list (S170 to S176). The target candidate selection unit 160 checks whether a scaling combined motion information candidate can be added (S171). Specifically, it is checked that the number of already added scaling combined motion information candidates is less than two. If scaling combined motion information candidates can be added (Y in S171), the target candidate selection unit 160 checks whether the candidate block is valid (S172). A candidate block is valid when the candidate block is not out of the region and is not in intra mode. If the candidate block is valid (Y in S172), the target candidate selection unit 160 checks whether the prediction direction of the candidate block is unidirectional (S173).

  If the prediction direction of the candidate block is unidirectional (Y in S173), the effective prediction direction is set as the LX direction, and the reference image setting unit 161 refers to the reference image of the candidate block in the LX direction of the scaled combined motion information candidate. A reference image in the LY direction of the scaling combined motion information candidate is set as an image (S174). Here, LX is either L0 or L1. LY represents a reference list opposite to LX. That is, if LX is L0, LY is L1, and if LX is L1, LY is L0. In the first embodiment, the reference image indicated by 0 in the reference image index in the LY direction is used as the reference image in the LY direction of the scaling combined motion information candidate. In the first embodiment, the reference image indicated by 0 in the reference image index in the LY direction is used as the reference image in the LY direction. However, the present invention is not limited to this as long as the reference image in the LY direction can be set. For example, it is possible to set the reference image in the LY direction determined in step S163 in generating the temporally combined motion information candidate.

  The scaling unit 162 calculates the motion vector in the LY direction of the scaling combined motion information candidate by scaling the motion vector in the LX direction of the scaling combined motion information candidate (S175). If the scaling combined motion information candidate cannot be added (N of S171), the process ends. If the candidate block is not valid (N in S172) or the prediction direction of the candidate block is not unidirectional (N in S173), the next candidate is examined (S176).

  Here, a method of calculating the LY direction motion vector mvLY of the scaled combined motion information candidate from the LX direction reference image RefLXPic, the LX direction motion vector mvLX, and the LY direction reference image RefLYPic will be described with reference to FIG. FIG. 29 is a diagram for explaining a method of calculating a motion vector in the LY direction of a scaling combined motion information candidate.

An inter-image distance CurLXDist between the processing target image CurPic and the reference image RefLXPic in the LX direction is calculated by the following equation 2.
CurLXDist = CurPOC-RefLXPOC (Formula 2)
The distance between images of the processing target image CurPic and the reference image RefLYPic in the LY direction is calculated by the following equation (3).
CurLYDist = CurPOC-RefLYPOC (Formula 3)
Here, CurPOC is the POC of the processing target image CurPic, RefLXPOC is the POC of the reference image RefLXPic in the LX direction, and RefLYPOC is the POC of the reference image RefLYPic in the LY direction. Note that CurLXDist and CurLYDist have a negative value when indicating a distance from a past reference image based on the processing target image CurPic, and have a positive value when indicating a distance from a future reference image.

Furthermore, a motion vector of the following expression 4 obtained by scaling mvLX with a distance ratio of CurLXDist and CurLYDist is defined as mvLY. Division is rounded off.
mvLY = mvLX × CurrLYDist / CurrLXDist (Expression 4)
Note that CurPic, RefLXPic, and RefLYPic in FIG. 29 are examples, and other relationships may be used.

  Here, the number of times corresponding to the number of combined motion information candidates (NCands) included in the second combined motion information candidate list is inspected, but it is only necessary to be able to determine scaling combined motion information candidates, and the present invention is not limited to this. For example, when generating a combined combined motion information candidate from only combined motion information candidates having a relatively high selection rate, the number of inspections is fixed to a predetermined number such as 2 or 3, and the processing amount is reduced and redundant scaling is performed. It is also possible to reduce the amount of merge index codes by reducing the possibility of generating combined motion information candidates. In addition, a time block that is bidirectional from the beginning can be removed from the inspection target.

(Configuration of moving picture decoding apparatus 200)
Next, the moving picture decoding apparatus according to the first embodiment will be described. FIG. 30 shows the moving picture decoding apparatus 200 according to the first embodiment. The video decoding device 200 is a device that generates a playback image by decoding the code string encoded by the video encoding device 100.

  The moving picture decoding apparatus 200 is realized by hardware such as an information processing apparatus including a CPU (Central Processing Unit), a frame memory, and a hard disk. The moving picture decoding apparatus 200 realizes functional components described below by operating the above components. Note that the position information and the prediction block size of the prediction block to be decoded are shared in the video decoding device 200 and are not shown. Further, the maximum number of combined motion information candidates included in the candidate number management table and the combined motion information candidate list is assumed to be shared in the moving image decoding apparatus 200 and is not illustrated.

  The moving picture decoding apparatus 200 according to Embodiment 1 includes a code string analysis unit 201, a prediction error decoding unit 202, an addition unit 203, a motion information reproduction unit 204, a motion compensation unit 205, a frame memory 206, and a motion information memory 207.

(Function of moving picture decoding apparatus 200)
Hereinafter, functions of each unit will be described. The code string analysis unit 201 decodes the code string supplied from the terminal 30 to predict prediction error encoded data, merge flag, merge candidate number, prediction direction of motion compensation prediction, reference image index, difference vector, and prediction vector index. Is decoded according to the syntax. Then, the prediction error coding data is transferred to the prediction error decoding unit 202, the merge flag, the merge candidate number, the prediction direction of the motion compensation prediction, the reference image index, the difference vector, and the prediction vector index as motion information. This is supplied to the playback unit 204. The merge candidate number is obtained by conversion from the merge index.

  The prediction error decoding unit 202 performs a process such as inverse quantization or inverse orthogonal transform on the prediction error encoded data supplied from the code string analysis unit 201 to generate a prediction error signal, and the prediction error signal is It supplies to the addition part 203.

  The adding unit 203 adds the prediction error signal supplied from the prediction error decoding unit 202 and the prediction signal supplied from the motion compensation unit 205 to generate a decoded image signal, and the decoded image signal is stored in the frame memory 206 and Supply to terminal 31.

  The motion information reproduction unit 204 is supplied from the motion information memory 207 and the merge flag, merge candidate number, motion compensation prediction direction, reference image index, difference vector, and prediction vector index supplied from the code string analysis unit 201. Motion information is reproduced from the candidate block group, and the motion information is supplied to the motion compensation unit 205. A detailed configuration of the motion information reproducing unit 204 will be described later.

  Based on the motion information supplied from the motion information reproducing unit 204, the motion compensation unit 205 performs motion compensation on the reference image indicated by the reference image index in the frame memory 206 based on the motion vector to generate a prediction signal. If the prediction direction is bidirectional, an average of the prediction signals in the L0 direction and the L1 direction is generated as a prediction signal, and the prediction signal is supplied to the adding unit 203.

  The frame memory 206 and the motion information memory 207 have the same functions as the frame memory 110 and the motion information memory 111 of the video encoding device 100.

(Detailed configuration of the motion information playback unit 204)
Next, a detailed configuration of the motion information reproducing unit 204 will be described with reference to FIG. FIG. 31 shows the configuration of the motion information playback unit 204. The motion information playback unit 204 includes an encoding mode determination unit 210, a motion vector playback unit 211, and a combined motion information playback unit 212. The terminal 32 is connected to the code string analysis unit 201, the terminal 33 is connected to the motion information memory 207, and the terminal 34 is connected to the motion compensation unit 205.

  Hereinafter, functions of each unit will be described. If the merge flag supplied from the code stream analysis unit 201 is “0”, the coding mode determination unit 210 determines the prediction direction, reference image index, difference vector, motion compensation prediction supplied from the code stream analysis unit 201, The prediction vector index is supplied to the motion vector reproduction unit 211. If the merge flag is “1”, the merge candidate number supplied from the code string analysis unit 201 is supplied to the combined motion information reproduction unit 212.

  The motion vector reproduction unit 211 receives motion information from the prediction direction of motion compensation prediction supplied from the encoding mode determination unit 210, the reference image index, the difference vector, and the prediction vector index, and the candidate block group supplied from the terminal 33. Is supplied to the terminal 34. A detailed configuration of the motion vector reproducing unit 211 will be described later.

  The combined motion information reproduction unit 212 reproduces motion information from the merge candidate number supplied from the encoding mode determination unit 210 and the candidate block group supplied from the terminal 33 and supplies the motion information to the terminal 34. A detailed configuration of the combined motion information reproducing unit 212 will be described later.

  Next, a detailed configuration of the motion vector reproduction unit 211 will be described with reference to FIG. FIG. 32 shows a configuration of the motion vector reproducing unit 211. The motion vector reproduction unit 211 includes a prediction vector candidate list generation unit 220, a prediction vector determination unit 221, and an addition unit 222. The terminal 35 is connected to the encoding mode determination unit 210.

  Hereinafter, functions of each unit will be described. The prediction vector candidate list generation unit 220 has the same function as the prediction vector candidate list generation unit 130 of the video encoding device 100. The prediction vector determination unit 221 determines a prediction vector from the prediction vector candidate list supplied from the prediction vector candidate list generation unit 220 and the prediction vector index supplied from the terminal 35, and supplies the prediction vector to the addition unit 222.

  The adder 222 adds the difference vector supplied from the terminal 35 and the prediction vector supplied from the prediction vector determination unit 221 to calculate a motion vector, and supplies the motion vector to the terminal 34.

  Next, a detailed configuration of the combined motion information reproduction unit 212 will be described with reference to FIG. FIG. 33 shows the configuration of the combined motion information playback unit 212. The combined motion information reproduction unit 212 includes a combined motion information candidate generation unit 230 and a combined motion information selection unit 231.

  Hereinafter, functions of each unit will be described. The combined motion information candidate generation unit 230 has the same function as the combined motion information candidate generation unit 140 shown in FIG. Based on the combined motion information candidate list supplied from the combined motion information candidate generation unit 230 and the merge candidate number supplied from the terminal 35, the combined motion information selection unit 231 selects motion information from the combined motion information candidate list. The motion information is selected and supplied to the terminal 34.

(Operation of the video decoding device 200)
Subsequently, the decoding operation in the video decoding device 200 according to Embodiment 1 will be described with reference to the flowchart of FIG. FIG. 34 is a flowchart for explaining the decoding operation of moving picture decoding apparatus 200 according to Embodiment 1 of the present invention. The code string analysis unit 201 decodes the code string supplied from the terminal 30 to predict prediction error encoded data, merge flag, merge candidate number, prediction direction of motion compensation prediction, reference image index, difference vector, and prediction vector index. Is decoded according to the syntax (S210).

  The motion information reproduction unit 204 is supplied from the motion information memory 207 and the merge flag, merge candidate number, motion compensation prediction direction, reference image index, difference vector, and prediction vector index supplied from the code string analysis unit 201. Motion information is reproduced from the candidate block group (S211).

  Based on the motion information supplied from the motion information reproducing unit 204, the motion compensation unit 205 performs motion compensation on the reference image indicated by the reference image index in the frame memory 206 based on the motion vector to generate a prediction signal. If the prediction direction is bidirectional, an average of the prediction signals in the L0 direction and the L1 direction is generated as a prediction signal (S212).

  The prediction error decoding unit 202 performs a process such as inverse quantization and inverse orthogonal transform on the prediction error encoded data supplied from the code string analysis unit 201 to generate a prediction error signal (S213). The adding unit 203 adds the prediction error signal supplied from the prediction error decoding unit 202 and the prediction signal supplied from the motion compensation unit 205 to generate a decoded image signal (S214).

  The frame memory 206 stores the decoded image signal supplied from the adding unit 203 (S215). The motion information memory 207 stores the motion vector supplied from the motion information reproducing unit 204 for one image in the minimum predicted block size unit (S216).

  Subsequently, the operation of the motion information reproducing unit 204 will be described with reference to the flowchart of FIG. FIG. 35 is a flowchart for explaining the operation of the motion information reproducing unit 204. The encoding mode determination unit 210 determines whether the merge flag supplied from the code string analysis unit 201 is “0” or “1” (S220). If the merge flag is “1”, the combined motion information reproduction unit 212 reproduces motion information from the merge candidate number supplied from the encoding mode determination unit 210 and the candidate block group supplied from the terminal 33 ( S221).

  If the merge flag is “0”, the motion vector reproduction unit 211 receives the prediction direction of motion compensation prediction, the reference image index, the difference vector, and the prediction vector index supplied from the encoding mode determination unit 210 and the terminal 33. Motion information is reproduced from the supplied candidate block group (S222).

  Subsequently, the operation of the motion vector reproducing unit 211 will be described with reference to the flowchart of FIG. FIG. 36 is a flowchart for explaining the operation of the motion vector reproducing unit 211. The prediction vector candidate list generation unit 220 generates a prediction vector candidate list by the same operation as the prediction vector candidate list generation unit 130 of the video encoding device 100 (S300).

  The prediction vector determination unit 221 selects a prediction vector candidate indicated by the prediction vector index supplied from the terminal 35 from the prediction vector candidate list supplied from the prediction vector candidate list generation unit 220, and determines a prediction vector. (S301). The adder 222 adds the difference vector supplied from the terminal 35 and the prediction vector supplied from the prediction vector determination unit 221 to calculate a motion vector (S302).

  Subsequently, the operation of the combined motion information reproducing unit 212 will be described with reference to the flowchart of FIG. FIG. 37 is a flowchart for explaining the operation of the combined motion information reproducing unit 212. The combined motion information candidate generation unit 230 generates a combined motion information candidate list by the same operation as the combined motion information candidate generation unit 140 of the video encoding device 100 (S310). The combined motion information selection unit 231 selects a combined motion information candidate indicated by the merge candidate number supplied from the terminal 35 from the combined motion information candidate list supplied from the combined motion information candidate generation unit 230, and combines them. The motion information is determined (S311).

(Modification of Embodiment 1)
The first embodiment can be modified as follows.

(Modification 1: Merge candidate number order)
In Embodiment 1 described above, FIG. 18 is given as an example of the candidate number management table. However, the maximum number of combined motion information candidates may be 1 or more, and the combined motion information candidates having a relatively high selection rate are smaller. The merge candidate number only needs to be assigned, and is not limited to FIG. The maximum number of combined motion information candidates included in the combined motion information candidate list is 7 (the maximum value of the merge index is 6), but it may be 2 or more. For example, when the selection rate of the combined motion information candidate for scaling is higher than the selection rate of the combined motion information candidate for block C and block E, it may be as shown in FIG. 38A and 38B are diagrams for explaining a candidate number management table according to the first modification. Also, as shown in FIG. 39, the scaling combined motion information candidates can be increased. FIG. 39 is a diagram for explaining another candidate number management table according to the first modification of the first embodiment.

  As described above, by changing the definition of the candidate number management table according to the selection rate of the scaling combined motion information candidate, the selection rate of the combined motion information candidate can be increased and the encoding efficiency of the motion information can be improved. .

(Variation 2: Exclusion of time candidate blocks)
In Embodiment 1 described above, the operation of the combined motion information candidate generation unit 140 is shown in FIG. 24, but it is only necessary to generate a scaled combined motion information candidate, which is not limited to this. For example, step S141 for generating a temporally combined motion information candidate that is a combined motion information candidate having a vector whose prediction direction is always bi-directionally scaled using a temporal candidate block may be deleted. This makes it possible to generate combined motion information candidates with scaled vectors without depending on temporal candidate blocks, and generate combined motion information candidates with scaled vectors even when temporal candidate blocks cannot be decoded correctly it can.

(Modification 3: Stop generation of scaling combined motion information candidates)
In Embodiment 1 described above, the operation of generating the scaled combined motion information candidate is shown in FIG. 28, but the scaled combined motion information candidate may be generated and is not limited to this. For example, the following steps may be added as shown in FIG. 40 for the purpose of reducing filtering processing and averaging processing related to bidirectional motion compensation prediction. FIG. 40 is a flowchart for explaining the operation of generating scaling combined motion information candidates according to the third modification of the first embodiment.

  It is determined whether the reference images in the LX direction and the LY direction are temporally the same direction (S177). If the reference images in the LX direction and the LY direction are not temporally the same direction (N in S177), step S175 is performed. To generate scaling combined motion information candidates. If the reference images in the LX direction and the LY direction are temporally in the same direction (Y in S177), the step S175 is skipped and no scaling combined motion information candidate is generated. Note that the same time direction means that CurL0Dist and CurL1Dist have the same sign (plus or minus). Further, step S177 can be further modified to determine whether the reference images in the LX direction and the LY direction are the same.

(Modification 4: Expansion of reference image setting)
In the first embodiment described above, the setting of the reference image in the LY direction (S174) in the operation of generating the scaled combined motion information candidate (FIG. 28) is performed using the reference image indicated by 0 in the reference image index in the LY direction. However, the present invention is not limited to this as long as a reference image in the LY direction can be set. For example, when the reference image in the LX direction and the reference image indicated by 0 in the reference image index in the LY direction are the same for the purpose of improving prediction efficiency such as a filtering effect of motion compensation prediction, the reference image in the LY direction Among them, a reference image closest to the processing target image, which is different from the reference image in the LX direction, can be used as the reference image in the LY direction. More specifically, among the reference images in the LY direction, the reference image indicated by 1 in the LY direction reference image index having a relatively high probability of being next closest to the reference image indicated by the reference image index 0 can do. Also, the reference image used most frequently in the reference image in the LY direction used by the adjacent block of the processing target block may be used as the reference image in the LY direction.

(Modification 5: Scaling of candidate block whose prediction direction is bidirectional)
In Embodiment 1 described above, the operation of generating the scaling combined motion information candidate is generated when the combined prediction motion information candidate is generated when the prediction direction of the candidate block is unidirectional as shown in FIG. The information candidate only needs to be generated, and the present invention is not limited to this. For example, for a candidate block whose prediction direction is bidirectional as shown in FIG. 41, a prediction direction to be used without scaling the motion vector and the reference image index is selected as a base direction (also referred to as a reference direction), and a prediction direction that is not the base direction By calculating the motion vector and the reference image index by scaling, it is possible to generate a scaled combined motion information candidate.

  FIG. 41 is a flowchart for explaining the operation of generating scaling combined motion information candidates according to the fifth modification of the first embodiment. In the flowchart of FIG. 41, the following steps are added to the flowchart of FIG. If the prediction direction of the candidate block is not unidirectional (N in S173), the base LX direction and the LY direction calculated by scaling are set (S178). In the present modification, the LX direction is a prediction direction that is close to the processing target image that is considered to have high correlation among the reference image in the L0 direction and the reference image in the L1 direction. By performing scaling on the basis of a motion vector that is considered highly correlated, a highly correlated prediction vector can be calculated.

  In the present modification, the LX direction is the prediction direction that is closer to the processing target image among the reference images in the L0 direction and the L1 direction. However, the LX direction may be determined and is not limited thereto. For example, a prediction direction having a small reference index value that is considered to have high correlation among the reference image in the L0 direction and the reference image in the L1 direction may be set as the LX direction.

  Further, of the reference image in the L0 direction and the reference image in the L1 direction, the prediction direction that is far or long from the processing target image is set as the LX direction, and the prediction direction that is close or short in distance is set as the LY direction. The calculation accuracy of the prediction vector can be improved. This is because the calculation accuracy is higher when the scaling factor is smaller in the calculation by scaling. The same effect can be obtained even if the prediction direction with a large reference index value is the LX direction and the prediction direction with a small reference index value is the LY direction. Whether to perform scaling based on highly correlated motion vectors or scaling with high calculation accuracy may be fixed, but it is set on the encoding side in sequence, picture or slice units, and those settings are set A flag to be identified may be encoded and switched on the decoding side according to the value of the flag.

  Subsequently, a reference image (reference index) in the LY direction is set (S183). At that time, the reference image (reference index) in the LY direction may be set as it is, or the reference image (reference index) in the LY direction may be set by the same processing as in step S174. When the reference image (reference index) in the LY direction is set as it is, a motion vector in the LY direction different from the existing motion vector of the highly correlated reference image can be calculated, and LY is obtained by the same processing as in step S174. When a direction reference image (reference index) is set, it is highly possible to calculate a scaling combined motion information candidate having a value different from that of a single combined motion information candidate, and the selection range can be widened.

  Further, as shown in FIG. 42, the generation operation of the scaling combined motion information candidate is selected as the base direction in both the L0 direction and the L1 direction for the bidirectional candidate block, and the motion vector in the prediction direction other than the base direction is referred to. A scaling combined motion information candidate can also be generated by calculating an image index by scaling.

  FIG. 42 is a flowchart for explaining the operation of generating another scaling combined motion information candidate according to the fifth modification of the first embodiment. In the flowchart of FIG. 42, the following steps are added to the flowchart of FIG. If the prediction direction of the candidate block is not unidirectional (N in S173), when the processing from step S180 to step S182 sets the LX direction to the L0 direction (i = 0) and the LX direction to the L1 direction (i = 1) is repeated twice (S180 to S182). The base LX direction is set as the Li direction, and the LY direction calculated by scaling is set as a predicted direction other than the LX direction (S180).

  In this modified example, the candidate combined block is generated as a combined motion information candidate that is a target of the combined motion information candidate without considering whether the prediction direction is unidirectional or bidirectional. A combined motion information candidate may be generated, and the present invention is not limited to this. For example, when there is a limit to the number of scaling combined motion information candidates that can be added to the combined motion information candidate list, a combination that is a target of the scaling combined motion information candidate is given priority to a candidate block whose prediction direction is unidirectional. It can also be a motion information candidate. For example, only candidate blocks having a unidirectional prediction direction may be combined motion information candidates that are candidates for scaling combined motion information candidates, or the degree of adjacency with a processing target block that is a candidate block having a unidirectional prediction direction. A candidate block having a relatively high value may be used as a combined motion information candidate that is a target of the scaled combined motion information candidate.

  In addition, a candidate block whose prediction direction is bidirectional can be preferentially used as a combined motion information candidate to be a target of the scaling combined motion information candidate. For example, only candidate blocks with bidirectional prediction directions may be combined motion information candidates that are candidates for scaling combined motion information candidates, or the degree of adjacency with processing target blocks that are candidate blocks with bidirectional prediction directions. A candidate block having a relatively high value may be used as a combined motion information candidate that is a target of the scaled combined motion information candidate.

  As described above, by generating a scaling combined motion information candidate for a candidate block whose prediction direction is bidirectional, the number of combined motion information candidates is increased and the selection rate of combined motion information candidates is increased, thereby increasing the motion information Encoding efficiency can be improved.

(Modification 6: Calculation of scaling)
In the first embodiment described above, the equation for calculating the motion vector calculated by scaling is (Equation 4), but it is only necessary to be able to calculate the scaled motion vector, and is not limited to this. For example, the division of (Equation 4) may be rounded down or rounded up so that the accuracy of division is different from that of (Equation 1), which is a motion vector calculation formula for temporally coupled motion information candidates. As a result, even if the motion information in the LX direction that is the base direction is the same as the motion information in the LX direction of the temporally combined motion information candidate, a scaling combined motion information candidate having motion information different from the temporally combined motion information candidate is generated and combined The efficiency of motion information encoding can be improved by increasing the selection rate of motion information candidates.

(Modification 7: Replacement of combined motion information candidates)
In Embodiment 1 described above, the operation of the combined motion information candidate generation unit 140 is shown in FIG. 24, but it is only necessary to generate a scaled combined motion information candidate, which is not limited to this. For example, step S144 can be performed as follows. The scaling combined motion information candidate generation unit 151 replaces the combined motion information candidate that is the target of the scaling combined motion information candidate in the first combined motion information candidate list with the scaling combined motion information candidate, and the second combined motion information candidate list. Is generated (S144). In general, when the accuracy of the motion vector is relatively high, the prediction efficiency is higher when the prediction direction is bidirectional than when the prediction direction is unidirectional. Therefore, by replacing single combined motion information whose prediction direction is unidirectional with scaling combined motion information whose prediction direction is bidirectional, it is possible to increase the selection rate of combined motion information candidates with high prediction efficiency and code the motion information. Efficiency can be improved.

(Example of effect of Embodiment 1)
An example of the effect of the first embodiment will be described with reference to FIGS. 43 and 44. FIG. FIG. 43 is a diagram for explaining the relationship between the processing target image CurPic and the reference image. FIG. 43 will be described. Reference image RefL1Pic [1] indicated by reference image index 1 in the L1 direction, reference image RefL1Pic [0] indicated by reference image index 0 in the L1 direction, reference image RefL1Pic [0] indicated by reference image index 0 in the L1 direction, reference indicated by reference image index 0 in the L0 direction Assume that the image RefL0Pic [0] and the reference image RefL0Pic [1] indicated by the reference image index 1 in the L0 direction are arranged at equal intervals. In addition, the motion information with the smallest prediction error for the processing target block Z is that the prediction direction is bidirectional (BI), mvL0Z = (4,4), mvL1Z = (− 2, −2), refIdxL0Z = 1, refIdxL1Z. = 0.

  At this time, it is assumed that the single combined motion information candidates are A, B, COL, C, and E in FIG. FIG. 44 is a diagram for explaining the effect of the first embodiment. Among these single coupled motion information candidates, there is no motion information identical to the motion information that minimizes the prediction error for the processing target block Z. Therefore, a single combined motion information candidate having a minimum rate distortion evaluation value is selected from these single combined motion information candidates. Then, the candidate rate distortion evaluation value and the rate distortion evaluation value calculated by the difference vector calculation unit 120 are compared, and the merge mode is used as the encoding mode only when the former is smaller than the latter. Become.

  When the merge mode is selected as the encoding mode, it is because the balance between the encoding efficiency of motion information and the prediction error is optimal, and the prediction error is not optimal. On the other hand, when the non-merge mode is selected as the encoding mode, it is necessary to encode the differential motion vector and the reference image index, so that the encoding efficiency of the motion information is not optimal.

Here, the block B and the block C are unidirectional prediction, and are candidates for scaling combined motion information candidates. The first scaling combined motion information candidate (BP0) is targeted for the block B, and the reference image index and motion vector of the block B in the L1 direction are represented by the reference image index of the BP0 in the L1 direction as shown in Equation 5 below. Let it be a motion vector. The reference image index and the motion vector of the block B in the L0 direction are obtained by the reference image setting unit 161 and the scaling unit 162 as shown in Equation 6 below.
mvL1BP0 = (4, 4), refIdxL1BP0 = 1 (Expression 5)
mvL0BP0 = (4,4) × −1 / 2 = (− 2, −2), refIdxL1BP0 = 0 (Expression 6)

The second scaling combined motion information candidate (BP1) is targeted for the block C, and the reference image index and motion vector of the block C in the L1 direction are represented by the reference image index of the BP1 in the L1 direction as shown in Equation 7 below. Let it be a motion vector. The reference image index and the motion vector of the block B in the L0 direction are obtained by the reference image setting unit 161 and the scaling unit 162 as shown in the following formula 8.
mvL1BP1 = (− 2, 8), refIdxL1BP1 = 0 (Expression 7)
mvL0BP1 = (− 2,8) × −1 / 1 = (2, −8), refIdxL1BP1 = 0 (Equation 8)

  At this time, it can be seen that BP1 has the same motion information as the motion information that minimizes the prediction error for the processing target block Z. That is, by selecting BP1, the prediction error can be minimized and the coding efficiency of motion information can be optimized.

(Effect of Embodiment 1)
As described above, in the case where the motion of the block to be processed moves at a constant speed with any reference image in the LX direction and any reference image in the LY direction, the same position of another encoded image There may be no motion information indicating the same speed motion as the processing target block in the motion information of the block or the block adjacent to the processing target block. In that case, only the single direction in the L0 direction or the L1 direction in the motion information of the same position block of another encoded image or the adjacent block of the processing target block is the same constant motion as the processing target block. There may be. In that case, by scaling the unidirectional motion information and generating a scaled combined motion information candidate, it is possible to encode only the merge index without encoding the motion information. Therefore, it is possible to realize a moving image encoding device and a moving image decoding device that can optimize encoding efficiency and prediction efficiency.

(Simplified video decoding process)
As described above, in the case where the motion of the block to be processed moves at a constant speed with any reference image in the LX direction and any reference image in the LY direction, the same position of another encoded image There may be no motion information indicating the same speed motion as the processing target block in the motion information of the block or the block adjacent to the processing target block. In that case, only the single direction in the L0 direction or the L1 direction in the motion information of the same position block of another encoded image or the adjacent block of the processing target block is the same constant motion as the processing target block. There may be. In that case, by scaling the unidirectional motion information to generate a scaled combined motion information candidate, decoding of the prediction direction, reference index and difference vector, addition processing between the prediction vector and the difference vector, etc. are not required. Processing of the image decoding device can be reduced.

(Merge candidate number assignment in order of selectivity)
As described above, by assigning a smaller merge candidate number to a combined motion information candidate having a relatively high selection rate, highly accurate scaling combined motion information using highly accurate unidirectional motion information in each direction. Candidates can be generated. In addition, the search process can be simplified, and a reduction in encoding efficiency can be suppressed even if the number of search processes is limited.

(Memory lead time)
As described above, the combined motion information candidates are generated without increasing the number of single combined motion information candidates by generating the scaling combined motion information candidates using the unidirectional motion information in each direction of the single combined motion information candidates. The number of can be increased. Therefore, in a moving picture encoding apparatus and moving picture decoding apparatus using a general LSI in which the memory read time is increased due to an increase in the number of single combined motion information candidates, a memory due to an increase in the number of single combined motion information candidates An increase in lead time can be suppressed.

[Embodiment 2]
(Configuration of Combined Motion Information Candidate Generation Unit 140 of Embodiment 2)
The configuration of the moving image encoding apparatus according to the second embodiment is the same as that of the moving image encoding apparatus 100 according to the first embodiment except for the combined motion information candidate generation unit 140. FIG. 45 is a diagram for explaining a candidate number management table according to the second embodiment. First, the candidate number management table in Embodiment 2 is shown in FIG. 45, and the maximum number of combined motion information candidates included in the combined motion information candidate list is 5. The difference is that the maximum number of combined motion information candidates included in the combined motion information candidate list is 5, and no merge candidate number is assigned to the dual combined motion information candidates. Hereinafter, the difference between the combined motion information candidate generation unit 140 in the second embodiment and the first embodiment will be described with reference to FIG. FIG. 46 is a diagram for explaining a configuration of the combined motion information candidate generation unit 140 according to the second embodiment.

  46 has a configuration in which a first combined motion information candidate list reduction unit 153 and a dual combined motion information candidate generation unit 154 are added to the combined motion information candidate generation unit 140 in FIG. Hereinafter, functions of each unit will be described.

(Function of Combined Motion Information Candidate Generation Unit 140 of Embodiment 2)
The first combined motion information candidate list reduction unit 153 has a plurality of combined motion information candidates having motion information that is duplicated from the first combined motion information candidate list supplied from the single combined motion information candidate list generation unit 150. The second combined motion information candidate list is generated by leaving one combined motion information candidate, deleting the remaining, and supplying the second combined motion information candidate list to the dual combined motion information candidate generating unit 154.

  The bi-coupled motion information candidate generating unit 154 generates bi-coupled motion information candidates from the second combined motion information candidate list supplied from the first combined motion information candidate list reducing unit 153, and the bi-coupled motion information candidate and the second The combined motion information candidate list is supplied to the scaling combined motion information candidate generation unit 151. In addition, the bi-coupled motion information candidate generation unit 154 sets a target candidate group of scaling combined motion information candidates, and supplies the target candidate group to the scaling combined motion information candidate generation unit 151.

  Here, the double coupled motion information is new motion information obtained by using the motion information of the two candidate blocks. In the second embodiment, it is assumed that a maximum of two bi-coupled motion information candidates (BD0) and a second bi-coupled motion information candidate (BD1) are generated as bi-coupled motion information candidates. It should be noted that the maximum number of bi-coupled motion information candidates is not limited to 2 and may be 1 or more. The detailed configuration of the dual coupled motion information candidate generation unit 154 will be described later.

  The scaling combined motion information candidate generation unit 151 generates a scaling combined motion information candidate from the second combined motion information candidate list supplied from the double combined motion information candidate generation unit 154 and the target candidate group described above. Then, the above-described dual combined motion information candidate and the scaled combined motion information candidate are added to the second combined motion information candidate list, and the second combined motion information candidate list is supplied to the combined motion information candidate list reduction unit 152.

(Configuration of the dual coupled motion information candidate generation unit 154)
Next, a detailed configuration of the dual coupled motion information candidate generation unit 154 will be described with reference to FIG. FIG. 47 shows the configuration of the dual coupled motion information candidate generation unit 154. The terminal 19 is connected to the first combined motion information candidate list reduction unit 153, and the terminal 20 is connected to the scaling combined motion information candidate generation unit 151. The double coupled motion information candidate generating unit 154 includes a reference direction determining unit 300, a reference direction motion information determining unit 301, a reverse direction motion information determining unit 302, a dual coupled motion information candidate determining unit 303, and a dual coupled motion information candidate adding unit 304. Including.

(Function of the dual coupled motion information candidate generation unit 154)
Hereinafter, functions of each unit will be described. The reference direction determination unit 300 determines the reference direction of the dual combined motion information candidate from the first combined motion information candidate list, and determines the reference direction motion information from the reference direction and the first combined motion information candidate list supplied from the terminal 19. Send to part 301. The reference direction of BD0 is the L0 direction, and the reference direction of BD1 is the L1 direction.

  The reference direction motion information determination unit 301 determines the motion vector and reference image index of the reference direction of the bi-coupled motion information candidate from the reference direction and the first combined motion information candidate list supplied from the reference direction determination unit 300, and The reference direction, the motion vector in the reference direction, the reference image index, and the first combined motion information candidate list are sent to the backward motion information determination unit 302.

  The reverse direction motion information determination unit 302 uses the reference direction, the reference direction motion vector and the reference image index, and the first combined motion information candidate list supplied from the reference direction motion information determination unit 301 to reverse the bi-join motion information candidate. A direction motion vector and a reference image index are determined. The backward motion information determination unit 302 sends the reference direction motion vector and reference image index, the backward motion vector and reference image index, and the first combined motion information candidate list to the double combined motion information candidate determination unit 303. .

  In Embodiment 2, if the reference direction is the L0 direction, the reverse direction is the L1 direction, and if the reference direction is the L1 direction, the reverse direction is the L0 direction. In the second embodiment, if the reference direction is the L0 direction, the reverse direction is the L1 direction, and if the reference direction is the L1 direction, the reverse direction is the L0 direction. Not. For example, the direction opposite to the reference direction can be set as the same prediction direction.

  The bi-join motion information candidate determination unit 303 predicts a bi-join motion information candidate from the base direction motion vector and reference image index supplied from the reverse direction motion information determination unit 302, and the reverse direction motion vector and reference image index. Is generated, and a double coupled motion information candidate is generated and supplied to the dual coupled motion information candidate adding unit 304.

  The bi-coupled motion information candidate adding unit 304 adds the bi-coupled motion information candidate supplied from the bi-coupled motion information candidate determining unit 303 to the first combined motion information candidate list, and the first combined motion information candidate list is added to the terminal 20. Send to.

(Operation of Combined Motion Information Candidate Generation Unit 140 of Embodiment 2)
Subsequently, the operation of the combined motion information candidate generation unit 140 according to the second embodiment will be described with reference to FIG. 48 and the difference from the first embodiment. FIG. 48 is a flowchart for explaining the operation of the combined motion information candidate generation unit 140 of the second embodiment. In the flowchart of FIG. 48, the following steps are added or changed to the flowchart of FIG.

  The first combined motion information candidate list reduction unit 153 has a plurality of combined motion information candidates having motion information that is duplicated from the first combined motion information candidate list supplied from the single combined motion information candidate list generation unit 150. In step S146, one combined motion information candidate is left and the rest is deleted to generate a second combined motion information candidate list.

  The combined motion information candidate generation unit 140 checks whether or not the number of valid combined motion information candidates included in the second combined motion information candidate list has reached a predetermined upper limit (step S147). In the second embodiment, the predetermined upper limit number is set to 5, which is the maximum number of combined motion information candidates included in the combined motion information candidate list. If the number of valid combined motion information candidates included in the second combined motion information candidate list has not reached the predetermined upper limit number (N in S147), the dual combined motion information candidate generation unit 154 generates the second combined motion information candidate list. Are generated in a range not exceeding the predetermined upper limit (S148).

  The scaling combined motion information candidate generation unit 151 sequentially adds the bi-coupled motion information candidate and the scaling combined motion information candidate to the second combined motion information candidate list to generate a second combined motion information candidate list (S144). In Embodiment 2, the addition order of the bi-join motion information candidate and the scaled joint motion information candidate to the second joint motion information candidate list is the order of the bi-join motion information candidate and the scaled joint motion information candidate. An information candidate and a scaling combined motion information candidate may be added, and the present invention is not limited to this. For example, it may be in the order of scaling combined motion information candidates and bi-coupled motion information candidates, and when there are a plurality of them, they may be added to the second combined motion information candidate list in an arbitrary order.

  If the number of valid combined motion information candidates included in the second combined motion information candidate list has reached the predetermined upper limit number (Y in S147), step S148, step S143, step S144, and step S145 are skipped. In addition, since the operation | movement of the production | generation of the scaling joint motion information candidate of Embodiment 2 differs from Embodiment 1, the detail of the operation | movement of the production | generation of the scaling joint motion information candidate of Embodiment 2 is mentioned later.

(Operation of the dual coupled motion information candidate generation unit 154)
Subsequently, the detailed operation of the dual coupled motion information candidate generation unit 154 will be described with reference to the flowchart of FIG. FIG. 49 is a flowchart for explaining the detailed operation of the dual-coupled motion information candidate generation unit 154. The reference direction determination unit 300 determines the reference direction of the dual combined motion information candidate from the second combined motion information candidate list (S400). The reference direction of BD0 is the L0 direction, and the reference direction of BD1 is the L1 direction.

  The reference direction motion information determination unit 301 determines the reference direction motion vector and the reference image index of the bi-join motion information candidate from the reference direction and the second combined motion information candidate list supplied from the reference direction determination unit 300 (S401). ). Detailed operation of the reference direction motion information determination unit 301 will be described later.

  The backward direction motion information determination unit 302 uses the reference direction, the reference direction motion vector, the reference image index, and the second combined motion information candidate list supplied from the reference direction motion information determination unit 301 to return the reverse direction of the bi-join motion information candidate. The motion vector and the reference image index are determined (S402). Detailed operation of the backward motion information determination unit 302 will be described later.

  The bi-join motion information candidate determination unit 303 uses the base direction, the base direction motion vector and the reference image index, and the reverse motion vector and the reference image index supplied from the reverse direction motion information determination unit 302. The candidate prediction direction is determined to generate a double coupled motion information candidate (S403).

  The dual coupled motion information candidate determination unit 303 checks whether the prediction direction of the dual coupled motion information candidate is bidirectional (BI) (S404). If the prediction direction of the bi-coupled motion information candidate is bidirectional (Y in S404), the bi-coupled motion information candidate adding unit 304 adds the bi-coupled motion information candidate to the second combined motion information candidate list (S405). If the prediction direction of the bi-coupled motion information candidate is not bidirectional (N in S404), it is checked whether the prediction direction of the bi-coupled motion information candidate is unidirectional (L0 direction or L1 direction) (S406). If the prediction direction of the bi-join motion information candidate is unidirectional (Y in S406), a candidate block having a motion vector in a valid prediction direction and a reference index is set as a target group of scaling joint motion information candidates (S407). .

  A candidate block having a motion vector in a valid prediction direction and a reference index is a candidate block having a motion vector and a reference image index determined by the base direction motion information determination unit 301 or the reverse direction motion information determination unit 302. Details will be described later. If the prediction direction of the bi-coupled motion information candidate is not unidirectional (N in S406), step S407 is skipped.

  Subsequently, the detailed operation of the reference direction motion information determination unit 301 will be described using the flowchart of FIG. FIG. 50 is a flowchart for explaining the operation of the reference direction motion information determination unit 301. It is assumed that the LX direction (X is 0 or 1) is selected as the reference direction for the double coupled motion information candidate. The validity of the reference direction LX is set to “0” (S190). The following process is repeated a number of times corresponding to the number of combined motion information candidates (NCands) included in the second combined motion information candidate list (S191 to S194). The validity of the combined motion information candidate in the LX direction is checked (S192). If the LX direction (LX prediction) of the combined motion information candidate is valid (Y in S192), the validity of the reference direction LX is set to “1”, and the motion vector and reference index in the standard direction are combined motion information. The process ends with the candidate motion vector in the LX direction and the reference index (S193). If the LX direction of the combined motion information candidate is invalid (N in S192), the next candidate is examined (S194).

  Here, the number of times corresponding to the number of combined motion information candidates (NCands) included in the second combined motion information candidate list is examined, but it is only necessary to determine the reference direction of the dual combined motion information candidates, and the present invention is not limited to this. For example, when generating a combined motion information candidate from only combined motion information candidates having a relatively high selection rate, the number of inspections is fixed to a predetermined number such as 1 or 2 to reduce the processing amount and redundant dual motion information candidates. It is also possible to reduce the amount of merge index codes by reducing the possibility of generating combined motion information candidates.

  Subsequently, the detailed operation of the backward motion information determination unit 302 will be described using the flowchart of FIG. FIG. 51 is a flowchart for explaining the operation of the backward motion information determination unit 302. The reverse direction of the reference direction is set as the reverse direction of the double coupled motion information candidate. It is assumed that the LY direction (Y is 0 or 1) is selected as the reverse direction. The validity of LY in the reverse direction is set to “0” (S200). The following processing is repeated a number of times corresponding to the number of combined motion information candidates (NCands) included in the second combined motion information candidate list (S201 to S205).

  It is checked that the combined motion information candidate is not selected in the reference direction (S202). If it is not the combined motion information candidate selected in the reference direction (Y in S202), the validity of the combined motion information candidate in the LY direction is checked (S203). If the LY direction (LY prediction) of the combined motion information candidate is valid (Y in S203), the validity of LY, which is the reverse direction, is set to “1”, and the reverse direction motion vector and the reference index are combined motion. The processing ends with the motion vector in the LY direction of the information candidate and the reference index (S204). If it is a combined motion information candidate selected in the reference direction (N in S202), or if the LY direction of the combined motion information candidate is invalid (N in S203), the next candidate is examined (S205).

  Here, the number of times corresponding to the number of combined motion information candidates (NCands) included in the second combined motion information candidate list is examined, but it is only necessary to determine the reference direction of the dual combined motion information candidates, and the present invention is not limited to this. For example, when generating a combined motion information candidate from only combined motion information candidates having a relatively high selectivity, the number of examinations is fixed to a predetermined number such as 2 or 3, and the processing amount is reduced and redundant dual information is also generated. It is also possible to reduce the amount of merge index codes by reducing the possibility of generating combined motion information candidates. In addition, by setting the block to start the inspection as the combined motion information candidate next to the combined motion information candidate selected in the reference direction, the possibility that BD0 and BD1 are the same can be eliminated, and step S202 can be reduced. .

  Subsequently, the detailed operation of determining the prediction direction of the dual coupled motion information candidate will be described with reference to the diagram of FIG. FIG. 52 is a diagram for explaining the determination of the prediction direction of the dual coupled motion information candidate. If the LX direction and the LY direction of the bi-coupled motion information candidate determined as described above are both valid, the prediction direction is a bidirectional BI, and if only the LX direction is valid, the prediction direction is a unidirectional LX direction. If only the LY direction is valid, the prediction direction is a single LY direction. If both the LX direction and the LY direction are invalid, the prediction direction is invalid.

  Subsequently, details of the operation of generating the scaled combined motion information candidate according to the second embodiment will be described with reference to FIG. FIG. 53 is a flowchart for explaining an operation of generating scaling combined motion information candidates according to the second embodiment. The target candidate group supplied from the double coupled motion information candidate generation unit 154 is set as a candidate block to be examined (S337). The following processing is repeated a number of times corresponding to the number of candidate blocks (MCands) included in the target candidate group (S338 to S339). The maximum number of candidate blocks included in the target candidate group in the second embodiment is two. Steps S334 and S335 are the same as those in the first embodiment.

  The configuration of the moving picture decoding apparatus according to the second embodiment is the same as that of the moving picture decoding apparatus 200 according to the first embodiment except for the function of the combined motion information candidate generation unit 140. The combined motion information candidate generation unit 140 of the video decoding device of the second embodiment is the same as the combined motion information candidate generation unit 140 of the video encoding device of the second embodiment.

(Modification of Embodiment 2)
The second embodiment can be modified as follows.

(Modification 1: Number of candidates for double coupled motion information)
In the above-described second embodiment, the first bi-coupled motion information candidate (BD0) having the first motion information effective in the L0 direction as the reference direction as the double-coupled motion information candidate and first effective in the L1 direction. Two second combined motion information candidates (BD1) having motion information as a reference direction are generated. However, the number is not limited to this as long as it is one or more. For example, the third bi-coupled motion information candidate (BD2) that uses the motion information that is the second most effective in the L0 direction as the reference direction and the fourth dual information that uses the motion information that is the second most effective in the L1 direction as the reference direction. A combined motion information candidate (BD3) or the like can be added, or a plurality of combined motion information candidates can be selected adaptively to make a single combined motion information candidate.

(Modification 2: Deletion process)
In the second embodiment described above, the combined motion information candidate generation unit shown in FIG. 46 is used to reduce the number of redundant combined motion information candidates and increase the number of added combined motion information candidates or scaling combined motion information candidates. The first combined motion information candidate list reduction unit 153 is included in 140, but when there are a plurality of combined motion information candidates having overlapping motion information, one combined motion information candidate is left and the rest is deleted For this purpose, the first combined motion information candidate list reduction unit 153 may not be included (step S146 in FIG. 48). For the same purpose, the combined motion information candidate list reduction unit 152 may not be included (step S145 in FIG. 48).

(Modification 3: Predetermined upper limit number)
In the above-described second embodiment, the predetermined upper limit number that is a condition for generating the bi-join motion information candidates is set to 5 that is the maximum number of combined motion information candidates included in the combined motion information candidate list. The number may be equal to or less than the maximum number of combined motion information candidates included in the motion information candidate list, and is not limited thereto. For example, the number of combined motion information candidates deleted by the first combined motion information candidate list reduction unit 153 may be used. As a result, NumMergeCands () can be determined before the operation of the first combined motion information candidate list reduction unit 153, merge_idx can be decoded, and the speed of the decoding process can be increased.

(Modification 4: Simple connection)
In Embodiment 2 described above, the scaled combined motion information candidate is generated when the prediction direction of the bi-coupled motion information candidate is unidirectional. However, it is only necessary to generate the bi-coupled motion information candidate and the scaled combined motion information candidate. However, the present invention is not limited to this. For example, in Embodiment 2, the operation of the dual coupled motion information candidate generation unit 154 is shown in FIG. 49, but step S406 and step S407 are deleted, and the prediction direction of the dual coupled motion information candidate is not bidirectional ( In step S404, the process may be terminated by skipping step S405, and the bi-coupled motion information candidate and the scaled combined motion information candidate may be sequentially generated without dependency as shown in FIG.

  FIG. 54 is a flowchart for explaining the operation of the combined motion information candidate generation unit 140 according to the fourth modification of the second embodiment. The flowchart of FIG. 54 adds the following steps to the flowchart of FIG. 48, which is a flowchart for explaining the operation of the combined motion information candidate generation unit 140 of the second embodiment.

  The combined motion information candidate generation unit 140 checks whether the number of valid combined motion information candidates included in the second combined motion information candidate list has reached a predetermined upper limit number (step S340). If the number of valid combined motion information candidates included in the second combined motion information candidate list does not reach the predetermined upper limit number (N in S340), the scaling combined motion information candidate generation unit 151 outputs the second combined motion information candidate list. Are generated in a range that does not exceed the predetermined upper limit number (S143). If the number of valid combined motion information candidates included in the second combined motion information candidate list has reached the predetermined upper limit (Y in S340), step S143 is skipped. In FIG. 54, the generation of the double coupled motion information candidate (step S143) and the generation of the scaling combined motion information candidate (step S148) can be interchanged.

(Modification 5: Simple connection 2)
The fourth modification of the second embodiment can be further modified as follows. In Embodiment 2, the operation of the dual coupled motion information candidate generation unit 154 is shown in FIG. 49, but step S407 is changed as follows. By combining the motion vector of the valid prediction direction with the motion vector of the valid prediction direction by setting the motion vector of the invalid prediction direction to (0, 0) and the reference image index to 0, the bi-join motion information candidate having the bidirectional prediction direction is obtained. It is generated and added to the second combined motion information candidate list (S407). In the fifth modification of the second embodiment, the motion vector in the invalid prediction direction is (0, 0) and the reference image index is 0. However, if the motion vector and the reference image in the invalid prediction direction can be set to predetermined values, Well, not limited to this.

(An example of the effect of Embodiment 2)
An example of the effect of the second embodiment will be described with reference to FIG. FIG. 55 is a diagram for explaining the effect of the second embodiment. The motion information with the smallest prediction error for the processing target block Z is bi-directional (BI), mvL0Z = (13, -2), mvL1Z = (4,4), refIdxL0Z = 0, refIdxL1Z = 1. Suppose there is. At this time, it is assumed that the single combined motion information candidates are A, B, COL, C, and E in FIG.

  Among these single coupled motion information candidates, there is no motion information identical to the motion information that minimizes the prediction error for the processing target block Z. Therefore, a single combined motion information candidate having a minimum rate distortion evaluation value is selected from these single combined motion information candidates. Then, the candidate rate distortion evaluation value and the rate distortion evaluation value calculated by the difference vector calculation unit 120 are compared, and the merge mode is used as the encoding mode only when the former is smaller than the latter. Become.

  When the merge mode is selected as the encoding mode, it is because the balance between the encoding efficiency of motion information and the prediction error is optimal, and the prediction error is not optimal. On the other hand, when the non-merge mode is selected as the encoding mode, it is necessary to encode the differential motion vector and the reference image index, so that the encoding efficiency of the motion information is not optimal.

  Here, BD0 whose reference direction is generated in the second embodiment is generated by combining the motion vector of the block A in the L0 direction and the reference image index, and the motion vector of the block C in the L1 direction and the reference image index. It becomes BD0 of FIG. For BD1 whose reference direction is the L1 direction, a scaling combined motion information candidate is generated from the motion vector of the block C in the L1 direction and the reference image index, and becomes BP0 in FIG.

  At this time, it can be seen that BD0 has the same motion information as the motion information that minimizes the prediction error for the processing target block Z. That is, by selecting BD0, the prediction error can be minimized and the coding efficiency of motion information can be optimized. Note that the motion information that minimizes the prediction error for the processing target block Z is bidirectional (BI) in the prediction direction, mvL0Z = (− 2, −2), mvL1Z = (4,4), refIdxL0Z = 0, refIdxL1Z. If = 1, by selecting BP0, the prediction error can be minimized and the coding efficiency of motion information can be optimized.

(Effect of Embodiment 2)
As described above, the motion of the block to be processed is the same as that of another encoded image by generating the double-coupled motion information candidate using the motion information in the L0 direction and the L1 direction of the single-coupled motion information candidate. Even when there is a deviation from the motion of the position block or the adjacent block of the processing target block, the motion information can be encoded only by the index without encoding the motion information. Therefore, it is possible to realize a moving image encoding device and a moving image decoding device that can optimize encoding efficiency and prediction efficiency.

  As described above, the combined motion information candidate and the scaling combined motion information candidate are used in combination so that the motion of the processing target block is the same position block of another encoded image or the adjacent block of the processing target block. Or when the motion of the block to be processed is moving at a constant speed with any reference image in the LX direction and any reference image in the LY direction. When there is no motion information indicating the same speed motion as the processing target block in the motion information of the same position block of another image or the adjacent block of the processing target block, and the same of another encoded image When only the single direction of the L0 direction or the L1 direction in the motion information of the position block and the adjacent block of the processing target block is the same constant motion as the processing target block It can be encoded only in the merge index without encoding motion information. Therefore, it is possible to realize a moving image encoding device and a moving image decoding device that can optimize encoding efficiency and prediction efficiency.

  The moving image encoded stream output from the moving image encoding apparatus according to the first and second embodiments described above is specified so that it can be decoded according to the encoding method used in the first and second embodiments. Data format. A moving picture decoding apparatus corresponding to the moving picture encoding apparatus can decode an encoded stream of this specific data format.

  Specifically, a merge index indicating a candidate for bi-join motion information and a candidate number management table are encoded in the encoded stream. Also, only the merge index indicating the bi-join motion information candidate is encoded in the encoded stream, and the candidate number management table is shared by the video encoding device and the video decoding device, so that the candidate number management table is included in the encoded stream. It may not be encoded.

  When a wired or wireless network is used to exchange an encoded stream between a moving image encoding device and a moving image decoding device, the encoded stream is converted into a data format suitable for the transmission form of the communication path. It may be transmitted. In that case, a video transmission apparatus that converts the encoded stream output from the video encoding apparatus into encoded data in a data format suitable for the transmission form of the communication channel and transmits the encoded data to the network, and receives the encoded data from the network Then, a moving image receiving apparatus that restores the encoded stream and supplies the encoded stream to the moving image decoding apparatus is provided.

  The moving image transmitting apparatus is a memory that buffers the encoded stream output from the moving image encoding apparatus, a packet processing unit that packetizes the encoded stream, and transmission that transmits the packetized encoded data via the network. Part. The moving image receiving apparatus generates a coded stream by packetizing the received data, a receiving unit that receives the packetized coded data via a network, a memory that buffers the received coded data, and packet processing. And a packet processing unit provided to the video decoding device.

  The above encoding and decoding processes can be realized as a transmission, storage, and reception device using hardware, as well as firmware stored in a ROM (Read Only Memory), flash memory, and the like. It can also be realized by software such as a computer. The firmware program and software program can be recorded and provided on a computer-readable recording medium, provided from a server through a wired or wireless network, or provided as a data broadcast of terrestrial or satellite digital broadcasting. Is also possible.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  DESCRIPTION OF SYMBOLS 100 moving image encoder, 101 prediction block image acquisition part, 102 subtraction part, 103 prediction error encoding part, 104 code stream production | generation part, 105 prediction error decoding part, 106 motion compensation part, 107 addition part, 108 motion vector detection , 109 motion information generation unit, 110 frame memory, 111 motion information memory, 120 difference vector calculation unit, 121 combined motion information determination unit, 122 prediction coding mode determination unit, 130 prediction vector candidate list generation unit, 131 prediction vector determination , 132 subtraction unit, 140 combined motion information candidate generation unit, 141 combined motion information selection unit, 150 single combined motion information candidate list generation unit, 151 scaling combined motion information candidate generation unit, 152 combined motion information candidate list reduction unit, 153 First coupled motion Information candidate list reduction unit, 154 double coupled motion information candidate generation unit, 160 target candidate selection unit, 161 reference image setting unit, 162 scaling unit, 163 scaling combined motion information candidate addition unit, 200 moving image decoding apparatus, 201 code string Analysis unit 202 Prediction error decoding unit 203 Addition unit 204 Motion information reproduction unit 205 Motion compensation unit 206 Frame memory 207 Motion information memory 210 Coding mode determination unit 211 Motion vector reproduction unit 212 Combined motion information Reproducing unit 220 prediction vector candidate list generation unit 221 prediction vector determination unit 222 addition unit 230 combined motion information candidate generation unit 231 combined motion information selection unit 300 reference direction determination unit 301 reference direction motion information determination unit 302 Reverse movement Information determination unit, 303 bi-binding motion information candidate determination section, 304 bi-binding motion information candidate adding unit.

Claims (13)

An image decoding apparatus that performs motion compensation prediction,
A candidate list generating unit that generates a candidate list by selecting a plurality of blocks having motion information including at least motion vector information and reference image information from a plurality of decoded blocks adjacent to the decoding target block;
A motion information acquisition unit that acquires motion information of blocks included in the candidate list;
A scaling unit that generates a second motion vector by scaling a first motion vector included in the motion information acquired by the motion information acquisition unit;
A selection candidate generating unit that generates a new selection candidate having motion information including the first motion vector and motion information including the second motion vector;
A code string analyzing unit for decoding candidate specifying information for specifying a candidate selected from the candidate list on the encoding side in the candidate list;
A selection unit that selects one candidate from the selection candidates included in the candidate list generated by the candidate list generation unit using the decoded candidate identification information;
An image decoding apparatus comprising: When the number of blocks included in the candidate list is less than the set maximum number, a new selection candidate generated by the selection candidate generation unit is added to the candidate list.
The image decoding apparatus according to claim 1. The plurality of decoded blocks adjacent to the decoding target block include a first type having unidirectional motion information and a second type having bidirectional information.
The motion information acquisition unit preferentially acquires the first type block among a plurality of blocks included in the candidate list.
The image decoding apparatus according to claim 1, wherein the image decoding apparatus is an image decoding apparatus. The plurality of decoded blocks adjacent to the decoding target block include a first type having unidirectional motion information and a second type having bidirectional motion vectors,
The motion information acquisition unit preferentially acquires the second type block among a plurality of blocks included in the candidate list.
The image decoding apparatus according to claim 1, wherein the image decoding apparatus is an image decoding apparatus. The motion information acquisition unit preferentially acquires a block having a relatively high degree of adjacency with the decoding target block among a plurality of blocks included in the candidate list.
The image decoding apparatus according to claim 1, wherein the image decoding apparatus is an image decoding apparatus. When the motion information acquired by the motion information acquisition unit is motion information of a block having a bidirectional motion vector, the scaling unit selects the longer motion vector of the two motion vectors of the block. Scaling as the first motion vector,
The image decoding device according to claim 1, wherein the image decoding device is an image decoding device. When the motion information acquired by the motion information acquisition unit is motion information of a block having a bidirectional motion vector, the scaling unit calculates a shorter motion vector of the two motion vectors of the block. Scaling as the first motion vector,
The image decoding device according to claim 1, wherein the image decoding device is an image decoding device. A reference image setting unit that sets a value of a reference index that identifies a reference image of the second motion vector generated by the scaling unit to zero;
The image decoding apparatus according to claim 1, further comprising: The plurality of decoded blocks adjacent to the decoding target block include blocks of images that are temporally different from the image including the decoding target block,
When the reference index value for specifying the reference image of the first motion vector is the same as the reference index value for specifying the reference image of the second motion vector, the reference image setting unit A reference index value that identifies a reference image of a motion vector of a block of a different image is set to a reference index value that identifies a reference image of the second motion vector;
The image decoding apparatus according to claim 8. The motion information acquisition unit acquires motion information in the first prediction direction from the first block included in the candidate list, and motion information in the second prediction direction from the second block included in the candidate list. ,
The selection candidate generating unit is a new selection having two pieces of motion information based on the motion information in the first prediction direction acquired by the motion information acquisition unit and the motion information in the second prediction direction. Generate suggestions,
The image decoding device according to claim 1, wherein the image decoding device is an image decoding device. The scaling unit is included in the motion information of the effective prediction direction when the motion information of the first prediction direction or the motion information of the second prediction direction acquired by the motion information acquisition unit is invalid. Generating a second motion vector by scaling a motion vector to be generated as the first motion vector;
The image decoding apparatus according to claim 10. An image decoding method for performing motion compensation prediction,
Selecting a plurality of blocks having motion information including at least motion vector information and reference image information from a plurality of decoded blocks adjacent to the decoding target block, and generating a candidate list;
Obtaining motion information of blocks included in the candidate list;
Scaling a first motion vector included in the acquired motion information to generate a second motion vector;
Generating a new selection candidate having motion information including the first motion vector and motion information including the second motion vector;
Decoding candidate identification information for identifying in the candidate list a candidate selected from the candidate list on the encoding side;
Selecting one candidate from the selection candidates included in the candidate list using the decoded candidate identification information;
An image decoding method comprising: An image decoding program for performing motion compensation prediction,
A process for generating a candidate list by selecting a plurality of blocks having motion information including at least motion vector information and reference image information from a plurality of decoded blocks adjacent to the decoding target block;
A process of acquiring motion information of blocks included in the candidate list;
Processing to generate a second motion vector by scaling the first motion vector included in the acquired motion information;
Processing for generating a new selection candidate having motion information including the first motion vector and motion information including the second motion vector;
Decoding candidate specifying information for specifying a candidate selected from the candidate list on the encoding side in the candidate list;
A process of selecting one candidate from the selection candidates included in the candidate list using the decoded candidate specifying information;
An image decoding program that causes a computer to execute the above.

Images