RU2783841C2 - Video encoding device, video encoding method, video decoding device and video decoding method - Google Patents

Abstract

FIELD: video encoding.

SUBSTANCE: invention relates to means for video encoding. List of triangle fusion candidates is then compiled, including spatial fusion candidates. From the list of triangle fusion candidates, a first triangle fusion candidate is selected, which is a unidirectional prediction. A second triangle merge candidate is chosen, which is a one-way prediction. Meanwhile, in the region where motion compensation is performed with the first triangle fusion candidate, motion information of the first triangle fusion candidate is stored. In the area where motion compensation is performed with the second triangle merge candidate, motion information of the second triangle merge candidate is stored. And in the area where motion compensation is performed by weighted averaging with the first and second triangle fusion candidates, motion information of the unidirectional prediction of the first or second triangle fusion candidate is stored.

EFFECT: improving the efficiency of video encoding.

4 cl, 67 dwg

Description

State of the art

The present invention relates to image encoding and decoding methods in which an image is divided into blocks and prediction is performed.

When encoding and decoding an image, the target image is divided into blocks, each of which is a group of a predetermined number of samples, and processing is performed in blocks of blocks. Dividing an image into appropriate blocks with appropriate intra-prediction and inter-prediction settings can improve coding efficiency.

When encoding/decoding a moving picture, inter-picture prediction is used, which performs prediction from an encoded/decoded picture, thereby improving encoding efficiency. Patent Document 1 describes a method for applying an affine transform during inter prediction. Moving images often have object shape distortions such as enlargement/reduction or rotation, and thus the application of the method in Patent Document 1 provides efficient coding.

[Patent Document 1] JP 9-172644A.

Disclosure of the essence of the invention

However, the method of Patent Document 1 involves performing an image conversion process, which results in a large processing load. The present invention has been proposed in view of the above technical problem, and provides a low load and an efficient encoding method.

In one aspect of the present invention, for solving the above technical problem, a method is provided, which includes: a triangle merge candidate list constructor, configured to construct a triangle merge candidate list including spatial merge candidates; a first triangle merge candidate selector, configured to select from the list of triangle merge candidates a first triangle merge candidate that is unidirectional prediction; and a second triangle merge candidate selector, configured to select from the triangle merge candidate list a second triangle merge candidate, which is a unidirectional prediction in which information is stored in the area where motion compensation is performed by weighted averaging by the first triangle merge candidate and the second triangle merge candidate. one-way prediction motions of one of the first triangle fusion candidate or the second triangle fusion candidate.

According to the present invention, a highly efficient low load image encoding/decoding process is provided.

Brief description of the drawings

Fig. 1 is a block diagram of an image encoding apparatus according to an embodiment of the present invention.

Fig. 2 is a block diagram of an apparatus for decoding an image according to an embodiment of the present invention.

Fig. 3 is a flowchart illustrating the tree block splitting operation.

Fig. 4 is a diagram illustrating a state of dividing an input image into tree blocks.

Fig. 5 is a diagram illustrating a z-scan.

Fig. 6A is a diagram illustrating the shape of a divided block.

Fig. 6B is a diagram illustrating the shape of a divided block.

Fig. 6C is a diagram illustrating the shape of a divided block.

Fig. 6D is a diagram illustrating the shape of a divided block.

Fig. 6E is a diagram illustrating the shape of a divided block.

Fig. 7 is a flowchart illustrating an operation of dividing a block into four.

Fig. 8 is a flowchart illustrating the operation of dividing a block into two or three.

Fig. 9 is the syntax for expressing the block division form.

Fig. 10A is a diagram illustrating intra prediction.

Fig. 10B is a diagram illustrating intra prediction.

Fig. 11 is a diagram illustrating reference blocks for inter prediction.

Fig. 12A is a syntax for expressing a coding block prediction mode.

Fig. 12B is a syntax for expressing a coding block prediction mode.

Fig. 13 is a diagram illustrating the correspondence between syntax elements and modes related to inter prediction.

Fig. 14 is a diagram illustrating affine motion compensation at two reference points.

Fig. 15 is a diagram illustrating affine motion compensation at three reference points.

Fig. 16 is a detailed configuration diagram of the inter prediction block 102 of FIG. one.

Fig. 17 is a detailed configuration diagram of the normal motion vector predictor mode acquisition block 301 of FIG. 16.

Fig. 18 is a detailed configuration diagram of the normal merge mode acquisition block 302 of FIG. 16.

Fig. 19 is a flowchart illustrating a normal motion vector predictor mode acquisition process of the normal motion vector predictor mode acquisition unit 301 of FIG. 16.

Fig. 20 is a flowchart illustrating a normal motion vector predictor mode acquisition processing procedure.

Fig. 21 is a flowchart illustrating a normal merge mode acquisition process processing procedure.

Fig. 22 is a detailed configuration diagram of the inter prediction block 203 of FIG. 2.

Fig. 23 is a detailed configuration diagram of the normal motion vector predictor mode acquisition block 401 of FIG. 22.

Fig. 24 is a detailed configuration diagram of the normal merge mode acquisition block 402 of FIG. 22.

Fig. 25 is a flowchart illustrating a normal motion vector predictor mode acquisition process of the normal motion vector predictor mode acquisition unit 401 of FIG. 22.

Fig. 26 is a diagram illustrating a process of initializing/updating a motion vector predictor candidate list based on history.

Fig. 27 is a flowchart of an identical element confirmation processing procedure in a history-based motion vector predictor candidate list initialization/updating processing procedure.

Fig. 28 is a flowchart of the element shift processing procedure in the history-based motion vector predictor candidate list initialization/updating processing procedure.

Fig. 29 is a flowchart illustrating a processing procedure for obtaining a motion vector predictor candidate based on history.

Fig. 30 is a flowchart illustrating a history-based merge candidate acquisition processing procedure.

Fig. 31A is a diagram illustrating an example of a process for updating a motion vector predictor candidate list based on history.

Fig. 31B is a diagram illustrating an example of a process for updating a motion vector predictor candidate list based on history.

Fig. 31C is a diagram illustrating a process of updating a motion vector predictor candidate list based on history.

Fig. 32 is a diagram illustrating motion compensation prediction in a case where L0 prediction is performed and an L0 reference picture (RefL0Pic) occurs at a time before the target picture (CurPic).

Fig. 33 is a diagram illustrating motion-compensated prediction in a case where L0 prediction is performed and the L0 prediction reference picture occurs at a time after the target picture.

Fig. 34 is a diagram illustrating a motion compensation prediction direction in bidirectional prediction in which L0 reference picture prediction occurs at a time before the target picture and L1 reference picture prediction occurs at a time after the target picture.

Fig. 35 is a diagram illustrating a direction of motion compensation prediction in bidirectional prediction in which L0 reference picture prediction and L1 reference picture prediction occur before the target frame.

Fig. 36 is a diagram illustrating a motion compensation prediction direction in bidirectional prediction in which L0 reference picture prediction and L1 reference picture prediction occur at a time after the target picture.

Fig. 37 is a diagram illustrating a hardware configuration example of an encoding/decoding device according to an embodiment of the present invention.

Fig. 38A is a diagram illustrating triangle fusion mode prediction.

Fig. 38B is a diagram illustrating triangle fusion mode prediction.

Fig. 39 is a flowchart illustrating an average merge candidate acquisition processing procedure.

Fig. 40 is a flowchart illustrating a process for obtaining a triangle merge candidate.

Fig. 41 is a flowchart illustrating the acquisition of triangle merge section 0 unidirectional prediction motion information according to the present embodiment.

Fig. 42 is a flowchart illustrating the acquisition of unidirectional prediction motion information for triangle merging section 1 in an embodiment of the present invention.

Fig. 43A is a diagram illustrating triangle merging mode weighting.

Fig. 43B is a diagram illustrating triangle fusion mode weighting.

Fig. 44A is a diagram illustrating partitions in a triangle fusion mode.

Fig. 44B is a diagram illustrating sections in a triangle fusion mode.

Fig. 44C is a diagram illustrating partitions in a triangle fusion mode.

Fig. 44D is a diagram illustrating partitions in a triangle fusion mode.

Fig. 44E is a diagram illustrating partitions in a triangle fusion mode.

Fig. 44F is a diagram illustrating sections in a triangle fusion mode.

Fig. 45A is a diagram illustrating stored information in a triangle fusion mode.

Fig. 45B is a diagram illustrating stored information in the triangle fusion mode.

Fig. 46A is a diagram illustrating stored information in the triangle fusion mode.

Fig. 46B is a diagram illustrating stored information in the triangle fusion mode.

Fig. 47A is a diagram illustrating stored information in the triangle fusion mode.

Fig. 47B is a diagram illustrating stored information in the triangle fusion mode.

Fig. 48A is a diagram illustrating stored information in the triangle fusion mode.

Fig. 48B is a diagram illustrating stored information in the triangle fusion mode.

Implementation of the invention

First of all, technology definitions and technical terms used in the present embodiment will be described.

wood block

In an embodiment, the target image of the encoding/decoding process (processing target picture) is equally divided into a given size. This block is defined as a tree block. While in FIG. 4, the tree block size is set to 128 × 128 samples, the tree block size is not limited to this, and any size can be set. The target tree block (corresponding to the encoding target in the encoding process and the decoding target in the decoding process) is switched in the raster scan order, that is, in order from left to right and top to bottom. The interior of each tree block can be further divided recursively. The coding/decoding block resulting from the recursive splitting of the tree block is defined as the coding block. The tree block and the coding block are collectively defined as a block. Performing an appropriate block split provides efficient coding. The tree block size may be a fixed value predetermined by the encoding device and the decoding device, or a configuration may be adopted in which the tree block size determined by the encoding device is transmitted to the decoding device. Here, the maximum tree block size is 128×128 samples and the minimum tree block size is 16×16 samples. The maximum coding block size is 64×64 samples and the minimum coding block size is 4×4 samples.

Prediction Mode

Switching is performed between intra-picture prediction (MODE_INTRA) for performing prediction from the processed image signal of the target image and inter-picture prediction (MODE_INTER) for performing prediction from the image signal of the processed image in blocks of coding target blocks.

The encoding process uses a processed image for an image obtained by decoding an encoded signal, an image signal, a tree block, a block, an encoding block, or the like. The decoding process uses the processed image for the decoded image, image signal, tree block, block, coding block, and the like.

The identification mode of intra-prediction (MODE_INTRA) and inter-prediction (MODE_INTER) is defined as a prediction mode (PredMode). As a value, the prediction mode (PredMode) has intra-frame prediction (MODE_INTRA) or inter-frame prediction (MODE_INTER).

Interframe prediction

In inter-picture prediction, in which prediction is performed from an image signal of a processed image, a plurality of processed images can be used as reference images. To manage a plurality of reference pictures, two types of look-up lists L0 (look-up list 0) and L1 (look-up list 1) are defined. In each of the lists, a reference image is indicated using a reference index. In the P-segment, L0 prediction (Pred_L0) can be used. In the B-segment, L0 prediction (Pred_L0), L1 prediction (Pred_L1), and bidirectional prediction (Pred_BI) can be used. L0 prediction (Pred_L0) is inter-picture prediction, which is related to the reference picture driven by L0, while L1 prediction (Pred_L1) is inter-picture prediction, which is related to the reference picture driven by L1. Bidirectional prediction (Pred_BI) is inter-picture prediction in which both L0 prediction and L1 prediction are performed and refers to one reference picture driven in each of L0 and L1. Information defining L0 prediction, L1 prediction, and bidirectional prediction is defined as an inter prediction mode. The following processing process assumes that processing will be performed for each of L0 and L1 for constants and variables in the LX suffix output.

Motion vector predictor mode

The motion vector predictor mode is an index transmission mode for specifying a motion vector predictor, a motion vector difference, an inter prediction mode, and a reference index, and determining target block inter prediction information. A motion vector predictor is obtained from a motion vector predictor candidate obtained from a processed block in a neighboring target block or a block belonging to the processed image and located at the same position or neighborhood (vicinity) of the target block, and from an index to indicate the motion vector predictor.

Merge mode

The merge mode is a mode that extracts target block inter prediction information from inter prediction information of a processed block in a neighboring block of the target block or a block belonging to the processed image and located at the same position as the target block or in the vicinity (vicinity) of the target block, without transmitting a motion vector difference or a reference index.

The processed block in the adjacent block of the target block and the inter prediction information of the processed block are determined as spatial merge candidates. Blocks belonging to the processed image and located at the same position as the target block, or in the vicinity (vicinity) of the target block, and inter prediction information obtained from the block inter prediction information are determined as temporal merge candidates. Each of the merger candidates is registered in the list of merger candidates. The merge candidate to be used to predict the target block is determined by the merge index.

neighboring block

Fig. 11 is a diagram illustrating reference blocks to be referred to for obtaining inter prediction information in the motion vector predictor mode and the merge mode. A0, A1, A2, B0, B1, B2, and B3 are processing blocks in the neighboring block of the target block. T0 is a block belonging to the image being processed and located at the same position as the target block, or in the neighborhood (vicinity) of the target block in the target image.

A1 and A2 are blocks located to the left of the target encoding block and next to the target encoding block. B1 and B3 are blocks located above the target encoding block and next to the target encoding block. A0, B0, and B2 are blocks located respectively at the bottom left, top right, and top left of the coding target block.

The following is a detailed description of the processing of adjacent blocks in the motion vector predictor mode and the merge mode.

Affine motion compensation

Affine motion compensation first partitions a coding block into subblocks of a given block, and then individually determines a motion vector for each of the partitioned subblocks to perform motion compensation. The motion vector of each of the sub-blocks is obtained based on one or more control points obtained from the inter-frame prediction information of the processed block in the neighboring block of the target block or a block belonging to the processed image and located at the same position as or in the neighborhood (vicinity) of the target block. While the present embodiment sets the sub-block size to 4×4 samples, the sub-block size is not limited to this, and a motion vector can be obtained in blocks of samples.

Fig. 14 illustrates an example of affine motion compensation in the case of two control points. In this case, each of the two control points has two parameters, i.e., a horizontal component and a vertical component. Accordingly, an affine transformation that has two control points is called a four-parameter affine transformation. In FIG. 14 CP1 and CP2 are control points.

Fig. 15 shows an example of affine motion compensation in the case of three control points. In this case, each of the three control points has two parameters, i.e., a horizontal component and a vertical component. Accordingly, an affine transformation that has three control points is called a six-parameter affine transformation. In FIG. 15 CP1, CP2 and CP3 are control points.

Affine motion compensation may be used in either of the motion vector predictor mode and the fusion mode. The affine motion compensation application mode in the motion vector predictor mode is defined as the sub-block motion vector predictor mode. The application mode of affine motion compensation in the merge mode is defined as the sub-block merge mode.

Code block syntax

Fig. 12A, FIG. 12B and FIG. 13 illustrate the syntax for expressing a coding block prediction mode. In FIG. 12A, pred_mode_flag is a flag indicating an inter prediction mode. Setting pred_mode_flag 0 indicates inter-prediction and setting pred_mode_flag 1 indicates intra-prediction. The intra prediction information intra_pred_mode is transmitted in the case of intra prediction, while the merge_flag is transmitted in the case of inter prediction. merge_flag is a flag indicating whether the mode used is a merge mode or a motion vector predictor mode. In the case of the motion vector predictor mode (merge_flag = 0), an inter_affine_flag flag is transmitted indicating whether to apply the subblock motion vector predictor mode. In case of applying the sub-block motion vector prediction mode (inter_affine_flag = 1), cu_affine_type_flag is transmitted. cu_affine_type_flag is a flag for specifying the number of breakpoints in subblock motion vector predictor mode.

On the contrary, as shown in FIG. 12B, in case of merge mode (merge_flag = 1), merge_subblock_flag is transmitted. merge_subblock_flag is a flag indicating whether to apply the subblock merge mode. In the case of the subblock merge mode (merge_subblock_flag = 1), the merge index merge_subblock_idx is passed. Conversely, in the case where the mode is not a subblock merge mode (merge_subblock_flag = 0), a merge_triangle_flag flag indicating whether to apply the triangle merge mode is transmitted. If the triangle merge mode is applied (merge_triangle_flag = 1), the merge_triangle_idx0 and merge_triangle_idx1 indices of the triangle merge are passed for each of the merge_triangle_split_dir block splitting directions and for each of the two split sections. If the triangle merge mode is not applied (merge_triangle_flag = 0), the merge_idx index of the merge is passed.

Fig. 13 illustrates the meaning of each of the syntax elements and the corresponding prediction mode. merge_flag = 0 and inter_affine_flag = 0 correspond to normal motion vector predictor mode (Inter Pred Mode). merge_flag = 0 and inter_affine_flag = 1 correspond to the subblock motion vector predictor mode (interaffine mode). merge_flag = 1, merge_subblock_flag = 0 and merge_trianlge_flag = 0 correspond to normal Merge Mode. merge_flag = 1, merge_subblock_flag = 0 and merge_trianlge_flag = 1 correspond to Triangle Merge Mode. merge_flag = 1, merge_subblock_flag = 1 correspond to the Affine Merge Mode.

POC

The Picture Order Count (POC) is a variable associated with the encoded picture and is set to a value that is incremented by one according to the output order of the pictures. The POC value allows you to distinguish whether the images are the same, to distinguish the sequential relationship of external images in the output order, or to get the distance between images. For example, it can be determined that two images having the same POC value are identical images. In the case where the POCs of two pictures have different values, the picture with the lower POC value can be determined as the picture that is output earlier. The difference between the POCs of two images indicates the distance between the images in the direction of the time axis.

First Embodiment

Next, description will be made of the image encoding apparatus 100 and the image decoding apparatus 200 according to the first embodiment of the present invention.

Fig. 1 is a block diagram of an apparatus 100 for encoding images according to the first embodiment. The image encoding apparatus 100 according to the embodiment includes a block separating unit 101, an inter prediction unit 102, an intra prediction unit 103, a decoded image memory 104, a prediction method determiner 105, a residual generating unit 106, an orthogonal transform/quantizer 107, an encoding unit 108 a bit string, an inverse quantizer/inverse orthogonal converter 109, a decoded image signal overlay unit 110, and an encoding information storage memory 111 .

The block splitter 101 recursively splits the input image to construct a coding block. The block dividing unit 101 includes: a quaternary dividing unit that divides a dividing target block in both the horizontal direction and the vertical direction; and a binary-ternary splitter that splits the target splitter in either the horizontal direction or the vertical direction. The block splitter 101 sets the constructed coding block as the target coding block, and supplies the image signal of the target coding block to the inter prediction section 102, the intra prediction section 103, and the residual generation section 106 . Further, the block splitter 101 supplies information indicative of the determined recursive split structure to the bit string encoding unit 108 . Next, detailed operation information of the block dividing unit 101 will be given.

The inter prediction block 102 performs inter prediction of the target coding block. The inter prediction unit 102 derives a plurality of inter prediction information candidates from the inter prediction information stored in the encoding information storage memory 111 and the decoded image signal stored in the decoded picture memory 104, selects an appropriate inter prediction mode from the plurality of obtained candidates, and supplies the selected mode prediction method and a predicted image signal corresponding to the selected inter prediction mode to the prediction method determiner 105 . Next, the detailed configuration and operation of the inter prediction block 102 will be given.

The intra prediction block 103 performs intra prediction for the target coding block. The intra prediction unit 103 refers to the decoded image signal stored in the decoded picture memory 104 as a reference sample, and performs intra prediction based on encoding information such as an intra prediction mode stored in the encoding information storage memory 111, and thereby generates a predicted image signal. In intra prediction, the intra prediction unit 103 selects a suitable intra prediction mode from a plurality of intra prediction modes, and supplies the selected intra prediction mode and the selected predicted image signal corresponding to the selected intra prediction mode to the prediction method determiner 105.

Fig. 10A and FIG. 10B illustrate examples of intra prediction. Fig. 10A illustrates a correspondence between an intra prediction prediction direction and an intra prediction mode number. For example, the intra prediction mode 50 copies the reference samples in the vertical direction and thereby generates an intra prediction image. Intra prediction mode 1 is a DC mode in which all target block sample values are set to the average of the reference samples. Intra prediction mode 0 is a planar mode in which a 2D intra prediction image is formed from reference samples in the vertical and horizontal directions. Fig. 10B is an example of constructing an intra prediction image in the case of the intra prediction mode 40 . The intra prediction block 103 copies, for each of the samples of the target block, the value of the reference sample in the direction indicated by the intra prediction mode. In the case where the reference sample in the intra prediction mode is not at an integer position, the intra prediction unit 103 determines the value of the reference sample by interpolating from the reference sample values at neighboring integer positions.

The decoded picture memory 104 stores decoded pictures generated by the decoded picture signal overlay unit 110 . The decoded picture memory 104 supplies the stored decoded pictures to the inter prediction unit 102 and the intra prediction unit 103 .

The prediction method determiner 105 evaluates each of intra prediction and inter prediction using encoding information, residual code amount, distortion amount between the predicted image signal and the target image signal, or the like, and thus determines the optimal prediction mode. In the case of intra prediction, the prediction method determiner 105 supplies intra prediction information such as an intra prediction mode to the bit string encoding unit 108 as encoding information. In the case of the inter prediction merge mode, the prediction method determiner 105 provides inter prediction information such as a merge index and information (sub-block merge flag) indicating whether the mode is a sub-block merge mode to the bit string encoding unit 108 as encoding information. In the case of an inter prediction motion vector predictor mode, the prediction method determiner 105 supplies inter prediction information such as an inter prediction mode, a motion vector predictor index, a reference index L0 or L1, a motion vector difference, or information indicating whether the mode is a subblock motion vector predictor mode. (sub-block motion vector predictor flag) to the bit string encoding block 108 as encoding information. The predictive method determiner 105 further supplies the determined encoding information to the encoding information storage memory 111 . The prediction method determiner 105 supplies the predicted image signal to the residual generation unit 106 and the decoded image signal overlay unit 110 .

The residual generation unit 106 generates a residual by subtracting the predicted image signal from the target image signal, and supplies the generated residual to the orthogonal converter/quantizer 107.

The orthogonal transform/quantizer 107 performs orthogonal transformation and quantization of the residual according to the quantization parameter, and thereby generates an orthogonally transformed and quantized residual, and then supplies the generated residual to the bit string encoding unit 108 and the inverse quantizer/inverse orthogonal transform 109.

The bit string coding unit 108 encodes, in addition to the sequences, images, segments, and information in coding block blocks, the bit string coding unit 108 encodes encoding information corresponding to the prediction method determined by the prediction method determiner 105 . for each of the coding blocks. In particular, the bit string encoding unit 108 encodes the PredMode prediction mode for each of the encoding units. In the case where the prediction mode is inter prediction (MODE_INTER), the bit string encoding section 108 encodes coding information (inter prediction information) such as a flag for determining whether the mode is a merge mode, a sub-block merge flag, a merge index in case of a merge mode , an inter-picture prediction mode in the case of non-merging modes, a motion vector predictor index, information related to the motion vector difference, and a sub-block motion vector predictor flag based on the prescribed syntax (by the bit string syntax rules), and thereby generates the first bit string. In the case where the prediction mode is intra prediction (MODE_INTRA), coding information (intra prediction information) such as the intra prediction mode is encoded according to a prescribed syntax (bit string syntax rules) to form a first bit string. Further, the bit string encoding section 108 performs entropy encoding of the orthogonally transformed and quantized residual based on the prescribed syntax, and thus generates the second bit string. The bit string encoding unit 108 multiplexes the first bit string and the second bit string based on the prescribed syntax, and outputs the bit stream.

The inverse quantizer/inverse orthogonal transformer 109 performs inverse quantization and inverse orthogonal transformation of the orthogonally transformed/quantized residual supplied from the orthogonal transformer/quantizer 107, and thus calculates the residual, and then supplies the calculated residual to the decoded image signal overlay unit 110.

The decoded image signal overlay section 110 superimposes the predicted image signal according to the determination of the residual prediction method determiner 105, which is subjected to inverse quantization/inverse orthogonal transform by the inverse quantizer/inverse orthogonal transform 109, thereby generating a decoded image, and stores the generated decoded picture in memory 104 decoded pictures. The decoded image signal overlay unit 110 may perform filtering processing to reduce distortion such as block distortion due to encoding of the decoded image, and may thereafter store the decoded image in the decoded image memory 104 .

The encoding information storage 111 stores encoding information such as a prediction mode (inter prediction or intra prediction) determined by the prediction method determiner 105 . In the case of inter prediction, encoding information is stored in the encoding information storage 111, which includes inter prediction information such as a determined motion vector, reference list indexes L0 and L1, and a history-based motion vector predictor candidate list. In the case of the inter prediction merge mode, encoding information stored in the encoding information storage memory 111 includes, in addition to the above-described information, a merge index and inter prediction information including information indicating whether the mode is a sub-block merge mode. (subblock merge flag). In the case of the inter prediction motion vector predictor mode, the encoding information stored in the encoding information storage memory 111 includes, in addition to the above information, inter prediction information such as the inter prediction mode, motion vector predictor index, motion vector difference, and information indicating whether the mode is a sub-block motion vector predictor mode (sub-block motion vector predictor flag). In the case of intra prediction, the encoding information stored in the encoding information storage memory 111 includes intra prediction information such as a specific intra prediction mode.

Fig. 2 is a block diagram illustrating the configuration of an image decoding apparatus according to an embodiment of the present invention corresponding to the image encoding apparatus of FIG. 1. The image decoding apparatus according to the embodiment includes a bit string decoding section 201, a block division section 202, an inter prediction section 203, an intra prediction section 204, an encoding information storage memory 205, an inverse quantizer/inverse orthogonal converter 206, and an overlay section 207. a decoded image signal and a decoded image memory 208 .

Since the decoding process of the image decoding device in FIG. 2 corresponds to the decoding process performed by the image encoding apparatus in FIG. 1. Accordingly, each of the configurations of the encoding information storage memory 205, the inverse quantizer/inverse orthogonal transformer 206, the decoded image signal overlay unit 207, and the decoded image memory 208 in FIG. 2 respectively have a function corresponding to each of the configurations of the encoding information storage memory 111, the inverse quantizer/inverse orthogonal converter 109, the decoded image signal overlay unit 110, and the decoded image memory 104 of the image encoding apparatus of FIG. one.

The bit stream supplied to the bit string decoder 201 is divided based on a prescribed syntax rule. The bit string decoding section 201 decodes the divided first bit string, and thus obtains a sequence, a picture, a segment, information in coding blocks, and coding information in coding blocks. Specifically, the bit string decoding section 201 decodes the prediction mode PredMode, which determines whether the prediction is inter-frame prediction (MODE_INTER) or intra-frame prediction (MODE_INTRA) in blocks of the coding block. In the case where the prediction mode is inter prediction (MODE_INTER), the bit string decoding section 201 decodes encoding information (inter prediction information) related to a flag that determines whether the mode is a merge mode, a merge index in the case of a merge mode, a sub-block merge flag and an inter prediction mode in the case of a motion vector predictor mode, a motion vector predictor index, a motion vector difference, a sub-block motion vector predictor flag, or the like. in accordance with the prescribed syntax, and then supplies encoding information (inter prediction information) to the encoding information storage memory 205 via the inter prediction section 203 and the block division section 202 . In the case where the prediction mode is intra prediction (MODE_INTRA), the bit string decoding section 201 decodes coding information (intra prediction information) such as the intra prediction mode according to a prescribed syntax, and then provides the decoded coding information (intra prediction information) to the memory 205 encoding information storage through block 203 inter prediction or block 204 intra prediction and through block 202 block division. The bit string decoding section 201 decodes the divided second bit string and calculates an orthogonally transformed/quantized residual, and then supplies the orthogonally transformed/quantized residual to the inverse quantizer/inverse orthogonal transform 206.

When the prediction mode PredMode of the target encoding block is inter prediction (MODE_INTER) and motion vector prediction mode, the inter prediction block 203 uses the encoding information of the already decoded image signal stored in the encoding information storage memory 205 to obtain a plurality of motion vector predictor candidates. The inter prediction block 203 then registers the plurality of received motion vector predictor candidates into a motion vector predictor candidate list described below. The inter prediction section 203 selects a motion vector predictor corresponding to the motion vector predictor index to be decoded and supplied by the bit string decoder 201 from the plurality of motion vector predictor candidates registered in the motion vector predictor candidate list. The inter prediction block 203 then calculates a motion vector based on the difference of the motion vectors decoded by the bit string decoding block 201 and the selected motion vector predictor, and stores the calculated motion vector in the encoding information storage memory 205 along with other encoding information. Here, the encoding information of a coding block to be set and stored includes a prediction mode PredMode, flags predFlagL0[xP][yP] and predFlagL1[xP][yP] indicating whether or not to use L0 prediction and L1 prediction, reference indices refIdxL0 [xP] [yP] and refIdxL1 [xP] [yP] L0 and L1; and vectors mvL0 [xP] [yP] and mvL1 [xP] [yP] of motion for L0 and L1, or the like. Here, xP and yP are indices indicating the position of the top left coding block sample in the picture. In the case where the prediction mode PredMode is inter prediction (MODE_INTER) and the inter prediction mode is L0 prediction (Pred_L0), a flag predFlagL0 indicating whether to use L0 prediction is set to 1, and a flag predFlagL1 indicating whether to use L1 prediction is set. to 0. In the case where the inter prediction mode is L1 prediction (Pred_L1), the flag predFlagL0 indicating whether to use L0 prediction is set to 0, and the flag predFlagL1 indicating whether to use L1 prediction is set to 1. In the case when the inter prediction mode is bidirectional prediction (Pred_BI), both the predFlagL0 flag indicating whether to use L0 prediction and the flag predFlagL1 indicating whether to use L1 prediction are set to 1. Further, when the prediction mode PredMode of the coding target block is the inter prediction mode prediction (MODE_INTER) and merge mode i, then it turns out the merge candidate. Using the encoding information of the already decoded encoding block stored in the encoding information storage memory 205, a plurality of merge candidates is output and registered in the merge candidate list described below. Then, a merge candidate corresponding to the merge index that is decoded by the bit string decoder 201 and provided is selected from a plurality of merge candidates registered in the merge candidate list, and then inter prediction information such as flags predFlagL0 [xP ] [yP] and predFlagL1 [xP ] [yP] indicating whether to use L0 prediction and L1 prediction of the selected merge candidate, reference indices refIdxL0 [xP] [yP] and refIdxL1 [xP] [yP] L0 and L1, and vectors mvL0 [xP] [yP] and mvL1 [xP] [yP] movements for L0 and L1 must be stored in the encoding information storage memory 205 . Here, xP and yP are indices indicating the position of the top left coding block sample in the picture. Next, a detailed description of the configuration and operation of the inter prediction block 203 will be given.

The intra prediction unit 204 performs intra prediction when the prediction mode PredMode of the target coding block is intra prediction (MODE_INTRA). The encoding information decoded by the bit string decoding section 201 includes an intra prediction mode. The intra prediction unit 204 generates a predicted image signal by intra prediction from a decoded image signal stored in the decoded image memory 208 according to the intra prediction mode in the encoding information decoded by the bit string decoding unit 201 . Next, the intra prediction block 204 supplies the generated predicted image signal to the decoded image signal overlay block 207 . The intra prediction unit 204 corresponds to the intra prediction unit 103 of the image encoding apparatus 100, and thus performs processing similar to that of the intra prediction unit 103 .

The inverse quantizer/inverse orthogonal transform 206 performs inverse orthogonal transformation/inverse quantization of the orthogonally transformed/quantized residual decoded in the bit string decoding section 201, and thus obtains the inversely orthogonally transformed/inversely quantized residual.

The decoded image signal overlay unit 207 superimposes the image signal predicted by inter prediction by the inter prediction unit 203 or the predictive image signal predicted by intra prediction by the intra prediction unit 204 with a residual that has been inversely orthogonally transformed/inversely quantized by an inverse quantizer /inverse orthogonal converter 206, thereby decoding the decoded image signal. The decoded picture signal overlay unit 207 then stores the decoded picture signal that has been decoded in the decoded picture memory 208 . When storing the decoded image in the decoded image memory 208, the decoded image signal overlay section 207 may perform filtering processing on the decoded image to reduce block distortion or the like due to encoding, and may thereafter store the decoded image in the decoded image memory 208.

Next, the operation of the block dividing unit 101 in the image encoding apparatus 100 will be described. Fig. 3 is a flowchart illustrating the operation of dividing an image into tree blocks and further dividing each of the tree blocks. First, the input image is divided into tree blocks of a given size (step S1001). Each of the tree blocks is scanned in a predetermined order, that is, in raster scan order (step S1002), and the target tree block is internally divided (step S1003).

Fig. 7 is a flowchart illustrating the detailed operation of the division process in step S1003. First, it is determined whether to divide the target block into four (step S1101).

In the case where it is determined that the target block is to be divided by four, the target block will be divided by four (step S1102). Each of the blocks obtained by dividing the target block is scanned in a Z-scan order, that is, in the order of top left, top right, bottom left, and bottom right (step S1103). Fig. 5 illustrates an example of a Z-scan order, 601 in FIG. 6A illustrates an example in which the target block is divided into four. Numbers 0 to 3 out of 601 in FIG. 6A indicate the processing order. Then, the splitting process of FIG. 7 for each of the blocks divided in step S1101 (step S1104).

In the case where it is determined that the target block is not to be divided into four, the target block will be divided into two or three, namely, binary-ternary division will be performed (step S1105).

Fig. 8 is a flowchart illustrating in detail the operation of the binary-ternary separation process in step S1105. First, it is determined whether binary-ternary separation will be performed for the target block, that is, whether any binary or ternary separation will be performed (step S1201).

In the case where it is not determined that binary-ternary separation is to be performed on the target block, that is, in the case where it is determined that the target block is not to be divided, the separation is completed (step S1211). That is, an additional recursive division process need not be performed on a block that has been divided by the recursive division process.

In the case where it is determined that binary-ternary division will be performed in the target block, it is further determined whether or not to divide the target block into two (step S1202).

In the case where it is determined that the target block is to be divided into two, it is further determined whether or not to split the target block in the top-bottom (vertical) direction (step S1203), and then, based on the result, the target block will be binary-divided in the top-bottom ( vertical) direction (step S1204), or the target block will be binary divided in the left-right (horizontal) direction (step S1205). As a result of step S1204, the target block is binary split in the up-down direction (vertical direction), as illustrated at 602 in FIG. 6b. As a result of step S1205, the target block is left-right binary split (horizontal direction), as illustrated in step 604 in FIG. 6d.

In step S1202, in the case where it is not determined that the target block is to be divided into two, that is, in the case where it is determined that the target block is to be divided into three, it is further determined whether the target block is to be divided into three parts: top, middle and bottom (vertical direction) (step S1206). Based on the result, the target block is divided into three parts: top, middle and bottom parts (vertical direction) (step S1207) or left, middle and right parts (horizontal direction) (step S1208). As a result of step S1207, the target block is divided into three parts: top, middle, and bottom (vertical direction), as illustrated in step 603 in FIG. 6C. As a result of step S1208, the target block is divided into three parts: left, middle, and right (horizontal direction), as illustrated in step 605 in FIG. 6E.

After performing one of steps S1204, S1205, S1207 or S1208, each of the blocks obtained by dividing the target block is scanned in order from left to right and top to bottom (step S1209). Numerals 0 to 2 602 to 605 in FIG. 6B-6E indicate the processing order. For each of the partition blocks, the binary-ternary partitioning process in FIG. 8 (step S1210).

In the recursive block division described herein, the correctness of the division may be limited based on the number of divisions, the size of the target block, and the like. The information that limits the correctness of the division can be provided in a configuration in which information is not transmitted by prior agreement between the encoder and the decoder, or in a configuration in which the encoder determines information for limiting the correctness of the division and writes the information in bit strings, thereby transmitting the information to the device for decoding.

When a particular block is split, the block before the split is called the parent box and each of the boxes after the split is called a child box.

Next, the functionality of the block splitter 202 in the device 200 for decoding an image will be described. The block splitter 202 splits the tree block using a processing procedure similar to the case of the block splitter 101 of the image encoding device 100 . Note that there is a difference, although the block separating unit 101 of the image encoding apparatus 100 determines the optimal shape of the block division by applying an optimization method such as optimal shape estimation by image recognition or distortion rate optimization, the block separating unit 202 of the image decoding apparatus 200 determines a block division form by decoding the block division information recorded in the bit string.

Fig. 9 illustrates the syntax (bit string syntax rules) related to block division according to the first embodiment. coding_quadtree() represents the syntax for the process of quadruple block splitting. multi_type_tree() represents the syntax for splitting a block into two or three. qt_split is a flag indicating whether the block should be split into four. In the case of a block split by four, the setting is qt_split = 1. In the case of no block split by four, the setting will be qt_split = 0. In the case of a block split by four (qt_split = 1), the split by four process will be performed recursively for each of blocks divided into four (coding_quadtree (0), coding_quadtree (1), coding_quadtree (2) and coding_quadtree (3), in which the arguments from 0 to 3 correspond to the numbers 601 in Fig. 6A). In the case where split by four is not to be performed (qt_split = 0), the subsequent split is determined according to multi_type_tree(). mtt_split is a flag indicating whether additional splitting should be performed. In the case where additional splitting is needed (mtt_split = 1), mtt_split_vertical, which is a flag indicating whether splitting should be performed in the vertical or horizontal direction, and mtt_split_binary, which is a flag specifying whether the block should be split into two, will be transmitted. or three. mtt_split_vertical = 1 indicates a split in the vertical direction and mtt_split_vertical = 0 indicates a split in the horizontal direction. mtt_split_binary = 1 indicates that the block has undergone a binary split, and mtt_split_binary = 0 indicates that the block has been split into three. In the case where a block is to be split by binary code (mtt_split_binary = 1), the splitting process is performed recursively for each of the two split blocks (multi_type_tree (0) and multi_type_tree (1), in which the arguments from 0 to 1 correspond to numbers in 602 or 604 in Fig. 6B-6D). In the case where a block is to be split into three (mtt_split_binary = 0), the splitting process is performed recursively for each of the three split blocks (multi_type_tree (0), multi_type_tree (1) and multi_type_tree (2), in which 0 2 correspond to numbers 603 on Fig. 6B or 605 in Fig. 6E). multi_type_tree is recursively used until mtt_split = 0 results in a hierarchical block split.

Interframe prediction

The inter prediction method according to the embodiment is implemented in the inter prediction unit 102 of the image encoding apparatus in FIG. 1 and in the inter prediction block 203 of the image decoding apparatus in FIG. 2.

Next, the inter prediction method according to the embodiment will be described with reference to the drawings. The inter-frame prediction method is implemented in any of the encoding and decoding processes in coding blocks.

Block 102 inter-picture prediction on the side of the coding

Fig. 16 is a diagram illustrating a detailed configuration of the inter frame prediction unit 102 of the image encoding apparatus in FIG. 1. The normal motion vector predictor mode obtaining unit 301 outputs a plurality of normal motion vector predictor candidates, selects a motion vector predictor, and calculates a motion vector difference between the selected motion vector predictor and the detected motion vector. The detected inter prediction mode, the reference index, the motion vector, and the calculated motion vector difference are the inter prediction information of the normal motion vector predictor mode. This inter prediction information is supplied to the inter prediction mode determiner 305 . Next, detailed configuration and processing information of the normal motion vector predictor mode acquisition unit 301 will be given.

The normal merge mode obtaining unit 302 outputs a plurality of normal merge candidates, selects a normal merge candidate, and obtains inter prediction information for the normal merge mode. This inter prediction information is supplied to the inter prediction mode determiner 305 . Next, detailed information on the configuration and processing of the normal merge mode acquisition block 302 will be given.

The sub-block motion vector predictor mode obtaining unit 303 outputs a plurality of sub-block motion vector predictor candidates, selects a sub-block motion vector predictor, and calculates a motion vector difference between the selected sub-block motion vector predictor and the detected motion vector. The detected inter prediction mode, reference index, motion vector, and calculated motion vector difference are sub-block motion vector predictor mode inter prediction information. This inter prediction information is supplied to the inter prediction mode determiner 305 .

The sub-block merge mode obtaining unit 304 outputs a plurality of sub-block merge candidates, selects a sub-block merge candidate, and obtains sub-block merge mode inter-picture prediction information. This inter prediction information is supplied to the inter prediction mode determiner 305 .

In the inter prediction mode determiner 305, inter prediction information is determined based on the inter prediction information supplied from the normal motion vector predictor mode obtaining block 301, the normal merge mode obtaining block 302, the subblock motion vector predictor mode obtaining block 303, and the subblock merge mode obtaining block 304. The inter prediction information according to the determination result is supplied from the inter prediction mode determiner 305 to the motion compensation predictor 306 .

The motion compensation predictor 306 performs inter prediction on the reference picture signal stored in the decoded picture memory 104 based on the determined inter prediction information. Next, detailed configuration and processing information of the motion compensation predictor 306 will be given.

Block 203 inter-picture prediction on the side of the decoding

Fig. 22 is a diagram illustrating a detailed configuration of the inter prediction unit 203 of the image decoding apparatus in FIG. 2.

The normal motion vector predictor mode obtaining unit 401 outputs a plurality of normal motion vector predictor candidates, selects a motion vector predictor, calculates an added value obtained by adding the selected motion vector predictor and the difference of the decoded motion vectors, and sets this added value as a motion vector. The decoded inter prediction mode, reference index, motion vector is the inter prediction information of the normal motion vector predictor mode. This inter prediction information is supplied to the motion compensation predictor 406 via the switch 408. Next, the configuration and processing of the normal motion vector predictor mode obtaining unit 401 will be described in detail.

The normal merge mode obtaining unit 402 outputs a plurality of normal merge candidates, selects a normal merge candidate, and obtains inter prediction information for the normal merge mode. This inter prediction information is supplied to the motion compensation prediction block 406 via the switch 408. The detailed configuration and processing of the normal fusion mode obtaining block 402 will be presented below.

The sub-block motion vector predictor mode obtaining unit 403 outputs a plurality of sub-block motion vector predictor candidates, selects a sub-block motion vector predictor, and calculates an added value obtained by adding the selected sub-block motion vector predictor and the decoded motion vector difference, and sets this added value as a motion vector. The decoded inter prediction mode, the reference index, and the motion vector are inter prediction information of the sub-block motion vector predictor mode. This inter prediction information is supplied to the motion compensation predictor 406 via the switch 408.

The sub-block merge mode obtaining unit 404 outputs a plurality of sub-block merge candidates, selects a sub-block merge candidate, and acquires sub-block merge mode inter-picture prediction information. This inter prediction information is supplied to the motion compensation predictor 406 via the switch 408.

The motion compensation predictor 406 performs inter prediction on the reference picture signal stored in the decoded picture memory 208 based on the determined inter prediction information. The detailed configuration and processing of the motion compensation predictor 406 is similar to the motion compensation predictor 306 on the encoding side.

Normal Motion Vector Predictor Mode (Normal AMVP) Obtaining Block

The normal motion vector predictor mode acquisition unit 301 in FIG. 17 includes a spatial motion vector predictor candidate deriving block 321, a temporal motion vector predictor candidate deriving block 322, a history-based motion vector predictor candidate deriving block 323, a motion vector predictor candidate replenisher 325, a normal motion vector determiner 326, a predictor candidate selector 327 motion vector subtractor 328 of the motion vector.

[0The normal motion vector predictor mode acquisition block 401 in FIG. 23 includes a spatial motion vector predictor candidate derivation unit 421, a temporal motion vector predictor candidate derivation unit 422, a history-based motion vector predictor candidate derivation unit 423, a motion vector predictor candidate adder 425, a motion vector predictor candidate selector 426, and a vector adder 427 movement.

With reference to the flowcharts in FIG. 19 and FIG. 25, respectively, the processing procedures of the normal motion vector predictor mode acquisition unit 301 on the encoding side and the normal motion vector predictor mode acquisition unit 401 on the decoding side are described. Fig. 19 is a flowchart illustrating a normal motion vector predictor mode acquisition processing procedure executed by the normal motion vector mode acquisition unit 301 on the encoding side. Fig. 25 is a flowchart illustrating a normal motion vector predictor mode acquisition processing procedure executed by the normal motion vector mode acquisition unit 401 on the decoding side.

Normal Motion Vector Predictor Mode (Normal AMVP) Obtaining Block: Encoding Side

With reference to FIG. 19, a normal motion vector predictor mode acquisition processing procedure on the encoding side will be described. In the description of the processing procedure in FIG. 19, the word "normal" illustrated in FIG. 19 is omitted in some cases.

First, the normal motion vector determiner 326 detects the normal motion vector for each of the inter prediction modes and reference indexes (step S100 in FIG. 19).

Subsequently, for each of L0 and L1, the motion vector difference of the motion vector used in inter-picture prediction in the normal motion vector predictor mode is calculated (steps S101 to S106 in FIG. 19) in the spatial motion vector predictor candidate acquisition block 321, the candidate acquisition block 322 temporal motion vector predictor, a history-based motion vector predictor candidate derivation block 323 , a motion vector predictor candidate adder 325 , a motion vector predictor candidate selector 327 , and a motion vector subtractor 328 . Specifically, in the case where the target block's PredMode is inter prediction (MODE_INTER) and the inter prediction mode is L0 prediction (Pred_L0), the L0 motion vector predictor candidate list mvpListL0 is calculated. Subsequently, the motion vector predictor mvpL0 is selected and then the motion vector difference mvdL0 of the motion vector mvL0 L0 is calculated. In the case where the inter prediction mode of the target block is L1 prediction (Pred_L1), the L1 motion vector predictor candidate list mvpListL1 is calculated. Then the predictor mvpL1 of the motion vector is selected, and then the difference mvdL1 of the motion vectors of the motion vector mvL1 of the motion L1 is calculated. In the case where the inter prediction mode of the target block is bidirectional prediction (Pred_BI), both L0 prediction and L1 prediction are performed. The motion vector predictor candidate list mvpListL0 for L0 is calculated, and the motion vector predictor mvpL0 for L0 is selected, and then the motion vector difference mvdL0 of the motion vector mvL0 for L0 is calculated. Along with this calculation, a motion vector predictor candidate list mvpListL1 for L1 is calculated, and a motion vector predictor mvpL1 for L1 is calculated, and then the motion vector difference mvdL1 of the motion vector mvL1 for L1 is calculated.

The motion vector difference calculation process is performed for each of L0 and L1, in which the calculation process is a common process for both L0 and L1. Accordingly, L0 and L1 will be referred to as LX as a general procedure. In the process of calculating the motion vector difference, L0, X, LX is set to 0, while in the process of calculating the motion vector difference, L1, X, LX is set to 1. the difference of the motion vectors of one LX, the other list will be represented as LY.

In the case where the LX motion vector mvLX is used (step S102 in FIG. 19: YES), LX motion vector predictor candidates are calculated, thereby compiling the LX motion vector predictor candidate list mvpListLX (step S103 in FIG. 19). In the normal motion vector predictor mode derivation block 301, the spatial motion vector predictor candidate derivation block 321, the temporal motion vector predictor candidate derivation block 322, the history-based motion vector predictor candidate derivation block 323, and the motion vector predictor candidate filler 325, a plurality of motion vector predictor candidates are output. motion vectors and thus constitute a list mvpListLX of motion vector predictor candidates. Next, with reference to FIG. 19, a detailed processing procedure of step S103 will be described using the flowchart in FIG. twenty.

Next, the motion vector predictor candidate selector 327 selects the LX motion vector predictor mvpLX of the LX motion vector predictor candidate list mvpListLX (step S104 in FIG. 19). Here, one element (i-th element, starting from 0) in the motion vector predictor candidate list mvpListLX is represented as mvpListLX[i]. Each motion vector difference is calculated, which is the difference between the mvLX motion vector and each of the motion vector predictor candidates mvpListLX[i] stored in the motion vector predictor candidate list mvpListLX. For each of the elements (motion vector predictor candidates) of the motion vector predictor candidate list mvpListLX, the amount of code is calculated at the time of encoding these motion vector differences. Then, a motion vector predictor candidate mvpListLX[i] is selected that minimizes the amount of code for each of the motion vector predictor candidates among the individual elements registered in the motion vector predictor candidate list mvpListLX as the motion vector predictor mvpLX, and its index i is obtained. In the case of a plurality of motion vector predictor candidates having a minimum generated amount of code in the motion vector predictor candidate list mvpListLX, the motion vector predictor candidate mvpListLX [i] having index i in the mvpListLX motion vector predictor candidate list represented by a small number is selected as the optimal predictor mvpLX of the motion vector, and its index i is obtained.

Then, the motion vector subtractor 328 subtracts the selected motion vector LX predictor mvpLX from the motion vector LX mvLX and calculates the difference of the LX motion vectors mvdLX as in: mvdLX = mvLX-mvpLX (step S105 in FIG. 19).

Normal motion vector predictor mode acquisition block (normal AMVP): decode side

Next, with reference to FIG. 25, a normal motion vector predictor mode processing procedure on the decoding side will be described. On the decoding side, a spatial motion vector predictor candidate deriving unit 421, a temporal motion vector predictor candidate deriving unit 422, a history-based motion vector predictor candidate deriving unit 423, and a motion vector predictor candidate enhancer 425 individually calculate the motion vectors used in inter-frame prediction of the normal predictor mode. a motion vector for each of L0 and L1 (steps S201 to S206 in FIG. 25). Specifically, in the case where the prediction mode PredMode of the target block is inter prediction (MODE_INTER) and the inter prediction mode of the target block is L0 prediction (Pred_L0), the L0 motion vector predictor candidate list mvpListL0 is calculated. Next, a predictor of the motion vector mvpL0 is selected, and then the motion vector mvL0 of L0 is calculated. In the case where the inter prediction mode of the target block is L1 prediction (Pred_L1), the L1 motion vector predictor candidate list mvpListL1 is calculated. Then, the motion vector mvpL1 predictor is selected and the motion vector mvL1 L1 is calculated. In the case where the inter prediction mode of the target block is bidirectional prediction (Pred_BI), both L0 prediction and L1 prediction are performed. A motion vector predictor candidate list mvpListL0 of L0 is computed, and a motion vector predictor mvpL0 of L0 is selected, and then a motion vector mvL0 of L0 is computed. Along with this calculation, the motion vector predictor candidate list mvpListL1 for L1 is calculated, and the motion vector predictor mvpL1 for L1 is calculated, and then the motion vector mvL1 for L1 is calculated.

Similar to the encoding side, the decoding side performs motion vector calculation processing for each of L0 and L1, the processing being a common process for both L0 and L1. Accordingly, L0 and L1 will be referred to as LX as a general procedure. LX represents an inter prediction mode used for inter prediction of a target coding block. X is 0 in the process of calculating the motion vector L0 and X is 1 in the process of calculating the motion vector L1. Additionally, in the case where other lookup list information is mentioned instead of the same lookup list as LX to be calculated in the LX motion vector calculation process, the other lookup list will be represented as LY.

In the case where motion vector mvLX for LX is used (step S202 in FIG. 25: YES), motion vector predictor candidates LX are computed to compile a motion vector predictor candidate list mvpListLX for LX (step S203 in FIG. 25). In the normal motion vector predictor mode derivation block 401, the spatial motion vector predictor candidate derivation block 421, the temporal motion vector predictor candidate derivation block 422, the history-based motion vector predictor candidate derivation block 423, and the motion vector predictor candidate filler 425, a plurality of vector predictor candidates are calculated. motion vectors and thus constitute a list mvpListLX of motion vector predictor candidates. Next, with reference to FIG. 25 and using the flowchart of FIG. 20, a detailed processing procedure of step S203 will be described.

Then, the motion vector predictor candidate selector 426 extracts the motion vector predictor candidate mvpListLX [mvpIdxLX] corresponding to the motion vector predictor index mvpIdxLX decoded and supplied by the bitstring decoder 201 from the motion vector predictor candidate list mvpListLX as the selected motion vector predictor mvpLX (step S204 at Fig. 25).

Then, the motion vector adder 427 adds the difference mvdLX of the LX motion vectors decoded and delivered by the bit string decoder 201 and the LX motion vector predictor mvpLX, and calculates the LX motion vector mvLX as in: + mvdLX (step S205 in FIG. 25).

Normal Motion Vector Predictor Mode Obtainer (Normal AMVP): Motion Vector Prediction Method

Fig. 20 is a flowchart illustrating a processing procedure for obtaining a normal motion vector predictor mode having a function common to the normal motion vector predictor mode obtaining unit 301 of the image encoding apparatus and the normal motion vector predictor mode obtaining unit 401 of the image decoding apparatus according to the embodiment implementation of the present invention.

Each of the normal motion vector predictor mode obtaining block 301 and the normal motion vector predictor mode obtaining block 401 includes a motion vector predictor candidate list mvpListLX. The motion vector predictor candidate list mvpListLX has a list structure and includes a storage area that stores as elements a motion vector predictor index indicating a location in the motion vector predictor candidate list and a motion vector predictor candidate corresponding to the index. The motion vector predictor index number starts from 0, and the motion vector predictor candidates shall be stored in the motion vector predictor candidate list mvpListLX storage area. In the present embodiment, it is assumed that the motion vector predictor candidate list mvpListLX can register at least two motion vector predictor candidates (as inter prediction information). In addition, the numCurrMvpCand variable indicating the number of motion vector predictor candidates registered in the motion vector predictor candidate list mvpListLX is set to 0.

Each of the spatial motion predictor candidate derivation blocks 321 and 421 derives a motion vector predictor candidate from blocks in the left side adjacent. This process outputs a motion vector predictor mvLXA with reference to the inter frame prediction information of the block in the left side adjacent (A0 or A1 in FIG. 11 ), namely a flag indicating whether a motion vector predictor candidate, a motion vector, a reference index, or the like can be used, and adds the obtained mvLXA to the motion vector predictor candidate list mvpListLX (step S301 in FIG. 20). Note that X is 0 in L0 prediction and X is 1 in L1 prediction (similarly applies later). Next, motion vector predictor candidate derivation blocks 321 and 421 extract motion vector predictor candidates from the top neighbor block. This process outputs the mvLXB motion vector predictor with reference to the upper neighbor block inter prediction information (B0, B1, or B2 in FIG. 11), namely, a flag indicating whether the motion vector predictor candidate, the motion vector, the reference index, etc., can be used. .P. When the received mvLXA and the received mvLXB are not equal, the mvLXB is added to the motion vector predictor candidate list mvpListLX (step S302 in FIG. 20). The processes in steps S301 and S302 in FIG. 20 are presented as a general process, except that the positions and numbers of reference adjacent blocks are different, and the flag availableFlagLXN indicating whether a coding block motion vector predictor candidate can be used, and motion vector mvLXN, reference index refIdxN (N indicates A or B, similarly applied later) in these processes.

Then, each of the motion vector temporal predictor candidate derivation blocks 322 and 422 extracts a motion vector predictor candidate from a block in the picture having a temporal difference from the target picture. This process outputs an availableFlagLXCol flag indicating whether the motion vector predictor candidate of the image coding block having the time difference and the motion vector mvLXCol, the reference index refIdxCol and the lookup list listCol can be used, and adds the mvLXCol to the motion vector predictor candidate list mvpListLX (step S303 in Fig. 20).

It should be noted that it is contemplated that the processes of temporal motion vector predictor candidate derivation blocks 322 and 422 may be omitted in Sequence (SPS), Picture (PPS), or Segment blocks.

Then, the history-based motion vector predictor candidate derivers 323 and 423 add the history-based motion vector predictor candidates registered in the history-based motion vector predictor candidate list HmvpCandList to the motion vector predictor candidate list mvpListLX. (Step S304 in FIG. 20). With reference to the flowchart in FIG. 29, next, a detailed description of the registration processing procedure in step S304 will be given.

Then, the motion vector predictor candidate fillers 325 and 425 add a motion vector predictor candidate having a predetermined value such as (0, 0) until the motion vector predictor candidate list mvpListLX is satisfied (S305 in FIG. 20).

Block receiving normal merge mode (normal merge)

Normal merge mode acquisition block 302 in FIG. 18 includes a spatial merge candidate derivation unit 341, a temporal merge candidate derivation unit 342, an average merge candidate derivation unit 344, a history-based merge candidate derivation unit 345, a merge candidate replenisher 346, and a merge candidate selector 347.

Normal merge mode acquisition block 402 in FIG. 24 includes a spatial merge candidate derivation unit 441, a temporal merge candidate derivation unit 442, an average merge candidate derivation unit 444, a history-based merge candidate derivation unit 445, and a merge candidate augmenter 446 and a merge candidate selector 447.

Fig. 21 is a flowchart illustrating a procedure for obtaining a normal fusion mode having a function common to the normal fusion mode obtaining unit 302 of the image encoding apparatus and the normal fusion mode obtaining unit 402 of the image decoding apparatus according to an embodiment of the present invention.

Later in this document, the various processes will be described step by step. The following description is for the case where the slice_type of the segment is segment B, unless otherwise noted. However, the present invention can also be applied to the case of a P segment. Note that there is only L0 prediction (Pred_L0) as the inter prediction mode, L1 prediction (Pred_L1) or bidirectional prediction (Pred_BI) in the case where the slice_type of the segment is a P-segment. Accordingly, in this case, the process associated with L1 can be skipped.

The normal merge mode getter 302 and the normal merge mode getter 402 include a list of mergeCandList merge candidates. The mergeCandList has a list structure and includes a storage area that stores as elements a merge index indicating a location in the merge candidate list and a merge candidate corresponding to that index. The merge index number starts at 0 and the merge candidate is stored in the merge candidate list mergeCandList storage area. In post-processing, the merge candidate of merge index i registered in the merge candidate list mergeCandList will be represented by mergeCandList[i]. In the present embodiment, it is assumed that the merge candidate list mergeCandList can register at least six merge candidates (as inter prediction information). In addition, the numCurrMergeCand variable indicating the number of merge candidates registered in the mergeCandList of merge candidates is set to 0.

The spatial merge candidate derivation unit 341 and the spatial merge candidate derivation unit 441 output a spatial merge candidate of each of the blocks (B1, A1, B0, A0, B2 in FIG. 11) in the neighboring block of the target block in the order of B1, A1, B0, A0, and B2 from encoding information stored in either the encoding information storage memory 111 of the image encoding apparatus or the encoding information storage memory 205 of the image decoding apparatus, and then registers the obtained spatial merge candidates into the mergeCandList merge candidate list (step S401 in FIG. 21) . Here, N will be defined denoting one of B1, A1, B0, A0, B2, or a temporary merge candidate Col. Items to be output include an availableFlagN flag indicating whether inter-frame prediction information of block N can be used as a spatial merge candidate, reference index refIdxL0N L0 and reference index refIdxL1N L1 of spatial merge candidate N, prediction flag predFlagL0N L0 indicating whether L0 prediction should be performed, a prediction flag predFlagL1N L1 indicating whether L1 prediction should be performed, an L0 motion vector mvL0N, and an L1 motion vector mvL1N. However, since the merge candidate in the present embodiment is output without referring to the inter prediction information of the block included in the encoding target block, a spatial merge candidate using the inter prediction information of the block included in the encoding target block cannot be obtained.

Then, the temporary merge candidate obtaining unit 342 and the temporary merge candidate obtaining unit 442 output the temporary merge candidates from the images having the time difference, and register the obtained temporary merge candidates in the merge candidate list mergeCandList (step S402 in FIG. 21). The elements to be received include an availableFlagCol flag indicating whether a temporary merge candidate can be used, a predFlagL0Col L0 prediction flag indicating whether an L0 temporary merge candidate prediction should be performed, a L1 prediction flag predFlagL1Col indicating whether L1 should be performed. prediction, L0 motion vector mvL0Col and L1 motion vector mvL1Col.

Note that it is contemplated that the processes of temporally merge candidate derivation blocks 342 and 442 may be omitted in Sequence (SPS), Picture (PPS), or Segment blocks.

Next, the history-based merge candidate acquisition unit 345 and the history-based merge candidate acquisition unit 445 register the history-based motion vector predictor candidates registered in the history-based motion vector predictor candidate list HmvpCandList in the merge candidate list mergeCandList (step S403 in FIG. .21).

In the case where the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList is less than the maximum number of merge candidates MaxNumMergeCand, obtain a merge candidate based on history with the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList limited by the maximum number of merge candidates MaxNumMergeCand and then registered in the mergeCandList of merge candidates.

Then, the average merge candidate derivation unit 344 and the average merge candidate acquisition unit 444 output the average merge candidate from the mergeCandList of merge candidates, and add the resulting average merge candidate to the mergeCandList of merge candidates (step S404 in FIG. 21).

In the case where the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList is less than the maximum number of merge candidates MaxNumMergeCand, an average merge candidate is obtained with the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList which is limited to the maximum number of merge candidates MaxNumMergeCand and then register in the list mergeCandList merge candidates.

Here, the average merge candidate is a new merge candidate including a motion vector obtained by averaging the motion vectors of the first merge candidate and the second merge candidate registered in the merge candidate list mergeCandList for each of prediction L0 and prediction L1.

Subsequently, in merge candidate augmenter 346 and merge candidate augmenter 446, in the case where the number of numCurrMergeCand merge candidates registered in the merge candidate list mergeCandList is less than the maximum number of merge candidates MaxNumMergeCand, an additional merge candidate with the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList limited by the maximum number of merge candidates MaxNumMergeCand and then registered in the merge candidate list mergeCandList (step S405 in FIG. 21). In the P-segment, a merge candidate having a motion vector of value (0, 0) and a prediction prediction mode L0 (Pred_L0) is added with the maximum number of merge candidates MaxNumMergeCand as an upper limit. In the B-segment, a merge candidate having a bidirectional prediction mode (Pred_BI) and a motion vector of value (0, 0) is added. The pivot index at the time the merge candidate was added is different from the pivot index that was already added.

Subsequently, the merge candidate selector 347 and the merge candidate selector 447 select a merge candidate from among the merge candidates registered in the merge candidate list mergeCandList. The coding-side merge candidate selector 347 calculates a code amount and a distortion amount, and thus selects a merge candidate, and then supplies a merge index indicating the selected merge candidate and inter frame prediction information of the merge candidate to the motion compensation predictor 306 via the inter frame mode determiner 305 predictions. In contrast, the merge candidate selector 447 on the decoding side selects the merge candidate based on the decoded merge index, and supplies the selected merge candidate to the motion compensation predictor 406 .

Update Motion Vector Predictor Candidate List Based on History

Next, the method of initializing and updating the motion vector predictor candidate list HmvpCandList based on the history provided in the encoding information storage memory 111 on the encoding side and the encoding information storage memory 205 on the decoding side will be described in detail. Fig. 26 is a flowchart illustrating a procedure for initializing/updating a motion vector predictor candidate list based on history.

In the present embodiment, the motion vector predictor candidate list HmvpCandList based on the history is updated in the encoding information storage memory 111 and the encoding information storage memory 205 . Alternatively, in the inter prediction block 102 and the inter prediction block 203, a history-based motion vector predictor candidate list updater may be provided to update the history-based motion vector predictor candidate list HmvpCandList.

In the segment header, the history-based motion vector predictor candidate list HmvpCandList is initialized. On the encoding side, the history-based motion vector predictor candidate list HmvpCandList is updated in the case that the prediction method determiner 105 selects the normal motion vector predictor mode or the normal merge mode. On the decoding side, the history-based motion vector predictor candidate list HmvpCandList is updated in the case where the prediction information decoded by the bit string decoding section 201 is the normal motion vector predictor mode or the normal merge mode.

The inter prediction information used at the time of performing inter prediction in the normal motion vector predictor mode or normal merge mode needs to be registered in the history-based motion vector predictor candidate list HmvpCandList as the inter prediction information candidate hMvpCand. The inter prediction information candidate hMvpCand includes a reference index refIdxL0 of L0 and a reference index refIdxL1 of L1, a prediction flag predFlagL0 L0 indicating whether L0 prediction should be performed, a prediction flag predFlagL1 L1 indicating whether L1 prediction should be performed, a motion vector mvL0 for L0 and motion vector mvL1 for L1.

In the case of having inter prediction information having the same value as the inter prediction information candidate hMvpCand among the elements (i.e., inter prediction information) registered in the motion vector predictor candidate list HmvpCandList based on the history provided in the encoding information storage memory 111 on the encoding side and the encoding information storage 205 on the decoding side, the item will be removed from the motion vector predictor candidate list HmvpCandList based on the history. On the contrary, in the case of no inter prediction information having the same value as the inter prediction information candidate hMvpCand, the header element of the history-based motion vector predictor candidate list HmvpCandList will be deleted, and the inter prediction information candidate hMvpCand will be added to the end of the vector predictor candidate list HmvpCandList movements based on history.

The number of elements of the motion vector predictor candidate list HmvpCandList provided in the coding information storage 111 on the coding side and the coding information storage 205 on the decoding side of the present invention is set to six.

First, the history-based motion vector predictor candidate list HmvpCandList is initialized in slot blocks (step S2101 in FIG. 26). All elements of the history-based motion vector predictor candidate list HmvpCandList are cleared at the beginning of the segment, and the number NumHmvpCand (current number of candidates) of the history-based motion vector predictor candidates registered in the motion vector predictor candidate list HmvpCandList is set to 0.

While the initialization of the history-based motion vector predictor candidate list HmvpCandList must be performed in segment blocks (the first segment coding block), the initialization may be performed in image blocks, tiles, or rows of the tree block.

Then, for each of the coding blocks in the segment, the following process of updating the motion vector predictor candidate list HmvpCandList based on the history is repeated (steps S2102 to S2107 in FIG. 26).

First of all, the initial settings are made in the coding blocks. The identityCandExist flag indicating the presence of an identical candidate is set to FALSE, the removal target index removeIdx indicating removal of the target candidate is set to 0 (step S2103 in FIG. 26).

The existence of an inter prediction information candidate hMvpCand to be registered is determined (step S2104 in FIG. 26). In the case where the encoding side prediction method determiner 105 determines the normal motion vector predictor mode or the normal merge mode, or when the bit string decoding section 201 on the decoding side performs decoding as the normal motion vector predictor mode or the normal merge mode, the corresponding inter prediction information is set. as an hMvpC candidate of inter-frame prediction information and is subject to registration. In the case where the encoding-side prediction method determiner 105 determines the intra-prediction mode, the sub-block motion vector predictor mode, or the sub-block merge mode, or, in the case that the bit-string decoding section 201 on the decoding side performs decoding as the intra-prediction mode, the vector prediction mode no motion vector predictor candidate list HmvpCandList update process based on the history, and no inter prediction information hMvpCand candidate for registration will be performed. If there is no inter prediction information candidate hMvpCand to be registered, steps S2105 to S2106 will be skipped (step S2104 in FIG. 26: NO). If there is an inter prediction information candidate hMvpCand to be registered, the process of step S2105 and later will be executed (step S2104 in FIG. 26: YES).

Subsequently, it is determined whether the individual elements of the history-based motion vector predictor candidate list HmvpCandList include an element (inter prediction information) having the same value as the inter prediction information candidate hMvpCand for registration, that is, whether an identical element exists. (step S2105 in FIG. 26). Fig. 27 is a flowchart of an identical element confirmation processing procedure. In the case where the history-based motion vector predictor candidate number value NumHmvpCand is 0 (S2121: NO in FIG. 27), the history-based motion vector predictor candidate list HmvpCandList is empty and there is no identical candidate. Accordingly, steps S2122-S2125 in FIG. 27 will be skipped, which will complete the identical element confirmation processing procedure. In the case where the value of the history-based motion vector predictor candidate number NumHmvpCand is greater than 0 (YES in step S2121 in FIG. 27), the process of step S2123 will be repeated from the history-based motion vector predictor index hMvpIdx from 0 to NumHmvpCand-1 (steps with S2122 to S2125 in Fig. 27). First, a comparison is made as to whether the 0-based element hMvpIdx-th HmvpCandList [hMvpIdx] in the history-based motion vector predictor candidate list is identical to the inter prediction information candidate hMvpCand (step S2123 in FIG. 27). In the case where they are identical (step S2123 in FIG. 27: YES), the identityCandExist flag indicating the existence of an identical candidate is set to TRUE, and the removal target index removeIdx indicating the position of the element to be removed is set to the current value of the motion vector predictor index hMvpIdx based on history, and identical item confirmation processing will be completed. In the case where they are not identical (step S2123 in FIG. 27: NO), hMvpIdx is incremented by one. In the case where the motion vector predictor index based on history hMvpIdx is less than or equal to NumHmvpCand-1, processing is performed in step S2123 and later.

Returning to the flowchart of the algorithm in FIG. 26, a process of shifting and adding elements of the motion vector predictor candidate list HmvpCandList based on the history is performed (step S2106 in FIG. 26). Fig. 28 is a flowchart of the shift/add processing procedure of elements in the history-based motion vector predictor candidate list HmvpCandList in step S2106 in FIG. 26. First, it is determined whether to add a new element after deleting the element stored in the history-based motion vector predictor candidate list HmvpCandList, or to add a new element without deleting the element. Specifically, a comparison is made as to whether the identityCandExist flag indicating the existence of an identical candidate is TRUE or NumHmvpCand is equal to 6 (step S2141 in FIG. 28). In the case where one of the conditions under which the identityCandExist flag indicating the existence of an identical candidate is TRUE or that the number of the current candidate NumHmvpCand is 6 is satisfied (step S2141: YES in FIG. 28), the item stored in the candidate list HmvpCandList history-based motion vector predictor is removed and a new element is added afterwards. The initial value of index i is set to removeIdx + 1. The element shifting process in step S2143 is repeated from this initial value to NumHmvpCand. (Steps S2142 to S2144 in FIG. 28). By copying the elements of HmvpCandList[i] to HmvpCandList[i-1], the elements are shifted forward (step S2143 in FIG. 28) and i is increased by one (steps S2142 to S2144 in FIG. 28). Then, the inter prediction information candidate hMvpCand is added to (NumHmvpCand-1) HmvpCandList [NumHmvpCand-1] counting from 0, which corresponds to the end of the history-based motion vector predictor candidate list (step S2145 in FIG. 28), and the list shift/add process HmvpCandList of motion vector predictor candidates based on history will be completed. On the contrary, in the case where none of the conditions under which the identityCandExist flag indicating the presence of an identical candidate is TRUE and that NumHmvpCand is 6 is satisfied (Step S2141: NO in FIG. 28), the inter prediction information candidate hMvpCand will be added to the end of the history-based motion vector predictor candidate list without deleting an entry stored in the history-based motion vector predictor candidate list HmvpCandList (step S2146 in FIG. 28). Here, the end of the history-based motion vector predictor candidate list is NumHmvpCand-th HmvpCandList [NumHmvpCand], counted from 0. Moreover, NumHmvpCand is incremented by one, and the process of shifting and adding elements of the history-based motion vector predictor candidate list HmvpCandList is completed.

Fig. 31 is a view illustrating an example of a process for updating a motion vector predictor candidate list based on history. In the case where a new element is to be added to the history-based motion vector predictor candidate list HmvpCandList in which six elements (inter prediction information) are already registered, the history-based motion vector predictor candidate list HmvpCandList is compared with the new inter prediction information in order of initial element (Fig. 31A). When the new element has the same value as the third element HMVP2 in the header of the history-based motion vector predictor candidate list HmvpCandList, the HMVP2 element is removed from the history-based motion vector predictor candidate list HmvpCandList, and the following HMVP3-HMVP5 elements are shifted (copied) one by one. others forward, and a new entry is added to the end of the history-based motion vector predictor candidate list HmvpCandList (FIG. 31B) to complete updating the history-based motion vector predictor candidate list HmvpCandList (FIG. 31C).

The Process of Deriving Motion Vector Predictor Candidates Based on History

Next, a method for obtaining a history-based motion vector predictor candidate from the history-based motion vector predictor candidate list HmvpCandList will be described in detail. This corresponds to the processing procedure of step S304 in FIG. 20 regarding the general processing performed by the motion vector predictor candidate obtaining unit 323 based on the history of the normal motion vector predictor mode obtaining unit 301 on the encoding side and the motion vector predictor candidate obtaining unit 423 based on the history from the normal motion vector predictor mode obtaining unit 401 on the side. decoding. Fig. 29 is a flowchart illustrating a processing procedure for obtaining a motion vector predictor candidate based on history.

In the case where the current number of motion vector predictor candidates numCurrMvpCand is greater than or equal to the maximum number of elements of the motion vector predictor candidate list mvpListLX (here 2) or the number of motion vector predictor candidates NumHmvpCand based on history is 0 (step S2201: NO in FIG. 29), the process of steps S2202 to S2209 in FIG. 29 will be omitted, and the history-based motion vector predictor candidate acquisition processing will be completed. In the case where the number numCurrMvpCand of the current motion vector predictor candidates is less than 2, which is the maximum number of elements of the motion vector predictor candidate list mvpListLX, and, in the case where the value of the number NumHmvpCand of the history-based motion vector predictor candidates is greater than 0 (step S2201: YES on 29), the process of steps S2202 to S2209 in FIG. 29.

Then, the process of steps S2203 to S2208 in FIG. 29 is repeated until index i is 1 to the lower of 4, or up to the number of history-based motion vector predictor candidates numCheckedHMVPCand (steps S2202-S2209 in FIG. 29). In the case where the current number of motion vector predictor candidates numCurrMvpCand is greater than or equal to 2, which is the maximum number of elements of the motion vector predictor candidate list mvpListLX (step S2203: NO in FIG. 29), steps S2204 to S2209 in FIG. 29 will be omitted, and the history-based motion vector predictor candidate acquisition process will be completed. In the case where the current number of motion vector predictor candidates numCurrMvpCand is less than 2, which is the maximum number of elements in the motion vector predictor candidate list mvpListLX (step S2203 in FIG. 29: YES), the process in step S2204 and later in FIG. 29.

Then, the process in steps S2205 to S2207 is performed for the cases where Y is 0 and Y is 1 (L0 and L1) (steps S2204 to S2208 in FIG. 29). In the case where the current number of motion vector predictor candidates numCurrMvpCand is greater than or equal to 2, which is the maximum number of elements of the motion vector predictor candidate list mvpListLX (S2205: NO in FIG. 29), the process of steps S2206 to S2209 in FIG. 29 will be omitted, and the history-based motion vector predictor candidate acquisition process will be completed. In the case where the current number of motion vector predictor candidates numCurrMvpCand is less than 2, which is the maximum number of elements in the motion vector predictor candidate list mvpListLX (S2205: YES in FIG. 29), the process in S2206 and later in FIG. 29.

Then, in the case where the history-based motion vector predictor candidate list HmvpCandList includes an element having the same reference index as the encoding/decoding target motion vector reference index refIdxLX and which is different from any element of the motion vector predictor list mvpListLX (step S2206: YES in FIG. 29), and the number numCurrMvpCand of current motion vector predictor candidates is incremented by one. In the case where there is no element in the history-based motion vector predictor candidate list HmvpCandList that has the same reference index as the encoding/decoding target motion vector reference index refIdxLX and is different from any element of the motion vector predictor list mvpListLX (step S2206: NO in Fig. 29), the additional process in step S2207 will be skipped.

The process of steps S2205-S2207 in FIG. 29 is performed for both L0 and L1 (steps S2204-S2208 in FIG. 29). Index i is incremented by one, and when index i is less than or equal to any of the smaller value 4 or the number of history-based motion vector predictor candidates NumHmvpCand, the process of step S2203 and later will be repeated (steps S2202 to S2209 in FIG. 29).

The process of obtaining merger candidates based on history

The following is a detailed description of the method for obtaining a history-based merge candidate from the history-based merge candidate list HmvpCandList, the process procedure of step S404 in FIG. 21, which is the general process of block 345 for obtaining merge candidates based on history from block 302 for obtaining normal merge mode on the encoding side, and block 445 for obtaining merge candidates based on the history of block 402 for obtaining normal merge mode on the decoding side. Fig. 30 is a flowchart illustrating a history-based merge candidate acquisition processing procedure.

First, an initialization process is performed (step S2301 in FIG. 30). The elements from 0 to (numCurrMergeCand -1) of isPruned[i] are each set to FALSE and numOrigMergeCand is set to numCurrMergeCand of the number of elements registered in the current merge candidate list.

Subsequently, the hMvpIdx index initial value is set to 1, and the sub-process from step S2303 to step S2310 in FIG. 30 is repeated from this initial value to NumHmvpCand (steps S2302 to S2311 in FIG. 30). When the number numCurrMergeCand of elements registered in the current merge candidate list is not less than or equal to (MaxNumMergeCand-1 maximum merge candidates), merge candidates have been added to all elements in the merge candidate list. Accordingly, the history-based merge candidate acquisition process will be completed (Step S2303: NO in FIG. 30) In the case where the number numCurrMergeCand of items registered in the current merge candidate list is less than or equal to (MaxNumMergeCand-1 maximum merge candidate list) will be the process of step S2304 and later is performed. sameMotion is set to FALSE (step S2304 in FIG. 30). Subsequently, the initial value of index i is set to 0, and the process of steps S2306 and S2307 in FIG. 30 is executed from this initial value to numOrigMergeCand-1 (S2305 to S2308 in FIG. 30). A comparison is made to see if the (NumHmvpCand-hMvpIdx) element of HmvpCandList [NumHmvpCand-hMvpIdx] counted from 0 in the history-based motion vector predictor candidate list is the same value as the ith element of mergeCandList [i] counted from from 0 in the merge candidate list (step S2306 in FIG. 30).

The merge candidates are determined to have the same value in the case where all constituent elements (inter prediction mode, reference index, motion vector) of the merge candidate have the same value. In the case where the merge candidates have the same value and isPruned[i] is set to FALSE (step S2306: YES in FIG. 30), sameMotion and isPruned[i] will be set to TRUE (step S2307 in FIG. 30). In case the values do not match (step S2306: NO in FIG. 30), the process in step S2307 will be skipped. After completion of repetition processing from step S2305 to step S2308 in FIG. 30, a comparison is made as to whether the sameMotion is FALSE (step S2309 in FIG. 30). In the case where sameMotion is FALSE (Step S2309: YES in FIG. 30), that is, (NumHmvpCand-hMvpIdx) there is no HmvpCandList [NumHmvpCand-hMvpIdx] item which is 0-based in the history-based motion vector predictor candidate list. in the mergeCandList and hence the element HmvpCandList [NumHmvpCand - hMvpIdx], which is the (NumHmvpCand - hMvpIdx) 0-based element in the history-based motion vector predictor candidate list, is added to mergeCandList [ngeumCurr], which is numCurrMergeCand in the merge candidate list , and numCurrMergeCand is increased by one (step S2310 in FIG. 30). The hMvpIdx index is incremented by one (step S2302 in FIG. 30), and the process of steps S2302 to S2311 in FIG. 30 is repeated.

After confirming all elements in the history-based motion vector predictor candidate list is completed, or adding merge candidates to all elements in the merge candidate list is completed, the history-based merge candidate derivation process ends.

The Process of Obtaining Average Merge Candidates

The following is a detailed description of the method for obtaining the average merge candidate, the process procedure of step S403 in FIG. 21, which is the common process of the average merge candidate obtaining block 344 of the normal merge mode obtaining block 302 on the encoding side, and the average merge candidate obtaining block 444 of the normal merge mode obtaining block 402 on the decoding side. Fig. 39 is a flowchart illustrating an average merge candidate acquisition processing procedure.

First, an initialization process is performed (step S1301 in FIG. 39). The variable numOrigMergeCand is set to the number of numCurrMergeCand elements registered in the current list of merge candidates.

A sequential scan is then performed, starting at the beginning of the merge candidate list, to determine two pieces of motion information. The index i indicating the first motion information is set such that the index i = 0, and the index j indicating the second motion information is set such that the index j=1. (Steps S1302-S1303 in Fig. 39). When the number numCurrMergeCand of elements registered in the current merge candidate list is not less than or equal to (MaxNumMergeCand-1 maximum merge candidates), merge candidates have been added to all elements in the merge candidate list. Accordingly, the process of obtaining merge candidates based on the history will be completed (step S1304 in FIG. 39). In the case where the number numCurrMergeCand of elements registered in the current merge candidate list is less than or equal to (the maximum number of merge candidates MaxNumMergeCand-1), the process of step S1305 and later will be executed.

It is determined whether both the i-th mergeCandList [i] motion information of the merge candidate list and the j-th mergeCandList [j] motion information of the merge candidate list are valid (step S1305 in FIG. 39). In the case where both are invalid, the process proceeds to the next element without obtaining an average merge candidate mergeCandList[i] and mergeCandList[j]. In the case where the condition that both mergeCandList[i] and mergeCandList[j] are invalid is not satisfied, the following process is repeated with X set to 0 and 1 (steps S1306 to S1314 in FIG. 39).

Determining whether the LX prediction mergeCandList [i] is valid (step S1307 in FIG. 39). In the case where the LX prediction mergeCandList[i] is valid, a determination is made whether the LX prediction mergeCandList[j] is valid (step S1308 in FIG. 39). In the case where the LX prediction of mergeCandList[j] is valid, that is, in the case where both the LX prediction of mergeCandList[i] and the LX prediction of mergeCandList[j] are valid, an LX prediction motion vector is obtained by averaging the LX prediction motion vector of mergeCandList [i] and the LX prediction motion vector of mergeCandList [j], and an LX prediction average merge candidate having an LX prediction reference index of mergeCandList [i] will be obtained to set LX prediction as averageCand, and LX prediction averageCand will be valid (step S1309 in Fig. 39). At step S1308 in FIG. 39, in the case where the LX prediction mergeCandList[j] is invalid, that is, in the case where the LX prediction mergeCandList[i] is valid and the LX prediction mergeCandList[j] is invalid, the LX prediction motion vector mergeCandList[i] and the averaged candidate will be obtained. merge LX prediction having a reference index to be set as LX prediction averageCand and LX prediction meanCand will be valid (step S1310 in FIG. 39). In the case where the LX mergeCandList [i] prediction is invalid in step S1307 in FIG. 39, a determination is made as to whether or not the LX prediction mergeCandList [j] is valid (step S1311 in FIG. 39). In the case where the LX prediction mergeCandList[j] is valid, that is, in the case where the LX prediction mergeCandList[i] is invalid and the LX prediction mergeCandList[j] is valid, the LX prediction motion vector mergeCandList[j] and the average merge candidate will be obtained LX prediction having a reference index to be set as LX prediction averageCand, and LX prediction averageCand will be valid (step S1312 in FIG. 39). At step S1311 in FIG. 39, in the case where LX prediction mergeCandList[j] is invalid, that is, in the case where LX prediction mergeCandList[i] and LX prediction mergeCandList[j] are both invalid, LX prediction averageCand will be invalid. (step S1312 in FIG. 39).

Thus obtained average merge candidate L0 prediction, L1 prediction or BI prediction is added to numCurrMergeCand-th mergeCandList [numCurrMergeCand] of the merge candidate list, and numCurrMergeCand is incremented by one (step S1315 in FIG. 39). This completes the process of obtaining an average merge candidate.

An average fusion candidate is obtained by averaging over each of the horizontal motion vector component and the vertical motion vector component.

Motion Compensation Prediction Process

The motion compensation prediction block 306 obtains the position and size of the block currently undergoing encoding prediction processing. Further, the motion compensation predictor 306 obtains inter prediction information from the inter prediction mode determiner 305 . The reference index and the motion vector are derived from the obtained inter prediction information, and the reference picture indicated by the reference index in the decoded picture memory 104 is shifted from the same position as the picture signal of the block that is subjected to prediction processing by the amount of the motion vector. After the shift, an image signal of this position is obtained, and then a prediction signal is generated.

In the case where prediction is made from one reference picture, for example, when the inter prediction mode in inter prediction is L0 prediction or L1 prediction, the prediction signal obtained from one reference picture is set as the motion compensation prediction signal. In the case where the prediction mode is performed from two reference pictures, for example, when the inter prediction mode is BI prediction, a weighted average of the prediction signals obtained from the two reference pictures is set as the motion compensation prediction signal. The received motion compensation prediction signal is supplied to the prediction method determiner 105 . Here, the weighted averaging ratio in bidirectional prediction is set to 1:1. Alternatively, weighted averaging may use a different ratio. For example, the weighted averaging factor may be set such that the shorter the picture interval between the prediction target picture and the reference picture, the higher the weighted averaging factor. The calculation of the weighted average coefficient can also be performed using a lookup table between the combination of image intervals and the weighted average coefficients.

The motion compensation predictor 406 has a function similar to the motion compensation predictor 306 on the encoding side. The motion compensation predictor 406 receives inter-picture prediction information from the normal motion vector predictor mode obtainer 401, the normal merge mode obtainer 402, the sub-block motion vector predictor mode obtainer 403, and the sub-block merge mode obtainer 404 through the switch 408. The motion compensation predictor 406 supplies the received motion compensation prediction signal to the decoded image signal overlay unit 207 .

Interframe prediction mode

The process of performing prediction from one reference picture is defined as unidirectional prediction. Unidirectional prediction performs L0 prediction or L1 prediction using one of two reference pictures registered in the L0 or L1 directory lists.

In FIG. 32 shows a case of unidirectional prediction in which the reference picture (RefL0Pic) L0 occurs at the time before the target picture (CurPic). Fig. 33 illustrates a case of unidirectional prediction in which the prediction reference picture L0 occurs at a time after the target picture. Similarly, unidirectional prediction can be performed by replacing the reference picture prediction L0 in FIG. 32 and FIG. 33 per prediction reference picture (RefL1Pic) L1.

The process of performing prediction from two reference pictures is defined as bidirectional prediction. Bidirectional prediction performs prediction expressed as BI prediction using both L0 prediction and L1 prediction. Fig. 34 illustrates a case of bidirectional prediction in which L0 reference picture prediction occurs at that point before the target picture and L1 reference picture prediction occurs at that point after the target picture. Fig. 35 illustrates a case of bidirectional prediction in which the prediction reference picture L0 and the prediction reference picture L1 occur at the same time as the target picture. Fig. 36 illustrates a case of bidirectional prediction in which a reference frame for prediction L0 and a reference picture L1 prediction occur at a time after the target picture.

Thus, it is possible to use prediction without limiting the relationship between L0/L1 prediction type and time, such that L0 for the past direction and L1 for the future direction. Moreover, bidirectional prediction can perform each of the L0 predictions and L1 predictions using the same reference picture. Determining whether to perform motion compensation prediction in unidirectional prediction or bidirectional prediction is performed based on information (eg, a flag) indicating whether to use L0 prediction and whether to use L1 prediction.

Reference index

In an embodiment of the present invention, it is possible to select optimal reference pictures from a plurality of reference pictures in motion compensation prediction to improve the accuracy of motion compensation prediction. Thus, a reference picture used in motion compensation prediction should be used as a reference index, and the reference index is encoded in the bitstream along with the motion vector difference.

Motion Compensation Process Based on Normal Motion Vector Predictor Mode

As illustrated in the inter prediction block 102 on the encoding side in FIG. 16, in the case where the inter prediction mode determiner 305 has been selected the inter prediction information by the normal motion vector predictor mode obtaining unit 301, the motion compensation predictor 306 obtains this inter prediction information from the inter prediction mode determiner 305 and obtains the inter prediction mode, reference index and a motion vector of the target block, and thus generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determiner 105 .

Similarly, as illustrated in the inter prediction block 203 on the decoding side in FIG. 22, in the case where the switch 408 is connected to the normal motion vector predictor mode obtaining unit 401 during the decoding process, the motion compensation prediction unit 406 acquires inter prediction information through the normal motion vector predictor mode acquisition unit 401, and obtains the inter prediction mode, the reference index, and the motion vector of the current target block and thus generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal overlay unit 207 .

Motion compensation processing based on normal fusion mode

As illustrated in the inter prediction block 102 on the encoding side in FIG. 16, in the case where the inter prediction mode determiner 305 has been selected the inter prediction information by the normal merge mode obtaining section 302, the motion compensation prediction section 306 obtains this inter prediction information from the inter prediction mode determiner 305, and obtains the inter prediction mode, the reference index, and the vector movement of the current target block, thereby generating a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determiner 105 .

Similarly, as illustrated in the inter prediction block 203 on the decoding side in FIG. 22, in the case where the switch 408 is connected to the normal merge mode obtaining unit 402 during the decoding process, the motion compensation prediction unit 406 obtains inter prediction information by the normal merge mode obtaining unit 402, and obtains the inter prediction mode, reference index, and motion vector of the current target block. , thereby generating a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal overlay unit 207 .

Motion Compensation Process Based on Subblock Motion Vector Predictor Mode

As illustrated in the inter prediction block 102 on the encoding side in FIG. 16, in the case where the inter prediction mode determiner 305 has selected the inter prediction information by the sub-block motion vector predictor mode obtaining unit 303, the motion compensation predictor 306 obtains this inter prediction information from the inter prediction mode determiner 305, and obtains the inter prediction mode, the reference index and a motion vector of the current target block, thereby generating a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determiner 105 .

Similarly, as illustrated in the inter prediction block 203 on the decoding side in FIG. 22, in the case where the switch 408 is connected to the sub-block motion vector predictor mode obtaining block 403 during the decoding process, the motion compensation prediction block 406 obtains inter-picture prediction information through the sub-block motion vector predictor mode obtaining block 403, and outputs the inter-prediction mode, the reference index, and the motion vector of the target block, thereby generating a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal overlay unit 207 .

Motion Compensation Process Based on Subblock Merge Mode

As illustrated in the inter prediction block 102 on the encoding side in FIG. 16, in the case where the inter prediction mode determiner 305 has selected the inter prediction information by the sub-block merge mode obtaining unit 304, the motion compensation predictor 306 obtains this inter prediction information from the inter prediction mode determiner 305 and outputs the inter prediction mode, the reference index, and the vector movement of the current target block, thereby generating a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determiner 105 .

Similarly, as illustrated in the inter prediction block 203 on the decoding side in FIG. 22, in the case where the switch 408 is connected to the sub-block merge mode obtaining block 404 during the decoding process, the motion compensation prediction block 406 acquires inter-prediction information through the sub-block merge mode obtaining block 404, and outputs the inter-prediction mode, reference index, and motion vector of the current target block, thereby generating a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal overlay unit 207 .

Motion Compensation Process Based on Affine Transform Prediction

In the normal motion vector predictor mode and the normal merge mode, motion compensation using an affine model can be used based on the following flags. The following flags are reflected in the following flags based on the inter prediction conditions determined by the inter prediction mode determiner 305 in the encoding process, and are encoded in the bitstream. In the decoding process, it is determined whether to perform motion compensation using the affine model based on the following flags in the bitstream.

sps_affine_enabled_flag indicates whether motion compensation using the affine model can be used in inter-prediction. When sps_affine_enabled_flag is 0, the process is suppressed not to perform affine model motion compensation on sequence blocks. Moreover, inter_affine_flag and cu_affine_type_flag are not conveyed in the coding unit (CU) syntax of the video coding sequence. When sps_affine_enabled_flag is equal to 1, affine model motion compensation can be used in the video coding sequence.

sps_affine_type_flag indicates whether motion compensation using the 6-parameter affine model can be used in inter-prediction. When sps_affine_type_flag is 0, the process is suppressed not to perform motion compensation using the 6-parameter affine model. Moreover, cu_affine_type_flag is not conveyed in the CU syntax of the video coding sequence. When sps_affine_type_flag is equal to 1, motion compensation based on a 6-parameter affine model can be used in the video coding sequence. If there is no sps_affine_type_flag, it should be 0.

In the case of P or B segment decoding, when inter_affine_flag is equal to 1 in the current target CU, motion compensation using an affine model is used to generate a motion compensation prediction signal of the current target CU. When inter_affine_flag is 0, no affine model is used for the current target CU. If there is no inter_affine_flag, it must be 0.

In the case of P or B segment decoding, when cu_affine_type_flag is equal to 1 in the current CU, motion compensation using a 6-parameter affine model is used to generate a motion compensation prediction signal of the current CU. When cu_affine_type_flag is 0, motion compensation using a four parametric affine model is used to generate a motion compensation prediction signal of the CU currently being processed.

The reference index and motion vector are output in blocks of sub-blocks in motion compensation based on the affine model. Accordingly, a motion-compensated prediction signal is generated using a reference index and a motion vector for sub-block processing.

The four-parameter affine model is a mode in which a sub-block motion vector is obtained from the four parameters of the horizontal component and the vertical component of each of the motion vectors of the two control points, and motion compensation is performed in blocks of sub-blocks.

Triangle Merge Mode

The triangle fusion mode is a type of fusion mode in which an encoding/decoding block is divided into diagonal parts to perform motion compensation prediction. The triangle merge mode is a type of geometric division merge mode in which an encoding/decoding block is divided into blocks having a non-rectangular shape. In the geometric division merge mode, this corresponds to a mode in which an encoding/decoding block is divided into two rectangular triangles by a diagonal line.

The geometric separation merge mode is expressed by a combination of two parameters, for example, an index (angleIdx) indicating a separation angle and an index (distanceIdx) indicating a distance from the center of the coding block. For example, 64 patterns are defined as the geometric separation merge mode, and fixed length encoding is performed. Of the 64 patterns, two modes in which the division angle index indicates the angle forming the diagonal line of the coding block (for example, 45 degrees (angleIdx = 4 in a configuration in which 360 degrees are represented by 32 divisions) or 135 degrees (angleIdx = 12 in configuration in which 360 degrees are represented by 32 divisions)), and the index indicating the distance from the center of the coding block is the minimum (distanceIdx = 0, indicating that the division boundary passes through the center of the coding block), indicate that the coding block is divided by a diagonal line, and corresponds to the mode of merging triangles.

The triangle fusion mode will be described with reference to FIG. 38A and FIG. 38b. Fig. 38A and FIG. 38B illustrate an example of 16×16 coding/decoding block prediction of triangle fusion mode. The triangle merge mode encoding/decoding block is divided into 4×4 subblocks, and each of the subblocks is assigned to three partitions, namely, unidirectional prediction partition 0 (UNI0), unidirectional prediction partition 1 (UNI1), and bidirectional prediction partition 2 (BI). Here, subblocks above the diagonal line are assigned to section 0, subblocks below the diagonal line are assigned to section 1, and subblocks on the diagonal line are assigned to section 2. When merge_triangle_split_dir is 0, sections are assigned as shown in FIG. 38A, and when merge_triangle_split_dir is 1, partitions are assigned as shown in FIG. 38b.

For section 0 motion compensation prediction, unidirectional prediction motion information, indicated by triangle merge index 0, is used. The unidirectional prediction motion information indicated by triangle fusion index 1 is used for partition motion compensation prediction 1. Bidirectional prediction motion information combined with unidirectional prediction motion information indicated by triangle fusion index 0 and unidirectional prediction motion information indicated by triangle fusion index 1 is used for section 2 motion compensation prediction.

Here, the unidirectional prediction motion information is a set of a motion vector and a reference index, while the bidirectional prediction motion information is generated with two sets of a motion vector and a reference index. The motion information represents either unidirectional prediction motion information or bidirectional prediction motion information.

Merge candidate selectors 347 and 447 use the resulting mergeCandList of merge candidates as the triangleMergeCandList of triangle merge candidates.

With reference to FIG. 40, a flowchart related to obtaining a triangle merge candidate will be described.

First, the mergeCandList of merge candidates is used as the triangleMergeCandList of triangle merge candidates (step S3501).

Subsequently, the merge candidate having the motion information list L0 receives priority, and the unidirectional prediction motion information for the triangle merge section 0 is obtained (step S3502).

Then, the merge candidate having the motion information list L1 is given priority, and unidirectional prediction motion information for triangle merging section 1 is obtained (step S3503).

Note that step S3502 and step S3503 may be output in random order, and may also be processed in parallel.

Fig. 41 is a flowchart illustrating acquisition of unidirectional prediction motion information for triangle merging section 0 according to the present embodiment.

First, for the Mth candidate in the obtained merge candidate list mergeCandList, a determination is made whether the candidate M has motion information of the motion information list L0 (step S3601). In the case where the candidate M has the motion information of the motion information list L0, the motion information of the motion information list L0 of the candidate M is set as a triangle merge candidate (step S3602). For M candidates (M = 0, 1, ..., numMergeCand-1), steps S3601 and step S3602 are executed in ascending order, and triangle merge candidates are additionally output.

Then, for the Mth candidate in the obtained merge candidate list mergeCandList, a determination is made whether the candidate M has motion information of the motion information list L1 (step S3603). In the case where the candidate M has the motion information of the motion information list L1, the motion information of the motion information list L1 of the candidate M is set as a triangle merge candidate (step S3604). For candidates M (M = numMergeCand-1, ..., 1, 0), steps S3603 and step S3604 are executed in descending order, and triangle merging candidates are additionally output.

Fig. 42 is a flowchart illustrating the acquisition of unidirectional prediction motion information for triangle merging section 1 according to the present embodiment.

First of all, it is determined for the M-th candidate in the obtained merge candidate list mergeCandList whether the candidate M has the motion information of the motion information list L1 (step S3701). In the case where the candidate M has the motion information of the motion information list L1, the motion information of the motion information list L1 of the candidate M is set as a triangle merge candidate (step S3702). For M candidates (M = 0, 1, ..., numMergeCand-1), steps S3701 and step S3702 are executed in ascending order, and triangle merge candidates are additionally output.

Then, for the M-th candidate in the obtained merge candidate list mergeCandList, it is determined whether the candidate M has motion information of the motion information list L0 (step S3703). In the case where the candidate M has the motion information of the motion information list L0, the motion information of the motion information list L0 of the candidate M is set as a triangle merge candidate (step S3704). For candidates M (M = numMergeCand-1, ..., 1, 0), steps S3703 and step S3704 are executed in descending order, and triangle merge candidates are additionally output.

The coding-side merge candidate selector 347 receives motion information from the obtained triangleMergeCandList of triangle merge candidates, and calculates code amounts and distortion amounts.

The merge candidate selector 347 compares the calculated sets of code values and distortion values, thereby selecting directions for splitting coding blocks and triangle merge candidates for the split sections. In the case where encoding is performed using the triangle merge mode, the merge candidate selector 347 supplies the selected information (the division directions of the encoding block merge_triangle_split_dir and the triangle merge indices merge_triangle_idx0, merge_triangle_idx1 indicating the triangle merge candidates of the split sections) and the interframe prediction information of the triangle merge candidates to block 306 motion compensation predictions. The bit string encoding unit 108 encodes the selected information.

On the other hand, in the case of the triangle merge mode, the merge candidate selector 447 on the decoding side selects the triangle merge candidates based on the decoded information (merge_triangle_split_dir coding block splitting directions and merge_triangle_idx0 and merge_triangle_idx1 triangle merge indices indicating split section triangle merge candidates) and provides inter-picture prediction information of the selected triangle merge candidates to the motion compensation predictor 406 .

In the case of the triangle merge mode, motion compensation predictors 306 and 406 perform weighted averaging, described below. In the case of brightness, motion compensation predictors 306 and 406 calculate nCbR = (nCbW > nCbH)? (nCbW / nCbH): (nCbH / nCbW) relative to the width nCbW and the height nCbH of the coding block. Then, at position (x, y) in the coding block, the weight of wValue in the case of FIG. 38A is calculated as wValue = (nCbW > nCbH)?

(Clip3(0, 8, x/nCbR - y + 4)):

(Clip3(0, 8, y/nCbR - x + 4)). On the other hand, the weight of wValue in the case of FIG. 38B is calculated as wValue = (nCbW > nCbH)?

(Clip3(0, 8, nCbH - 1 - x / nCbR - y + 4)):

(Clip3(0, 8, nCbW - 1 - y / nCbR - x + 4)). In addition, motion compensation predictors 306 and 406 calculate shift1=max(5, 17-bitDepth).

offset1 = 1 << (shift1 - 1) relative to bitDepth. The result of the weighted average pbSamples is then calculated as pbSamples = Clip3 (0, (1 << bitDepth) - 1,

(predSamplesLA * wValue +

predSamplesLB * (8 - wValue) + offset1) >> shift 1). Here, predSamplesLA is a motion-compensated pixel value using the mvLA motion vector, and predSamplesLB is a motion-compensated pixel value using mvLB.

The process of storing encoding information in storage memory

The inter prediction information obtained in the triangle fusion mode is stored in the encoding information storage memory, so that the inter prediction information can be referred to as inter prediction information adjacent to the target block when encoding and decoding are performed. The coding information storage memory process is performed in 4×4 sub-block blocks, and the inter prediction information specified by the partitions is stored.

This inter prediction information is inter prediction information for unidirectional prediction section 0 (UNI0) and unidirectional prediction section 1 (UNI1). Bidirectional prediction section 2 (BI) is obtained using the inter prediction information UNI0 and UNI1.

For sub-blocks on diagonal lines divided as sections, in the case where the weighting shown in FIG. 43, separation 2 shown in FIG. 44. FIG. 44A illustrates a case where the area in which the weighing illustrated in FIG. 43A is set as section 2. Similarly, FIG. 44B corresponds to FIG. 43b. Fig. 44C illustrates the case where the sub-blocks for which weighting is performed in FIG. 43A are set as section 2. Similarly, FIG. 44D corresponds to FIG. 43b. Fig. 44E illustrates the case where the sub-blocks for which weighting is performed in FIG. 43A are set as section 2. Similarly, FIG. 44F corresponds to FIG. 43b.

Section 2 shown in FIG. 44 can also be stored as BI as shown in FIG. 45. However, in this embodiment, for the sub-blocks for which part of the weighting is performed, as shown in FIG. 46, inter prediction information belonging to a section having a larger weight value is stored as unidirectional prediction (UNIY). Here Y is 0 or 1.

Additionally, as illustrated in FIG. 47 and FIG. 48, sub-blocks originally stored as BI are stored as unidirectional prediction using the predetermined section inter prediction information. Fig. 47A and FIG. 47B represent the case where UNI0 is a predetermined partition. Fig. 48A and FIG. 48B represent the case where UNI1 is a predetermined partition.

As described in this embodiment, by storing bidirectional prediction section 2 (BI) in the encoding information storage memory as unidirectional prediction UNIY, the storage memory can be reduced. Further, since the motion information of the motion information list L0 and the motion information list L1 is stored in the encoding information storage memory without being converted to BI prediction, the amount of processing can be reduced. Moreover, it is possible to reduce the amount of processing in the case where the inter prediction information given by the triangle fusion mode is referred to and used in subsequent encoding and decoding.

By selecting UNI1 as a predetermined partition, as illustrated in FIG. 48A and FIG. 48B, in the case where the right block of the triangle merge mode illustrated in FIG. 48B is in triangle fusion mode in FIG. 48A, the continuity of the stored motion information in the area where the two triangle merge modes continue to be performed can also be maintained. Therefore, the coding efficiency is improved compared to selecting UNI0 as the predetermined partition, as shown in FIG. 47A and FIG. 47b.

Selecting UNI1 as the predetermined partition, as illustrated in FIG. 48A and FIG. 48B can also maintain processing continuity because after encoding/decoding, partition motion information is accumulated. Therefore, the amount of processing is reduced compared to selecting UNI0 as the predetermined partition, as shown in FIG. 47A and FIG. 47b.

By fixing the unidirectional prediction as L0 prediction to be stored as UNI1, since the L0 prediction can be processed in the same way as the P-segment, the amount of processing is reduced compared to the case where the unidirectional prediction is stored as L1 prediction.

In all of the embodiments described above, a plurality of technologies may be combined with each other.

In all the embodiments described above, the bitstream output from the image encoding device has a specific data format to be decoded according to the encoding method used in the embodiment. Moreover, the image decoding apparatus corresponding to the image encoding apparatus is configured to decode a bitstream of a specific data format.

In the case where a wired or wireless network is used to exchange the bit stream between the image encoding apparatus and the image decoding apparatus, the bit stream can be converted to a data format suitable for a transmission form of a communication channel in transmission. In this case, a transmission device is provided that converts an output bitstream from an image encoding device into coded data in a data format suitable for a transmission form of a communication channel, and transmits the coded data to the network, and a reception device that receives the coded data from the network, which needs to be restored to a bitstream, and supplies the bitstream to an image decoding device. The transmission apparatus includes a memory that buffers a bitstream output from the image encoding apparatus, a packet processing unit that packetizes the bitstream, and a transmitter that transmits the packetized encoded data via a network. The receiving device includes a receiver that receives packetized coded data via a network, a memory that buffers the received coded data, and a packet processing unit that decompresses the coded data to form a bitstream and transmits the generated bitstream to an image decoding device.

Moreover, as a display device, a display unit that displays an image decoded by the image decoding apparatus can be added to the configuration. In this case, the display unit reads the decoded image signal generated by the decoded image signal overlay unit 207 and stored in the decoded image memory 208, and displays the signal on a screen.

Additionally, an imaging unit may be added to the configuration to allow it to function as an imaging device by inputting a captured image into an image encoding device. In this case, the imaging section inputs the captured image signal to the block separating section 101 .

Fig. 37 illustrates a hardware configuration example of an encoding/decoding device according to the present embodiment. The encoding/decoding apparatus includes configurations of an image encoding apparatus and an image decoding apparatus according to embodiments of the present invention. The encoder/decoder device 9000 includes a CPU 9001, an IC codec 9002, an input/output interface 9003, a memory 9004, an optical disc drive 9005, a network interface 9006, and a video interface 9009 in which the individual units are connected via a bus 9010.

Image encoder 9007 and image decoder 9008 are typically implemented as codec IC 9002. The image encoding process of the image encoding device according to the embodiments of the present invention is performed by the image encoder 9007 . The image decoding process in the image decoding apparatus according to the embodiment of the present invention is performed by the image decoder 9008. The input/output interface 9003 is implemented, for example, via a USB interface, and is connected to an external keyboard 9104, mouse 9105, or the like. The CPU 9001 controls the encoder/decoder device 9000 based on the user's operation input via the I/O interface 9003 to perform the user's operation. User operations on keyboard 9104, mouse 9105, etc. include selecting the encoding or decoding function to be performed, setting the encoding quality, bitstream I/O assignment, image I/O assignment, or the like, the like, the like.

In the case where the user requires an operation to reproduce an image recorded on the disc recording medium 9100, the optical disc drive 9005 reads the bitstream from the inserted disc recording medium 9100 and transmits the read bitstream to the image decoder 9008 of the IC codec 9002 via the bus 9010. The image decoder 9008 performs an image decoding process in an image decoding apparatus according to embodiments of the present invention on an input bitstream, and transmits the decoded image to an external monitor 9103 via a video interface 9009. The encoding/decoding apparatus 9000 has a network interface 9006 and can be connected to an external distribution server 9106 and to the mobile terminal device 9107 via the network 9101. In the case where the user wishes to reproduce the image recorded in distribution, the server 9106 or the mobile terminal device 9107 instead of the image recorded on the disk medium 9100 recording, the network interface 9006 receives the bitstream from the network 9101 instead of reading the bitstream from the input disk recording medium 9100. In the case where the user wishes to reproduce the image recorded in the memory 9004, the image decoding processing is performed by the device for decoding the image of the bitstream recorded in the memory 9004 according to embodiments in the present invention.

In the case where the user wishes to perform the operation of encoding an image captured by the external camera 9102 and writing the image to the memory 9004, the video interface 9009 inputs the image from the camera 9102 and transmits the image to the image encoder 9007 of the codec 9002 via the bus 9010. The image encoder 9007 performs an image encoding process by an image encoding device according to an embodiment of the present invention at the image input through the video interface 9009 and thereby generates a bit stream. The bitstream is then transferred to the memory 9004 via the bus 9010. In the case where the user wishes to record the bitstream on the disk recording medium 9100 instead of the memory 9004, the optical disk drive 9005 writes the bitstream on the inserted disk of the medium 9100.

It is also possible to realize an equipment configuration having an image encoding device and no image decoding device, or an equipment configuration having an image decoding device and no image encoding device. This hardware configuration is implemented by replacing the codec IC 9002 with an image encoder 9007 or an image decoder 9008.

The above-described process relating to encoding and decoding can naturally be implemented as a transmission, storage, and reception device using hardware, and alternatively, the process can be implemented using firmware stored in read-only memory (ROM) flash memory or the like, or using software provided for a computer or the like. The firmware program and the software may be provided by being recorded on a computer-readable recording medium or the like, may be provided from a server via a wired or wireless network, or may be provided through a digital terrestrial or satellite broadcast.

The present invention has been described with reference to the present embodiments. The above described embodiment has been described for illustrative purposes only. Rather, those skilled in the art can readily appreciate that various examples of modifications can be made by forming various combinations of the components or processes described above, which are also within the technical scope of the present invention.

The present invention can be used for image encoding and decoding methods that divide an image into blocks to perform prediction.

100 - image encoding device

101 - block division block

102 - interframe prediction block

103 - intra prediction block

104 - decoded image memory

105 - determinant of the prediction method

106 - block generating the remainder

107 - orthogonal converter/quantizer

108 - bit string encoding block

109 - inverse quantizer / inverse orthogonal converter

110 - decoded image signal overlay block

111 - encoding information storage memory

200 - image decoding device

201 - bit string decoding block

202 - block division block

203 - interframe prediction block

204 - intra prediction block

205 - encoding information storage memory

206 - inverse quantizer/inverse orthogonal converter

207 - decoded image signal overlay block

208 - decoded image memory

Claims (28)

1. A device for encoding a moving image, containing: a triangle merge candidate list constructor, configured to construct a triangle merge candidate list including spatial merge candidates; a first triangle merge candidate selector, configured to select from the list of triangle merge candidates a first triangle merge candidate that is unidirectional prediction; and a second triangle merge candidate selector, configured to select from a list of triangle merge candidates a second triangle merge candidate that is unidirectional prediction, wherein in the area where motion compensation is performed with the first triangle fusion candidate, motion information of the first triangle fusion candidate is stored, in the area where motion compensation is performed with the second triangle fusion candidate, motion information of the second triangle fusion candidate is stored, and in a region where motion compensation is performed by weighted averaging with the first triangle merge candidate and the second triangle merge candidate, unidirectional prediction motion information of one of the first triangle merge candidate or the second triangle merge candidate is stored. 2. A method for encoding a moving image, comprising: a triangle fusion candidate listing step of compiling a triangle fusion candidate list including spatial fusion candidates; a first triangle merge candidate selection step of selecting from the triangle merge candidate list a first triangle merge candidate that is unidirectional prediction; and a second triangle fusion candidate selection step of selecting from a list of triangle fusion candidates a second triangle fusion candidate that is unidirectional prediction, wherein in the area where motion compensation is performed with the first triangle fusion candidate, motion information of the first triangle fusion candidate is stored, in the area where motion compensation is performed with the second triangle fusion candidate, motion information of the second triangle fusion candidate is stored, and in a region where motion compensation is performed by weighted averaging with the first triangle merge candidate and the second triangle merge candidate, motion information of the unidirectional prediction of the first triangle merge candidate or the second triangle merge candidate is stored. 3. A device for decoding a moving image, comprising: a triangle merge candidate list constructor, configured to construct a triangle merge candidate list including spatial merge candidates; a first triangle merge candidate selector, configured to select from the list of triangle merge candidates a first triangle merge candidate that is unidirectional prediction; and a second triangle merge candidate selector, configured to select from a list of triangle merge candidates a second triangle merge candidate that is unidirectional prediction, wherein in the area where motion compensation is performed with the first triangle fusion candidate, motion information of the first triangle fusion candidate is stored, in the area where motion compensation is performed with the second triangle fusion candidate, motion information of the second triangle fusion candidate is stored, and in a region where motion compensation is performed by weighted averaging with the first triangle merge candidate and the second triangle merge candidate, motion information of the unidirectional prediction of the first triangle merge candidate or the second triangle merge candidate is stored. 4. A method for decoding a moving image, comprising: a triangle fusion candidate listing step of compiling a triangle fusion candidate list including spatial fusion candidates; a first triangle merge candidate selection step of selecting from the triangle merge candidate list a first triangle merge candidate that is unidirectional prediction; and a second triangle fusion candidate selection step of selecting from a list of triangle fusion candidates a second triangle fusion candidate that is unidirectional prediction, wherein in the area where motion compensation is performed with the first triangle fusion candidate, motion information of the first triangle fusion candidate is stored, in the area where motion compensation is performed with the second triangle fusion candidate, motion information of the second triangle fusion candidate is stored, and in a region where motion compensation is performed by weighted averaging with the first triangle merge candidate and the second triangle merge candidate, motion information of the unidirectional prediction of the first triangle merge candidate or the second triangle merge candidate is stored.

Images