RU2781517C1 - Dynamic image encoding device, dynamic image encoding method, dynamic image encoding program, dynamic image decoding device, dynamic image decoding method and dynamic image decoding program - Google Patents

Abstract

FIELD: imaging technology.

SUBSTANCE: group of inventions relates to image encoding and decoding technologies in which the image is divided into blocks and prediction is performed. An image encoding device is proposed, characterized by the fact that it divides a block of the encoding tree into one or more standard blocks of in-block copying. The device contains a block for obtaining a block candidate vector, a selector, a storage device, a block for correcting the boundary of the reference area, and a bit string encoding block. In this case, the block of obtaining a block candidate vector is made with the possibility of obtaining block candidate vectors of the target encoding block in the target encoding image from encoding information stored in memory for storing encoding information. The selector is made with the ability to select the selected block vector from the candidate block vectors. The storage device is designed with the ability to store encoded images of a given number of standard blocks of in-block copying located directly in front of the target standard block of in-block copying encoding.

EFFECT: increase in the efficiency of encoding/decoding images.

6 cl, 83 dwg

Description

State of the art

The present invention relates to image encoding and decoding methods in which an image is divided into blocks and prediction is performed.

When encoding and decoding an image, the target image is divided into blocks, each of which is a group of a predetermined number of samples, and processing is performed in blocks of blocks. Dividing an image into appropriate blocks with appropriate intra-prediction and inter-prediction settings can improve coding efficiency.

Patent Document 1 discloses an intra-frame prediction technique in which a predicted image is obtained using decoded samples in a block adjacent to an encoding/decoding target block.

[Patent Document 1] JP 9-172644A.

Disclosure of the essence of the invention

However, in the technology set forth in Patent Document 1, only decoded samples in a block adjacent to an encoding/decoding target block are used for prediction, and this results in poor prediction performance.

In an aspect of the present invention, in order to solve the above-described problem, an apparatus is provided, including: a block vector candidate deriving unit that obtains candidate block vectors of an encoding target block in an encoding target image from encoding information stored in a memory for storing encoding information; a selector that selects a block vector to be selected from among the candidate block vectors; a storage device that stores encoded images of a predetermined number of intra-block copy building blocks immediately before the encoding target block; and a reference area boundary correction unit that removes an encoded image of one standard block copy stored in the memory from the referenced area after the encoding process of the encoding target block is completed, and determines whether both the upper left position and the bottom right position of the reference block specified by the selected block vector into the referenced area.

According to the present invention, a highly efficient low load image encoding/decoding process is provided.

Brief description of the drawings

Fig. 1 is a block diagram of an image encoding apparatus according to an embodiment of the present invention.

Fig. 2 is a block diagram of an image decoding apparatus according to an embodiment of the present invention.

Fig. 3 is a block diagram of the algorithm illustrating the operation of dividing a tree block.

Fig. 4 is a diagram illustrating the state of splitting the input image into tree blocks.

Fig. 5 is a diagram illustrating z-scan.

Fig. 6A is a diagram illustrating the shape of a divided block.

Fig. 6B is a diagram illustrating the shape of a divided block.

Fig. 6C is a diagram illustrating the shape of a divided block.

Fig. 6D is a diagram illustrating the shape of a divided block.

Fig. 6E is a diagram illustrating the shape of a divided block.

Fig. 7 is a block diagram of the algorithm illustrating the operation of dividing a block into four.

Fig. 8 is a flowchart illustrating the operation of dividing a block into two or three.

Fig. 9 is the syntax for expressing the shape of a block split.

Fig. 10A is a diagram illustrating intra prediction.

Fig. 10B is a diagram illustrating intra prediction.

Fig. 11 is a diagram illustrating reference blocks for inter prediction.

Fig. 12A is a syntax for expressing a coding block prediction mode.

Fig. 12B is a syntax for expressing a coding block prediction mode.

Fig. 13 is a table illustrating the correspondence between syntax elements and modes that are related to inter prediction.

Fig. 14 is a diagram illustrating affine motion compensation at two control points.

Fig. 15 is a diagram illustrating affine motion compensation at three control points.

Fig. 16 is a block diagram of a detailed configuration of the inter prediction block 102 (FIG. 1).

Fig. 17 is a block diagram of a detailed configuration of the normal motion vector predictor mode acquisition block 301 (FIG. 16).

Fig. 18 is a block diagram of a detailed configuration of the normal merge mode acquisition block 302 (FIG. 16).

Fig. 19 is a flowchart illustrating a normal motion vector predictor mode acquisition process of the normal motion vector predictor mode acquisition unit 301 (FIG. 16).

Fig. 20 is a flowchart illustrating a normal motion vector predictor mode acquisition processing procedure.

Fig. 21 is a flowchart illustrating a normal merge mode acquisition process processing procedure.

Fig. 22 is a block diagram of a detailed configuration of the inter prediction block 203 (FIG. 2).

Fig. 23 is a block diagram of a detailed configuration of the normal motion vector predictor mode acquisition block 401 (FIG. 22).

Fig. 24 is a block diagram of a detailed configuration of the normal merge mode acquisition block 402 (FIG. 22).

Fig. 25 is a flowchart illustrating the normal motion vector predictor mode acquisition process of the normal motion vector predictor mode acquisition unit 401 (FIG. 22).

Fig. 26 is a flowchart illustrating a process of initializing/updating a motion vector predictor candidate list based on history.

Fig. 27 is a flowchart of the identity confirmation processing procedure in the history-based motion vector predictor candidate list initialization/updating processing procedure.

Fig. 28 is a flowchart of the element shift processing procedure in the history-based motion vector predictor candidate list initialization/updating processing procedure.

Fig. 29 is a flowchart illustrating a processing procedure for obtaining a motion vector predictor candidate based on history.

Fig. 30 is a flowchart illustrating a history-based merge candidate acquisition processing procedure.

Fig. 31A is a diagram illustrating an example of a process for updating a motion vector predictor candidate list based on history.

Fig. 31B is a diagram illustrating an example of a process for updating a motion vector predictor candidate list based on history.

Fig. 31C is a diagram illustrating a process of updating a motion vector predictor candidate list based on history.

Fig. 32 is a diagram illustrating motion compensation prediction in a case where L0 prediction is performed and an L0 reference picture (RefL0Pic) occurs at a time before the target picture (CurPic).

Fig. 33 is a diagram illustrating motion-compensated prediction in a case where L0 prediction is performed and an L0 prediction reference picture occurs at a time after the target picture.

Fig. 34 is a diagram illustrating a motion compensation prediction direction in bidirectional prediction in which L0 reference picture prediction occurs at a time before the target picture and L1 reference picture prediction occurs at a time after the target picture.

Fig. 35 is a diagram illustrating a motion compensation prediction direction in bidirectional prediction in which L0 reference picture prediction and L1 reference picture prediction occur at a time before the target frame.

Fig. 36 is a diagram illustrating a motion compensation prediction direction in bidirectional prediction in which L0 reference picture prediction and L1 reference picture prediction occur at a time after the target picture.

Fig. 37 is a block diagram illustrating a hardware configuration example of an encoding/decoding device according to an embodiment of the present invention.

38 is a flowchart illustrating a procedure for obtaining an average merge candidate.

39A is a diagram illustrating the actual in-block copy reference area.

39B is a diagram illustrating the actual in-block copy reference area.

FIG. 40 is a block diagram of a detailed configuration of the intra prediction block 103 (FIG. 1).

FIG. 41 is a block diagram of a detailed configuration of the intra prediction block 204 (FIG. 2).

FIG. 42 is a block diagram of block 352 of intra-block copy prediction.

43 is a block diagram of an intra-block copy prediction block 362 .

44 is a flowchart illustrating the process of copying an intra-frame prediction block of the intra-block copy prediction block 352 .

45 is a flowchart illustrating the process of copying an intra-frame prediction block of the intra-block copy prediction block 362 .

Fig. 46 is a flowchart illustrating the intra-block copy process in merge mode.

FIG. 47 is a flowchart illustrating a process flow for obtaining a block vector prediction mode with intra-block copy.

48 is a flowchart illustrating the process performed by the reference position correction block 380 and the reference position correction block 480.

Fig. 49 is a schematic diagram illustrating a state of reference position correction.

Fig. 50A is a schematic diagram illustrating the top left and bottom right positions when the area that can be referred to is rectangular.

50B is a schematic diagram illustrating the top left and bottom right positions when the area that can be referred to is rectangular.

50C is a schematic diagram illustrating the top left and bottom right positions when the area that can be referred to is rectangular.

50D is a schematic diagram illustrating the top left and bottom right positions when the area that can be referred to is rectangular.

Fig. 51 is a flowchart illustrating a process for correcting a reference position of a portion in which an area that can be referred to is not rectangular.

Fig. 52A is a schematic diagram illustrating the status of the reference position correction.

Fig. 52B is a schematic diagram illustrating a state of correction of the reference position.

53 is a flowchart illustrating the process performed by the reference position correction block 380 and the reference position correction block 480.

Fig. 54A is a schematic view illustrating a state of dividing an area that can be referred to into two areas.

Fig. 54B is a schematic view illustrating a state of dividing an area that can be referred to into two areas.

Fig. 54C is a schematic view illustrating a state of dividing an area that can be referred to into two areas.

Fig. 54D is a schematic view illustrating a state of dividing an area that can be referred to into two areas.

55 is a flowchart illustrating the process of dividing a reference area into two areas and correcting each of the reference positions.

56A is a schematic diagram illustrating a reference area memory area when a coding tree block unit is used as an intra-block copy standard block.

56B is a schematic diagram illustrating a reference area memory area when a coding tree block unit is used as an intra-block copy standard block.

56C is a schematic diagram illustrating a reference area memory area when a coding tree block unit is used as an intra-block copy standard block.

57A is a schematic diagram illustrating a reference area memory area when a valid reference area is determined under the assumption that a unit is obtained by quartering a coding tree block as an intra-block copy standard block.

57B is a schematic diagram illustrating a reference area memory area when a valid reference area is determined under the assumption that a unit is obtained by quartering a coding tree block as an intra-block copy standard block.

57C is a schematic diagram illustrating a reference area memory area when a valid reference area is determined under the assumption that a unit is obtained by quartering a coding tree block as an intra-block copy standard block.

57D is a schematic diagram illustrating a reference area memory area when a valid reference area is determined under the assumption that a unit is obtained by quartering a coding tree block as an intra-block copy standard block.

58 is a flowchart illustrating the procedure for correcting the boundary of the reference area.

59A is a schematic diagram illustrating the division of a reference block in a memory area when a coding tree block unit is used as an intra-block copy standard block.

59B is a schematic diagram illustrating the division of a reference block in a memory area when a coding tree block unit is used as an intra-block copy standard block.

60A is a schematic diagram illustrating the division of a reference block in a memory area when a valid reference area is determined under the assumption that a unit is obtained by quartering a coding tree block as an intra-block copy standard block.

60B is a schematic diagram illustrating the division of a reference block in a memory area when a valid reference area is determined under the assumption that a unit is obtained by quartering a coding tree block as an intra-block copy standard block.

Implementation of the invention

First of all, technology definitions and technical terms used in the present embodiment will be described.

wood block

In an embodiment, the encoding/decoding target image is equally divided into a predetermined size. This block is defined as a tree block. While in FIG. 4, the tree block size is set to 128×128 samples, the tree block size is not limited to this, and any size can be set. The target tree block (corresponding to the encoding target in the encoding process and the decoding target in the decoding process) is switched in the raster scan order, that is, in the order from left to right and top to bottom. The interior of each tree block can be further divided recursively. The coding/decoding block resulting from the recursive splitting of the tree block is defined as the coding block. The tree block and the coding block are collectively defined as a block. Performing an appropriate block split provides efficient coding. The tree block size may be a fixed value predetermined by the encoding device and the decoding device, or a configuration may be adopted in which the tree block size determined by the encoding device is transmitted to the decoding device. Here, the maximum tree block size is 128×128 samples and the minimum tree block size is 16×16 samples. The maximum coding block size is 64×64 samples and the minimum coding block size is 4×4 samples.

Prediction mode

Switching is performed between intra-picture prediction (MODE_INTRA) for performing prediction from the processed image signal of the target image and inter-picture prediction (MODE_INTER) for performing prediction from the image signal of the processed image in blocks of coding target blocks.

The encoding process uses a processed image for an image obtained by decoding an encoded signal, an image signal, a tree block, a block, an encoding block, or the like. The decoding process uses the processed image for the decoded image, image signal, tree block, block, coding block, and the like.

The identification mode of intra-prediction (MODE_INTRA) and inter-prediction (MODE_INTER) is defined as a prediction mode (PredMode). As a value, the prediction mode (PredMode) has intra-frame prediction (MODE_INTRA) or inter-frame prediction (MODE_INTER).

Prediction with intraframe block copy

Block copy prediction is a process of encoding/decoding a target block with respect to a decoded sample in the target picture as a prediction value. The distance from the target block to the reference sample is represented by a block vector. The block vector refers to the target image, and the reference image is uniquely defined. Thus, there is no need to provide a reference index. The difference between a block vector and a motion vector is whether the reference image is the target image or the processed image. In addition, in the case of a block vector, sampling precision 1 or sampling precision 4 can be selected using adaptive motion vector resolution (AMVR).

With intrablock copy, you can choose one of two modes: intrablock copy prediction mode or intrablock copy merge mode.

The intra-block copy prediction mode is a mode of determining a block vector of a target block based on the block vector predictor obtained from the processed information and based on the block vector difference. The block vector predictor is obtained from the processed block near the target block and from the index to fine-tune the block vector predictor. An index to fine-tune the block vector predictor and block vector differences are transmitted in the bitstream.

The intra-block copy-merge mode is a mode of obtaining intra-block copy prediction information of a target block from intra-block copy prediction information of a processed block near the target block without transmitting a motion vector difference.

Interframe prediction

In inter-picture prediction, in which prediction is performed from an image signal of a processed image, a plurality of processed images can be used as reference images. To manage a plurality of reference pictures, two types of look-up lists L0 (look-up list 0) and L1 (look-up list 1) are defined. In each of the lists, a reference image is indicated using a reference index. In the P-segment, L0 prediction (Pred_L0) can be used. In the B-segment, L0 prediction (Pred_L0), L1 prediction (Pred_L1), and bidirectional prediction (Pred_BI) can be used. L0 prediction (Pred_L0) is inter-picture prediction, which is related to the reference picture driven by L0, while L1 prediction (Pred_L1) is inter-picture prediction, which is related to the reference picture driven by L1. Bidirectional prediction (Pred_BI) is inter-picture prediction in which both L0 prediction and L1 prediction are performed and refers to one reference picture driven in each of L0 and L1. Information defining L0 prediction, L1 prediction, and bidirectional prediction is defined as an inter prediction mode. The following processing process assumes that processing will be performed for each of L0 and L1 for constants and variables in the LX suffix output.

Motion vector predictor mode

The motion vector predictor mode is an index transmission mode for specifying a motion vector predictor, a motion vector difference, an inter prediction mode, and a reference index, and determining target block inter prediction information. A motion vector predictor is obtained from a motion vector predictor candidate obtained from a processed block in a neighboring target block or a block belonging to the processed image and located at the same position or neighborhood (vicinity) of the target block, and from an index to indicate the motion vector predictor.

Merge Mode

The merge mode is a mode of extracting target block inter prediction information from inter prediction information of a processed block in a neighboring block of the target block or a block belonging to the processed image and located at the same position as the target block or in the vicinity of the target block, without transmitting a motion vector difference or base index.

The processed block in the adjacent block of the target block and the inter prediction information of the processed block are determined as spatial merge candidates. Blocks belonging to the processed image and located at the same position as the target block, or in the vicinity (vicinity) of the target block, and inter prediction information obtained from the block inter prediction information are determined as temporal merge candidates. Each of the merger candidates is registered in the list of merger candidates. The merge candidate to be used to predict the target block is determined by the merge index.

neighboring block

In FIG. 11 is a diagram illustrating reference blocks to be referenced for obtaining inter prediction information in the motion vector predictor mode and the merge mode. A0, A1, A2, B0, B1, B2, and B3 are processing blocks in the neighboring block of the target block. T0 is a block belonging to the image being processed and located at the same position as the target block, or in the neighborhood (vicinity) of the target block in the target image.

A1 and A2 are blocks located to the left of the target encoding block and next to the target encoding block. B1 and B3 are blocks located above the target encoding block and next to the target encoding block. A0, B0, and B2 are blocks located respectively at the bottom left, top right, and top left of the coding target block.

The following is a detailed description of the processing of adjacent blocks in the motion vector predictor mode and the merge mode.

Affine motion compensation

Affine motion compensation first partitions a coding block into subblocks of a given block, and then individually determines a motion vector for each of the partitioned subblocks to perform motion compensation. The motion vector of each of the sub-blocks is obtained based on one or more control points obtained from the inter-frame prediction information of the processed block in the neighboring block of the target block or a block belonging to the processed image and located at the same position as or in the neighborhood (vicinity) of the target block. While the present embodiment sets the sub-block size to 4×4 samples, the sub-block size is not limited to this, and a motion vector can be obtained in blocks of samples.

In FIG. 14 shows an example of affine motion compensation in the case of two control points. In this case, each of the two control points has two parameters, ie a horizontal component and a vertical component. Accordingly, an affine transformation that has two control points is called a four-parameter affine transformation. In FIG. 14 CP1 and CP2 are control points.

In FIG. 15 shows an example of affine motion compensation in the case of three control points. In this case, each of the three control points has two parameters, ie a horizontal component and a vertical component. Accordingly, an affine transformation that has three control points is called a six-parameter affine transformation. In FIG. 15 CP1, CP2 and CP3 are control points.

Affine motion compensation may be used in either of the motion vector predictor mode and the fusion mode. The affine motion compensation application mode in the motion vector predictor mode is defined as the sub-block motion vector predictor mode. The application mode of affine motion compensation in the merge mode is defined as the sub-block merge mode.

Code block syntax

The syntax for expressing a coding block prediction mode will be described with reference to FIGS. 12A, 12B, and 13. In FIG. 12A, pred_mode_flag is a flag indicating whether the mode is an inter prediction mode. Setting pred_mode_flag to 0 indicates inter-prediction, while setting pred_mode_flag to 1 indicates intra-prediction. pred_mode_ibc_flag indicating whether the mode is an intra-block copy prediction mode is transmitted in the case of intra-prediction. merge_flag is passed in case of intrablock copy prediction (pred_mode_ibc_flag = 1). merge_flag is a flag indicating whether the mode used is an intra-block merge copy mode or an intra prediction block copy mode. merge_idx is passed in intra-block merge copy mode (merge_flag = 1). The mode is set to normal intra prediction, and normal intra prediction information intra_pred_mode is transmitted when the mode is not an intra block copy prediction mode (pred_mode_ibc_flag = 0).

intra_pred_mode is transmitted in case of intra prediction. merge_flag is a flag indicating whether the mode used is a merge mode or a motion vector prediction mode. In the case of the motion vector predictor mode (merge_flag = 0), an inter_affine_flag flag is transmitted indicating whether to apply the subblock motion vector predictor mode. In case of applying the sub-block motion vector prediction mode (inter_affine_flag = 1), cu_affine_type_flag is transmitted. cu_affine_type_flag is a flag for specifying the number of breakpoints in subblock motion vector predictor mode.

On the contrary, as shown in FIG. 12B, in case of merge mode (merge_flag = 1), merge_subblock_flag is transmitted. merge_subblock_flag is a flag indicating whether to apply the subblock merge mode. In the case of the subblock merge mode (merge_subblock_flag = 1), the merge index merge_subblock_idx is passed. Conversely, in the case where the mode is not a subblock merge mode (merge_subblock_flag = 0), a merge_triangle_flag flag indicating whether to apply the triangle merge mode is transmitted. If the triangle merge mode is applied (merge_triangle_flag = 1), the merge_triangle_idx0 and merge_triangle_idx1 indices of the triangle merge are passed for each of the merge_triangle_split_dir block splitting directions and for each of the two split sections. If the triangle merge mode is not applied (merge_triangle_flag = 0), the merge_idx index of the merge is passed.

In FIG. 13 shows the meaning of each of the inter prediction syntax elements and the corresponding prediction mode. merge_flag = 0 and inter_affine_flag = 0 correspond to normal motion vector predictor mode (Inter Pred Mode). merge_flag = 0 and inter_affine_flag = 1 correspond to the subblock motion vector predictor mode (interaffine mode). merge_flag = 1, merge_subblock_flag = 0 and merge_trianlge_flag = 0 correspond to normal Merge Mode. merge_flag = 1, merge_subblock_flag = 0 and merge_trianlge_flag = 1 correspond to Triangle Merge Mode. merge_flag = 1, merge_subblock_flag = 1 correspond to the Affine Merge Mode.

POC

The Picture Order Count (POC) is a variable associated with the encoded picture and is set to a value that is incremented by one according to the output order of the pictures. The POC value allows you to distinguish whether the images are the same, to distinguish the sequential relationship of external images in the output order, or to get the distance between images. For example, it can be determined that two images having the same POC value are identical images. In the case where the POCs of two pictures have different values, the picture with the lower POC value can be determined as the picture that is output earlier. The difference between the POCs of two images indicates the distance between the images in the direction of the time axis.

First Embodiment

Next, a description will be given of the image encoding apparatus 100 and the image decoding apparatus 200 according to the first embodiment of the present invention.

In FIG. 1 shows a block diagram of an image encoding apparatus 100 according to the first embodiment. The image encoding apparatus 100 according to the embodiment includes a block separating unit 101, an inter prediction unit 102, an intra prediction unit 103, a decoded image memory 104, a prediction method determiner 105, a residual generating unit 106, an orthogonal transform/quantizer 107, an encoding unit 108 a bit string, an inverse quantizer/inverse orthogonal converter 109, a decoded image signal overlay unit 110, and an encoding information storage memory 111 .

The block splitter 101 recursively splits the input image to construct a coding block. The block dividing unit 101 includes: a quaternary dividing unit that divides a dividing target block in both the horizontal direction and the vertical direction; and a binary-ternary splitter that splits the target splitter in either the horizontal direction or the vertical direction. The block separating unit 101 sets the constructed coding block as the target coding unit, and supplies the image signal of the target coding unit to the inter prediction unit 102, the intra prediction unit 103, and the residual generating unit 106 . Further, the block splitter 101 supplies information indicative of the determined recursive split structure to the bit string encoding unit 108 . Next, detailed operation information of the block dividing unit 101 will be given.

The inter prediction block 102 performs inter prediction of the target coding block. The inter prediction unit 102 derives a plurality of inter prediction information candidates from the inter prediction information stored in the encoding information storage memory 111 and the decoded image signal stored in the decoded picture memory 104, selects an appropriate inter prediction mode from the plurality of obtained candidates, and supplies the selected mode prediction method and a predicted image signal corresponding to the selected inter prediction mode to the prediction method determiner 105 . Next, the detailed configuration and operation of the inter prediction block 102 will be given.

The intra prediction block 103 performs intra prediction for the target coding block. The intra prediction unit 103 refers to the decoded image signal stored in the decoded picture memory 104 as a reference sample, and performs intra prediction based on encoding information such as an intra prediction mode stored in the encoding information storage memory 111, and thereby generates a predicted image signal. In intra prediction, the intra prediction section 103 selects a suitable intra prediction mode from a plurality of intra prediction modes, and supplies the selected intra prediction mode and the selected predicted image signal corresponding to the selected intra prediction mode to the prediction method determiner 105. Next, the detailed configuration and operation of the intra prediction unit 103 will be described.

The decoded picture memory 104 stores decoded pictures generated by the decoded picture signal overlay unit 110 . The decoded picture memory 104 supplies the stored decoded pictures to the inter prediction unit 102 and the intra prediction unit 103 .

The prediction method determiner 105 evaluates each of intra prediction and inter prediction using encoding information, residual code amount, distortion amount between the predicted image signal and the target image signal, or the like, and thus determines the optimal prediction mode. In the case of intra prediction, the prediction method determiner 105 supplies intra prediction information such as an intra prediction mode to the bit string encoding unit 108 as encoding information. In the case of the inter prediction merge mode, the prediction method determiner 105 provides inter prediction information such as a merge index and information (sub-block merge flag) indicating whether the mode is a sub-block merge mode to the bit string encoding unit 108 as encoding information. In the case of an inter prediction motion vector predictor mode, the prediction method determiner 105 supplies inter prediction information such as an inter prediction mode, a motion vector predictor index, a reference index L0 or L1, a motion vector difference, or information indicating whether the mode is a subblock motion vector predictor mode. (sub-block motion vector predictor flag) to the bit string encoding block 108 as encoding information. The predictive method determiner 105 further supplies the determined encoding information to the encoding information storage memory 111 . The prediction method determiner 105 supplies the predicted image signal to the residual generation unit 106 and the decoded image signal overlay unit 110 .

The residual generation unit 106 generates a residual by subtracting the predicted image signal from the target image signal and supplies the generated residual to the orthogonal converter/quantizer 107.

The orthogonal transform/quantizer 107 performs orthogonal transformation and quantization of the residual according to the quantization parameter, and thereby generates an orthogonally transformed and quantized residual, and then supplies the generated residual to the bit string encoding unit 108 and the inverse quantizer/inverse orthogonal transform 109.

The bit string coding unit 108 encodes, in addition to the sequences, images, segments, and information in coding block blocks, the bit string coding unit 108 encodes encoding information corresponding to the prediction method determined by the prediction method determiner 105 . for each of the coding blocks. In particular, the bit string encoding unit 108 encodes the PredMode prediction mode for each of the encoding units. In the case where the prediction mode is inter prediction (MODE_INTER), the bit string encoding section 108 encodes coding information (inter prediction information) such as a flag for determining whether the mode is a merge mode, a sub-block merge flag, a merge index in the merge mode, an inter-picture prediction mode in non-merge modes, a motion vector predictor index, motion vector difference related information, and a sub-block motion vector predictor flag based on a prescribed syntax (bit string syntax rule), and thereby generates a first bit string. In the case where the prediction mode is intra-frame prediction (MODE_INTRA), the bit string encoding section 108 encodes a flag to determine whether the mode is an intra-block copy mode according to a prescribed syntax. In the case where the mode is intra-block copy, coding information (intra-prediction information) such as the merge index in the case of the merge mode, the block vector predictor index in the case of the non-merge mode, and the block vector difference are encoded according to the prescribed syntax. In the case where the mode is not intra-block copy, coding information (intra prediction information) such as the intra prediction mode is encoded according to a prescribed syntax. With the above encoding method, the first bit string is created. In addition, the bit string encoding unit 108 performs entropy encoding of the orthogonally transformed and quantized residual based on the prescribed syntax, and thus generates the second bit string. The bit string encoding unit 108 multiplexes the first bit string and the second bit string based on the prescribed syntax and outputs the bit stream.

The inverse quantizer/inverse orthogonal transformer 109 performs inverse quantization and inverse orthogonal transformation of the orthogonally transformed/quantized residual supplied from the orthogonal transformer/quantizer 107, and thus calculates the residual, and then supplies the calculated residual to the decoded image signal overlay unit 110.

The decoded image signal overlay section 110 superimposes the predicted image signal according to the determination of the residual prediction method determiner 105, which is subjected to inverse quantization/inverse orthogonal transform by the inverse quantizer/inverse orthogonal transform 109, thereby generating a decoded image, and stores the generated decoded picture in memory 104 decoded pictures. The decoded image signal overlay unit 110 may perform filtering processing to reduce distortion such as block distortion due to encoding of the decoded image, and may thereafter store the decoded image in the decoded image memory 104 .

The encoding information storage 111 stores encoding information such as a prediction mode (inter prediction or intra prediction) determined by the prediction method determiner 105 . In the case of inter prediction, encoding information is stored in encoding information storage 111, which includes inter prediction information such as a determined motion vector, reference list indexes L0 and L1, and a motion vector predictor candidate list based on history. In the case of the inter prediction merge mode, encoding information stored in the encoding information storage memory 111 includes, in addition to the above-described information, a merge index and inter prediction information including information indicating whether the mode is a sub-block merge mode. (subblock merge flag). In the case of the inter prediction motion vector predictor mode, encoding information stored in the encoding information storage memory 111 includes, in addition to the above information, inter prediction information such as an inter prediction mode, a motion vector predictor index, a motion vector difference, and information indicating whether the mode is a sub-block motion vector predictor mode (sub-block motion vector predictor flag). In the case of intra prediction, the encoding information stored in the encoding information storage memory 111 includes intra prediction information such as a specific intra prediction mode.

In FIG. 2 is a block diagram illustrating the configuration of an image decoding apparatus according to an embodiment of the present invention corresponding to the image encoding apparatus (FIG. 1). The image decoding apparatus according to the embodiment includes a bit string decoding section 201, a block splitting section 202, an inter prediction section 203, an intra prediction section 204, an encoding information storage memory 205, an inverse quantizer/inverse orthogonal converter 206, and a decoded signal overlay section 207. image and decoded image memory 208 .

Since the decoding process of the image decoding device in FIG. 2 corresponds to the decoding process performed by the image encoding apparatus in FIG. 1. Accordingly, each of the configurations of the encoding information storage memory 205, the inverse quantizer/inverse orthogonal transformer 206, the decoded image signal overlay unit 207, and the decoded image memory 208 in FIG. 2 respectively have a function corresponding to each of the configurations of the encoding information storage memory 111, the inverse quantizer/inverse orthogonal converter 109, the decoded image signal overlay unit 110, and the decoded image memory 104 of the image encoding apparatus (FIG. 1).

The bit stream supplied to the bit string decoding section 201 is divided based on a prescribed syntax rule. The bit string decoding unit 201 decodes the divided first bit string, and thus obtains a sequence, a picture, a segment, information in coding blocks, and coding information in coding blocks. Specifically, the bit string decoding section 201 decodes the prediction mode PredMode, which determines whether the prediction is inter-frame prediction (MODE_INTER) or intra-frame prediction (MODE_INTRA) in blocks of a coding block. In the case where the prediction mode is inter prediction (MODE_INTER), the bit string decoding section 201 decodes encoding information (inter prediction information) related to a flag that determines whether the mode is a merge mode, a merge index in the case of a merge mode, a sub-block merge flag and inter-picture prediction in the case of a motion vector predictor mode, a motion vector predictor index, a motion vector difference, a sub-block motion vector predictor flag, or the like. in accordance with the prescribed syntax, and then supplies encoding information (inter prediction information) to the encoding information storage memory 205 via the inter prediction section 203 and the block division section 202 . In the case where the prediction mode is intra-frame prediction (MODE_INTRA), the bit string decoding section 201 decodes a flag to determine whether the mode is intra-block copy mode. In the case where the mode is intra-block copy, coding information (intra-prediction information) such as the merge index in the case of the merge mode, the block vector predictor index in the case of the non-merge mode, and the block vector difference are decoded according to the prescribed syntax. In the case where the mode is not intra-block copy, the bit string decoding section 201 decodes coding information (intra prediction information) such as the intra prediction mode according to the prescribed syntax. With the above decoding method, the decoded encoding information (intra prediction information) is supplied to the encoding information storage memory 205 via the inter prediction unit 203 or the intra prediction unit 204 and through the block division unit 202 . The bit string decoding section 201 decodes the divided second bit string and calculates an orthogonally transformed/quantized residual, and then supplies the orthogonally transformed/quantized residual to the inverse quantizer/inverse orthogonal transform 206.

When the prediction mode PredMode of the target encoding block is inter prediction (MODE_INTER) and motion vector prediction mode, the inter prediction block 203 uses the encoding information of the already decoded image signal stored in the encoding information storage memory 205 to obtain a plurality of motion vector predictor candidates. The inter prediction block 203 then adds the plurality of received motion vector predictor candidates to the motion vector predictor candidate list described below. The inter prediction section 203 selects a motion vector predictor corresponding to the motion vector predictor index to be decoded and supplied by the bit string decoder 201 from the plurality of motion vector predictor candidates registered in the motion vector predictor candidate list. The inter prediction block 203 then calculates a motion vector based on the difference of the motion vectors decoded by the bit string decoding block 201 and the selected motion vector predictor, and stores the calculated motion vector in the encoding information storage memory 205 along with other encoding information. Here, the encoding information of a coding block to be set and stored includes a prediction mode PredMode, flags predFlagL0[xP][yP] and predFlagL1[xP][yP] indicating whether or not to use L0 prediction and L1 prediction, reference indices refIdxL0 [xP] [yP] and refIdxL1 [xP] [yP] L0 and L1; and vectors mvL0 [xP] [yP] and mvL1 [xP] [yP] of motion for L0 and L1. Here, xP and yP are indices indicating the position of the top left coding block sample in the picture. In the case where the prediction mode PredMode is inter prediction (MODE_INTER) and the inter prediction mode is L0 prediction (Pred_L0), a flag predFlagL0 indicating whether to use L0 prediction is set to 1, and a flag predFlagL1 indicating whether to use L1 prediction is set. to 0. In the case where the inter prediction mode is L1 prediction (Pred_L1), the flag predFlagL0 indicating whether to use L0 prediction is set to 0, and the flag predFlagL1 indicating whether to use L1 prediction is set to 1. In the case when the inter prediction mode is bidirectional prediction (Pred_BI), both the predFlagL0 flag indicating whether to use L0 prediction and the flag predFlagL1 indicating whether to use L1 prediction are set to 1. Further, when the prediction mode PredMode of the coding target block is the inter prediction mode prediction (MODE_INTER) and merge mode i, then it turns out the merge candidate. Using the encoding information of the already decoded encoding block stored in the encoding information storage memory 205, a plurality of merge candidates is output and registered in the merge candidate list described below. Then, a merge candidate corresponding to the merge index that is decoded by the bit string decoder 201 and provided is selected from a plurality of merge candidates registered in the merge candidate list, and then inter prediction information such as flags predFlagL0 [xP ] [yP] and predFlagL1 [xP ] [yP] indicating whether to use L0 prediction and L1 prediction of the selected merge candidate, reference indices refIdxL0 [xP] [yP] and refIdxL1 [xP] [yP] L0 and L1, and vectors mvL0 [xP] [yP] and mvL1 [xP] [yP] movements for L0 and L1 must be stored in the encoding information storage memory 205 . Here, xP and yP are indices indicating the position of the top left coding block sample in the picture. Next, a detailed description of the configuration and operation of the inter prediction block 203 will be given.

The intra prediction unit 204 performs intra prediction when the prediction mode PredMode of the target coding block is intra prediction (MODE_INTRA). The encoding information decoded by the bit string decoding section 201 includes an intra prediction mode. The intra prediction unit 204 generates a predicted image signal by intra prediction from a decoded image signal stored in the decoded image memory 208 according to the intra prediction mode in the coding information decoded by the bit string decoding unit 201 . Next, the intra prediction block 204 supplies the generated predicted image signal to the decoded image signal overlay block 207 . The intra prediction unit 204 corresponds to the intra prediction unit 103 of the image encoding apparatus 100, and thus performs processing similar to that of the intra prediction unit 103 .

The inverse quantizer/inverse orthogonal transform 206 performs inverse orthogonal transformation/inverse quantization of the orthogonally transformed/quantized residual decoded in the bit string decoding section 201, and thus obtains the inversely orthogonally transformed/inversely quantized residual.

The decoded image signal overlay unit 207 superimposes the image signal predicted by inter prediction by the inter prediction unit 203 or the predictive image signal predicted by intra prediction by the intra prediction unit 204 with a residual that has been inversely orthogonally transformed/inversely quantized by an inverse quantizer /inverse orthogonal converter 206, thereby decoding the decoded image signal. The decoded picture signal overlay unit 207 then stores the decoded picture signal that has been decoded in the decoded picture memory 208 . When storing the decoded image in the decoded image memory 208, the decoded image signal overlay section 207 may perform filtering processing on the decoded image to reduce block distortion or the like due to encoding, and may thereafter store the decoded image in the decoded image memory 208.

Next, the operation of the block dividing unit 101 in the image encoding apparatus 100 will be described. In FIG. 3 is a flowchart illustrating the operation of dividing an image into tree blocks and further dividing each of the tree blocks. First, the input image is divided into tree blocks of a given size (step S1001). Each of the tree blocks is scanned in a predetermined order, that is, in a raster scan order (step S1002), and the target tree block is internally divided (step S1003).

In FIG. 7 is a flowchart illustrating the detailed operation of the division process in step S1003. First, it is determined whether to divide the target block into four (step S1101).

In the case where it is determined that the target block is to be divided by four, the target block will be divided by four (step S1102). Each of the blocks obtained by dividing the target block is scanned in a Z-scan order, that is, in the order of top left, top right, bottom left, and bottom right (step S1103). In FIG. 5 shows an example of a Z-scan order, 601 in FIG. 6A illustrates an example in which the target block is divided into four. Numbers 0 to 3 out of 601 in FIG. 6A indicate the processing order. Then, the division process (FIG. 7) is recursively executed for each of the blocks divided in step S1101 (step S1104).

In the case where it is determined that the target block is not to be divided into four, the target block will be divided into two or three, namely, binary-ternary division will be performed (step S1105).

In FIG. 8 is a flowchart illustrating in detail the operation of the binary-ternary separation process in step S1105. First, it is determined whether binary-ternary separation will be performed for the target block, that is, whether any binary or ternary separation will be performed (step S1201).

In the case where it is not determined that binary-ternary separation is to be performed on the target block, that is, in the case where it is determined that the target block is not to be divided, the separation is completed (step S1211). That is, an additional recursive division process need not be performed on a block that has been divided by the recursive division process.

In the case where it is determined that binary-ternary division will be performed in the target block, it is further determined whether or not to divide the target block into two (step S1202).

In the case where it is determined that the target block is to be divided into two, it is further determined whether or not to split the target block in the top-bottom (vertical) direction (step S1203), and then, based on the result, the target block will be binary-divided in the top-bottom ( vertical) direction (step S1204), or the target block will be binary divided in the left-right (horizontal) direction (step S1205). As a result of step S1204, the target block is binary split in the up-down direction (vertical direction), as illustrated at 602 in FIG. 6b. As a result of step S1205, the target block is left-right binary split (horizontal direction), as illustrated in step 604 in FIG. 6d.

In step S1202, in the case where it is not determined that the target block is to be divided into two, that is, in the case where it is determined that the target block is to be divided into three, it is further determined whether the target block is to be divided into three parts: upper , middle and bottom (vertical direction) (step S1206). Based on the result, the target block is divided into three parts: top, middle and bottom parts (vertical direction) (step S1207) or left, middle and right parts (horizontal direction) (step S1208). As a result of step S1207, the target block is divided into three parts: top, middle, and bottom (vertical direction), as illustrated in step 603 in FIG. 6C. As a result of step S1208, the target block is divided into three parts: left, middle, and right (horizontal direction), as illustrated in step 605 in FIG. 6E.

After performing one of steps S1204, S1205, S1207 or S1208, each of the blocks obtained by dividing the target block is scanned in order from left to right and top to bottom (step S1209). Numerals 0 to 2 602 to 605 in FIG. 6B-6E indicate the processing order. For each of the partition blocks, the binary-ternary partitioning process in FIG. 8 (step S1210).

In the recursive block division described herein, the correctness of the division may be limited based on the number of divisions, the size of the target block, and the like. The information that limits the correctness of the division may be provided in a configuration in which information is not transmitted by a prior agreement between the encoder and the decoder, or in a configuration in which the encoder determines information for limiting the correctness of the division and writes the information in bit strings, thereby transmitting the information to the device for decoding.

When a particular block is split, the block before the split is called the parent box and each of the boxes after the split is called a child box.

Next, the functionality of the block splitter 202 in the device 200 for decoding an image will be described. The block splitter 202 splits the tree block using a processing procedure similar to the case of the block splitter 101 of the image encoding device 100 . Note that there is a difference, although the block separating unit 101 of the image encoding apparatus 100 determines the optimal shape of the block division by applying an optimization method such as estimating the optimal shape by image recognition or distortion rate optimization, the block separating unit 202 of the image decoding apparatus 200 determines a block division form by decoding the block division information recorded in the bit string.

9 illustrates the syntax (bit string syntax rules) related to block division according to the first embodiment. coding_quadtree() represents the syntax for the quad block process. multi_type_tree() represents the syntax for splitting a block into two or three. qt_split is a flag indicating whether the block should be split into four. In the case of a block split by four, the setting is qt_split = 1. In the case of no block split by four, the setting will be qt_split = 0. In the case of a block split by four (qt_split = 1), the split by four process will be performed recursively for each of blocks divided into four (coding_quadtree (0), coding_quadtree (1), coding_quadtree (2) and coding_quadtree (3), in which the arguments from 0 to 3 correspond to the numbers 601 in Fig.6A). In the case where split by four is not to be performed (qt_split = 0), the subsequent split is determined according to multi_type_tree(). mtt_split is a flag indicating whether additional splitting should be performed. In the case where additional splitting needs to be performed (mtt_split = 1), mtt_split_vertical which is a flag indicating whether splitting should be performed in the vertical or horizontal direction and mtt_split_binary which is a flag specifying whether binary splitting should be performed or ternary block division. mtt_split_vertical = 1 indicates a split in the vertical direction and mtt_split_vertical = 0 indicates a split in the horizontal direction. mtt_split_binary = 1 indicates that the block has undergone a binary split, and mtt_split_binary = 0 indicates that the block has been split into three. In the case where a block is to be split by binary code (mtt_split_binary = 1), the splitting process is performed recursively for each of the two split blocks (multi_type_tree (0) and multi_type_tree (1), in which the arguments from 0 to 1 correspond to numbers in 602 or 604 in Figs. 6B-6D). In the case where a block is to be split into three (mtt_split_binary = 0), the splitting process is performed recursively for each of the three split blocks (multi_type_tree (0), multi_type_tree (1) and multi_type_tree (2), in which 0 2 correspond to numbers 603 on Fig.6B or 605 in Fig.6E). multi_type_tree is recursively used until mtt_split = 0 results in a hierarchical block split.

Intra-frame prediction

The intra prediction method according to the embodiment is implemented in the intra prediction section 103 of the image encoding apparatus 100 shown in FIG. 1 and the intra prediction section 204 of the image decoding apparatus 200 shown in FIG. 2.

An intra prediction method according to an embodiment will be described with reference to the drawings. The intra-frame prediction method is implemented in any of the encoding and decoding processes in units of coding blocks.

Block 103 intra prediction on the side of the coding

FIG. 40 is a schematic diagram illustrating the detailed configuration of the intra prediction unit 103 of the image encoding apparatus 100 shown in FIG. 1. FIG. The normal intra prediction unit 351 generates a predicted image signal by normal intra prediction from decoded pixels adjacent to the coding target block, selects an appropriate intra prediction mode from a plurality of intra prediction modes, and supplies the selected intra prediction mode and the predicted image signal corresponding to the selected intra prediction mode, to the determinant 105 of the prediction method. 10A and 10B show examples of intra prediction. 10A illustrates a correspondence between a normal intra prediction prediction direction and an intra prediction mode number. For example, the "intra-prediction" mode 50 copies the reference samples in the vertical direction and thereby creates an intra-prediction image. Intra prediction mode 1 is a constant current mode in which all sample values of the target block are set equal to the average value of the reference samples. Intra prediction mode 0 is a planar mode in which 2D intra prediction is generated from reference samples in the vertical and horizontal directions. FIG. 10B shows an example of constructing an intra prediction image in the case of the intra prediction mode 40 . For each of the samples of the target block, the value of the reference sample is copied in the direction indicated by the intra prediction mode. In the case where the reference sample in the intra prediction mode is not at an integer position, the value of the reference sample is determined by interpolation from the values of the reference samples at adjacent integer positions.

The block copy prediction block 352 obtains a decoded region of the same image signal as the encoding target block, generates a predicted image signal in the block copy process, and supplies the predicted image signal to the prediction method determiner 105 . The detailed configuration and process performed by the intra-block copy predictor 352 will be described below.

Block 204 intra prediction on the side of the decoding

FIG. 41 is a schematic diagram illustrating a detailed configuration of the intra prediction unit 204 of the image decoding apparatus 200 shown in FIG. 2. FIG.

The normal intra prediction unit 361 generates a prediction signal by normal intra prediction from decoded pixels adjacent to the coding target block, selects an appropriate intra prediction mode from a plurality of intra prediction modes, and obtains the selected intra prediction mode and the predicted image signal corresponding to the selected intra prediction mode. This prediction signal is supplied to the decoded image superposition section 207 via the switch 364. The process performed by the normal intra prediction section 361 shown in FIG. 41 corresponds to the process performed by the normal intra prediction section 351 shown in FIG. description will be omitted.

The intra-block copy prediction block 362 obtains a decoded area of the same image signal as a target coding block, and obtains a predicted image signal by an intra-block copy process. This predicted image signal is supplied to the decoded image signal overlay section 207 via the switch 364. The detailed configuration and process performed by the intra-block copy prediction section 362 will be described below.

Interframe prediction

The inter prediction method according to the embodiment is implemented in the inter prediction unit 102 of the image encoding apparatus in FIG. 1 and in the inter prediction block 203 of the image decoding apparatus in FIG. 2.

Next, a description will be given of the inter prediction method according to the embodiment with reference to the drawings. The inter-frame prediction method is implemented in any of the encoding and decoding processes in coding blocks.

Block 102 inter-picture prediction on the side of the coding

In FIG. 16 is a diagram illustrating a detailed configuration of the inter prediction unit 102 of the image encoding apparatus in FIG. 1. The normal motion vector predictor mode obtaining unit 301 outputs a plurality of normal motion vector predictor candidates, selects a motion vector predictor, and calculates a motion vector difference between the selected motion vector predictor and the detected motion vector. The detected inter prediction mode, the reference index, the motion vector, and the calculated motion vector difference are the inter prediction information of the normal motion vector predictor mode. This inter prediction information is supplied to the inter prediction mode determiner 305 . Next, detailed configuration and processing information of the normal motion vector predictor mode acquisition unit 301 will be given.

The normal merge mode obtaining unit 302 outputs a plurality of normal merge candidates, selects a normal merge candidate, and obtains inter prediction information for the normal merge mode. This inter prediction information is supplied to the inter prediction mode determiner 305 . Next, detailed information on the configuration and processing of the normal merge mode acquisition block 302 will be given.

The sub-block motion vector predictor mode derivation unit 303 outputs a plurality of sub-block motion vector predictor candidates, selects a sub-block motion vector predictor, and calculates a motion vector difference between the selected sub-block motion vector predictor and the detected motion vector. The detected inter prediction mode, reference index, motion vector, and calculated motion vector difference are sub-block motion vector predictor mode inter prediction information. This inter prediction information is supplied to the inter prediction mode determiner 305 .

The sub-block merge mode obtaining unit 304 outputs a plurality of sub-block merge candidates, selects a sub-block merge candidate, and obtains sub-block merge mode inter-picture prediction information. This inter prediction information is supplied to the inter prediction mode determiner 305 .

In the inter prediction mode determiner 305, inter prediction information is determined based on the inter prediction information supplied from the normal motion vector predictor mode obtaining block 301, the normal merge mode obtaining block 302, the subblock motion vector predictor mode obtaining block 303, and the subblock merge mode obtaining block 304. The inter prediction information according to the determination result is supplied from the inter prediction mode determiner 305 to the motion compensation predictor 306 .

The motion compensation predictor 306 performs inter prediction on the reference picture signal stored in the decoded picture memory 104 based on the determined inter prediction information. Next, detailed configuration and processing information of the motion compensation predictor 306 will be given.

Block 203 inter-picture prediction on the side of the decoding

In FIG. 22 is a diagram illustrating a detailed configuration of the inter prediction unit 203 of the image decoding apparatus in FIG. 2.

The normal motion vector predictor mode obtaining unit 401 outputs a plurality of normal motion vector predictor candidates, selects a motion vector predictor, calculates an added value obtained by adding the selected motion vector predictor and the difference of the decoded motion vectors, and sets this added value as a motion vector. The decoded inter prediction mode, reference index, motion vector is the inter prediction information of the normal motion vector predictor mode. This inter prediction information is supplied to the motion compensation predictor 406 via the switch 408. Next, the configuration and processing of the normal motion vector predictor mode obtaining unit 401 will be described in detail.

The normal merge mode obtaining unit 402 outputs a plurality of normal merge candidates, selects a normal merge candidate, and obtains inter prediction information for the normal merge mode. This inter prediction information is supplied to the motion compensation prediction block 406 via the switch 408. The detailed configuration and processing of the normal fusion mode obtaining block 402 will be presented below.

The sub-block motion vector predictor mode obtaining unit 403 outputs a plurality of sub-block motion vector predictor candidates, selects a sub-block motion vector predictor, and calculates an added value obtained by adding the selected sub-block motion vector predictor and the decoded motion vector difference, and sets this added value as a motion vector. The decoded inter prediction mode, the reference index, and the motion vector are inter prediction information of the sub-block motion vector predictor mode. This inter prediction information is supplied to the motion compensation predictor 406 via the switch 408.

The sub-block merge mode obtaining unit 404 outputs a plurality of sub-block merge candidates, selects a sub-block merge candidate, and obtains sub-block merge mode inter-picture prediction information. This inter prediction information is supplied to the motion compensation predictor 406 via the switch 408.

The motion compensation predictor 406 performs inter prediction on the reference picture signal stored in the decoded picture memory 208 based on the determined inter prediction information. The detailed configuration and processing of the motion compensation predictor 406 is similar to the motion compensation predictor 306 on the encoding side.

Normal Motion Vector Predictor Mode (Normal AMVP) Obtaining Block

The normal motion vector predictor mode acquisition unit 301 in FIG. 17 includes a spatial motion vector predictor candidate deriving block 321, a temporal motion vector predictor candidate deriving block 322, a history-based motion vector predictor candidate deriving block 323, a motion vector predictor candidate replenisher 325, a normal motion vector determiner 326, a predictor candidate selector 327 motion vector subtractor 328 of the motion vector.

The normal motion vector predictor mode acquisition unit 401 in FIG. 23 includes a spatial motion vector predictor candidate derivation unit 421, a temporal motion vector predictor candidate derivation unit 422, a history-based motion vector predictor candidate derivation unit 423, a motion vector predictor candidate adder 425, a motion vector predictor candidate selector 426, and a vector adder 427 movement.

With reference to the flowcharts in FIG. 19 and FIG. 25, respectively, the processing procedures of the normal motion vector predictor mode acquisition unit 301 on the encoding side and the normal motion vector predictor mode acquisition unit 401 on the decoding side are described. In FIG. 19 is a flowchart illustrating a normal motion vector predictor mode acquisition processing procedure executed by the normal motion vector mode acquisition unit 301 on the encoding side. In FIG. 25 is a flowchart illustrating a normal motion vector predictor mode acquisition processing procedure executed by the normal motion vector mode acquisition unit 401 on the decoding side.

Normal Motion Vector Predictor Mode (Normal AMVP) Obtaining Block: Encoding Side

With reference to FIG. 19, a normal motion vector predictor mode acquisition processing procedure on the encoding side will be described. In the description of the processing procedure in FIG. 19, the word "normal" illustrated in FIG. 19 is omitted in some cases.

First, the normal motion vector determiner 326 detects the normal motion vector for each of the inter prediction modes and reference indexes (step S100 in FIG. 19).

Subsequently, for each of L0 and L1, the motion vector difference of the motion vector used in inter-picture prediction in the normal motion vector predictor mode is calculated (steps S101 to S106 in FIG. 19) in the spatial motion vector predictor candidate acquisition block 321, the candidate acquisition block 322 temporal motion vector predictor, a history-based motion vector predictor candidate derivation block 323 , a motion vector predictor candidate adder 325 , a motion vector predictor candidate selector 327 , and a motion vector subtractor 328 . Specifically, in the case where the target block's PredMode is inter prediction (MODE_INTER) and the inter prediction mode is L0 prediction (Pred_L0), the L0 motion vector predictor candidate list mvpListL0 is calculated. Subsequently, the motion vector predictor mvpL0 is selected and then the motion vector difference mvdL0 of the motion vector mvL0 L0 is calculated. In the case where the inter prediction mode of the target block is L1 prediction (Pred_L1), the L1 motion vector predictor candidate list mvpListL1 is calculated. Then the predictor mvpL1 of the motion vector is selected, and then the difference mvdL1 of the motion vectors of the motion vector mvL1 of the motion L1 is calculated. In the case where the inter prediction mode of the target block is bidirectional prediction (Pred_BI), both L0 prediction and L1 prediction are performed. The motion vector predictor candidate list mvpListL0 for L0 is calculated, and the motion vector predictor mvpL0 for L0 is selected, and then the motion vector difference mvdL0 of the motion vector mvL0 for L0 is calculated. Along with this calculation, a motion vector predictor candidate list mvpListL1 for L1 is calculated, and a motion vector predictor mvpL1 for L1 is calculated, and then the motion vector difference mvdL1 of the motion vector mvL1 for L1 is calculated.

The motion vector difference calculation process is performed for each of L0 and L1, in which the calculation process is a common process for both L0 and L1. Accordingly, L0 and L1 will be referred to as LX as a general procedure. In the process of calculating the motion vector difference L0, X, LX is set to 0, while in the process of calculating the motion vector difference L1, X, LX is set to 1. the difference of the motion vectors of one LX, the other list will be represented as LY.

In the case where the LX motion vector mvLX is used (YES in step S102 in FIG. 19), the LX motion vector predictor candidates are calculated, thereby compiling the LX motion vector predictor candidate list mvpListLX (Step S103 in FIG. 19). In the normal motion vector predictor mode derivation block 301, the spatial motion vector predictor candidate derivation block 321, the temporal motion vector predictor candidate derivation block 322, the history-based motion vector predictor candidate derivation block 323, and the motion vector predictor candidate filler 325, a plurality of motion vector predictor candidates are output. motion vectors and thus constitute a list mvpListLX of motion vector predictor candidates. Next, with reference to FIG. 19, a detailed processing procedure of step S103 will be described using the flowchart in FIG. twenty.

Next, the motion vector predictor candidate selector 327 selects the LX motion vector predictor mvpLX of the LX motion vector predictor candidate list mvpListLX (step S104 in FIG. 19). Here, one element (i-th element, starting from 0) in the motion vector predictor candidate list mvpListLX is represented as mvpListLX[i]. Each motion vector difference is calculated, which is the difference between the mvLX motion vector and each of the motion vector predictor candidates mvpListLX[i] stored in the motion vector predictor candidate list mvpListLX. For each of the elements (motion vector predictor candidates) of the motion vector predictor candidate list mvpListLX, the amount of code is calculated at the time of encoding these motion vector differences. Then, a motion vector predictor candidate mvpListLX[i] is selected that minimizes the amount of code for each of the motion vector predictor candidates among the individual elements registered in the motion vector predictor candidate list mvpListLX as the motion vector predictor mvpLX, and its index i is obtained. In the case of a plurality of motion vector predictor candidates having a minimum generated amount of code in the motion vector predictor candidate list mvpListLX, the motion vector predictor candidate mvpListLX [i] having index i in the mvpListLX motion vector predictor candidate list represented by a small number is selected as the optimal predictor mvpLX of the motion vector, and its index i is obtained.

Then, the motion vector subtractor 328 subtracts the selected LX motion vector predictor mvpLX from the LX motion vector mvLX and calculates the difference of the LX motion vectors mvdLX as in:

mvdLX = mvLX - mvpLX (step S105 in FIG. 19).

Normal motion vector predictor mode acquisition block (normal AMVP): decode side

Next, with reference to FIG. 25, a normal motion vector predictor mode processing procedure on the decoding side will be described. On the decoding side, a spatial motion vector predictor candidate deriving unit 421, a temporal motion vector predictor candidate deriving unit 422, a history-based motion vector predictor candidate deriving unit 423, and a motion vector predictor candidate enhancer 425 individually calculate the motion vectors used in inter-frame prediction of the normal predictor mode. a motion vector for each of L0 and L1 (steps S201-S206 in FIG. 25). Specifically, in the case where the prediction mode PredMode of the target block is inter prediction (MODE_INTER) and the inter prediction mode of the target block is L0 prediction (Pred_L0), the L0 motion vector predictor candidate list mvpListL0 is calculated. Next, a predictor of the motion vector mvpL0 is selected, and then the motion vector mvL0 of L0 is calculated. In the case where the inter prediction mode of the target block is L1 prediction (Pred_L1), the L1 motion vector predictor candidate list mvpListL1 is calculated. Then, the motion vector mvpL1 predictor is selected and the motion vector mvL1 L1 is calculated. In the case where the inter prediction mode of the target block is bidirectional prediction (Pred_BI), both L0 prediction and L1 prediction are performed. A motion vector predictor candidate list mvpListL0 of L0 is computed, and a motion vector predictor mvpL0 of L0 is selected, and then a motion vector mvL0 of L0 is computed. Along with this calculation, the motion vector predictor candidate list mvpListL1 for L1 is calculated, and the motion vector predictor mvpL1 for L1 is calculated, and then the motion vector mvL1 for L1 is calculated.

Similar to the encoding side, the decoding side performs motion vector calculation processing for each of L0 and L1, the processing being a common process for both L0 and L1. Accordingly, L0 and L1 will be referred to as LX as a general procedure. LX represents an inter prediction mode used for inter prediction of a target coding block. X is 0 in the process of calculating the motion vector L0 and X is 1 in the process of calculating the motion vector L1. Additionally, in the case where other lookup list information is mentioned instead of the same lookup list as LX to be calculated in the process of calculating the motion vector LX, the other lookup list will be represented as LY.

In the case where the LX motion vector mvLX is used (YES in step S202 in FIG. 25), the LX motion vector predictor candidates are calculated to compile the LX motion vector predictor candidate list mvpListLX (step S203 in FIG. 25). In the normal motion vector predictor mode derivation block 401, the spatial motion vector predictor candidate derivation block 421, the temporal motion vector predictor candidate derivation block 422, the history-based motion vector predictor candidate derivation block 423, and the motion vector predictor candidate filler 425, a plurality of vector predictor candidates are calculated. motion vectors and thus constitute a list mvpListLX of motion vector predictor candidates. Next, with reference to FIG. 25 and using the flowchart of FIG. 20, a detailed processing procedure of step S203 will be described.

Then, the motion vector predictor candidate selector 426 extracts the motion vector predictor candidate mvpListLX [mvpIdxLX] corresponding to the motion vector predictor index mvpIdxLX decoded and supplied by the bitstring decoder 201 from the motion vector predictor candidate list mvpListLX as the selected motion vector predictor mvpLX (step S204 at Fig. 25).

Then, the motion vector adder 427 sums the difference mvdLX of the LX motion vectors and the predictor mvpLX of the LX motion vector that are decoded and supplied by the bit string decoder 201, and calculates the motion vector mvLX of the LX motion as in:

mvLX = mvpLX + mvdLX (step S205 in FIG. 25).

Normal Motion Vector Predictor Mode Obtainer (Normal AMVP): Motion Vector Prediction Method

In FIG. 20 is a flowchart illustrating a processing procedure for obtaining the normal motion vector predictor mode having a function common to the normal motion vector predictor mode obtaining unit 301 of the image encoding apparatus and the normal motion vector predictor mode obtaining unit 401 of the image decoding apparatus according to the embodiment implementation of the present invention.

Each of the normal motion vector predictor mode obtaining block 301 and the normal motion vector predictor mode obtaining block 401 includes a motion vector predictor candidate list mvpListLX. The motion vector predictor candidate list mvpListLX has a list structure and includes a storage area that stores as elements a motion vector predictor index indicating a location in the motion vector predictor candidate list and a motion vector predictor candidate corresponding to the index. The motion vector predictor index number starts from 0, and the motion vector predictor candidates must be stored in the motion vector predictor candidate list mvpListLX storage area. In the present embodiment, it is assumed that the motion vector predictor candidate list mvpListLX can register at least two motion vector predictor candidates (as inter prediction information). In addition, the numCurrMvpCand variable indicating the number of motion vector predictor candidates registered in the motion vector predictor candidate list mvpListLX is set to 0.

Each of the spatial motion predictor candidate derivation blocks 321 and 421 extracts a motion vector predictor candidate from the blocks in the adjacent left side. This process outputs the motion vector predictor mvLXA with reference to the block inter-prediction information in the adjacent left side (A0 or A1 in FIG. 11), namely a flag indicating whether the motion vector predictor candidate, motion vector, reference index, or t can be used. .p., and adds the obtained mvLXA to the motion vector predictor candidate list mvpListLX (step S301 in FIG. 20). Note that X is 0 in L0 prediction and X is 1 in L1 prediction (similarly applies later). Next, motion vector predictor candidate derivation blocks 321 and 421 extract motion vector predictor candidates from the top neighbor block. This process outputs the mvLXB motion vector predictor with reference to the upper neighbor block inter prediction information (B0, B1, or B2 in FIG. 11), namely, a flag indicating whether the motion vector predictor candidate, the motion vector, the reference index, etc., can be used. .P. When the received mvLXA and the received mvLXB are not equal, the mvLXB is added to the motion vector predictor candidate list mvpListLX (step S302 in FIG. 20). The processes in steps S301 and S302 in FIG. 20 are presented as a general process, except that the positions and numbers of reference adjacent blocks are different, and the flag availableFlagLXN indicating whether a coding block motion vector predictor candidate can be used, and motion vector mvLXN, reference index refIdxN (N indicates A or B, similarly applied later) in these processes.

Then, each of the motion vector temporal predictor candidate derivation blocks 322 and 422 extracts a motion vector predictor candidate from a block in the picture having a temporal difference from the target picture. This process outputs an availableFlagLXCol flag indicating whether the motion vector predictor candidate of the image coding block having the time difference and the motion vector mvLXCol, the reference index refIdxCol and the lookup list listCol can be used, and adds mvLXCol to the motion vector predictor candidate list mvpListLX (step S303 in Fig. 20).

It should be noted that it is contemplated that the processes of temporal motion vector predictor candidate derivation blocks 322 and 422 may be omitted in Sequence (SPS), Picture (PPS), or Segment blocks.

Then, the history-based motion vector predictor candidate derivers 323 and 423 add the history-based motion vector predictor candidates registered in the history-based motion vector predictor candidate list HmvpCandList to the motion vector predictor candidate list mvpListLX. (Step S304 in FIG. 20). With reference to the flowchart in FIG. 29, next, a detailed description of the registration processing procedure in step S304 will be given.

Then, the motion vector predictor candidate fillers 325 and 425 add a motion vector predictor candidate having a predetermined value such as (0, 0) until the motion vector predictor candidate list mvpListLX is satisfied (S305 in FIG. 20).

Block receiving normal merge mode (normal merge)

Normal merge mode acquisition block 302 in FIG. 18 includes a spatial merge candidate derivation unit 341, a temporal merge candidate derivation unit 342, an average merge candidate derivation unit 344, a history-based merge candidate derivation unit 345, a merge candidate replenisher 346, and a merge candidate selector 347.

Normal merge mode acquisition block 402 in FIG. 24 includes a spatial merge candidate derivation unit 441, a temporal merge candidate derivation unit 442, an average merge candidate derivation unit 444, a history-based merge candidate derivation unit 445, and a merge candidate augmenter 446 and a merge candidate selector 447.

In FIG. 21 is a flowchart illustrating a procedure for obtaining a normal fusion mode having a function common to the normal fusion mode obtaining unit 302 of the image encoding apparatus and the normal fusion mode obtaining unit 402 of the image decoding apparatus according to an embodiment of the present invention.

Later in this document, the various processes will be described step by step. The following description is for the case where the slice_type of the segment is segment B, unless otherwise noted. However, the present invention can also be applied to the case of a P segment. Note that there is only L0 prediction (Pred_L0) as the inter prediction mode, L1 prediction (Pred_L1) or bidirectional prediction (Pred_BI) in the case where the slice_type of the segment is a P-segment. Accordingly, in this case, the process associated with L1 can be skipped.

The normal merge mode getter 302 and the normal merge mode getter 402 include a list of mergeCandList merge candidates. The mergeCandList has a list structure and includes a storage area that stores as elements a merge index indicating a location in the merge candidate list and a merge candidate corresponding to that index. The merge index number starts at 0, and the merge candidate is stored in the merge candidate list mergeCandList storage area. In post-processing, the merge candidate of merge index i registered in the mergeCandList of merge candidates will be represented by mergeCandList[i]. In the present embodiment, it is assumed that the merge candidate list mergeCandList can register at least six merge candidates (as inter prediction information). In addition, the numCurrMergeCand variable indicating the number of merge candidates registered in the mergeCandList of merge candidates is set to 0.

The spatial merge candidate derivation unit 341 and the spatial merge candidate derivation unit 441 output a spatial merge candidate of each of the blocks (B1, A1, B0, A0, B2 in FIG. 11) in the neighboring block of the target block in the order of B1, A1, B0, A0, and B2 from encoding information stored in either the encoding information storage memory 111 of the image encoding apparatus or the encoding information storage memory 205 of the image decoding apparatus, and then registers the obtained spatial merge candidates into the mergeCandList merge candidate list (step S401 in FIG. 21) . Here, N will be defined denoting one of B1, A1, B0, A0, B2, or a temporary merge candidate Col. Items to be output include an availableFlagN flag indicating whether inter-frame prediction information of block N can be used as a spatial merge candidate, reference index refIdxL0N L0 and reference index refIdxL1N L1 of spatial merge candidate N, prediction flag predFlagL0N L0 indicating whether L0 prediction should be performed, a prediction flag predFlagL1N L1 indicating whether L1 prediction should be performed, an L0 motion vector mvL0N, and an L1 motion vector mvL1N. However, since the merge candidate in the present embodiment is output without referring to the inter prediction information of the block included in the encoding target block, a spatial merge candidate using the inter prediction information of the block included in the encoding target block cannot be obtained.

Then, the temporary merge candidate obtaining unit 342 and the temporary merge candidate obtaining unit 442 output the temporary merge candidates from the images having the time difference, and register the obtained temporary merge candidates in the merge candidate list mergeCandList (step S402 in FIG. 21). The elements to be received include an availableFlagCol flag indicating whether a temporary merge candidate can be used, a predFlagL0Col L0 prediction flag indicating whether an L0 temporary merge candidate prediction should be performed, a L1 prediction flag predFlagL1Col indicating whether L1 should be performed. prediction, L0 motion vector mvL0Col and L1 motion vector mvL1Col.

Note that it is contemplated that the processes of temporally merge candidate derivation blocks 342 and 442 may be omitted in Sequence (SPS), Picture (PPS), or Segment blocks.

Next, the history-based merge candidate acquisition unit 345 and the history-based merge candidate acquisition unit 445 register the history-based motion vector predictor candidates registered in the history-based motion vector predictor candidate list HmvpCandList in the merge candidate list mergeCandList (step S403 in FIG. .21).

In the case where the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList is less than the maximum number of merge candidates MaxNumMergeCand, obtain a merge candidate based on history with the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList limited by the maximum number of merge candidates MaxNumMergeCand and then registered in the mergeCandList of merge candidates.

Then, the average merge candidate derivation unit 344 and the average merge candidate acquisition unit 444 output the average merge candidate from the mergeCandList of merge candidates, and add the resulting average merge candidate to the mergeCandList of merge candidates (step S404 in FIG. 21).

In the case where the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList is less than the maximum number of merge candidates MaxNumMergeCand, an average merge candidate is obtained with the number of merge candidates numCurrMergeCand registered in the merge candidate list mergeCandList which is limited to the maximum number of merge candidates MaxNumMergeCand and then register in the list mergeCandList merge candidates.

Here, the average merge candidate is a new merge candidate including a motion vector obtained by averaging the motion vectors of the first merge candidate and the second merge candidate registered in the merge candidate list mergeCandList for each of prediction L0 and prediction L1.

Then, in merge candidate adder 346 and merge candidate adder 446, in the case where the number numCurrMergeCand of merge candidates registered in the merge candidate list mergeCandList is less than the maximum number of merge candidates MaxNumMergeCand, an additional merge candidate with the number of merge candidates numCurrMergeCand registered in the list is obtained. mergeCandList of merge candidates limited by the maximum number of merge candidates MaxNumMergeCand and then registered in the merge candidate list mergeCandList (step S405 in FIG. 21). In the P-segment, a merge candidate having a motion vector of value (0, 0) and a prediction prediction mode L0 (Pred_L0) is added with the maximum number of merge candidates MaxNumMergeCand as an upper limit. In the B-segment, a merge candidate having a bidirectional prediction mode (Pred_BI) and a motion vector of value (0, 0) is added. The pivot index at the time the merge candidate was added is different from the pivot index that was already added.

Then, the merge candidate selector 347 and the merge candidate selector 447 select the merge candidate from among the merge candidates registered in the merge candidate list mergeCandList. The coding-side merge candidate selector 347 calculates a code amount and a distortion amount, and thus selects a merge candidate, and then supplies a merge index indicating the selected merge candidate and inter frame prediction information of the merge candidate to the motion compensation predictor 306 via the inter frame mode determiner 305 predictions. In contrast, the merge candidate selector 447 on the decoding side selects the merge candidate based on the decoded merge index, and supplies the selected merge candidate to the motion compensation predictor 406 .

Update Motion Vector Predictor Candidate List Based on History

Next, the method of initializing and updating the motion vector predictor candidate list HmvpCandList based on the history provided in the encoding information storage memory 111 on the encoding side and the encoding information storage memory 205 on the decoding side will be described in detail. In FIG. 26 is a flowchart illustrating a procedure for initializing/updating a motion vector predictor candidate list based on history.

In the present embodiment, the motion vector predictor candidate list HmvpCandList based on the history is updated in the encoding information storage memory 111 and the encoding information storage memory 205 . Alternatively, in the inter prediction block 102 and the inter prediction block 203, a history-based motion vector predictor candidate list updater may be provided to update the history-based motion vector predictor candidate list HmvpCandList.

In the segment header, the history-based motion vector predictor candidate list HmvpCandList is initialized. On the encoding side, the history-based motion vector predictor candidate list HmvpCandList is updated in the case that the prediction method determiner 105 selects the normal motion vector predictor mode or the normal merge mode. On the decoding side, the history-based motion vector predictor candidate list HmvpCandList is updated in the case where the prediction information decoded by the bit string decoding section 201 is the normal motion vector predictor mode or the normal merge mode.

The inter prediction information used at the time of performing inter prediction in the normal motion vector predictor mode or normal merge mode needs to be registered in the history-based motion vector predictor candidate list HmvpCandList as the inter prediction information candidate hMvpCand. The inter prediction information candidate hMvpCand includes a reference index refIdxL0 of L0 and a reference index refIdxL1 of L1, a prediction flag predFlagL0 indicating whether L0 prediction should be performed, a prediction flag predFlagL1 indicating whether L1 prediction should be performed, a motion vector mvL0 for L0, and motion vector mvL1 for L1.

In the case of having inter prediction information having the same value as an inter prediction information candidate hMvpCand among the elements (i.e., inter prediction information) registered in the history-based motion vector predictor candidate list HmvpCandList provided in the side encoding information storage memory 111 encoding and encoding information storage 205 on the decoding side, the element will be removed from the motion vector predictor candidate list HmvpCandList based on the history. In contrast, in the case of no inter prediction information having the same value as the inter prediction information candidate hMvpCand, the header element of the history-based motion vector predictor candidate list HmvpCandList will be deleted, and the inter prediction information candidate hMvpCand will be added to the end of the vector predictor candidate list HmvpCandList movements based on history.

The number of elements of the motion vector predictor candidate list HmvpCandList provided in the coding information storage 111 on the coding side and the coding information storage 205 on the decoding side of the present invention is set to six.

First, the history-based motion vector predictor candidate list HmvpCandList is initialized in segment blocks (step S2101 in FIG. 26). All elements of the history-based motion vector predictor candidate list HmvpCandList are cleared at the beginning of the segment, and the number NumHmvpCand (current number of candidates) of the history-based motion vector predictor candidates registered in the motion vector predictor candidate list HmvpCandList is set to 0.

Although the initialization of the history-based motion vector predictor candidate list HmvpCandList must be performed in segment blocks (the first segment coding block), the initialization may be performed in image blocks, tiles, or rows of the tree block.

Then, for each of the coding blocks in the segment, the following process of updating the motion vector predictor candidate list HmvpCandList based on the history is repeated (steps S2102 to S2107 in FIG. 26).

First of all, initial settings are made in the coding blocks. The identityCandExist flag indicating the presence of an identical candidate is set to FALSE, the removal target index removeIdx indicating removal of the target candidate is set to 0 (step S2103 in FIG. 26).

The existence of an inter prediction information candidate hMvpCand to be registered is determined (step S2104 in FIG. 26). In the case where the encoding side prediction method determiner 105 determines the normal motion vector predictor mode or the normal merge mode, or when the bit string decoding section 201 on the decoding side performs decoding as the normal motion vector predictor mode or the normal merge mode, the corresponding inter prediction information is set. as an hMvpC candidate of inter-frame prediction information and is subject to registration. In the case where the encoding-side prediction method determiner 105 determines the intra-prediction mode, the sub-block motion vector predictor mode, or the sub-block merge mode, or, in the case that the bit-string decoding section 201 on the decoding side performs decoding as the intra-prediction mode, the vector prediction mode no motion vector predictor candidate list HmvpCandList update process based on the history, and no inter prediction information hMvpCand candidate for registration will be performed. If there is no inter prediction information candidate hMvpCand to be registered, steps S2105 to S2106 will be skipped (NO in step S2104 in FIG. 26). If there is an inter prediction information candidate hMvpCand to be registered, the process of step S2105 and later will be executed (YES in step S2104 in FIG. 26).

It is then determined whether the individual elements of the history-based motion vector predictor candidate list HmvpCandList include an element (inter prediction information) having the same value as the inter prediction information candidate hMvpCand for registration, that is, whether an identical element exists (step S2105 in Fig. 26). In FIG. 27 is a flowchart of the same item confirmation processing procedure. In the case where the history-based motion vector predictor candidate number NumHmvpCand value is 0 (NO in step S2121 in FIG. 27), the history-based motion vector predictor candidate list HmvpCandList is empty and there is no identical candidate. Accordingly, steps S2122-S2125 in FIG. 27 will be skipped, which will complete the identical element confirmation processing procedure. In the case where the value of the history-based motion vector predictor candidate number NumHmvpCand is greater than 0 (YES in step S2121 in FIG. 27), the process of step S2123 will be repeated from the history-based motion vector predictor index hMvpIdx from 0 to NumHmvpCand-1 (steps S2122 -S2125 in Fig. 27). First, a comparison is made as to whether the 0-based element hMvpIdx-th HmvpCandList [hMvpIdx] in the history-based motion vector predictor candidate list is identical to the inter prediction information candidate hMvpCand (step S2123 in FIG. 27). In the case that they are identical (YES in step S2123 in FIG. 27), the identityCandExist flag indicating the existence of an identical candidate is set to TRUE, and the removal target index indicating the position of the element to be removed is set to the current value of the motion vector predictor index hMvpIdx based on history, and identical item confirmation processing will be completed. In the case where they are not identical (NO in step S2123 in FIG. 27), hMvpIdx is incremented by one. In a case where the motion vector predictor index based on history hMvpIdx is less than or equal to NumHmvpCand-1, processing is performed in step S2123 and later.

Returning to the flowchart of the algorithm in FIG. 26, a process of shifting and adding elements of the motion vector predictor candidate list HmvpCandList based on the history is performed (step S2106 in FIG. 26). Fig. 28 is a flowchart of the shift/add processing procedure of elements in the history-based motion vector predictor candidate list HmvpCandList in step S2106 in FIG. 26. First, it is determined whether to add a new element after deleting the element stored in the history-based motion vector predictor candidate list HmvpCandList, or to add a new element without deleting the element. Specifically, a comparison is made as to whether the identityCandExist flag indicating the existence of an identical candidate is TRUE or NumHmvpCand is equal to 6 (step S2141 in FIG. 28). In the case where one of the conditions under which the identityCandExist flag indicating the presence of an identical candidate is TRUE or that the current number of candidates NumHmvpCand is 6 is satisfied (YES in S2141 in FIG. 28), the item stored in the HmvpCandList history-based motion vector predictor candidates are removed and a new element is added afterwards. The initial value of index i is set to removeIdx + 1. The element shifting process in step S2143 is repeated from this initial value to NumHmvpCand. (steps S2142-S2144 in FIG. 28). By copying the elements of HmvpCandList[i] to HmvpCandList[i - 1], the elements are shifted forward (step S2143 in FIG. 28) and i is incremented (steps S2142-S2144 in FIG. 28). Then, the inter prediction information candidate hMvpCand is added to (NumHmvpCand-1) HmvpCandList [NumHmvpCand-1] counting from 0, which corresponds to the end of the history-based motion vector predictor candidate list (step S2145 in FIG. 28), and the list shift/add process HmvpCandList of motion vector predictor candidates based on history will be completed. On the contrary, in the case where none of the conditions under which the identityCandExist flag indicating the existence of an identical candidate is TRUE and that NumHmvpCand is equal to 6 is satisfied (NO in step S2141 in FIG. 28), the inter prediction information candidate hMvpCand will be added to the end of the history-based motion vector predictor candidate list without deleting an entry stored in the history-based motion vector predictor candidate list HmvpCandList (step S2146 in FIG. 28). Here, the end of the history-based motion vector predictor candidate list is NumHmvpCand-th HmvpCandList [NumHmvpCand], counted from 0. Moreover, NumHmvpCand is incremented by one, and the process of shifting and adding elements of the history-based motion vector predictor candidate list HmvpCandList is completed.

In FIG. 31 is a view illustrating an example of a process for updating a motion vector predictor candidate list based on history. In the case where a new element is to be added to the history-based motion vector predictor candidate list HmvpCandList in which six elements (inter prediction information) are already registered, the history-based motion vector predictor candidate list HmvpCandList is compared with the new inter prediction information in order of initial element (Fig. 31A). When the new element has the same value as the third element HMVP2 from the header of the history-based motion vector predictor candidate list HmvpCandList, the HMVP2 element is removed from the history-based motion vector predictor candidate list HmvpCandList, and the elements HMVP3-HMVP5 are shifted (copied) one by one. forward, and a new element is added to the end of the history-based motion vector predictor candidate list HmvpCandList (FIG. 31B) to complete updating the history-based motion vector predictor candidate list HmvpCandList (FIG. 31C).

The Process of Deriving Motion Vector Predictor Candidates Based on History

Next, a method for obtaining a history-based motion vector predictor candidate from the history-based motion vector predictor candidate list HmvpCandList will be described in detail. This corresponds to the processing procedure of step S304 in FIG. 20 regarding the general processing performed by the motion vector predictor candidate obtaining unit 323 based on the history of the normal motion vector predictor mode obtaining unit 301 on the encoding side and the motion vector predictor candidate obtaining unit 423 based on the history from the normal motion vector predictor mode obtaining unit 401 on the side. decoding. Fig. 29 is a flowchart illustrating a processing procedure for obtaining a motion vector predictor candidate based on history.

In the case where the current number of motion vector predictor candidates numCurrMvpCand is greater than or equal to the maximum number of elements of the motion vector predictor candidate list mvpListLX (here 2) or the number of motion vector predictor candidates NumHmvpCand based on history is 0 (NO in step S2201 in FIG. 29), the process of steps S2202-S2209 in FIG. 29 will be omitted, and the history-based motion vector predictor candidate acquisition process will be completed. In the case where the number numCurrMvpCand of the current motion vector predictor candidates is less than 2, which is the maximum number of elements of the motion vector predictor candidate list mvpListLX, and, in the case where the value of the number NumHmvpCand of the history-based motion vector predictor candidates is greater than 0 (YES in step S2201 in 29), the process of steps S2202-S2209 in FIG. 29.

Then, the process of steps S2203-S2208 in FIG. 29 is repeated until index i is 1 to the lower of 4, or up to the number of history-based motion vector predictor candidates numCheckedHMVPCand (steps S2202-S2209 in FIG. 29). In the case where the current number of motion vector predictor candidates numCurrMvpCand is greater than or equal to 2, which is the maximum number of elements of the motion vector predictor candidate list mvpListLX (NO in step S2203 in FIG. 29), process steps S2204 to S2209 in FIG. 29 will be omitted, and the history-based motion vector predictor candidate acquisition process will be completed. In the case where the current number of motion vector predictor candidates numCurrMvpCand is less than 2, which is the maximum number of elements in the motion vector predictor candidate list mvpListLX (YES in S2203 in FIG. 29), the process in S2204 and later in FIG. 29.

Then, the process in steps S2205 to S2207 is performed for the cases where Y is 0 and Y is 1 (L0 and L1) (steps S2204 to S2208 in FIG. 29). In the case where the current number of motion vector predictor candidates numCurrMvpCand is greater than or equal to 2, which is the maximum number of elements of the motion vector predictor candidate list mvpListLX (NO in step S2205 in FIG. 29), the process of steps S2206 to S2209 in FIG. 29 will be omitted, and the history-based motion vector predictor candidate acquisition process will be completed. In the case where the current number of motion vector predictor candidates numCurrMvpCand is less than 2, which is the maximum number of elements in the motion vector predictor candidate list mvpListLX (YES in S2205 in FIG. 29), the process in S2206 and later in FIG. 29.

Then, in the case that the history-based motion vector predictor candidate list HmvpCandList includes an element having the same reference index as the encoding/decoding target motion vector reference index refIdxLX and which is different from any element of the motion vector predictor list mvpListLX (YES in step S2206 in FIG. 29) and the number numCurrMvpCand of current motion vector predictor candidates is incremented by one. In the case where there is no element in the history-based motion vector predictor candidate list HmvpCandList that has the same reference index as the reference index refIdxLX of the encode/decode target motion vector and is different from any element of the motion vector predictor list mvpListLX (NO in step S2206 in Fig. 29), the additional process in step S2207 will be skipped.

The process of steps S2205-S2207 in FIG. 29 is performed for both L0 and L1 (steps S2204-S2208 in FIG. 29). Index i is incremented by one, and when index i is less than or equal to any of the smaller value 4 or the number of history-based motion vector predictor candidates NumHmvpCand, the process of step S2203 and later will be repeated (steps S2202-S2209 in FIG. 29).

The process of obtaining merger candidates based on history

The following is a detailed description of the method for obtaining a history-based merge candidate from the history-based merge candidate list HmvpCandList, the process procedure of step S404 in FIG. 21, which is the general process of block 345 for obtaining merge candidates based on history from block 302 for obtaining normal merge mode on the encoding side, and block 445 for obtaining merge candidates based on the history of block 402 for obtaining normal merge mode on the decoding side. In FIG. 30 is a flowchart illustrating a history-based merge candidate acquisition processing procedure.

First, an initialization process is performed (step S2301 in FIG. 30). The elements from 0 to (numCurrMergeCand -1) of isPruned[i] are each set to FALSE and numOrigMergeCand is set to numCurrMergeCand of the number of elements registered in the current merge candidate list.

Then, the hMvpIdx index initial value is set to 1, and the sub-process from step S2303 to step S2310 in FIG. 30 is repeated from this initial value to NumHmvpCand (steps S2302-S2311 in FIG. 30). When the number numCurrMergeCand of elements registered in the current merge candidate list is not less than or equal to (MaxNumMergeCand-1 maximum merge candidates), merge candidates have been added to all elements in the merge candidate list. Accordingly, the history-based merge candidate acquisition process will be completed (NO at S2303 in FIG. 30). In the case where the number numCurrMergeCand of items registered in the current merge candidate list is less than or equal to (the maximum number of merge candidates MaxNumMergeCand-1), the process of step S2304 and later will be executed. sameMotion is set to FALSE (step S2304 in FIG. 30). Then, the initial value of index i is set to 0, and the process of steps S2306 and S2307 in FIG. 30 is executed from this initial value to numOrigMergeCand-1 (steps S2305-S2308 in FIG. 30). A comparison is made to see if the (NumHmvpCand-hMvpIdx) element of HmvpCandList [NumHmvpCand-hMvpIdx] counted from 0 in the history-based motion vector predictor candidate list is the same value as the ith element of mergeCandList [i] counted from from 0 in the merge candidate list (step S2306 in FIG. 30).

The merge candidates are determined to have the same value in the case where all constituent elements (inter prediction mode, reference index, motion vector) of the merge candidate have the same value. In the case where the merge candidates have the same value and isPruned[i] is set to FALSE (YES in step S2306 in FIG. 30), sameMotion and isPruned[i] will be set to TRUE (step S2307 in FIG. 30). In case the values do not match (NO in step S2306 in FIG. 30), the process in step S2307 will be skipped. After completion of repetition processing from step S2305 to step S2308 in FIG. 30, a comparison is made as to whether the sameMotion is FALSE (step S2309 in FIG. 30). In the case where sameMotion is FALSE (YES at step S2309 in FIG. 30), that is, (NumHmvpCand-hMvpIdx), the HmvpCandList [NumHmvpCand-hMvpIdx] item, which is 0-based in the history-based motion vector predictor candidate list, is not present in mergeCandList and hence the element HmvpCandList [NumHmvpCand - hMvpIdx], which is the (NumHmvpCand - hMvpIdx) element 0-based in the history-based motion vector predictor candidate list, is added to mergeCandList [ngeumCurr], which is numCurrMergeCand in the merge candidate list, and numCurrMergeCand is incremented (step S2310 in FIG. 30). The hMvpIdx index is incremented by one (step S2302 in FIG. 30), and the process of steps S2302-S2311 in FIG. 30 is repeated.

When all elements in the history-based motion vector predictor candidate list are confirmed or adding merge candidates to all elements in the merge candidate list is completed, the history-based merge candidate derivation process ends.

The Process of Obtaining Average Merge Candidates

The following is a detailed description of the average merge candidate acquisition method, the processing procedure of step S403 in FIG. 21, which is the common process of the average merge candidate obtaining block 344 of the normal merge mode obtaining block 302 on the encoding side, and the average merge candidate obtaining block 444 of the normal merge mode obtaining block 402 on the decoding side. In FIG. 38 is a flowchart illustrating an average merge candidate acquisition processing procedure.

First, an initialization process is performed (step S1301 in FIG. 38). The variable numOrigMergeCand is set to the number of numCurrMergeCand elements registered in the current list of merge candidates.

A sequential scan is then performed, starting at the beginning of the merge candidate list, to determine two pieces of motion information. An index i indicating the first motion information is set such that the index i = 0, and an index j indicating the second motion information is set such that the index j=1. (Steps S1302-S1303 in FIG. 38). When the number numCurrMergeCand of elements registered in the current merge candidate list is not less than or equal to (the maximum number of merge candidates MaxNumMergeCand-1), merge candidates have been added to all elements in the merge candidate list. Accordingly, the process of obtaining merge candidates based on the history will be completed (step S1304 in FIG. 38). In the case where the number numCurrMergeCand of elements registered in the current merge candidate list is less than or equal to (the maximum number of merge candidates MaxNumMergeCand-1), the process of step S1305 and later will be executed.

It is determined whether both the i-th mergeCandList [i] motion information of the merge candidate list and the j-th mergeCandList [j] motion information of the merge candidate list are valid (step S1305 in FIG. 38). In the case where both are invalid, the process proceeds to the next element without obtaining an average merge candidate mergeCandList[i] and mergeCandList[j]. In the case where the condition that both mergeCandList[i] and mergeCandList[j] are invalid is not satisfied, the following process is repeated with X set to 0 and 1 (steps S1306 to S1314 in FIG. 38).

Determining whether the LX prediction mergeCandList [i] is valid (step S1307 in FIG. 38). In the case where the LX prediction mergeCandList[i] is valid, a determination is made whether the LX prediction mergeCandList[j] is valid (step S1308 in FIG. 38). In the case where the LX prediction of mergeCandList[j] is valid, that is, in the case where both the LX prediction of mergeCandList[i] and the LX prediction of mergeCandList[j] are valid, obtain an LX prediction motion vector by averaging the LX prediction motion vector of mergeCandList[ i] and the LX prediction motion vector of mergeCandList[j], and an LX prediction average merge candidate having a LX prediction reference index of mergeCandList[i] will be obtained to set the LX prediction to averageCand, and the LX prediction averageCand will be valid (step S1309 on Fig. 38). At step S1308 in FIG. 38, in the case where the LX prediction mergeCandList[j] is invalid, that is, in the case where the LX prediction mergeCandList[i] is valid and the LX prediction mergeCandList[j] is invalid, the LX prediction motion vector mergeCandList[i] and the average merge candidate will be obtained. LX prediction having a reference index to be set as LX prediction averageCand and LX prediction meanCand will be valid (step S1310 in FIG. 38). In the case where the LX mergeCandList [i] prediction is invalid in step S1307 in FIG. 38, a determination is made as to whether or not the LX mergeCandList [j] prediction is valid (step S1311 in FIG. 38). In the case where the LX prediction of mergeCandList[j] is valid, i.e., in the case where the LX prediction of mergeCandList[i] is invalid and the LX prediction of mergeCandList[j] is valid, the LX prediction motion vector mergeCandList[j] and the LX average merge candidate will be obtained prediction having a reference index to be set as LX prediction averageCand, and LX prediction averageCand will be valid (step S1312 in FIG. 38). At step S1311 in FIG. 38, in the case where LX prediction mergeCandList[j] is invalid, that is, in the case where LX prediction mergeCandList[i] and LX prediction mergeCandList[j] are both invalid, LX prediction averageCand will be invalid (step S1312 in FIG. 38).

Thus obtained average merge candidate L0 prediction, L1 prediction or BI prediction is added to numCurrMergeCand-th mergeCandList [numCurrMergeCand] of the merge candidate list, and numCurrMergeCand is incremented by one (step S1315 in FIG. 38). This completes the process of obtaining an average merge candidate.

An average fusion candidate is obtained by averaging over each of the horizontal motion vector component and the vertical motion vector component.

Motion Compensation Prediction Process

The motion compensation prediction block 306 obtains the position and size of the block currently undergoing encoding prediction processing. Further, the motion compensation predictor 306 obtains inter prediction information from the inter prediction mode determiner 305 . The reference index and the motion vector are derived from the obtained inter prediction information, and the reference image indicated by the reference index in the decoded image memory 104 is shifted from the same position as the image signal of the prediction target block to be subjected to prediction processing by the amount of the motion vector. After the shift, an image signal of this position is obtained, and then a prediction signal is generated.

In the case where the prediction is made from the signal reference picture, for example, when the inter prediction mode in the inter prediction is L0 prediction or L1 prediction, the prediction signal obtained from one reference picture is set as the motion compensation prediction signal. In the case where prediction is performed from two reference pictures, for example, when the inter prediction mode is BI prediction, a weighted average of the prediction signals obtained from the two reference pictures is set as the motion compensation prediction signal. The received motion compensation prediction signal is supplied to the prediction method determiner 105 . Here, the ratio of the weighted averages in bidirectional prediction is set to 1:1. Alternatively, another ratio of weighted averages can be used. For example, the weighted averaging factor may be set such that the shorter the picture interval between the prediction target picture and the reference picture, the higher the weighted averaging factor. The calculation of the weighted average coefficient can also be performed using a lookup table between the combination of image intervals and the weighted average coefficients.

The motion compensation predictor 406 has a function similar to the motion compensation predictor 306 on the encoding side. The motion compensation predictor 406 obtains inter-picture prediction information from the normal motion vector predictor mode obtainer 401, the normal merge mode obtainer 402, the sub-block motion vector predictor mode obtainer 403, and the sub-block merge mode obtainer 404 through the switch 408. The motion compensation predictor 406 supplies the received motion compensation prediction signal to the decoded image signal overlay unit 207 .

Interframe prediction mode

The process of performing prediction from one reference picture is defined as unidirectional prediction. Unidirectional prediction performs L0 prediction or L1 prediction using one of two reference pictures registered in the L0 or L1 directory lists.

In FIG. 32 shows a case of unidirectional prediction in which the reference picture (RefL0Pic) L0 occurs at the time before the target picture (CurPic). Fig. 33 illustrates a case of unidirectional prediction in which the prediction reference picture L0 occurs at a time after the target picture. Similarly, unidirectional prediction can be performed by replacing the reference picture prediction L0 in FIG. 32 and FIG. 33 per prediction reference picture (RefL1Pic) L1.

The process of performing prediction from two reference pictures is defined as bidirectional prediction. Bidirectional prediction performs prediction expressed as BI prediction using both L0 prediction and L1 prediction. Fig. 34 illustrates a case of bidirectional prediction in which L0 reference picture prediction occurs at that point before the target picture and L1 reference picture prediction occurs at that point after the target picture. Fig. 35 illustrates a case of bidirectional prediction in which the prediction reference picture L0 and the prediction reference picture L1 occur at the same time as the target picture. Fig. 36 illustrates a case of bidirectional prediction in which a reference frame for prediction L0 and a reference picture L1 prediction occur at a time after the target picture.

Thus, it is possible to use prediction without limiting the relationship between L0/L1 prediction type and time, such that L0 for the past direction and L1 for the future direction. Moreover, bidirectional prediction can perform each of the L0 predictions and L1 predictions using the same reference picture. Determining whether to perform motion compensation prediction in unidirectional prediction or bidirectional prediction is performed based on information (eg, a flag) indicating whether to use L0 prediction and whether to use L1 prediction.

Reference index

In an embodiment of the present invention, it is possible to select optimal reference pictures from a plurality of reference pictures in motion compensation prediction to improve the accuracy of motion compensation prediction. Thus, a reference picture used in motion compensation prediction should be used as a reference index, and the reference index is encoded in the bitstream along with the motion vector difference.

Motion Compensation Process Based on Normal Motion Vector Predictor Mode

As illustrated in the inter prediction block 102 on the encoding side in FIG. 16, in the case where the inter prediction mode determiner 305 has selected the inter prediction information by the normal motion vector predictor mode obtaining unit 301, the motion compensation predictor 306 obtains this inter prediction information from the inter prediction mode determiner 305, and obtains the inter prediction mode, the reference index and a motion vector of the target block, and thus generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determiner 105 .

Similarly, as illustrated in the inter prediction block 203 on the decoding side in FIG. 22, in the case where the switch 408 is connected to the normal motion vector predictor mode obtaining unit 401 during the decoding process, the motion compensation prediction unit 406 acquires inter prediction information through the normal motion vector predictor mode acquisition unit 401, and obtains the inter prediction mode, the reference index, and the motion vector of the target block, and thus generates a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal overlay unit 207 .

Motion compensation process based on normal fusion mode

As illustrated in the inter prediction block 102 on the encoding side in FIG. 16, in the case where inter prediction information has been selected in the inter prediction mode determiner 305 by the normal merge mode obtaining section 302, the motion compensation prediction section 306 obtains this inter prediction information from the inter prediction mode determiner 305, and obtains the inter prediction mode, the reference index, and the vector motion of the target block, thereby generating a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determiner 105 .

Similarly, as illustrated in the inter prediction block 203 on the decoding side in FIG. 22, in the case where the switch 408 is connected to the normal merge mode obtaining unit 402 during the decoding process, the motion compensation prediction unit 406 obtains inter prediction information by the normal merge mode obtaining unit 402, and obtains the inter prediction mode, reference index, and motion vector of the target block, thereby generating a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal overlay unit 207 .

Motion Compensation Process Based on Subblock Motion Vector Predictor Mode

As illustrated in the inter prediction block 102 on the encoding side in FIG. 16, in the case where inter prediction mode determiner 305 has selected inter prediction information by sub-block motion vector predictor mode obtaining unit 303, motion compensation predictor 306 obtains this inter prediction information from inter prediction mode determiner 305 and obtains inter prediction mode, reference index and a motion vector of the target block, thereby generating a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determiner 105 .

Similarly, as illustrated in the inter prediction block 203 on the decoding side in FIG. 22, in the case where the switch 408 is connected to the sub-block motion vector predictor mode obtaining block 403 during the decoding process, the motion compensation prediction block 406 acquires inter-frame prediction information through the sub-block motion vector predictor mode obtaining block 403, and outputs the inter-prediction mode, the reference index, and the motion vector of the target block, thereby generating a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal overlay unit 207 .

Motion Compensation Process Based on Subblock Merge Mode

As illustrated in the inter prediction block 102 on the encoding side in FIG. 16, in the case where the inter prediction mode determiner 305 has selected the inter prediction information by the sub-block merge mode obtaining unit 304, the motion compensation predictor 306 obtains this inter prediction information from the inter prediction mode determiner 305 and outputs the inter prediction mode, the reference index, and the vector motion of the target block, thereby generating a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the prediction method determiner 105 .

Similarly, as illustrated in the inter prediction block 203 on the decoding side in FIG. 22, in the case where the switch 408 is connected to the sub-block merge mode obtaining block 404 during the decoding process, the motion compensation prediction block 406 obtains inter-prediction information through the sub-block merge mode obtaining block 404, and outputs the inter-prediction mode, reference index, and motion vector of the target block. , thereby generating a motion compensation prediction signal. The generated motion compensation prediction signal is supplied to the decoded image signal overlay unit 207 .

Motion Compensation Process Based on Affine Transform Prediction

In the normal motion vector predictor mode and the normal merge mode, motion compensation using an affine model can be used based on the following flags. The following flags are reflected in the following flags based on the inter prediction conditions determined by the inter prediction mode determiner 305 in the encoding process, and are encoded in the bitstream. In the decoding process, it is determined whether to perform motion compensation using the affine model based on the following flags in the bitstream.

sps_affine_enabled_flag indicates whether motion compensation using the affine model can be used in inter-prediction. When sps_affine_enabled_flag is 0, the process is suppressed to not perform motion compensation with the affine model in sequence blocks. Moreover, inter_affine_flag and cu_affine_type_flag are not conveyed in the coding unit (CU) syntax of the video coding sequence. When sps_affine_enabled_flag is equal to 1, affine model motion compensation can be used in the video coding sequence.

sps_affine_type_flag indicates whether motion compensation using the 6-parameter affine model can be used in inter-prediction. When sps_affine_type_flag is 0, the process is suppressed to not perform motion compensation using the 6-parameter affine model. Moreover, cu_affine_type_flag is not conveyed in the CU syntax of the video coding sequence. When sps_affine_type_flag is equal to 1, motion compensation based on a 6-parameter affine model can be used in the video coding sequence. If there is no sps_affine_type_flag, it should be 0.

In the case of P or B segment decoding, when inter_affine_flag is equal to 1 in the current CU, motion compensation using an affine model is used to generate a motion compensation prediction signal of the current CU. When inter_affine_flag is 0, no affine model is used for the current CU. If there is no inter_affine_flag, it must be 0.

In the case of P or B segment decoding, when cu_affine_type_flag is equal to 1 in the current CU, motion compensation using a 6-parameter affine model is used to generate a motion compensation prediction signal of the current CU. When cu_affine_type_flag is 0, motion compensation using a four-parameter affine model is used to generate a motion compensation prediction signal of the CU currently being processed.

The reference index and motion vector are output in blocks of sub-blocks in motion compensation based on the affine model. Accordingly, a motion-compensated prediction signal is generated using a reference index and a motion vector for sub-block processing.

The four-parameter affine model is a mode in which a sub-block motion vector is obtained from the four parameters of the horizontal component and the vertical component of each of the motion vectors of the two control points, and motion compensation is performed in blocks of sub-blocks.

For section 0 motion compensation prediction, unidirectional prediction motion information, indicated by triangle merge index 0, is used. The unidirectional prediction motion information indicated by triangle fusion index 1 is used for section 1 motion compensation prediction. for section 2 motion compensation prediction.

Intrablock Copy (IBC)

The actual intra-block copy reference area will be described with reference to FIG. FIG. 39A shows an exemplary case where a valid reference area is determined under the condition that a coding tree block unit is used as an intra-block copy standard block. Blocks 500, 501, 502, 503, and 504 in FIG. 39A are coding tree blocks, and block 504 among them is the target coding tree block. Block 505 is a target coding block. The coding tree blocks are processed in the order 500, 501, 502, 503, and 504. In this case, all three coding tree blocks 501, 502, and 503 that are processed immediately before the coding tree block 504, including the target coding tree block 505, are set as valid reference regions. target block 505 coding. The coding tree block that is processed before the coding tree block 501 and the regions included in the coding tree block 504, including the target encoding block 505, are all set as invalid reference regions regardless of whether processing is completed before the target encoding block 505.

FIG. 39B shows an exemplary case where a valid reference area is determined under the condition that a unit obtained by dividing a coding tree block into four units is used as an intra-block copy standard unit. 39B, blocks 515 and 516 are coding tree blocks, and block 516 is the target coding tree block. The coding tree block 515 is divided into blocks 506, 507, 508, and 509, and the coding tree block 516 is divided into blocks 510, 511, 512, and 513. Reference numeral 514 denotes a target coding block. Intrablock copy building blocks 506, 507, 508, 509, 510, 511, 512, and 513 are processed. In this case, three intrablock copy building blocks 508, 509, 510 are processed immediately before intrablock copy building block 511, including encoding target block 514 , are set as the actual reference areas of the target encoding block 514 . The coding tree block processed before the intra-block copy standard 508 and the entire area included in the intra-block copy standard 511 including the target encoding block 514 are all set as invalid reference areas regardless of whether processing is completed before the target encoding block 514.

Reference area memory space

The memory space for storing the processed image of the reference area will be described below. 56 is a schematic diagram illustrating the memory space of a reference area when a coding tree block unit is used as an intra-block copy standard block. Blocks 500, 501, 502, 503, 504, and 505 shown in FIG. 56A are coding tree blocks. The coding tree block 503 is the target coding tree block. The coding tree block 500, the coding tree block 501, the coding tree block 502 are processed coding tree blocks and correspond to the reference area of the target coding tree block 503. Encoding tree blocks 504 and 505 are raw coding tree blocks. On figa gaps 600, 601, 602 and 603 represent the memory space. Memory space 600 stores the processed tree coding block image 500. Similarly, memory space 601 stores the processed tree coding block 501 image, while memory space 602 stores the processed coding tree block 502 image. The memory space 603 sequentially stores the processed images in accordance with the processing of the target block 503 of the coding tree. When the processing of the coding tree block 503 is completed, then the processing of the coding tree block 504 is started.

The process performed when processing of the coding tree block 503 is completed will be described using FIG. 56B. At this point, coding tree block 504 will be the target coding tree block. In addition, coding tree block 503 will be a reference region, and coding tree block 500 will no longer be a reference region. At this stage, the processed image of the coding tree block 503 has been stored in the memory space 603 that sequentially stored the processed image of the coding tree block 503. Since the coding tree block 500 is no longer the coding reference area of the tree block 504, the processed image of the coding tree block 500 stored in the memory space 600 will be waste information. Accordingly, the processed images of the target coding tree block 504 will be sequentially stored in the memory space 600 . When the processing of the coding tree block 504 is completed, then the processing of the coding tree block 505 begins.

Similarly, an example of completing processing of the coding tree block 504 is shown in FIG. 56C. At this point, coding tree block 505 will be the target coding tree block, and coding tree block 504 will be the reference region. Block 501 of the coding tree is no longer a reference area. The processed images of the target block 504 of the coding tree are sequentially stored in the memory space 601 . When processing of coding tree block 505 is completed, processing of the next coding tree block is started.

Fig. 57 is a schematic diagram illustrating the memory space of a reference area when a valid reference area is determined under the assumption that a unit is obtained by quartering a coding tree block as an intra-block copy standard block. On figa 510 and 511 shows blocks of the coding tree. Encoding tree block 510 is a processed coding tree block, and coding tree block 511 is a target coding tree block. Code tree block 510 includes blocks 500, 501, 502, and 503. Code tree block 511 includes blocks 504, 505, 506, and 507. Block 504 is the target block. Blocks 501, 502 and 503 are processed blocks and correspond to the reference area of the target block 504. Blocks 505, 506, 507 are raw blocks. The spaces 600, 601, 602, and 603 in FIG. 57A are memory spaces. The memory space 603 stores the processed image of the block 501. Similarly, the processed image of the block 502 is stored in the memory space 602. The processed image of the block 503 is stored in the memory space 603. The processed images are sequentially stored in the memory space 600 according to the processing of the target block 504. When processing of block 504 is completed, then processing of block 505 begins.

The process performed at the completion of block 504 processing will be described using FIG. 57B. At this point, block 505 will be the target block of the coding tree. Also, block 504 will be a reference area, and block 501 will no longer be a reference area. At this stage, the processed image of the block 504 is stored in the memory space 600, in which the processed image of the block 504 is sequentially stored. , will be unnecessary information. Accordingly, the processed images of the target coding tree block 505 will be sequentially stored in the memory space 601 . When processing of block 505 is completed, processing of block 506 of the coding tree then begins.

The process performed at the completion of block 504 processing will be described using Fig. 57C. At this point, block 506 will be the target block. In addition, block 505 will be a reference region, and coding tree block 502 will no longer be a reference region. Similar to the above example, the processed images of the target block 506 of the coding tree will be sequentially stored in the memory space 602.

The process performed at the completion of block 505 processing will be described with reference to FIG. 57D. At this point, block 507 will be the target block. In addition, block 506 will be a reference region, and coding tree block 503 will no longer be a reference region. Similar to the above example, the processed images of the target block 507 of the coding tree will be sequentially stored in the memory space 603 .

Intrablock Predictive Copy: Encoding Side

The encoding-side intra-block copy copy processing procedure will be described with reference to FIG.

First, the block vector detector 375 detects the block vector mvL (step S4500 in FIG. 44). Then, an IBC spatial block vector candidate derivation block 371, a history-based block vector IBC predictor candidate derivation block 372, an IBC block vector predictor candidate enhancer 373, an IBC block vector predictor candidate selector 376, a block vector subtractor 378 calculate the block vector difference for the block vector used in the block vector predictor mode (steps S4501-S4503 in FIG. 44).

The block vector predictor candidates are computed to construct the block vector candidate list mvpList (step S4501 in FIG. 44). In intra-block copy prediction block 352, an IBC spatial block vector candidate derivation block 371, a history-based block vector IBC candidate derivation block 372, and an IBC block vector predictor candidate augmenter 373 obtain a plurality of block vector predictor candidates and compile a block vector predictor candidate list mvpList. The detailed procedure of the processing performed in step S4501 in FIG. 44 will be described below using the flowchart in FIG.

Then, the IBC block vector predictor candidate selector 376 selects the block vector predictor mvpL from the block vector predictor candidate list mvpListL (step S4502 in FIG. 44). Each block vector difference is computed, which is the difference between the block vector mvL and each of the block vector predictor candidate list mvpListL[i] stored in the block vector predictor candidate list mvpListL. The amount of code during encoding of these block vector differences is computed for each of the elements of the block vector predictor candidate list mvpListL. Then, a block vector predictor candidate mvpListL[i] that minimizes the amount of code for each of the block vector predictor candidates among the individual elements registered in the block vector predictor candidate list mvpListL is selected as the block vector predictor mvpL, and its index i is obtained. In the case where there are a plurality of block vector predictor candidates having the minimum amount of code generated in the block vector predictor candidate list mvpListL, the block vector predictor candidate mvpListL[i] having index i in the block vector predictor candidate list mvpListL represented by a small number is selected in as the optimal predictor of the block vector mvpL, and its index i is obtained.

The block vector subtractor 378 then subtracts the selected block vector predictor mvpL from the block vector mvL and thus computes the block vector difference mvdL as:

mvdL = mvL - mvpL (step S4503 in FIG. 44).

Intrablock Predictive Copy: Decoding Side

Next, the block vector normal predictor mode processing procedure on the decoding side will be described with reference to FIG. On the decoding side, an IBC spatial block vector predictor candidate deriver 471, a history-based block vector IBC candidate deriver 472, and an IBC block vector predictor enhancer 473 calculate a block vector used in the block vector predictor mode (steps S4600 to S4602 in FIG. 45). In particular, the block vector predictor candidate list mvpListL is calculated, the block vector predictor mvpL is selected, and then the block vector mvL is calculated.

The block vector predictor candidates are computed to construct a block vector predictor candidate list mvpListL (step S4601 in FIG. 45). In intra-block copy prediction block 362, an IBC spatial block vector candidate derivation block 471, a history-based block vector IBC candidate derivation block 472, and an IBC block vector predictor augmenter 473 calculate a set of block vector predictor candidates and compile a list mvpListL of block vector predictor candidates. Description of the detailed processing procedure performed in step S4601 in FIG. 45 has been omitted. Then, the block vector predictor candidate selector 476 extracts the block vector predictor candidate mvpListL[mvpIdxL] corresponding to the block vector predictor index mvpIdxL decoded and provided by the bitstring decoder 201 from the block vector predictor candidate list mvpListL as the selected block predictor vector mvpL( step S4601 in Fig. 45). Then, the block vector adder 478 adds the difference mvdL of the block vectors and the block vector predictor mvpL decoded and provided by the bit string decoder 201, and calculates (step S4602 in FIG. 45) the block vector mvL as follows:

mvL = mvpL + mvdL

Block vector prediction mode: block vector prediction method

Fig. 47 is a flowchart illustrating a prediction processing procedure of an intra-block copy mode acquisition process having a function in common with an intra-block copy predictor 352 of a moving picture coding apparatus and an intra-block copy predictor 362 of a moving image decoder according to an embodiment of the present. inventions.

Each of the intra-block copy predictor 352 and the intra-block copy predictor 362 includes a block vector predictor candidate list mvpListL. The block vector predictor candidate list mvpListL has a list structure and includes a storage area that stores as elements a block vector predictor index indicating a location in the list of block vector predictor candidates and a block vector predictor candidate corresponding to that index. The block vector predictor index number starts from 0, and the block vector predictor candidates shall be stored in the block vector predictor candidate list mvpListL storage area. In the present embodiment, it is assumed that the block vector predictor candidate list mvpListL can register three block vector predictor candidates. In addition, the numCurrMvpIbcCand variable indicating the number of block vector predictor candidates registered in the block vector predictor candidate list mvpListL is set to 0.

IBC spatial block vector candidate derivation blocks 371 and 471 obtain a block vector predictor candidate from an adjacent block on the left side (step S4801 in FIG. 47). In this process, the availableFlagLA flag indicating whether the block vector predictor candidate of the adjacent block (A0 or A1) on the left side can be used and the mvLA block vector are obtained, and the resulting mvLA will be added to the block vector predictor candidate list mvpListL. Next, IBC spatial block vector candidate derivation blocks 371 and 471 obtain the block vector predictor candidate from the neighboring block (B0, B1 or B2) on the upper side (step S4802 in FIG. 47). This process obtains an availableFlagLB flag indicating whether the neighbor block motion vector predictor candidate on the upper side can be used, and the mvLB block vector. When mvLA and mvLB are not equal, the mvLB block vector will be added to the block vector predictor candidate list mvpListL. The processes in steps S4801 and S4802 in FIG. 47 are shown as a general process, except that the positions and numbers of reference neighboring blocks are different. These processes receive an availableFlagLN flag indicating whether a block vector predictor candidate for a coding block can be used, and a motion vector mvLN (N indicates A or B, similarly applied hereinafter).

Then, the history-based IBC block vector candidate derivers 372 and 472 add the history-based block vector candidates registered in the history-based block vector candidate list HmvpIbcCandList to the block vector predictor candidate list mvpListL. (Step S4803 in FIG. 47). For the details of the registration processing procedure, in step S4803, it is sufficient to perform an operation similar to the operation shown in the flowchart of FIG. the history-based list HmvpIbcCandList of history-based candidate block vectors is replaced, and thus their description will be omitted.

Then, the IBC block vector predictor fillers 373 and 473 add a block vector having a predetermined value such as (0, 0) until the block vector predictor candidate list mvpListL is satisfied (S4804 in FIG. 47).

In-block copy-and-merge mode receive block

The intra-block copy prediction block 352 shown in FIG. a reference area boundary correction block 381 , an IBC merge candidate selector 374 , and an IBC prediction mode determiner 377 .

The intra-block copy prediction block 362 of FIG. reference position, block 481 correction of the boundary of the reference area and block 477 block copy.

FIG. 46 is a flowchart illustrating a process of obtaining the intra-block copy mode in the merge mode having a function in common with the intra-copy predictor 352 of the moving picture encoding apparatus and the intra-copy predictor 362 of the moving picture decoding apparatus according to an embodiment of the present invention. .

The intra-block copy predictor 352 and the intra-block copy predictor 362 include a list of mergeIbcCandList in-block copy candidates in merge mode. The list mergeIbcCandList of in-block merge-copy candidates has a list structure and includes: a storage area in which a merge index is stored as elements indicating a location within the in-block merge copy candidate; and the in-block merge-copy candidate corresponding to the index. The merge index number starts at 0, and the in-block merge-mode copy candidate is stored in the merge-in-block copy candidate list storage area mergeIbcCandList. In subsequent processes, the merge index merge candidate registered in the merge mode in-block copy merge candidate list mergeIbcCandList will be represented as mergeIbcCandList[i]. In the present embodiment, it is assumed that the merge candidate list mergeCandList allows at least three in-block copy candidates to be registered in merge mode. Moreover, the variable numCurrMergeIbcCand, which indicates the number of in-block merge copy candidates registered in the mergeIbcCandList of in-block merge copy candidates, has a value of 0.

The IBC spatial block vector candidate deriving unit 371 and the IBC spatial block vector candidate deriving unit 471 obtain spatial merging candidates A and B of neighboring blocks on the left and upper sides of the target block from the encoding information stored in the encoding information storage memory 111 of the encoding device. of the moving picture or in the memory 205 for storing encoding information of the moving picture decoding device, and then adding the obtained spatial merge candidates to the mergeIbcCandList of intra-block copy merge candidates (step S4701 in FIG. 46). In this document, N will be defined denoting one of the spatial merging candidates A and B. An availableFlagN flag and a block vector mvL are obtained indicating whether the intra-block copy prediction information of block N can be used as a spatial block vector merge candidate N. However, in the present embodiment, a block vector merge candidate is obtained without reference to other coding blocks included in the block, including the target coding block. Accordingly, a spatial block vector merge candidate included in a block including the target coding block will not be obtained.

Then, the history-based IBC block vector candidate derivation unit 372 and the history-based IBC block vector candidate derivation unit 472 add the history-based block vector predictor candidates registered in the history-based block vector predictor candidate list HmvpIbcCandList to the intra-block copy candidate list mergeIbcCandList in the merge mode (step S4702 in FIG. 46). In the present example, in the case where the block vector already added to mergeIbcCandList and the block vector of history-based block vector predictor candidate have the same value, no addition to mergeIbcCandList will be performed.

Then, in the case where the number of merge candidates numCurrMergeIbcCand registered in the merge mode in-block copy candidate list mergeIbcCandList is less than the maximum number of in-block merge candidates numCurrMergeIbcCand, the IBC block vector predictor pad 373 and the IBC block vector predictor pad 473 obtain an additional intra-block merge candidate, thus that the number of merge candidates numCurrMergeIbcCand registered in the merge mode in-block copy candidate list mergeIbcCandList is within the upper limit of the maximum number of in-block merge candidates numCurrMergeIbcCand, and the resulting candidate is added to the in-block merge mode copy candidate list mergeIbcCandList3 (FIG. 46). A block vector with a value of (0,0) will be added to the list of mergeIbcCandList in-block copy candidates within the upper limit of the maximum number of in-block merge candidates numCurrMergeIbcCand.

Then, the IBC merge candidate selector 374 and the IBC merge candidate selector 474 select one of the intra-block merge candidates registered in the merge mode intra-block copy candidate list mergeIbcCandList (step S4704 in FIG. 46). The IBC merge candidate selector 374 receives the decoded image at the reference position from the decoded image memory 104, calculates the code amount and distortion amount, thereby selecting the merge candidate, and then supplies the merge index indicating the selected intra-block merge candidate to the IBC prediction mode determiner 377. The IBC prediction mode determiner 377 calculates the code amount and the amount of distortion, thereby selecting the merge mode, and then supplies the result to the prediction method determiner 105 . On the other hand, the IBC merge candidate selector 474 on the decoding side selects an intra-block merge candidate based on the decoded merge index, and then passes the selected intra-block merge candidate to the reference position corrector 480 .

Then, the reference position correction block 380 and the reference position correction block 480 execute the reference position correction process on the intra-block merge candidate (step S4705 in FIG. 46). The details of the processes of the reference position correction block 380 and the reference position correction block 480 will be described below.

Then, the reference area boundary correction unit 381 and the reference area boundary correction unit 481 execute the reference area boundary correction process on the intra-block merge candidate (step S4706 in FIG. 46). Details of the processes performed by the reference position correction unit 381 and the reference position correction unit 481 will be described below.

The block copy unit 477 receives the decoded image at the reference position from the decoded image memory 208, and transmits the obtained image to the decoded image signal overlay unit 207 . Here, the block copy block 477 copies the luminance component and the color difference component.

The above block vector mvL represents the luminance block vector. When the color difference format is 420, the mvC block vector of the color difference is obtained as follows:

mvC = ((mvL >> (3 + 2)) * 32

The above equation will process each of the x and y components of mvC.

Reference position correction block

FIG. 48 is a flowchart illustrating a process performed by the reference position correction block 380 and the reference position correction block 480. FIG. This document assumes that the unit of the standard intra-block copy unit is a coding tree unit (CTU) and its size is not 128×128 samples.

First, the upper left and lower right positions of the reference block are calculated (S6001). The reference block refers to the block referenced by the target encoding block using the block vector. When it is determined in the reference block that the top left position has coordinates (xRefTL, yRefTL) and the bottom right position has coordinates (xRefBR, yRefBR), the following expression will be obtained:

(xRefTL, yRefTL) = (xCb + (mvL[0] >> 4), yCb + (mvL[1] >> 4))

(xRefBR, yRefBR) = (xRefTL + cbWidth - 1, yRefTL + cbHeight - 1)

In this document, the position of the target encoding block has coordinates (xCb, yCb), the block vector is (mvL[0], mvL[1]), the width of the target encoding block is cbWidth, and the height is cbHeight.

It is then determined whether the CTU size is 128×128 samples (S6002). Herein, since the size is not 128×128 samples (S6002: No), the top left and bottom right positions of the area that can be referred to are calculated (S6003). When the top left position (xAvlTL, yAvlTL) and bottom right position (xAvlBR, yAvlBR) are in the area that can be referenced, the individual positions will be obtained as follows:

NL = Min(1, 7-CtbLog2SizeY) - (1 << ((7-CtbLog2SizeY) << 1))

(xAvlTL, yAvlTL) = (((xCb >> CtbLog2SizeY) + NL) << CtbLog2SizeY,

(yCb >> CtbLog2SizeY) << CtbLog2SizeY))

(xAvlBR, yAvlBR) = (((xCb >> CtbLog2SizeY) << CtbLog2SizeY) -1,

(((yCb >> CtbLog2SizeY) + 1) << CtbLog2SizeY) -1)

In this document, the CTU size is CtbLog2SizeY.

It is then determined whether the reference position of the reference block in the x direction is smaller than the upper left portion of the area that can be referred to. When the determination is false (S6004: No), the process advances to the next process (S6006). On the contrary, when the determination is true (S6004: Yes), the reference position in the x direction will be corrected according to the upper left portion of the area (S6005) that can be referred to.

Fig. 49 is a schematic diagram illustrating a state of reference position correction. Reference numeral 6001 denotes a target coding tree block, 6002 denotes a target coding unit, and 6003 denotes an area that can be referred to. Now, assuming that the reference block r2 is located at position 6011, the reference position in the x direction is smaller than the upper left portion of the referenced area (S6004: Yes). Thus, the reference position is corrected before position 6012 under the condition that xRefTL = xAvlTL (S6005). In this document, since xRefBR = xRefTL + cbWidth-1 is kept as in S6001, xRefBR will also be adjusted along with the xRefTL adjustment. With this reference position correction, the block vector mvL[0] can be corrected. That is, the correction is performed as follows:

mvL[0] = (xAvlTL - xCb) << 4.

This satisfies xRefTL = xAvlTL, which allows the reference position to be corrected.

Thus, in the case where the reference block is located outside the area that can be referred to, the correction of the reference position will allow the reference block to be referenced.

It is now assumed that some of the block vectors in the list of block vector candidates compiled by intra-block copy predictor 352 are outside the region that can be referenced. In the case where the reference position is not subject to correction, these block vectors will not be referenced, disallowing the addition of these block vectors to candidates for the IBC merge mode. In contrast, in the case where the reference position is to be adjusted in the present invention, all of the block vectors in the compiled list of block vector candidates will be within the area that can be referenced. This will make all block vectors linkable, allowing all block vectors to be added to candidates for IBC merge mode. Thus, the IBC merge mode selector 374 can select the optimal prediction mode from each of the IBC merge mode candidates corresponding to all block vectors, resulting in improved coding efficiency.

It is now assumed that some of the block vectors in the list of block vector candidates compiled by intra-block copy predictor 362 are outside the region that can be referenced. In the case where the reference position is not to be adjusted, these block vectors will not be accessible, which will disable IBC merge mode decoding using these block vectors. In an encoder that is not the present invention, the merge index indicates the IBC merge mode, provided that these block vectors act as unencoded. However, such a merge index may be coded to generate a bitstream due to an operation failure or the like. In other cases, a portion of the bitstream may be lost due to packet loss or the like, and such a merge index may be the result of decoding. When attempting to decode such an incomplete bitstream, there is a possibility that decoded image memory is accessed at the wrong position when attempting to access outside of the area that can be referenced. Thus, the decoding result differs depending on the decoding apparatus, or the decoding process is stopped. In contrast, in the case where the reference position is to be adjusted in the present invention, all of the block vectors in the compiled list of block vector candidates will be within the region that can be referenced. Thus, even when such an incomplete bitstream is decoded, the reference position will be adjusted within the area that can be referred to, allowing it to be a reference. Thus, the reference position correction makes it possible to guarantee the memory access range. This leads to the same decoding result by the decoding device, which allows the decoding process to continue, which in turn leads to an increase in the stability of the decoding device.

In addition, in the case where the block vector is to be corrected when correcting the reference position, the target is the luminance block vector. In this document, a color difference block vector is computed from a luminance block vector. That is, the luminance block vector correction also corrects the color difference block vector. Thus, it is not necessary to correct the reference position for the color difference again. The amount of processing can be reduced compared to the case where no block vector correction is performed and it becomes necessary to determine whether a reference is possible based on both brightness and color difference.

In addition, in the case where the block vector needs to be corrected when correcting the reference position, the corrected block vector will be stored in the encoding information storage memory 111 or the encoding information storage memory 205 as the block vector of the encoding target block. That is, the adjusted reference position and the position indicated by the block vector are the same. Herein, storing a decoding result in decoded picture memory sometimes includes a deblocking filtering process. In this filtering process, the filter power is controlled by the difference between the block vectors of the two blocks before the block boundary. Compared with the case where the block vector is uncorrected and the corrected reference position differs from the position indicated by the block vector, additional corresponding filter power can be obtained, which results in improved coding efficiency.

It is then determined whether the reference position of the reference block in the y direction is smaller than the upper left portion of the area that can be referred to (S6006). When the determination is false (S6006: No), the process advances to the next process (S6008). On the contrary, when the determination is true (S6006: Yes), the reference position in the y direction will be corrected according to the upper left portion of the area (S6007) that can be referred to.

Now, assuming that the reference block r4 is located at position 6021, the reference position in the y direction is smaller than the upper left portion of the referenced area (S6006: Yes). Thus, the reference position is corrected before position 6022, provided that yRefTL=yAvlTL (S6007). In this document, since yRefBR = yRefTL + cbHeight-1 is performed as in S6001, yRefBR will also be corrected along with the yRefTL correction. With this correction of the reference position, the block vector mvL [1] can be corrected. That is, the correction is performed as follows:

mvL[1] = (yAvlTL - yCb) << 4

This satisfies yRefTL = yAvlTL, which allows the reference position to be corrected.

It is then determined whether the reference position of the reference block in the x direction is larger than the lower right portion of the area that can be referred to (S6008). When the determination is false (S6008: No), the process advances to the next process (S6010). On the contrary, when the determination is true (S6008: Yes), the reference position in the x direction will be corrected according to the lower right portion of the referenced area (S6009).

Now, assuming that the reference block r7 is located at position 6031, the reference position in the x direction is larger than the lower right portion of the referenced area (S6008: Yes). Thus, the reference position is corrected before position 6032 under the condition that xRefBR = xAvlBR (S6009). In this document, since xRefBR = xRefTL + cbWidth-1 is performed as in S6001, that is, xRefTL = xRefBR-(cbWidth-1) is performed, xRefTL will also be adjusted along with the xRefBR correction. With this reference position correction, the block vector mvL[0] can be corrected. That is, the correction is performed as follows:

mvL[0]=(xAvlBR-(xCb+cbWidth-1)) << 4

This satisfies xRefBR = xAvlBR, which allows the reference position to be corrected.

It is then determined whether the reference position of the reference block in the y direction is larger than the lower right portion of the area that can be referred to (S6010). When the determination is false (S6010: No), the process ends. On the contrary, when the determination is true (S6010: Yes), the reference position in the y direction will be adjusted according to the lower right portion of the referenced area (S6011).

Now, assuming that the reference block r5 is located at position 6041, the reference position in the y direction is larger than the lower right portion of the referenced area (S6010: Yes). Thus, the reference position is corrected before position 6042, provided that yRefBR = yAvlBR (S6011). In this document, since yRefBR = yRefTL + cbHeight-1 is performed as in S6001, i.e., yRefTL = yRefBR-(cbHeight-1) is performed, yRefTL will also be adjusted along with the yRefBR correction. With this correction of the reference position, the block vector mvL [1] can be corrected. That is, the correction is performed as follows:

mvL[1] = (yAvlBR-(yCb+cbHeitght-1)) << 4

This satisfies yRefBR = yAvlBR, which allows the reference position to be corrected.

Herein, the case where the reference block r1 is located at position 6051 will be described. In this case, the reference position in the x direction is adjusted in the same way as when the reference block is r2. In addition, similar to the case where the reference block is r4, the reference position in the y direction is corrected. As a result, reference block r1 will be located at position 6052 within the area that can be referenced.

In the case where the reference block r3 is located at position 6061, where the reference block r6 is located at position 6062, or where the reference block r8 is located at position 6063, the reference position in each of the x and y directions is adjusted similarly to the above case. As a result, each anchor block is located within the area that can be referenced.

The above procedure terminates the process for the case where the CTU size is not 128×128 samples. On the contrary, in the case where the CTU size is 128×128 samples (S6002: Yes), the top left and bottom right positions are calculated when the area that can be referred to is rectangular (S6012).

Fig. 50 is a schematic diagram illustrating the top left and bottom right positions when the area that can be referred to is rectangular. In the case shown in FIG. 50A, the target coding tree block 6101 is divided into four parts, and the target coding block 6102 is located at the top left of the divided positions. At this stage, the referenced area has an inverted L-shape, such as the hatched portion at 6103. In the case where the referenced area is rectangular, the range will be the range of the rectangular area 6103. In the case where the reference area is rectangular, and when the top left of the reference block is defined as (xRefTL, yRefTL) and its bottom right is defined as (xRefBR, yRefBR), the following expression will be obtained:

offset[4] = {0, 64, 128, 128}

NL=-offset[3-blk_idx], NR=offset[blk_idx]

(xAvlTL, yAvlTL) = ((xCb >> CtbLog2SizeY) << CtbLog2SizeY + NL,

(yCb >> CtbLog2SizeY) << CtbLog2SizeY))

(xAvlBR, yAvlBR) = (((xCb >> CtbLog2SizeY) << CtbLog2SizeY)-1 + NR,

(((yCb >> CtbLog2SizeY) + 1) << CtbLog2SizeY)-1).

Here, blk_idx is an index indicating the position of the target coding block. In the case where the target coding block is located at the top left position after the division into four parts of the target block of the coding tree, the tuning parameter will be: blk_idx = 0. Similarly, in the case where the target coding blocks are located at the top right, bottom left, and bottom right positions, blk_idx will be set to 1, 2, and 3. FIG. 50A is a schematic diagram illustrating the case where blk_idx = 0. Similarly, FIGS. 50B-50D are schematic diagrams illustrating cases where blk_idx = 1 – blk_idx = 3, respectively.

Then, the reference position of the non-rectangular reference area is corrected (S6013). FIG. 51 is a flowchart illustrating a process for correcting the reference position of a portion of an area that can be referenced and is not rectangular. First, the upper left position of the area that can be referred to is calculated (S6021). Since the area that can be referred to is the shaded area in FIG. 50, there are two upper left positions 6111 and 6112 except when blk_idx = 3. When these two points have coordinates (X1, Y1), (X2 , Y2), respectively, positions are calculated as follows:

offset[4] = {64, 128, 64, 0}, NL = offset[blk_idx]

(X1, Y1) = (xAvlTL, yAvlTL + 64)

(X2, Y2) = (xAvlTL + NL, yAvlTL)

Then, it is determined (S6022) whether the reference position is to be corrected according to the upper left portion of the referenced area. In this determination, it is determined that it will be true in the case where blk_idx = 3 is not satisfied and the reference block is located in an area smaller than X2 and Y1 (S6022: Yes). In case of a false determination (S6022: No), the process advances to the next process (S6026).

It is then determined whether the difference between the reference block and the referenced area in the x direction is smaller than the difference between the reference block and the referenced area in the y direction (S6023). In the case where the determination is true (S6023: Yes), the reference position in the x direction will be corrected (S6024). On the contrary, in the case where the determination is false (S6023: No), the reference position in the y direction will be corrected (S6025).

52A is a schematic diagram illustrating the state of the reference position correction in steps S6024 and S6025. Now, blk_idx = 0. Assuming that the reference block r1 is located at position 6201, blk_idx = 3 is not satisfied, and the upper left reference block is located in an area less than X2 (x direction 6112) and Y1 (y direction 6111) (S6022: Yes). In addition, the difference between the reference block and the referenced area in the x direction is smaller than the difference between the reference block and the referenced area in the y direction (S6023: Yes). Thus, the reference position in the x direction will be corrected before position 6202, provided that xRefTL = xAvlTL + NL (S6024). In this document, since xRefBR = xRefTL + cbWidth-1 is kept as in S6001, xRefBR will also be adjusted along with the xRefTL adjustment. With this reference position correction, the block vector mvL[0] can be corrected. That is, the correction is performed as follows:

mvL[0] = (xAvlTL + NL-xCb) << 4

This satisfies xRefTL = xAvlTL + NL, which allows the reference position to be corrected.

In contrast, when the reference block r2 is located at position 6203, blk_idx = 3 is not satisfied, and the upper left portion of the reference block is located in an area smaller than X2 (6112 x direction) and Y1 (6111 y direction). (S6022: Yes). In addition, the difference between the reference block and the referenced area in the x direction is not less than the difference between the reference block and the referenced area in the y direction (S6023: No). Thus, the reference position in the y direction is corrected before position 6204, provided that yRefTL = yAvlTL + 64 (S6025). In this document, since yRefBR = yRefTL + cbHeight-1 is performed as in S6001, yRefBR will also be corrected along with the yRefTL correction. With this reference position correction, the block vector mvL[0] can be corrected. That is, the correction is performed as follows:

mvL[1] = (yAvlTL + 64-yCb) << 4

This satisfies yRefTL = yAvlTL + 64, which allows the reference position to be corrected.

In this document, reference block r3 is assumed to be located at position 6205. In this case, the difference between the reference block and the referenced area in the x direction is smaller than the difference between the reference block and the referenced area in the y direction. (S6023: Yes). Accordingly, the reference position in the x direction is corrected similarly to the reference block r1 to be at position 6206 (S6024). At this point, the control block is outside of the area that can be referenced. However, the reference position in the y direction is corrected by processes S6006 and S6007 described below. Thus, the reference block is located within the area that can be referenced.

Then, the lower right position of the area that can be referred to is calculated (S6026). Since the area that can be referred to is the shaded area in Fig. 50, there are two lower right positions, 6113 and 6114, except when blk_idx = 0. When these two points have coordinates (X3, Y3) and ( X4, Y4), respectively, positions are calculated as follows:

offset[4] = {0, 64, 128, 64}, NR = offset[blk_idx]

(X3, Y3) = (xAvlBR, yAvlBR-64)

(X4, Y4) = (xAvlBR-NR, yAvlBR)

It is then determined whether the reference position will be corrected according to the bottom right portion of the referenced area (S6027). In this determination, it is determined that it is true when blk_idx is not equal to 0 and the reference block is located in an area larger than X4 and Y3 (S6027: Yes). In the case where the determination is false (S6027: No), the process ends.

It is then determined whether the difference between the reference block and the referenced area in the x direction is smaller than the difference between the reference block and the referenced area in the y direction (S6028). In the case where the determination is true (S6028: Yes), the reference position will be corrected (S6029) in the x direction. On the contrary, in the case where the determination is false (S6028: No), the reference position will be corrected (S6030) in the y direction.

Fig. 52B is a schematic diagram illustrating the state of the reference position correction in steps S6029 and S6030. Now, blk_idx = 3. Assuming that the reference block r1 is located at position 6211, blk_idx = 0 is not satisfied, and the bottom right portion of the reference block is located in an area larger than X4 (x direction 6114) and Y3 (y direction 6113) (S6027 : Yes). In addition, the difference between the reference block and the referenced area in the x direction is smaller than the difference between the reference block and the referenced area in the y direction (S6028: Yes). Thus, the reference position in the x direction is corrected before position 6212, provided that xRefBR = xAvlBR (S6029). In this document, since xRefBR = xRefTL + cbWidth-1 is performed as in S6001, that is, xRefTL = xRefBR-(cbWidth-1) is performed, xRefTL will also be adjusted along with the xRefBR correction. With this reference position correction, the block vector mvL[0] can be corrected. That is, the correction is performed as follows:

mvL[0] = (xAvlBR-NR-(xCb + cbWitdh-1)) << 4

This satisfies xRefBR = xAvlBR, which allows the reference position to be corrected.

On the contrary, assuming that the reference block r2 is located at the position 6213, blk_idx = 0 is not satisfied, and the bottom right portion of the reference block located in the area is larger than X4 (x direction 6114) and Y3 (y direction 6113) (S6027: Yes). In addition, the difference between the reference block and the referenced area in the x direction is not less than the difference between the reference block and the referenced area in the y direction (S6028: No). Thus, the reference position in the y direction is corrected before position 6214, provided that yRefBR = yAvlBR (S6030). In this document, since yRefBR = yRefTL + cbHeight-1 is performed as in S6001, i.e., yRefTL = yRefBR-(cbHeight-1) is performed, yRefTL will also be adjusted along with the yRefBR correction. With this correction of the reference position, the block vector mvL [1] can be corrected. That is, the correction is performed as follows:

mvL[1] = (yAvlBR-64-(yCb + cbHeight - 1)) << 4

This satisfies yRefBR = yAvlBR, which allows the reference position to be corrected.

In this document, reference block r3 is assumed to be located at position 6215. In this case, the difference between the reference block and the referenced area in the x direction is no less than the difference between the reference block and the referenced area in the x direction. y (S6028: No). Accordingly, the reference position in the y direction is adjusted similarly to the reference block r2 to be located at position 6216 (S6030). At this point, the control block is outside of the area that can be referenced. However, the reference position in the x direction is corrected by processes S6008 and S6009 described below. Thus, the reference block is located within the area that can be referenced.

52, the reference position correction process has been described using the exemplary cases where blk_idx=0 and blk_idx=3. = 3.

After the reference position correction process of the non-rectangular area that can be referenced (S6013), processes S6004 to S6011 are executed. The above procedure terminates the process for the case where the CTU size is 128×128 samples.

Now, it is assumed that in the process of correcting the reference position of the non-rectangular portion of the referenced area (S6013), the process of correcting the reference position in the x direction is performed (S6024) in accordance with the upper left portion of the referenced area. In this case, the reference position in the x direction of the reference block does not become smaller than the upper left portion of the referenced area, and thus the determination in S6004 is always false (S6004: No). Thus, in the case where process S6024 has been executed, processes S6004 and S6005 can be omitted. Similarly, in the case where processing S6025 has been performed, processes S6006 and S6007 can be omitted. In the case where process S6029 has been executed, processes S6008 and S6009 may be omitted. In the case where process S6030 has been executed, processes S6010 and S6011 may be omitted.

Also, the configuration shown in the flowchart shown in FIG. 51 in which the comparison process in step S6023 is omitted and step S6024 is always executed, or step S6025 is always performed can be used. Similarly, a configuration in which the comparison process in step S6028 is omitted and step S6029 is always executed, or step S6030 is always performed can be used. In such a configuration, the reference position can be corrected by a simple process.

48, the reference position is adjusted using processes S6012, S6013, and S6004-S6011 in the case where the CTU size is 128×128 samples. Instead, as shown in FIG. 53, this can also be realized by a process of dividing the referenced area into two areas and then correcting each of the reference positions (S6101).

Fig. 54 is a schematic diagram illustrating a state of dividing an area that can be referred to into two areas. In contrast to the rectangular area that can be referred to and which is shown in Fig.50, the reference area is divided into two in Fig.54. In the case where the coding target block (6102) is located at the upper left position after the division into four parts of the coding tree target block (6101 ), the tuning parameter will be: blk_idx = 0. Similarly, in the case where the coding target blocks are located in top right, bottom left, and bottom right positions, blk_idx will be set to 1, 2, and 3. FIG. 54A is a schematic diagram illustrating the case where blk_idx = 0. Similarly, FIGS. cases where blk_idx = 1, blk_idx = 2 and blk_idx = 3, respectively. In addition, one area that can be referenced (6301) is defined as area A that can be referred to, and another area that can be referred to (6302) is defined as area B that can be referred to.

55 is a flowchart illustrating a process (S6101) of dividing an area into two referenced areas and correcting each of the reference positions of the referenced areas. In Fig. 55, the same process as in Fig. 48 is indicated by the same step numbers, and its description will be omitted. First, the top left and bottom right positions of the referenced area A are calculated (S6111). When the upper left portion of the referenced region A is defined as (xAvlTL, yAvlTL) and the lower right portion is defined as (xAvlBR, yAvlBR), the following expressions are obtained:

xOffsetTL[4] = {-128, -128, -64, 0}, yOffsetTL[4] = {64, 64, 64, 0}

xOffsetBR[4] = {0, 0, 0, 128}, yOffsetBR[4] = {128, 128, 128, 64}

(xAvlTL, yAvlTL) = ((xCb >> CtbLog2SizeY) << CtbLog2SizeY

+xOffsetTL[blk_idx],

(yCb >> CtbLog2SizeY) << CtbLog2SizeY + yOffsetTL[blk_idx])

(xAvlBR, yAvlBR) = (((xCb >> CtbLog2SizeY) << CtbLog2SizeY) - 1

+xOffsetBR[blk_idx],

(((yCb >> CtbLog2SizeY) + 1) << CtbLog2SizeY) - 1 + yOffsetBR[blk_idx])

Then, a calculation as to whether the reference block is outside the referenced area A is performed as follows (S6112):

out_xRefTL = xRefTL<xAvlTL

out_yRefTL = yRefTL<yAvlTL

out_xRefBR = xRefBR > xAvlBR

out_yRefBR = yRefBR > yAvlBR

Then, the top left and bottom right positions of the referable area B are calculated (S6113). When the upper left portion of the referenced region B is defined as (xAvlTL, yAvlTL) and the lower right portion is defined as (xAvlBR, yAvlBR), the following expressions are obtained:

xOffsetTL[4] = {-64, 0, 0, 0}, yOffsetTL[4] = {0, 0, 0, 0}

xOffsetBR[4] = {0, 64, 128, 64}, yOffsetBR[4] = {128, 64, 64, 128}

(xAvlTL, yAvlTL) = ((xCb >> CtbLog2SizeY) << CtbLog2SizeY

+xOffsetTL[blk_idx],

(yCb >> CtbLog2SizeY) << CtbLog2SizeY + yOffsetTL[blk_idx])

(xAvlBR, yAvlBR) = (((xCb >> CtbLog2SizeY) << CtbLog2SizeY) - 1

+xOffsetBR[blk_idx],

(((yCb >> CtbLog2SizeY) + 1) << CtbLog2SizeY) - 1 + yOffsetBR[blk_idx])

It is then determined whether the reference block reference position in the x direction is smaller than the upper left portion of the referenced region A, and whether the reference block reference position in the x direction is smaller than the upper left portion of the referenced region B, ( S6114). When the determination is false (S6114: No), the process advances to the next process (S6116). On the contrary, when the determination is true (S6114: Yes), the reference position in the x direction will be corrected according to the upper left portion of the area B that can be referred to (S6005). Since the process S6005 has already been described, its description is omitted.

It is then determined whether the reference position of the reference block in the y direction is smaller than the upper left portion of the referenced region A, and whether the reference position of the reference block in the y direction is smaller than the upper left portion of the referenced region B (S6116 ). When the determination is false (S6116: No), the processing proceeds to the next step (S6118). On the contrary, when the determination is true (S6116: Yes), the reference position in the y direction will be corrected according to the upper left portion of the area B that can be referred to (S6007). Since the process S6007 has already been described, its description is omitted.

It is then determined whether the reference position in the x direction of the reference block is larger than the lower right portion of the referenced area A, and whether the reference position of the reference block in the x direction is greater than the lower right portion of the referenced area B (S6118 ). When the determination is false (S6118: No), the process advances to the next process (S6120). On the contrary, when the determination is true (S6118: Yes), the reference position in the x direction will be corrected according to the lower right portion of the area B that can be referred to (S6009). Since the process S6009 has already been described, its description is omitted.

It is then determined whether the reference position of the reference block in the y direction is greater than the lower right portion of the referenced area A, and whether the reference position of the reference block in the y direction is greater than the lower right portion of the referenced region B (S6120) . When the determination is false (S6120: No), the process ends. On the contrary, when the determination is true (S6120: Yes), the reference position in the y direction will be corrected according to the lower right portion of the area B that can be referred to (S6011). Since the process S6011 has already been described, its description is omitted.

As described above, even in the case where the reference block is located outside the area that can be referred to, when the CTU size is 128×128 samples, the reference position can be adjusted and can become a reference position that can be referred to. In addition, you can simplify the process and reduce the amount of calculations by dividing the area that can be referenced into two parts and correcting each of the reference positions. In this document, one area (6301) that can be referred to is defined as area A that can be referred to, while the other area (6302) that can be referred to is defined as area B that can be referred to. Alternatively, the referenced area A and the referenced area B may be replaced, and one referenced area (6301) may be treated as the referenced area B, while while the other reference area (6302) can be treated as an A area that can be referred to.

In this example, it is determined whether the CTU size is 128×128 samples (S6002) and the process switches accordingly. This determination may be replaced by determining whether an intrablock copy standard unit is provided in a unit obtained by quadrupling a coding tree block, or determining whether the CTU size exceeds the maximum coding unit size.

Reference area border correction block

FIG. 58 is a flowchart illustrating the process performed by the reference area boundary correction unit 381 and the reference area boundary correction unit 481. FIG.

First, the upper left position of the reference block is determined (step S1401) as follows:

(xRefTL, yRefTL) = (xCb + (mvL[0] >> 4), yCb + (mvL[1] >> 4))

(xRefBR, yRefBR) = (xRefTL + cbWidth - 1, yRefTL + cbHeight - 1)

Since this procedure is the same as step S6001 in Fig. 48, its description will be omitted.

Then, the top left and bottom right positions of the regions A, B, and C that can be referred to are calculated (steps S1402 to S1403). Here it is assumed that the region A that can be referenced is an in-block copy building block processed immediately before the target in-block copy building block, the region B that can be referenced is an intra-block copy building block processed immediately before region A, on that can be referenced and the region C that can be referred to is the block copy building block processed immediately before the region B that can be referenced. FIG. 56 shows an example in which a coding tree block unit is used as an intra-block copy standard block. In the example shown in FIG. 56A, coding tree block 503 is the target coding tree block. At this point, coding tree block 502 corresponds to region A that can be referenced, coding tree block 501 corresponds to region B that can be referred to, and coding tree block 500 corresponds to region C that can be referred to. FIG. 57 shows an example of using a unit obtained by quartering a coding tree block as an intra-block copy standard block. In the example shown in FIG. 57A, coding tree block 504 is the target building block of the intra-block copy. At this point, coding tree block 503 corresponds to region A that can be referred to, coding tree block 502 corresponds to region B that can be referred to, and coding tree block 501 corresponds to region C that can be referred to.

The top left positions of regions A, B, and C that can be referenced are set to (xAvlATL, yAvlATL), (xAvlBTL, yAvlBTL), and (xAvlCTL, yAvlCTL), respectively. In addition, the bottom right positions of regions A, B, and C that can be referenced are set to (xAvlABR, yAvlABR), (xAvlBBR, yAvlBBR), and (xAvlCBR, yAvlCBR), respectively. When the coding tree block unit is used as the standard intrablock copy block, each of (xAvlATL, yAvlATL), (xAvlBTL, yAvlBTL), (xAvlCTL, yAvlCTL), (xAvlABR, yAvlABR), (xAvlBR), (xAvlBR), (xAvlBR) , (xAvlBR), yAvlCBR) are set as follows:

(xAvlATL, yAvlATL) = ((xCb >> CtbLog2SizeY) << (CtbLog2SizeY) - (1 << CtbLog2SizeY),

(yCb >> CtbLog2SizeY) << (CtbLog2SizeY))

(xAvlABR, yAvlABR) = (xAvlATL + cbWidth - 1, yAvlATL + cbHeight - 1)

(xAvlATL, yAvlATL) = ((xCb >> CtbLog2SizeY) << (CtbLog2SizeY) - 2 * (1 << CtbLog2SizeY)),

(yCb >> CtbLog2SizeY) << (CtbLog2SizeY))

(xAvlBBR, yAvlBBR) = (xAvlBTL + cbWidth -1, yAvlBTL + cbHeight - 1)

(xAvlATL, yAvlATL) = ((xCb >> CtbLog2SizeY) << (CtbLog2SizeY) - 3 * (1 << CtbLog2SizeY),

(yCb >> CtbLog2SizeY) << (CtbLog2SizeY))

(xAvlCBR, yAvlCBR) = (xAvlCTL + cbWidth -1, yAvlCTL + cbHeight - 1)

When a unit obtained by dividing the coding tree block into four parts is used as an intrablock copy standard block, each of the parameters (xAvlATL, yAvlATL), (xAvlBTL, yAvlBTL), (xAvlCTL, yAvlCTL) is set as follows:

(xAvlATL, yAvlATL) = ((xCb >> (CtbLog2SizeY-1)) << (CtbLog2SizeY-1) - (1 << (CtbLog2SizeY-1)),

(yCb >> (CtbLog2SizeY-1)) << ((CtbLog2SizeY-1)) + (1 << (CtbLog2SizeY-1)))

(xAvlABR, yAvlABR) = (xAvlATL + cbWidth - 1, yAvlATL + cbHeight - 1)

(xAvlBTL, yAvlBTL) = ((xCb >> (CtbLog2SizeY-1)) << (CtbLog2SizeY-1) - (1 << (CtbLog2SizeY)),

(yCb >> (CtbLog2SizeY-1)) << ((CtbLog2SizeY-1)) + (1 << (CtbLog2SizeY-1)))

(xAvlBBR, yAvlBBR) = (xAvlBTL + cbWidth -1, yAvlBTL + cbHeight - 1)

(xAvlCTL, yAvlCTL) = ((xCb >> (CtbLog2SizeY-1)) << (CtbLog2SizeY-1) - (1 << (CtbLog2SizeY-1)),

(yCb >> (CtbLog2SizeY-1)) << ((CtbLog2SizeY-1)) + (1 << (CtbLog2SizeY-1)))

(xAvlCBR, yAvlCBR) = (xAvlCTL + cbWidth -1, yAvlCTL + cbHeight - 1)

Then, an area that can be referred to including the upper left position of the reference block is determined (Step S1405). Specifically, the top left and bottom right positions of the referenced areas A, B, C are compared with the top left position of the reference block, and then the referenced area including the top left position of the reference block is determined to be the reference the area including the reference block.

It is then determined whether the reference area including the upper left position of the reference block includes the lower right position of the reference block (step S1406). In the case where the reference area including the upper left position of the reference block includes the lower right position of the reference block, the algorithm shown in FIG. 58 ends.

In a case where the referenced area including the upper left position of the reference block does not include the lower right position of the reference block, the reference block is corrected (step S1407). Reference block correction will be described with reference to FIG. FIG. 59 shows an example in which a coding tree block unit is used as an intra-block copy standard block. FIG. 59A shows the case where the block vector 507 is set to the target encoding block 505 . The region B (503) that can be referred to includes the upper left position of the reference block and does not include the lower right position of the reference block. At this stage, as shown in FIG. 59B, reference block 508 is divided into divided reference block 509 and divided reference block 510 in the memory space. The divided reference block 509 is the area included in the referenced area B (503), which is the referenced area including the upper left portion of the reference block, while the divided reference block 510 is the reference block included to a reference area other than area B (503) that can be referred to. In this example, when constructing a reference image of the reference block 510, a different process is performed for each of the area corresponding to the divided reference block 509 and the area corresponding to the divided reference block 510. In particular, with regard to the area corresponding to the divided reference block 509, the reference image is created based on the position indicated by the block vector. On the contrary, with regard to the area corresponding to the divided reference block 510, the reference image is created by setting a predetermined value without reference to the memory space 600 . In this document, the set value will be MaxPixelValue/2 when the maximum sample value is MaxPixelValue. However, with regard to the area corresponding to the divided reference block 510, it is possible to adopt a configuration in which the reference image is constructed from the memory area 603 including the divided reference block 509 instead of a predetermined value. For example, a reference image may be constructed by performing a horizontal copy of the rightmost sample of the divided reference block 509.

FIG. 60 shows an example of using a unit obtained by quartering a coding tree block as an intra-block copy standard block. On figa shows the case when the block vector 508 is set to the target block 509 coding. The area C (501) that can be referred to includes the upper left position of the reference block and does not include the lower right position of the reference block. At this stage, as shown in FIG. 60B, reference block 510 is divided into divided reference block 511 and divided reference block 512 in the memory space. The split reference block 511 is the area included in the referenced region C (501), which is the reference area including the top left portion of the reference block, while the split reference block 512 is the reference block included in the reference block. an area other than area C (501) that can be referred to. In this example, when constructing a reference image of the reference block 510, a different process is performed for each of the area corresponding to the divided reference block 511 and the area corresponding to the divided reference block 512. In particular, with regard to the area corresponding to the divided reference block 511, the reference image is created based on the position indicated by the block vector. On the contrary, for the area corresponding to the divided reference block 511, the reference image is created by setting a predetermined value without reference to the memory space 602. In this document, the set value will be MaxPixelValue/2 when the maximum sample value is equal to MaxPixelValue. However, with regard to the area corresponding to the divided reference block 512, it is permissible to use a configuration in which the reference image is constructed from the memory area 601 including the division reference block 511 instead of a predetermined value. For example, a reference image may be constructed by performing vertical copying of a sample located at the bottom of the divided reference block 511.

By using this configuration, when determining the intra-block copy reference block, access to the divided memory space multiple times in the case where the reference image is divided and allocated to the memory space can be avoided. This allows you to reduce the amount of processing when building an intrablock copy reference block.

In all of the embodiments described above, multiple technologies may be combined with each other.

In all the embodiments described above, the bitstream output from the image encoding device has a specific data format to be decoded according to the encoding method used in the embodiment. The bitstream may be recorded and provided on a computer-readable recording medium such as an HDD, SSD, flash memory, optical disc, or the like, or the bitstream may be provided from a server via a wired or wireless network. Thus, no matter how the bitstream is provided, the image decoding apparatus corresponding to the image encoding apparatus is configured to decode the bitstream of a specific data format.

In the case where a wired or wireless network is used to exchange the bit stream between the image encoding apparatus and the image decoding apparatus, the bit stream can be converted to a data format suitable for a transmission form of a communication channel in transmission. In this case, a transmission device is provided that converts an output bitstream from an image encoding device into coded data in a data format suitable for a transmission form of a communication channel, and transmits the coded data to the network, and a reception device that receives the coded data from the network, which needs to be restored to a bitstream, and supplies the bitstream to an image decoding device. The transmission apparatus includes a memory that buffers a bitstream output from the image encoding apparatus, a packet processing unit that packetizes the bitstream, and a transmitter that transmits the packetized encoded data via a network. The receiving device includes a receiver that receives packetized coded data via a network, a memory that buffers the received coded data, and a packet processing unit that decompresses the coded data to form a bitstream and transmits the generated bitstream to an image decoding device.

In the case where a wired or wireless network is used to exchange the bit stream between the image encoding apparatus and the image decoding apparatus, a relay apparatus may be provided in addition to the transmitter apparatus and the receiver apparatus, which receives the encoded data transmitted by the transmitter and supplies the encoded data to the receiving device. The relay apparatus includes a receiver that receives packetized coded data transmitted by a transmitter, a memory that buffers the received coded data, and a transmitter that transmits the packetized coded data to a network. The relay apparatus may further include a received packet processing unit that packetizes the packetized encoded data to create a bitstream, a recording medium that stores the bitstream, and a transmitted packet processing unit that packetizes the bitstream.

Moreover, as a display device, a display unit that displays an image decoded by the image decoding apparatus can be added to the configuration. In this case, the display unit reads the decoded image signal generated by the decoded image signal overlay unit 207 and stored in the decoded image memory 208, and displays the signal on a screen.

Moreover, an imager can be added to the configuration to function as an imager by inputting the captured image into the image encoder. In this case, the imaging section inputs the captured image signal to the block separating section 101 .

FIG. 37 shows an example of a hardware configuration of an encoding/decoding apparatus according to the present embodiment. The encoding/decoding apparatus includes configurations of an image encoding apparatus and an image decoding apparatus according to embodiments of the present invention. The encoder/decoder device 9000 includes a CPU 9001, an IC codec 9002, an input/output interface 9003, a memory 9004, an optical disc drive 9005, a network interface 9006, and a video interface 9009 in which the individual units are connected via a bus 9010.

Image encoder 9007 and image decoder 9008 are typically implemented as codec IC 9002. The image encoding process of the image encoding device according to the embodiments of the present invention is performed by the image encoder 9007 . The image decoding process in the image decoding apparatus according to the embodiment of the present invention is performed by the image decoder 9008. The input/output interface 9003 is implemented, for example, via a USB interface, and is connected to an external keyboard 9104, mouse 9105, or the like. The CPU 9001 controls the encoder/decode device 9000 based on the input of the user's operation via the input/output interface 9003 to perform the user's operation. User operations on keyboard 9104, mouse 9105, etc. include selecting the encoding or decoding function to be performed, setting the encoding quality, bitstream I/O assignment, image I/O assignment, or the like, the like, the like.

In the case where the user requires an operation to reproduce an image recorded on the disc recording medium 9100, the optical disc drive 9005 reads the bitstream from the inserted disc recording medium 9100 and transmits the read bitstream to the image decoder 9008 of the IC codec 9002 via the bus 9010. The image decoder 9008 performs an image decoding process in an image decoding apparatus according to embodiments of the present invention on an input bitstream, and transmits the decoded image to an external monitor 9103 via a video interface 9009. The encoding/decoding apparatus 9000 has a network interface 9006 and can be connected to an external distribution server 9106 and to the mobile terminal device 9107 via the network 9101. In the case where the user wishes to reproduce the image recorded in the distribution, the server 9106 or the mobile terminal device 9107 instead of the image recorded on the recording disk 9100 and, the network interface 9006 receives the bitstream from the network 9101 instead of reading the bitstream from the input disk recording medium 9100. In the case where the user wishes to reproduce the image recorded in the memory 9004, the image decoding processing is performed by the apparatus for decoding the image of the bitstream recorded in the memory 9004 according to embodiments in the present invention.

In the case where the user wishes to perform the operation of encoding an image captured by the external camera 9102 and writing the image to the memory 9004, the video interface 9009 inputs the image from the camera 9102 and transmits the image to the image encoder 9007 of the codec 9002 via the bus 9010. The image encoder 9007 performs an image encoding process by an image encoding device according to an embodiment of the present invention at the image input through the video interface 9009 and thereby generates a bitstream. The bitstream is then transferred to the memory 9004 via the bus 9010. In the case where the user wishes to record the bitstream on the disk recording medium 9100 instead of the memory 9004, the optical disk drive 9005 writes the bitstream on the inserted disk of the medium 9100.

It is also possible to realize an equipment configuration having an image encoding device and no image decoding device, or an equipment configuration having an image decoding device and no image encoding device. This hardware configuration is implemented by replacing the codec IC 9002 with an image encoder 9007 or an image decoder 9008.

The above-described process relating to encoding and decoding can naturally be implemented as a transmission, storage, and reception device using hardware, and alternatively, the process can be implemented using firmware stored in read-only memory (ROM) flash memory or the like, or using software provided for a computer or the like. The firmware program and the software may be provided by being recorded on a computer-readable recording medium or the like, may be provided from a server via a wired or wireless network, or may be provided by a digital terrestrial or satellite broadcast.

The present invention has been described with reference to the present embodiments. The above described embodiment has been described for illustrative purposes only. Rather, those skilled in the art can readily appreciate that various examples of modifications can be made by forming various combinations of the components or processes described above, which are also within the technical scope of the present invention.

The present invention can be used for image encoding and decoding methods that divide an image into blocks to perform prediction.

100 - Image encoding device

101 - Block division block

102 - Block interframe prediction

103 - Intra prediction block

104 - Decoded picture memory

105 - Determinant of the method of prediction

106 - Residue generation unit

107 - Orthogonal Converter/Quantizer

108 - Bit string encoding block

109 - Inverse quantizer/inverse orthogonal converter

110 - Decoded image signal overlay block

111 - Memory for storing encoding information

200 - Image decoding device

201 - Bit string decoding block

202 - Block division block

203 - Block interframe prediction

204 - Intra prediction block

205 - Memory for storing encoding information

206 - Inverse quantizer/inverse orthogonal converter

207 - Decoded image signal overlay block

208 - Decoded picture memory.

Claims (34)

1. A device for encoding an image, characterized in that it divides a block of the coding tree into one or more standard blocks of intrablock copying, while the device contains: a candidate block vector deriving unit, configured to obtain candidate block vectors of the coding target block in the coding target image from the coding information stored in the coding information storage memory; a selector configured to select a selected block vector from the candidate block vectors; a storage device configured to store encoded images of a predetermined number of intra-block copy building blocks immediately before the target intra-block copy coding building block; block for correcting the boundary of the reference area, configured to removing all encoded images of intra-block copy building blocks that include the target encoding block from the referenced area, and determining whether the top left position and bottom right position of the reference block indicated by the selected block vector are included in the area that can be referred to; and a bit string encoding unit configured to encode encoding information corresponding to the prediction method for each encoding unit. 2. The image encoding device according to claim 1, wherein the reference area boundary correction unit is configured to remove all encoded images of intra-block copy building blocks processed to a predetermined number of intra-block copy building blocks immediately before the target intra-block copy encoding building block from the area, to which you can refer. 3. A method for encoding images, in which a block of the coding tree is divided into one or more standard blocks of intrablock copying, while the method contains: a candidate block vector deriving step of obtaining candidate block vectors of an encoding target block in an encoding target image from the encoding information stored in the encoding information storage memory; a selection step of selecting a selected block vector from the candidate block vectors; a storage step of storing encoded pictures of a predetermined number of intra-block copy building blocks immediately before the target intra-block copy coding building block; and the stage of correction of the boundary of the reference area, in which: removing all encoded pictures of intra-block copy building blocks that include the decoding target block from the referenced area, and determining whether the top left position and the bottom right position of the reference block indicated by the selected block vector are included in the area that can be referred to; and an encoding step in which encoding information corresponding to the prediction method for each encoding block is encoded. 4. A device for decoding an image, characterized in that it divides a block of the coding tree into one or more standard blocks of intrablock copying, while the device contains: a block vector candidate deriving unit, configured to obtain block vector candidates of the decoding target block in the decoding target image from the encoding information stored in the encoding information storage memory; a selector configured to select a selected block vector from the candidate block vectors; a storage device configured to store decoded images of a predetermined number of intra-block copy building blocks immediately before the target intra-block copy decoding building block; block for correcting the boundary of the reference area, configured to removing all decoded intra-block copy building block pictures that include the decoding target block from the referenced area, and determining whether the top left position and the bottom right position of the reference block indicated by the selected block vector are included in the area that can be referred to; and a decoding unit, configured to decode the bit string to obtain encoded data, and decode the encoded data to obtain recovered data. 5. The image decoding apparatus according to claim 4, wherein the reference area boundary correction unit is configured to remove all decoded intrablock copy building block images processed to a predetermined number of intrablock copy building blocks immediately before the decoding target intrablock copy building block from the area, to which you can refer. 6. An image decoding method, in which the coding tree block is divided into one or more intrablock copy standard blocks, the method comprising: a candidate block vector derivation step of obtaining candidate block vectors of the decoding target block in the decoding target image from the encoding information stored in the encoding information storage memory; a selection step of selecting a selected block vector from the candidate block vectors; a storage step of storing decoded pictures of a predetermined number of intra-block copy building blocks just before the target intra-block copy decoding building block; the stage of correction of the boundary of the reference area, in which: removing all decoded intra-block copy building block pictures that include the decoding target block from the referenced area, and determining whether the top left position and the bottom right position of the reference block indicated by the selected block vector are included in the area that can be referred to; and a decoding step of decoding the bit string to obtain encoded data, and decoding the encoded data to obtain recovered data.

Images