TW201828705A - Encoding device, decoding device, encoding method, and decoding method - Google Patents

Abstract

An encoding device comprising a memory and a circuit that can access the memory, wherein the circuit that can access the memory maps each of a plurality of reference blocks to a corresponding region of an image to be encoded, according to motion vectors of said reference blocks, said reference blocks including one or more reference blocks that are included in a first reference picture constituting a first reference picture list and one or more reference blocks that are included in a second reference picture constituting a second reference picture list, and if the corresponding region overlaps with a block to be encoded of the image to be encoded, the circuit derives a candidate motion vector, as one of a plurality of candidate motion vectors, from the motion vectors of the reference blocks that were mapped to the corresponding region, selects a predicted motion vector from among the plurality of candidate motion vectors, and uses the predicted motion vector to encode information of the block to be encoded.

Description

Encoding device, decoding device, encoding method and decoding method

FIELD OF THE INVENTION The present disclosure relates to an encoding apparatus or the like that encodes information of a moving image.

Background Art Heretofore, H.265 has been used as a specification for encoding a moving image. H.265 is also known as HEVC (High Efficiency Video Coding).

Advance Technical Literature Non-Patent Literature Non-Patent Document 1: H.265 (ISO/IEC 23008-2 HEVC (High Efficiency Coding))

SUMMARY OF THE INVENTION Problem to be Solved by the Invention However, when the amount of coding of a moving image increases, the transmission delay of a moving image becomes large, and the memory capacity of a moving image also becomes large. As a result, the consumption of resources and energy has increased.

Here, the present disclosure provides an encoding apparatus or the like that can support reduction in the amount of encoding of a moving image.

Means for Solving the Problem The encoding device of the present disclosure encodes information of a moving image, and includes a memory and a circuit for accessing the memory, and the memory can be accessed. The foregoing circuit is: according to the motion vector of the reference block, respectively mapping a plurality of the foregoing reference blocks to corresponding regions in the image to be encoded, wherein the plurality of reference blocks include 1 included in the first reference picture. One or more reference blocks and one or more reference blocks included in the second reference picture, the first reference picture is a first reference picture list constituting two reference picture lists for bi-prediction, the foregoing The reference picture is a second reference picture list among the two reference picture lists; when the corresponding area overlaps the coding target block in the encoding target image, the reference area that has been mapped to the corresponding area is a motion vector of the block, the candidate motion vector is derived as one of a plurality of candidate motion vectors for the pre-motion vector of the encoding target block; and the plurality of candidate movements from the foregoing Among the selected amount of the prediction motion vector; and using the predicted motion vector, the information about the coding target block is encoded.

In addition, the generality or the specific aspect can be realized by a non-transitory recording medium such as a system, a device, a method, an integrated circuit, a computer program, or a computer-readable CD-ROM. This can be achieved by any combination of systems, devices, methods, integrated circuits, computer programs, and recording media.

Advantageous Effects of Invention The coding apparatus and the like of one aspect of the present invention can support the reduction of the amount of coding associated with moving pictures.

MODE FOR CARRYING OUT THE INVENTION (Discussion on the basis of the present disclosure) In the coding of a block constituting a moving image, there is a case where inter-picture prediction is used. Inter-picture prediction is also known as motion compensation.

For example, when the encoding apparatus uses inter-picture prediction for encoding of the encoding target block, the moving of the moving image is detected (estimated), thereby detecting the motion vector of the encoding target block. Next, the encoding apparatus refers to an image indicated by a motion vector of the encoding target block among the reference pictures different in temporal property from the encoding target block, and generates a predicted image of the encoding target block. Then, the encoding device encodes the difference image between the image of the encoding target block and the predicted image of the encoding target block.

The encoding device can also encode the motion vector of the encoding target block. At this time, the decoding apparatus decodes the motion vector and the difference image of the decoding target block, and decodes the decoding target block using the motion vector and the difference image of the decoding target block.

Specifically, the decoding apparatus refers to the image indicated by the motion vector of the decoding target block among the reference pictures different in temporal property from the decoding target block, and generates a predicted image of the decoding target block. Then, the decoding device adds the predicted image of the decoding target block to the difference image of the decoding target block, thereby reconstructing the image of the decoding target block.

The encoding apparatus may encode the difference motion vector between the motion vector of the encoding target block and the prediction motion vector of the encoding target block in the encoding of the motion vector of the encoding target block.

At this time, the encoding device derives a plurality of candidate motion vectors with respect to the predicted motion vector from a plurality of motion vectors of the plurality of blocks. The plurality of blocks include, for example, a plurality of blocks different in spatial nature from the coding target block, and one block that is temporally different from the coding target block. Then, the encoding device encodes the predicted motion vector index for the specific predicted motion vector from among the plurality of candidate motion vectors.

Next, the decoding device decodes the differential motion vector and the predicted motion vector index of the decoding target block. Further, the decoding means derives a plurality of candidate motion vectors with respect to the prediction motion vector from a plurality of motion vectors of the plurality of blocks. The plurality of blocks are corresponding to a plurality of blocks used when deriving a plurality of candidate motion vectors at the time of encoding, for example, including a plurality of blocks different in spatial nature from the decoding target block, and in time nature It is different from the 1 block of the decoding target block.

Next, the decoding apparatus uses the predicted motion vector index of the decoding target block to specify a prediction motion vector of the decoding target block from among the plurality of candidate motion vectors. Then, the decoding device adds the predicted motion vector of the decoding target block to the differential motion vector of the decoding target block, thereby deriving the motion vector of the decoding target block. Thereby, the encoding device and the decoding device can use the same motion vector.

The coding apparatus and the decoding apparatus may perform an operation using the differential motion vector as described above in an operation mode called a prediction motion vector designation mode.

Further, the encoding apparatus may also generate a predicted image by using the predicted motion vector of the encoding target block as the motion vector of the encoding target block. Similarly, the decoding apparatus can also generate a predicted image by using the predicted motion vector of the decoding target block as the motion vector of the decoding target block. The encoding device and the decoding device can also perform such an operation in an operation mode called a merge mode.

Further, for example, when the prediction apparatus selects the prediction motion vector from among the plurality of candidate motion vectors, the coding apparatus evaluates each of the plurality of candidate motion vectors, and selects the candidate motion vector that is the most highly evaluated among the plurality of candidate motion vectors. Predictive motion vectors are also available. In this case, the higher the degree of suitability of the two evaluation target regions is, the higher the candidate motion vector of the evaluation target is, and the two evaluation target regions are two regions different from the coding target block. And at least one of them is two regions determined in accordance with the candidate motion vector of the evaluation object.

At this time, the decoding apparatus may evaluate each of the plurality of candidate motion vectors when selecting the prediction motion vector from among the plurality of candidate motion vectors, and select the candidate motion that is most highly evaluated among the plurality of candidate motion vectors. Vector, as a predictive motion vector. In this case, the higher the degree of suitability of the two evaluation target regions, the higher the evaluation of the candidate motion vector of the evaluation target, wherein the two evaluation target regions are two regions different from the decoding target block. And at least one of them is two regions determined in accordance with the candidate motion vector of the evaluation object.

Thereby, the encoding device and the decoding device can derive the same predicted motion vector according to the same selection method. Then, the encoding device and the decoding device use the derived prediction motion vector as the motion vector, whereby the same motion vector can be used without encoding and decoding of the motion vector. This technique is also known as FRUC (Frame Rate Up-Conversion).

In each of the above aspects, the predicted motion vector is selected from a plurality of candidate motion vectors. At this time, if a suitable candidate motion vector is not included in the plurality of candidate motion vectors, an appropriate prediction motion vector cannot be derived. Then, if an appropriate prediction motion vector cannot be derived, appropriate prediction processing cannot be performed, and the coding efficiency is lowered and the amount of coding is increased. Next, the transmission delay of the moving image is made larger, and the memory capacity of the moving image is also made larger. As a result, the consumption of resources and energy will increase.

Herein, an encoding apparatus according to an aspect of the present invention encodes information of a moving image, and includes a memory and a circuit that can access the memory, and the foregoing circuit that can access the memory is And correspondingly mapping a plurality of the foregoing reference blocks to corresponding regions in the image to be encoded according to the motion vector of the reference block, wherein the plurality of reference blocks include one or more included in the first reference picture. a reference block and one or more reference blocks included in the second reference picture, wherein the first reference picture is a first reference picture list constituting a list of two reference pictures used for bi-prediction, and the second reference picture Is a second reference picture list among the two reference picture lists; when the corresponding corresponding area overlaps the coding target block in the encoding target image, the motion vector from the reference block that has been mapped to the corresponding area is Deriving a candidate motion vector as one of a plurality of candidate motion vectors for the pre-motion vector of the encoding target block; selecting the foregoing from the plurality of candidate motion vectors Measuring motion vector; and using the predicted motion vector, the information about the coding target block is encoded.

Thereby, the encoding device may derive a suitable candidate motion vector from the reference block, wherein the reference block is among the plurality of reference blocks in the first reference picture list and the second reference picture list, and is assumed to be the coding target block. Reference block with high correlation. Therefore, the encoding apparatus can increase the likelihood that a plurality of candidate motion vectors for selecting a predicted motion vector contain suitable candidate motion vectors.

Then, the encoding device can support the derivation of a suitable prediction motion vector, and can support the reduction of the encoding amount of the moving image.

For example, the above-described circuit may perform mapping of one or more reference blocks in the first reference picture in the encoding target image, and then one or more reference areas in the second reference picture. The blocks are respectively mapped in a blank area, and the blank area is an area in which one or more reference blocks in the first reference picture are not mapped in the encoding target image.

Thereby, the encoding apparatus can more preferentially map one or more reference blocks in the first reference picture list than one or more reference blocks in the second reference picture list. There is a case where the reliability of the motion vector of the reference block in the first reference picture list is higher than the reliability of the motion vector of the reference block in the second reference picture list. In this case, the encoding device can derive a suitable candidate motion vector according to the highly reliable motion vector.

Further, for example, the above-described circuit may overlap two or more of the two or more corresponding regions in which two or more reference blocks of the plurality of reference blocks are mapped in the coding target block. The candidate motion vector is derived by referring to at least one of two or more motion vectors of the block.

Thereby, the encoding device may derive a suitable candidate motion vector from at least one of the plurality of reference blocks, the plurality of reference blocks being assumed to have a high correlation with the coding target block.

Further, for example, the circuit may be mapped to overlap when there is no corresponding region overlapping the encoding target block and there is a corresponding region overlapping the peripheral region of the encoding target block. The candidate motion vector is derived from the motion vector of the reference block of the aforementioned corresponding block in the foregoing peripheral area.

Thereby, even if the reference block in the coding target block is not mapped, the encoding device may derive the appropriate candidate motion vector according to the motion vector of the reference block that has been mapped in the periphery of the coding target block.

Further, for example, the circuit may evaluate each of the plurality of candidate motion vectors when selecting the prediction motion vector from among the plurality of candidate motion vectors, and select the highest evaluated among the plurality of candidate motion vectors. The candidate motion vector is used as the prediction motion vector. When evaluating each of the plurality of candidate motion vectors, the degree of suitability between the reconstructed images of the two comparison target regions is higher, and the candidate motion of the evaluation object is highly evaluated. The vector, the two comparison target regions are two regions different from the coding target block, and at least one of them is two regions determined in accordance with the candidate motion vector of the evaluation target.

Thereby, the encoding apparatus can evaluate each candidate motion vector with reference to the reconstructed image of the region different from the encoding target block, and select the predicted motion vector from among the plurality of candidate motion vectors. Therefore, the encoding device and the decoding device can select the predicted motion vector from among the plurality of candidate motion vectors in the same manner. Thereby, the encoding apparatus can omit the encoding of the information for selecting the prediction motion vector, and can support the reduction of the encoding amount.

Moreover, for example, the foregoing circuit may also derive a motion vector of the reference block to which a scaling ratio is applied when the candidate motion vector is derived from a motion vector of a reference block that has been mapped to the corresponding region. As the foregoing candidate motion vector, the foregoing scaling ratio value is a time difference from a coding target picture including the foregoing encoding target image to a reference picture including the foregoing reference block, with respect to from a reference picture including the aforementioned reference block to the foregoing The ratio of the time differences of the reference pictures of the reference area indicated by the motion vector of the reference block.

Thereby, the encoding device can appropriately scale the motion vector of the reference block, and can derive the scaled motion vector as the candidate motion vector.

Further, for example, the encoding target block may be a block determined as a coding unit or a sub-block determined by a predetermined size in a block determined as a coding unit.

Thereby, the encoding device can derive a suitable candidate motion vector for the block of the coding unit or the sub-block within the block of the coding unit.

Furthermore, for example, the above-described circuit may further be used for each of the plurality of encoding target blocks in the encoding target image, when the corresponding region overlaps the encoding target block, and is mapped from the corresponding region. The candidate motion vector of the predicted motion vector for the coding target block is derived by referring to the motion vector of the block.

Thereby, the encoding device can derive an appropriate candidate motion vector for each encoding target block in the encoding target image.

Furthermore, the decoding device of one aspect of the present disclosure is a decoding device for decoding information of a moving image, comprising a memory and a circuit for accessing the memory, and the memory can be accessed. The foregoing circuit is: mapping a plurality of the foregoing reference blocks to corresponding regions in the decoding target image according to the motion vector of the reference block, wherein the plurality of reference blocks include the first reference picture. One or more reference blocks and one or more reference blocks included in the second reference picture, the first reference picture is a first reference picture list constituting two reference picture lists for bi-prediction, the foregoing The reference picture is a second reference picture list among the two reference picture lists; when the corresponding corresponding area is superimposed on the decoding target block in the decoding target picture, the reference has been mapped from the corresponding area. The motion vector of the block derives the candidate motion vector as one of a plurality of candidate motion vectors for the prediction motion vector of the decoding target block; from the plurality of candidate movements Among the selected amount of the prediction motion vector; and using the predicted motion vector, the information about the decoding target block is decoded.

Thereby, the decoding apparatus can derive a suitable reference block from the plurality of reference blocks in the first reference picture list and the second reference picture list from the reference block that is assumed to have high correlation with the decoding target block. Candidate motion vector. Therefore, the decoding apparatus can increase the likelihood that a suitable candidate motion vector is included in the plurality of candidate motion vectors used to select the predicted motion vector.

Then, the decoding device can support the derivation of the appropriate prediction motion vector, and can support the reduction of the coding amount of the motion image.

For example, in the above circuit, after one or more reference blocks in the first reference picture are mapped to the decoding target image, one or more reference blocks in the second reference picture may be used. The mapping is performed in a blank area, and the blank area is an area in which one or more reference blocks in the first reference picture among the decoding target images are not mapped.

Thereby, the decoding apparatus can more preferentially map one or more reference blocks in the first reference picture list than one or more reference blocks in the second reference picture list. The reliability of the motion vector of the reference block in the first reference picture list is higher than the reliability of the motion vector of the reference block in the second reference picture list. In this case, suitable candidate motion vectors can be derived according to the highly reliable motion vector.

Further, for example, the above-described circuit may overlap two or more references when two corresponding regions in which two or more reference blocks among the plurality of reference blocks have been mapped are overlapped in the decoding target block. The candidate motion vector is derived by at least one motion vector among two or more motion vectors of the block.

Thereby, the decoding apparatus can derive a suitable candidate motion vector from at least one of the plurality of reference blocks that are assumed to have high correlation with the decoding target block.

Further, for example, the above-described circuit may be derived from the motion vector of the reference block when there is no corresponding region overlapping the decoding target block and there is a corresponding region overlapping the peripheral region of the decoding target block. In the foregoing candidate motion vector, the reference block is mapped to the aforementioned corresponding region overlapping the peripheral region.

Thereby, even if the reference block is not mapped in the decoding target block, the decoding device may derive a suitable candidate motion vector according to the motion vector of the reference block that has been mapped in the periphery of the decoding target block. .

Further, for example, when the prediction motion vector is selected from among the plurality of candidate motion vectors, the circuit may evaluate each of the plurality of candidate motion vectors, and select the most highly evaluated among the plurality of candidate motion vectors. The candidate motion vector is used as the prediction motion vector, and when evaluating each of the plurality of candidate motion vectors, the degree of suitability between the reconstructed images of the two comparison target regions is higher, and the candidate motion vector of the evaluation object is The two comparison target areas are two areas different from the decoding target block, and at least one of them is two areas determined in accordance with the candidate motion vector of the evaluation target.

Thereby, the decoding apparatus can evaluate each candidate motion vector with reference to the reconstructed image of the region different from the decoding target block, and select the predicted motion vector from among the plurality of candidate motion vectors. Therefore, the encoding device and the decoding device can select the predicted motion vector from among the plurality of candidate motion vectors in the same manner. Thereby, the decoding apparatus can save decoding for selecting information for predicting the motion vector, and can support the reduction of the amount of coding.

Further, for example, when the foregoing candidate motion vector is derived from the motion vector of the reference block that has been mapped to the corresponding region, the circuit may derive a motion vector of the reference block to which the scaling value is applied as The candidate motion vector, wherein the scaling ratio is a time difference from a decoding target picture including the decoding target image to a reference picture including the reference block, with respect to the reference picture from the reference block to the foregoing reference The ratio of the time differences of the reference pictures of the reference area indicated by the motion vector of the block.

Thereby, the decoding device can appropriately scale the motion vector of the reference block, and can derive the scaled motion vector as the candidate motion vector.

Further, for example, the decoding target block may be a block determined as a decoding unit or a sub-block determined by a predetermined size among the blocks determined as the decoding unit.

Thereby, the decoding apparatus can derive a suitable candidate motion vector for the block of the decoding unit or the sub-block within the block of the decoding unit.

Further, for example, the circuit further includes, for each of the plurality of decoding target blocks in the decoding target image, when the corresponding region overlaps the decoding target block, from a reference region that has been mapped to the corresponding region The motion vector of the block derives a candidate motion vector for the predicted motion vector of the decoding target block.

Thereby, the decoding device can derive an appropriate candidate motion vector for each decoding target block in the decoding target image.

Moreover, the encoding method of the present disclosure is a method for encoding information of a moving image, and performing mapping of the plurality of the reference blocks to the image of the encoding object according to the motion vector of the reference block. Corresponding region, wherein the plurality of reference blocks include one or more reference blocks included in the first reference picture and one or more reference blocks included in the second reference picture, and the first reference picture is Forming a first reference picture list among the two reference picture lists for bi-prediction, wherein the second reference picture is a second reference picture list constituting the two reference picture lists; and the foregoing corresponding region overlaps the foregoing reference picture When encoding the target block in the object image, the candidate motion vector is derived from the motion vector of the reference block that has been mapped to the corresponding region as a plurality of candidate motion vectors for the prediction motion vector of the encoding target block. One of the plurality of candidate motion vectors, the foregoing prediction motion vector is selected; and the information of the foregoing coding target block is used using the prediction motion vector Line coding.

Thereby, the apparatus or the like using the encoding method can be used as a reference block which is assumed to have high correlation with the coding target block among the plurality of reference blocks in the first reference picture list and the second reference picture list. , export the appropriate candidate motion vector. Therefore, the apparatus or the like using the encoding method can increase the possibility of including a suitable candidate motion vector among a plurality of candidate motion vectors for selecting a prediction motion vector.

Then, the device or the like using the encoding method can support the derivation of an appropriate prediction motion vector, and can support the reduction of the coding amount of the moving image.

Moreover, the decoding method of the present disclosure is a method for decoding information of a moving image, and performing mapping of the plurality of the reference blocks to the decoding target image according to the motion vector of the reference block. Corresponding region, wherein the plurality of reference blocks include one or more reference blocks included in the first reference picture and one or more reference blocks included in the second reference picture, and the first reference picture is Forming a first reference picture list among the two reference picture lists used for bi-prediction, wherein the second reference picture is a second reference picture list constituting the two reference picture lists; and the foregoing corresponding region overlaps the decoding When decoding the target block in the object image, the candidate motion vector is derived from the motion vector of the reference block that has been mapped to the corresponding region as a plurality of candidate motion vectors for the prediction motion vector of the decoding target block. One of the plurality of candidate motion vectors; the prediction motion vector is selected; and the information of the decoding target block is used by using the prediction motion vector The row decoder.

Thereby, the apparatus or the like using the decoding method can be used as a reference block which is assumed to have high correlation with respect to the decoding target block among the plurality of reference blocks in the first reference picture list and the second reference picture list. , export the appropriate candidate motion vector. Therefore, the apparatus or the like using the decoding method can increase the possibility of including a suitable candidate motion vector among a plurality of candidate motion vectors for selecting a prediction motion vector.

Then, the device or the like using the decoding method can support the derivation of an appropriate predicted motion vector, and can support the reduction of the amount of coding associated with the moving image.

Furthermore, the general or specific aspects may be implemented by a non-transitory recording medium such as a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable CD-ROM, or a system. The device, the method, the integrated circuit, the computer program, and any combination of recording media are implemented.

Hereinafter, embodiments will be specifically described with reference to the accompanying drawings.

In addition, each of the embodiments described below is an example showing generality or specificity. The numerical values, shapes, materials, constituent elements, arrangement positions, connection forms, steps, and order of steps shown in the following embodiments are merely examples, and the scope of the claims is not limited. Further, among the constituent elements in the following embodiments, the constituent elements of the independent request items that are not described in the uppermost concept are described as arbitrary constituent elements.

(Embodiment 1) First, an outline of Embodiment 1 will be described with respect to an example of an encoding device and a decoding device to which the processing and/or configuration described in each aspect of the present disclosure to be described later can be applied. However, the first embodiment is merely an example of an encoding device and a decoding device to which the processing and/or configuration described in the various aspects of the present disclosure can be applied. The processing and/or configuration described in the present disclosure may be implemented. The coding apparatus and the decoding apparatus different from the first embodiment.

In the first embodiment, when the processing and/or configuration described in each aspect of the present disclosure is applied, for example, any of the following aspects may be employed. (1) The coding device or the decoding device according to the first embodiment, among the plurality of constituent elements constituting the coding device or the decoding device, constituent elements corresponding to the constituent elements described in the aspects of the present disclosure, (2) The coding device or the decoding device according to the first embodiment first applies to a part of a plurality of constituent elements constituting the coding device or the decoding device. Any changes in the addition, replacement, deletion, and the like of the function or the process to be performed, and the components corresponding to the components described in the aspects of the present disclosure are replaced by the various aspects of the cost disclosure. (3) The method to be performed by the encoding device or the decoding device according to the first embodiment, the processing addition, and/or the processing of a part of the plurality of processing included in the method is first replaced or deleted. After any arbitrary changes, the processing corresponding to the processing described in each aspect of the present disclosure is replaced with the various aspects disclosed by the cost. (4) The constituent elements of a part of the plurality of constituent elements constituting the encoding apparatus or the decoding apparatus of the first embodiment, and the constituent elements described in the respective aspects of the present disclosure, and the various aspects of the present disclosure The constituent elements of one of the functions of the constituent elements described in the example or the constituent elements of the processing to be performed by the constituent elements described in the various aspects of the present disclosure are combined; (5) A component of one of the functions of a part of the plurality of components constituting the coding device or the decoding device of the first embodiment, or a plurality of components constituting the coding device or the decoding device of the first embodiment Some of the components of the processing performed by the constituent elements of the present invention, and the constituent elements described in the various aspects of the present disclosure, and a part of the functions of the constituent elements described in the various aspects of the present disclosure The constituent elements or the processing performed by the constituent elements described in the various aspects of the present disclosure A part of the components are combined and implemented. (6) The method to be implemented by the encoding device or the decoding device according to the first embodiment, among the plurality of processes included in the method, corresponds to each of the aspects of the present disclosure. The processing of the description process replaces the processing to be described in each aspect of the cost disclosure; (7) processing a part of the plurality of processing included in the method to be implemented by the encoding apparatus or the decoding apparatus of the first embodiment, and The processes illustrated in the various aspects disclosed are combined to be implemented.

Further, the embodiments of the processes and/or configurations described in the various aspects of the present disclosure are not limited to the above examples. For example, it may be implemented in a device different from the moving image/image encoding device or the moving image/image decoding device disclosed in the first embodiment, or may be implemented separately. The processing and/or composition illustrated in the aspects. Further, the processes and/or configurations already described in the different aspects may be combined and implemented.

[Outline of Encoding Device] First, an outline of the encoding device according to the first embodiment will be described. Fig. 1 is a block diagram showing the functional configuration of an encoding apparatus 100 according to the first embodiment. The encoding device 100 is a moving image/image encoding device that encodes moving images/images in block units.

As shown in FIG. 1, the encoding apparatus 100 is an apparatus for encoding an image in units of blocks, and includes a division unit 102, a subtraction unit 104, a conversion unit 106, a quantization unit 108, an entropy coding unit 110, and an inverse quantization unit 112. The inverse conversion unit 114, the addition unit 116, the block memory 118, the loop filter unit 120, the frame memory 122, the intra prediction unit 124, the inter prediction unit 126, and the prediction control unit 128.

The encoding device 100 is realized by, for example, a general purpose processor and a memory. At this time, when the software program stored in the memory is executed by the processor, the processor functions as the dividing unit 102, the subtraction unit 104, the conversion unit 106, the quantization unit 108, the entropy coding unit 110, and the inverse quantization unit 112. The inverse conversion unit 114, the addition unit 116, the loop filter unit 120, the intra prediction unit 124, the inter prediction unit 126, and the prediction control unit 128 function. Further, the encoding device 100 can be realized by one or more dedicated electronic circuits corresponding to the dividing unit 102, the subtracting unit 104, the converting unit 106, the quantization unit 108, and the entropy. The coding unit 110, the inverse quantization unit 112, the inverse conversion unit 114, the addition unit 116, the loop filter unit 120, the intra prediction unit 124, the inter prediction unit 126, and the prediction control unit 128.

Hereinafter, each component included in the encoding device 100 will be described.

[Division Unit] The division unit 102 divides each picture included in the input moving image into a plurality of blocks, and outputs each block to the subtraction unit 104. For example, the dividing unit 102 first divides the picture into blocks of a fixed size (for example, 128 × 128). This fixed size block is sometimes also referred to as a coding tree unit (CTU). Next, the dividing unit 102 divides each of the fixed-size blocks into variable sizes (for example, 64×64 or less) according to the recursive quadtree and/or binary tree partitioning. ) The block. This variable size block is sometimes also referred to as a coding unit (CU), a prediction unit (PU), or a conversion unit (TU). Further, in the present embodiment, there is no need to distinguish between CU, PU, and TU, and some or all of the blocks in the picture may be processing units of CU, PU, and TU.

Fig. 2 is a view showing an example of block division in the first embodiment; In Fig. 2, the solid line indicates the block boundary obtained by the division of the quaternary tree block, and the broken line indicates the block boundary obtained by the division of the binary tree block.

Here, block 10 is a square block of 128 x 128 pixels (128 x 128 blocks). The 128 x 128 block 10 is first divided into 64 square 64 blocks (quaternary tree block partitioning) divided into 4 squares.

The upper left 64×64 block is a 32×64 block that is further vertically divided into two rectangles, and the left 32×64 block is a 16×64 block that is further vertically divided into two rectangles (binary tree area) Block split). As a result, the upper left 64x64 block is divided into two 16x64 blocks 11, 12, and 32x64 blocks 13.

The upper right 64x64 block is a 64x32 block 14, 15 (binary tree block partition) that is horizontally divided into two rectangles.

The lower left 64x64 block is a 32x32 block (quaternary tree block partition) that is divided into 4 squares. Among the four 32×32 blocks, the upper left block and the lower right block are further divided. The upper left 32×32 block is a 16×32 block vertically divided into two rectangles, and the right 16×32 block is further horizontally divided into two 16×16 blocks (binary tree block division). The lower right 32x32 block is horizontally divided into two 32x16 blocks (binary tree block partitioning). As a result, the lower left 64×64 block is divided into one 16×32 block 16, two 16×16 blocks 17, 18, two 32×32 blocks 19, 20, and two 32×. Block 16 21, 22.

The 64×64 block 23 at the lower right is not divided.

As above, in Fig. 2, the block 10 is divided into 13 variable-sized blocks 11 to 23 according to the recursive quadtree and binary tree block division. This division is sometimes referred to as QTBT (quad-tree plus binary tree) segmentation.

In addition, in FIG. 2, one block is divided into four or two blocks (quaternary tree or binary tree block division), and the division is not limited thereto. For example, one block can also be divided into three blocks (three-dimensional tree block division). Segmentation including ternary tree block partitioning is sometimes referred to as MBT (multi type tree) segmentation.

[Subtraction Unit] The subtraction unit 104 is a block unit divided by the division unit 102, and subtracts the prediction signal (predicted sample) from the original signal (original sample). In other words, the subtraction unit 104 is a prediction error (also referred to as a residual) for calculating a coding target block (hereinafter referred to as a current block). Next, the subtraction unit 104 outputs the calculated prediction error to the conversion unit 106.

The original signal is an input signal of the encoding device 100, and is a signal (for example, a luma signal and two chroma signals) indicating an image of each picture constituting the moving image. In the following, the signal representing the image is also referred to as a sample.

[Conversion Unit] The conversion unit 106 converts the prediction error of the spatial region into a conversion coefficient of the frequency region, and outputs the conversion coefficient to the quantization unit 108. Specifically, the conversion unit 106 performs, for example, a discrete cosine transform (DCT) or a discrete sine transform (DST) that has been determined in advance for the prediction error of the spatial region.

Further, the conversion unit 106 may adaptively select a conversion pattern from among a plurality of conversion patterns, and convert a prediction error into a conversion coefficient using a transform basis function corresponding to the selected conversion pattern. Such a conversion is sometimes referred to as an EMT (explicit multiple core transform) or an AMT (adaptive multiple transform).

A plurality of conversion patterns include, for example, DCT-II, DCT-V, DCT-VIII, DST-I, and DST-VII. Figure 3 is a table showing the conversion basis functions corresponding to the respective conversion patterns. In Figure 3, N is the number of input pixels displayed. The selection of the conversion pattern from among the plurality of conversion patterns may be based on, for example, the type of prediction (internal prediction and inter prediction) or the internal prediction mode.

Information indicating whether such EMT or AMT is applicable (for example, referred to as an AMT flag) and information indicating the selected conversion pattern is signal processed at the CU level. In addition, the signal processing of the information is not necessarily limited to the CU level, but may be other levels (such as sequence level, picture level, slice level, tile level or CTU level).

Further, the conversion unit 106 may reconvert the conversion coefficient (conversion result). Such reconversion is sometimes referred to as AST (adapive secondary transform) or NSST (non-separable secondary transform). For example, the conversion unit 106 performs reconversion for each of the sub-blocks (for example, 4 × 4 sub-blocks) included in the block corresponding to the conversion coefficient of the intra prediction error. The information showing whether NSST is applicable and the information about the conversion matrix used in NSST is signal processing at the CU level. In addition, the signal processing of the information is not necessarily limited to the CU level, but may be other levels (such as sequence level, picture level, slice level, block level or CTU level).

Here, the separable conversion refers to a method of separating the dimension of the input according to the direction and performing the conversion several times. The non-separable conversion refers to a dimension of 2 or more when the input is multi-dimensional. Whole, but as a one-dimensional, then the way to convert together.

For example, in the case of an inseparable conversion, for example, when a block of 4×4 is input, the block is regarded as having one of 16 elements, and the arrangement is 16 The conversion matrix of ×16 performs conversion processing.

Also, similarly, a 4×4 input block is regarded as having an entire arrangement of one of 16 elements, and then the arrangement is performed several times of Givens rotation (Hypercube Givens Transform/Hybrid Givens Transform) ), also an example of non-separable (Non-Separable) conversion.

[Quantization Unit] The quantization unit 108 quantizes the conversion coefficients output from the conversion unit 106. Specifically, the quantization unit 108 scans the conversion coefficient of the current block in a predetermined scanning order, and quantizes the conversion coefficient according to the quantization parameter (QP) corresponding to the scanned conversion coefficient. Then, the quantization unit 108 outputs the quantized conversion coefficients (hereinafter referred to as quantized coefficients) of the current block to the entropy encoding unit 110 and the inverse quantization unit 112.

The predetermined order is the order in which the quantization/dequantization of the conversion coefficients is used. For example, the predetermined scanning order is defined by an ascending order of frequency (in order from low frequency to high frequency) or a descending order of power (in order of high frequency to low frequency).

The quantization parameter refers to a parameter that defines a quantization step (quantization width). For example, if the value of the quantization parameter increases, the quantization step also increases. That is, if the value of the quantization parameter increases, the quantization error also becomes large.

[Entropy Encoding Unit] The entropy encoding unit 110 performs variable length encoding on the quantized coefficients input from the quantization unit 108, thereby generating an encoded signal (encoded bit stream). Specifically, the entropy coding unit 110 binarizes the quantized coefficients, for example, and arithmetically encodes the binarized signals.

[Inverse Quantization Unit] The inverse quantization unit 112 inversely quantizes the input quantized coefficients from the quantization unit 108. Specifically, the inverse quantization unit 112 inversely quantizes the quantized coefficients of the current block in a predetermined scanning order. Then, the inverse quantization unit 112 outputs the inverse quantized conversion coefficient of the current block to the inverse conversion unit 114.

[Inverse Conversion Unit] The inverse conversion unit 114 inversely converts the input conversion coefficient of the inverse quantization unit 112, thereby restoring the prediction error. Specifically, the inverse conversion unit 114 performs inverse conversion corresponding to the conversion performed by the conversion unit 106 on the conversion coefficient, thereby restoring the prediction error of the current block. Then, the inverse conversion unit 114 outputs the predicted error that has been restored to the addition unit 116.

Further, since the prediction error that has been recovered is that the information is lost by the progress of the quantization, the prediction error calculated by the subtraction unit 104 does not match. That is, the quantization error is included in the predicted error that has been recovered.

[Addition Unit] The addition unit 116 adds the prediction error input from the inverse conversion unit 114 to the prediction sample input from the prediction control unit 128, thereby reconstructing the current block. Then, the addition unit 116 outputs the reconstructed block to the block memory 118 and the loop filter unit 120. Reconstituted blocks are sometimes referred to as local decoded blocks.

[Block Memory] The block memory 118 is a memory for storing a block, wherein the block is a block for intra prediction, and is a region within a coded object picture (hereinafter referred to as a current picture). Piece. Specifically, the tile memory 118 stores the reconstructed block output from the addition unit 116.

[Loop Filter Unit] The loop filter unit 120 applies loop filtering to the block reconstructed via the adder 116, and outputs the filtered reconstructed block to the frame memory 122. Loop filtering refers to the filter (intra-loop filter) used in the coding loop, and includes, for example, a deblocking filter (DF), a sample adaptive compensation (SAO), and an adaptive loop filter (ALF).

In ALF, the least square error filter used to remove the coding distortion is applied, for example, according to each of the 2×2 sub-blocks in the current block, according to the direction and activity of the local gradient (activity) ), one filter selected from a plurality of filters is applied.

Specifically, first, sub-blocks (for example, 2×2 sub-blocks) are classified into a plurality of categories (for example, 15 or 25 types). The classification of sub-blocks is based on the direction and activity of the gradient. For example, the classification value C (for example, C=5D+A) is calculated using the direction value D of the gradient (for example, 0 to 2 or 0 to 4) and the activity value A of the gradient (for example, 0 to 4). Then, based on the classification value C, the sub-blocks are classified into a plurality of categories (for example, 15 or 25 types).

The direction value D of the gradient is derived, for example, by comparing gradients in a plurality of directions (eg, horizontal, vertical, and 2 diagonal directions). Further, the gradient activity value A is derived, for example, by adding the gradients of the plurality of directions and quantizing the addition result.

Based on the result of such classification, a filter for a sub-block is determined from among a plurality of filters.

For the shape of the filter used in the ALF, for example, a circularly symmetrical shape is utilized. 4A to 4C are diagrams showing a few examples of the shape of a filter used in the ALF. 4A is a 5×5 diamond shape filter, 図4B is a 7×7 diamond shape filter, and FIG. 4C is a 9×9 diamond shape filter. The information of the display filter is signaled at the picture level. In addition, the signalization of the information indicating the shape of the filter is not limited to the picture level, and may be other levels (for example, sequence level, slice level, block level, CTU level, or CU level).

The ALF is turned on/off, for example, by picture level or CU level. For example, for brightness, it is determined whether or not ALF is applied by the CU level, and for the color difference, whether or not ALF is applied is determined by the picture level. The information showing the ALF on/off is signalized by the picture level or CU level. In addition, the information indicating the on/off of the ALF is not limited to the picture level or the CU level, and may be other levels (such as sequence level, slice level, block level, or CTU level).

The combination of coefficients of a selectable plurality of filters (e.g., filters up to 15 or 25) is signaled at the picture level. In addition, the signalization of the coefficient combination is not limited to the picture level, but may be other levels (such as sequence level, slice level, block level, CTU level, CU level, or sub-block level).

[Frame Memory] The frame memory 122 is a memory for storing reference pictures that are used for inter prediction, and is sometimes referred to as a frame buffer. Specifically, the frame memory 122 stores the reconstructed block that has been filtered by the loop filter unit 120.

[Internal Prediction Unit] The intra prediction unit 124 refers to a block in the current picture stored in the reference block memory 118, and performs intra prediction (also referred to as intra-picture prediction) of the current block to generate a prediction signal (inter prediction) Signal)). Specifically, the intra prediction unit 124 performs intra prediction by referring to samples (for example, luminance values and color difference values) of the blocks adjacent to the current block, thereby generating an intra prediction signal, and outputting the intra prediction signal to the prediction control unit. 128.

For example, the intra prediction unit 124 performs intra prediction by using one of a plurality of intra prediction modes that have been previously defined. The plurality of intra prediction modes include one or more non-directional prediction modes and a plurality of directional prediction modes.

One or more non-directional prediction modes include, for example, a Planar prediction mode and a direct current (DC) defined by the H.265/HEVC (High-Efficiency Video Coding) specification (Non-Patent Document 1). ) Prediction mode.

A plurality of directional prediction modes include, for example, prediction modes of 33 directions specified by the H.265/HEVC specification. In addition, the plurality of directional prediction modes may further include prediction modes of 32 directions (a total of 65 directional prediction modes) in addition to 33 directions. FIG. 5A is a diagram showing 67 intra prediction modes (two non-directional prediction modes and 65 directional prediction modes) in intra prediction. The solid arrow symbol indicates 33 directions defined by the H.265/HEVC standard, and the dotted arrow symbol indicates the 32 additional directions added.

In addition, in the intra prediction of the color difference block, the luminance block can also be referred to. That is, the color difference component of the current block can also be predicted based on the luminance component of the current block. Such intra prediction is sometimes referred to as CCLM (cross-component linear model) prediction. An intra prediction mode such as a color difference block of the reference luminance block (for example, referred to as CCLM mode) can also be added as an intra prediction mode of one color difference block.

The intra prediction unit 124 may correct the intra-predicted pixel value based on the gradient of the reference pixels in the horizontal/vertical direction. The intra prediction accompanying the correction like this is sometimes referred to as PDPC (position dependent intra prediction combination). Applicable information showing the presence or absence of a PDPC (for example, referred to as a PDPC flag), for example, is signaled at the CU level. In addition, the signalization of the information need not be limited to the CU level, but may be other levels (such as sequence level, picture level, slice level, block level, or CTU level).

[Inter-prediction unit] The inter-prediction unit 126 is a reference reference picture for performing inter-block prediction (also called inter-picture prediction) to generate a prediction signal (inter-predictive signal), wherein the reference picture is a frame memory 122 stored reference pictures, and is a reference picture different from the current picture. The inter-prediction is performed in units of a current block or a sub-block within the current block (e.g., a 4x4 block). For example, the inter prediction unit 126 performs motion estimation within the reference picture for the current block or sub-block. Next, the inter prediction unit 126 performs motion compensation by using motion information (for example, a motion vector) obtained by the motion estimation, thereby generating an inter-predicted signal of the current block or the sub-block. Then, the inter prediction unit 126 outputs the generated inter prediction signal to the prediction control unit 128.

The mobile information for motion compensation is signaled. For signal vectorization of motion vectors, a motion vector predictor can also be used. That is, the difference between the motion vector and the predicted motion vector can also be signaled.

In addition, not only the movement information of the current block obtained by the motion estimation but also the movement information of the adjacent block may be used to generate the inter prediction signal. Specifically, the prediction signal according to the mobile information obtained by the motion estimation and the prediction signal according to the mobile information of the adjacent block may be weighted and added, thereby using the sub-block unit in the current block. Generate inter-predictive signals. Such prediction (motion compensation) is sometimes referred to as OBMC (overlapped block motion compensation).

In such an OBMC mode, information showing the size of a sub-block for OBMC (for example, referred to as an OBMC block size) is signaled at a sequence level. Also, information indicating whether or not the OBMC mode is applicable (for example, referred to as an OBMC flag) is signaled by the CU level. In addition, the level of signalization of such information need not be limited to the sequence level and CU level, but may also be other levels (such as picture level, slice level, block level, CTU level, or sub-block level).

The OBMC mode will be described more specifically. 5B and 5C are a flow chart and a conceptual diagram for explaining an outline of a predicted image correction process performed by the OBMC process.

First, a motion vector (MV) assigned to a coding target block is used to obtain a predicted image (Pred) obtained by normal motion compensation.

Next, the motion vector (MV_L) of the encoded left adjacent block is applied to the coding target block, the predicted image (Pred_L) is obtained, and the predicted image and the Pred_L are weighted and superimposed to perform the predicted image. The first correction.

In the same manner, the motion vector (MV_U) of the adjacent block that has been encoded is applied to the coding target block, and the predicted image (Pred_U) is obtained, and the predicted image with the first correction and the Pred_U are weighted. And superimposing, to perform the second correction of the predicted image, and use this as the final predicted image.

In addition, a method of using the two-stage correction of the left adjacent block and the upper adjacent block is described here, but it is also possible to adopt a configuration in which the right adjacent block or the lower adjacent block is used for more times than the 2 stages. The composition of the correction.

In addition, the superimposed region may be an area that is only a part of the vicinity of the block boundary, rather than the pixel area of the block as a whole.

Here, although the prediction image correction processing from one reference picture is described here, the same applies to the case where the prediction image is corrected from the plurality of reference pictures, and the corrected prediction map is obtained from each reference picture. After the image, the obtained predicted image is further superimposed as the final predicted image.

In addition, the foregoing processing target block may also be a prediction block unit, or may be a sub-block unit that further divides the prediction block.

As a method of judging whether or not the OBMC processing is applied, for example, there is a method of using obmc_flag which is a signal indicating whether or not OBMC processing is applied. In a specific example, in the encoding apparatus, it is determined whether the encoding target block belongs to an area where the movement is complicated, and when the movement is a complex area, the set value is 1 as obmc_flag, and the OBMC processing is applied for encoding, When the movement is a complex area, the value is set to 0 as obmc_flag, and OBMC processing is not applied for encoding. Further, in the decoding apparatus, the obmc_flag described in the stream is decoded, and the OBMC processing is switched to perform decoding in accordance with the value.

In addition, the mobile information can also be derived on the decoding device side without being signaled. For example, a merge mode specified by the H.265/HEVC specification can also be used. Further, for example, the motion estimation may be performed on the decoding device side to derive the mobile information. At this time, the motion estimation can be performed without using the pixel value of the current block.

Here, a mode in which the motion estimation is performed on the decoding device side will be described. The mode in which the motion estimation is performed on the decoding device side is sometimes referred to as a PMMVD (pattern matched motion vector derivation) mode or a FRUC (frame rate up-conversion) mode.

An example of FRUC processing is shown in Figure 5D. First, a reference list of spatially or temporally adjacent coded blocks of the current block is generated, and a list of a plurality of candidates (which may also be common to the merge list) is generated, the list of the plurality of candidates each having a predicted movement vector. Next, the best candidate MV is selected from among a plurality of candidate MVs that have been registered in the candidate list. For example, the evaluation value of each candidate included in the candidate list is calculated, and one candidate is selected based on the evaluation value.

Next, the motion vector for the current block is derived based on the selected candidate motion vector. Specifically, for example, the selected candidate motion vector (the best candidate MV) is thus derived as the motion vector for the current block. Also, pattern matching is performed, for example, in a peripheral region of a position within a reference picture, whereby a motion vector for the current block can also be derived, wherein the reference picture is a motion vector corresponding to the selected candidate. That is, for the area around the best candidate MV, the search is performed in the same way, and when there is an MV whose evaluation value is a good number, the best candidate MV is updated to the aforementioned MV, and the MV is regarded as the current block. The last MV is also available. Alternatively, the configuration may be omitted.

When processing in sub-block units, it is also possible to configure the same processing.

Further, the evaluation value is calculated by matching the region in the reference picture corresponding to the motion vector with the pattern between the predetermined region and obtaining a difference value of the reconstructed image. Further, in addition to the difference value, the evaluation value may be calculated using information other than this.

For pattern matching, the first pattern matching or the second pattern matching is used. The first pattern matching and the second pattern matching are sometimes referred to as bidirectional matching and template matching, respectively.

In the first pattern matching, pattern matching is performed between two blocks, which are two blocks in two different reference pictures, and are movement trajectories along the current block (motion Trajectory). Therefore, in the first pattern matching, an area in another reference picture along the movement trajectory of the current block is used as a predetermined area for calculating the evaluation value of the candidate.

Fig. 6 is a view for explaining an example of pattern matching (bidirectional matching) between two blocks along a movement trajectory. As shown in FIG. 6, under the first pattern matching, two blocks in the moving trajectory along the current block (Cur block) and two regions in different two reference pictures (Ref0, Ref1) Among the pair of blocks, the most matching pair is searched, thereby deriving two motion vectors (MV0, MV1). Specifically, for the current block, the reconstructed image in the specified position in the first encoded reference picture (Ref0) specified by the candidate MV is derived, and the symmetricity in which the candidate MV has been scaled at the display time interval is derived. The difference between the reconstructed images in the designated position in the second encoded reference picture (Ref1) designated by the MV is used, and the obtained difference value is used to calculate the evaluation value. Among the plurality of candidate MVs, the candidate MV whose evaluation value is the best is selected as the last MV.

Under the assumption of continuous moving trajectory, the distance between the motion vector (MV0, MV1) of the two reference blocks relative to the current picture (Cur Pic) and the two reference pictures (Ref0, Ref1) is indicated (TD0) TD1) is proportional. For example, the current picture is temporally located between two reference pictures, and when the distance from the current picture to the two reference pictures is equal, the mirror-symmetric two-way motion vector can be derived on the first pattern matching. .

On the second pattern matching, between the template in the current picture (the block adjacent to the current block in the current picture (for example, the upper and/or left adjacent block)) and the block in the reference picture are Pattern matching. Therefore, in the second pattern matching, a block adjacent to the current block in the current picture is used as a predetermined area for calculation of the above-described candidate evaluation value.

FIG. 7 is a diagram for explaining an example of pattern matching (template matching) between a template in a current picture and a block in a reference picture. As shown in FIG. 7, in the second pattern matching, a block that matches the block closest to the current block (Cur block) within the current picture (Cur Pic) is searched within the reference picture (Ref0), thereby Export the motion vector of the current block. Specifically, for the current block, the reconstructed image of the coded region of the left adjacent and the adjacent two sides or either side is derived in the same position as the encoded reference picture (Ref0) specified by the candidate MV. The difference between the images is reconstructed, and the evaluation value is calculated using the obtained difference value, and the candidate MV whose evaluation value is the optimum value is selected among the plurality of candidate MVs as the best candidate MV.

Such information showing whether the FRUC mode is applicable (for example, referred to as the FRUC flag) is signaled by the CU level. Moreover, when the FRUC mode is applied (for example, when the FRUC flag is true), the information of the method of displaying the pattern matching (the first pattern matching or the second pattern matching) (for example, referred to as the FRUC mode flag) is based on the CU level. Be signaled. In addition, the signalization of such information is not limited to the CU level, but may be other levels (such as sequence level, picture level, slice level, block level, CTU level or sub-block level).

Here, a description will be given of a mode in which a motion vector is derived from a model, which is a model assumed to be a constant-speed linear motion. This mode is sometimes referred to as BIO (bi-directional optical flow) mode.

Fig. 8 is a view for explaining a model assumed to be a constant-speed linear motion. In Fig. 8, (v _x , v _y ) represents a velocity vector, and τ ₀ and τ _{1 are} each represented as a temporal distance between a current picture (Cur Pic) and two reference pictures (Ref ₀ , Ref ₁ ). (MVx ₀ , MVy ₀ ) is a motion vector indicating a reference picture Ref ₀ , and (MVx ₁ , MVy ₁ ) is a motion vector indicating a reference picture Ref ₁ .

At this time, the velocity vectors (v _x , v _y ) are under the assumption of constant-speed linear motion, and (MVx ₀ , MVy ₀ ) and (MVx ₁ , MVy ₁ ) are expressed as (v _x τ ₀ , v _y τ ₀ And (-v _x τ ₁ , -v _y τ ₁ ), the following optical flow equation (1) is established. (Number 1)

Here, I ^(k) is a luminance value indicating a reference image k (k = 0, 1) after the motion compensation. The optical flow equation is a product that displays (i) time differential of the luminance value, (ii) the horizontal direction velocity, and the horizontal component of the spatial gradient of the reference image, and (iii) the velocity in the vertical direction and the space of the reference image. The sum of the products of the vertical components of the gradient is equal to zero. According to the combination of the optical flow equation and the Hermitian interpolation, the motion vector of the block unit obtained from the merge list or the like is corrected in units of pixels.

Alternatively, the motion vector may be derived on the decoding device side differently from the method of deriving the motion vector of the model of the assumed constant velocity linear motion. For example, the motion vector may also be derived in sub-block units based on the motion vectors of the plurality of contiguous blocks.

Here, a mode in which a motion vector is derived in units of sub-blocks based on a motion vector of a plurality of adjacent blocks will be described. This mode has a time called an affine motion compensation prediction mode.

Fig. 9A is a diagram for explaining the derivation of a motion vector of a sub-block unit, which is performed based on a motion vector of a plurality of adjacent blocks. In Figure 9A, the current block contains 16 4x4 sub-blocks. Here, the motion vector v ₀ of the upper left corner control point of the current block is derived according to the motion vector of the adjacent block, and the motion vector v ₁ of the upper right corner control point of the current block is derived according to the motion vector of the adjacent subblock. . Next, using two motion vectors v ₀ and v ₁ , the motion vectors (v _x , v _y ) of the respective sub-blocks in the current block are derived via the following equation (2). (number 2)

Here, x and y each represent the horizontal position and the vertical position of the sub-block, and w represents the weight coefficient which has been previously set.

In such an affine motion compensation prediction mode, it is also possible to include several modes in which the motion vectors of the upper left and upper right control points are different. Information showing such an affine motion compensated prediction mode (e.g., referred to as an affine flag) is signaled at the CU level. In addition, the signalization of the information showing the affine motion compensation prediction mode need not be limited to the CU level, but may be other levels (eg, sequence level, picture level, slice level, block level, CTU level, or sub-block level).

[Predictive Control Unit] The prediction control unit 128 selects any one of the intra prediction signal and the inter prediction signal, and outputs the selected signal as a prediction signal to the subtraction unit 104 and the addition unit 116.

Here, an example of deriving a motion vector of a coding target picture via a merge mode will be described. Fig. 9B is a diagram for explaining an outline of a motion vector derivation process by a merge mode.

First, a list of predicted MVs of candidates for the registered prediction MV is generated. The candidate for predicting the MV includes: a spatial adjacency prediction MV, which is a MV of a plurality of coded blocks located in the periphery of the spatial region of the coding target block; the temporal adjacent prediction MV is a projection to the encoded reference. The MV of the vicinity block of the position of the coding target block in the picture; the combined prediction MV is the MV generated by combining the MV value of the spatial adjacent prediction MV and the temporal adjacent prediction MV; and the zero prediction MV whose value is zero MV and so on.

Next, from among the plurality of prediction MVs that have been registered in the prediction MV list, one prediction MV is selected, and this is determined as the MV of the encoding target block.

Further, in the variable length coding unit, the merge_idx is described in the stream, and the merge_idx is a signal indicating which prediction MV has been selected.

In addition, the predicted MV registered in the predicted MV list illustrated in FIG. 9B is only an example, and may be a number different from the number in the figure, or a type including a part of the predicted MV in the figure, or The structure of the prediction MV other than the type of the prediction MV in the figure is added.

Alternatively, the MV of the encoding target block derived by the merge mode may be used to perform the DMVR processing described later, thereby determining the last MV.

Here, an example in which MV is determined using DMVR processing will be described.

Fig. 9C is a conceptual diagram for explaining an outline of DMVR processing.

First, the most suitable MVP that has been set in the processing target block is used as the candidate MV. According to the candidate MV, the processed picture from the L0 direction, that is, the first reference picture and the processed picture in the L1 direction are the second. Referring to the picture, the reference pixels are respectively obtained, and the average of each reference pixel is taken to generate a template.

Next, using the template, the peripheral regions of the candidate MVs of the first reference picture and the second reference picture are respectively searched, and the MV whose cost is the smallest is determined as the last MV. Further, the cost value is calculated by using a difference value between each pixel value of the template and each pixel value of the search area, an MV value, and the like.

Further, in the encoding device and the decoding device, the outline of the processing described herein is basically common.

Further, even if it is not the processing content described here, other processing may be used as long as it is a process of searching for the periphery of the candidate MV and deriving the last MV.

Here, a mode in which a predicted image is generated using the LIC processing will be described.

9D is a diagram for explaining an outline of a prediction image generation method using luminance correction processing by LIC processing.

First, the MV is derived from the reference picture, wherein the reference picture is an encoded picture, and the MV is used to obtain a reference picture corresponding to the coding target block.

Next, for the coding target block, the brightness pixel value of the left adjacent and upper adjacent coded surrounding reference area and the brightness pixel value located at the same position in the reference picture specified by the MV are used to extract information and calculate brightness correction. The parameter, where the information is displayed, shows how the brightness value changes in the reference picture and the encoded object picture.

For the reference image in the reference picture specified by the MV, the brightness correction processing is performed using the aforementioned luminance correction parameter, thereby generating a predicted image with respect to the encoding target block.

In addition, the shape of the aforementioned peripheral reference area in FIG. 9D is only one example, and other shapes may be used.

Here, the process of generating a predicted image from one reference picture has been described here, but the same is true for the case where a predicted picture is generated from a plurality of reference pictures. First, the reference picture obtained from each reference picture is first. For example, the brightness correction process is performed in the same manner, and then the predicted image is generated.

For the method of judging whether or not the LIC processing is applicable, for example, there is a method of using lic_flag, which is a signal indicating whether LIC processing is applicable. In a specific example, the encoding apparatus determines whether the encoding target block belongs to an area where the luminance change occurs, and if it belongs to the area where the luminance change occurs, sets the value to 1 for lic_flag, and applies LIC processing to perform encoding. If it is not in the area where the brightness change occurs, set the value to 0 for lic_flag, and do not apply LIC processing to encode. On the other hand, in the decoding apparatus, the lic_flag described in the stream is decoded, and the LIC processing is switched to perform decoding in accordance with the value.

For another method of judging whether or not the LIC processing is applicable, for example, there is also a method of judging whether or not the LIC processing is applied to the peripheral block. In a specific example, when the coding target block is in the merge mode, it is determined whether the coded block in the vicinity of the MV in the merge mode process is encoded by the LIC process, and the result is whether the switch is performed. Apply LIC processing and encode. In addition, in the case of this example, the processing in decoding is also identical.

[Outline of Decoding Device] Next, an outline of a decoding device that can decode an encoded signal (encoded bit stream) output from the encoding device 100 will be described. FIG. 10 is a block diagram showing a functional configuration of a decoding device 200 according to the first embodiment. The decoding device 200 is a moving image/image decoding device that decodes a moving image/image in units of blocks.

As shown in FIG. 10, the decoding apparatus 200 includes an entropy decoding unit 202, an inverse quantization unit 204, an inverse conversion unit 206, an addition unit 208, a block memory 210, a loop filter unit 212, a frame memory 214, and an intra prediction. The unit 216, the inter prediction unit 218, and the prediction control unit 220.

The decoding device 200 can be realized, for example, by a general purpose processor and a memory. At this time, when the software program stored in the memory is executed by the processor, the processor functions as the entropy decoding unit 202, the inverse quantization unit 204, the inverse conversion unit 206, the addition unit 208, the loop filter unit 212, and the internal prediction unit 216. The inter prediction unit 218 and the prediction control unit 220 operate. Further, the decoding device 200 may correspond to the entropy decoding unit 202, the inverse quantization unit 204, the inverse conversion unit 206, the addition unit 208, the loop filter unit 212, the intra prediction unit 216, the inter prediction unit 218, and the prediction control unit 220. One or more dedicated electronic circuits are dedicated to implementation.

Hereinafter, each component included in the decoding device 200 will be described.

[Entropy Decoding Unit] The entropy decoding unit 202 performs entropy decoding on the encoded bit stream. Specifically, the entropy decoding unit 202 performs arithmetic decoding from the encoded bit stream to become a binary signal, for example. Next, the entropy decoding unit 202 demultiplexes the binary signal. Thereby, the entropy decoding unit 202 outputs the quantized coefficients to the inverse quantization unit 204 in units of blocks.

[Inverse Quantization Unit] The inverse quantization unit 204 inversely quantizes the quantized coefficients of the decoding target block (hereinafter referred to as the current block) which is an input from the entropy decoding unit 202. Specifically, the inverse quantization unit 204 is for each of the quantized coefficients of the current block, and inversely quantizes the quantized coefficients according to the quantization parameter corresponding to the quantized coefficients. Then, the inverse quantization unit 204 outputs the dequantized quantized coefficients (i.e., conversion coefficients) of the current block to the inverse conversion unit 206.

[Inverse Conversion Unit] The inverse conversion unit 206 restores the prediction error by inversely converting the conversion coefficient, which is an input from the inverse quantization unit 204.

For example, when the information that has been interpreted from the encoded bit stream is displayed when EMT or AMT is applied (for example, the AMT flag is true), the inverse conversion unit 206 converts the current block according to the information of the converted conversion pattern. Perform a reverse conversion.

Further, for example, when the information that has been interpreted from the encoded bit stream is the display applicable NSST, the inverse conversion unit 206 applies inverse retransformation to the conversion coefficient.

[Addition Unit] The addition unit 208 adds the prediction error to the prediction block, thereby reconstructing the current block, wherein the prediction error is an input from the inverse conversion unit 206, which is an input from the prediction control unit 220. Then, the addition unit 208 outputs the reconstructed block to the block memory 210 and the loop filter unit 212.

[Block Memory] The tile memory 210 is a memory portion for storing a block referenced in the intra prediction and for decoding a block in the target picture (hereinafter referred to as a current picture). Specifically, the tile memory 210 stores the reconstructed block output from the addition unit 208.

[Loop Filter Unit] The loop filter unit 212 performs loop filtering on the block that has been reconstructed by the adder unit 208, and outputs the filtered reconstructed block to the frame memory 214, the display device, and the like.

Displaying the information that the ALF is turned on/off from the encoded bit stream is displayed. When the ALF is turned on, one filter is selected from a plurality of filters according to the direction and activity of a part of the gradient. The selected filter is suitable for reconstructing blocks.

[Frame Memory] The frame memory 214 is a memory for storing reference pictures that are used for prediction, and is sometimes referred to as a frame buffer. Specifically, the frame memory 214 stores the reconstructed block filtered by the loop filter unit 212.

[Internal Prediction Unit] The intra prediction unit 216 performs intra prediction by referring to the intra-prediction mode that has been interpreted from the encoded bit stream, and refers to the block in the current picture stored in the block memory 210, thereby generating a prediction signal. (internal prediction signal). Specifically, the intra prediction unit 216 performs intra prediction by referring to samples (for example, luminance values and color difference values) of the blocks adjacent to the current block, thereby generating an intra prediction signal, and outputting the intra prediction signal to the prediction control. Department 220.

Further, in the intra prediction of the chroma block, when the intra prediction mode of the reference luma block is selected, the intra prediction unit 216 may also predict the chroma component of the current block based on the luminance component of the current block.

Further, when the information display PDPC is interpreted from the encoded bit stream, the intra prediction unit 216 corrects the intra-predicted pixel value based on the gradient of the reference pixels in the horizontal/vertical direction.

The inter-prediction unit 218 is a reference picture stored in the reference frame memory 214 to predict the current block. The prediction is made in units of the current block or sub-blocks within the current block (eg, 4x4 blocks). For example, the inter prediction unit 218 performs motion compensation using the mobile information (for example, motion vector) that has been interpreted from the encoded bit stream, thereby generating an inter-prediction signal of the current block or sub-block, and outputting the inter-predicted signal. To the prediction control unit 220.

In addition, when the information that has been interpreted from the encoded bit stream is displayed in the applicable OBMC mode, the inter prediction unit 218 not only uses the mobile information of the current block obtained through the motion estimation, but also uses the mobile information of the adjacent block. Generate inter-predictive signals.

Further, when the information that has been interpreted from the encoded bit stream is displayed in the applicable FRUC mode, the inter prediction unit 218 performs the motion estimation in accordance with the pattern matching method (bidirectional matching or template matching) that has been interpreted from the encoded stream. This allows you to export mobile information. Then, the inter prediction unit 218 performs motion compensation using the derived movement information.

Further, when the BIO mode is applied, the inter prediction unit 218 derives a motion vector based on a model of a hypothetical linear motion. Further, when the affine motion compensation prediction mode is applied to the information read from the coded bit stream, the inter prediction unit 218 derives the motion vector in units of sub-blocks based on the motion vectors of the plurality of adjacent blocks.

[Predictive Control Unit] The prediction control unit 220 selects either one of the intra prediction signal and the inter prediction signal, and outputs the selected signal as a prediction signal to the addition unit 208.

[Predictive Motion Vector Correlation Processing in Encoding Device and Decoding Device] In the moving image encoding method such as H.264 and HEVC, 3 is called I picture, P picture, and B picture due to information amount compression. Picture type. I pictures are encoded by intra-picture prediction. Intra-picture prediction is also called inter-picture prediction or intra prediction.

The P picture is encoded by referring to the prediction between the coded pictures that are displayed in front of or behind the picture to be encoded in the display time order. The B picture is encoded by referring to prediction between the two pictures encoded in front of or behind the picture in the display chronological order.

In inter prediction, a list of reference pictures that are useful for a particular reference picture is generated. In the reference picture list, the reference picture index is assigned to the encoded reference picture referenced by the inter prediction. For example, in the encoding of a B picture, two pictures can be referred to. To this end, two reference picture lists (L0 reference picture list and L1 reference picture list) are generated.

Regarding the method of encoding the motion vector of the coding target block in the B picture or the P picture, there is a prediction motion vector designation mode. In the prediction motion vector designation mode, a neighboring block and a co-located block are used, wherein the adjacent block is a block adjacent to the encoding target block, and the co-located block is located in the reference picture. Encode the same block of the object block.

Specifically, the predicted motion vector candidate is derived from the motion vectors that have been encoded in the adjacent blocks and the co-located blocks. The predicted motion vector candidate is also referred to as a candidate motion vector. Further, the block used for the prediction of the motion vector candidate and the adjacent block and the parity block are also referred to as candidate blocks.

Then, a prediction motion vector is selected from among a plurality of prediction motion vector candidates, so that the motion vector of the coding target block is encoded. At this time, an index or the like of the selected predicted motion vector is added to the bit stream.

In addition, regarding the prediction of the motion vector, the method of using the motion vector of the co-located block is called TMVP (Temporal Motion Vector Prediction). Also, the co-located block is sometimes called a Col block. Also, the motion vector and reference picture of each block are motion vectors and reference pictures that have been used for (or for) inter-block prediction.

Figure 11 is a conceptual diagram showing the motion vectors of adjacent blocks and adjacent blocks. In the example of Fig. 11, four adjacent blocks A, B, C, and D are four blocks adjacent to the encoding target block. Four prediction motion vector candidates can also be derived from four motion vectors mvA, mvB, mvC, and mvD of four adjacent blocks A, B, C, and D.

Specifically, the motion vector mvA of the adjacent block A can also be derived as a prediction motion vector candidate as it is. Alternatively, the motion vector mvA of the adjacent block A may be scaled to derive the scaled motion vector mvA as a predicted motion vector candidate. Here, the scaling of the motion vector refers to scaling the size of the motion vector, and also includes inverting the motion vector using a negative ratio.

For the above scaling, the ratio of the two time differences may be used, that is, the time difference from the encoding target picture to the reference picture of the encoding target block relative to the reference picture from the encoding target picture to the adjacent block A The ratio of time differences. Further, for example, the time difference from the first picture to the second picture is a value obtained by subtracting the display order of the first picture from the display order of the second picture, and a negative value is obtained.

Further, similarly to the motion vector mvA of the adjacent block A, the motion vector predictor candidates can be derived from the motion vectors mvB, mvC, and mvD adjacent to the blocks B, C, and D.

Figure 12 is a conceptual diagram showing the motion vectors of the Col block and the Col block. In addition, each picture is two-dimensional, but each picture in FIG. 12 and the like is presented in one dimension for convenience of explanation.

In the example of FIG. 12, the Col block is encoded using the motion vector mvCol and referring to the first reference picture. For example, a predicted motion vector candidate is derived from the motion vector mvCol of the Col block.

Specifically, the motion vector mvCol of the Col block can also be derived as a prediction motion vector candidate as it is. Alternatively, the motion vector mvCol of the Col block may be scaled, and the scaled motion vector mvCol may be derived as a prediction motion vector candidate. For the scaling, the following two time difference ratios may also be used, and the time difference from the encoding target picture to the reference picture of the encoding target block is relative to the reference picture including the second reference picture of the Col block up to the Col block. The ratio of the time difference.

Furthermore, the next generation of HEVC's next-generation dynamic image coding method is being discussed by JVET (Joint Video Exploration Team), a review team jointly operated by MPEG and ITU-T. Among them, TMVP (Advanced Temporal Motion Vector Prediction), which is an improved method of TMVP, has been proposed.

For example, in ATMVP, the motion vector of the coding target block is temporarily determined according to the motion vector of the adjacent block. Next, the block indicated by the temporarily determined motion vector is determined to be the ATMVP block. Next, the motion vector (ATMV: Advanced Temporal Motion Vector) in the ATMVP block is used as a prediction motion vector candidate.

In addition, the ATMVP block is sometimes referred to as an Adv block. An ATMVP block is an example of a candidate block that is used in the derivation of a candidate motion vector.

Figure 13 is a conceptual diagram showing the motion vectors of Adv blocks and Adv blocks. For example, the motion vector mvD of the adjacent block D shown in FIG. 11 is temporarily determined as the motion vector of the encoding target block. Next, the block indicated by the encoding target block by the motion vector mvD is determined to be an Adv block.

Further, for example, when the region indicated by the second reference picture by the motion vector mvD is the generation of the prediction image used in the adjacent block D, the Adv block is included in the second reference picture. That is, the reference picture containing the Adv block is represented by a reference picture index showing the reference picture of the adjacent block D. Then, the predicted motion vector candidate is derived from the motion vector mvAdv of the Adv block.

Specifically, the motion vector mvAdv of the Adv block can also be derived as a prediction motion vector candidate as it is. Alternatively, the motion vector mvAdv of the Adv block may be scaled to derive the scaled motion vector mvAdv as a prediction motion vector candidate. For the scaling, a ratio of two time differences, that is, a time difference from the encoding target picture to the reference picture of the encoding target block, relative to the second reference picture including the Adv block to the Adv block, may be used. The ratio of the time difference of the picture.

For example, even in the case where the encoding target block is an image within the range of the moving object, the Adv block can follow the movement of the moving object, and is determined to be in the same position relative to the encoding target block within the range of the moving object. . The motion vector of such an Adv block may have the same characteristics as the motion vector of the encoding target block. Therefore, suitable prediction motion vector candidates can be derived from the motion vector mvAdv of the Adv block.

Further, in the decision of the Adv block, the motion vector mvD and the second reference picture used for the inter prediction of the adjacent block D need not be used as they are. The Adv block may also be determined among reference pictures of the second reference picture used for inter prediction from the adjacent block D. For example, the Adv block can also be determined in the front of the encoding object picture or in the reference picture immediately after the picture.

Next, scaling may be applied to the motion vector mvD used for inter prediction of the adjacent block D, and the deviated block is determined using the scaled motion vector mvD. For the scaling, the ratio of the two time differences may be used, that is, the time difference from the encoding target picture to the reference picture determined by the Adv block relative to the prediction from the encoding target picture to the adjacent block D. The ratio of the time difference of the second reference picture used.

As described above, in the ATMVP currently being discussed, the motion vector of the coding target block is temporarily determined based on the motion vector of the adjacent block, and the block decision indicated by the temporarily determined motion vector is determined as the ATMVP area. Piece. That is, a single ATMVP block is determined. The predicted motion vector candidate can then be derived from the motion vector of a single ATMVP block.

Similarly, TMVP determines a single co-located block and derives a predicted motion vector candidate from the motion vector of a single co-located block. That is, whether it is ATMVP or TMVP, the predicted motion vector candidate is derived from the motion vector of a single block. Therefore, among the blocks included in the L0 reference picture list and the blocks included in the L1 reference block list, only the predicted motion vector candidates are derived from one of the blocks.

The predicted motion vector candidate derived as described above is not necessarily a suitable prediction motion vector candidate. When the coding target block is located near the boundary of the range of the moving object in the image, and the movement within the range of the moving object is different from the movement outside the range of the moving object, the movement vector of the ATMVP block and the coding target block are The motion vectors are likely to be very different.

Fig. 14 is a conceptual diagram showing an example of a relationship between a coding target block, four adjacent blocks A, B, C, and D, and a range of moving objects. In this example, the encoding target block is a boundary spanning the range of the moving object, and further, the encoding target block is mostly included in the range of the moving object. In this example, the adjacent block D is located outside the range of the moving object, specifically included in the background.

Figure 15 is a conceptual diagram showing an example of an unsuitable Adv block. Specifically, FIG. 15 is an Adv block determined in accordance with the motion vector mvD of the adjacent block D in the example of FIG.

When the Adv block is determined according to the motion vector mvD of the adjacent block D, the Adv block is located outside the range of the moving object, and is highly likely to be included in the background. Therefore, at this time, the motion vector of the Adv block is a movement showing the background, and the movement of most of the coding target blocks included in the range of the mobile object cannot be correctly displayed. Therefore, from the motion vector of the Adv block, suitable predictive motion vector candidates are not derived.

The encoding apparatus 100 and the decoding apparatus 200 of the present embodiment use the blocks in the reference picture included in the L0 reference picture list and the blocks in the reference picture included in the L1 reference picture list to derive the temporal movement of the predicted movement. Vector candidate. Thereby, the encoding apparatus 100 and the decoding apparatus 200 can improve the possibility that the predicted motion vector candidate of the temporal property integrated with the movement of the processing target block is derived even when the processing target block is located near the boundary of the range of the moving object. Sex.

In addition, the temporal property prediction motion vector candidate is a prediction motion vector candidate derived from a tile determined with respect to the relationship of the processing object block use time property, and is also referred to as a temporal candidate motion vector. Further, the spatial motion prediction motion vector candidate is a prediction motion vector candidate derived from a tile determined by the relationship of the spatial properties of the processing target tile, and is also referred to as a spatial candidate motion vector.

[Detailed Configuration of Encoding Device and Decoding Device] FIG. 16 is a block diagram showing more specifically the functional configuration of the encoding device 100 shown in FIG. 1. Compared with FIG. 1, the picture type determining unit 132, the candidate list generating unit 134, and the candidate block information memory 136 are further shown in FIG. In other words, the encoding device 100 further includes a picture type determining unit 132, a candidate list generating unit 134, and a candidate block information memory 136.

As described above, the encoding device 100 is realized by, for example, a general-purpose processor and a memory. At this time, when the software program stored in the memory is executed by the processor, the processor functions as the picture type determining unit 132 and the candidate list generating unit 134. Further, the encoding device 100 can be realized as one or more dedicated electronic circuits corresponding to a plurality of constituent elements including the picture type determining unit 132 and the candidate list generating unit 134.

The picture pattern determining unit 132 determines the picture type for each of a plurality of pictures constituting the moving picture. For example, the picture type determining unit 132 determines a picture type of each picture from among three picture types called an I picture, a P picture, and a B picture, and selects a picture type for each picture. The picture pattern determining unit 132 may determine the picture type in accordance with the encoding order or the display order of the pictures.

Further, the picture type determining unit 132 outputs the picture type determined for each picture, thereby inputting the picture type to the prediction control unit 128. The prediction control unit 128 selects one of the intra prediction signal and the inter prediction signal according to the input picture pattern, and outputs the selected signal as a prediction signal.

For example, when the input picture type is an I picture, the prediction control unit 128 selects the intra prediction signal and outputs the intra prediction signal. On the other hand, when the input picture type is a P picture or a B picture, the prediction control unit 128 selects the inter prediction signal and outputs the inter prediction signal. Alternatively, in the P picture or the B picture, the prediction control unit 128 may select any one of the intra prediction signal and the inter prediction signal for each block, and use the selected signal as the prediction signal output.

The candidate list generation unit 134 is a list that generates candidate motion vectors. In other words, the candidate list generating unit 134 generates a list of one or more candidate motion vectors. The list can also be expressed as a candidate list, a candidate motion vector list, or a predicted motion vector candidate list. The details of the generation of this list will be described later using FIGS. 18A to 30 and the like.

Further, the candidate list generating unit 134 outputs the generated candidate list, thereby inputting the candidate list to the inter prediction unit 126. The inter prediction unit 126 selects a prediction motion vector from among one or more candidate motion vectors included in the input candidate list.

Next, the entropy coding unit (variable length coding unit) 110 encodes the prediction motion vector index and the difference motion vector, wherein the prediction motion vector index is an index of the selected prediction motion vector, and the difference motion vector is selected. A differential motion vector that predicts the motion vector and the motion vector of the encoding object block. For example, in the prediction motion vector designation mode, the entropy coding unit 110 encodes the prediction motion vector index and the difference motion vector.

Further, in the merge mode, the inter prediction unit 126 generates a predicted image by using the selected predicted motion vector as a motion vector of the encoding target block. Further, in the merge mode, the entropy coding unit 110 does not encode the difference motion vector, but encodes the prediction motion vector index.

Further, the inter prediction unit 126 may select a prediction motion vector in accordance with the FRUC, and use the selected prediction motion vector as a motion vector of the coding target block to generate a prediction image. At this time, the entropy coding unit 110 does not encode the motion vector information such as the prediction motion vector index and the difference motion vector.

The candidate block information memory 136 is a memory for storing information used in the generation of the candidate motion vectors. Specifically, the candidate block information memory 136 stores the information of the candidate block in the encoding target picture or the reference picture. The information of the candidate block is, for example, information showing the motion vector of the candidate block.

More specifically, the inter prediction unit 126 may also store the following information in the candidate block information memory 136, wherein the information is information of the motion vector used to display the predicted image generation in the encoding target block. Then, the candidate list generating unit 134 may also use the information stored in the candidate block information memory 136 as the information of the candidate block, and refer to when generating the predicted image of the other encoding target block.

The inter prediction unit 126 may include the candidate list generation unit 134 or the candidate block information memory 136. That is, the inter prediction unit 126 can exhibit the role of the candidate list generation unit 134, and can also exhibit the role of the candidate block information memory 136.

Further, inter prediction coding is performed by the subtraction unit 104, the entropy coding unit 110, the addition unit 116, the inter prediction unit 126, and the like. These constituent elements may constitute an inter-prediction encoding unit.

Fig. 17 is a block diagram showing more specifically the functional configuration of the decoding device 200 shown in Fig. 10. Compared with FIG. 10, the candidate list generating unit 234 and the candidate block information memory 236 are further shown in FIG. That is, the decoding device 200 further includes a candidate list generating unit 234 and a candidate block information memory 236.

As described above, the decoding device 200 is realized by, for example, a general-purpose processor and a memory. At this time, when the software program stored in the memory is executed by the processor, the processor further functions as the candidate list generating unit 234 or the like. Further, the decoding device 200 can be realized as one or more dedicated electronic circuits corresponding to a plurality of constituent elements including the candidate list generating unit 234 and the like.

The candidate list generating unit 234 of the decoding device 200 operates in the same manner as the candidate list generating unit 134 of the encoding device 100. In other words, the candidate list generating unit 234 generates a candidate list composed of one or more candidate motion vectors. The generation of the candidate list is detailed later using FIGS. 18A to 30. 18A to 30, etc., mainly the operation of the encoding apparatus 100 is shown. However, the decoding apparatus 200 operates in the same manner as the encoding apparatus 100 with respect to the generation of the candidate list. The codes in these descriptions can be appropriately replaced with decoding.

Further, the candidate list generating unit 234 outputs the generated candidate list, thereby inputting the candidate list to the inter prediction unit 218. The inter prediction unit 218 selects a prediction motion vector from among one or more candidate motion vectors included in the input candidate list.

For example, the entropy decoding unit (variable length decoding unit) 202 decodes the predicted motion vector index and the differential motion vector in the predicted motion vector designation mode. At this time, the inter prediction unit 218 selects the predicted motion vector in accordance with the predicted motion vector index.

Next, the inter prediction unit 218 adds the selected prediction motion vector to the decoded difference motion vector, thereby deriving the motion vector of the decoding target block. Further, the inter prediction unit 218 generates a predicted image of the decoding target block using the motion vector of the decoding target block.

Further, in the merge mode, the entropy decoding unit 202 does not decode the differential motion vector, but decodes the predicted motion vector index. At this time, the inter prediction unit 218 also selects the predicted motion vector in accordance with the predicted motion vector index. Then, the inter prediction unit 218 uses the selected predicted motion vector as the motion vector of the decoding target block to generate a predicted image of the decoding target block.

Further, the inter prediction unit 218 may select a prediction motion vector in accordance with the FRUC, and use the selected prediction motion vector as a motion vector of the decoding target block to generate a prediction image. At this time, the entropy decoding unit 202 does not decode the motion vector information such as the predicted motion vector index and the differential motion vector.

The candidate block information memory 236 is a type of memory for storing information generated using the candidate motion vectors. That is, the candidate block information memory 236 of the decoding device 200 can function as the candidate block information memory 136 of the encoding device 100. Specifically, the candidate block information memory 236 stores the information of the candidate block in the decoding target picture or the reference picture.

More specifically, the inter prediction unit 218 can also store the following information in the candidate block information memory 236, wherein the information is a motion vector generated by displaying the predicted image that has been used in the decoding target block. Then, the candidate list generation unit 234 may also use the information stored in the candidate block information memory 236 as the information of the candidate block, and refer to when generating the predicted image of the other decoding target block.

Further, the entropy decoding unit 202 decodes the picture type of each picture. Next, the entropy decoding unit 202 outputs a picture type of each picture, thereby inputting the picture type of each picture to the prediction control unit 220. Similarly to the prediction control unit 128 of the encoding device 100, the prediction control unit 220 selects one of the intra prediction signal and the inter prediction signal in accordance with the input picture pattern, and outputs the selected signal as a prediction signal.

The inter prediction unit 218 may include the candidate list generation unit 234 or the candidate block information memory 236. In other words, the inter-prediction unit 218 can also function as the candidate list generating unit 234, or can function as the candidate block information memory 236.

Further, inter-prediction decoding is performed by the entropy decoding unit 202, the addition unit 208, the inter prediction unit 218, and the like. These constituent elements may also constitute an inter prediction decoding unit.

[Candidate Motion Vector Derived from Co-located Block] When the encoding target picture is a B picture, the L0 reference picture list and the L1 reference picture list are determined for the encoding target picture. For example, the temporal candidate motion vector is derived from each of the co-located block of the reference picture included in the L0 reference picture list and the co-located block of the reference picture included in the L1 reference picture list.

Basically, the L0 reference picture list is a reference picture including a picture that is earlier than the picture of the coding object in the display order, and the L1 reference picture list is a reference picture that is later than the picture of the picture to be encoded. However, not limited to this, the L0 reference picture list may also include a reference picture that is later than the coding target picture in the display order, and the L1 reference picture list may also include a reference picture that is earlier than the coding target picture in the display order.

Here, the reference picture included in the L0 reference picture list is sometimes expressed as the reference picture L0[n]. Also, the reference picture included in the L1 reference picture list is sometimes expressed as the reference picture L1[n]. n in the reference pictures L0[n] and L1[n] is a reference picture index and is an integer of 0 or more.

Also, the co-located block in the reference picture included in the L0 reference picture list is sometimes referred to as a Col0 block. Also, the co-located block in the reference picture included in the L1 reference picture list is sometimes referred to as a Col1 block.

For example, the Col0 block is in the L0 reference picture list, and is included in the reference picture L0[0] whose reference picture index is 0, which is a block at the same position as the coding target block. Also, for example, the Col1 block is a block in the L1 reference picture list, and is included in the reference picture L1[0] whose reference picture index is 0, which is the same position as the coding target block.

Then, the temporal candidate motion vector is derived from each of the Col0 block and the Col1 block. First, an example in which each of the Col0 block and the Col1 block is encoded by bi-prediction will be described with reference to FIGS. 18A and 18B.

Figure 18A is a conceptual diagram showing candidate motion vectors derived from a bi-predicted Col0 block. In this example, the reference pictures L0[0] and L0[1] contained in the L0 reference picture list are located in front of the picture to be encoded, and the reference pictures L1[0] and L1[1] included in the L1 reference picture list. Is located later than the encoding object image.

The Col0 block is included in the reference picture L0[0] and is encoded by bi-prediction. In the bi-prediction of the Col0 block, the reference picture L0[1] in the front direction is referred to by the motion vector mvCol0L0, and the reference picture L1[1] in the back direction is referred to by the motion vector mvCol0L1.

In this example, the motion vector mvCol0L1 is performed with a ratio of the time difference from the encoding target picture to the reference picture L0[0] with respect to the time difference from the reference picture L0[0] to the reference picture L1[1]. target. Thereby, the motion vector mvCol0L1L0 of the reference reference picture L0[0] is derived as a temporal candidate motion vector.

Further, the motion vector mvCol0L1 is scaled by the ratio of the time difference from the encoding target picture to the reference picture L1[0] with respect to the time difference from the reference picture L0[0] to the reference picture L1[1]. Thereby, the motion vector mvCol0L1L1 of the reference reference picture L1[0] is derived as a temporal candidate motion vector.

In other words, the candidate list generation unit 134 scales the motion vector mvCol0L1 of the Col0 block, and derives each of the two motion vectors mvCol0L1L0 and mvCol0L1L1 that can be used for bi-prediction as the temporal candidate motion vector.

In this example, among the two motion vectors mvCol0L0 and mvCol0L1 of the Col0 block, the motion vector mvCol0L1 whose reference direction and the reference picture L0[0] including the Col0 block are aligned to the direction of the encoding target picture is used. . Thereby, among the two motion vectors mvCol0L0 and mvCol0L1, the temporal candidate motion vector is derived using the motion vector mvCol0L1 which is assumed to have high correlation with the coding target block.

Alternatively, the motion vector mvCol0L0 can be used instead of the motion vector mvCol0L1. For example, the candidate list generating unit 134 may evaluate each of the two motion vectors mvCol0L0 and mvCol0L1 in accordance with a predetermined evaluation criterion, and select one of the two motion vectors mvCol0L0 and mvCol0L1. Then, the candidate list generation unit 134 may derive the temporal candidate motion vector from the motion vector of one of the selected ones.

Specifically, one motion vector may be selected from the two motion vectors mvCol0L0 and mvCol0L1 of the Col0 block according to the size of each motion vector or the temporal distance from the reference picture referenced by each motion vector. For example, a moving vector having a small size may be selected, or a moving vector having a short temporal distance may be selected.

Alternatively, the candidate list generating unit 134 may calculate a motion vector having a calculated prediction residual or a small R-D cost among the two motion vectors mvCol0L0 and mvCol0L1 based on the prediction residual or the R-D cost of each motion vector. Here, the R-D cost is a weighted sum of the coding distortion and the amount of generated coding.

Next, the entropy coding unit 110 may encode the identification information of the selected motion vector. Further, in the decoding device 200, the entropy decoding unit 202 may decode the identification information of the motion vector, and the candidate list generation unit 234 selects the motion vector in accordance with the decoded identification information.

In particular, when the Col0 block is encoded by bi-prediction in only one direction among the front direction and the backward direction, it may also be derived by reference to one of the two motion vectors of the Col0 block. Time candidate motion vector in the opposite direction.

That is, the candidate list generating unit 134 may select one motion vector from the two motion vectors in accordance with the above-described selection method, and apply a negative ratio to the selected one motion vector. Standard. Thereby, a temporal candidate motion vector whose reference direction is opposite to the selected one motion vector can be derived.

Also, in addition to the motion vector mvCol0L1, the motion vector mvCol0L0 can also be used. For example, the candidate list generation unit 134 may derive two temporal candidate motion vectors available for bi-prediction from each of the two motion vectors mvCol0L0 and mvCol0L1. Thereby, the candidate list generating unit 134 may derive four time candidate motion vectors from the two motion vectors mvCol0L0 and mvCol0L1.

Figure 18B is a conceptual diagram showing candidate motion vectors derived from a bi-predicted Col1 block. Each picture and coding target block in the example is the same as the example of FIG. 18A.

The Col1 block is included in the reference picture L1[0] and is encoded by bi-prediction. In the double prediction of the Col1 block, the reference picture L0[1] in the front direction is referenced by the motion vector mvCol1L0, and the reference picture L1[1] in the back direction is referenced by the motion vector mvCol1L1.

In this example, the motion vector mvCol1L0 is scaled by the ratio of the time difference from the encoding target picture to the reference picture L0[0] with respect to the time difference from the reference picture L1[0] to the reference picture L0[1]. . Thereby, the motion vector mvCol1L0L0 of the reference reference picture L0[0] is derived as a temporal candidate motion vector.

Further, the motion vector mvCol1L0 is scaled by the ratio of the time difference from the encoding target picture to the reference picture L1[0] with respect to the time difference from the reference picture L1[0] to the reference picture L0[1]. Thereby, the motion vector mvCol1L0L1 of the reference reference picture L1[0] is derived as a temporal candidate motion vector.

That is, the candidate list generating unit 134 scales the motion vector mvCol1L0 of the Col1 block, and derives each of the two motion vectors mvCol1L0L0 and mvCol1L0L1 available for bi-prediction as the temporal candidate motion vector.

In this example, among the two motion vectors mvCol1L0 and mvCol1L1 of the Col1 block, the motion vector mvCol1L0 whose reference direction is the same as the direction from the reference picture L1[0] including the Col1 block to the picture to be encoded is used. . Thereby, among the two motion vectors mvCol1L0 and mvCol1L1, the temporal candidate motion vector is derived using the motion vector mvCol1L0 which is assumed to have high correlation with the coding target block.

Alternatively, the motion vector mvCol1L1 can be used instead of the motion vector mvCol1L0. For example, the candidate list generating unit 134 may evaluate each of the two motion vectors mvCol1L0 and mvCol1L1 in accordance with a predetermined evaluation criterion, and select one of the two motion vectors mvCol1L0 and mvCol1L1. Then, the candidate list generation unit 134 may also derive the temporal candidate motion vector from one of the selected motion vectors.

Specifically, the candidate list generation unit 134 can select one motion vector from among the two motion vectors mvCol1L0 and mvCol1L1 of the Col1 block, as in the case of selecting one motion vector for the Col0 block. In particular, similarly to the Col0 block, when the Col1 block is encoded by bi-prediction in only one of the front direction and the backward direction, it can also be derived by scaling one of the two motion vectors. Used to refer to the temporal candidate motion vector in the opposite direction.

Further, in addition to the motion vector mvCol1L0, the motion vector mvCol1L1 can also be used. For example, the candidate list generating unit 134 may derive two time candidate motion vectors usable in the bi-prediction from each of the two motion vectors mvCol1L0 and mvCol1L1. Thereby, the candidate list generating unit 134 can derive four time candidate motion vectors from the two motion vectors mvCol1L0 and mvCol1L1.

Next, an example in which each of the Col0 block and the Col1 block is coded by the prediction is described using FIG. 19A and FIG. 19B.

Figure 19A is a conceptual diagram showing candidate motion vectors derived from a single predicted Col0 block. In this example, as in FIG. 18A, the reference pictures L0[0] and L0[1] are located further ahead of the encoding target picture, and the reference pictures L1[0] and L1[1] are located later than the encoding target picture. The Col0 block is included in the reference picture L0[0] and is encoded by the borrowing prediction. In the single prediction of the Col0 block, the reference picture L0[1] in the front direction is referred to by the motion vector mvCol0L0.

In this example, the ratio of the time difference from the encoding target picture to the reference picture L0[0] with respect to the time difference from the reference picture L0[0] to the reference picture L0[1] is performed, and the motion vector mvCol0L0 is performed. target. Thereby, the motion vector mvCol0L0L0 of the reference reference picture L0[0] is derived as a temporal candidate motion vector.

Further, the motion vector mvCol0L0 is scaled by the ratio of the time difference from the encoding target picture to the reference picture L1[0] with respect to the time difference from the reference picture L0[0] to the reference picture L0[1]. Thereby, the motion vector mvCol0L0L1 of the reference reference picture L1[0] is derived as a temporal candidate motion vector.

In other words, the candidate list generating unit 134 scales the motion vector mvCol0L0 of the Col0 block, and derives each of the two motion vectors mvCol0L0L0 and mvCol0L0L1 that can be used in the bi-prediction as the temporal candidate motion vector.

Figure 19B is a conceptual diagram showing candidate motion vectors derived from a single predicted Col1 block. The pictures and the coding target blocks in the example are the same as those in the example of Fig. 19A. The Col1 block is included in the reference picture L1[0] and is encoded by a single prediction. In the single prediction of the Col1 block, the reference picture L0[1] in the front direction is referred to by the motion vector mvCol1L0.

In this example, the motion vector mvCol1L0 is determined by the ratio of the time difference from the encoding target picture to the reference picture L0[0] with respect to the time difference from the reference picture L1[0] to the reference picture L0[1]. Standard. Thereby, the motion vector mvCol1L0L0 of the reference reference picture L0[0] is derived as a temporal candidate motion vector.

Further, the motion vector mvCol1L0 is scaled by the ratio of the time difference from the encoding target picture to the reference picture L1[0] with respect to the time difference from the reference picture L1[0] to the reference picture L0[1]. Thereby, the motion vector mvCol1L0L1 of the reference reference picture L1[0] is derived as a temporal candidate motion vector.

In other words, the candidate list generating unit 134 scales the motion vector mvCol1L0 of the Col1 block, and derives each of the two motion vectors mvCol1L0L0 and mvCol1L0L1 that can be used for bi-prediction as the temporal candidate motion vector.

Next, an example in which the encoding target block is encoded by bi-prediction in only one of the front direction and the backward direction will be described with reference to FIGS. 20A and 20B.

FIG. 20A is a conceptual diagram showing a candidate motion vector derived from a bi-predicted Col0 block for a bi-predicted coding target block of only one direction. In this example, the reference pictures L0[0] and L0[1] contained in the L0 reference picture list, and the reference picture L1[0] contained in the L1 reference picture list are located further ahead of the picture to be encoded.

The Col0 block is included in the reference picture L0[0] and is encoded by bi-prediction. In the double prediction of the Col0 block, the reference picture L0[1] in the front direction is referred to by the motion vector mvCol0L0, and the picture NL in the backward direction is referred to by the motion vector mvCol0L1.

In this example, the motion vector mvCol0L0 is determined by the ratio of the time difference from the encoding target picture to the reference picture L0[0] with respect to the time difference from the reference picture L0[0] to the reference picture L0[1]. Standard. Thereby, the motion vector mvCol0L0L0 of the reference reference picture L0[0] is derived as a temporal candidate motion vector.

Further, the motion vector mvCol0L1 is scaled by the ratio of the time difference from the encoding target picture to the reference picture L1[0] with respect to the time difference from the reference picture L0[0] to the picture NL. Thereby, the motion vector mvCol0L1L1 of the reference reference picture L1[0] is derived as a temporal candidate motion vector.

That is, the candidate list generating unit 134 scales each of the motion vectors mvCol0L0 and mvCol0L1 of the Col0 block. Thereby, the candidate list generating unit 134 derives each of the two motion vectors mvCol0L0L0 and mvCol0L1L1 available for bi-prediction as the temporal candidate motion vector.

In this example, the motion vector mvCol0L0L0 is derived from one of the motion vectors mvCol0L0 of the Col0 block as a temporal candidate motion vector, wherein the motion vector mvCol0L0L0 is a reference picture L0[0] included in the reference L0 reference picture list. Next, a motion vector mvCol0L1L1 is derived from another motion vector mvCol0L1 of the Col0 block as a temporal candidate motion vector, wherein the motion vector mvCol0L1L1 is a reference picture L1[0] included in the reference L1 reference picture list.

For example, in the bi-prediction of the Col0 block, the reference picture L0[1] referenced by the motion vector mvCol0L0 of the Col0 block is a reference picture already included in the L0 reference picture list relative to the Col0 block. Then, in the bi-prediction of the Col0 block, the reference picture NL referenced by the motion vector mvCol0L1 of the Col0 block is a reference picture already included in the L1 reference picture list relative to the Col0 block.

That is, from the reference to the motion vector mvCol0L0 of the L0 reference picture list with respect to the Col0 block, the motion vector mvCol0L0L0 of the reference L0 reference picture list with respect to the encoding target block is derived as the temporal candidate motion vector. Next, the motion vector mvCol0L1L1 of the reference L1 reference picture list with respect to the coding target block is derived from the reference motion vector mvCol0L1 with respect to the L1 reference picture list of the Col0 block as a temporal candidate motion vector.

Thereby, the two temporal candidate motion vectors with reference to the two reference picture lists of the coding target block are derived from the two motion vectors mvCol0L0 and mvCol0L1 with reference to the two reference picture lists of the Col0 block.

In addition, in the time candidate motion vector for deriving the reference L0 reference picture list, the motion vector mvCol0L1 may be used instead of the motion vector mvCol0L0. Similarly, in the time candidate motion vector for deriving the reference L1 reference picture list, the motion vector mvCol0L0 may be used instead of the motion vector mvCol0L1.

For example, the candidate list generating unit 134 may evaluate two motion vectors mvCol0L0 and mvCol0L1 for each of the two reference picture lists, and select one of the two motion vectors mvCol0L0 and mvCol0L1. Then, the candidate list generating unit 134 may also derive a temporal candidate motion vector from one of the selected ones for each of the two reference picture lists.

Further, in the derivation of the temporal candidate motion vector with reference to the L0 reference picture list, the motion vector mvCol0L1 may be used in addition to the motion vector mvCol0L0. Similarly, in the derivation of the temporal candidate motion vector with reference to the L1 reference picture list, the motion vector mvCol0L0 may be used in addition to the motion vector mvCol0L1.

For example, the candidate list generation unit 134 may also derive two temporal candidate motion vectors from the two motion vectors mvCol0L0 and mvCol0L1, wherein the two temporal candidate motion vectors are reference pictures L0[0] included in the reference L0 reference picture list. Further, the candidate list generating unit 134 may derive two time candidate motion vectors from the two motion vectors mvCol0L0 and mvCol0L1, wherein the two time candidate motion vectors are reference pictures L1[0] included in the reference L1 reference picture list.

Thereby, the candidate list generating unit 134 may derive four time candidate motion vectors from the two motion vectors mvCol0L0 and mvCol0L1.

Further, when the encoding target block is encoded in a single prediction, the candidate list generating unit 134 extracts only one of the temporal candidate motion vector of the reference L0 reference picture list and the temporal candidate motion vector of the reference L1 reference picture list. By.

For example, in the above case, the candidate list generating unit 134 may derive at least one temporal candidate motion vector of the reference reference picture L0[0] from at least one of the two motion vectors mvCol0L0 and mvCol0L1. Alternatively, the candidate list generating unit 134 may derive at least one temporal candidate motion vector of the reference reference picture L1[0] from at least one of the two motion vectors mvCol0L0 and mvCol0L1.

Figure 20B is a conceptual diagram showing candidate motion vectors derived from bi-predicted Col1 blocks with respect to bi-predicted coding object blocks in only one direction. The pictures and the coding target blocks in the example are the same as those in the example of Fig. 20A. The Col1 block is included in the reference picture L1[0] and is encoded by bi-prediction. In the double prediction of the Col1 block, the reference picture L0[0] in the front direction is referred to by the motion vector mvCol1L0, and the picture NL in the backward direction is referred to by the motion vector mvCol1L1.

In this example, the motion vector mvCol1L0 is determined by the ratio of the time difference from the encoding target picture to the reference picture L0[0] with respect to the time difference from the reference picture L1[0] to the reference picture L0[1]. Standard. Thereby, the motion vector mvCol1L0L0 of the reference reference picture L0[0] is derived as a temporal candidate motion vector.

Further, the motion vector mvCol1L1 is scaled by the ratio of the time difference from the encoding target picture to the reference picture L1[0] with respect to the time difference from the reference picture L1[0] to the picture NL. Thereby, the motion vector mvCol1L1L1 of the reference reference picture L1[0] is derived as a temporal candidate motion vector.

That is, the candidate list generating unit 134 scales each of the motion vectors mvCol1L0 and mvCol1L1 of the Col1 block. Thereby, the candidate list generating unit 134 derives each of the two motion vectors mvCol1L0L0 and mvCol1L1L1 available for bi-prediction as the temporal candidate motion vector.

In this example, the motion vector mvCol1L0L0 is derived from one of the motion vectors mvCol1L0 of the Col1 block as a temporal candidate motion vector, wherein the motion vector mvCol1L0L0 is a reference picture L0[0] included in the reference L0 reference picture list. Next, the motion vector mvCol1L1L1 is derived from the other motion vector mvCol1L1 of the Col1 block as a temporal candidate motion vector, wherein the motion vector mvCol1L1L1 is a reference picture L1[0] included in the reference L1 reference picture list.

For example, in the bi-prediction of the Col1 block, the reference picture L0[1] referenced by the motion vector mvCol1L0 of the Col1 block is a reference picture included in the list of L0 reference pictures relative to the Col1 block. Moreover, in the double prediction of the Col1 block, the reference picture NL referenced by the motion vector mvCol1L1 of the Col1 block is a reference picture included in the L1 reference picture list relative to the Col1 block.

That is, the motion vector mvCol1L0L0 is derived as a temporal candidate motion vector from the reference motion vector mvCol1L0 with respect to the L0 reference picture list of the Col1 block, wherein the motion vector mvCol1L0L0 is a reference to the L0 reference picture list relative to the encoding target block. . Next, the motion vector mvCol1L1L1 is derived as a temporal candidate motion vector from the reference motion vector mvCol1L1 with respect to the L1 reference picture list of the Col1 block, wherein the motion vector mvCol1L1L1 is a reference to the L1 reference picture list relative to the encoding target block. .

Thereby, the two temporal candidate motion vectors with reference to the two reference picture lists of the coding target block are derived from the two motion vectors mvCol1L0 and mvCol1L1 with reference to the two reference picture lists of the Col1 block.

In addition, in the derivation of the temporal candidate motion vector with reference to the L0 reference picture list, the motion vector mvCol1L1 may be used instead of the motion vector mvCol1L0. Similarly, in the derivation of the temporal candidate motion vector with reference to the L1 reference picture list, the motion vector mvCol1L0 can also be used instead of the motion vector mvCol1L1.

For example, the candidate list generating unit 134 may evaluate two motion vectors mvCol1L0 and mvCol1L1 for each of the two reference picture lists, and select one of the two motion vectors mvCol1L0 and mvCol1L1. Then, the candidate list generation unit 134 may derive a temporal candidate motion vector from one of the selected ones for each of the two reference picture lists.

Further, in the derivation of the temporal candidate motion vector with reference to the L0 reference picture list, the motion vector mvCol1L1 may be used in addition to the motion vector mvCol1L0. Similarly, in the derivation of the temporal candidate motion vector with reference to the L1 reference picture list, the motion vector mvCol1L0 can also be used in addition to the motion vector mvCol1L1.

For example, the candidate list generating unit 134 may derive two time candidate motion vectors of the reference picture L0[0] included in the reference L0 reference picture list from the two motion vectors mvCol1L0 and mvCol1L1. Further, the candidate list generating unit 134 may derive two time candidate motion vectors of the reference picture L1[0] included in the reference L1 reference picture list from the two motion vectors mvCol1L0 and mvCol1L1.

Thereby, the candidate list generating unit 134 can derive four time candidate motion vectors from the two motion vectors mvCol1L0 and mvCol1L1.

Further, when the encoding target block is encoded in a single prediction, the candidate list generating unit 134 derives only one of the temporal candidate motion vector of the reference L0 reference picture list and the temporal candidate motion vector of the reference L1 reference picture list.

For example, in the above case, the candidate list generating unit 134 may derive at least one temporal candidate motion vector of the reference reference picture L0[0] from at least one of the two motion vectors mvCol1L0 and mvCol1L1. Alternatively, the candidate list generating unit 134 may derive at least one temporal candidate motion vector of the reference reference picture L1[0] from at least one of the two motion vectors mvCol1L0 and mvCol1L1.

In the above-described examples of FIGS. 18A to 20B, the operation of the encoding device 100 is mainly displayed, but the decoding device 200 also performs the same operation as the encoding device 100. In other words, each component of the decoding device 200 also performs the same operation as the corresponding component in the encoding device 100. Next, the encoding in the above description can be appropriately replaced with decoding.

When encoding the predicted motion vector index, each of the plurality of candidate motion vectors is respectively provided with an index, wherein the predicted motion vector index is a prediction motion vector selected to be selected from among the plurality of candidate motion vectors. Specifically, the encoding device 100 and the decoding device 200 assign an index to each of a plurality of candidate motion vectors in accordance with a common index assignment rule that has been previously defined. The index assigned to the candidate motion vector selected as the predicted motion vector is then encoded and decoded.

The index assigned to each of the plurality of candidate motion vectors can also be represented by a number.

Moreover, in the above example, each of the Col0 block and the Col1 block is encoded by bi-prediction or single prediction, respectively. When one of the Col0 block and the Col1 block is encoded by bi-prediction and another borrowing prediction, the temporal candidate motion vector may be derived by the combination of the above examples. For example, when the Col0 block is encoded by bi-prediction, and the Col1 block is encoded by the single prediction, the time candidate motion vector can be derived by combining the example of FIG. 18A with the example of FIG. 19B.

Further, it is not limited to the combination of the example of Fig. 18A and the example of Fig. 19B, and any of the above plural examples may be combined.

Further, in the above plural example, a reference picture whose reference picture index is 0 is used as a reference picture containing a Col0 block or a reference picture containing a Col1 block. However, a reference picture whose reference picture index is not 0 may also be used as a reference picture containing a Col0 block or a reference picture containing a Col1 block.

As described above, candidate motion vectors are derived from each of the Col0 block and the Col1 block. For example, sometimes one of the Col0 block and the Col1 block is included in the same moving object range as the encoding target block. Therefore, by deriving the candidate motion vector by each of the Col0 block and the Col1 block, an appropriate candidate motion vector having the same characteristics as the motion vector of the encoding target block can be derived.

Alternatively, the candidate motion vector may be derived from each of the Col0 block and the Col1 block for all of the coding target blocks. For example, the candidate motion vector may be derived from each of the Col0 block and the Col1 block for one coding target block. Then, for other coding target blocks, the candidate motion vector may not be derived from the Col1 block, but the candidate motion vector may be derived from the Col0 block.

Further, when the encoding target picture is not a B picture and the two reference picture lists are not determined, the temporal candidate motion vector may be derived from the co-located block included in one reference picture list.

[Candidate Motion Vector Derived from ATMVP Block] In the complex example of FIG. 18A to FIG. 20B, the temporal candidate motion vector will be from the co-located block included in the L0 reference picture list, and the co-located area included in the L1 reference picture list. The blocks are exported separately. Instead of the co-located blocks in these examples, ATMVP blocks can also be used.

That is, the temporal candidate motion vector may also be derived from the ATMVP block in the reference picture included in the L0 reference picture list and the ATMVP block in the reference picture included in the L1 reference picture list. Here, there is a case where the ATMVP block in the reference picture included in the L0 reference picture list is called an Adv0 block. Further, there is a case where the ATMVP block in the reference picture included in the L1 reference picture list is called an Adv1 block.

That is, instead of the Col0 block and the Col1 block in the plural examples of FIGS. 18A to 20B, the Adv0 block and the Adv1 block may be used.

Hereinafter, an example in which the temporal candidate motion vector is derived from the Adv0 block and the Adv1 block will be described with reference to FIGS. 21A and 21B. Specifically, in the examples of FIGS. 21A and 21B, the Ad0 block and the Adv1 block are used instead of the Col0 block and the Col1 block in the examples of FIGS. 18A and 18B.

Figure 21A is a conceptual diagram showing candidate motion vectors derived from Adv0 blocks. In this example, the Adv0 block is the reference picture L0[0] contained in the L0 reference picture list. Such an Adv0 block is determined in accordance with the temporal motion vector of the coding target block.

For example, the candidate list generating unit 134 uses the motion vector of the reference picture L0[0] of the reference L0 reference picture list as the temporary block of the coding target block in accordance with the motion vector of the coded block adjacent to the coding target block. The movement vector is determined by the sex. At this time, the candidate list generating unit 134 may determine the block from the plurality of blocks adjacent to the encoded target block in accordance with the relative position with respect to the encoding target block, wherein the block is A motion vector used to determine the temporality of the encoding object block.

For example, the relative position of the adjacent block D with respect to the encoding target block in Fig. 11 may be determined in advance as the position of the adjacent block for determining the temporal motion vector of the encoding target block. Further, the candidate list generating unit 134 may search for the adjacent block predicted in the order of the adjacent blocks D, A, B, and C in FIG. 11, and use the adjacent block that is originally found to be used for determining the temporary. Adjacent blocks of the moving vector.

Alternatively, scaling can be applied to the motion vector of the adjacent block, and the motion vector to which the scaling has been applied is used as the temporary motion vector of the coding target block. For scaling, the ratio of the following two time differences can also be used, that is, the time difference from the encoding target picture to the reference picture L0[0] contained in the L0 reference picture list relative to the moving vector from the encoding target picture to the adjacent block. The ratio of the time differences of the reference pictures referenced.

Alternatively, a negative ratio scaling may be applied to the motion vector of the adjacent block to invert the motion vector of the adjacent block. Then, the inverted motion vector can also be used as a temporary motion vector of the coding target block. Thereby, even if the adjacent block is coded only by referring to one of the front direction and the backward direction, the motion vector with reference to the appropriate reference direction can be used as the temporary motion vector of the coding target block.

Next, the candidate list generating unit 134 determines the following block as the Adv0 block from the encoding target block by using the temporary motion vector of the encoding target block, wherein the block is a reference picture included in the L0 reference picture list. The indicated block in L0[0].

In the example of FIG. 21A, the candidate list generating unit 134 determines the motion vector mvCurL0 as the temporary motion vector of the encoding target block. Then, the candidate list generating unit 134 determines the block from the encoding target block as the Adv0 block by the motion vector mvCurL0, wherein the block is in the reference picture L0[0] included in the L0 reference picture list. The indicated block.

The motion vector mvCurL0 is, for example, a motion vector of the adjacent block D shown in FIG. 11, and is a motion vector of the reference picture L0[0] included in the reference L0 reference picture list. Alternatively, scaling can be applied to the motion vector of the adjacent block D, and the motion vector to which the scaling has been applied is used as the motion vector mvCurL0. For example, in order to obtain the motion vector mvCurL0 of the reference reference picture L0[0], scaling can also be applied to the motion vector of the adjacent block D.

In this example, the Adv0 block is coded through bi-prediction. In the double prediction of the Adv0 block, the reference picture L0[1] in the front direction has been referred to by the motion vector mvAdv0L0, and the reference picture L1[1] in the backward direction has been referred to by the motion vector mvAdv0L1.

Then, in this example, the motion vector mvAdv0L1 is performed with a ratio of the time difference from the encoding target picture to the reference picture L0[0] with respect to the time difference from the reference picture L0[0] to the reference picture L1[1]. target. Thereby, the motion vector mvAdv0L1L0 of the reference reference picture L0[0] is derived as a temporal candidate motion vector.

Further, the motion vector mvAdv0L1 is scaled by the ratio of the time difference from the encoding target picture to the reference picture L1[0] with respect to the time difference from the reference picture L0[0] to the reference picture L1[1]. Thereby, the motion vector mvAdv0L1L1 of the reference reference picture L1[0] is derived as a temporal candidate motion vector.

In other words, the candidate list generation unit 134 scales the motion vector mvAdv0L1 of the Adv0 block, and derives the two motion vectors mvAdv0L1L0 and mvAdv0L1L1 that can be used in the bi-prediction as the temporal candidate motion vector.

For other explanations, the Col0 block in the example of FIG. 18A, and the motion vectors mvCol0L0 and mvCol0L1, etc., may be replaced with the Adv0 block in the example of FIG. 21A, and the motion vectors mvAdv0L0 and mvAdv0L1, and the like. Therefore, the detailed description is omitted.

Figure 21B is a conceptual diagram showing candidate motion vectors derived from Adv1 blocks. In this example, the Adv1 block is included in the reference picture L1[0] of the L1 reference picture list. Such an Adv1 block is determined in accordance with a temporary motion vector of the coding target block.

For example, the candidate list generating unit 134 determines the motion vector of the reference picture L1[0] of the reference L1 reference picture list as the temporary block of the coding target block in accordance with the motion vector of the coded block adjacent to the coding target block. Sexual movement vector. At this time, the candidate list generating unit 134 may determine the block from among the plurality of blocks adjacent to the encoding target block in accordance with the relative position with respect to the encoding target block, wherein the block is A motion vector used to determine the temporality of the encoding object block.

For example, the relative position of the adjacent block D relative to the encoding target block in FIG. 11 may be determined in advance as the position of the adjacent block, wherein the adjacent block is used to determine the temporary block of the encoding target block. Sexual movement vector. Further, the candidate list generating unit 134 may search the adjacent blocks predicted in the order of the adjacent blocks D, A, B, and C in FIG. 11, and use the adjacent blocks of the inter-predicted first found to determine the temporary. Adjacent blocks of the moving vector.

Alternatively, scaling may be applied to the motion vector of the adjacent block, and the motion vector to which the scaling has been applied may be used as the temporary motion vector of the coding target block. For scaling, the time difference from the reference picture L1[0] contained in the encoding target picture to the L1 reference picture list relative to the reference picture referenced from the encoding target picture to the moving picture of the adjacent block may also be used. ratio.

Alternatively, a negative ratio scaling may be applied to the motion vector of the adjacent block, and the motion vector of the adjacent block may be inverted. Then, the motion vector that has been inverted can also be used as the temporary motion vector of the coding target block. Thereby, even if the adjacent block is only coded in one of the front direction and the backward direction, the motion vector referring to the appropriate reference direction is used as the temporary motion vector of the coding target block.

Then, the candidate list generating unit 134 determines the block as the Adv1 block from the encoding target block by using the temporary motion vector of the encoding target block, wherein the block is a reference included in the L1 reference picture list. The block indicated in picture L1[0].

In the example of FIG. 21B, the candidate list generating unit 134 determines the temporal motion vector mvCurL1 as the motion vector of the encoding target block. Then, the candidate list generating unit 134 determines the block as the Adv1 block from the encoding target block by the motion vector mvCurL1, wherein the block is indicated in the reference picture L1[0] included in the L1 reference picture list. Block.

The motion vector mvCurL1 is, for example, a motion vector of the adjacent block D shown in FIG. 11, and is a motion vector of the reference picture L1[0] included in the reference L1 reference picture list. Alternatively, scaling can be applied to the motion vector of the adjacent block D, and the motion vector to which the scaling has been applied is used as the motion vector mvCurL1. For example, in order to obtain the motion vector mvCurL1 of the reference reference picture L1[0], scaling can also be applied to the motion vector of the adjacent block D.

Further, two motion vectors mvCurL0 and mvCurL1 may be derived from the same one of the motion vectors adjacent to the block D. Alternatively, the motion vector mvCurL0 may be derived from one of the two motion vectors of the adjacent block D that has been encoded by the bi-prediction, and the motion vector mvCurL1 may be derived from the other. Alternatively, the motion vector mvCurL0 may be derived from the motion vector of the adjacent block D, and the motion vector mvCurL1 may be derived from the motion vectors of other adjacent blocks A, B, or C.

In this example, the Adv1 block has been encoded by bi-prediction. In the double prediction of the Adv1 block, the reference picture L0[1] in the front direction has been referred to by the motion vector mvAdv1L0, and the reference picture L1[1] in the back direction has been referred to by the motion vector mvAdv1L1.

Next, in this example, the motion vector mvAdv1L0 is performed with a ratio of the time difference from the encoding target picture to the reference picture L0[0] with respect to the time difference from the reference picture L1[0] to the reference picture L0[1]. target. Thereby, the motion vector mvAdv1L0L0 of the reference reference picture L0[0] is derived as a temporal candidate motion vector.

Further, the motion vector mvAdv1L0 is scaled by the ratio of the time difference from the encoding target picture to the reference picture L1[0] with respect to the time difference from the reference picture L1[0] to the reference picture L0[1]. Thereby, the motion vector mvAdv1L0L1 of the reference reference picture L1[0] is derived as a temporal candidate motion vector.

In other words, the candidate list generation unit 134 scales the motion vector mvAdv1L0 of the Adv1 block, and derives the two motion vectors mvAdv1L0L0 and mvAdv1L0L1 that can be used in the bi-prediction as the temporal candidate motion vector.

For other explanations, the Col1 block in the example of FIG. 18B, and the motion vectors mvCol1L0 and mvCol1L1, etc., may be replaced with the Adv1 block in the example of FIG. 21B, and the motion vectors mvAdv1L0 and mvAdv1L1. Therefore, the detailed description is omitted.

Further, as described above, in the examples of FIGS. 21A and 21B, the Adv0 block and the Adv1 block are used instead of the Col0 block and the Col1 block in the examples of FIGS. 18A and 18B. However, the Ad00 block and the Adv1 block may be used instead of the Col0 block and the Col1 block in the examples of FIGS. 19A to 20B.

Therefore, the Col0 block and the Col1 block in the examples of FIGS. 19A to 20B can also be replaced by the Adv0 block and the Adv1 block. Further, in addition to the Col0 block and the Col1 block in the examples of FIGS. 18A to 20B, an Adv0 block, an Adv1 block, or the like may be used.

As described above, the candidate motion vector is derived from the Adv0 block in the reference picture included in the L0 reference picture list and the Adv1 block in the reference picture included in the L1 reference picture list, respectively.

For example, when the coding target block is located near the boundary of the moving object range, there is a possibility that, by occlusion, there is no region corresponding to the coding target block in one of the reference pictures, and another The reference picture has an area corresponding to the block of the encoding object. Therefore, especially when the coding target block is located near the boundary of the moving object range, the candidate motion vector is derived from the Adv0 block and the Adv1 block, respectively, so that a suitable candidate motion vector can be derived from the region corresponding to the coding target block.

Further, for all the coding target blocks, the candidate motion vectors may not be derived from the Adv0 block and the Adv1 block, respectively. For example, the candidate motion vector may be derived from the Adv0 block and the Adv1 block, respectively, for one coding target block. Then, for other coding target blocks, candidate motion vectors are not derived from the Adv1 block, and candidate motion vectors are derived from the Adv0 block.

Further, when the encoding target picture is not a B picture and the two reference picture lists are not determined, the time candidate motion vector may be derived from the ATMVP block included in one reference picture list.

[Candidate List] FIG. 22 is a conceptual diagram showing a plurality of groups with respect to a plurality of candidate motion vectors. The predicted motion vector used to encode the information of the coding target block may be specified by an index corresponding to each of the plurality of candidate motion vectors. For example, for a maximum of N candidate motion vectors, there are respectively corresponding indexes.

A plurality of candidate motion vectors are constituent candidate lists. The plurality of candidate motion vectors may contain spatial candidate motion vectors, may also contain temporal candidate motion vectors, and may also contain other candidate motion vectors. That is, a maximum of N candidate motion vectors may include a group of spatial candidate motion vectors, a group of temporal candidate motion vectors, and a group of other candidate motion vectors.

The spatial candidate motion vector is, for example, a candidate motion vector derived from a block spatially adjacent to the coding target block. The spatial candidate motion vector may also be derived from a block located next to the encoding object block and not adjacent to the encoding object block.

The temporal candidate motion vector is a candidate motion vector derived from a block included in a reference picture different from the encoding target picture. For example, the temporal candidate motion vector may be derived from a Col0 block, a Col1 block, an Adv0 block, or an Adv1 block.

For one coding target block, a temporal candidate motion vector may also be derived from the Col0 block and the Col1 block, respectively. Then, for other coding target blocks, the temporal candidate motion vector may also be derived from one of the Col0 block and the Col1 block.

Similarly, for one coding target block, the temporal candidate motion vector can also be derived from the Adv0 block and the Adv1 block, respectively. Then, for other coding target blocks, the temporal candidate motion vector may also be derived from one of the Adv0 block and the Adv1 block.

The other candidate motion vectors are candidate motion vectors that are different from the spatial candidate motion vector and also different from the temporal candidate motion vector. Other candidate motion vectors may also be derived from a plurality of candidate motion vectors of the spatial candidate motion vector or the temporal candidate motion vector, respectively. For example, the average or intermediate value of a plurality of candidate motion vectors can also be derived as other candidate motion vectors.

Also, a motion vector that has been determined in advance can also be used as other candidate motion vectors. For example, a motion vector of size zero can also be used as other candidate motion vectors.

Further, the fixed candidate motion vector may be determined in a sequence unit, a picture unit, a slice unit, or a predetermined unit including one or more blocks.

For example, the entropy encoding unit 110 may also encode a fixed candidate motion vector of a sequence unit into a sequence parameter combination. Further, the entropy coding unit 110 may encode a fixed candidate motion vector of a picture unit into a picture parameter combination. Further, the entropy coding unit 110 may encode a fixed candidate motion vector of a slice unit into a slice header.

Next, the entropy decoding unit 202 may combine the sequence parameter parameters to decode the fixed candidate motion vector of the sequence unit. Further, the entropy decoding unit 202 may decode the fixed candidate motion vector of the picture unit from the picture parameter combination. Further, the entropy decoding unit 202 may decode the fixed candidate motion vector of the slice unit from the slice header.

For example, when the image as a whole is moving at a constant speed, the motion vector of the motion of the entire image may be used as a fixed candidate motion vector, and included in the sequence parameter combination, the picture parameter combination, or the slice header. Head information. Further, when the entire image is moving, and the image includes an object having a motion different from the motion of the entire image, the candidate motion vector of the motion of the entire display image and the candidate motion of the motion of the display object may be moved. The vector is included in the header information.

Further, when updating the candidate motion vector such as a picture unit or a slice unit, the encoding apparatus 100 may encode the difference with respect to the candidate motion vector before the update.

Further, the method of deriving the candidate motion vector, the method of assigning the index, and the maximum number of candidate motion vectors may be determined by a sequence unit, a picture unit, a slice unit, or a predetermined unit including one or more blocks. For example, the method for deriving the candidate motion vector may be determined by a sequence unit by a combination of sequence parameters, or may be determined by a picture unit by a combination of picture parameters, or may be determined by a slice unit by a slice header.

For example, when the coding target block is near the boundary of the moving object range, the candidate list generation unit 134 may also derive the temporal candidate motion vector from the Col0 block and the Col1 block, respectively. Alternatively, at this time, the candidate list generating unit 134 may derive the temporal candidate motion vector from each of the Adv0 block and the Adv1 block.

Next, when the encoding target block is not near the boundary of the moving object range, the candidate list generating unit 134 may derive the temporal candidate motion vector from one of the Col0 block and the Col1 block. Alternatively, at this time, the candidate list generating unit 134 may derive the temporal candidate motion vector from one of the Adv0 block and the Adv1 block.

The candidate list generating unit 134 may determine whether or not the encoding target block is near the boundary of the moving object range based on an object capturing technique including edge detection or mechanical learning. Alternatively, the candidate list generation unit 134 may determine whether or not the encoding target block is located near the boundary of the moving object range based on the movement of each pixel or each movement of the rectangular area of a predetermined size.

Further, the candidate list generating unit 134 may determine whether or not the difference between the motion vector of the Col0 block and the motion vector of the Col1 block is equal to or larger than the threshold. Next, when the difference is equal to or larger than the threshold, the candidate list generating unit 134 may derive the temporal candidate motion vector from the Col0 block and the Col1 block, respectively. On the other hand, if the difference is lower than the threshold, the candidate list generation unit 134 may derive the temporal candidate motion vector from one of the Col0 block and the Col1 block.

Alternatively, when the difference between the motion vector of the Adv0 block and the motion vector of the Adv1 block is equal to or greater than the threshold value, the candidate list generation unit 134 may derive the temporal candidate motion vector from the Adv0 block and the Adv1 block, respectively. On the other hand, if the difference is lower than the threshold, the candidate list generating unit 134 may derive the temporal candidate motion vector from one of the Adv0 block and the Adv1 block.

The method of deriving the spatial candidate motion vector and the like is also the same as the temporal candidate motion vector, and can be changed in accordance with whether or not the coding target block is near the boundary of the moving object range.

For example, when the encoding target block is near the boundary of the moving object range, the candidate list generating unit 134 may also derive the spatial candidate motion vector from the plurality of adjacent blocks, respectively. Then, when the encoding target block is not near the boundary of the moving object range, the candidate list generating unit 134 may derive the spatial candidate moving vector from one of the plurality of adjacent blocks.

Further, for example, the candidate list generating unit 134 may determine whether or not the difference between the two motion vectors of the two adjacent blocks is equal to or larger than the threshold. Then, when the difference is equal to or larger than the threshold value, the candidate list generating unit 134 may derive the spatial candidate motion vector from each of the two adjacent blocks. On the other hand, when the difference is lower than the threshold, the candidate list generating unit 134 may derive the spatial candidate motion vector from one of the two adjacent blocks.

Also, the group can be reflected in the index allocation method. Specifically, the candidate list generation unit 134 may group the plurality of candidate motion vectors into a plurality of groups in accordance with the number of temporal candidate motion vectors, the number of spatial candidate motion vectors, and the like.

For example, the candidate list generation unit 134 may specify the number of temporal candidate motion vectors based on whether or not the temporal candidate motion vector can be derived from the block included in the L0 reference picture list and the block included in the L1 reference picture list. Further, the candidate list generation unit 134 may specify the number of spatial candidate motion vectors based on whether or not the spatial candidate motion vector can be derived from the block that is not adjacent to the encoding target block.

Next, for each of the plurality of candidate motion vectors, the candidate list generation unit 134 may use the combination of the identification number of the group including the candidate motion vector and the index number of the candidate motion vector in the group as the index of the candidate motion vector. .

For the coding target block near the boundary of the moving object range, the candidate motion vector is derived from each of the spatially and temporally wide ranges, whereby a more suitable predicted motion vector can be selected. For example, as described above, the temporal candidate motion vector is derived from the block included in the L0 reference picture list and the block included in the L1 reference picture list, thereby deriving the temporal candidate movement from each block in the temporally broad range. vector. Thereby, a more suitable predicted motion vector can be selected.

Further, for each of the two reference picture lists, in addition to the block included in the reference picture whose reference picture index is 0, the time candidate motion vector may be derived from the block included in the reference picture whose reference picture index is not 0. Thereby, the temporal candidate motion vector can be derived from each of the blocks in a wider time range, and a more suitable predicted motion vector can be selected.

Alternatively, the temporal candidate motion vector may be derived from a co-located block or a block around the ATMVP block. Here, the range around the block may be a range of a predetermined distance from the block, or may be a range adjacent to one or more blocks of the block.

Further, for example, it is assumed that the difference in movement is large in the foreground and the background, and the aforementioned foreground is in the range of the moving object, and the aforementioned background is outside the range of the moving object. Therefore, it is also possible to extract two motion vectors having large differences from a plurality of blocks in the vicinity of the parity block or the ATMVP block. Then, the two motion vectors extracted can also be regarded as two motion vectors of the foreground and the background. Then, the time candidate motion vector can also be derived from the two extracted motion vectors, respectively.

Further, the candidate list generating unit 134 may group a plurality of similar motion vectors of a plurality of similar blocks in a plurality of motion vectors of a plurality of blocks in the vicinity of the co-located block or the ATMVP block. Then, the candidate list generation unit 134 may derive the temporal candidate motion vector for each group by the average or intermediate value of the plurality of motion vectors belonging to the group.

In the above, each motion vector can also be normalized according to the distance in time. That is, each motion vector may also be scaled to refer to the motion vector of the reference picture L0[0] or L1[0] from the encoding target picture.

Further, in order to obtain a more appropriate prediction motion vector, the spatial candidate motion vector may be derived from each of the blocks that are spatially larger than the range adjacent to the coding target block. For example, in addition to the adjacent block, the adjacent block may also be used to derive the spatial candidate motion vector, wherein the adjacent block is a block adjacent to the encoding target block, and the adjacent block is adjacent to the adjacent block. Block.

Figure 23 is a conceptual diagram showing a block used to derive a spatial candidate motion vector. In the example of FIG. 23, in addition to the adjacent blocks A, B, C, and D, a spatial candidate motion vector is derived from at least one of the adjacent blocks C1, C2, and C3, wherein the re-contiguous block C1. C2 and C3 are blocks adjacent to the adjacent block C. Among the adjacent blocks C1, C2, and C3, the adjacent blocks for the derivation of the spatial candidate motion vector may be fixed or may be appropriately switched.

Further, in the example of Fig. 23, it is determined that the adjacent blocks C1, C2, and C3 are adjacent to the adjacent block C, but the adjacent blocks can be similarly determined for the adjacent blocks A, B, and D. Then, among the adjacent blocks A, B, C, and D, the adjacent blocks in which the adjacent blocks for the derivation of the spatial candidate motion vector have been determined may be fixed, or may be appropriately switched.

For example, in order to cover as much as possible a wide range of spatial movements, the contiguous blocks for the derivation of the spatial candidate motion vectors may also be determined relative to adjacent blocks C and D.

Moreover, not only the adjacent blocks that are adjacent to the corners of the coding target block, for example, the blocks between the adjacent blocks B and C can also be used for the derivation of the spatial candidate motion vector. The use of such an intermediate block is effective in the case where, for example, the size of the encoding target block having a size of the encoding target block of 16×16 or more is large.

The encoding apparatus 100 selects a predicted motion vector from among a plurality of candidate motion vectors including a spatial candidate motion vector, a temporal candidate motion vector, and the like, wherein the predicted motion vector is predicted to be moved as a motion vector of the encoding target block. vector. The encoding apparatus 100 may calculate a prediction residual or RD cost based on each candidate motion vector, and select a candidate motion vector whose calculated prediction residual or RD cost is the smallest among the plurality of candidate motion vectors as a prediction motion vector. .

Then, the encoding device 100 encodes the information of the encoding target block using the predicted motion vector.

For example, the encoding device 100 may also encode a differential motion vector of a motion vector of a coding target block and a prediction motion vector. Thereby, the encoding apparatus 100 can also encode the motion vector of the encoding target block using the predicted motion vector. Alternatively, the encoding device 100 may encode an image of the encoding target block and a difference image of the image predicted in accordance with the predicted motion vector. Thereby, the encoding apparatus 100 can also encode the image of the encoding target block using the predicted motion vector.

Furthermore, the encoding apparatus 100 may also encode the candidate motion vector selected as the predicted motion vector as the identification information for identifying the predicted motion vector from among the plurality of candidate motion vectors. Yuan stream.

Next, the decoding device 200 can also decode the index from the bit stream. Next, the decoding apparatus 200 may also select a candidate motion vector corresponding to the decoded index from among a plurality of candidate motion vectors, where the plurality of candidate motion vectors include a spatial candidate motion vector and a temporal candidate motion. Vector, etc. Then, the decoding device 200 decodes the information of the decoding target block using the predicted motion vector.

For example, the decoding device 200 may also decode the differential motion vector and add the predicted motion vector to the differential motion vector. Thereby, the decoding device 200 can also decode the motion vector of the decoding target block using the predicted motion vector. Alternatively, the decoding device 200 may decode the difference image and add the image predicted in accordance with the predicted motion vector to the difference image. Thereby, the decoding device 200 can also decode the image of the decoding target block using the predicted motion vector.

Further, for example, the encoding device 100 may not encode the index of the prediction motion vector. Next, the decoding device 200 may not decode the index of the predicted motion vector. At this time, in the encoding device 100 and the decoding device 200, the prediction motion vector may be selected by the template matching method or the bidirectional matching method in the FRUC.

In other words, when the encoding device 100 and the decoding device 200 select a prediction motion vector from among a plurality of candidate motion vectors, they may evaluate a plurality of candidate motion vectors. The encoding device 100 and the decoding device 200 may also select a candidate motion vector that is most highly evaluated among the plurality of candidate motion vectors as a predicted motion vector.

For example, the encoding device 100 and the decoding device 200 determine two comparison target regions in the evaluation of each candidate motion vector. At this time, the encoding device 100 and the decoding device 200 determine at least one of the two comparison target regions in accordance with the candidate motion vector to be evaluated. Then, the higher the degree of suitability of the reconstructed image of the two comparison target regions by the encoding device 100 and the decoding device 200, the higher the candidate motion vector of the evaluation target is.

Thereby, the encoding device 100 and the decoding device 200 can evaluate the plurality of candidate motion vectors in the same manner, and can select the same predicted motion vector from among the plurality of candidate motion vectors.

For example, as the number of candidate motion vectors increases, the index used to identify the predicted motion vector from among the plurality of candidate motion vectors becomes larger, and there is a possibility that the code amount of the index is increased. However, in FRUC, since the index of the predicted motion vector is not encoded, the increase in the amount of coding caused by the increase in the number of candidate motion vectors can be suppressed.

Therefore, in the FRUC, more spatial candidate motion vectors, temporal candidate motion vectors, and the like can be derived. Thereby, the encoding device 100 and the decoding device 200 can improve the prediction accuracy while suppressing an increase in the amount of encoding.

Moreover, the candidate list may also be divided into an L0 reference candidate list, which is a candidate motion vector containing a reference L0 reference picture list, and the L1 reference candidate list, which includes the reference L1. Refer to the candidate motion vector of the picture list.

Also, the candidate list may also include a candidate motion vector combination of the candidate motion vector of the reference L0 reference picture list and the candidate motion vector of the reference L1 reference picture list. Furthermore, the candidate list may also contain candidate motion vector combinations of candidate motion vectors with reference to the L0 reference picture list, and may also include candidate motion vector combinations of candidate motion vectors with reference to the L1 reference picture list.

Then, from the plurality of candidate motion vector combinations included in the candidate list, one candidate motion vector combination is selected as the predicted motion vector combination, thereby also being among the plurality of candidate motion vectors included in the candidate list. Select the predicted motion vector.

[Candidate Motion Vector Derived by Alignment] The candidate list generation unit 134 can replace the use of the parity block and the ATMVP block, or the use of the reference block and the use of the same block and the ATMVP block. Export time candidate motion vectors.

For example, the candidate list generating unit 134 maps a plurality of reference blocks of the plurality of reference pictures to the encoding target picture when the processing of the encoding target picture is started. Specifically, the candidate list generating unit 134 is a corresponding region in which each reference block is mapped to the encoding target picture. Each corresponding area can also be represented as an enantiing area.

Then, the candidate list generating unit 134 derives the candidate motion vector for each block of the encoding target picture in accordance with the result of the mapping. Specifically, the candidate list generation unit 134 derives candidate motion vectors from the motion vectors of the reference blocks, wherein the aforementioned reference blocks are mapped to corresponding regions of the blocks superimposed in the encoding target picture.

Further, the derivation processing of the candidate motion vector may be performed in a picture unit, or may be performed by a slice unit obtained by dividing the picture, or may be performed in another area unit.

Figure 24 is a conceptual diagram showing the mapping of the reference block contained in the L0 reference picture list and the reference block included in the L1 reference picture list. For example, the candidate list generating unit 134 normalizes the motion vector of the reference block Blk01 included in the reference picture L0[0], thereby deriving the motion vector mvBlk01 of the reference encoding target picture.

That is, the candidate list generating unit 134 scales the motion vector of the reference block Blk01, thereby deriving the motion vector mvBlk01 of the reference encoding target picture. For the calibration, the ratio is used, that is, the ratio of the time difference of the reference picture L0[0] to the picture of the encoding target relative to the time difference of the reference picture referenced from the reference picture L0[0] to the motion vector of the reference block Blk01. .

Next, the candidate list generating unit 134 maps the reference block Blk01 to the corresponding region Blk01c in the encoding target picture in accordance with the motion vector mvBlk01. That is, the candidate list generating unit 134 assigns the reference block Blk01 to the corresponding region Blk01c in the encoding target picture in accordance with the motion vector mvBlk01. Further, in other words, the candidate list generating unit 134 is a corresponding region Blk01c of the specific reference block Blk01 among the encoding target pictures in accordance with the motion vector mvBlk01.

Similarly, the candidate list generating unit 134 normalizes the motion vector of the reference block Blk02 included in the reference picture L0[0], thereby deriving the motion vector mvBlk02 of the reference encoding target picture. Then, the candidate list generating unit 134 maps the reference block Blk02 to the corresponding region Blk02c in the encoding target picture in accordance with the motion vector mvBlk02.

Similarly, the candidate list generating unit 134 normalizes the motion vector of the reference block Blk03 included in the reference picture L0[0], thereby deriving the motion vector mvBlk03 of the reference encoding target picture. Then, the candidate list generating unit 134 maps the reference block Blk03 to the corresponding region Blk03c in the encoding target picture in accordance with the motion vector mvBlk03.

Then, the candidate list generating unit 134 derives the temporal candidate motion vector from the motion vector of the reference block mapped to each block of the encoding target picture.

When only the picture L0[0] is referred to, as in the area between the corresponding area Blk01c and the corresponding area Blk02c, there is a case where the reference block is not mapped. To this end, the candidate list generating section 134 uses the reference picture L1[0].

For example, the candidate list generating unit 134 normalizes the motion vector of the reference block Blk11 included in the reference picture L1[0], thereby deriving the motion vector mvBlk11 of the reference encoding target picture.

That is, the candidate list generating unit 134 scales the motion vector of the reference block Blk11, thereby deriving the motion vector mvBlk11 of the reference encoding target picture. In the scaling, a ratio is used, that is, the time difference from the reference picture L1[0] to the encoding target picture relative to the reference picture referenced from the reference picture L1[0] to the motion vector of the reference block Blk11 The ratio of time differences.

Then, the candidate list generating unit 134 maps the reference block Blk11 to the corresponding region Blk11c in the encoding target picture in accordance with the motion vector mvBlk11. However, even if both the reference picture L0[0] and the reference picture L1[0] are used, there is still a possibility that a blank area remains.

Fig. 25 is a conceptual diagram showing a blank area. The candidate list generation unit 134 is a block for the blank area, and derives the temporal candidate motion vector from the surrounding motion vector. That is, the candidate list generation section 134 derives a temporal candidate motion vector from the motion vector of the reference block, which is an area of the periphery of the tile that has been mapped to the blank area.

Specifically, in the example of FIG. 25, the candidate list generating unit 134 is a coding target block for the blank area, and derives the temporal candidate motion vector from the motion vector of the corresponding region Blk11c and the motion vector of the corresponding region Blk02c. For example, the candidate list generating unit 134 may derive the temporal candidate motion vector by the interpolation of the motion vector of the corresponding region Blk11c and the motion vector of the corresponding region Blk02c for the encoding target block of the blank region.

Alternatively, the candidate list generating unit 134 may derive the temporal candidate motion vector by using a motion vector having a smaller size among the motion vector of the corresponding region Blk11c and the motion vector of the corresponding region Blk02c without interpolation. Alternatively, the candidate list generating unit 134 may derive the temporal candidate motion vector from the motion vector selected from the motion vector of the corresponding region Blk11c and the motion vector of the corresponding region Blk02c by other selection methods.

Further, in the encoding target picture, there is a possibility that there is a blank area and there is a duplicate area. That is, there is a possibility that a plurality of reference blocks are overlapped by a plurality of corresponding regions.

Fig. 26 is a conceptual diagram showing a repeating area. In this example, the reference block Blk02 is mapped to the corresponding region Blk02c by the motion vector mvBlk02, and the reference block Blk03 is mapped to the corresponding region Blk03c by the motion vector mvBlk03. Then, the corresponding area Blk02c and the corresponding area Blk03c are partially overlapped. The temporal candidate motion vector for such a repeated region is described later in detail using FIG.

Fig. 27 is a conceptual diagram showing an example of a coding target block and a corresponding area. In this example, the corresponding area Blk01c of the reference block Blk01 is superimposed on the encoding target block. When the corresponding region of only one reference block is superimposed on the coding target block, the candidate list generation unit 134 derives the temporal candidate motion vector for the prediction motion vector of the coding target block from the motion vector of the one reference block.

In the example of FIG. 27, the coding target block includes a portion of the corresponding region Blk01c of the reference block Blk01, and a blank region, and does not include a corresponding region of other reference blocks. Therefore, the candidate list generation unit 134 derives the temporal candidate motion vector from the motion vector of the reference block Blk01.

For example, the candidate list generating section 134 scales the motion vector of the reference block Blk01, thereby deriving the temporal candidate motion vector of the reference reference picture L0[0] from the encoding target block. The scaling is a ratio of the time difference from the encoding target picture to the reference picture L0[0] with respect to the time difference of the reference picture referenced from the reference picture L0[0] to the motion vector of the reference block Blk01.

In addition, the temporal candidate motion vector can also be obtained by inverting the normalized motion vector mvBlk01 of the reference block Blk01.

Fig. 28 is a conceptual diagram showing an example of a coding target block and two corresponding areas. In this example, the corresponding area Blk02c of the reference block Blk02 and the corresponding area Blk03c of the reference block Blk03 are superimposed on the encoding target block. When a plurality of corresponding regions of the plurality of blocks overlap in the encoding target block, the candidate list generating portion 134 derives a temporal candidate motion vector from the motion vector of the reference block, wherein the reference block is mapped in the encoding target block. The area occupied is the largest corresponding area.

In the example of FIG. 28, in the coding target block, the area occupied by the corresponding area Blk03c of the reference block Blk03 is larger than the area occupied by the corresponding area Blk02c of the reference block Blk02. Therefore, the candidate list generation unit 134 derives the temporal candidate motion vector from the motion vector of the reference block Blk03.

Alternatively, the candidate list generating unit 134 derives a temporal candidate motion vector by using an average value, an intermediate value, a maximum value, or a minimum value of a plurality of motion vectors of the plurality of blocks, wherein the plurality of blocks correspond to the overlap in the encoding. A plurality of corresponding regions of the object block. Further, the candidate list generating unit 134 may weight a plurality of motion vectors corresponding to the plurality of corresponding regions in accordance with an area occupied by each corresponding region of the encoding target block, and weight the average of the plurality of moving vectors. To derive the temporal candidate motion vector.

For example, when using the average, intermediate value, maximum value, minimum value, or weighted average of a plurality of motion vectors, the average, intermediate, and maximum of the normalized multiple motion vectors may also be used according to the temporal distance. Value, minimum, or weighted average. That is, an average value, an intermediate value, a maximum value, a minimum value, or a weighted average of a plurality of motion vectors that have been respectively scaled from the motion vector of the reference picture L0[0] or L1[0] of the encoding target picture may be used. value.

Alternatively, the candidate list generating unit 134 may also derive a temporal candidate motion vector from a plurality of motion vectors of the plurality of reference blocks, wherein the plurality of reference blocks are mapped to a plurality of correspondences that are overlapped in the coding target block. region. At this time, the upper limit of the number of time candidate motion vectors may be determined in advance.

For example, in FRUC, as described above, since the index of the predicted motion vector is not encoded, it is possible to suppress an increase in the amount of coding caused by an increase in the number of candidate motion vectors. Therefore, even if the time candidate motion vectors are respectively derived from the plurality of motion vectors of the plurality of reference blocks, the increase in the amount of coding caused by the increase in the number of candidate motion vectors over time can be suppressed, wherein the plurality of reference blocks It is mapped to a plurality of corresponding regions overlapping the coding target block. Therefore, in order to suppress an increase in the processing load, the upper limit of the number of time candidate motion vectors can also be determined in advance.

29 is a conceptual diagram showing an example of a coding target block, a plurality of adjacent blocks, and a plurality of corresponding regions. When the encoding target block does not include the corresponding region of the reference block and is configured by a blank area, the candidate list generating unit 134 derives the temporal candidate motion vector by transmitting the motion vector of the peripheral block of the encoding target block. That is, at this time, the candidate list generating unit 134 derives the temporal candidate motion vector from the motion vector of the reference block that has been mapped to the corresponding region around the encoding target block.

For example, the candidate list generation unit 134 derives a temporal candidate motion vector from a motion vector of the reference block Blk01 that has been mapped to the corresponding region Blk01c, wherein the corresponding region Blk01c is overlapped in the first adjacent region adjacent to the right side of the encoding target block. Piece.

Furthermore, the candidate list generating unit 134 may determine whether each of the plurality of adjacent blocks is a blank area in a predetermined order, wherein the plurality of adjacent blocks include the first adjacent block and the second adjacent area. Block and third adjacent block, etc. Then, the candidate list generation unit 134 derives a temporal candidate motion vector from the motion vector of the reference block that has been mapped to the corresponding region, wherein the corresponding region is overlapped in the region that was originally determined not to be a blank region.

Alternatively, the candidate list generating unit 134 may weight the plurality of motion vectors of the plurality of reference blocks according to the spatial distance from the encoding target block to the corresponding regions, wherein the plurality of reference blocks are A plurality of corresponding regions that have been mapped around the block of the encoding object. Then, the candidate list generating unit 134 may derive the temporal candidate motion vector by weighting the average of the plurality of motion vectors.

For example, when a weighted average of a plurality of motion vectors is used, a weighted average of the normalized plurality of motion vectors may also be used in accordance with the temporal distance. That is, a weighted average of a plurality of motion vectors that have been respectively scaled to the motion vector of the reference picture L0[0] or L1[0] from the encoding target picture may be used.

Figure 30 is a conceptual diagram showing a block determined as a coding unit and a sub-block determined by a predetermined size. The predetermined size is, for example, a size determined to be M × N pixels. Specifically, in the block in which the coding unit is determined by the size of 16 × 16 pixels, 16 sub-blocks may be determined by the size of 4 × 4 pixels.

The coding target block may be a block determined as a coding unit, or may be a sub-block determined by a predetermined size in a block determined as a coding unit. Similarly, the reference block may be a block determined as a coding unit, or may be a sub-block determined by a predetermined size in a block determined as a coding unit.

That is, the candidate list generating unit 134 may derive the candidate motion vector for each block determined in the coding unit, or may derive the candidate motion vector for each of the sub-blocks determined by the predetermined size. Then, the inter prediction unit 126 may select a prediction motion vector for each of the blocks determined by the coding unit to perform prediction, or may select a prediction motion vector for each sub-block determined by a predetermined size. Make predictions.

For example, there are cases where foreground and background are mixed in a block of coding units. At this time, the candidate list generating unit 134 performs processing in each of the sub-blocks, whereby the candidate motion vectors suitable for the motions of the sub-blocks belonging to the foreground and the sub-blocks belonging to the background can be derived. Then, the inter prediction section 126 performs processing in each sub-block, whereby prediction can be performed by selecting a prediction motion vector suitable for the motion of each of the sub-blocks belonging to the foreground and the sub-blocks belonging to the background.

Further, when the candidate list generation unit 134 derives the temporal candidate motion vector for the sub-block of the blank region, the candidate list generation unit 134 may derive the time from the motion vector of the other sub-blocks in the same block corresponding to the sub-block including the blank region. Candidate motion vector. In other words, the candidate list generating unit 134 may use the range of the coding unit instead of the reference block outside the range of the coding unit, and use it as the range around the sub-block. Thereby, the candidate list generating section 134 can appropriately limit the reference range.

In addition, the action of deriving the candidate motion vector per sub-block and selecting the prediction motion vector may also be applied to the method of using the co-located block or the ATMVP block in each of the two reference picture lists.

Further, the operation of deriving the candidate motion vector for each sub-block and selecting the prediction motion vector may be performed as the processing of the FRUC. That is, the prediction motion vector may be selected from the plurality of candidate motion vectors in each sub-block according to the template matching method or the bidirectional matching method in the FRUC.

For example, when a predicted motion vector has been selected for each block, the number of predicted motion vectors per coding unit is increased. Thereby, the number of indexes of the predicted motion vector also increases, and there is a possibility that the amount of coding is increased. However, in FRUC, since the index of the predicted motion vector is not encoded, the increase in the amount of coding caused by the increase in the number of candidate motion vectors can be suppressed.

Further, when the candidate motion vector is derived by mapping the reference block, the candidate list generation unit 134 may derive two or more from one reference block as in the case of using the parity block or the ATMVP block. Candidate motion vector.

Specifically, the candidate list generating unit 134 may also derive two candidate motion vectors that can be utilized in the bi-prediction from one motion vector of the reference block, or may derive from the two motion vectors of the reference block and may be utilized in the bi-prediction. 2 candidate motion vectors. Moreover, the candidate list generating unit 134 may also derive two candidate motion vectors that can be utilized in the bi-prediction from the two motion vectors of the reference block, thereby extracting four candidate motion vectors from the two motion vectors of the reference block. .

Further, when the reference block has been encoded by bi-prediction, the candidate list generating unit 134 may map the reference block to the two corresponding regions in the encoding target picture in accordance with the two motion vectors of the reference block.

Moreover, the candidate list generating unit 134 is not limited to the reference picture whose reference picture index is 0, and the reference block of the other reference picture may be mapped to the corresponding area in the encoding target picture according to the motion vector of the reference block.

For example, instead of the plurality of reference blocks of the reference pictures L0[0] and L1[0], a plurality of reference blocks of the reference pictures L0[1] and L1[1] may also be used for mapping. Alternatively, in addition to the plurality of reference blocks of the reference pictures L0[0] and L1[0], a plurality of reference blocks of the reference pictures L0[1] and L1[1] may be used for mapping.

[Example of Mounting of Encoding Device] Fig. 31 is a block diagram showing an example of mounting of the encoding device 100 according to the first embodiment. The encoding device 100 includes a circuit 160 and a memory 162. For example, a plurality of components of the encoding device 100 shown in FIGS. 1 and 16 are mounted through the circuit 160 and the memory 162 shown in FIG.

The circuit 160 is a circuit for performing information processing and is a circuit that can access the memory 162. For example, circuit 160 is a dedicated or general purpose electronic circuit that encodes moving images. Circuitry 160 can also be a CPU-like processor. Also, circuit 160 can be an aggregate of a plurality of electronic circuits. Further, for example, the circuit 160 can realize the function of a plurality of constituent elements other than the constituent elements for storing information among the plurality of constituent elements of the encoding apparatus 100 shown in FIG.

Memory 162 is a general purpose or dedicated memory that memorizes information used by circuit 160 to encode moving images. The memory 162 can be an electronic circuit or can be connected to the circuit 160. Also, memory 162 may be included in circuit 160. Further, the memory 162 may be an aggregate of a plurality of electronic circuits. Further, the memory 162 may be a magnetic disk or a compact disk, or may be represented as a storage or a recording medium. Moreover, the memory 162 can be a non-volatile memory or a volatile memory.

For example, the memory 162 can also memorize the moving image that is encoded, or the bit string corresponding to the dynamic image that has been encoded. Further, a program for causing the circuit 160 to encode a moving image can be stored in the memory 162.

Further, for example, the memory 162 may also function as a constituent element for storing information among a plurality of constituent elements of the encoding device 100 shown in FIG. 1 and the like. Specifically, the memory 162 can also function as the block memory 118, the frame memory 122, and the candidate block information memory 136 shown in FIG. More specifically, the memory 162 can also memorize the reconstructed block, reconstruct the completed picture, and the motion vector that has been used for inter prediction.

Further, in the coding apparatus 100, it is not necessary to install all of the plurality of constituent elements shown in FIG. 1 and the like, and it is not necessary to perform all of the above-described plural processing. One of the plurality of constituent elements shown in FIG. 1 and the like may be included in other devices, and one of the plurality of processing described above may also be executed by other means. Then, in the encoding apparatus 100, a part of the plurality of constituent elements shown in FIG. 1 and the like is mounted, and a part of the above-described plural processing is performed, whereby the moving image can be appropriately processed with a small amount of encoding.

Fig. 32A is a flowchart showing a first operation example regarding the derivation of the candidate motion vectors in the case of moving picture coding. The encoding apparatus 100 shown in Fig. 31 can also perform the operation shown in Fig. 32A when encoding a moving image.

Specifically, the circuit 160 of the encoding device 100 derives one or more candidate motion vectors from the prediction motion vector of the encoding target block from one or more motion vectors of the first reference block (S101). Here, the first reference block is a block included in the first reference picture, wherein the first reference picture is a first reference picture list among the two reference picture lists constituting the double prediction.

Further, the circuit 160 derives one or more candidate motion vectors from the predicted motion vector from one or more motion vectors of the second reference block (S102). Here, the second reference block is included in the block included in the second reference picture, wherein the second reference picture is a second reference picture list among the two reference picture lists constituting the double prediction.

Next, the circuit 160 selects a predicted motion vector from among a plurality of candidate motion vectors (S103). Here, the plurality of candidate motion vectors are one or more candidate motion vectors derived from one or more motion vectors of the first reference block, and one or more motion vectors that have been derived from the second reference block. One or more candidate motion vectors derived. Then, the circuit 160 encodes the information of the encoding target block using the predicted motion vector (S104).

Thereby, the encoding apparatus 100 can derive the candidate motion vector from the first reference block in the first reference picture list, and can derive the candidate motion vector from the second reference block in the second reference picture list. Therefore, the encoding apparatus 100 can increase the possibility that a plurality of candidate motion vectors for selecting a predicted motion vector contain appropriate candidate motion vectors.

Then, the encoding device 100 can support the derivation of a suitable predicted motion vector, and can support the reduction of the encoding amount of the moving image.

For example, when the first reference block has been encoded by bi-prediction, the circuit 160 may apply two scaling ratio values to one of the two motion vectors of the first reference block. By applying this, the circuit 160 may also derive a candidate motion vector for referring to the first reference picture list and a candidate motion vector for referring to the second reference picture list from one motion vector of the first reference block.

Further, when the second reference block has been encoded by bi-prediction, the circuit 160 may apply two scaling ratio values to one of the two motion vectors of the second reference block. By applying this, the circuit 160 may derive a candidate motion vector for referring to the first reference picture list and a candidate motion vector for referring to the second reference picture list from one motion vector of the second reference block.

Thereby, the encoding apparatus 100 can derive the candidate motion vector combination from one motion vector of the first reference block and one motion vector of the second reference block, respectively, wherein the candidate motion vector combination includes two references. Refer to the 2 candidate motion vectors of the picture list. That is, the encoding apparatus 100 may derive two sets of candidate motion vector combinations from two motion vectors having different characteristics, wherein the candidate motion vector combinations are each composed of two candidate motion vectors, which are available in the dual prediction.

Further, for example, when the first reference block has been encoded by bi-prediction in only one direction among the upper and lower directions in the display order, the circuit 160 may also use one of the two motion vectors of the first reference block. The motion vector applies to two scaling ratios. By applying this, the circuit 160 may also derive a candidate motion vector for referring to the first reference picture list from the one motion vector of the two motion vectors of the first reference block, and refer to the second reference picture list. Candidate motion vector.

Further, when the second reference block has been encoded by bi-prediction in only one direction among the front and rear directions in the display order, the circuit 160 may move one of the two motion vectors of the second reference block. The vector applies to 2 scaling ratios. By applying this, the circuit 160 may also derive a candidate motion vector for referring to the first reference picture list from the one motion vector of the two motion vectors of the second reference block, and refer to the second reference picture list. Candidate motion vector.

Thereby, the encoding apparatus 100 can derive the candidate motion vector combination from only one motion vector among the two motion vectors assumed to be similar, wherein the candidate motion vector combination is two of the reference two reference picture lists. The candidate motion vector is composed. Therefore, the encoding device 100 can improve processing efficiency.

Further, for example, when the first reference block has been encoded by bi-prediction, the circuit 160 may apply two scaling ratio values to the two motion vectors of the first reference block. By applying this, the circuit 160 may also derive two candidate motion vectors for referring to the first reference picture list and two candidate movements for referring to the second reference picture list from the two motion vectors of the first reference block. vector.

Further, when the second reference block has been encoded by bi-prediction, the circuit 160 may apply two scaling ratio values to the two motion vectors of the second reference block. By applying this, the circuit 160 may also derive two candidate motion vectors for referring to the first reference picture list and two candidate movements for referring to the second reference picture list from the two motion vectors of the second reference block. vector.

Thereby, the encoding apparatus 100 can derive the candidate motion vector combination from the four motion vectors of the two reference blocks, wherein the candidate motion vector combination is formed by referring to the two candidate motion vectors of the two reference picture lists. That is, the encoding apparatus 100 may derive four sets of candidate motion vector combinations from four motion vectors of two reference blocks, wherein the candidate motion vector combinations are each composed of two candidate motion vectors, which are Available in double prediction.

Further, for example, when the first reference block has been encoded by bi-prediction, the circuit 160 may also derive a reference vector from the one of the two motion vectors of the first reference block for reference to the first reference picture. The candidate movement vector for the list. Next, at this time, the circuit 160 may derive a candidate motion vector for referring to the second reference picture list from another one of the two motion vectors of the first reference block.

Moreover, when the second reference block has been encoded by bi-prediction, the circuit 160 may also derive a reference vector from the first reference picture list by using one of the two motion vectors of the second reference block. Candidate motion vector. Next, at this time, the circuit 160 may also derive a candidate motion vector for referring to the second reference picture list from another one of the two motion vectors of the second reference block.

Thereby, the encoding apparatus 100 can derive four candidate motion vectors from four motion vectors of two reference blocks, and can derive two sets of candidate motion vector combinations composed of four candidate motion vectors. That is, the encoding apparatus 100 can appropriately reflect the four motion vectors of the two reference blocks in the four candidate motion vectors included in the two sets of candidate motion vector combinations.

Further, for example, when the first reference block has been encoded by single prediction, the circuit 160 may apply two scaling ratio values to one motion vector of the first reference block. By applying this, the circuit 160 may derive a candidate motion vector for referring to the first reference picture list and a candidate motion vector for referring to the second reference picture list from one motion vector of the first reference block.

Further, when the second reference block has been encoded by single prediction, the circuit 160 may apply two scaling ratio values to one motion vector of the second reference block. By applying this, the circuit 160 may derive a candidate motion vector for referring to the first reference picture list and a candidate motion vector for referring to the second reference picture list from one motion vector of the second reference block.

Thereby, the encoding apparatus 100 can derive a candidate motion vector combination composed of two candidate motion vectors with reference to two reference picture lists from only one motion vector of each reference block. That is, the encoding apparatus 100 can derive a candidate motion vector combination from only one motion vector of each reference block, wherein the candidate motion vector combination is composed of two candidate motion vectors available for bi-prediction.

Further, for example, the encoding target block may be encoded by bi-prediction in only one direction among the front direction and the rear direction in the display order. At this time, the circuit 160 may derive two or more candidate motion vectors from one or more motion vectors of the first reference block, wherein the candidate motion vector is used for reference to the double prediction used in the coding target block. One direction. Then, at this time, the circuit 160 may derive two or more candidate motion vectors from one or more motion vectors of the second reference block, wherein the candidate motion vector is used to refer to the pair used in the coding target block. Predict one direction.

Thereby, when the encoding target block uses bi-prediction in only one direction, the coding apparatus 100 can derive two or more candidate motion vectors in one direction from each reference block. Therefore, the encoding apparatus 100 can derive one or more sets of candidate motion vectors that are suitable for bi-prediction in only one direction from each reference block.

Also, for example, the first reference picture list may also be a reference picture list for referring to the front direction in the display order. The second reference picture list may also be a reference picture list for referring to the rear direction in the display order. The first reference block can also be encoded by bi-prediction in both directions of the front and back directions in the display order. The second reference block can also be encoded by bi-prediction in both directions of the front and back directions in the display order. The coding target block can also be coded by bi-prediction in both directions of the front direction and the backward direction in the display order.

At this time, the circuit 160 may apply two scaling ratio values to one of the two motion vectors of the first reference block for reference to one motion vector in the upper and lower directions of the display order. By applying this, the circuit 160 may derive a candidate motion vector for referring to the first reference picture list and a candidate motion vector for referring to the second reference picture list from one motion vector of the first reference block.

Further, at this time, the circuit 160 may apply two scaling ratio values to one of the two motion vectors of the second reference block for reference to one motion vector in the upward direction of the display order. By applying this, the circuit 160 may derive a candidate motion vector for referring to the first reference picture list and a candidate motion vector for referring to the second reference picture list from one motion vector of the second reference block.

Thereby, the encoding apparatus 100 can derive a candidate motion vector combination of the reference two reference picture lists from the motion vector of the reference direction, which will be described later, among the two motion vectors of each reference block, wherein the reference direction is time-based self-reference The block faces the direction of the encoding object block. That is, the encoding apparatus 100 can appropriately derive the candidate motion vector usable in the bi-prediction from among the two motion vectors of each reference block from the motion vector having high correlation with respect to the encoding target block in temporal nature. combination.

Further, for example, the position of the spatial property of the first reference block may be the same as the position of the spatial property of the coding target block, and the position of the spatial property of the second reference block may also be the position of the spatial property of the coding target block. the same.

Fig. 32B is a flowchart showing a second operation example of deriving the candidate motion vector at the time of moving picture coding. The encoding apparatus 100 shown in Fig. 31 can also perform the operation shown in Fig. 32B when encoding a moving image.

Specifically, the circuit 160 of the encoding apparatus 100 specifies the first reference block included in the first reference picture using the first motion vector of the first adjacent block, wherein the first adjacent block is spatially adjacent. A block of the encoding target block (S201). Here, the first reference picture is a picture constituting the first reference picture list among the two reference picture lists for bi-prediction. Then, the circuit 160 derives one or more candidate motion vectors with respect to the prediction motion vector from one or more motion vectors of the first reference block, where the prediction motion vector is a prediction motion vector of the coding target block (S202) ).

Further, the circuit 160 specifies the second reference picture by using the first motion vector, the second motion vector of the first adjacent block, or the third motion vector of the second adjacent block spatially adjacent to the coding target block. The second reference block is included (S203). Here, the second reference picture is a picture constituting the second reference picture list among the two reference picture lists. Then, the circuit 160 derives one or more candidate motion vectors from the predicted motion vector from one or more motion vectors of the second reference block (S204).

Next, the circuit 160 selects a predicted motion vector from among a plurality of candidate motion vectors (S205). Here, the plurality of candidate motion vectors are one or more candidate motion vectors derived from one or more motion vectors of the first reference block, and one or more motion vectors that have been derived from the second reference block. One or more candidate motion vectors derived. Then, the circuit 160 encodes the information of the encoding target block using the predicted motion vector (S206).

Thereby, the encoding apparatus 100 can specify the first reference block in the first reference picture list and the second reference area in the second reference picture list by using one or more motion vectors of one or more adjacent blocks. Piece. In other words, the encoding apparatus 100 can specify the first reference block and the second reference block which are assumed to have high correlation with the encoding target block by using one or more motion vectors of one or more adjacent blocks.

Then, the encoding device 100 may derive a candidate motion vector from the first reference block and derive a candidate motion vector from the second reference block. Accordingly, the encoding apparatus 100 can increase the likelihood that a suitable candidate motion vector will be included among a plurality of candidate motion vectors used to select the predicted motion vector. Then, the encoding device 100 can support the derivation of a suitable predicted motion vector, and can support the reduction of the encoding amount of the moving image.

Further, in the second operation example shown in FIG. 32B, the modification or specific example described with reference to FIG. 32A can be applied.

The second operation example shown in FIG. 32B is an example in which the first reference block and the second reference block in the operation example shown in FIG. 32A are each an ATMVP block. In the first operation example shown in FIG. 32A, each of the first reference block and the second reference block may not be an ATMVP block but a parity block.

Specifically, the first reference block in the first operation example shown in FIG. 32A may be the same block as the encoding target block in the first reference picture. Then, the second reference block in the first operation example shown in FIG. 32A may be the same block as the encoding target block in the second reference picture.

Fig. 32C is a flowchart showing a third operation example in which the candidate motion vectors are derived in the case of moving picture coding. The encoding apparatus 100 shown in Fig. 31 can also perform the operation shown in Fig. 32C when encoding a moving image.

Specifically, the circuit 160 of the encoding device 100 maps each reference block in the plurality of reference blocks to a corresponding region in the image to be encoded according to the motion vector of the reference block (S301).

Here, the plurality of reference blocks include one or more reference blocks included in the first reference picture and one or more reference blocks included in the second reference picture. The first reference picture is a picture of the first reference picture list among the two reference picture lists for bi-prediction. The second reference picture is a picture constituting the second reference picture list among the two reference picture lists.

Then, when the corresponding region overlaps the coding target block in the coding target image, the circuit 160 derives the candidate motion vector from the motion vector of the reference block that has been mapped to the corresponding region, as one of the plurality of candidate motion vectors. One (S302). The plurality of candidate motion vectors are a plurality of candidate motion vectors with respect to a prediction motion vector of the coding target block.

Next, the circuit 160 selects a predicted motion vector from among a plurality of candidate motion vectors (S303). Then, the circuit 160 encodes the information of the encoding target block using the predicted motion vector (S304).

Thereby, the encoding apparatus 100 can derive a suitable reference block from the plurality of reference blocks in the first reference picture list and the second reference picture list from the reference block that is assumed to have high correlation with the coding target block. Candidate motion vector. Accordingly, the encoding apparatus 100 can increase the likelihood that a suitable candidate motion vector will be included among a plurality of candidate motion vectors used to select the predicted motion vector.

Then, the encoding device 100 can support the derivation of a suitable predicted motion vector, and can support the reduction of the encoding amount of the moving image.

For example, the circuit 160 may map each of the one or more reference blocks in the first reference picture to the encoding target image. After that, the circuit 160 may also separate one or more reference blocks in the second reference picture from the one of the first reference pictures in the encoding target image. Screening in the area.

Thereby, the encoding apparatus 100 can more preferentially map one or more reference blocks in the first reference picture list than one or more reference blocks in the second reference picture list. The reliability of the motion vector of the reference block in the first reference picture list is higher than the reliability of the motion vector of the reference block in the second reference picture list. In this case, the encoding device 100 can derive a suitable candidate motion vector in accordance with a highly reliable motion vector.

Further, for example, two or more corresponding regions in which two or more reference blocks among the plurality of reference blocks are mapped may be superimposed on the coding target block. At this time, the circuit 160 may derive a candidate motion vector from at least one of two or more motion vectors of two or more reference blocks.

Thereby, the encoding apparatus 100 can derive a suitable candidate motion vector from at least one of the plurality of reference blocks assumed to have high correlation with respect to the encoding target block.

Further, for example, there is no overlap in the corresponding region of the encoding target block, and there is a corresponding region overlapping the peripheral region of the encoding target block. At this time, the circuit 160 may also derive the candidate motion vector from the motion vector of the reference block that is mapped to the corresponding region of the peripheral region.

Thereby, the encoding apparatus 100 can derive the appropriate candidate motion vector according to the motion vector of the reference block mapped in the periphery of the encoding target block even if there is no reference block mapping in the encoding target block.

Further, for example, when the prediction motion vector is selected from among a plurality of candidate motion vectors, the circuit 160 may evaluate each of the plurality of candidate motion vectors, and select the most highly evaluated candidate motion vector among the plurality of candidate motion vectors as the candidate motion vector. Predict the motion vector.

When the circuit 160 evaluates each of the plurality of candidate motion vectors, the higher the degree of suitability between the reconstructed images of the two comparison target regions, the higher the candidate motion vector of the evaluation target is. Here, the two comparison target areas are two areas different from the coding target block, and at least one of them is two areas determined in accordance with the candidate motion vector of the evaluation target.

Thereby, the encoding apparatus 100 can evaluate each candidate motion vector with reference to a reconstructed image different from the region of the encoding target block, and select the predicted motion vector from among the plurality of candidate motion vectors. Therefore, the encoding device 100 and the decoding device 200 can select the predicted motion vector from among the plurality of candidate motion vectors in the same manner. Thereby, the encoding apparatus 100 can omit the encoding for selecting the information of the predicted motion vector, and can support the reduction of the encoding amount.

Also, for example, when the circuit 160 derives the candidate motion vector from the motion vector mapped to the reference block of the corresponding region, the motion vector of the reference block to which the scaling ratio is applied may also be derived as the candidate motion vector.

Here, the scaling ratio is a reference picture from the encoding target picture to the reference picture containing the reference block relative to the reference picture from the reference picture containing the reference block to the reference vector indicated by the motion vector containing the reference block. The ratio of the time difference. The encoding target picture is a picture containing an encoding target image.

Thereby, the encoding device 100 can appropriately scale the motion vector of the reference block and derive the scaled motion vector as the candidate motion vector.

Further, for example, the coding target block may be a block determined in coding units, or may be a sub-block determined in a predetermined size in a block determined in coding units. Thereby, the encoding apparatus 100 can derive a suitable candidate motion vector for the block of the coding unit or the sub-block within the block of the coding unit.

Further, for example, the circuit 160 may also apply, for each of the plurality of coding target blocks in the coding target image, the motion vector of the reference block mapped to the corresponding region when the corresponding region overlaps the coding target block. Export candidate motion vectors. The candidate motion vector is a candidate motion vector with respect to a prediction motion vector of the coding target block.

Thereby, the encoding device 100 can derive an appropriate candidate motion vector for each of the encoding target blocks in the encoding target image.

[Installation Example of Decoding Device] FIG. 33 is a block diagram showing an example of mounting of the decoding device 200 according to the first embodiment. The decoding device 200 includes a circuit 260 and a memory 262. For example, the plurality of components of the decoding device 200 shown in FIGS. 10 and 17 are mounted by the circuit 260 and the memory 262 shown in FIG.

The circuit 260 is a circuit for performing information processing and is a circuit that can access the memory 262. For example, circuit 260 is a dedicated or general purpose electronic circuit that decodes moving images. Circuit 260 can also be like a processor of a CPU. Also, circuit 260 can be an aggregate of a plurality of electronic circuits. Further, for example, the circuit 260 may exhibit the function of a plurality of constituent elements other than the constituent elements for storing information among the plurality of constituent elements of the decoding device 200 shown in FIG. 10 and the like.

Memory 262 is a general purpose or dedicated memory that stores information for causing circuit 260 to decode the moving image. The memory 262 can also be an electronic circuit or can be connected to the circuit 260. Also, memory 262 may be included in circuit 260. Further, the memory 262 may be an aggregate of a plurality of electronic circuits. Further, the memory 262 may be a magnetic disk or a compact disk, or may be embodied as a storage or a recording medium. Further, the memory 262 may be a non-volatile memory or a volatile memory.

For example, the memory 262 can also memorize the bit column corresponding to the encoded dynamic image, and can also memorize the dynamic image corresponding to the bit column that has been decoded. Further, the memory 262 can also store a program for causing the circuit 260 to decode the moving image.

Further, for example, the memory 262 may also function as a constituent element for storing information among a plurality of constituent elements of the decoding device 200 shown in FIG. Specifically, the memory 262 can also function as the block memory 210, the frame memory 214, and the candidate block information memory 236 shown in FIG. More specifically, the memory 262 can also memorize the reconstructed block, reconstruct the completed picture, and the motion vector that has been used for inter prediction.

Further, in the decoding device 200, it is not necessary to install all of the plurality of constituent elements shown in FIG. 10 and the like, and it is not necessary to perform all of the above-described plural processing. One of the plurality of constituent elements shown in FIG. 10 and the like may be included in other devices, and one of the plurality of processing described above may be executed by another device. Then, in the decoding device 200, a part of the plurality of constituent elements shown in FIG. 10 and the like are mounted, and a part of the above-described plural processing is performed, whereby the moving image can be appropriately processed with a small amount of encoding.

Fig. 34A is a flowchart showing a first operation example regarding the derivation of the candidate motion vectors in the case of decoding of a moving picture. The decoding device 200 shown in Fig. 33 can also perform the operation shown in Fig. 34A when decoding a moving image.

Specifically, the circuit 260 of the decoding device 200 derives one or more candidate motion vectors from the prediction motion vector of the decoding target block from one or more motion vectors of the first reference block (S401). Here, the first reference block is a block included in the first reference picture, wherein the first reference picture is a first reference picture list among the two reference picture lists constituting the double prediction.

Further, the circuit 260 derives one or more candidate motion vectors from the predicted motion vector from one or more motion vectors of the second reference block (S402). Here, the second reference block is included in the block included in the second reference picture, wherein the second reference picture is a second reference picture list among the two reference picture lists constituting the double prediction.

Next, the circuit 260 selects a predicted motion vector from among a plurality of candidate motion vectors (S403). Here, the plurality of candidate motion vectors are one or more candidate motion vectors derived from one or more motion vectors of the first reference block, and one or more motion vectors that have been derived from the second reference block. One or more candidate motion vectors derived. Then, the circuit 260 decodes the information of the decoding target block using the predicted motion vector (S404).

Thereby, the decoding device 200 can derive the candidate motion vector from the first reference block in the first reference picture list, and can derive the candidate motion vector from the second reference block in the second reference picture list. Therefore, the decoding apparatus 200 can increase the possibility that the plurality of candidate motion vectors used to select the predicted motion vector contain appropriate candidate motion vectors.

Then, the decoding device 200 can support the derivation of a suitable prediction motion vector, and can support the reduction of the coding amount of the moving image.

For example, when the first reference block has been decoded by bi-prediction, the circuit 260 may apply two scaling ratio values to one of the two motion vectors of the first reference block. By applying this, the circuit 260 may also derive a candidate motion vector for referring to the first reference picture list and a candidate motion vector for referring to the second reference picture list from one motion vector of the first reference block.

Further, when the second reference block has been decoded by bi-prediction, the circuit 260 may apply two scaling ratio values to one of the two motion vectors of the second reference block. By applying this, the circuit 260 may also derive a candidate motion vector for referring to the first reference picture list and a candidate motion vector for referring to the second reference picture list from one motion vector of the second reference block.

Thereby, the decoding apparatus 200 can derive the candidate motion vector combination from one motion vector of the first reference block and one motion vector of the second reference block, respectively, wherein the candidate motion vector combination includes two references. Refer to the 2 candidate motion vectors of the picture list. That is, the decoding apparatus 200 may derive two sets of candidate motion vector combinations from two motion vectors having different characteristics, wherein the candidate motion vector combinations are each composed of two candidate motion vectors, which are available in the dual prediction.

Further, for example, when the first reference block has been decoded by bi-prediction in only one direction among the upper and lower directions in the display order, the circuit 260 may also use one of the two motion vectors of the first reference block. The motion vector applies to two scaling ratios. By applying this, the circuit 260 may also derive a candidate motion vector for referring to the first reference picture list from the one motion vector of the two motion vectors of the first reference block, and refer to the second reference picture list. Candidate motion vector.

Further, when the second reference block has been decoded by bi-prediction in only one direction among the front and rear directions in the display order, the circuit 260 may move one of the two motion vectors of the second reference block. The vector applies to 2 scaling ratios. By applying this, the circuit 260 may also derive a candidate motion vector for referring to the first reference picture list from the one motion vector of the two motion vectors of the second reference block, and refer to the second reference picture list. Candidate motion vector.

Thereby, the decoding apparatus 200 can derive the candidate motion vector combination from only one motion vector among the two motion vectors assumed to be similar, wherein the candidate motion vector combination is two of the reference two reference picture lists. The candidate motion vector is composed. Therefore, the decoding device 200 can improve processing efficiency.

Further, for example, when the first reference block has been decoded by bi-prediction, the circuit 260 may apply two scaling ratio values to the two motion vectors of the first reference block. By applying this, the circuit 260 may also derive two candidate motion vectors for referring to the first reference picture list and two candidate movements for referring to the second reference picture list from the two motion vectors of the first reference block. vector.

Further, when the second reference block has been decoded by bi-prediction, the circuit 260 may apply two scaling ratio values to the two motion vectors of the second reference block. By applying this, the circuit 260 may also derive two candidate motion vectors for referring to the first reference picture list and two candidate movements for referring to the second reference picture list from the two motion vectors of the second reference block. vector.

Thereby, the decoding device 200 can derive the candidate motion vector combination from the four motion vectors of the two reference blocks, wherein the candidate motion vector combination is formed by referring to the two candidate motion vectors of the two reference picture lists. That is, the decoding apparatus 200 may derive four sets of candidate motion vector combinations from four motion vectors of two reference blocks, wherein the candidate motion vector combinations are each composed of two candidate motion vectors, which are Available in double prediction.

Further, for example, when the first reference block has been decoded by bi-prediction, the circuit 260 may also derive a reference vector from the one of the two motion vectors of the first reference block to refer to the first reference picture. The candidate movement vector for the list. Next, at this time, the circuit 260 may also derive a candidate motion vector for referring to the second reference picture list from another one of the two motion vectors of the first reference block.

Moreover, when the second reference block has been decoded by bi-prediction, the circuit 260 may also derive a reference vector from the first reference picture list from one of the two motion vectors of the second reference block. Candidate motion vector. Next, at this time, the circuit 260 may also derive a candidate motion vector for referring to the second reference picture list from another one of the two motion vectors of the second reference block.

Thereby, the decoding apparatus 200 can derive four candidate motion vectors from four motion vectors of two reference blocks, and can derive two sets of candidate motion vector combinations composed of four candidate motion vectors. That is, the decoding apparatus 200 can appropriately reflect the four motion vectors of the two reference blocks in the four candidate motion vectors included in the two sets of candidate motion vector combinations.

Further, for example, when the first reference block has been decoded by single prediction, the circuit 260 may apply two scaling ratio values to one motion vector of the first reference block. By applying this, the circuit 260 may also derive a candidate motion vector for referring to the first reference picture list and a candidate motion vector for referring to the second reference picture list from one motion vector of the first reference block.

Further, when the second reference block has been decoded by single prediction, the circuit 260 may apply two scaling ratio values to one motion vector of the second reference block. By applying this, the circuit 260 may also derive a candidate motion vector for referring to the first reference picture list and a candidate motion vector for referring to the second reference picture list from one motion vector of the second reference block.

Thereby, the decoding apparatus 200 can derive a candidate motion vector combination composed of two candidate motion vectors with reference to two reference picture lists from only one motion vector of each reference block. That is, the decoding apparatus 200 can derive a candidate motion vector combination from only one motion vector of each reference block, wherein the candidate motion vector combination is composed of two candidate motion vectors that can be utilized in bi-prediction.

Further, for example, the decoding target block can also be decoded by bi-prediction in only one direction among the front direction and the rear direction in the display order. At this time, the circuit 260 may derive two or more candidate motion vectors from one or more motion vectors of the first reference block, wherein the candidate motion vector is used to refer to the double prediction used in the decoding target block. One direction. Then, at this time, the circuit 260 may derive two or more candidate motion vectors from one or more motion vectors of the second reference block, wherein the candidate motion vector is used to refer to the pair used in the decoding target block. Predict one direction.

Thereby, the decoding apparatus 200 can derive two or more candidate motion vectors in the reference one direction from each reference block when the decoding target block uses bi-prediction in only one direction. Therefore, the decoding apparatus 200 can derive one or more sets of candidate motion vectors that are suitable for bi-prediction in only one direction from each reference block.

Also, for example, the first reference picture list may also be a reference picture list for referring to the front direction in the display order. The second reference picture list may also be a reference picture list for referring to the rear direction in the display order. The first reference block can also be decoded by bi-prediction in both directions of the front and back directions in the display order. The second reference block can also be decoded by bi-prediction in both directions of the front and back directions in the display order. The decoding target block can also be decoded by bi-prediction in both directions of the front direction and the backward direction in the display order.

At this time, the circuit 260 may apply two scaling ratio values to one of the two motion vectors of the first reference block for reference to one motion vector in the upper and lower directions of the display order. By applying this, the circuit 260 may also derive a candidate motion vector for referring to the first reference picture list and a candidate motion vector for referring to the second reference picture list from one motion vector of the first reference block.

Further, at this time, the circuit 260 may apply two scaling ratio values to one of the two motion vectors of the second reference block for reference to one motion vector in the upper direction of the display order. By applying this, the circuit 260 may also derive a candidate motion vector for referring to the first reference picture list and a candidate motion vector for referring to the second reference picture list from one motion vector of the second reference block.

Thereby, the decoding apparatus 200 can derive a candidate motion vector combination of the reference two reference picture lists from the motion vector of the reference direction, which will be described later, among the two motion vectors of each reference block, wherein the reference direction is time-based self-reference The block faces the direction of the decoding target block. That is, the decoding apparatus 200 can appropriately derive the candidate motion vector usable in the bi-prediction from among the two motion vectors of each reference block from the motion vector having high correlation with respect to the decoding target block in temporal nature. combination.

Further, for example, the position of the spatial property of the first reference block may be the same as the position of the spatial property of the decoding target block, and the position of the spatial property of the second reference block may also be the position of the spatial property of the decoding target block. the same.

Fig. 34B is a flowchart showing a second operation example of deriving the candidate motion vector at the time of moving picture decoding. The decoding device 200 shown in FIG. 33 can also perform the operation shown in FIG. 34B when decoding a moving image.

Specifically, the circuit 260 of the decoding device 200 specifies the first reference block included in the first reference picture using the first motion vector of the first adjacent block, wherein the first adjacent block is spatially adjacent. The block of the decoding target block is decoded (S501). Here, the first reference picture is a picture of the first reference picture list among the two reference picture lists constituting the double prediction. Then, the circuit 260 derives one or more candidate motion vectors with respect to the prediction motion vector of the decoding target block from one or more motion vectors of the first reference block (S502).

Further, the circuit 260 specifies the second reference picture by using the first motion vector, the second motion vector of the first adjacent block, or the third motion vector of the second adjacent block spatially adjacent to the decoding target block. The second reference block is included (S503). Here, the second reference picture is a picture constituting the second reference picture list among the two reference picture lists. Then, the circuit 260 derives one or more candidate motion vectors with respect to the predicted motion vector from one or more motion vectors of the second reference block (S504).

Next, the circuit 260 selects a predicted motion vector from among a plurality of candidate motion vectors (S505). Here, the plurality of candidate motion vectors include one or more candidate motion vectors derived from one or more motion vectors of the first reference block, and one or more motion vectors derived from the second reference block. More than one candidate motion vector. Then, the circuit 260 decodes the information of the decoding target block using the predicted motion vector (S506).

Thereby, the decoding apparatus 200 can specify the first reference block in the first reference picture list and the second reference block in the second reference picture list by using one or more motion vectors of one or more adjacent blocks. . In other words, the decoding apparatus 200 can specify the first reference block and the second reference block which are assumed to have high correlation with the decoding target block by using one or more motion vectors of one or more adjacent blocks.

Next, the decoding device 200 may derive a candidate motion vector from the first reference block and may derive a candidate motion vector from the second reference block. Accordingly, decoding apparatus 200 may increase the likelihood that a plurality of candidate motion vectors used to select a predicted motion vector contain suitable candidate motion vectors. Then, the decoding device 200 can support the derivation of an appropriate prediction motion vector, and can support the reduction of the coding amount of the motion image.

Further, in the second operation example shown in FIG. 34B, the modification or specific example described using FIG. 34A can be applied.

The second operation example shown in FIG. 34B is an example in which the first reference block and the second reference block in the operation example shown in FIG. 34A are in the form of ATMVP blocks. In the first operation example shown in FIG. 34A, each of the first reference block and the second reference block may not be an ATMVP block but a parity block.

Specifically, the first reference block in the first operation example shown in FIG. 34A may be the same block as the decoding target block position in the first reference picture. Then, the second reference block in the first operation example shown in FIG. 34A may be the same block as the decoding target block position in the second reference picture.

Fig. 34C is a flowchart showing a third operation example regarding the derivation of the candidate motion vector at the time of decoding of a moving picture. The decoding device 200 shown in FIG. 33 can also perform the operation shown in FIG. 34C when decoding a moving image.

Specifically, the circuit 260 of the decoding device 200 maps each reference block in the plurality of reference blocks to a corresponding region in the decoding target image according to the motion vector of the reference block (S601).

Here, the plurality of reference blocks include one or more reference blocks included in the first reference picture and one or more reference blocks included in the second reference picture. The first reference picture is a picture of the first reference picture list among the two reference picture lists used for the dual prediction. The second reference picture is a picture constituting a list of the second reference picture among the two reference picture lists.

Then, when the corresponding area overlaps the decoding target block in the decoding target image, the circuit 260 derives the candidate motion vector from the motion vector of the reference block that has been mapped to the corresponding area, as a plurality of candidate motion vectors. 1 (S602). The plurality of candidate motion vectors are a plurality of candidate motion vectors with respect to a prediction motion vector of the decoding target block.

Next, the circuit 260 selects a predicted motion vector from among a plurality of candidate motion vectors (S603). Then, the circuit 260 decodes the information of the decoding target block using the predicted motion vector (S604).

Thereby, the decoding apparatus 200 can derive a suitable reference block from the plurality of reference blocks in the first reference picture list and the second reference picture list from the reference block that is assumed to have high correlation with respect to the decoding target block. Candidate motion vector. Accordingly, decoding apparatus 200 may increase the likelihood that a plurality of candidate motion vectors used to select a predicted motion vector contain suitable candidate motion vectors.

Then, the decoding device 200 can support the derivation of an appropriate prediction motion vector, and can support the reduction of the coding amount of the motion image.

For example, the circuit 260 may perform mapping of one or more reference blocks in the first reference picture in the decoding target image. After that, the circuit 260 may also map one or more reference blocks in the second reference picture to each other in a blank area, where the blank area is one of the first reference pictures among the decoding target images. More than one reference block is not mapped.

Thereby, the decoding device 200 is one or more reference blocks in the second reference picture list, and one or more reference blocks in the first reference picture list can be preferentially mapped. The reliability of the motion vector of the reference block in the first reference picture list is higher than the reliability of the motion vector of the reference block in the second reference picture list. In this case, the decoding device 200 can derive a suitable candidate motion vector in accordance with a more reliable motion vector.

Further, for example, two or more corresponding regions may be superimposed on the decoding target block, wherein the two or more corresponding regions are two or more reference blocks of the plurality of reference blocks that have been mapped. region. At this time, the circuit 260 may derive a candidate motion vector from at least one of two or more motion vectors of two or more reference blocks.

Thereby, the decoding device 200 can derive a suitable candidate motion vector from at least one of the plurality of reference blocks that are assumed to have high correlation with respect to the decoding target block.

Further, for example, there is no overlap in the corresponding region of the decoding target block, and there may be a corresponding region overlapping the peripheral region of the decoding target block. At this time, the circuit 260 may also derive the candidate motion vector from the motion vector of the reference block that is mapped to the corresponding region of the peripheral region.

Thereby, the decoding apparatus 200 can derive the appropriate candidate motion vector according to the motion vector of the reference block mapped in the periphery of the decoding target block even if there is no reference block mapping in the decoding target block.

Further, for example, when the prediction motion vector is selected from among a plurality of candidate motion vectors, the circuit 260 may evaluate each of the plurality of candidate motion vectors, and select a candidate motion vector that is most highly evaluated among the plurality of candidate motion vectors. To predict the motion vector.

When the circuit 260 evaluates each of the plurality of candidate motion vectors, the higher the degree of suitability between the reconstructed images of the two comparison target regions, the higher the candidate motion vector of the evaluation target is evaluated. Here, the two comparison target areas are two areas different from the decoding target block, and at least one of them is two areas determined in accordance with the candidate motion vector of the evaluation target.

Thereby, the decoding device 200 can evaluate each candidate motion vector with reference to a reconstructed image different from the region of the decoding target block, and select the predicted motion vector from among the plurality of candidate motion vectors. Therefore, the encoding device 100 and the decoding device 200 can select the predicted motion vector from among the plurality of candidate motion vectors in the same manner. Thereby, the decoding apparatus 200 can omit the decoding of the information for selecting the predicted motion vector, and can support the reduction of the amount of encoding.

Also, for example, when the circuit 260 derives the candidate motion vector from the motion vector of the reference block that has been mapped to the corresponding region, the motion vector of the reference block to which the scaling ratio has been applied may also be derived as the candidate motion vector.

Here, the scaling ratio refers to a reference of a time difference from a decoding target picture to a reference picture containing a reference block with respect to a reference area indicated by a motion vector containing a reference block to a motion vector containing a reference block. The ratio of the time difference of the picture. The decoding target picture is a picture containing the image of the decoding target.

Thereby, the decoding device 200 can appropriately scale the motion vector of the reference block, and can derive the scaled motion vector as a candidate motion vector.

Further, for example, the decoding target block may be a block determined in units of decoding, or may be a sub-block determined in a predetermined size in a block determined in decoding units.

Thereby, the decoding device 200 can derive a suitable candidate motion vector for the block of the decoding unit or the sub-block within the block of the decoding unit.

Further, for example, the circuit 260 may also apply, for each of the plurality of decoding target blocks in the decoding target image, the motion vector of the reference block mapped to the corresponding region when the corresponding region overlaps the decoding target block. Export candidate motion vectors. Here, the candidate motion vector is a candidate motion vector with respect to the prediction motion vector of the decoding target block.

Thereby, the decoding device 200 can derive an appropriate candidate motion vector for each decoding target block in the decoding target image.

[Supplement] The first reference picture list described above is one of the L0 reference picture list and the L1 picture list, and the second reference picture list described above is the other of the L0 reference picture list and the L1 picture list. For example, the first reference picture list is the L0 reference picture list, and the second reference picture list is the L1 reference picture list. Alternatively, the first reference picture list is the L1 reference picture list, and the second reference picture list is the L0 reference picture list.

Moreover, the motion vector of the reference block may also be scaled to refer to candidate motion vectors of other reference pictures from the processing target block, and the other reference pictures are other reference pictures different from the reference picture whose reference picture index is 0. Moreover, the motion vector of the reference block may also be unscaled and derived as a candidate motion vector.

Further, when the scaling ratio is 1, the scaling processing may be skipped in the application of the scaling ratio.

Further, each of the encoding device 100 and the decoding device 200 in the present embodiment may be used as an image encoding device and an image decoding device, or may be used as a moving image encoding device and a moving image decoding device. use. Alternatively, the encoding device 100 and the decoding device 200 may each be utilized as an inter prediction device.

In other words, each of the encoding device 100 and the decoding device 200 may correspond to only the prediction unit 126 including the candidate list generation unit 134 and the prediction unit 218 including the candidate list generation unit 234. The entropy coding unit 110, the entropy decoding unit 202, and the like may be included in other devices.

Further, in the present embodiment, each component can be realized by a dedicated hardware or by executing a software program suitable for each component. Each of the components can be realized by causing a program execution unit such as a CPU or a processor to execute a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

Specifically, each of the encoding device 100 and the decoding device 200 may have a processing circuit (Processing Circuitry) and a memory device (Storage), and the memory device is electrically connected to the processing circuit and can be accessed by the processing circuit. For example, the processing circuit corresponds to circuit 160 or 260, and the memory device is corresponding memory 162 or 262.

The processing circuit includes at least one of a dedicated hardware and a program execution unit that performs processing using a memory device. Further, when the processing circuit includes the program execution unit, the memory device stores the software program executed by the program execution unit.

Here, the software that implements the encoding device 100 or the decoding device 200 of the present embodiment is the following program.

That is, the program is to let the computer execute an encoding method, which is a method of encoding the information of the moving image by performing mapping of the plurality of the aforementioned reference blocks to the encoding according to the motion vector of the reference block. a corresponding region in the target image, wherein the plurality of reference blocks include one or more reference blocks included in the first reference picture and one or more reference blocks included in the second reference picture, and the first reference The picture is a first reference picture list constituting two reference picture lists for bi-prediction, and the second reference picture is a second reference picture list constituting the two reference picture lists; When the encoding target block in the encoding target image is encoded, the candidate motion vector is derived from the motion vector of the reference block that has been mapped to the corresponding region as a plurality of candidates for the prediction motion vector of the encoding target block. One of the motion vectors; selecting the aforementioned prediction motion vector from among the plurality of candidate motion vectors; and using the prediction motion vector to encode the foregoing Like a block of information is encoded.

Alternatively, the program may also cause the computer to perform a decoding method, which is a method of decoding the information of the moving image by: mapping the plurality of the reference blocks to the reference vector according to the motion vector of the reference block a corresponding region in the decoding target image, wherein the plurality of reference blocks include one or more reference blocks included in the first reference picture and one or more reference blocks included in the second reference picture, and the foregoing The first reference picture is a first reference picture list constituting two reference picture lists for bi-prediction, and the second reference picture is a second reference picture list constituting the two reference picture lists; When the region overlaps the decoding target block in the decoding target image, the candidate motion vector is derived from the motion vector of the reference block that has been mapped to the corresponding region as the predicted motion vector with respect to the decoding target block. One of a plurality of candidate motion vectors; selecting the aforementioned predicted motion vector from among the plurality of candidate motion vectors; and using the predicted motion vector, The aforementioned decoding target block to decode the information.

Further, as described above, each component may be a circuit. These circuits may be integrally formed as one circuit or as individual circuits. Further, each component may be implemented by a general-purpose processor or by a dedicated processor.

Further, it is also possible to allow other constituent elements to execute the processing to be executed by the specific constituent elements. Further, the order in which the processing is to be executed may be changed, and a plurality of processing may be executed in parallel. Further, the codec device may include the encoding device 100 and the decoding device 200.

The first and second orthographic numbers used in the description may be appropriately replaced. Further, the constituent numbers may be re-attached to the constituent elements or the like, and may be removed.

Although the aspects of the encoding device 100 and the decoding device 200 have been described above based on the embodiments, the aspects of the encoding device 100 and the decoding device 200 are not limited to the embodiment. Any form that can be implemented by combining the various embodiments of the present invention or the components of different embodiments can be included in the present invention without departing from the scope of the present disclosure. Within the scope of the aspect of the encoding device 100 and the decoding device 200.

(Embodiment 2) In each of the above embodiments, each of the functional blocks can be realized by an MPU, a memory, or the like. Further, the processing performed by each of the function blocks can be realized by executing a software (program) recorded on a recording medium such as a ROM by a program execution unit such as a processor. The software can also be distributed by downloading or the like, or can be recorded on a recording medium such as a semiconductor memory. In addition, it is also possible to implement each functional block through a hardware (dedicated circuit).

Further, the processing described in the respective embodiments may be realized by performing centralized processing using a single device (system), or may be realized by performing distributed processing using a plurality of devices. Moreover, the processor that executes the above program may be singular or plural. That is, the centralized processing may be performed or the dispersion processing may be performed.

The aspects of the present disclosure are not limited to the above embodiments, and various changes can be made, and variations thereof are also included in the scope of the present disclosure.

Further, an application example of the moving image encoding method (image encoding method) or the moving image decoding method (image decoding method) shown in each of the above embodiments and a system using the same will be described. This system is characterized by an image encoding device using an image encoding method, an image decoding device using an image decoding method, and an image encoding and decoding device having both. The other components in the system can be appropriately changed in accordance with the needs of the situation.

[Example of Use] Fig. 35 is a view showing the overall configuration of the content supply system ex100 that realizes the content distribution service. The area of communication service is divided into desired sizes, and base stations ex106, ex107, ex108, ex109, and ex110, which are fixed wireless stations, are provided in each cell.

In the content supply system ex100, various devices such as the computer ex111, the game machine ex112, the camera ex113, the home appliance ex114, and the smart phone ex115 are connected via the Internet service provider ex102 or the communication network ex104 and the base stations ex106 to ex110. Connected to the Internet ex101. The content supply system ex100 can be configured to combine the above-described arbitrary elements. It is also possible to connect the machines directly or indirectly via a telephone line network or a short-range wireless device, without going through the base stations ex106 to ex110 of the fixed wireless station. Further, the streaming server ex103 is connected to each of the devices such as the computer ex111, the game machine ex112, the camera ex113, the home appliance ex114, and the smartphone ex115 via the Internet ex101 or the like. Further, the streaming server ex103 is connected to a terminal or the like in the hot spot in the aircraft ex117 via the satellite ex116.

Alternatively, the base station ex106 to ex110 may be replaced by a wireless access point or a hot spot or the like. Further, the streaming server ex103 may be directly connected to the communication network ex104 without via the Internet ex101 or the Internet service provider ex102, or may be directly connected to the aircraft ex117 without via the satellite ex116.

The camera ex113 is a machine that can perform still image shooting and moving image shooting such as a digital camera. In addition, the smart phone ex115 generally refers to a smart phone, a mobile phone, or a PHS (Personal Handyphone System) that corresponds to 2G, 3G, 3.9G, 4G, and the future is called a 5G mobile communication system.

The home appliance ex118 is a machine included in a refrigerator or a domestic fuel cell thermoelectric symbiosis system.

In the content supply system ex100, the terminal having the photographing function is connected to the streaming server ex103 via the base station ex106 or the like, thereby enabling live broadcast or the like. In the live broadcast, the terminal (computer ex111, game machine ex112, camera ex113, home appliance ex114, smart phone ex115, terminal in the aircraft ex117, etc.) transmits the data obtained as follows to the stream server ex103. The data is obtained by performing the encoding process described in the above embodiments on the still image or the moving image content captured by the user using the terminal device, and the image data obtained by the encoding and the image corresponding to the image. The voice-coded sound data is obtained by multiplexing. That is, each terminal functions as an image coding apparatus which is one aspect of the disclosure.

On the other hand, the streaming server ex103 is to stream-distribute the content material to be transmitted to the client having the request. The client is a computer ex111, a game machine ex112, a camera ex113, a home appliance ex114, a smart phone ex115, or a terminal in the aircraft ex117, which can decode the encoded data. Each device that has received the distributed data decodes the received data and performs reproduction. In other words, each device functions as an image decoding device which is one aspect of the disclosure.

[Distributing Process] Further, the streaming server ex103 may be a plurality of servers or a plurality of computers, and the data may be distributed or recorded to the distributor. For example, the streaming server ex103 can also be implemented by a CDN (Contents Delivery Network), and content distribution can be realized by connecting a network of a plurality of edge servers distributed around the world. In the CDN, a physical proximity edge server is dynamically allocated in response to the client. The content is then cached and distributed by the edge server, which reduces latency. Moreover, when any error occurs or the communication state is changed due to an increase in traffic, etc., a plurality of edge servers may be distributed, or the distribution body may be switched to another edge server, and the network portion where the obstacle has occurred may be performed. It is roundabout to continue distribution, so high-speed and stable distribution is possible.

Further, not only the distribution processing of the distribution itself, but also the encoding processing of the photographed data may be performed at each terminal, or may be performed on the server side or may be performed separately. For example, in the encoding process, the processing loop is generally performed twice. The first cycle detects the complexity of the image in frame or scene unit, or the amount of code. Moreover, in the second cycle, the image quality is maintained, and the process of improving the coding efficiency is performed. For example, the terminal performs the first encoding process, and the server side that has received the content performs the second encoding process, thereby reducing the processing load on each terminal and improving the quality and efficiency of the content. . In this case, if there is a request to be decoded almost in real time, the first encoded data that has been transmitted by the terminal can be received and reproduced in another terminal, so that softer real-time distribution can be achieved.

For another example, the camera ex113 or the like extracts the feature amount from the image, compresses the data about the feature amount, and transmits it as a meta material to the server. The server determines the importance of the target from the feature amount, for example, and switches the quantization accuracy and the like in accordance with the meaning of the image. The feature quantity data is particularly effective for the accuracy and efficiency improvement of the motion vector prediction when recompressing on the server. Further, the terminal device performs simple coding such as VLC (variable length coding), and the server may perform coding with a large processing load such as CABAC (Context Adaptive Binary Arithmetic Coding).

Further, in another example, in a stadium, a shopping mall, a factory, or the like, a plurality of pieces of video data in which almost the same scene is captured via a plurality of terminals may exist. At this time, use a plurality of terminals that have been photographed, and use other terminal devices and servers that are not photographed as needed, such as a GOP (Group of Picture) unit, a picture unit, or a square unit that divides the picture. The encoding process is separately assigned to perform the distributed processing. Thereby, the delay can be reduced, and real-time performance can be achieved.

Moreover, since the plurality of image data are almost the same scene, it is also possible to manage and/or instruct the server to refer to the image data captured by each terminal. Alternatively, the server may receive the encoded data from each terminal, change the reference relationship between the plurality of data, or correct or exchange the image itself to re-encode. Thereby, a stream that improves the quality and efficiency of one piece of data can be generated.

In addition, the server may first perform transcoding to change the encoding mode of the image data, and then distribute the image data. For example, the server can also convert the MPEG-based encoding method into a VP system, or convert H.264 to H.265.

In this way, the encoding process can be performed by a terminal or by one or more servers. In the following, the main body of the processing is a description using a "server" or a "terminal", but some or all of the processing by the server may be performed at the terminal. Part or all of the processing performed by the terminal is performed on the server. Also, regarding these parts, the same applies to the decoding process.

[3D, multi-view] In recent years, different scenes photographed by terminals such as camera ex113 and/or smart phone ex115 that are almost synchronized with each other, or images or images that are shot at different angles in the same scene are integrated. The situation of using it has also increased. The images captured by the respective terminals are integrated based on the relative positional relationship between the separately obtained terminals, or the area where the feature points included in the images match.

The server not only encodes the two-dimensional moving image, but also encodes the still image automatically or at the time specified by the user according to the scene analysis of the moving image, and then transmits the static image to the receiving terminal. Further, when the server can obtain the relative positional relationship between the photographing terminals, not only the two-dimensional moving image but also the three-dimensional shape of the scene can be generated from the images captured from the different scenes from the same scene. In addition, the server can additionally encode three-dimensional data generated by a point cloud or the like, or can recognize or track the result of a person or a target based on using three-dimensional data, and images taken from a plurality of terminals. Selecting, or reconfiguring, to generate an image to be sent to the receiving terminal.

In this way, the user may arbitrarily select each of the images corresponding to each of the photographing terminals to view the scene, and may view the content of the image of the arbitrary viewpoint from the three-dimensional data reconstructed using a plurality of images or images. Further, similar to the image, the sound can be collected from a plurality of different viewing angles, and the server can cooperate with the image to perform multiplexing with the sound and image from a specific viewing angle or space, and then transmit.

In addition, in recent years, content corresponding to the real world and the virtual world, such as Virtual Reality (VR/Virtual Reality) and Augmented Reality (AR/Augmented Reality), has gradually spread. In the case of an VR image, the server can also make viewpoint images for the left eye and the right eye, and allow for reference between the viewpoint images by Multi-View Coding (MVC/Multi-view coding). The encoding may also be encoded as a different stream without referring to each other. When decoding different streams, it is preferable to synchronize and reproduce the virtual three-dimensional space in response to the user's viewpoint.

In the case of the image of the AR, the server superimposes the virtual object information on the virtual space on the camera information in the real space according to the position of the three-dimensional property or the movement of the viewpoint of the user. The decoding device can also acquire or maintain virtual object information and three-dimensional data, and generate a two-dimensional image in response to the movement of the user's viewpoint, and smoothly connect to create overlapping data. Alternatively, the decoding device may also send the movement of the user's viewpoint to the server in addition to the request instruction of the virtual object information, and the server creates overlapping data from the three-dimensional data held by the server in accordance with the movement of the received viewpoint, and The overlapping data is encoded and distributed to the decoding device. In addition, the superimposed data has an alpha value indicating the transmittance in addition to RGB, and the server sets the alpha value of the portion other than the target made from the three-dimensional data to 0 or the like, and encodes the portion in a state of being penetrated. ,also may. Alternatively, the server may generate data as a chroma key, which sets the RGB value of the predetermined value as the background, and sets the portion other than the target as the background color.

Similarly, the decoding process of the distributed data may be performed at each terminal of the client, or may be performed on the server side, or may be shared with each other. For example, when a terminal sends a reception request to the server, the other terminal receives the content requested, performs decoding processing, and transmits the decoded signal to the device having the display. It is possible to reproduce the processing and to select the appropriate content without depending on the performance of the communication terminal itself, thereby reproducing the image quality. Further, in another example, while a large-sized image data is received by a TV or the like, a part of a region such as a square divided by the image is decoded and displayed on the personal terminal of the viewer. By this, the entire image can be shared, and the area of responsibility for itself or the area to be confirmed in more detail can be confirmed by the side.

In the future, in the case of indoor or outdoor use, it is possible to use a distribution system specification such as MPEG-DASH in a situation where a plurality of types of wireless communication such as a short distance, a medium distance, or a long distance can be used. Data, while receiving content seamlessly, is predictable. Thereby, the user can switch in real time not only by the terminal device but also by a decoding device or a display device such as a display provided indoors or outdoors. Further, it is possible to perform decoding while switching the decoded terminal device and the displayed terminal device based on the own location information and the like. Thereby, it is also possible to make it possible to move the map information while displaying the map information on the wall surface of the building or the part of the ground on the side where the displayable element is embedded in the movement to the destination. Moreover, based on the ease of accessing the encoded material on the network, such as the encoded data is cached by the server, the server can be accessed from the receiving terminal in a short time, or copied to the content distribution. It is also possible to switch the bit rate of the received data by an edge server or the like in the Contents Delivery Service.

[Adjustable coding] The switching of the content is explained by the expandable stream shown in Fig. 36, which is a string compressed and encoded by the moving picture coding method shown in each of the above embodiments. flow. The server has a plurality of streams, and the plurality of streams are identical in content but different in quality. The server having the plurality of streams may be any other, but may be configured as follows. The content is switched by using the temporal/space-adjustable stream feature, wherein the temporally/spatially adjustable stream is implemented by layering as shown in the figure. In other words, the decoding side determines which layer to decode based on factors such as the inherent factors of performance and the state of the communication band, so that the decoding side can freely switch between low-resolution content and high-resolution content. And decode it. For example, if you want to put a subsequent part of the image that was viewed on the mobile phone ex115 on the mobile home and watch it on a network TV or the like, the machine only needs to decode the same stream to different layers. Therefore, the burden on the server side can be reduced.

Further, as described above, in addition to the configuration in which the picture is encoded in each layer and the scalability of the enhancement layer exists in the upper layer of the base layer, the enhancement layer may contain an interpretation based on image-based statistical information or the like. Information, the decoding side based on the interpretation information, the base layer of the image is super-image analysis, in order to produce high-quality content. The super-image analysis may be any one of the improvement of the SN ratio and the expansion of the resolution at the same resolution. The interpretation information is information including linear or non-linear filter coefficients used for a specific super image analysis process, or filter values used in a specific super image analysis process, mechanical learning, or parameter values in a least squares operation. Information, etc.

Alternatively, the picture may be divided into squares or the like in accordance with the meaning of an object or the like in the image, and the decoding side selects the block to be decoded, thereby decoding only a part of the area. In addition, the attributes of the target (person, car, ball, etc.) and the position in the image (coordinate position in the same image, etc.) are stored as interpretation information, so that the decoding side can specify the desired information based on the interpretation information. The location of the target to determine the square containing the target. For example, as shown in FIG. 37, the interpretation information is stored using a material storage structure different from the pixel material, such as an SEI message in HEVC. The interpretation information is, for example, displaying the position, size, or color of the main target.

Alternatively, the interpretation information may be stored in a unit composed of a plurality of pictures such as a stream, a sequence, or a random access unit. Thereby, the decoding side can obtain the time when the specific person appears in the image, and the information of the picture unit can be used, so that the picture existing in the specific target and the position of the target in the picture can be specified.

[Optimization of Web Page] FIG. 38 is a diagram showing an example of a display screen of a web page in a computer ex111 or the like. FIG. 39 is a view showing an example of a display screen of a web page of the smart phone ex115 or the like. As shown in FIG. 38 and FIG. 39, the webpage includes a plurality of linked images, wherein the linked images are links to image content, and the manner in which the linked images are viewed depends on the components being viewed. Different. When a plurality of linked images are viewed on the screen, the display device (decoding device) until the user understands that the link image is selected, or until the link image approaches the center of the screen or the entire link image enters the screen. ) is to display a still image or an I picture of each content as a link image, or to display a gif-like image in a plurality of still images or I pictures, or to decode the image only by receiving the base layer and display.

When the linked image is selected by the user, the display device regards the base layer as the highest priority for decoding. Further, when the HTML constituting the web page has information for displaying the content of the adjustable type, the display device can also decode until the enhancement layer. Moreover, in order to ensure real-time performance, before the selection or the communication band is extremely narrow, the display device can only decode and display the picture in front of the reference picture (I picture, P picture, only the B picture in front), thereby reducing the head. The delay between the decoding time of the picture and the display time (delay from the start of decoding of the content to the start of display). Moreover, the display device can also ignore the reference relationship of the picture, so that all the B pictures and the P picture are referred to the front, and the decoding is roughly performed first, and then, after a period of time, the normal picture is added as the received picture increases. decoding.

[Automatic driving] In addition, when a static image or image data such as two-dimensional or three-dimensional map information is transmitted and received for automatic driving or support driving of a car, the receiving terminal has an image of one level or more. In addition to the data, it is also possible to receive weather or construction information as an interpretation of the information and to decode the information accordingly. In addition, the interpretation of information can also belong to the layer, or simply multiplex with the image data.

At this time, since the car, the aerial camera, or the airplane including the receiving terminal moves, the receiving terminal can transmit the location information of the receiving terminal when requesting reception, thereby switching the base stations ex106 to ex110. Seamless reception and decoding on one side. Moreover, the receiving terminal can dynamically switch to which extent the interpretation information is received or to what extent the map information is updated, depending on the user's selection, the state of the user, or the state of the communication band.

As described above, in the content supply system ex100, the client can receive the encoded information transmitted by the user in real time and decode it, and perform reproduction.

[Distribution of Personal Content] In the content supply system ex100, not only the long-term content is transmitted by the high-quality image transmitted by the image distribution company, but also the short-time content can be transmitted through the low image quality performed by the individual. Distribute by unicast or multicast. In addition, the content of the individual is considered to increase in the future. In order to make the personal content into more excellent content, the server can also perform editing processing and then perform encoding processing. This can be realized, for example, as follows.

In the real-time or first-time storage after photography, the server performs recognition processing such as shooting error, scene search, meaning analysis, and target detection from the original image or the encoded data. Then, according to the identification result, the server manually or automatically corrects the out-of-focus or the jitter, or deletes a scene with a low importance such as a scene with a lower brightness than the other pictures or a focal length, or an object that emphasizes the target. Edges, or edits that change tones, etc. The server encodes the edited data based on the edited result. Moreover, it is known that when the shooting time is too long, the viewing rate is lowered, and the server can automatically edit the scenes based on the image processing results, not only for scenes of low importance as described above, but also for scenes with less motion. It becomes content within a specific time range in response to the shadow time. Alternatively, the server may also generate a digest based on the result of the semantic analysis of the scene and encode it.

In addition, in the personal content, if it is the original state, there are cases in which the copyright, the author's personality right, or the portrait right are infringed, and the scope of sharing is more than the intended range, which is not suitable for the individual. Case. Therefore, for example, the server can also intentionally change the face of the peripheral portion of the screen or the home to an unfocused image to perform encoding. Moreover, the server can also recognize whether the face of the person is photographed in the image to be encoded, wherein the character is different from the person who has previously registered, and if it is photographed, the part of the face is decorated with a mosaic or the like. . Alternatively, in the pre-processing or post-processing of the encoding, from the viewpoint of copyright, etc., the user specifies a person or a background area in which the image is to be processed, and the server replaces the designated area with another image or blurs the focus. Processing can also be done. In the case of a character, in the moving image, it is possible to replace a part of the image of the face while tracking the person.

Further, since the viewing of personal content having a small amount of data is highly demanded in real time, although the bandwidth varies depending on the bandwidth, the decoding device first receives the highest priority in the base layer, and then performs decoding and reproduction. The decoding device can also receive the enhancement layer therebetween, and when there are two or more reproductions in the reproduction cycle, the high-quality image is reproduced together with the enhancement layer. If you have done such a stream of scalable coding, you can provide the following experience, that is, when you are not selected or at the beginning of the viewing phase, it is a rough animation, but the stream is gradually refined, and the image is Will get better. In addition to the scalable coding, the same stream can be provided by using a rough stream that is reproduced for the first time and a second stream that is encoded with reference to the first animation as one stream.

[Other use cases] Further, these encoding or decoding processes are generally processed in the LSI ex500 included in each terminal. The LSI ex500 may be a single wafer or a plurality of wafers. In addition, the software for encoding or decoding a moving image can also be loaded into some recording medium (CD-ROM, floppy disk, or hard disk, etc.) readable by a computer ex111, and encoded using the software or Decoding processing. Further, when the smartphone ex115 is attached to a camera, it can also transmit animation data acquired by the camera. The animation data at this time is data that has been encoded by the LSI ex500 which is included in the smart phone ex115.

In addition, the LSIex500 can also be configured to download an application software program. At this time, first, the terminal determines whether the terminal corresponds to the encoding mode of the content, or whether it has the execution capability of the specific service. When the terminal does not correspond to the encoding method of the content, or does not have the execution capability of the specific service, the terminal device downloads the codec or the application software program, and then acquires and reproduces the content.

Further, the content encoding system ex100 via the Internet ex101 is not limited, and at least the moving image encoding device (image encoding device) or the moving image decoding device of the above embodiments may be incorporated in the digital broadcasting system. Any of the decoding devices. Since it is a multiplexed data that has been subjected to multiplex processing of video and audio by radio waves for broadcasting by satellite or the like, and is transmitted and received by the radio wave for broadcasting, it is easy to perform unicast propagation with respect to the content supply system ex100, and digitally. Although the broadcast system is advantageous for multicast transmission, the same application can be applied to the encoding processing and the decoding processing.

[Hardware Configuration] FIG. 40 is a diagram showing the smartphone ex115. Moreover, FIG. 41 is a view showing an example of the configuration of the smartphone ex115. The smart phone ex115 includes an antenna ex450 for transmitting and receiving radio waves with the base station ex110, a camera portion ex465 for capturing images and a still image, and a display portion ex458 for displaying the camera portion ex465. The captured image and the data decoded by the image received by the antenna ex450. The smart phone ex115 further includes an operation unit ex466, which is a touch panel or the like, a sound output unit ex457, which is a speaker for outputting sound or sound, and an audio input unit ex456, which is a microphone for inputting sound, and the like. Ex467, can save the captured image or still image, the recorded sound, the received image or the still image, the encoded data such as mail, or the decoded data; and the slot portion ex464, The interface between SIMex468, in which SIMex468 is used for specific users, from the network to implement access to a variety of materials. Alternatively, an external memory can be used instead of the memory unit ex467.

Further, the main control unit ex460, which is integrally controlled by the display unit ex458 and the operation unit ex466, and the power supply circuit unit ex461, the operation input control unit ex462, the video signal processing unit ex455, the camera interface ex463, and the display control unit ex459 are adjusted. The variable/demodulation unit ex452, the multiplex/separation unit ex453, the audio signal processing unit ex454, the slot unit ex464, and the memory unit ex467 are connected via the bus bar ex470.

When the power supply switch is turned on by the user's operation, the power supply circuit unit ex461 supplies power to each unit from the battery pack, thereby starting the smart phone ex115 to be operable.

The smartphone ex115 performs processing such as call and data communication based on the control of the main control unit ex460 such as a CPU, a ROM, and a RAM. At the time of the call, the audio signal received by the voice input unit ex456 is converted into a digital audio signal by the audio signal processing unit ex454, and the signal is subjected to spectrum diffusion processing in the modulation/demodulation unit ex452, and the transmission/reception unit ex451 The digital analog conversion processing and the frequency conversion processing are performed, and then transmitted via the antenna ex450. Further, the received data is amplified, and the frequency conversion processing and the analog digital conversion processing are performed, and the spectrum de-diffusion processing is performed in the modulation/demodulation unit ex452, and converted into an analog sound signal by the audio signal processing unit ex454, and then the signal is transmitted from the analog signal. The sound output unit ex457 outputs. In the data communication mode, the text, the still image, or the video data is sent to the main control unit ex460 via the operation input control unit ex462 through the operation of the operation unit ex466 or the like of the main unit, and the transmission and reception processing is performed in the same manner. In the data communication mode, when transmitting a video, a still image, or a video or a sound, the video signal processing unit ex455 transmits the video signal stored in the memory unit ex467 or the video signal input from the camera unit ex465 through the respective The moving picture coding method shown in the embodiment performs compression coding, and the coded video data is sent to the multiplex/separation unit ex453. Further, the audio signal processing unit ex454 encodes the audio signal received by the audio input unit ex456 during the imaging of the video or the still image by the camera unit ex465, and sends the encoded audio data to the multiplex/separation unit ex453. . The multiplex/separation unit ex453 multiplexes the encoded video data and the encoded audio data in a predetermined manner, and performs modulation and demodulation (modulation/demodulation circuit unit) ex452 and transmission. The receiving unit ex451 performs the modulation processing and the conversion processing, and transmits it via the antenna ex450.

When receiving an image attached to an e-mail or a chat or an image linked to a web page or the like, in order to decode the multiplexed data received via the antenna ex450, the multiplex/separation unit ex453 performs the multiplexed data. Separating, thereby dividing the multiplexed data into a bit stream of the image data and a bit stream of the sound data, and supplying the encoded image data to the image signal processing unit ex455 via the synchronous bus ex470, and encoding the data The sound data is supplied to the audio signal processing unit ex454. The video signal processing unit ex455 decodes the video signal by the moving picture decoding method corresponding to the moving picture coding method described in each of the above embodiments, and displays the connected unit from the display unit ex458 via the display control unit ex459. An image or still image contained in a dynamic image file. Further, the audio signal processing unit ex454 decodes the audio signal and outputs the sound from the audio output unit ex457. In addition, since real-time streaming has become widespread, depending on the user's situation, the reproduction of sound may also occur in socially inappropriate scenes. For this reason, as an initial value, the sound signal is not reproduced, and only the composition of the image data reproduction is more desirable. It is also possible to reproduce the sound synchronously only when the user performs an operation such as clicking on image data.

Here, the smart phone ex115 has been described as an example. The terminal device can also be considered in the following three installation forms, except for the signal transceiving type terminal having both the encoder and the decoder. A messaging terminal having an encoder and a receiving terminal having only a decoder. Further, in the digital broadcasting system, the multiplexed data for performing multiplex processing on the video data is received or transmitted, but the multiplexed data may be combined with the sound data. The image has associated text data and other multiplex processing, and can also receive or send the image data itself, rather than multiplex data.

Further, the case where the main control unit ex460 including the CPU controls the encoding process or the decoding process will be described, but the case where the terminal has the GPU is also large. Therefore, the configuration may be as follows, that is, the memory that is shared by the CPU and the GPU, or the memory that manages the address to form a common use state, and utilizes the performance of the GPU to integrate the vast area. Dealers together. Thereby, the encoding time can be shortened, real-time performance can be ensured, and low latency can be realized. In particular, it is efficient to use the CPU instead of the GPU to perform motion estimation, deblocking filter, SAO (Sample Adaptive Offset), and conversion and quantization processing in units of pictures and the like. .

Industrial Applicability The present disclosure is applicable to, for example, television receivers, digital video recorders, car navigation, mobile phones, digital cameras, digital video cameras, video conferencing systems, or electronic mirrors.

100‧‧‧ coding device
102‧‧‧ Division
104‧‧‧Subtraction Department
106‧‧‧Transition Department
108‧‧‧Quantity Department
110‧‧‧ Entropy Coding Department
112, 204‧‧‧Anti-Quantization Department
114, 206‧‧‧Anti-conversion department
116, 208‧‧ Addition Department
118, 210‧‧‧ Block memory
120, 212‧‧‧Circuit Filtering Department
122, 214‧‧‧ frame memory
124, 216‧‧ Internal Forecasting Department
126, 218‧ ‧ forecasting department
128, 220‧‧‧Predictive Control Department
132‧‧‧ Picture Type Determination Department
134, 234‧‧‧ Candidate List Generation Department
136, 236‧‧‧ candidate block information memory
160, 260‧‧‧ circuits
162, 262‧‧‧ memory
200‧‧‧ decoding device
202‧‧‧ Entropy Decoding Department
Ex100‧‧‧Content Supply System
Ex101‧‧‧Internet
Ex102‧‧‧Internet Service Provider
Ex103‧‧‧Streaming server
Ex104‧‧‧Communication Network
Ex106 to ex110‧‧‧ base station
Ex111‧‧‧ computer
Ex112‧‧‧game machine
Ex113‧‧‧Camera
Ex114‧‧‧Home appliances
Ex115‧‧‧Smart mobile phone
Ex116‧‧‧ satellite
Ex117‧‧ aircraft
Ex450‧‧‧Antenna
Ex451‧‧‧Send/Receive Department
Ex452‧‧‧Modulation/Demodulation Department
Ex453‧‧‧Multiplex/Separation Department
Ex454‧‧‧Sound Signal Processing Department
Ex455‧‧‧Image Signal Processing Department
Ex456‧‧‧Sound Input Department
Ex457‧‧‧Sound Output Department
Ex458‧‧‧Display Department
Ex459‧‧‧Display Control Department
Ex460‧‧‧Main Control Department
Ex461‧‧‧Power Circuit Department
Ex462‧‧‧Operation Input Control Department
Ex463‧‧‧Camera face
Ex464‧‧‧Slots
Ex465‧‧‧ camera department
Ex466‧‧‧Operation Department
Ex467‧‧‧Memory Department
Ex468‧‧‧SIM
Ex470‧‧‧ busbar

Fig. 1 is a block diagram showing a functional configuration of an encoding apparatus according to a first embodiment. Fig. 2 is a view showing an example of block division in the first embodiment; Figure 3 is a table showing the conversion basis functions corresponding to the respective conversion patterns. Fig. 4A is a view showing an example of the shape of a filter used in ALF. Fig. 4B is a view showing another example of the shape of the filter used in the ALF. Fig. 4C is a view showing another example of the shape of the filter used in the ALF. Figure 5A is a graph showing 67 intra prediction modes in intra prediction. Fig. 5B is a flow chart for explaining an outline of a predicted image correction process by OBMC processing. Fig. 5C is a conceptual diagram for explaining an outline of prediction image correction processing by OBMC processing. Fig. 5D is a diagram showing an example of FRUC. Figure 6 is a diagram for explaining pattern matching (bidirectional matching) between two blocks along a movement trajectory. FIG. 7 is a diagram for explaining picture matching (template matching) between a template in a current picture and a block in a reference picture. Fig. 8 is a view for explaining a model assuming constant-speed linear movement. FIG. 9A is a diagram for explaining the derivation of a motion vector of a sub-block unit, and the motion vector of the sub-block unit is based on a motion vector of a plurality of adjacent blocks. Fig. 9B is a diagram for explaining an outline of a motion vector derivation process in a merge mode. Fig. 9C is a conceptual diagram for explaining an outline of DMVR processing. 9D is a diagram for explaining an outline of a method of generating a predicted image, which is a brightness correction process using LIC processing. Figure 10 is a block diagram showing the functional configuration of a decoding apparatus according to the first embodiment. Figure 11 is a conceptual diagram showing the motion vectors of adjacent blocks and adjacent blocks. Figure 12 is a conceptual diagram showing the motion vectors of co-located blocks and co-located blocks. Figure 13 is a conceptual diagram of the motion vectors of the ATMVP block and the ATMVP block. Fig. 14 is a conceptual diagram showing an example of a relationship between a coding target block, an adjacent block, and a range of a moving object. Figure 15 is a conceptual diagram showing an example of an unsuitable ATMVP block. Fig. 16 is a block diagram showing the functional configuration of the coding apparatus of the first embodiment. Fig. 17 is a block diagram showing the functional configuration of the decoding apparatus of the first embodiment. Figure 18A is a conceptual diagram showing candidate motion vectors derived from the bi-predicted co-located blocks contained in the L0 reference picture list. Figure 18B is a conceptual diagram showing candidate motion vectors derived from the bi-predicted co-located blocks contained in the L1 reference picture list. Figure 19A is a conceptual diagram showing candidate motion vectors derived from a single predicted co-located block contained in the L0 reference picture list. Figure 19B is a conceptual diagram showing candidate motion vectors derived from a single predicted co-located block contained in the L1 reference picture list. FIG. 20A is a conceptual diagram showing a candidate motion vector derived from a bi-predicted co-located block included in the L0 reference picture list with respect to a bi-predicted coding target block of only one direction. FIG. 20B is a conceptual diagram showing a candidate motion vector which is derived from a bi-predicted parity block included in the L1 reference picture list with respect to a bi-predicted coding target block of only one direction. Figure 21A is a conceptual diagram showing candidate motion vectors derived from ATMVP blocks contained in the L0 reference picture list. Figure 21B is a conceptual diagram showing candidate motion vectors derived from ATMVP blocks contained in the L1 reference picture list. Figure 22 is a conceptual diagram showing a plurality of groups with respect to a plurality of candidate motion vectors. Figure 23 is a conceptual diagram showing blocks used to derive spatial candidate motion vectors. Figure 24 is a conceptual diagram showing the matching of the blocks contained in the L0 reference picture list and the blocks contained in the L1 reference picture list. Figure 25 is a conceptual diagram showing a blank area. Fig. 26 is a conceptual diagram showing a repeating area. Fig. 27 is a conceptual diagram showing an example of a coding target block and a corresponding area. Fig. 28 is a conceptual diagram showing an example of a coding target block and two corresponding areas. 29 is a conceptual diagram showing an example of a coding target block, a plurality of adjacent blocks, and a plurality of corresponding regions. Fig. 30 is a conceptual diagram showing a block determined as a coding unit and a sub-block determined in a predetermined size. Figure 31 is a block diagram showing an example of mounting of the encoding device of the first embodiment. Fig. 32A is a flowchart showing a first operation example regarding the derivation of the candidate motion vector at the time of moving picture coding. Fig. 32B is a flowchart showing a second operation example regarding the derivation of the candidate motion vector at the time of moving picture coding. Fig. 32C is a flowchart showing a third operation example regarding the derivation of the candidate motion vector at the time of moving picture coding. Figure 33 is a block diagram showing an example of mounting of the decoding apparatus of the first embodiment. Fig. 34A is a flowchart showing a first operation example regarding the derivation of the candidate motion vector at the time of moving picture decoding. Fig. 34B is a flowchart showing a second operation example regarding the derivation of the candidate motion vector at the time of moving picture decoding. Fig. 34C is a flowchart showing a third operation example regarding the derivation of the candidate motion vector at the time of moving picture decoding. Fig. 35 is a view showing the overall configuration of a content supply system for realizing a content distribution service. Fig. 36 is a view showing an example of a coding structure in the case of adjustable coding. Fig. 37 is a view showing an example of a coding structure in the case of adjustable coding. 38 is a diagram showing an example of a display screen of a web page. 39 is a diagram showing an example of a display screen of a web page. Figure 40 is a diagram showing an example of a smart phone. Fig. 41 is a block diagram showing a configuration example of a smart phone.

Claims (18)

An encoding device for encoding information of a moving image includes: a memory and a circuit for accessing the memory; and the foregoing circuit for accessing the memory is: according to a reference block Transmitting a vector, and mapping the plurality of reference blocks to corresponding regions in the image to be encoded, wherein the plurality of reference blocks include one or more reference blocks included in the first reference picture, and 2 refers to one or more reference blocks included in the picture, wherein the first reference picture is a first reference picture list constituting a list of two reference pictures used for bi-prediction, and the second reference picture constitutes the two a second reference picture list in the reference picture list; when the corresponding corresponding area overlaps the coding target block in the encoding target image, the candidate movement is derived from the motion vector of the reference block already mapped to the corresponding area a vector as one of a plurality of candidate motion vectors for the prediction motion vector of the encoding target block; and selecting the aforementioned pre-selection from among the plurality of candidate motion vectors Motion vector; and, using the prediction motion vector, the information about the coding target block is encoded. The coding apparatus according to claim 1, wherein the circuit is configured to map one or more reference blocks in the first reference picture to the coding target image, and then to one of the second reference pictures. The above reference blocks are respectively mapped in a blank area, wherein the blank area is an area in which one or more reference blocks in the first reference picture are not mapped among the encoding target images. The coding apparatus according to claim 1, wherein the circuit moves from at least one of two or more motion vectors of two or more reference blocks when two or more corresponding regions overlap the coding target block. The vector extracts the candidate motion vector, wherein the two or more corresponding regions are regions in which the two or more reference blocks among the plurality of reference blocks have been mapped. The encoding device of claim 1, wherein the foregoing circuit is in the absence of the aforementioned corresponding region overlapping the encoding target block, and there is a corresponding region overlapping the peripheral region of the encoding target block, The candidate motion vector is derived by mapping a motion vector of a reference block that overlaps the aforementioned corresponding region of the peripheral region. The encoding apparatus according to any one of claims 1 to 4, wherein said circuit is: when said predictive motion vector is selected from said plurality of candidate motion vectors, each of said plurality of candidate motion vectors is evaluated, in said plural Among the candidate motion vectors, the most highly evaluated candidate motion vector is selected as the predicted motion vector, and the degree of fit between the reconstructed images of the two comparison target regions is evaluated when evaluating each of the plurality of candidate motion vectors. The higher the evaluation, the more highly evaluated the candidate motion vector of the evaluation object, wherein the two comparison target regions are two regions different from the encoding target block, and at least one of them is determined according to the candidate motion vector of the evaluation object. 2 areas. The encoding apparatus of any one of claims 1 to 4, wherein the foregoing circuit derives the aforementioned reference to which the scaling value is applied when the foregoing candidate motion vector is derived from a motion vector of a reference block that has been mapped to the corresponding region. The motion vector of the block is used as the aforementioned candidate motion vector, wherein the aforementioned scaling ratio value is: a time difference from the encoding target picture including the foregoing encoding target image to the reference picture including the foregoing reference block, with respect to the slave including the aforementioned reference block The ratio of the reference picture to the time difference of the reference picture of the reference area indicated by the motion vector of the aforementioned reference block. The encoding apparatus according to any one of claims 1 to 4, wherein the encoding target block is a block determined as a coding unit, or is determined by a predetermined size in a block determined as the coding unit. Sub-block. The encoding device according to any one of claims 1 to 4, wherein the foregoing circuit further, for each of the plurality of encoding target blocks in the encoding target image, when the corresponding region overlaps the encoding target block, The motion vector of the reference block that has been mapped to the corresponding corresponding region is derived, and the candidate motion vector for the predicted motion vector of the encoding target block is derived. A decoding device for decoding information of a moving image includes: a memory and a circuit for accessing the memory; the foregoing circuit for accessing the memory is: according to a reference block Transmitting a vector, and mapping the plurality of reference blocks to corresponding regions in the decoding target image, wherein the plurality of reference blocks include one or more reference blocks included in the first reference picture, and 2 refers to one or more reference blocks included in the picture, wherein the first reference picture is a first reference picture list constituting a list of two reference pictures used for bi-prediction, and the second reference picture constitutes the two a second reference picture list in the reference picture list; when the corresponding corresponding area overlaps the decoding target block in the decoding target image, the candidate movement is derived from the motion vector of the reference block that has been mapped to the corresponding area a vector as one of a plurality of candidate motion vectors for the prediction motion vector of the decoding target block; and selecting the foregoing pre-selection among the plurality of candidate motion vectors Motion vectors; and the use of the prediction motion vector, the block of the decoded information decoded. The decoding device of claim 9, wherein the circuit is configured to map one or more reference blocks in the first reference picture to the decoding target image, and then to one of the second reference pictures. The above reference blocks are respectively mapped in a blank area, wherein the blank area is an area in which one or more reference blocks in the first reference picture are not mapped among the decoding target images. The decoding device of claim 9, wherein the circuit is that when two or more corresponding regions in which two or more reference blocks of the plurality of reference blocks have been mapped overlap in the encoding target block, The candidate motion vector is derived by at least one of two or more motion vectors of the two or more reference blocks. The decoding device of claim 9, wherein the foregoing circuit is in the absence of the aforementioned corresponding region overlapping the decoding target block, and there is a corresponding region overlapping the peripheral region of the decoding target block, The candidate motion vector is derived by mapping a motion vector of a reference block that overlaps the aforementioned corresponding region of the peripheral region. The decoding apparatus according to any one of claims 9 to 12, wherein the foregoing circuit is: when each of the plurality of candidate motion vectors is selected from the plurality of candidate motion vectors, evaluating each of the plurality of candidate motion vectors, in the foregoing plural Among the candidate motion vectors, the most highly evaluated candidate motion vector is selected as the predicted motion vector, and the degree of fit between the reconstructed images of the two comparison target regions is evaluated when evaluating each of the plurality of candidate motion vectors. The higher the evaluation, the more highly evaluated the candidate motion vector of the evaluation object, wherein the two comparison target regions are two regions different from the decoding target block, and at least one of them is determined according to the candidate motion vector of the evaluation object. 2 areas. The decoding device of any one of claims 9 to 12, wherein the foregoing circuit derives the aforementioned reference to which the scaling value is applied when the foregoing candidate motion vector is derived from a motion vector of a reference block that has been mapped to the corresponding region. a motion vector of the block as the foregoing candidate motion vector, wherein the aforementioned scaling ratio is a time difference from the encoding target picture including the foregoing encoding target image to the reference picture including the foregoing reference block, relative to the slave reference block including the foregoing reference block The ratio of the reference picture to the time difference of the reference picture of the reference area indicated by the motion vector of the aforementioned reference block. The decoding device according to any one of claims 9 to 12, wherein the decoding target block is a block determined as a decoding unit, or is determined in a block determined as the decoding unit. The sub-block determined by the size. The decoding device according to any one of claims 9 to 12, wherein the foregoing circuit further, for each of the plurality of decoding target blocks in the decoding target image, when the corresponding region overlaps the decoding target block, The motion vector of the reference block that has been mapped to the corresponding region is derived, and the candidate motion vector with respect to the prediction motion vector of the decoding target block is derived. An encoding method is a method for encoding information of a moving image, performing: mapping, according to a motion vector of a reference block, a plurality of the reference blocks to corresponding regions in the image to be encoded, wherein the foregoing plurality The reference blocks include one or more reference blocks included in the first reference picture and one or more reference blocks included in the second reference picture, and the first reference picture is composed of two for prediction. a reference picture list in the reference picture list, wherein the second reference picture is a second reference picture list constituting the two reference picture lists; and when the corresponding area overlaps the coding target block in the coding target image Deriving a candidate motion vector from a motion vector of a reference picture that has been mapped to the corresponding region as one of a plurality of candidate motion vectors for the prediction motion vector of the encoding target block; from the plurality of candidate motion vectors The foregoing predicted motion vector is selected; and the information of the foregoing encoding target block is encoded using the aforementioned predicted motion vector. A decoding method is a method for decoding information of a moving image, and performing: mapping, according to a motion vector of a reference block, a plurality of the reference blocks to corresponding regions in a decoding target image, where the foregoing plurality The reference blocks include one or more reference blocks included in the first reference picture and one or more reference blocks included in the second reference picture, and the first reference picture is composed of two for prediction. Referring to a first reference picture list in the picture list, the second reference picture is a second reference picture list constituting the two reference picture lists; and when the corresponding area overlaps the decoding target block in the decoding target picture Deriving a candidate motion vector from a motion vector of a reference picture that has been mapped to the corresponding region as one of a plurality of candidate motion vectors for the prediction motion vector of the decoding target block; from the plurality of candidate motion vectors The foregoing predicted motion vector is selected; and the information of the decoding target block is decoded using the aforementioned predicted motion vector.