TW201834456A - Image coding apparatus, image decoding apparatus, and method - Google Patents

Abstract

An encoding device (100), when extracting at least one predicted motion vector candidate for a block to be encoded from a plurality of candidate motion vectors, encodes mode information for identifying an extraction method, selects an extraction method identified by the mode information for the block to be encoded from among a first extraction method and a second extraction method, and extracts the at least one predicted motion vector candidate in accordance with the selected extraction method. The first extraction method is an extraction method based on evaluation results for each of the plurality of candidate motion vectors obtained using a reconstructed image of an encoded region in a moving image without using the image region of the block to be encoded, and the second extraction method is an extraction method based on a predetermined priority for the plurality of candidate motion vectors.

Description

Encoding device, decoding device, encoding method and decoding method

FIELD OF THE INVENTION The present disclosure relates to an encoding apparatus or the like that encodes a moving image composed of a plurality of pictures.

Background of the Invention Conventionally, H.265 exists as a specification for encoding a moving image. H.265 is also known as HEVC (High Efficiency Video Coding). Prior technical literature

Non-patent literature Non-Patent Document 1: H.265 (ISO/IEC 23008-2 HEVC (High Efficiency Video Coding))

SUMMARY OF THE INVENTION Problem to be Solved by the Invention However, on the other hand, in order to further improve coding efficiency, there is a problem that the processing load is increased due to an increase in coding efficiency.

Accordingly, the present disclosure provides an encoding apparatus or the like which has an possibility of suppressing an increase in processing load and which can improve an encoding efficiency. Means to solve the problem

An encoding device according to an aspect of the present disclosure is an encoding device that encodes a moving image, and includes a processing circuit and a memory connected to the processing circuit, wherein the processing circuit uses the memory and is based on the dynamic Obtaining, by the motion vector of each of the plurality of coding completion blocks corresponding to the coding target block in the image, to obtain a plurality of candidate motion vectors, and extracting at least the foregoing coding target block from the plurality of candidate motion vectors a motion vector predictor candidate, and deriving a motion vector of the encoding target block with reference to a reference picture included in the moving image, and predicting a motion vector among the at least one motion vector predictor candidate that has been captured Encoding the difference between the sub- and the derived motion vector of the coding target block, and dynamically using the derived motion vector of the coding target block to dynamically compensate the coding target block, and predicting at least one motion vector In the extraction of the sub-candidate, the mode information for identifying the extraction method is encoded, and the first extraction side is obtained. In the method and the second extraction method, for the coding target block, a capture method identified by the mode information is selected, and the at least one motion vector prediction is captured according to the selected extraction method. a sub-candidate, the first extraction method is a method for extracting an evaluation result according to each of the plurality of candidate motion vectors, wherein the plurality of candidate motion vectors do not use the image region of the encoding target block, but are used a candidate motion vector of a reconstructed image of the coded completion region in the moving image, wherein the second capture method is a capture method according to a priority order, and the priority order is a predetermined priority for the plurality of candidate motion vectors order.

Furthermore, these comprehensive or specific aspects may be implemented by a system, device, method, integrated circuit, computer program, or non-transitory recording medium such as a computer-readable CD-ROM, or by means of a non-transitory recording medium. Any combination of systems, devices, methods, integrated circuits, computer programs, and recording media is implemented. Effect of the invention

An encoding apparatus or the like which is one aspect of the present disclosure can suppress an increase in processing load and can improve encoding efficiency.

MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments will be specifically described with reference to the drawings.

Furthermore, the embodiments described below are embodiments that exhibit comprehensive or specific examples. Numerical values, shapes, materials, constituent elements, arrangement positions and connection forms of the constituent elements, procedures, procedures, and the like, which are shown in the following embodiments, are merely examples, and are not intended to limit the scope of the claims. Further, among the constituent elements of the following embodiments, the constituent elements that are not described in the independent request items indicating the highest-level concept are described as arbitrary constituent elements. (Embodiment 1)

First, an outline of the first embodiment will be described as an example of an encoding device and a decoding device to which the processing and/or configuration described in each aspect of the present disclosure to be described later can be applied. However, the first embodiment is merely an example of an encoding device and a decoding device to which the processing and/or configuration described in each aspect of the present disclosure can be applied, and the processing and/or configuration described in each aspect of the present disclosure may be used. It is implemented in an encoding apparatus and a decoding apparatus different from the first embodiment.

In the case where the processing and/or configuration described in each aspect of the present disclosure is applied to the first embodiment, any of the following may be performed. (1) In the encoding device or the decoding device according to the first embodiment, among the plurality of constituent elements constituting the encoding device or the decoding device, the constituent elements corresponding to the constituent elements described in the respective aspects of the present disclosure are replaced with The constituent elements described in the various aspects disclosed. (2) The encoding device or the decoding device according to the first embodiment can arbitrarily add, replace, or delete a function or a process to be applied to a part of a plurality of constituent elements constituting the encoding device or the decoding device. After the change, the constituent elements corresponding to the constituent elements described in the various aspects of the present disclosure are replaced with the constituent elements described in the various aspects of the present disclosure. (3) The method implemented by the encoding device or the decoding device according to the first embodiment may be performed by adding or deleting a part of the plurality of processes included in the method, or adding or deleting any of the plurality of processes included in the method. After the change, the processing corresponding to the processing described in each aspect of the present disclosure is replaced with the processing explained in each aspect of the disclosure. (4) The constituent elements of a part of the plurality of constituent elements constituting the encoding apparatus or the decoding apparatus of the first embodiment can be combined with the following constituent elements: the constituent elements described in the respective aspects of the present disclosure, and There are constituent elements of a part of the functions of the constituent elements described in the aspects of the present disclosure, and constituent elements of a part of the processing performed by the constituent elements described in the various aspects of the present disclosure. (5) A component including a part of the functions of a part of a plurality of constituent elements constituting the encoding device or the decoding device of the first embodiment, or an encoding device or a decoding device according to the first embodiment The constituent elements of the processing performed by the constituent elements of a part of the plurality of constituent elements are implemented in combination with the following constituent elements: constituent elements described in the aspects of the present disclosure, and various aspects of the present disclosure are provided. The constituent elements of one of the functions of the constituent elements described in the example, or the constituent elements of the processing performed by the constituent elements described in the various aspects of the present disclosure. (6) With respect to the method implemented by the encoding device or the decoding device according to the first embodiment, the processing corresponding to the processing described in each aspect of the present disclosure among the plurality of processing included in the method can be replaced with The processing illustrated in the various aspects of the present disclosure. (7) The processing of one of the plurality of processes included in the method performed by the encoding device or the decoding device according to the first embodiment can be implemented in combination with the processing described in each aspect of the present disclosure.

Furthermore, the manner in which the processes and/or configurations described in the various aspects of the present disclosure are implemented is not limited to the above examples. For example, it may be implemented in a device that is used for different purposes from the moving image/image encoding device or the moving image/image decoding device disclosed in the first embodiment, or may be separately implemented in various aspects. The processing and/or composition of the description. Further, the processes and/or configurations described in the different aspects may be combined and implemented. [Summary of coding device]

First, an outline of an encoding apparatus according to the first embodiment will be described. Fig. 1 is a block diagram showing the functional configuration of an encoding apparatus 100 according to the first embodiment. The encoding device 100 is a moving image/image encoding device that encodes moving images/images in block units.

As shown in FIG. 1, the encoding apparatus 100 is an apparatus for encoding an image in units of blocks, and includes a dividing unit 102, a subtracting unit 104, a converting unit 106, a quantization unit 108, an entropy encoding unit 110, and an inverse quantization unit 112. The inverse conversion unit 114, the addition unit 116, the block memory 118, the loop filter unit 120, the frame memory 122, the in-frame prediction unit 124, the inter-frame prediction unit 126, and the prediction control unit 128.

The encoding device 100 can be implemented by, for example, a general purpose processor and a memory. In this case, when the processor executes the software program stored in the memory, the processor functions as the division unit 102, the subtraction unit 104, the conversion unit 106, the quantization unit 108, the entropy coding unit 110, the inverse quantization unit 112, and the inverse. The conversion unit 114, the addition unit 116, the loop filter unit 120, the in-frame prediction unit 124, the inter-frame prediction unit 126, and the prediction control unit 128 function. Further, the encoding device 100 may correspond to the division unit 102, the subtraction unit 104, the conversion unit 106, the quantization unit 108, the entropy coding unit 110, the inverse quantization unit 112, the inverse conversion unit 114, the addition unit 116, and the loop filter unit 120. One or more dedicated electronic circuits of the in-frame prediction unit 124, the inter-frame prediction unit 126, and the prediction control unit 128 are realized.

Hereinafter, each constituent element included in the encoding device 100 will be described. [Division Department]

The division unit 102 divides each picture included in the input moving image into a plurality of blocks, and outputs each block to the subtraction unit 104. For example, the segmentation section 102 first divides the picture into blocks of a fixed size (for example, 128x128). This fixed size block is called a coding tree unit (CTU). Moreover, the segmentation unit 102 divides each fixed size block into a variable size (for example, 64×64 or less) according to the recursive quadtree and/or binary tree segmentation. Piece. This variable size block is sometimes referred to as a coding unit (CU), a prediction unit (PU), or a conversion unit (TU). Furthermore, in the present embodiment, it is not necessary to distinguish between the CU, the PU, and the TU, and some or all of the blocks in the picture are processed by the CU, the PU, and the TU.

Fig. 2 is a view showing an example of block division in the first embodiment. In FIG. 2, the solid line indicates the block boundary divided by the quaternary tree block, and the broken line indicates the block boundary divided by the binary tree block.

Here, block 10 is a 128x128 pixel square block (128x128 block). This 128x128 block 10 is first partitioned into 4 square 64x64 blocks (quaternary tree block partitioning).

The upper left 64x64 block is further vertically divided into two rectangular 32x64 blocks, and the left 32x64 block is further vertically divided into two rectangular 16x64 blocks (binary tree block partitioning). As a result, the upper left 64x64 block is divided into two 16x64 blocks 11, 12, and 32x64 blocks 13.

The upper right 64x64 block is horizontally divided into two rectangular 64x32 blocks 14, 15 (binary tree block partitioning).

The lower left 64x64 block is divided into 4 square 32x32 blocks (quaternary tree block partitioning). Among the four 32x32 blocks, the upper left block and the lower right block are further divided. The upper left 32x32 block is vertically divided into two rectangular 16x32 blocks, and the right 16x32 block is further horizontally divided into two 16x16 blocks (binary tree block partitioning). The lower right 32x32 block is horizontally divided into two 32x16 blocks (binary tree block partitioning). As a result, the lower left 64x64 block can be divided into: 16x32 block 16, two 16x16 blocks 17, 18, two 32x32 blocks 19, 20, and two 32x16 blocks 21, 22.

The lower right 64x64 block 23 is not split.

As above, in FIG. 2, the block 10 is divided into 13 variable-sized blocks 11 to 23 according to the recursive quaternary tree and binary tree block division. This segmentation is called QTBT (quad tree plus binary tree) segmentation.

Furthermore, in FIG. 2, although one block is divided into four or two blocks (quaternary tree or binary tree block division), the division is not limited thereto. For example, one block may be divided into three blocks (three-dimensional tree block division). This segmentation, which includes ternary tree block partitioning, is called MBT (multi-type tree) segmentation. [Subtraction Department]

The subtraction unit 104 subtracts the prediction signal (predicted sample) from the original signal (original sample) in the block unit divided by the division unit 102. In other words, the subtraction unit 104 calculates a prediction error (which may also be referred to as a residual) of the coding target block (hereinafter referred to as the current block). Further, the subtraction unit 104 outputs the calculated prediction error to the conversion unit 106.

The original signal is an input signal of the encoding device 100, and is a signal (for example, a luma signal and two chroma signals) indicating images of the respective pictures constituting the moving image. In the following, the signal representing the image is sometimes referred to as a sample. [conversion department]

The conversion unit 106 converts the prediction error of the spatial region into a conversion coefficient of the frequency region, and outputs the conversion coefficient to the quantization unit 108. Specifically, the conversion unit 106 is, for example, performing predetermined discrete cosine transform (DCT) or discrete sine transform (DST) on the prediction error of the spatial region.

Furthermore, the conversion unit 106 may also adaptively select a conversion type from among a plurality of conversion types, and convert a prediction error into a conversion coefficient using a transform basis function corresponding to the selected conversion type. . This conversion is sometimes referred to as EMT (explicit multiple core transform) or AMT (adaptive multiple transform).

A plurality of conversion types include, for example, DCT-II, DCT-V, DCT-VIII, DST-I, and DST-VII. Figure 3 is a table showing a conversion basis function corresponding to each conversion type. In Fig. 3, N is the number of input pixels. The selection of the conversion types from among the plurality of conversion types may be determined according to, for example, the type of prediction (intra-prediction and inter-prediction), or may be based on the intra-frame prediction mode. set.

This display shows whether EMT or AMT information (which can be called, for example, an AMT flag) and information showing the selected conversion type can be signaled at the CU level. Moreover, the signalization of the information does not need to be limited to the CU level, but may be other levels (for example, a sequence level, a picture level, a slice level, and a tile level) (eg, a sequence level, a picture level, a slice level, and a tile level). Tile level) or CTU level).

Further, the conversion unit 106 may reconvert the conversion coefficient (conversion result). This retransformation is sometimes referred to as AST (adaptive secondary transform) or NSST (non-separable secondary transform). For example, the conversion unit 106 performs retransformation for each sub-block (for example, a 4×4 sub-block) included in the block corresponding to the conversion coefficient of the intra-frame prediction error. The information showing whether NSST is applicable and the information related to the conversion matrix used by NSST is signalized at the CU level. Moreover, the signalization of the information is not limited to the CU level, but may be other levels (for example, sequence level, picture level, fragment level, tile level or CTU level).

Here, the separable conversion refers to a method of separating and performing a plurality of conversions in various directions in a number corresponding to the input dimension, and the non-separable conversion means that when the input is multi-dimensional, A method in which two or more dimensions are combined and regarded as one dimension, and conversion is performed at a time.

For example, as an example of the inseparable conversion, a conversion is exemplified: in the case where a block of 4×4 is input, it is regarded as an arrangement having 16 elements, and the arrangement is performed by a conversion matrix of 16×16. Conversion processing.

Also, similarly, after considering the 4x4 input block as an arrangement having 16 elements, the arrangement is subjected to a plurality of conversions such as Givens rotation (hypercube Givens conversion, Hypercube Givens Transform) ) is also an example of an inseparable transformation. [Quantization Department]

The quantization unit 108 quantizes the conversion coefficients output from the conversion unit 106. Specifically, the quantization unit 108 scans the conversion coefficient of the current block in a predetermined scanning order, and quantizes the conversion coefficient according to a quantization parameter (QP) corresponding to the scanned conversion coefficient. Further, the quantization unit 108 outputs the quantized conversion coefficients (hereinafter referred to as quantized coefficients) of the current block to the entropy coding unit 110 and the inverse quantization unit 112.

The prescribed order is the order of quantization/inverse quantization for the conversion coefficients. For example, the prescribed scanning order is defined in the order of increasing frequency (in order of low frequency to high frequency) or descending order (in order of high frequency to low frequency).

The so-called quantization parameter is a parameter that defines the quantization step size (quantization width). For example, if the value of the quantization parameter is increased, the quantization step size will also increase. That is to say, if the value of the quantization parameter is increased, the quantization error is increased. [Entropy coding unit]

The entropy coding unit 110 generates a coded signal (code bit stream) by performing variable length coding on the quantized coefficients input from the quantization unit 108. Specifically, the entropy coding unit 110 binarizes the quantized coefficients, for example, and arithmetically encodes the binary signals. [Inverse Quantization Department]

The inverse quantization unit 112 inversely quantizes the quantized coefficients which are inputs from the quantization unit 108. Specifically, the inverse quantization unit 112 inversely quantizes the quantized coefficients of the current block in a predetermined scanning order. Further, the inverse quantization unit 112 outputs the inversely quantized conversion coefficient of the current block to the inverse conversion unit 114. [Inverse Conversion Department]

The inverse conversion unit 114 inversely converts the conversion coefficient, which is an input from the inverse quantization unit 112, to restore the prediction error. Specifically, the inverse transform unit 114 restores the prediction error of the current block by performing inverse conversion corresponding to the conversion performed by the conversion unit 106 on the conversion coefficient. Further, the inverse conversion unit 114 outputs the restored prediction error to the addition unit 116.

Furthermore, since the predicted prediction error is lost due to quantization, the prediction error calculated by the subtraction unit 104 does not match. That is, the predicted error of the restoration includes a quantization error. [Addition Department]

The addition unit 116 adds the prediction error, which is an input from the inverse conversion unit 114, and the prediction sample, which is an input from the prediction control unit 128, to the current block. Further, the addition unit 116 outputs the reconstructed block to the block memory 118 and the loop filter unit 120. The reconstructed block is sometimes referred to as a local decoding block. [block memory]

The block memory 118 is a storage unit for storing a block referred to in the in-frame prediction and also a block in the encoding target picture (hereinafter referred to as a current picture). Specifically, the tile memory 118 stores the reconstructed block output from the addition unit 116. [loop filter unit]

The loop filter unit 120 performs loop filtering on the block reconstructed by the adder 116, and outputs the filtered reconstructed block to the frame memory 122. The loop filter is a filter (in-loop filter) used in the coding loop, and includes, for example, a Deblocking Filter (DF), a sample adaptive offset (Sample). Adaptive Offset (SAO) and Adaptive Loop Filter (ALF).

In ALF, the least square error filter for removing coding distortion can be applied. For example, it can be applied: according to each of the 2x2 sub-blocks in the current block, according to the direction and activity of the local gradient. One filter selected from a plurality of filters.

Specifically, first, sub-blocks (eg, 2x2 sub-blocks) may be classified into a plurality of classes (eg, 15 or 25 categories). The classification of sub-blocks is based on the direction and activity of the gradient. For example, the classification value C (for example, C=5D+A) can be calculated by using the gradient direction value D (for example, 0 to 2 or 0 to 4) and the gradient activity value A (for example, 0 to 4). Moreover, the sub-blocks are classified into a plurality of categories (for example, 15 or 25 categories) according to the classification value C.

The direction value D of the gradient can be derived, for example, by comparing gradients in a plurality of directions (eg, horizontal, vertical, and 2 diagonal directions). Further, the gradient activity value A is derived by, for example, adding a gradient in a plurality of directions and quantizing the addition result.

According to the result of this classification, the filter for the sub-block can be determined from among a plurality of filters.

As the shape of the filter used in the ALF, for example, a circularly symmetrical shape can be utilized. 4A to 4C are diagrams showing a plurality of examples of shapes of filters used in ALF. 4A shows a 5x5 diamond shape filter, FIG. 4B shows a 7x7 diamond shape filter, and FIG. 4C shows a 9x9 diamond shape filter. The information showing the shape of the filter is signalized at the picture level. Furthermore, the signalization of the information of the shape of the display filter need not be limited to the picture level, and may be other levels (for example, sequence level, slice level, tile level, CTU level or CU level).

The on/off of the ALF is determined at, for example, the picture level or the CU level. For example, whether the ALF is applied to the brightness at the CU level or not is determined at the picture level for the color difference. The information showing the ALF on/off can be signaled at the picture level or the CU level. Furthermore, the signalization of the information indicating the on/off of the ALF is not limited to the picture level or the CU level, and may be other levels (for example, a sequence level, a slice level, a tile level, or a CTU level).

A set of coefficients of a selectable plurality of filters (e.g., filters up to 15 or 25) is signaled at the picture level. Furthermore, the signalization of the coefficient set does not need to be limited to the picture level, but may be other levels (eg, sequence level, slice level, tile level, CTU level, CU level, or sub-block level). [frame memory]

The frame memory 122 is a storage unit for storing a reference picture used for inter-frame prediction, and is sometimes referred to as a frame buffer. Specifically, the frame memory 122 stores the reconstructed block that has been filtered by the loop filter unit 120. [In-frame prediction department]

The in-frame prediction unit 124 performs intra-frame prediction (also referred to as intra-picture prediction) of the current block by referring to the block stored in the current picture of the block memory 118, thereby generating a prediction signal (in-frame prediction signal) ). Specifically, the in-frame prediction unit 124 performs intra-frame prediction by referring to samples (for example, luminance values and color difference values) of the blocks adjacent to the current block, thereby generating an in-frame prediction signal and predicting the in-frame prediction. The signal is output to the prediction control unit 128.

For example, the in-frame prediction unit 124 performs intra-frame prediction using one of a plurality of predetermined intra-frame prediction modes. The plurality of in-frame prediction modes include one or more non-directional prediction modes and a plurality of directional prediction modes.

One or more non-directional prediction modes include, for example, a Planar prediction mode and a DC prediction mode defined by the H.265/HEVC (High-Efficiency Video Coding) specification (Non-Patent Document 1).

The plurality of directional prediction modes include, for example, prediction modes of 33 directions specified by the H.265/HEVC specification. Furthermore, the plurality of directional prediction modes may further include prediction modes of 32 directions (total of 65 directional prediction modes) in addition to 33 directions. FIG. 5 is a diagram showing 67 in-frame prediction modes (two non-directional prediction modes and 65 directional prediction modes) in the in-frame prediction. The solid arrows indicate 33 directions defined by the H.265/HEVC specifications, and the dotted arrows indicate 32 additional directions.

Furthermore, in the in-frame prediction of the color difference block, the luminance block can also be referred to. That is to say, the color difference component of the current block can also be predicted based on the luminance component of the current block. Such intra-frame predictions can sometimes be referred to as CCLM (cross-component linear model) prediction. Such an intra-frame prediction mode (for example, referred to as CCLM mode) of the color difference block of the reference luminance block may also be added as an intra-frame prediction mode of one color difference block.

The in-frame prediction unit 124 may also correct the intra-frame predicted pixel value based on the gradient of the reference pixels in the horizontal/vertical direction. This intra-frame prediction of companion correction is sometimes referred to as PDPC (position dependent intra prediction combination). Information showing whether or not the PDPC is applicable (referred to as a PDPC flag, for example) is signalized at, for example, the CU level. Moreover, the signalization of this information does not need to be limited to the CU level, but may be other levels (such as sequence level, picture level, fragment level, tile level, or CTU level). [Inter-frame prediction unit]

The inter-frame prediction unit 126 refers to the reference picture stored in the frame memory 122 and is also a reference picture different from the current picture, and performs inter-frame prediction (also referred to as inter-picture prediction) of the current block, thereby generating a prediction signal ( Inter-frame prediction signal). Inter-frame prediction is performed in units of current blocks or sub-blocks within the current block (eg, 4x4 blocks). For example, the inter-frame prediction unit 126 performs motion search (motion estimation) in the reference picture for the current block or sub-block. Further, the inter-frame prediction unit 126 performs motion compensation using motion information (for example, motion vector) obtained by motion search, thereby generating an inter-frame prediction signal of the current block or sub-block. Further, the inter-frame prediction unit 126 outputs the generated inter-frame prediction signal to the prediction control unit 128.

Motion information used for dynamic compensation is signaled. In the signalization of motion vectors, a motion vector predictor can also be used. That is to say, the difference between the motion vector and the motion vector predictor can also be signaled.

Moreover, not only the motion information of the current block obtained by the motion search, but also the motion information of the adjacent block can also be utilized to generate the inter-frame prediction signal. Specifically, the prediction signals of the motion information obtained according to the motion search and the prediction signals according to the motion information of the adjacent blocks may be weighted and added, and the sub-block units in the current block are used. Generate inter-frame prediction signals. This inter-frame prediction (dynamic compensation) is sometimes referred to as OBMC (overlapped block motion compensation).

In this OBMC mode, information showing the size of the sub-blocks used by the OBMC (for example, referred to as OBMC block size) is signalized at the sequence level. Also, information indicating whether or not the OBMC mode is applied (for example, referred to as an OBMC flag) is signalized at the CU level. Moreover, the signalization level of the information does not need to be limited to the sequence level and the CU level, and may be other levels (such as picture level, slice level, tile level, CTU level or sub-block level).

Furthermore, the motion information may not be signaled but may be derived on the decoding device side. For example, a merge mode specified by the H.265/HEVC specification may also be used. Further, the motion information can also be derived by performing motion search on the decoding device side, for example. In this case, motion search can be performed without using the pixel values of the current block.

Here, a mode in which motion search is performed on the decoding device side will be described. The mode in which the motion search is performed on the decoding device side is sometimes referred to as a PMMVD (pattern matched motion vector derivation) mode or a FRUC (frame rate up-conversion) mode.

First, a list of a plurality of candidates each having a motion vector predictor may be generated by referring to a motion vector of a coded block that is spatially or temporally adjacent to the current block (common with the merge list). Then, the evaluation values of the candidates included in the candidate list are calculated, and one candidate is selected based on the evaluation values.

Moreover, the motion vector for the current block can be derived from the selected candidate motion vector. Specifically, for example, the selected candidate motion vector is derived as it is as the motion vector for the current block. Further, for example, in the peripheral region of the position in the reference picture corresponding to the selected candidate motion vector, the motion vector for the current block is derived by performing pattern matching.

Furthermore, the evaluation value is calculated by matching the region in the reference picture corresponding to the motion vector with the pattern between the predetermined regions.

As a pattern match, a Type 1 match or a Type 2 match can be used. The first type matching and the second type matching are sometimes referred to as bidirectional matching and template matching, respectively.

In the first type matching, pattern matching is performed between two blocks in two different reference pictures and also between two blocks of the motion trajectory of the current block. Therefore, in the first pattern matching, the predetermined region for calculating the candidate evaluation value is the region in the other reference picture along the motion trajectory of the current block.

Fig. 6 is a diagram for explaining pattern matching (bidirectional matching) between two blocks along a motion trajectory. As shown in FIG. 6, in the first type matching, there are two blocks in the motion trajectory along the current block (Cur block) and two in the two different reference pictures (Ref0, Ref1). In the pairing of the blocks, the most matching pair is searched, thereby deriving two motion vectors (MV0, MV1).

Under the assumption of continuous motion trajectory, it is pointed out that the motion vectors (MV0, MV1) of the two reference blocks are relative to the temporal distance between the current picture (Cur Pic) and the two reference pictures (Ref0, Ref1). (TD0, TD1) is proportional. For example, if the current picture is located between two reference pictures in time, and the distance from the current picture to the two reference pictures is equal, in the first type matching, the mirror-symmetrical two-way is derived. Motion vector.

In the second type matching, it is a template in the current picture (a block adjacent to the current block in the current picture (for example, upper and/or left adjacent blocks)) and a block in the reference picture. Pattern matching between them. Therefore, in the second type matching, a predetermined area for calculating the candidate evaluation value is a block adjacent to the current block in the current picture.

FIG. 7 is a diagram for explaining pattern matching (template matching) between a template in a current picture and a block in a reference picture. As shown in FIG. 7, in the second type matching, by searching within the reference picture (Ref0) and matching the block adjacent to the current block (Cur block) in the current picture (Cur Pic), Block to derive the motion vector of the current block.

This kind of display is applicable to the information of the FRUC mode (for example, it can be called FRUC flag), which is signalized at the CU level. Also, in the case where the FRUC mode is applied (for example, when the FRUC flag is true), the information of the pattern matching method (the first type matching or the second type matching) is displayed (for example, it may be called the FRUC mode flag). The standard is signalized at the CU level. Moreover, the signalization of the information does not need to be limited to the CU level, and may be other levels (for example, sequence level, picture level, slice level, tile level, CTU level or sub-block level).

Furthermore, the motion information can also be derived on the decoding device side by a different method from the motion search. For example, the correction amount of the motion vector may be calculated using the peripheral pixel values in units of pixels based on a model that assumes a constant-speed linear motion.

Here, a mode in which a motion vector is derived based on a model in which a constant-speed linear motion is assumed will be described. This mode is sometimes referred to as BIO (bi-directional optical flow) mode.

Fig. 8 is a view for explaining a model assuming constant-speed linear motion. In Fig. 8, (v _x , v _y ) represents a velocity vector, and τ ₀ , τ ₁ respectively represent the temporal distance between the current picture (Cur Pic) and two reference pictures (Ref ₀ , Ref ₁ ). (MVx ₀ , MVy ₀ ) represents a motion vector corresponding to the reference picture Ref ₀ , and (MVx ₁ , MVy ₁ ) represents a motion vector corresponding to the reference picture Ref ₁ .

At this time, under the assumption of the constant velocity linear motion of the velocity vector (v _x , v _y ), (MVx ₀ , MVy ₀ ) and (MVx ₁ , MVy ₁ ) are expressed as (v _x τ ₀ , v _y τ _{0 , respectively).} And (-v _x τ ₁ , -v _y τ ₁ ), and the following optical flow equation (1) holds. [Math 1]

Here, I ^(k) represents the luminance value of the reference image k (k = 0, 1) after the motion compensation. The optical flow equation is expressed by the sum of (i), (ii), and (iii) below being equal to zero: (i) time differential of the luminance value, (ii) velocity in the horizontal direction, and spatial gradient of the reference image. The product of the product of the horizontal component and ( ) the velocity in the vertical direction and the vertical component of the spatial gradient of the reference image. According to the combination of the optical flow equation and the Hermite interpolation formula, the motion vector of the block unit obtained from the merge list or the like can be corrected in pixel units.

Furthermore, the motion vector can also be derived on the decoding device side by a different method from the derivation of the motion vector of the model which assumes a constant velocity linear motion. For example, the motion vector may also be derived in sub-block units based on motion vectors of a plurality of adjacent blocks.

Here, a mode in which a motion vector is derived in units of sub-blocks based on motion vectors of a plurality of adjacent blocks will be described. This mode is called an affine motion compensation prediction mode.

Figure 9 is a diagram for explaining the derivation of motion vectors of sub-block units according to motion vectors of a plurality of adjacent blocks. In Figure 9, the current block contains 16 4x4 sub-blocks. Here, the motion vector v ₀ of the upper left corner control point of the current block is derived according to the motion vector of the adjacent block, and the motion of the upper right corner control point of the current block is derived according to the motion vector of the adjacent sub-block. Vector v ₁ . Further, using two motion vectors v ₀ and v ₁ , the motion vectors (v _x , v _y ) of the respective sub-blocks in the current block are derived by the following equation (2). [Math 2]

Here, x and y each represent the horizontal position and the vertical position of the sub-block, and w is a predetermined weighting coefficient.

In this affine dynamic compensation prediction mode, the method of deriving the motion vectors of the upper left and upper right control points may also include several different modes. Information showing such an affine dynamic compensated prediction mode (which may be referred to as an affine flag) is signaled at the CU level. Furthermore, the signalization of the information showing the affine dynamic compensation prediction mode does not need to be limited to the CU level, and may also be other levels (eg, sequence level, picture level, slice level, tile level, CTU level, or sub- Block level). [Predictive Control Department]

The prediction control unit 128 selects any one of the in-frame prediction signal and the inter-frame prediction signal, and outputs the selected signal as a prediction signal to the subtraction unit 104 and the addition unit 116. [Summary of decoding device]

Next, an outline of a decoding device that can decode an encoded signal (encoded bit stream) output from the encoding device 100 will be described. FIG. 10 is a block diagram showing a functional configuration of a decoding device 200 according to the first embodiment. The decoding device 200 is a moving image/image decoding device that decodes a moving image/image in units of blocks.

As shown in FIG. 10, the decoding apparatus 200 includes an entropy decoding unit 202, an inverse quantization unit 204, an inverse conversion unit 206, an addition unit 208, a block memory 210, a loop filter unit 212, a frame memory 214, and an in-frame prediction unit 216. The inter-frame prediction unit 218 and the prediction control unit 220.

The decoding device 200 can be implemented by, for example, a general purpose processor and a memory. In this case, when the processor executes the software program stored in the memory, the processor functions as the entropy decoding unit 202, the inverse quantization unit 204, the inverse conversion unit 206, the addition unit 208, the loop filter unit 212, and the in-frame prediction. The unit 216, the inter-frame prediction unit 218, and the prediction control unit 220 function. Further, the decoding device 200 may correspond to the entropy decoding unit 202, the inverse quantization unit 204, the inverse conversion unit 206, the addition unit 208, the loop filter unit 212, the in-frame prediction unit 216, the inter-frame prediction unit 218, and the prediction control unit. One or more of 220 dedicated electronic circuits are implemented.

Hereinafter, each constituent element included in the decoding device 200 will be described. [Entropy decoding unit]

The entropy decoding unit 202 performs entropy decoding on the encoded bit stream. Specifically, the entropy decoding unit 202 performs arithmetic decoding of the binary signal from the encoded bit stream, for example. Further, the entropy decoding unit 202 debinarizes the binary signal. Thereby, the entropy decoding unit 202 outputs the quantized coefficients to the inverse quantization unit 204 in units of blocks. [Inverse Quantization Department]

The inverse quantization unit 204 inversely quantizes the quantized coefficients of the decoding target block (hereinafter referred to as the current block) which is an input from the entropy decoding unit 202. Specifically, the inverse quantization unit 204 is inverse quantized for each of the quantized coefficients of the current block based on the quantization parameter corresponding to the quantized coefficient. Further, the inverse quantization unit 204 outputs the quantized coefficients (that is, the conversion coefficients) of the current block that have been inversely quantized to the inverse transform unit 206. [Inverse Conversion Department]

The inverse conversion unit 206 inversely converts the conversion coefficient, which is an input from the inverse quantization unit 204, to restore the prediction error.

For example, in the case where the information that has been interpreted from the encoded bit stream shows that EMT or AMT is applied (for example, the AMT flag is true), the inverse conversion unit 206 compares the information indicating the type of conversion that has been interpreted. The conversion coefficient of the current block is inversely converted.

Further, for example, when the information that has been read from the encoded bit stream indicates that the NSST is applied, the inverse conversion unit 206 applies inverse retransformation to the conversion coefficient. [Addition Department]

The addition unit 208 adds the prediction error, which is an input from the inverse conversion unit 206, and the prediction sample input from the prediction control unit 220, thereby reconstructing the current block. Further, the addition unit 208 outputs the reconstructed block to the block memory 210 and the loop filter unit 212. [block memory]

The block memory 210 is a storage unit for storing a block referred to in the in-frame prediction and also a block in the decoding target picture (hereinafter referred to as a current picture). Specifically, the tile memory 210 stores the reconstructed block output from the addition unit 208. [loop filter unit]

The loop filter unit 212 performs loop filtering on the block reconstructed by the adder 208, and outputs the filtered reconstructed block to the frame memory 214, the display device, and the like.

When the information indicating that the ALF is turned on/off from the encoded bit stream is displayed, the ALF is turned on, and one filter can be selected from the plurality of filters according to the direction and activity of the local gradient. And the selected filter is applied to the reconstructed block. [frame memory]

The frame memory 214 is a storage unit for storing a reference picture used for inter-frame prediction, and is sometimes referred to as a frame buffer. Specifically, the frame memory 214 stores the reconstructed block that has been filtered by the loop filter unit 212. [In-frame prediction department]

The in-frame prediction unit 216 performs in-frame prediction based on the intra-frame prediction mode that has been interpreted from the encoded bit stream, and refers to the block stored in the current picture of the block memory 210, thereby generating a prediction signal ( In-frame prediction signal). Specifically, the in-frame prediction unit 216 performs intra-frame prediction by referring to samples (for example, luminance values and color difference values) of the blocks adjacent to the current block, thereby generating an in-frame prediction signal and predicting the in-frame prediction. The signal is output to the prediction control unit 220.

Furthermore, in the case where the in-frame prediction mode of the reference luminance block is selected in the intra-frame prediction of the chroma block, the in-frame prediction unit 216 may predict the chroma component of the current block based on the luminance component of the current block.

Further, when the information read from the encoded bit stream indicates that the PDPC is applied, the in-frame prediction unit 216 corrects the intra-frame predicted pixel value based on the gradient of the reference pixels in the horizontal/vertical direction. [Inter-frame prediction unit]

The inter-frame prediction unit 218 refers to the reference picture stored in the frame memory 214 to predict the current block. The prediction is made in units of the current block or a sub-block within the current block (for example, a 4x4 block). For example, the inter-frame prediction unit 218 performs motion compensation using motion information (for example, motion vector) interpreted from the encoded bit stream, thereby generating an inter-frame prediction signal of the current block or sub-block, and inter-frame prediction signals. The prediction signal is output to the prediction control unit 220.

Furthermore, in the case where the information read from the encoded bit stream shows that the OBMC mode is applied, the inter-frame prediction unit 218 uses not only the motion information of the current block obtained by the motion search but also the motion information of the current block obtained by the motion search. Motion information of adjacent blocks to generate inter-frame prediction signals.

Further, when the information read from the encoded bit stream shows that the FRUC mode is applied, the inter-frame prediction unit 218 follows the pattern matching method (bidirectional matching or template matching) interpreted from the encoded stream. Perform a motion search to derive sports information. Further, the inter-frame prediction unit 218 performs dynamic compensation using the derived motion information.

Further, when the BIO mode is applied, the inter-frame prediction unit 218 derives a motion vector based on a model that assumes a constant-speed linear motion. Further, in the case where the information read from the encoded bit stream indicates that the affine dynamic compensation prediction mode is applied, the inter-frame prediction unit 218 sets the sub-block unit based on the motion vectors of the plurality of adjacent blocks. Export motion vectors. [Predictive Control Department]

The prediction control unit 220 selects any one of the in-frame prediction signal and the inter-frame prediction signal, and outputs the selected signal as a prediction signal to the addition unit 208. (Embodiment 2)

The coding apparatus and the decoding apparatus according to the present embodiment have the same configuration and function as those of the first embodiment, but are characterized by the processing operations of the inter-frame prediction units 126 and 218. [Knowledge of knowledge that is the basis of this disclosure]

Figure 11 is a flow chart showing the dynamic compensation performed by other encoding devices that are the basis of the present disclosure. Furthermore, in each of the figures subsequent to Fig. 11, the motion vector is represented as MV.

The encoding device dynamically compensates the prediction block according to each prediction block corresponding to the prediction unit. At this time, the encoding apparatus first acquires a plurality of candidate motion vectors for the prediction block based on the information of the motion vectors of the plurality of coded completion blocks temporally or spatially located around the prediction block (step S101).

Next, the encoding apparatus extracts each of N (N is an integer of 2 or more) candidate motion vectors as a motion vector prediction from among a plurality of candidate motion vectors acquired in step S101 in accordance with a predetermined priority order. Sub-candidate (step S102). Furthermore, the priority order is predetermined for each of the N candidate motion vectors.

Next, the encoding apparatus selects one motion vector predictor candidate from among the N motion vector predictor candidates as a motion vector predictor of the prediction block. At this time, the encoding device encodes the motion vector predictor selection information for identifying the selected motion vector predictor into the stream (step S103). Furthermore, the stream is the above encoded signal or encoded bit stream.

Next, the encoding apparatus derives the motion vector of the prediction block by referring to the encoding completion reference picture (step S104). At this time, the encoding device further encodes the difference value of the derived motion vector and the motion vector predictor as the difference motion vector information in the stream. Furthermore, the coded completion reference picture is a picture formed by a plurality of blocks constructed by coding.

Finally, the encoding device dynamically compensates the prediction block by using the derived motion vector and the encoded completion reference picture, thereby generating a predicted image of the prediction block (step S105). Furthermore, the predicted image is the inter-frame prediction signal described above.

Figure 12 is a flow chart showing the dynamic compensation performed by other decoding devices that are the basis of the present disclosure.

The decoding device dynamically compensates the prediction block for each prediction block. At this time, the decoding apparatus first acquires a plurality of candidate motion vectors for the prediction block based on the information of the motion vectors of the plurality of decoding completion blocks temporally or spatially located around the prediction block (step S111).

Next, the decoding apparatus extracts each of N (N is an integer of 2 or more) candidate motion vectors as motion vector prediction from among a plurality of candidate motion vectors acquired in step S111 according to a predetermined priority order. Sub-candidate (step S112). Furthermore, the priority order is predetermined for each of the N candidate motion vectors.

Next, the decoding device decodes the motion vector predictor selection information from the input stream, and uses the decoded motion vector predictor selection information, and one of the N motion vector predictor candidates The motion vector predictor candidate is selected as the motion vector predictor of the prediction block (step S113).

Then, the decoding device decodes the differential motion vector information from the input stream, and adds the decoded difference motion vector information, that is, the difference value and the selected motion vector predictor, to derive the prediction. The motion vector of the block (step S114).

Finally, the decoding device dynamically compensates the prediction block by using the derived motion vector and the decoded complete reference picture, thereby generating a predicted image of the prediction block (step S115).

Here, in the example shown in FIGS. 11 and 12, in order to extract N motion vector predictor candidates, a predetermined priority order is used. However, each of the plurality of candidate motion vectors may also be evaluated in order to obtain a motion vector predictor that is smaller than the difference between the motion vectors of the prediction blocks. In other words, the evaluation value of each of the plurality of candidate motion vectors acquired in step S101 or step 111 may be calculated, and N motion vectors may be extracted from the plurality of candidate motion vectors based on the calculated evaluation values. Predictor candidates.

FIG. 13 is a view for explaining an example of a method of calculating an evaluation value.

As a method of calculating the evaluation value, for example, there is a template matching method. In this template matching method, a reconstruction image of a coding completion region or a decoding completion region in a moving image is used for calculation of an evaluation value. In addition, in FIG. 13, the encoding completion area and the decoding completion area are collectively referred to as a processing completion area, and the encoding target picture and the decoding target picture are collectively referred to as a processing target picture. Further, the prediction block to be coded and the prediction block to be decoded are collectively referred to as a processing target prediction block.

Specifically, the encoding apparatus calculates a difference value between two types of reconstructed images which are reconstructed from the coded completion area located around the processing target prediction block in the encoding target picture. The image, and the reconstructed image of the coded completion region located in the periphery of the block designated by the candidate motion vector in the encoded reference picture. For example, the difference value can be calculated as the sum of absolute differences of the pixel values.

Similarly to the encoding device, the decoding device calculates a difference value between two types of reconstructed images, which are located in the decoding completion region around the processing target prediction block in the decoding target picture. A reconstructed image constituting the image and the decoded completion region located in the periphery of the block designated by the candidate motion vector in the decoded reference picture. For example, the difference value can be calculated as the sum of absolute differences of the pixel values.

Furthermore, the block specified by the candidate motion vector in the reference picture is hereinafter referred to as a designated block. The designated block is located at a position indicated by the candidate motion vector with reference to the position on the spatial space of the prediction block of the processing object. Further, the relative position of the processing completion area with respect to the specified block in the reference picture and the relative position of the processing completion area with respect to the processing object prediction block in the processing target picture are equal. Moreover, the processing completion area located at the periphery of the processing target prediction block or the designated block may also be an area adjacent to the left side of the block and an area adjacent to the upper part, and may be adjacent only to the left side. The area may be an area adjacent only to the upper side. For example, if both the area adjacent to the left side and the area adjacent to the upper side exist, the areas can be used for the calculation of the evaluation value, and if any of the areas does not exist, only the existing area can be used for evaluation. The calculation of the value.

The encoding device and the decoding device calculate the evaluation value using the obtained difference value. For example, the smaller the difference value, the higher the evaluation value is calculated. Furthermore, the encoding device and the decoding device may calculate the evaluation value by using information other than the difference value.

Furthermore, the method of calculating the evaluation value by the template matching method shown in the example of FIG. 13 is merely an example, and is not limited thereto. For example, the position of the area for evaluation or the method of determining whether or not the area is used is not limited to the example of FIG. 13, and may be other positions or methods.

FIG. 14 is a view for explaining another example of the method of calculating the evaluation value.

As a method of calculating the evaluation value, for example, there is a two-way matching method. In this two-way matching method, the reconstructed image of the coded completion area or the decoded completed area in the moving image is also used for calculation of the evaluation value. In addition, in FIG. 14, the encoding completion area and the decoding completion area are collectively referred to as a processing completion area, and the encoding target picture and the decoding target picture are collectively referred to as a processing target picture. Further, the prediction block to be encoded and the prediction block to be decoded are collectively referred to as a processing target prediction block.

Specifically, the encoding apparatus calculates a difference value between two types of reconstructed images, which are reconstructed maps of the blocks specified by the candidate motion vectors in the encoded completed reference picture 1. The image and the reconstructed image of the block specified by the symmetric motion vector in the coded completion reference picture 2. The block specified by the candidate motion vector and the block specified by the symmetric motion vector are all coded completion regions. For example, the difference value can be calculated as the sum of absolute differences of the pixel values.

Similarly, the decoding device calculates a difference value between two types of reconstructed images in the same manner as the encoding device. The two reconstructed images are reconstructed by the block specified by the candidate motion vector in the decoded completed reference picture 1. The image, and the reconstructed image of the block specified by the symmetric motion vector in the decoded reference picture 2 is reconstructed. The block specified by the candidate motion vector and the block specified by the symmetric motion vector are all decoding completion areas. For example, the difference value can be calculated as the sum of absolute differences of the pixel values.

Furthermore, the symmetric motion vector is a motion vector generated by scaling the candidate motion vector in response to the display time interval described above. Further, the block specified by each of the candidate motion vector and the symmetric motion vector is located at a position indicated on the basis of the position on the space in which the prediction block is processed.

The encoding device and the decoding device calculate the evaluation value using the obtained difference value. For example, the smaller the difference value, the higher the evaluation value is calculated. Furthermore, the encoding device and the decoding device may calculate the evaluation value by using information other than the difference value.

Furthermore, the method of calculating the evaluation value by the two-way matching method shown in the example of FIG. 14 is merely an example, and is not limited thereto. For example, the method of specifying the position of the processing completion area for evaluation is not limited to the example shown in FIG.

By extracting N motion vector predictor candidates from a plurality of candidate motion vectors based on such evaluation values, there is a possibility that the prediction accuracy of the prediction block can be improved. Furthermore, the template matching method and the bidirectional matching method are the methods used in the above FRUC mode. Therefore, the method of extracting the motion vector predictor candidate based on such an evaluation value is also referred to as a method of extracting based on the evaluation result by the FRUC.

Here, in order to extract N motion vector predictor candidates from a plurality of candidate motion vectors for the prediction block, the first extraction method based on the evaluation result by the FRUC and the prioritized priority order may be used. The second method of capture.

For example, in order to extract two motion vector predictor candidates from a plurality of candidate motion vectors, the encoding apparatus and the decoding apparatus use the first extraction method to extract one motion vector predictor candidate, and use the second capture method. To retrieve the remaining one motion vector predictor candidate. In this case, it is conceivable that an individual candidate list can be utilized for each of the first extraction method and the second extraction method. These candidate lists are a list showing a plurality of candidate motion vectors.

Therefore, in the case as described above, there is a possibility that the prediction accuracy can be improved. However, it is necessary to create a plurality of candidate lists different from each other for one prediction block, and there is a problem that the processing load increases.

Therefore, the encoding apparatus 100 and the decoding apparatus 200 according to the present embodiment perform the dynamic compensation of the prediction block by using the extraction method based on the evaluation result by the FRUC and using one candidate list for the prediction block. .

Specifically, the encoding apparatus 100 according to the present embodiment extracts at least one motion vector predictor candidate of the encoding target block from a plurality of candidate motion vectors. At this time, the encoding apparatus 100 extracts all of the at least one motion vector predictor candidate according to the evaluation result of each of the plurality of candidate motion vectors, wherein the plurality of candidate motion vectors do not use the map of the encoding target block. The image area, but the candidate motion vector of the reconstructed image of the coded completion area in the moving image is used.

Further, the decoding apparatus 200 according to the present embodiment extracts at least one motion vector predictor candidate of the decoding target block from the plurality of candidate motion vectors. At this time, the decoding apparatus 200 extracts all of the at least one motion vector predictor candidate according to the evaluation result of each of the plurality of candidate motion vectors, wherein the plurality of candidate motion vectors do not use the map of the decoding target block. The image area, but the candidate motion vector of the reconstructed image of the decoding completion area in the moving image is used.

That is, the encoding device 100 and the decoding device 200 in the present embodiment extract all the motion vector predictor candidates by the extraction method based on the evaluation result by the FRUC. In other words, all motion vector predictor candidates are retrieved without using the acquisition method according to the priority order determined in advance. The all motion vector predictor candidates may be one motion vector predictor candidate, or may be a plurality of motion vector predictor candidates. Therefore, since the extraction method based on the priority order determined in advance is not used, a dedicated candidate list for the extraction method is not required, and dynamic compensation using one candidate list can be performed on the prediction block. [Use only FRUC to extract 1 motion vector predictor candidate]

Fig. 15 is a flowchart showing an example of dynamic compensation performed by the encoding device 100 in the present embodiment. When the encoding device 100 shown in FIG. 1 encodes a moving image composed of a plurality of pictures, the inter-frame prediction unit 126 of the encoding device 100 executes the processing shown in FIG.

Specifically, the inter-frame prediction unit 126 dynamically compensates the prediction block, that is, the coding target block, in accordance with each prediction block corresponding to the prediction unit. At this time, the inter-frame prediction unit 126 first acquires a plurality of candidate motion vectors for the prediction block according to information such as motion vectors of a plurality of coded completion blocks temporally or spatially located around the prediction block (step S201). . For example, the information such as the motion vector of the coded completion block may be the motion vector that has been used for the dynamic compensation of the coded completion block, or may be not only the motion vector, but also includes a display time interval, where the display time interval is The display time interval between the picture containing the coded completion block and the picture to be encoded. For example, the plurality of candidate motion vectors are motion vectors obtained by scaling each of the motion vectors of the plurality of coded completion blocks in response to the display time interval. Moreover, the plurality of coded completion blocks located around the prediction block may also be, for example, the following blocks: a plurality of coded completion blocks adjacent to each of the lower left, upper left, and upper right of the prediction block of the coding object. And an encoding completion block of all or a part of the plurality of coding completion blocks included in the picture different from the coding target picture.

Next, the inter-frame prediction unit 126 calculates an evaluation value for each of the plurality of candidate motion vectors acquired in step S201 using the reconstructed image of the encoding completion region. In other words, the inter-frame prediction unit 126 calculates an evaluation value based on the FRUC, that is, the template matching method or the two-way matching method. Further, the inter-frame prediction unit 126 selects one candidate motion vector having the highest evaluation value from among the plurality of candidate motion vectors as the motion vector predictor of the prediction block (step S202). In other words, the inter-frame prediction unit 126 extracts all of the at least one motion vector predictor candidate from among a plurality of candidate motion vectors by selecting only one candidate motion vector having the best evaluation result.

Furthermore, the inter-frame prediction unit 126 may also finely move the selected motion vector predictor in the peripheral region to increase the evaluation value by the FRUC, thereby correcting the motion vector predictor. In other words, the inter-frame prediction unit 126 can correct the motion vector predictor by searching for a region in which the evaluation value by the FRUC is made higher.

Next, the inter-frame prediction unit 126 refers to the coded completion reference picture to derive the motion vector of the prediction block (step S203). At this time, the inter-frame prediction unit 126 further calculates a difference value between the derived motion vector and the motion vector predictor. The entropy coding unit 110 encodes the difference value as a difference motion vector information in the stream. That is, the entropy coding unit 110 encodes the difference between the selected candidate motion vector, that is, the motion vector predictor, and the motion vector of the derived coding target block.

Finally, the inter-frame prediction unit 126 dynamically compensates the prediction block by using the derived motion vector and the coded completion reference picture, thereby generating a prediction picture of the prediction block (step S204).

Furthermore, the inter-frame prediction unit 126 may also replace the motion compensation in units of prediction blocks as described above, and equally derive the motion vector by sub-block units obtained by dividing the prediction block, and sub-regions. Block units for dynamic compensation.

Fig. 16 is a flowchart showing an example of dynamic compensation performed by the decoding device 200 in the embodiment. When the decoding device 200 shown in FIG. 10 decodes a moving image composed of a plurality of encoded pictures, the inter-frame prediction unit 218 of the decoding device 200 or the like executes the processing shown in FIG. 16.

Specifically, the inter-frame prediction unit 218 dynamically compensates the prediction block, that is, the decoding target block, in accordance with each prediction block corresponding to the prediction unit. At this time, the inter-frame prediction unit 218 first acquires a plurality of candidate motion vectors for the prediction block according to information such as motion vectors of a plurality of decoding completion blocks temporally or spatially located around the prediction block (step S211). . For example, the information such as the motion vector of the decoding completion block may be a motion vector that has been used for dynamic compensation of the decoding completion block, or may be not only the motion vector, but also includes a display time interval, where the display time interval is The display time interval between the picture including the decoding completion block and the picture to be decoded. For example, the plurality of candidate motion vectors are motion vectors obtained by scaling each of the motion vectors of the plurality of decoded completion blocks in response to the display time interval. Moreover, the plurality of decoding completion blocks located around the prediction block may also be, for example, the following blocks: a plurality of decoding completion blocks adjacent to the lower left, upper left, and upper right of the prediction block of the decoding target. And decoding completion blocks of all or a part of the plurality of decoding completion blocks included in the picture different from the decoding target picture.

Next, the inter-frame prediction unit 218 calculates an evaluation value for each of the plurality of candidate motion vectors acquired in step S211 using the reconstructed image of the decoding completion region. In other words, the inter-frame prediction unit 218 calculates an evaluation value based on the FRUC, that is, the template matching method or the two-way matching method. Further, the inter-frame prediction unit 218 selects one candidate motion vector having the highest evaluation value from among the plurality of candidate motion vectors as the motion vector predictor of the prediction block (step S212). In other words, the inter-frame prediction unit 218 extracts all of the at least one motion vector predictor candidate from among a plurality of candidate motion vectors by selecting only one candidate motion vector having the best evaluation result.

Furthermore, the inter-frame prediction unit 218 may also finely move the selected motion vector predictor in the peripheral region to increase the evaluation value by the FRUC, thereby correcting the motion vector predictor. In other words, the inter-frame prediction unit 218 can correct the motion vector predictor by searching for a region in which the evaluation value by the FRUC is made higher.

Next, the inter-frame prediction unit 218 derives a motion vector of the prediction block by using the difference motion vector information, which is decoded by the entropy decoding unit 202 from the stream that has been input to the decoding device 200. Information (step S213). Specifically, the inter-frame prediction unit 218 adds the decoded difference motion vector information, that is, the difference value, and the selected motion vector predictor, thereby deriving the motion vector of the prediction block. That is, the entropy decoding unit 202 decodes the difference motion vector information which is differential information showing the difference between the two motion vectors. Further, the inter-frame prediction unit 218 adds the motion vector predictor which is the selected candidate motion vector to the difference indicated by the decoded difference information, thereby deriving the motion of the decoding target block, that is, the prediction block. vector.

Finally, the inter-frame prediction unit 218 dynamically compensates the prediction block by using the derived motion vector and the decoded completion reference picture, thereby generating a prediction image of the prediction block (step S214).

Furthermore, the inter-frame prediction unit 218 may also replace the motion compensation in units of prediction blocks as described above, and equally derive the motion vector by sub-block units obtained by dividing the prediction block, and sub-regions. Block units for dynamic compensation.

Although in the example shown in FIG. 15 and FIG. 16, one motion vector predictor candidate is extracted, a plurality of motion vector predictor candidates may be extracted. [Use only FRUC to retrieve multiple motion vector predictor candidates]

Fig. 17 is a flowchart showing another example of dynamic compensation performed by the encoding device 100 in the embodiment. When the encoding device 100 shown in FIG. 1 encodes a moving image composed of a plurality of pictures, the inter-frame prediction unit 126 of the encoding device 100 or the like executes the processing shown in FIG.

Specifically, the inter-frame prediction unit 126 dynamically compensates the prediction block, that is, the coding target block, in accordance with each prediction block corresponding to the prediction unit. At this time, the inter-frame prediction unit 126 first obtains a plurality of candidate motion vectors for the prediction block according to information such as a motion vector of a plurality of coded completion blocks temporally or spatially located around the prediction block (step S201). .

Next, the inter-frame prediction unit 126 calculates an evaluation value for each of the plurality of candidate motion vectors acquired in step S201 using the reconstructed image of the encoding completion region. In other words, the inter-frame prediction unit 126 calculates an evaluation value based on the FRUC, that is, the template matching method or the two-way matching method. Further, the inter-frame prediction unit 126 extracts each of N (N is an integer of 2 or more) candidate motion vectors from the plurality of candidate motion vectors as motion vector prediction based on the evaluation values of the plurality of candidate motion vectors. Sub-candidate (step S202a). In other words, the inter-frame prediction unit 126 extracts all of the N candidate motion vectors from the plurality of candidate motion vectors as the at least one motion vector predictor candidate based on the evaluation result described above. More specifically, the inter-frame prediction unit 126 extracts, from among the plurality of candidate motion vectors, the top N candidate motion vectors ranked in the ranks of the evaluation results as all of the at least one motion vector predictor candidate. . In other words, the inter-frame prediction unit 126 extracts, from the plurality of candidate motion vectors, the first N candidate motion vectors ranked higher in the rank having the higher evaluation value as the motion vector predictor candidates.

Furthermore, the inter-frame prediction unit 126 may finely move the selected motion vector predictor candidate in the peripheral region for each of the N motion vector predictor candidates that have been captured to evaluate the FRUC. The value becomes higher, whereby the motion vector predictor candidate is corrected. In other words, the inter-frame prediction unit 126 can correct the motion vector predictor candidates by searching for a region in which the evaluation value by the FRUC is higher.

Further, the inter-frame prediction unit 126 selects a motion vector predictor of the prediction block from among the N motion vector predictor candidates that have been captured (step S202b). At this time, the inter-frame prediction unit 126 outputs the motion vector predictor selection information, which is information for identifying the selected motion vector predictor. The entropy coding unit 110 encodes the motion vector predictor selection information in the stream.

In the selection of the motion vector predictor, the inter-frame prediction unit 126 may also use the original image of the prediction target block, that is, the prediction block. For example, the inter-frame prediction unit 126 calculates a difference for each of the N motion vector predictor candidates, wherein the difference is an image of a block specified by the motion vector predictor candidate, and an original map of the prediction block. Like the difference. Further, the inter-frame prediction unit 126 is a motion vector predictor candidate that minimizes the difference, and selects a motion vector predictor as the prediction block. Alternatively, the inter-frame prediction unit 126 may perform motion search using the original image of the prediction block, thereby deriving the motion vector of the prediction block. Further, the inter-frame prediction unit 126 calculates a difference for each of the N motion vector predictor candidates, wherein the difference is an image of a block specified by the motion vector predictor candidate, and the derived prediction region The difference in the image of the block specified by the motion vector of the block. Further, the inter-frame prediction unit 126 is a motion vector predictor candidate that minimizes the difference, and selects a motion vector predictor as the prediction block.

Next, the inter-frame prediction unit 126 refers to the coded completion reference picture to derive the motion vector of the prediction block (step S203). At this time, the inter-frame prediction unit 126 further calculates a difference value between the derived motion vector and the motion vector predictor. The entropy coding unit 110 encodes the difference value as a difference motion vector information in the stream.

Finally, the inter-frame prediction unit 126 dynamically compensates the prediction block by using the derived motion vector and the coded completion reference picture, thereby generating a prediction picture of the prediction block (step S204).

Furthermore, the inter-frame prediction unit 126 may also replace the motion compensation in units of prediction blocks as described above, and equally derive the motion vector by sub-block units obtained by dividing the prediction block, and sub-regions. Block units for dynamic compensation.

Fig. 18 is a flowchart showing another example of dynamic compensation performed by the decoding device 200 in the embodiment. When the decoding device 200 shown in FIG. 10 decodes a moving image composed of a plurality of encoded pictures, the inter-frame prediction unit 218 of the decoding device 200 executes the processing shown in FIG. 18.

Specifically, the inter-frame prediction unit 218 dynamically compensates the prediction block, that is, the decoding target block, in accordance with each prediction block corresponding to the prediction unit. At this time, the inter-frame prediction unit 218 first acquires a plurality of candidate motion vectors for the prediction block based on information such as motion vectors of a plurality of decoding completion blocks temporally or spatially located around the prediction block (step S211). .

Next, the inter-frame prediction unit 218 calculates an evaluation value for each of the plurality of candidate motion vectors acquired in step S211 using the reconstructed image of the decoding completion region. In other words, the inter-frame prediction unit 218 calculates an evaluation value based on the FRUC, that is, the template matching method or the two-way matching method. Further, the inter-frame prediction unit 218 extracts each of N (N is an integer of 2 or more) candidate motion vectors from the plurality of candidate motion vectors as motion vector prediction based on the evaluation values of the plurality of candidate motion vectors. Sub-candidate (step S212a). In other words, the inter-frame prediction unit 218 extracts all of the N candidate motion vectors from the plurality of candidate motion vectors as the at least one motion vector predictor candidate based on the evaluation result described above. More specifically, the inter-frame prediction unit 218 extracts, from among the plurality of candidate motion vectors, the top N candidate motion vectors ranked in the ranks of the evaluation results as all of the at least one motion vector predictor candidate. . In other words, the inter-frame prediction unit 218 extracts, from the plurality of candidate motion vectors, the first N candidate motion vectors ranked higher in the rank having the higher evaluation value as the motion vector predictor candidates.

Furthermore, the inter-frame prediction unit 218 may also finely move the selected motion vector predictor candidate in the peripheral region for each of the N motion vector predictor candidates that have been captured, so as to be evaluated by the FRUC. The value becomes higher, whereby the motion vector predictor candidate is corrected. In other words, the inter-frame prediction unit 218 can correct the motion vector predictor candidates by searching for regions in which the evaluation values by the FRUC are made higher.

Next, the inter-frame prediction unit 218 selects one motion vector predictor candidate from the captured N motion vector predictor candidates as the motion vector predictor of the prediction block by using the motion vector predictor selection information. The motion vector predictor selection information is information decoded by the entropy decoding unit 202 from the stream that has been input to the decoding device 200 (step S212b). That is, the entropy decoding unit 202 decodes the motion vector predictor selection information which is information for identifying the motion vector predictor. Further, the inter-frame prediction unit 218 selects, as the motion vector predictor, the motion vector predictor candidate identified by the decoded motion vector predictor selection information from among the N motion vector predictor candidates that have been captured. .

Next, the inter-frame prediction unit 218 derives a motion vector of the prediction block by using the difference motion vector information, which is decoded by the entropy decoding unit 202 from the stream that has been input to the decoding device 200. Information (step S213). Specifically, the inter-frame prediction unit 218 adds the decoded difference motion vector information, that is, the difference value, and the selected motion vector predictor, thereby deriving the motion vector of the prediction block. That is, the entropy decoding unit 202 decodes the difference motion vector information which is differential information showing the difference between the two motion vectors. Further, the inter-frame prediction unit 218 adds the selected motion vector predictor to the difference indicated by the decoded difference information, thereby deriving the motion vector of the prediction block, that is, the prediction block.

Finally, the inter-frame prediction unit 218 dynamically compensates the prediction block by using the derived motion vector and the decoded completion reference picture, thereby generating a prediction image of the prediction block (step S214).

Furthermore, the inter-frame prediction unit 218 may also replace the motion compensation in units of prediction blocks as described above, and equally derive the motion vector by sub-block units obtained by dividing the prediction block, and sub-regions. Block units for dynamic compensation.

19 is a diagram for explaining a method of extracting N motion vector predictor candidates from a plurality of candidate motion vectors.

In the example shown in FIG. 17 and FIG. 18, the inter-frame prediction units 126 and 218 extract the top N candidate motion vectors from the plurality of candidate motion vectors among the plurality of candidate motion vectors. Motion vector predictor candidate. Specifically, in the case of N=2, as shown in (a) of FIG. 19, the inter-frame prediction units 126 and 218 are ranked from among all candidate motion vectors in order to rank higher in the evaluation value. Two candidate motion vectors are respectively taken as motion vector predictor candidates 1 and 2.

However, the inter-frame prediction units 126 and 218 may also classify a plurality of candidate motion vectors into N groups, and extract one candidate motion having the best evaluation result in the group from each of the N groups. The vector is used to retrieve all of the at least one motion vector predictor candidate. Specifically, in the case of N=2, as shown in FIG. 19(b), the inter-frame prediction units 126 and 218 classify all the candidate motion vectors into two groups. The first group is, for example, a group to which the candidate motion vector obtained based on the motion vector of the block in the encoding target picture or the like belongs. The second group is, for example, a group to which the candidate motion vector is obtained based on the motion vector of the block in the picture different from the encoding target picture or the like.

Further, the inter-frame prediction units 126 and 218 extract, from the first group, one candidate motion vector having the highest evaluation value in the group as the motion vector predictor candidate 1. Furthermore, the inter-frame prediction units 126 and 218 extract, from the second group, one candidate motion vector having the highest evaluation value among the groups as the motion vector predictor candidate 2.

Alternatively, the inter-frame prediction units 126 and 218 may also classify a plurality of candidate motion vectors into M (M is an integer greater than N) group. Further, the inter-frame prediction units 126 and 218 select, as the representative candidate motion vector, one candidate motion vector that has the best evaluation result in the group from each of the M groups. Then, the inter-frame prediction units 126 and 218 may also extract, from the selected M representative candidate motion vectors, the top N representative candidate motion vectors in the ranking with the preferred evaluation result as the at least one motion. All of the vector predictor candidates.

Specifically, in the case of M=3, as shown in (c) of FIG. 19, the inter-frame prediction units 126 and 218 classify all the candidate motion vectors into three groups. The first group is, for example, a group to which the candidate motion vector obtained based on the motion vector of the block on the left side of the encoding target block in the encoding target picture or the like. The second group is, for example, a group to which the candidate motion vector obtained based on the motion vector of the block on the upper side of the encoding target block in the encoding target picture or the like. The third group is a group to which the candidate motion vector is obtained, for example, based on a motion vector of a block in a picture different from the encoding target picture.

Further, the inter-frame prediction units 126 and 218 select one candidate motion vector having the highest evaluation value among the first groups as the representative candidate motion vector 1 from the first group. Further, the inter-frame prediction units 126 and 218 select, as the representative candidate motion vector 2, one candidate motion vector having the highest evaluation value among the second groups from the second group. Furthermore, the inter-frame prediction units 126 and 218 select, as the representative candidate motion vector 3, one candidate motion vector having the highest evaluation value among the third group from the third group.

Next, the inter-frame prediction units 126 and 218 extract, from the three representative candidate motion vectors that have been captured, the first two representative candidate motion vectors ranked higher in the rank having the higher evaluation value as the motion vector predictor candidates. .

Further, in the example shown in FIG. 19, two motion vector predictor candidates are extracted, but the number is not limited to two, and three or more motion vector predictor candidates may be extracted. [Effects of Embodiment 2, etc.]

The encoding device according to the present embodiment is an encoding device that encodes a moving image, and includes a processing circuit and a memory connected to the processing circuit, wherein the processing circuit uses the memory and is based on the moving image. And encoding a motion vector of each of the plurality of coding completion blocks corresponding to the coding target block to obtain a plurality of candidate motion vectors, and extracting at least one motion vector of the foregoing coding target block from the plurality of candidate motion vectors Deriving a sub-candidate, and deriving a motion vector of the foregoing encoding target block with reference to a reference picture included in the foregoing moving image, and predicting a motion vector predictor among the at least one motion vector predicting sub-candidate that has been extracted, and having derived Encoding the difference of the motion vector of the encoding target block, and using the derived motion vector of the encoding target block to dynamically compensate the foregoing coding target block, and extracting at least one motion vector predictor candidate The above is based on the evaluation result of each of the plurality of candidate motion vectors described above. All of the motion vector predictor candidates, wherein the plurality of candidate motion vectors do not use the image region of the foregoing encoding target block, but use the candidate motion vector of the reconstructed image of the encoded complete region in the foregoing moving image . Furthermore, the memory may be the frame memory 122 or other memory, and the processing circuit may include, for example, the inter-frame prediction unit 126 and the entropy coding unit 110.

Thereby, all of the at least one motion vector predictor candidate is extracted according to the evaluation result of each of the plurality of candidate motion vectors, that is, the result of the evaluation performed by the FRUC, wherein the plurality of candidate motion vectors are extracted. Instead of using the image area of the coding target block, a candidate motion vector of the reconstructed image of the coded completion area in the moving image is used. Thereby, the prediction accuracy of the coding target block, that is, the prediction block, can be improved, and the coding efficiency can be improved. Furthermore, in the present embodiment, it is not necessary to extract motion vector predictor candidates based on a predetermined priority order. Therefore, as long as the candidate list for the evaluation based on the evaluation result by the FRUC is generated, all the motion vector predictor candidates can be retrieved without generating the candidate list for the priority order. Thereby, an increase in processing load can be suppressed and an improvement in coding efficiency can be achieved.

Furthermore, the processing circuit may select only one candidate motion vector having the best evaluation result from the plurality of candidate motion vectors in the capture of the at least one motion vector predictor candidate. At least one motion vector predictor candidate, and encoding, in the encoding of the difference, a difference between the selected candidate motion vector, that is, the motion vector predictor, and the derived motion vector of the encoding target block .

Thereby, for example, as shown in FIG. 15, one motion vector predictor candidate can be retrieved, and the motion vector predictor candidate can be selected as a motion vector predictor. On the other hand, in the case of extracting a plurality of motion vector predictor candidates and selecting one motion vector predictor from the plurality of motion vector predictor candidates, it is necessary to identify the selected motion vector predictor. The information is encoded and included in the stream. However, in the example shown in FIG. 15, since one motion vector predictor candidate is extracted and the motion vector predictor candidate is selected as the motion vector predictor, it is not necessary to encode such information. Therefore, it is possible to reduce the amount of coding.

Furthermore, in the processing of the at least one motion vector predictor candidate, the processing circuit may select N (N is an integer of 2 or more) candidate motion vectors from the plurality of candidate motion vectors based on the evaluation result. Extracting all of the at least one motion vector predictor candidate, the processing circuit further selecting the motion vector predictor from the extracted N motion vector predictor candidates, and identifying the selected The selection information of the motion vector predictor is encoded, and in the encoding of the difference, the difference between the selected motion vector predictor and the derived motion vector of the encoding target block is encoded.

Thereby, for example, as shown in FIG. 17, a plurality of motion vector predictor candidates can be retrieved, and from which the image of the encoding target block, that is, the prediction block, can be used, the prediction precision is high. The motion vector predictor candidate is selected as a motion vector predictor. Therefore, it is possible to improve the coding efficiency. Further, since the selection information for identifying the motion vector predictor selected as such can be encoded, the decoding means can appropriately select the motion vector predictor selected in the encoding device by decoding the selected information. Motion vector predictor candidate. Thereby, the decoding device can be made to appropriately decode the encoded moving image.

Furthermore, the processing circuit may, in the extraction of the at least one motion vector predictor candidate, rank the top N candidate motion vectors from the plurality of candidate motion vectors. All of the at least one motion vector predictor candidate is extracted. For example, the evaluation result of each of the plurality of candidate motion vectors is an evaluation result that is smaller as the difference is smaller, wherein the difference is a reconstructed image of the first coded completion region specified by the candidate motion vector, and 2 The encoding is completed to reconstruct the difference of the image.

Thereby, for example, as shown in (a) of FIG. 19, N motion vector predictor candidates having higher prediction accuracy can be preferentially selected from a plurality of candidate motion vectors.

Furthermore, the processing circuit may classify the plurality of candidate motion vectors into N groups in the extraction of the at least one motion vector predictor candidate, and from each of the N groups. The first candidate motion vector having the best evaluation result in the group is extracted, thereby extracting all of the at least one motion vector predictor candidate. For example, as shown in (b) of FIG. 19, a plurality of candidate motion vectors are N groups classified as being different in nature from each other. Moreover, since one motion vector predictor candidate having the best evaluation result is obtained from each of the N groups, N motion vector predictor candidates having different properties and high prediction accuracy can be extracted. . As a result, the selection range of the motion vector predictor can be expanded, and the possibility of selecting a motion vector predictor with higher prediction accuracy can be improved.

Furthermore, the processing circuit may classify the plurality of candidate motion vectors into M (M is an integer greater than N) group in the extraction of the at least one motion vector predictor candidate, and from the foregoing M In each of the groups, one candidate motion vector having the best evaluation result in the group is selected as the representative candidate motion vector, and from the selected M representative candidate motion vectors, the foregoing evaluation will be performed. The result is that the top N representative candidate motion vectors in the preferred order are all taken as the at least one motion vector predictor candidate.

Thereby, for example, as shown in (c) of FIG. 19, even in the case of classifying a plurality of candidate motion vectors into more groups than the number of captured motion vector predictor candidates (ie, N) It is still possible to extract N motion vector predictor candidates that are different in nature from each other and have higher prediction accuracy.

Further, the decoding device according to the present embodiment is a decoding device that decodes the encoded moving image, and includes a processing circuit and a memory connected to the processing circuit, wherein the processing circuit uses the memory and And acquiring a motion vector of each of the plurality of decoding completion blocks corresponding to the decoding target block in the foregoing moving image, to obtain a plurality of candidate motion vectors, and extracting the decoding target block from the plurality of candidate motion vectors. At least one motion vector predictor candidate, and decoding differential information showing the difference between the two motion vectors, and adding the at least one motion that has been captured to the difference indicated by the decoded differential information a motion vector predictor among the vector predictor candidates, thereby deriving a motion vector of the decoding target block, and dynamically compensating the decoding target block by using the derived motion vector of the decoding target block, at least 1 The motion vector predictor candidate is extracted based on the evaluation result of each of the plurality of candidate motion vectors. Taking all of the at least one motion vector predictor candidate, wherein the plurality of candidate motion vectors do not use the image region of the decoding target block, but use a reconstructed map of the decoded complete region in the dynamic image. The candidate motion vector of the image. Furthermore, the memory may be the frame memory 214 or other memory, and the processing circuit may include, for example, the inter-frame prediction unit 218 and the entropy decoding unit 202.

Thereby, all of the at least one motion vector predictor candidate is extracted according to the evaluation result of each of the plurality of candidate motion vectors, that is, the result of the evaluation performed by the FRUC, wherein the plurality of candidate motion vectors are extracted. Instead of using the image area of the decoding target block, the candidate motion vector of the reconstructed image of the decoding completion area in the moving image is used. Therefore, the prediction accuracy of the decoding target block, that is, the prediction block can be improved, and the coding efficiency can be improved. Further, in the present embodiment, it is not necessary to extract motion vector predictor candidates based on a predetermined priority order. Therefore, as long as the candidate list for the evaluation based on the evaluation result by the FRUC is generated, all the motion vector predictor candidates can be retrieved without generating the candidate list for the priority order. Thereby, an increase in processing load can be suppressed and an improvement in coding efficiency can be achieved.

Furthermore, the processing circuit may select only one candidate motion vector having the best evaluation result from the plurality of candidate motion vectors in the capture of the at least one motion vector predictor candidate. At least one motion vector predictor candidate, and in the derivation of the motion vector of the decoding target block, adding the selected candidate motion vector to the difference indicated by the decoded difference information The motion vector predictor is thereby used to derive the motion vector of the aforementioned decoding target block.

Thereby, for example, as shown in FIG. 16, one motion vector predictor candidate can be retrieved, and the motion vector predictor candidate can be selected as a motion vector predictor. On the other hand, in the case where a plurality of motion vector predictor candidates have been retrieved, the information for identification must be decoded from the stream, wherein the information is used to identify from the motion vector predictor candidates. The information of the sub-predictor is predicted by the motion vector selected by the encoding device. However, in the example shown in FIG. 16, since one motion vector predictor candidate is extracted and the motion vector predictor candidate is selected as the motion vector predictor, such information does not need to be decoded. Therefore, it is possible to reduce the amount of coding.

Furthermore, in the processing of the at least one motion vector predictor candidate, the processing circuit may select N (N is an integer of 2 or more) candidate motion vectors from the plurality of candidate motion vectors based on the evaluation result. Taking all of the at least one motion vector predictor candidate, the foregoing processing circuit further decodes the selection information for identifying the motion vector predictor, and from the N motion vector predictor candidates that have been captured And selecting a motion vector predictor candidate identified by the decoded candidate information as the motion vector predictor, and in deriving the motion vector of the decoding target block, as indicated by the decoded differential information On the difference, the selected motion vector predictor is added, thereby deriving the motion vector of the aforementioned decoding target block.

Thereby, for example, as shown in FIG. 18, a plurality of motion vector predictor candidates can be retrieved, and among them, a motion vector predictor candidate with a higher prediction precision can be selected as a motion vector by selecting information. Forecaster. Therefore, it is possible to improve the coding efficiency.

Furthermore, the processing circuit may, in the extraction of the at least one motion vector predictor candidate, rank the top N candidate motion vectors from the plurality of candidate motion vectors. All of the at least one motion vector predictor candidate is extracted. For example, the evaluation result of each of the plurality of candidate motion vectors is a result of a better evaluation, wherein the difference is a reconstructed image of the first decoding completion region specified by the candidate motion vector, And the second decoding is completed to reconstruct the difference of the image.

Thereby, for example, as shown in (a) of FIG. 19, N motion vector predictor candidates having higher prediction accuracy can be preferentially selected from a plurality of candidate motion vectors.

Furthermore, the processing circuit may classify the plurality of candidate motion vectors into N groups and extract from each of the N groups in the capture of the at least one motion vector predictor candidate. The one candidate motion vector having the best evaluation result in the group is used to extract all of the at least one motion vector predictor candidate. For example, as shown in (b) of FIG. 19, a plurality of candidate motion vectors may be classified into N groups having different properties from each other. Moreover, since one motion vector predictor candidate having the best evaluation result is obtained from each of the N groups, N motion vector predictor candidates having different properties and high prediction accuracy can be extracted. . As a result, the selection range of the motion vector predictor can be expanded, and the possibility that the motion vector predictor with higher prediction accuracy is selected can be improved.

Furthermore, the processing circuit may classify the plurality of candidate motion vectors into M (M is an integer greater than N) group in the extraction of the at least one motion vector predictor candidate, and from the foregoing M In each of the groups, one candidate motion vector having the best evaluation result in the group is selected as the representative candidate motion vector, and from the selected M representative candidate motion vectors, the foregoing evaluation will be performed. The result is that the top N representative candidate motion vectors in the preferred order are all taken as the at least one motion vector predictor candidate.

Thereby, for example, as shown in (c) of FIG. 19, even in the case of classifying a plurality of candidate motion vectors into more groups than the number of captured motion vector predictor candidates (ie, N) It is still possible to extract N motion vector predictor candidates that are different in nature from each other and have higher prediction accuracy.

These comprehensive or specific aspects may be implemented by systems, devices, methods, integrated circuits, computer programs, or non-transitory recording media such as computer-readable CD-ROMs, or by systems and devices. , any combination of methods, integrated circuits, computer programs, and recording media. (Embodiment 3) [FRUC/Priority Switching]

The coding apparatus and the decoding apparatus according to the present embodiment have the same configuration as that of the first embodiment, but are characterized by the processing operations of the inter-frame prediction units 126 and 218. In other words, the present embodiment is also a problem in solving the above-mentioned [knowledge knowledge which is the basis of the present disclosure], that is, it is necessary to create a plurality of candidate lists different from each other for one prediction block. A problem that increases the processing burden.

The encoding apparatus 100 according to the present embodiment encodes mode information for identifying the capturing method in the extraction of the at least one motion vector predictor candidate. Further, the encoding apparatus 100 selects a capture method identified by the mode information for the encoding target block from the first capture method and the second capture method, and according to the selected capture method. The at least one motion vector predictor candidate is retrieved. Here, the first extraction method is a method of capturing an evaluation result according to each of a plurality of candidate motion vectors, wherein the plurality of candidate motion vectors do not use an image region of the encoding target block, but a dynamic image is used. The candidate motion vector of the reconstructed image of the coded completion region. Further, the second capture method is a capture method based on a priority order which is a predetermined priority order for a plurality of candidate motion vectors.

Further, the decoding apparatus 200 according to the present embodiment decodes the mode information for identifying the capture method in the capture of the at least one motion vector predictor candidate. Further, the decoding apparatus 200 selects a capture method identified by the decoded mode information for the decoding target block from the first capture method and the second capture method, and according to the selected capture method. And extracting the at least one motion vector predictor candidate.

In other words, the encoding apparatus 100 and the decoding apparatus 200 in the present embodiment switch the extraction method of at least one motion vector predictor candidate to the acquisition result by the FRUC for each prediction block, And methods of obtaining according to prioritized priorities.

Thereby, it is not necessary to create a plurality of candidate lists different from each other for the prediction block, and it is possible to suppress an increase in the processing load.

FIG. 20 is a flowchart showing a method of selecting a motion vector predictor by the encoding device 100 and the decoding device 200 in the present embodiment.

The inter-frame prediction units 126 and 218 determine whether the mode information is display 0 or display 1 (step S301). The mode information is information for identifying a method of capturing at least one motion vector predictor candidate. Specifically, in the case of mode information=0, the mode information is a display method of displaying the first extraction method, that is, based on the evaluation result by the FRUC. In the case of mode information=1, the mode information is a method of displaying the second extraction method, that is, a method of capturing in accordance with a predetermined priority order.

Here, when it is determined that the mode information is 0, the inter-frame prediction units 126 and 218 extract at least one motion vector predictor candidate based on the evaluation result by the FRUC in the same manner as in the second embodiment ( Step S302). Specifically, the inter-frame prediction units 126 and 218 perform evaluation of a reconstructed image using encoding completion or decoding for each of a plurality of candidate motion vectors. Further, the inter-frame prediction units 126 and 218 extract at least one motion vector predictor candidate from the plurality of candidate motion vectors based on the evaluation result.

On the other hand, when the inter-frame prediction units 126 and 218 determine in step S301 that the mode information is 1 is displayed, N is similar to the example shown in FIGS. 11 and 12 (N is 2 or more). Integer) motion vector predictor candidate (step S303). Specifically, the inter-frame prediction units 126 and 218 extract N motion vector predictor candidates from a plurality of candidate motion vectors in accordance with a predetermined priority order.

When at least one motion vector predictor candidate has been extracted in step S302, the inter-frame prediction units 126 and 218 determine whether or not the number of the extracted motion vector predictor candidates is plural (step S304). Here, when the inter-frame prediction units 126 and 218 determine that the number of motion vector predictor candidates that have been captured is one (NO in step S304), the captured motion vector predictor candidate is selected as the code. The target block is the motion vector predictor of the prediction block (step S305).

On the other hand, when the inter-frame prediction units 126 and 218 determine that the number of motion vector predictor candidates that have been captured is plural (YES in step S304), it is determined from the plurality of motion vector predictor candidates. The motion vector predictor candidate identified by the vector predictor selection information is selected as the motion vector predictor of the prediction block (step S306).

When the N motion vector predictor candidates have been retrieved in step S303, the inter-frame prediction units 126 and 218 predict the motion vector indicated by the motion vector predictor selection information from the N motion vector predictor candidates. The sub-candidate is selected as the motion vector predictor of the prediction block (step S306).

Furthermore, the mode information can be encoded in the stream by the entropy encoding unit 110 of the encoding device 100, and can be decoded from the stream by the entropy decoding unit 202 of the decoding device 200.

This mode information can be encoded in the header area of any of the sequence layer, the picture layer, and the slice layer. That is, the entropy coding unit 110 encodes mode information for identifying a method of capturing the blocks included in the layer in the header area of the layer. Further, the entropy decoding unit 202 decodes the mode information for identifying the acquisition method for each of the blocks included in the layer from the header area of the layer.

Thereby, the method of capturing the FRUC and the method of capturing according to the predetermined priority order can be switched for each sequence, picture, or segment. Also, the mode information can also be predicted in block units to be encoded in the stream. That is, the entropy encoding unit 110 encodes the mode information for each of the blocks included in the moving image, wherein the mode information is information for identifying the capturing method with respect to the block. Further, the entropy decoding unit 202 decodes the mode information for each of the blocks included in the moving image, wherein the mode information is information for identifying the capturing method of the block. Thereby, the method of capturing can be switched for each prediction block.

Furthermore, the above numerical value (0 or 1) indicated by the mode information is only an example, and may be a numerical value other than these numerical values. Also, the mode information can display an identification code other than the value. That is, as long as it is an identification code that can distinguish between the extraction method based on the evaluation result by the FRUC and the extraction method according to the predetermined priority order, the mode information can indicate which identification code is acceptable.

Further, the motion vector predictor selection information is encoded in the stream by the entropy encoding unit 110 of the encoding apparatus 100 in the prediction block unit, and is streamed by the entropy decoding unit 202 of the decoding device 200. Decoded out. [Effects of Embodiment 3, etc.]

The encoding device according to the present embodiment is an encoding device that encodes a moving image, and includes a processing circuit and a memory connected to the processing circuit. The processing circuit uses the memory and is based on the moving image. Obtaining, by the motion vector of each of the plurality of coding completion blocks corresponding to the coding target block, a plurality of candidate motion vectors, and extracting at least one of the foregoing coding target blocks from the plurality of candidate motion vectors a motion vector predictor candidate, and deriving a motion vector of the encoding target block with reference to a reference picture included in the moving image, and a motion vector predictor among the at least one motion vector predictor candidate that has been captured, and Deriving a difference of motion vectors of the foregoing coding target block, and performing motion compensation on the coding target block by using the derived motion vector of the coding target block, where the at least one motion vector predictor candidate is used In the capture, the mode information used to identify the capture method is encoded, and the first capture method is used. And in the second extraction method, the capture target block is selected for the encoding target block, and the at least one motion vector predictor is captured according to the selected extraction method. a candidate, the first extraction method is a method for extracting an evaluation result according to each of the plurality of candidate motion vectors, wherein the plurality of candidate motion vectors do not use the image region of the encoding target block, but are used The candidate motion vector of the reconstructed image of the coded completion region in the moving image, the second capture method is a capture method according to a priority order, wherein the priority order is a predetermined priority order for the plurality of candidate motion vectors. Furthermore, the memory may be the frame memory 122 or other memory, and the processing circuit may include, for example, the inter-frame prediction unit 126 and the entropy coding unit 110.

Thereby, the first extraction method based on the evaluation result by the FRUC or the second extraction method according to the predetermined priority order can be applied to the prediction target block, that is, the prediction block in response to the mode information. . That is, the capture method can be switched. Thereby, the prediction accuracy of the coding target block, that is, the prediction block, can be improved, and the coding efficiency can be improved. Further, in the present embodiment, since any of the first extraction method and the second extraction method can be applied to the prediction block, it is not necessary to separately generate the first extraction method for the prediction block. Candidate list and candidate list for the second method. Thereby, an increase in processing load can be suppressed and an improvement in coding efficiency can be achieved.

In addition, in the coding of the mode information, mode information may also be encoded in a header area of any one of a sequence layer, a picture layer, and a slice layer in the stream of the moving image, where the mode information is Information for identifying a method of capturing the blocks included in the aforementioned layer.

In this way, the capture method can be switched in sequence, picture, or segment unit. Further, the amount of coding of the mode information can be further suppressed as compared with the case of switching the capturing method by, for example, a block unit such as a prediction block.

In addition, in the encoding of the foregoing mode information, the mode information may be encoded according to each of the blocks included in the dynamic image, wherein the mode information is information for identifying a method for capturing the block. .

Thereby, the capture method can be switched by using a block unit such as a prediction block. Moreover, the possibility of improving the prediction accuracy of the block can be further improved as compared with the case where the capture method is switched in units of sequences, pictures, or segments.

Further, the decoding device according to the present embodiment is a decoding device that decodes the encoded moving image, and includes a processing circuit and a memory connected to the processing circuit, wherein the processing circuit uses the memory, according to the foregoing Obtaining, by the motion vector of each of the plurality of decoding completion blocks corresponding to the decoding target block in the dynamic image, to obtain a plurality of candidate motion vectors, and extracting the decoding target block from the plurality of candidate motion vectors At least one motion vector predictor candidate, and decoding differential information showing the difference of the two motion vectors, and adding the at least one motion vector that has been captured to the difference indicated by the decoded differential information Predicting a motion vector predictor among the sub-candidates, thereby deriving a motion vector of the decoding target block, and dynamically compensating the decoding target block by using the derived motion vector of the decoding target block, at least one of the foregoing In the capture of the motion vector predictor candidate, the mode information used to identify the capture method is decoded, and from the first In the method and the second extraction method, for the decoding target block, a capture method identified by the decoded mode information is selected, and the at least one of the at least one is captured according to the selected extraction method. a motion vector predictor candidate, wherein the first scooping method is a scribing method according to an evaluation result of each of the plurality of candidate motion vectors, wherein the plurality of candidate motion vectors do not use an image region of the encoding target block, Instead, the candidate motion vector of the reconstructed image of the coded completion region in the moving image is used, and the second capture method is a capture method according to a priority order in which the plurality of candidate motion vectors are specified in advance. Priority. Furthermore, the memory may be the frame memory 214 or other memory, and the processing circuit may include, for example, the inter-frame prediction unit 218 and the entropy decoding unit 202.

Thereby, the first extraction method based on the evaluation result by the FRUC or the second extraction method according to the predetermined priority order can be applied to the prediction target block, that is, the prediction block in response to the mode information. . That is, the capture method can be switched. Therefore, the prediction accuracy of the decoding target block, that is, the prediction block can be improved, and the coding efficiency can be improved. Further, in the present embodiment, since any of the first extraction method and the second extraction method can be applied to the prediction block, it is not necessary to separately generate the first extraction method for the prediction block. Candidate list and candidate list for the second method. Thereby, an increase in processing load can be suppressed and an improvement in coding efficiency can be achieved.

Moreover, in the decoding of the mode information, the mode information may be decoded from a header region of any one of a sequence layer, a picture layer, and a slice layer in the stream of the moving image, where the mode information is Information for identifying a method of capturing the blocks included in the aforementioned layer.

In this way, the capture method can be switched in sequence, picture, or segment unit. Further, the amount of coding of the mode information can be further suppressed as compared with the case of switching the capturing method by, for example, a block unit such as a prediction block.

Moreover, in the decoding of the foregoing mode information, the mode information may be decoded for each block included in the dynamic image, wherein the mode information is information for identifying a method for capturing the block. .

Thereby, the capture method can be switched by using a block unit such as a prediction block. Moreover, the possibility of improving the prediction accuracy of the block can be further improved as compared with the case where the capture method is switched in units of sequences, pictures, or segments.

These comprehensive or specific aspects may be implemented by systems, devices, methods, integrated circuits, computer programs, or non-transitory recording media such as computer-readable CD-ROMs, or by systems and devices. , any combination of methods, integrated circuits, computer programs, and recording media. (Embodiment 4) [Selecting motion vector predictor candidates from FRUC and priority order from a common candidate list]

The coding apparatus and the decoding apparatus according to the present embodiment have the same configuration as that of the first embodiment, but are characterized by the processing operations of the inter-frame prediction units 126 and 218. In other words, in the same manner as in the second and third embodiments, the present embodiment solves the above-mentioned problem in the knowledge knowledge which is the basis of the present disclosure, that is, it is necessary to create a plurality of candidates different from each other in one prediction block. List, which causes problems in processing burden.

The encoding apparatus 100 according to the present embodiment extracts N (N is an integer of 2 or more) motion vector predictor candidates from the plurality of candidate motion vectors with respect to the encoding target block. At this time, the encoding apparatus 100 generates a candidate list in which a plurality of candidate motion vectors are displayed, and a candidate list common to the first extraction method and the second extraction method. Further, the encoding apparatus 100 extracts M (M is an integer of 1 or more and less than N) motion vector predictor candidate from the plurality of candidate motion vectors indicated by the common candidate list in accordance with the first acquisition method. Furthermore, the encoding apparatus 100 extracts L (L=N-M) motion vector predictor candidates from the plurality of candidate motion vectors indicated by the common candidate list in accordance with the second acquisition method. Here, the first extraction method is a method of extracting the evaluation result according to each of the plurality of candidate motion vectors, specifically, a method of extracting the evaluation result by the FRUC, wherein the plurality of candidate motions The vector does not use the image area of the encoding target block, but uses the candidate motion vector of the reconstructed image of the coded completion area in the moving image. Further, the second capture method is a capture method based on a priority order which is a predetermined priority order for a plurality of candidate motion vectors.

Further, the decoding apparatus 200 according to the present embodiment extracts N (N is an integer of 2 or more) motion vector predictor candidates from the plurality of candidate motion vectors with respect to the decoding target block. At this time, the decoding device 200 generates a candidate list in which a plurality of candidate motion vectors are displayed, and a candidate list common to the first extraction method and the second extraction method. Further, the decoding apparatus 200 extracts M (M is an integer of 1 or more and less than N) motion vector predictor candidate from the plurality of candidate motion vectors indicated by the common candidate list. Furthermore, the decoding apparatus 200 extracts L (L=N-M) motion vector predictor candidates from the plurality of candidate motion vectors indicated by the common candidate list in accordance with the second extraction method.

Fig. 21 is a flowchart showing an example of dynamic compensation performed by the encoding device 100 in the embodiment. When the encoding apparatus 100 shown in FIG. 1 encodes a moving image composed of a plurality of pictures, the inter-frame prediction unit 126 of the encoding apparatus 100 or the like executes the processing shown in FIG. 21.

Specifically, the inter-frame prediction unit 126 dynamically compensates the prediction block, that is, the coding target block, in accordance with each prediction block corresponding to the prediction unit. At this time, the inter-frame prediction unit 126 first obtains a plurality of candidate motion vectors for the prediction block according to information such as a motion vector of a plurality of coded completion blocks temporally or spatially located around the prediction block (step S201). . At this time, the inter-frame prediction unit 126 generates a candidate list in which a plurality of candidate motion vector candidates acquired in step S201 are displayed, and a method based on the evaluation result by the FRUC and a predetermined method are provided. A list of candidates common to the method of prioritization.

Next, the inter-frame prediction unit 126 calculates an evaluation value for each of the plurality of candidate motion vectors acquired in step S201 using the reconstructed image of the encoding completion region. That is, the inter-frame prediction unit 126 calculates the evaluation values based on the FRUC, that is, the template matching method or the two-way matching method. Further, the inter-frame prediction unit 126 extracts each of the M candidate motion vectors as a motion vector from among the plurality of candidate motion vectors indicated by the common candidate list, based on the evaluation values of the plurality of candidate motion vectors. The sub-candidate 1 is predicted (step S202aa). In other words, the inter-frame prediction unit 126 extracts, from the plurality of candidate motion vectors, the candidate motion vectors of the top M in the higher order of the evaluation value as the motion vector predictor candidates. Furthermore, the inter-frame prediction unit 126 may also finely move the selected motion vector predictor candidate 1 in the peripheral region for each of the M motion vector predictor candidates that have been captured to be performed by the FRUC. The evaluation value becomes higher, thereby correcting the motion vector predictor candidate 1. In other words, the inter-frame prediction unit 126 can also finely search for a region in which the evaluation value by the FRUC is higher, thereby correcting the motion vector predictor candidates 1.

Further, the inter-frame prediction unit 126 extracts each of the L candidate motion vectors as a motion vector predictor candidate 2 from a plurality of candidate motion vectors indicated by the common candidate list in accordance with a predetermined priority order. (Step S202ab).

Further, the inter-frame prediction unit 126 selects one of the M motion vector predictor candidates 1 and the L motion vector predictor candidates 2 that have been captured, and selects motion vector prediction as a prediction block. Sub (step S202b). At this time, the inter-frame prediction unit 126 outputs motion vector predictor selection information for identifying the selected motion vector predictor. The entropy coding unit 110 encodes the motion vector predictor selection information in the stream.

Next, the inter-frame prediction unit 126 refers to the encoding completion reference picture, and derives the motion vector of the prediction block (step S203). At this time, the inter-frame prediction unit 126 further calculates a difference value between the derived motion vector and the motion vector predictor. The entropy coding unit 110 encodes the difference value as a difference motion vector information in the stream.

Finally, the inter-frame prediction unit 126 dynamically compensates the prediction block by using the derived motion vector and the coded completion reference picture, thereby generating a prediction picture of the prediction block (step S204).

Furthermore, the inter-frame prediction unit 126 may also perform motion compensation in units of prediction blocks as described above, and similarly derive motion vectors by sub-block units obtained by dividing prediction blocks, and sub-areas Block units for dynamic compensation.

Fig. 22 is a flowchart showing an example of dynamic compensation performed by the decoding device 200 in the embodiment. When the decoding device 200 shown in FIG. 10 decodes a moving image composed of a plurality of encoded pictures, the inter-frame prediction unit 218 of the decoding device 200 executes the processing shown in FIG.

Specifically, the inter-frame prediction unit 218 dynamically compensates the prediction block, that is, the decoding target block, in accordance with each prediction block corresponding to the prediction unit. At this time, the inter-frame prediction unit 218 first acquires a plurality of candidate motion vectors for the prediction block based on information such as motion vectors of a plurality of decoding completion blocks temporally or spatially located around the prediction block (step S211). .

At this time, the inter-frame prediction unit 218 generates a candidate list in which a plurality of candidate motion vector candidates acquired in step S211 are displayed, and a method based on the evaluation result by the FRUC and a predetermined method are provided. A list of candidates common to the method of prioritization.

Next, the inter-frame prediction unit 218 calculates an evaluation value for each of the plurality of candidate motion vectors acquired in step S211 using the reconstructed image of the decoding completion region. That is, the inter-frame prediction unit 218 calculates the evaluation values based on the FRUC, that is, the template matching method or the two-way matching method. Further, the inter-frame prediction unit 218 extracts each of the M candidate motion vectors as a motion vector from among the plurality of candidate motion vectors indicated by the common candidate list, based on the evaluation values of the plurality of candidate motion vectors. The sub-candidate 1 is predicted (step S212aa). In other words, the inter-frame prediction unit 218 extracts, from the plurality of candidate motion vectors, the candidate M motion vectors ranked first in the higher order of the evaluation value as the motion vector predictor candidates. Furthermore, the inter-frame prediction unit 218 may also finely move the selected motion vector predictor candidate 1 in the peripheral region for each of the M motion vector predictor candidates that have been captured to be performed by the FRUC. The evaluation value becomes higher, whereby the motion vector predictor candidate 1 is corrected so that the evaluation value by the FRUC becomes higher. In other words, the inter-frame prediction unit 218 can also finely search for regions in which the evaluation values by the FRUC are higher, thereby correcting the motion vector predictor candidates 1.

Furthermore, the inter-frame prediction unit 218 extracts each of the L candidate motion vectors as a motion vector predictor candidate from a plurality of candidate motion vectors indicated by the common candidate list in accordance with a predetermined priority order. 2 (step S212ab).

Next, the inter-frame prediction unit 218 uses the motion vector prediction sub-selection information to select one motion vector predictor candidate from the M motion vector predictor candidates 1 and the L motion vector predictor candidates 2 that have been extracted. The motion vector predictor of the block is predicted (step S212b). That is, the entropy decoding unit 202 decodes the motion vector predictor selection information which is information for identifying the motion vector predictor of the decoding target block, that is, the prediction block. Further, the inter-frame prediction unit 218 selects, as the prediction region, the motion vector predictor candidate identified by the decoded motion vector predictor selection information from the N motion vector predictor candidates 1 and 2 that have been captured. The motion vector predictor of the block.

Next, the inter-frame prediction unit 218 derives a motion vector of the prediction block by using the difference motion vector information, which is obtained by decoding by the entropy decoding unit 202 from the stream that has been input to the decoding device 200. Information (step S213). Specifically, the inter-frame prediction unit 218 adds the decoded difference motion vector information, that is, the difference value, and the selected motion vector predictor, thereby deriving the motion vector of the prediction block. That is, the entropy decoding unit 202 decodes the difference motion vector information which is differential information showing the difference between the two motion vectors. Further, the inter-frame prediction unit 218 adds the selected motion vector predictor to the difference indicated by the decoded difference information, thereby deriving the motion vector of the prediction block, that is, the prediction block.

Finally, the inter-frame prediction unit 218 dynamically compensates the prediction block by using the derived motion vector and the decoded completion reference picture, thereby generating a prediction image of the prediction block (step S214).

Furthermore, the inter-frame prediction unit 218 may also replace the motion compensation in units of prediction blocks as described above, and equally derive the motion vector by sub-block units obtained by dividing the prediction block, and sub-regions. Block units for dynamic compensation.

Here, in the present embodiment, in the capture according to the second capture method performed by the inter-frame prediction units 126 and 218, that is, in the capture according to the predetermined priority order, the first use may be utilized. The results of the extraction method, that is, the results obtained from the evaluation results performed by FRUC. In other words, the inter-frame prediction units 126 and 218 extract L motion vector predictor candidates from at least one candidate motion vector in accordance with the priority order of the evaluation results in the first acquisition method, wherein the at least one The candidate motion vectors are among the plurality of candidate motion vectors shown on the common candidate list, except for at least one candidate motion other than the M motion vector predictor candidates that have been extracted by the first acquisition method. vector.

Fig. 23 is a view for explaining a method of capturing a motion vector predictor candidate in the embodiment;

For example, the inter-frame prediction units 126 and 218 classify a plurality of candidate motion vectors indicated by the common candidate list into K (K is an integer of 2 or more) group. Further, in the acquisition of the M motion vector predictor candidates 1, the inter-frame prediction units 126 and 218 are the preferred ones from the plurality of candidate motion vectors indicated by the common candidate list. The top M candidate motion vectors are extracted as M motion vector predictor candidates 1. Furthermore, in the acquisition of the L motion vector predictor candidates 2, the inter-frame prediction units 126 and 218 extract L motion vector predictor candidates 2 from one or more candidate motion vectors in accordance with a priority order, wherein Among the common candidate lists, the one or more candidate motion vectors belong to at least one group other than the group to which each of the M motion vector predictor candidates 1 belongs.

Specifically, as shown in (a) of FIG. 23, when K=3, M=1, and L=1, the inter-frame prediction units 126 and 218 are first a plurality of common candidate lists. The candidate motion vectors are classified into three groups G1 to C3. The group G1 is a group to which the candidate motion vector is obtained according to, for example, a motion vector of a block on the left side, and the block on the left side is a block on the left side of the coding target block in the coding target picture. The group G2 is a group to which the candidate motion vector is obtained based on, for example, a motion vector of the block on the upper side, and the block on the upper side is a block on the upper side of the coding target block in the picture to be encoded. The group G3 is a group to which the candidate motion vector is obtained based on, for example, a motion vector of a block in a picture different from the encoding target picture.

Next, the inter-frame prediction units 126 and 218 extract the candidate motion vector having the highest evaluation value as the motion vector predictor candidate 1 from the plurality of candidate motion vectors shown in the common candidate list. Next, the inter-frame prediction units 126 and 218 extract one candidate motion vector as one motion vector predictor candidate 2 from one or more candidate motion vectors in accordance with a priority order, wherein the one or more candidate motion vectors are Among the common candidate lists, any one of the groups G2 and G3 other than the group G1 to which the motion vector predictor candidate 1 belongs is attached.

Alternatively, the inter-frame prediction units 126 and 218 classify the plurality of candidate motion vectors shown in the common candidate list into K groups. Further, in the acquisition of the M motion vector predictor candidates 1, the inter-frame prediction units 126 and 218 are selected from the plurality of candidate motion vectors shown in the common candidate list, and the evaluation result is preferable. The top M candidate motion vectors are ranked as the M motion vector predictor candidates 1 in the order. Further, the inter-frame prediction units 126 and 218 specify, from among a plurality of candidate motion vectors, a candidate motion vector that is optimal in the evaluation result as a next motion vector predictor candidate, wherein the plurality of candidate motion vectors are among the common candidate lists. Is any one of at least one group other than the group to which each of the M motion vector predictor candidates 1 belongs. Further, in the acquisition of the L motion vector predictor candidates 2, the inter-frame prediction units 126 and 218 extract L motion vector predictors 2 from one or more candidate motion vectors in accordance with the priority order, wherein One or more candidate motion vectors are among the common candidate list, and belong to the same group as the group to which the specific next motion vector predictor candidate belongs.

Specifically, as shown in (b) of FIG. 23, when K=3, M=1, and L=1, the inter-frame prediction units 126 and 218 firstly share common candidates as in the above example. The plurality of candidate motion vectors shown on the list are classified into three groups.

Next, the inter-frame prediction units 126 and 218 extract the candidate motion vector having the highest evaluation value from the plurality of candidate motion vectors shown in the common candidate list as the motion vector predictor candidate 1. Furthermore, the inter-frame prediction units 126 and 218 specify, from among the plurality of candidate motion vectors, the candidate motion vector 4 having the highest evaluation value as the next motion vector predictor candidate, wherein the plurality of candidate motion vectors are in the common candidate list. Among them, any one of the groups G2 and G3 other than the group G1 to which the motion vector predictor candidate 1 belongs is affiliated. Further, the inter-frame prediction units 126 and 218 extract one candidate motion vector as a motion vector predictor candidate 2 from among one or more candidate motion vectors in a priority order, wherein the one or more candidate motion vectors are common. Among the candidate lists, belonging to the same group as the group G2, and the group G2 is a specific next motion vector predictor candidate, that is, a group to which the candidate motion vector 4 belongs.

Fig. 24 is a diagram showing an example of a common candidate list.

For example, the inter-frame prediction units 126 and 218 are the coding target block or the decoding target block (hereinafter simply referred to as the processing target block) shown in (a) of FIG. 24, and the (b) of FIG. 24 is generated. A list of common candidates. This common candidate list is composed of a list of L0 and a list of L1.

Specifically, the inter-frame prediction units 126 and 218 include the candidate motion vectors in a common candidate list, wherein the candidate motion vectors are based on adjacent blocks 1, 2, and 5 adjacent to the processing target block. The candidate motion vector of the motion vector. The adjacent block 1 is a block in the lower left adjacent to the block to be processed, the adjacent block 2 is a block in the upper right adjacent to the block to be processed, and the adjacent block 5 is adjacent to the processing target block. The upper left block of the block.

For example, adjacent block 1 is encoded or decoded by motion vectors mvL01 and mvL11. The adjacent block 2 is encoded or decoded by the motion vectors mvL02 and mvL12. The adjacent block 5 is encoded or decoded by the motion vector mvL05. In this case, as shown in (b) of FIG. 24, the inter-frame prediction units 126 and 218 include the candidate motion vectors based on the motion vectors as spatial candidate motion vectors in the common candidate list. Furthermore, the inter-frame prediction units 126 and 218 may also include candidate motion vectors based on motion vectors of other adjacent blocks as spatial candidate motion vectors in the common candidate list. The other adjacent blocks may be, for example, adjacent blocks 3 located immediately to the right of adjacent blocks 2, or adjacent blocks 4 located below the adjacent block 1, and the like. Moreover, the inter-frame prediction units 126 and 218 may also scale the motion vector of the adjacent block according to the display time interval, and include the scaled motion vector as a candidate motion vector in the candidate list.

Furthermore, the inter-frame prediction units 126 and 218 may also include the temporal candidate motion vector and the combined bi-predictive candidate motion vector (mvL0b, mvL1b) in the candidate list. In the temporal candidate motion vector, there may be, for example, a Col candidate motion vector (mvL0t, mvL1t) and a Unilateral candidate motion vector (mvL0u, mvL1u). The Col candidate motion vector (mvL0t, mvL1t) is a candidate motion vector according to a motion vector: a block located in a picture different from a picture containing a processing target block, for example, a block located at the same position as the processing target block. Motion vector. Furthermore, the Col candidate motion vector (mvL0t, mvL1t) may also be a candidate motion vector obtained by scaling a motion vector at a display time interval, wherein the motion vector is a block located in a picture different from the picture containing the processing target block. Motion vector. Also, the Col candidate motion vector may also be a candidate motion vector according to a motion vector, where the motion vector is a motion vector of a block located at a different location from the processing target block. Furthermore, a plurality of Col candidate motion vectors different from each other may also be included in the candidate list. The Unilateral candidate motion vector (mvL0u, mvL1u) is a candidate motion vector according to the motion vector of the block described below: a block in a picture different from the picture containing the block to be processed, and considering the amount of movement A block on the position where the amount of movement is an amount of movement over time with respect to the position of the aforementioned processing target block. The combined bi-predicted candidate motion vector (mvL0b, mvL1b) is a candidate motion vector generated by combining the L0 list of the candidate list with the respective motion vectors of the L1 list.

In the present embodiment, for example, the candidate list shown in (b) of FIG. 24 can be commonly used for the extraction method based on the evaluation result by the FRUC and the extraction method according to the predetermined priority order. [Effects of Embodiment 4, etc.]

The encoding device according to the present embodiment is an encoding device that encodes a moving image, and includes a processing circuit and a memory connected to the processing circuit. The processing circuit uses the memory and is based on the moving image. Obtaining, by the motion vector of each of the plurality of coding completion blocks corresponding to the coding target block, a plurality of candidate motion vectors, and extracting N from the plurality of candidate motion vectors relative to the foregoing coding target block (N is an integer of 2 or more) motion vector predictor candidate, and selecting a motion vector predictor from the aforementioned N motion vector predictor candidates, and identifying the selected motion vector predictor Selecting information for encoding, deriving a motion vector of the encoding target block with reference to a reference picture included in the moving image, and extracting a motion vector of the encoded target block and the selected motion vector predictor Differentially encoding, and using the derived motion vector of the coding target block to move the foregoing coding target block Compensating, in the extraction of the N motion vector predictor candidates, is a candidate list that displays a list of the plurality of candidate motion vectors, and is in the first capture method and the second capture method. a common candidate list, wherein, from the plurality of candidate motion vectors shown in the common candidate list, M (M is an integer greater than 1 and less than N) motion vector predictor is extracted according to the first extraction method a candidate, and extracting L (L=N-M) motion vector predictor candidates according to the second extraction method from the plurality of candidate motion vectors shown in the common candidate list, the first The acquisition method is a method for extracting an evaluation result of each of the plurality of candidate motion vectors, wherein the plurality of candidate motion vectors do not use the image region of the encoding target block, but the encoding in the foregoing dynamic image is used. Completing the candidate motion vector of the reconstructed image of the region, the second capturing method is a method of capturing according to a priority order, which is predetermined for the plurality of candidate motion vectors Priority order. Furthermore, the memory may be the frame memory 122 or other memory, and the processing circuit may include, for example, the inter-frame prediction unit 126 and the entropy coding unit 110.

Thereby, the M motion vector predictor candidates can be extracted according to the first acquisition method, that is, based on the evaluation result by the FRUC. Thereby, the prediction accuracy of the coding target block, that is, the prediction block, can be improved, and the coding efficiency can be improved. Further, in the present embodiment, a candidate list common to the first extraction method and the second extraction method can be generated. That is, whether the M motion vector predictor candidates are extracted according to the first acquisition method or the L motion vector predictor candidates are captured according to the second extraction method according to the predetermined priority order. In the case, you can refer to the common candidate list. As a result, it is not necessary to separately generate a candidate list for the first acquisition method and a candidate list for the second extraction method for the prediction block. Thereby, an increase in processing load can be suppressed and an improvement in coding efficiency can be achieved.

Further, in the acquisition by the second extraction method, the processing circuit may further select, from at least one of the candidate motion vectors, according to the priority order of the evaluation result in the first extraction method. Taking the foregoing L motion vector predictor candidates, wherein the at least one candidate motion vector is among the plurality of candidate motion vectors shown in the common candidate list, except for the foregoing by the first extraction method The remaining at least one candidate motion vector other than the M motion vector predictor candidates.

For example, as shown in FIG. 21, in the extraction according to the second extraction method (for example, step S202ab), the extraction result according to the first extraction method (for example, step S202aa) can be referred to. Therefore, it is possible to suppress the case where the same candidate motion vector is extracted as the motion vector predictor candidate by the first extraction method and the second extraction method.

Further, the processing circuit may classify the plurality of candidate motion vectors shown in the common candidate list into K (K is an integer of 2 or more) group in the capture of the N motion vector predictor candidates. In the acquisition according to the first extraction method, the first plurality of candidate motion vectors shown in the common candidate list are ranked in the top M of the preferred evaluation results. The candidate motion vector is extracted as the M motion vector predictor candidates. In the following method according to the second capture method, the L motion vectors are extracted from one or more candidate motion vectors according to the priority order. a prediction sub-candidate, wherein the one or more candidate motion vectors are among the aforementioned common candidate lists, belonging to at least one group other than the group to which each of the M motion vector predictor candidates belongs Either. For example, the evaluation result of each of the plurality of candidate motion vectors is a smaller evaluation result, which is a reconstructed image of the first coded completion region specified by the candidate motion vector, and The second encoding is completed to reconstruct the difference of the image.

For example, as shown in (a) of FIG. 23, a plurality of candidate motion vectors may be classified into K groups different in nature from each other. And, one (M=1) motion vector predictor candidate having the best evaluation result can be extracted from the whole of the K groups, and from the group other than the group to which the motion vector predictor candidate is attached, Another one (L=1) motion vector predictor candidate is retrieved according to a predetermined priority order. Thereby, two (N=2) motion vector predictor candidates which are different in nature from each other and have high prediction precision can be extracted. As a result, the selection range of the motion vector predictor can be expanded, and the possibility that the motion vector predictor with higher prediction accuracy is selected can be improved.

Further, the processing circuit may classify the plurality of candidate motion vectors shown in the common candidate list into K (K is an integer of 2 or more) group in the capture of the N motion vector predictor candidates. In the acquisition according to the first extraction method, the first plurality of candidate motion vectors shown in the common candidate list are ranked in the top M of the preferred evaluation results. The candidate motion vector is extracted as the foregoing M motion vector predictor candidates, and further, from the plurality of candidate motion vectors, the candidate motion vector having the best evaluation result is specified as a next motion vector predictor candidate, wherein the plurality of candidate motion vector candidates The candidate motion vector is one of at least one group other than the group to which each of the M motion vector predictor candidates belongs, among the aforementioned common candidate lists, according to the second aspect. In the acquisition method, the L motion vector predictor candidates are extracted from one or more candidate motion vectors according to the priority order, wherein the one or more candidates are selected Trends in the amount of common among the candidate list, the group is part of the same group with the next has a specific motion vector predictor candidate belongs.

For example, as shown in (b) of FIG. 23, a plurality of candidate motion vectors may be classified into K groups different in nature from each other. And, one (M=1) motion vector predictor candidate having the best evaluation result can be extracted from the whole of the K groups, and from the group other than the group to which the motion vector predictor candidate is attached, A specific next motion vector predictor candidate. Furthermore, another (L=1) motion vector predictor candidate may be retrieved in accordance with the priority order from the same group as the group to which the next motion vector predictor candidate belongs. Thereby, two (N=2) motion vector predictor candidates which are different in nature from each other and have high prediction precision can be extracted. As a result, the selection range of the motion vector predictor can be expanded, and the possibility that the motion vector predictor with higher prediction accuracy is selected can be improved.

Further, the decoding device according to the present embodiment is a decoding device that decodes the encoded moving image, and includes a processing circuit and a memory connected to the processing circuit, wherein the processing circuit uses the memory, according to the foregoing Obtaining, by the motion vector of each of the plurality of decoding completion blocks corresponding to the decoding target block in the dynamic image, to obtain a plurality of candidate motion vectors, and extracting from the plurality of candidate motion vectors relative to the decoding target area N (N is an integer of 2 or more) motion vector predictor candidates of the block, decoding the selection information of the motion vector predictor for identifying the decoding target block, and predicting from the N motion vectors that have been captured In the sub-candidate, the motion vector predictor candidate identified by the decoded selection information is selected as the motion vector predictor, and the difference information indicating the difference of the two motion vectors is decoded, in the foregoing Adding the selected motion vector predictor to the difference indicated by the difference information, thereby deriving the decoding target block a motion vector, and dynamically using the derived motion vector of the decoding target block to dynamically compensate the decoding target block. In the capturing of the N motion vector predictor candidates, a candidate list is generated, and the candidate list is Displaying a list of the plurality of candidate motion vectors, and is a candidate list common to the first extraction method and the second extraction method, and from the plurality of candidate motion vectors shown in the common candidate list, according to the foregoing The first method of extracting M (M is an integer greater than 1 and less than N) motion vector predictor candidate, and from the plurality of candidate motion vectors shown in the common candidate list, according to the second Extracting a method for extracting L (L=N-M) motion vector predictor candidates, wherein the first scooping method is a method for extracting an evaluation result according to each of the plurality of candidate motion vectors, wherein the plurality of The candidate motion vector does not use the image region of the encoding target block, but uses the candidate motion vector of the reconstructed image of the encoded completion region in the aforementioned moving image, the second The method is based on capturing method priority order, the priority is the priority of the plurality of candidate motion vectors specified in advance. Furthermore, the memory may be the frame memory 214 or other memory, and the processing circuit may include, for example, the inter-frame prediction unit 218 and the entropy decoding unit 202.

Thereby, the M motion vector predictor candidates can be extracted according to the first acquisition method, that is, based on the evaluation result by the FRUC. Therefore, the prediction accuracy of the decoding target block, that is, the prediction block can be improved, and the coding efficiency can be improved. Further, in the present embodiment, a candidate list common to the first extraction method and the second extraction method can be generated. That is, whether the M motion vector predictor candidates are extracted according to the first acquisition method or the L motion vector predictor candidates are captured according to the second extraction method according to the predetermined priority order. In the case, you can refer to the common candidate list. As a result, it is not necessary to separately generate a candidate list for the first acquisition method and a candidate list for the second extraction method for the prediction block. Thereby, an increase in processing load can be suppressed and an improvement in coding efficiency can be achieved.

Further, in the acquisition by the second extraction method, the processing circuit may further select, from at least one of the candidate motion vectors, according to the priority order of the evaluation result in the first extraction method. Taking the foregoing L motion vector predictor candidates, wherein the at least one candidate motion vector is among the plurality of candidate motion vectors shown in the foregoing common candidate list, except that the first capture method is used. The remaining at least one candidate motion vector other than the M motion vector predictor candidates.

For example, as shown in FIG. 22, in the extraction according to the second extraction method (for example, step S212ab), the extraction result according to the first extraction method is referred to (for example, step S212aa). Therefore, it is possible to suppress the case where the same candidate motion vector is extracted as the motion vector predictor candidate by the first extraction method and the second extraction method.

Further, the processing circuit may classify the plurality of candidate motion vectors shown in the common candidate list into K (K is an integer of 2 or more) in the capture of the N motion vector predictor candidates. In the acquisition according to the first extraction method, the group is ranked from the plurality of candidate motion vectors shown in the common candidate list, and the first M rankings are ranked in the preferred evaluation result. The candidate motion vector is captured as the M motion vector predictor candidates. In the following method according to the second capture method, the L motions are extracted from one or more candidate motion vectors according to the priority order. a vector predictor candidate, wherein the one or more candidate motion vectors among the aforementioned common candidate lists are at least one group belonging to a group to which each of the M motion vector predictor candidates belongs Either. For example, the evaluation result of each of the plurality of candidate motion vectors is an evaluation result that is smaller as the difference is smaller, wherein the difference is a reconstructed image of the first decoding completion region specified by the candidate motion vector, and 2 The decoding is completed to reconstruct the difference of the image.

For example, as shown in (a) of FIG. 23, a plurality of candidate motion vectors may be classified into K groups different in nature from each other. And, one (M=1) motion vector predictor candidate having the best evaluation result can be extracted from the whole of the K groups, and from the group other than the group to which the motion vector predictor candidate is attached, Another one (L=1) motion vector predictor candidate is retrieved according to a predetermined priority order. Thereby, two (N=2) motion vector predictor candidates which are different in nature from each other and have high prediction precision can be extracted. As a result, the selection range of the motion vector predictor can be expanded, and the possibility that the motion vector predictor with higher prediction accuracy is selected can be improved.

Further, the processing circuit may classify the plurality of candidate motion vectors shown in the common candidate list into K (K is an integer of 2 or more) in the capture of the N motion vector predictor candidates. In the acquisition according to the first extraction method, the group is ranked from the plurality of candidate motion vectors shown in the common candidate list, and the first M rankings are ranked in the preferred evaluation result. The candidate motion vector is extracted as the foregoing M motion vector predictor candidates, and further, from the plurality of candidate motion vectors, the candidate motion vector having the best evaluation result is specified as a next motion vector predictor candidate, wherein the complex number Among the aforementioned common candidate lists, the candidate motion vectors are any one of at least one group other than the group to which each of the M motion vector predictor candidates belongs, in accordance with the second In the acquisition method, the L motion vector predictor candidates are extracted from one or more candidate motion vectors according to the priority order, wherein the one or more candidates In the motion vector candidate list in common among the same group it is part of a particular group and has the next motion vector predictor candidate belongs.

For example, as shown in (b) of FIG. 23, a plurality of candidate motion vectors may be classified into K groups different in nature from each other. And, one (M=1) motion vector predictor candidate having the best evaluation result can be extracted from the whole of the K groups, and from the group other than the group to which the motion vector predictor candidate is attached, A specific next motion vector predictor candidate. Furthermore, another (L=1) motion vector predictor candidate may be retrieved in accordance with the priority order from the same group as the group to which the next motion vector predictor candidate belongs. Thereby, two (N=2) motion vector predictor candidates which are different in nature from each other and have high prediction precision can be extracted. As a result, the selection range of the motion vector predictor can be expanded, and the possibility that the motion vector predictor with higher prediction accuracy is selected can be improved.

These comprehensive or specific aspects may be implemented by systems, devices, methods, integrated circuits, computer programs, or non-transitory recording media such as computer-readable CD-ROMs, or by systems and devices. , any combination of methods, integrated circuits, computer programs, and recording media. [Assembly example]

Fig. 25 is a block diagram showing an example of assembly of the coding apparatus 100 of each of the above embodiments. The encoding device 100 includes a processing circuit 160 and a memory 162. For example, the plurality of components of the encoding device 100 shown in FIG. 1 are assembled by the processing circuit 160 and the memory 162 shown in FIG.

The processing circuit 160 is a circuit that performs information processing and is a circuit that can access the memory 162. For example, processing circuit 160 can be a dedicated or general purpose electronic circuit that encodes dynamic images. Processing circuit 160 can also be a processor such as a CPU. Moreover, the processing circuit 160 can also be an aggregate of a plurality of electronic circuits. Further, for example, the processing circuit 160 may function as a plurality of constituent elements other than the constituent elements for storing information among the plurality of constituent elements of the encoding apparatus 100 shown in FIG.

The memory 162 is a general-purpose or dedicated memory that stores information for encoding a moving image by the processing circuit 160. The memory 162 can be an electronic circuit or can be connected to the processing circuit 160. Further, the memory 162 may be included in the processing circuit 160. Further, the memory 162 may be an aggregate of a plurality of electronic circuits. Further, the memory 162 may be a magnetic disk, a compact disk, or the like, and may be embodied as a memory or a recording medium. Moreover, the memory 162 can also be a non-volatile memory or a volatile memory.

For example, a dynamic image to be encoded may be stored in the memory 162, or a bit string corresponding to the encoded dynamic image may be stored. Further, a program for encoding the moving image by the processing circuit 160 may be stored in the memory 162.

Further, for example, the memory 162 may function as a constituent element for storing information among a plurality of constituent elements of the encoding device 100 shown in FIG. Specifically, the memory 162 can also function as the block memory 118 and the frame memory 122 shown in FIG. More specifically, the processing completion sub-block, the processing completion block, and the processing completion picture may be stored in the memory 162.

Furthermore, in the encoding apparatus 100, all of the plurality of constituent elements shown in FIG. 1 and the like may not be assembled, and all of the above-described plural processing may not be performed. A part of the plurality of constituent elements shown in FIG. 1 and the like may also be included in other devices, and a part of the plurality of processes may be executed by other devices. Further, in the encoding apparatus 100, a part of the plurality of constituent elements shown in FIG. 1 and the like can be assembled, and a part of the plurality of processing units can be performed, whereby the moving image can be appropriately processed with a small amount of encoding. .

Fig. 26 is a block diagram showing an example of assembly of the decoding device 200 of each of the above embodiments. The decoding device 200 includes a processing circuit 260 and a memory 262. For example, the plurality of components of the decoding device 200 shown in FIG. 10 are assembled by the processing circuit 260 and the memory 262 shown in FIG.

The processing circuit 260 is a circuit that performs information processing and is a circuit that can access the memory 262. For example, processing circuit 260 is a general purpose or special purpose electronic circuit that decodes moving images. Processing circuit 260 can also be a processor such as a CPU. Moreover, the processing circuit 260 can also be an aggregate of a plurality of electronic circuits. Further, for example, the processing circuit 260 may function as a plurality of constituent elements other than the constituent elements for storing information among the plurality of constituent elements of the decoding device 200 shown in FIG.

The memory 262 is a general-purpose or dedicated memory that stores information for decoding a moving image by the processing circuit 260. The memory 262 can be an electronic circuit or can be connected to the processing circuit 260. Further, the memory 262 may be included in the processing circuit 260. Further, the memory 262 may be an aggregate of a plurality of electronic circuits. Moreover, the memory 262 may be a magnetic disk, a compact disk, or the like, and may be embodied as a memory or a recording medium. Moreover, the memory 262 can also be a non-volatile memory or a volatile memory.

For example, a bit string corresponding to the encoded dynamic image may be stored in the memory 262, and a dynamic image corresponding to the decoded bit string may also be stored. Further, a program for decoding the moving image by the processing circuit 260 may be stored in the memory 262.

Further, for example, the memory 262 may function as a constituent element for storing information among a plurality of constituent elements of the decoding device 200 shown in FIG. Specifically, the memory 262 can also function as the block memory 210 and the frame memory 214 shown in FIG. More specifically, the processing completion sub-block, the processing completion block, and the processing completion picture may be stored in the memory 262.

Furthermore, in the decoding device 200, all of the plurality of constituent elements shown in FIG. 10 and the like may not be assembled, and all of the plurality of processing may not be performed. A part of the plurality of constituent elements shown in FIG. 10 and the like may be included in other devices, and a part of the plurality of processes may be executed by other devices. Further, in the decoding device 200, a part of the plurality of constituent elements shown in FIG. 10 and the like can be assembled, and a part of the plurality of processing can be performed, whereby the moving image can be appropriately processed with a small amount of encoding. . [Supplementary note]

The encoding device 100 and the decoding device 200 in each of the above embodiments may be used as an image encoding device and an image decoding device, respectively, or may be used as a moving image encoding device and a moving image decoding device, respectively. Alternatively, the encoding device 100 and the decoding device 200 can be utilized as inter-frame prediction devices, respectively. That is to say, the encoding device 100 and the decoding device 200 may correspond to only the inter-frame prediction unit 126 and the inter-frame prediction unit 218, respectively.

Further, in each of the above embodiments, the prediction block is encoded or decoded as the coding target block or the decoding target block, but the coding target block or the decoding target block is not limited to the prediction block, and may be Sub-blocks, and can also be other blocks.

Further, in each of the above embodiments, each constituent element may be constituted by a dedicated hardware, or may be realized by executing a software program suitable for each constituent element. Each component can also be realized by reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory by a program execution unit such as a CPU or a processor.

Specifically, the encoding device 100 and the decoding device 200 may each include a processing circuit (Processing Circuitry) and a storage device (Storage) electrically connected to the processing circuit and accessible by the processing circuit.

The processing circuit includes at least one of a dedicated hardware and a program execution unit, and the storage device is used to perform processing. Further, when the processing circuit includes the program execution unit, the storage device stores the software program executable by the program execution unit.

Here, the software for realizing the encoding device 100, the decoding device 200, and the like of the above embodiments is as follows.

That is, the program is a process for causing the computer to execute the flowchart shown in any one of FIGS. 15 to 18 and FIGS. 20 to 22.

Further, as described above, each constituent element may be an electric circuit. These circuits may be integrally formed as one circuit or may be different circuits. Further, each component may be implemented by a general-purpose processor or by a dedicated processor.

Further, it is also possible to cause another constituent element to execute the processing performed by the specific constituent element. Further, the order of executing the processing may be changed, or a plurality of processing may be performed in parallel. Further, the coding and decoding apparatus may be provided with the coding apparatus 100 and the decoding apparatus 200.

The number of the first and second numbers used in the description may be appropriately replaced. Further, for the constituent elements and the like, the ordinal number can be re-applied or removed.

Although the aspects of the encoding device 100 and the decoding device 200 have been described above based on the respective embodiments, the aspects of the encoding device 100 and the decoding device 200 are not limited to the embodiments. Any form in which the various modifications conceivable by those of ordinary skill in the art to which the present invention pertains can be applied to the embodiments or combined with the constituent elements of the different embodiments can be included without departing from the spirit of the present disclosure. Within the scope of the encoding device 100 and the decoding device 200. (Embodiment 5)

In each of the above embodiments, each of the functional blocks can be generally implemented by an MPU, a memory, or the like. Further, the processing performed by each of the function blocks is usually realized by causing a program execution unit such as a processor to read and execute a software (program) recorded on a recording medium such as a ROM. The software can be distributed by downloading or the like, or can be recorded on a recording medium such as a semiconductor memory. Furthermore, it is of course also possible to implement the functional blocks by hardware (dedicated circuit).

Further, the processing described in each embodiment can be realized by a centralized processing using a single device (system), or can be realized by distributed processing using a plurality of devices. Moreover, the processor executing the above program may be a single or a plurality of processors. That is, it is possible to perform centralized processing or distributed processing.

The present invention is not limited to the above embodiments, and various modifications can be made without departing from the scope of the invention.

Here, an application example of the moving image encoding method (image encoding method) or the moving image decoding method (image decoding method) described in each of the above embodiments and a system using the same will be further described. This system is characterized by an image coding apparatus using an image coding method, an image decoding apparatus using an image decoding method, and an image coding and decoding apparatus including both. Other configurations in the system can be appropriately changed depending on the situation. [usage]

FIG. 27 is a diagram showing the overall configuration of a content supply system ex100 that realizes a content delivery service. The area where the communication service is provided is divided into a desired size, and base stations ex106, ex107, ex108, ex109, and ex110, which are fixed radio stations, are provided in each of the cells.

In the content supply system ex100, each of the machines such as the computer ex111, the game machine ex112, the camera ex113, the home appliance ex114, and the smart phone ex115 can be accessed via the Internet service provider ex102 or the communication network ex104 and the base stations ex106 to ex110. Connect to the Internet ex101. The content supply system ex100 may also be configured to combine and connect any of the above requirements. Alternatively, each device may be directly or indirectly connected to each other via a telephone network or short-range wireless without passing through the base station ex106 to ex110 as a fixed radio station. Further, the streaming server ex103 is connected to each of the devices such as the computer ex111, the game machine ex112, the camera ex113, the home appliance ex114, and the smart phone ex115 via the Internet ex101 or the like. Further, the streaming server ex103 is connected to a terminal or the like in a hot spot in the aircraft ex117 via the satellite ex116.

Furthermore, it is also possible to use a wireless access point or hotspot instead of the base station ex106~ex110. Further, the streaming server ex103 can directly connect to the communication network ex104 without passing through the Internet ex101 or the Internet service provider ex102, or can directly connect to the aircraft ex117 without passing through the satellite ex116.

The camera ex113 is a machine that can perform still picture shooting and dynamic picture shooting such as a digital camera. In addition, the smart phone ex115 is a smart phone, a mobile phone, or a PHS (Personal Handyphone System) corresponding to a mobile communication system generally called 2G, 3G, 3.9G, 4G, and 5G. Handheld phone system)) and so on.

The home appliance ex118 may be a refrigerator or a machine included in a household fuel cell cogeneration system.

In the content supply system ex100, the terminal having the photographing function is connected to the streaming server ex103 via the base station ex106 or the like, thereby making it possible to transmit live live or the like. In live delivery, the terminal (computer ex111, game machine ex112, camera ex113, home appliance ex114, smart phone ex115, terminal in aircraft ex117, etc.) is a static or dynamic picture content captured by the user using the terminal. Performing the encoding process described in each of the above embodiments, and multiplexing the image data obtained by the encoding and the sound data encoded by the sound corresponding to the image, and transmitting the obtained data to Streaming server ex103. That is, each terminal functions as an image coding apparatus according to an aspect of the present invention.

On the other hand, the streaming server ex103 performs streaming of the content material, which is the content material transmitted to the client (client) that is required. The client is a computer ex111, a game machine ex112, a camera ex113, a home appliance ex114, a smart phone ex115, and a terminal in the aircraft ex117, which can decode the data that has undergone the above-described encoding processing. Each machine that has received the transmitted data decodes and plays the received data. That is, each machine functions as an image decoding device according to an aspect of the present invention. [Distributed processing]

Moreover, the streaming server ex103 can also be a plurality of servers or a plurality of computers, and the data is distributed and processed or recorded for transmission. For example, the streaming server ex103 can be implemented by a CDN (Contents Delivery Network), or by a plurality of edge servers distributed between the world and the edge server. The network is used to implement content delivery. On the CDN, physically similar edge servers are dynamically allocated in response to the client. Also, the delay can be reduced by caching and sending the content to the edge server. Moreover, since the communication state can be changed when an error occurs or the flow rate is increased, the processing is dispersed by a plurality of edge servers, or the transmission body is switched to another edge server to bypass the obstacle that has occurred. The part of the network is continuously transmitted, so high-speed and stable transmission can be achieved.

Further, not only the distributed processing of the transmission itself but also the encoding processing of the photographed data may be performed at each terminal, or may be performed on the server side or shared with each other. As an example, generally, in the encoding process, the processing loop is performed twice. In the first loop, the complexity or amount of encoding of the image in the frame or scene unit is detected. Further, in the second loop, processing for maintaining image quality and improving encoding efficiency is performed. For example, by causing the terminal to perform the first encoding process and the second encoding process by the server side receiving the content, the processing load on each terminal can be reduced and the quality and efficiency of the content can be improved. At this time, as long as there is a request for receiving and decoding in a near-instant manner, other terminals can also be used to receive and play the first coded completion data that the terminal has performed, so that more flexible instant transmission can be achieved.

As another example, the camera ex113 or the like performs feature amount extraction from an image, and compresses and associates the material related to the feature amount as meta data to the server. The server performs, for example, compression of the meaning of the image corresponding to the importance of the object from the feature amount and switching the quantization precision. The feature quantity data is particularly effective for the accuracy and efficiency improvement of motion vector prediction when the server is again compressed. Further, it is also possible to perform simple coding such as VLC (Variable Length Coding) in the terminal, and perform processing with a large load such as CABAC (Critical Reference Adaptive Binary Arithmetic Coding) on the server.

Further, as another example, in a sports field, a shopping mall, a factory, or the like, a plurality of pieces of image data in which almost the same scene is captured by a plurality of terminals may exist. In this case, it is possible to use, for example, a GOP (Group of Picture) unit, a picture unit, or a picture by using a plurality of terminals that have been photographed, and other terminals and servers that are not photographed as needed. The tile units and the like are assigned to each other to perform a distributed process. Thereby, the delay can be reduced, and real-time can be more realized.

Moreover, since the plurality of image data are almost the same scene, the server can be managed and/or instructed to refer to the image data captured by each terminal. Alternatively, the server may be configured to receive the coded completion data from each terminal, change the reference relationship between the plurality of data, or correct or replace the picture itself and re-encode. Thereby, it is possible to generate a stream in which the quality and efficiency of one piece of data have been improved.

Moreover, the server can also transmit the image data after transcoding the encoding method of the changed image data. For example, the server can also convert the encoding method of the MPEG class into a VP class, and can also convert H.264 to H.265.

In this way, the encoding process can be performed by the terminal or by one or more servers. Therefore, although the following describes the main body of the processing using the descriptions such as "server" or "terminal", some or all of the processing performed by the server may be performed at the terminal, or the server may perform the processing at the terminal. Part or all of the treatment. Further, regarding the above, the same applies to the decoding process. [3D, multi-angle]

In recent years, the following practices are gradually increasing, that is, different scenes photographed by terminals such as a plurality of cameras ex113 and/or a smart phone ex115 that are almost synchronized with each other, or the same scene will be photographed from different angles. The images or images are integrated and utilized. The image captured by each terminal is integrated based on the relative positional relationship between the acquired terminals or the area in which the feature points of the video are identical.

The server not only encodes the two-dimensional moving image, but also encodes the static image and transmits it to the receiving terminal automatically or at a time specified by the user according to scene analysis of the moving image or the like. In addition, when the server can obtain the relative positional relationship between the photographing terminals, the server can generate not only a two-dimensional moving image but also a three-dimensional shape of the scene from images captured from different angles in the same scene. Furthermore, the server can also encode the three-dimensional data generated by the point cloud, and can also capture images from a plurality of terminals according to the use of the three-dimensional data to identify or track the result of the person or the target. Selecting, or reconstituting and generating an image to be transmitted to the receiving terminal.

In this way, the user can arbitrarily select the respective images corresponding to the respective imaging terminals to enjoy the scene, and can also enjoy the content of the image obtained by cutting out arbitrary viewpoints from the three-dimensional data reconstructed from the plurality of images or images. In addition, similar to the image, the sound can be collected from a plurality of different angles, and the server can also cooperate with the image to multiplex and transmit the sound and image from a specific angle or space.

In addition, in recent years, contents such as Virtual Reality (VR) and Augmented Reality (AR) have become popular in the real world and the virtual world. In the case of a VR image, the server can also create viewpoint images for the right eye and the left eye, and allow for reference between the viewpoint images by Multi-View Coding (MVC) or the like. Encoding may also be encoded as a different stream without cross-referencing. When decoding different streams, they can be synchronized with each other to play back, in order to reproduce the virtual three-dimensional space in response to the user's viewpoint.

In the case of an AR image, the server superimposes the virtual object information on the virtual space with the camera information in the real space according to the position of the three-dimensional or the movement of the user's viewpoint. The decoding device can also acquire or maintain virtual object information and three-dimensional data, and generate a two-dimensional image according to the movement of the user's viewpoint and smoothly connect to create overlapping data. Alternatively, the decoding device may transmit the movement of the user's viewpoint to the server in addition to the virtual object information, and the server cooperates with the movement of the viewpoint received from the three-dimensional data held by the server. The overlapping data is created and the overlapping data is encoded and sent to the decoding device. Furthermore, the overlapping data may have an alpha value indicating the transmittance in addition to the RGB, and the server sets the alpha value of the portion other than the target created by the three-dimensional data to 0, etc., and penetrates in the portion. Encoding in the state. Alternatively, the server may also set the RGB value of the specified value as the background in the form of a chroma key, and the part other than the generated target is formed as the background color.

Similarly, the decoding process of the transmitted data can be performed on the client side, that is, on each terminal, or on the server side, or can be shared with each other. As an example, a terminal may temporarily transmit the reception request to the server, receive the content requested in the other terminal, perform decoding processing, and transmit the decoded signal to the device having the display. By dispersing the processing and selecting the appropriate content without relying on the performance of the communicable terminal itself, it is possible to play back a good picture quality. Further, as another example, a large-sized image data may be received by a TV or the like, and a part of a region such as a tile divided by the image may be decoded and displayed on the personal terminal of the appreciator. By this, the overall picture can be shared, and the area in which it is responsible or the area in which it is desired to be confirmed in more detail can be confirmed.

In addition, in the future, it is expected that, in the case where the wireless communication at a short distance, a medium distance, or a long distance is used in a plurality of places, the transmission system specifications such as MPEG-DASH are used, and the connection is in progress. The communication switches the appropriate data while seamlessly receiving the content. Thereby, the user can freely select and instantly switch not only the terminal of the terminal, but also the display device or the display device provided in the display or the like. Further, it is possible to switch between the terminal to be decoded and the terminal to be displayed, in accordance with the position information of the user, and the like. Thereby, it is also possible to move the map information while displaying the map information on a part of the wall or the ground of the adjacent building in which the displayable component is embedded during the movement to the destination. Further, it is also possible to make the encoded data cached to a server that can be accessed from the receiving terminal in a short time, or to an edge server in a content delivery server, etc. The bit rate of the received data is switched according to the ease of accessing the encoded data on the network. [Adjustable coding]

The switching of the content is described using an adjustable stream shown in Fig. 25, and the moving picture encoding method shown in each of the above embodiments is applied and compression-encoded. Although the server has a plurality of streams of the same content but different qualities as individual streams, it may be configured to be encoded by layering as shown in the figure, and the time/space can be adjusted. Streaming, and using the characteristics of the stream to switch content. In other words, by deciding which layer to decode to the external factor such as the inherent factor of the decoding side and the state of the communication band, the decoding side can freely switch between the low-resolution content and the high-resolution content. To decode. For example, when you want to watch the video that is viewed by the smartphone ex115 on the mobile device after going home, the machine only needs to decode the same stream to different layers, so the servo can be reduced. The burden on the side of the device.

Further, as described above, in addition to the configuration in which the picture is encoded for each layer and the enhancement layer has an enhancement layer on the base layer, the enhancement layer may be included in the image according to the image. Meta-information such as statistical information, and the decoding side super-analyzes the picture of the base layer based on the meta-information, thereby generating high-quality content. The super-analysis may be any one of an increase in the SN ratio and an increase in the resolution in the same resolution. The meta information includes: information for linear or non-linear filter coefficients used in a specific super-resolution processing, or information on filter values used in a specific super-resolution processing, parameter values in mechanical learning or least squares operations, and the like.

Alternatively, the picture may be divided into tiles or the like in accordance with the meaning of the object or the like in the image, and the decoding side may select a block to be decoded, thereby decoding only a part of the area. Moreover, by storing the attributes of the target (person, car, ball, etc.) and the position within the image (coordinate position in the same image, etc.) as meta information, the decoding side can specify the desired target based on the meta information. Location and decide which tile contains the target. For example, as shown in FIG. 29, the meta information may be saved using a data save structure different from the pixel data, such as an SEI message in HEVC. This meta information indicates, for example, the position, size, or color of the main target.

Further, the meta information may be stored in a unit composed of a plurality of pictures such as a stream, a sequence, or a random access unit. Thereby, the decoding side can obtain the time when the specific person appears in the image, and the like, and by comparing with the information of the picture unit, the picture in which the target exists and the position of the target in the picture can be specified. [Optimization of the page]

FIG. 30 is a view showing an example of a display screen of a web page in the computer ex111 or the like. FIG. 31 is a view showing an example of a display screen of a web page in the smartphone ex115 or the like. As shown in FIG. 30 and FIG. 31, when a web page includes a plurality of links to image content, that is, a link image, the appearance thereof differs depending on the components to be viewed. When a plurality of link images are visible on the screen, the display device (decoding device) until the user explicitly selects the link image or the link image approaches the center of the screen or the entire link image enters the screen. ) is to display a still picture or an I picture (intra picture) having each content as a link image, or to display an image in the form of a gif animation in a plurality of still pictures or I pictures, or to receive only basic Layer to decode and display images.

In the case where the linked image has been selected by the user, the display device will set the base layer to be the highest priority for decoding. Furthermore, the display device may be decoded to the enhancement layer as long as it has information indicating that the content belongs to the adjustable content in the HTML constituting the web page. Moreover, in order to guarantee the immediacy, the display device can decode and display only the forward reference picture (I picture (in-frame coded picture), P picture (before selection) or the communication band is very tight. Predictive Picture, B-picture (Bidirectionally Predictive Picture), to reduce the delay between the decoding time and the display time of the first picture (from the decoding of the content to the start of the display) The delay between). Moreover, the display device may also deliberately ignore the reference relationship of the pictures, and set all the B pictures and P pictures as forward references to roughly decode, and increase the received pictures over time to perform normal decoding. [Automatic driving]

In addition, when a still image or video data such as two-dimensional or three-dimensional map information is transmitted for automatic driving or driving support of a car, the receiving terminal may be in addition to image data belonging to one or more layers. Information such as weather or construction is also received as meta information, and is decoded corresponding to these. Furthermore, the meta information can belong to the layer, and can be simply multiplexed with the image data.

At this time, since the car including the receiving terminal, the drone or the airplane moves, the base station ex106~ex110 can be switched by the receiving terminal transmitting the position information of the receiving terminal when receiving the request. Seamless reception and decoding on one side. Moreover, the receiving terminal can dynamically switch to what extent the meta-information is to be received, or to what extent the map information is to be updated, depending on the user's choice, the state of the user, or the state of the communication band.

As described above, in the content supply system ex100, the client can receive the encoded information transmitted by the user in real time, and decode and play the information. [Send of personal content]

Further, in the content supply system ex100, not only the high-quality and long-time content from the video transmission provider, but also unicast or multicast transmission of the low-quality and short-time content of the individual. It can also be done. Moreover, this personal content is considered to continue to increase in the future. In order to make personal content into better content, the server can also perform encoding processing after performing editing processing. This can be achieved by, for example, the following constitution.

The server will perform recognition processing such as shooting error, scene search, meaning analysis, and target detection from the original image or the encoded completion data immediately or after the shooting. Moreover, the server performs the following editing manually or automatically according to the identification result: correcting out-of-focus or jitter, deleting scenes with low importance such as low or unfocused scenes, highlighting the edge of the target, and changing Hue and so on. The server will encode the edited data based on the edited results. Moreover, when the shooting time is too long, the viewing rate will be reduced. The server will automatically follow the image processing results, such as the above, not only for the scenes with low importance, but also for dynamic comparison. A small scene or the like is clipped so that the shooting time becomes a content within a specific time range. Alternatively, the server may also generate and encode the digest based on the result of the semantic analysis of the scene.

In addition, in the personal content, there may be cases in which the content of copyright, copyright, or portrait rights is infringed, and it is inconvenient for the individual to share the range beyond the scope of sharing. Case. According to this, for example, the server can also intentionally change the face of the peripheral portion of the screen or the inside of the house to an unfocused image and encode it. Further, the server can recognize whether or not a face of a person different from the person registered in advance is captured in the image to be encoded, and when the image is captured, a process of mosaicing the face portion or the like is performed. Alternatively, as a pre-coding process or a post-processing, the user may specify a person or a background area to be processed in the image based on the viewpoint of copyright, etc., and then cause the server to replace the designated area with another image, Or the focus is blurred and the like. If it is a character, you can replace the image of the part of the face while tracking the character in the moving image.

Moreover, since the audiovisual content of the personal content having a small amount of data has a strong demand for immediacy, the decoding apparatus first receives the base layer and decodes and plays it first, depending on the frequency bandwidth. The decoding device can also receive the enhancement layer during this period, and if the playback is played twice or more in a loop, the enhancement layer is also included to play the high-quality image. In this way, as long as the stream of the adjustable encoding is performed, it is possible to provide a dynamic picture which is rough when the stage is not selected or is first seen, but the stream is gradually smart and the image is changed. Good experience. In addition to the tunable coding, even if the rough stream of the first play and the second stream coded with reference to the first motion picture are configured as one stream, the same experience can be provided. [Other use cases]

Moreover, these encoding or decoding processes are generally processed in the LSI ex500 included in each terminal. The LSI ex500 may be a one chip or may be formed of a plurality of wafers. Furthermore, the software for encoding or decoding a moving image can be mounted to a recording medium (CD-ROM, flexible disk, or hard disk) that can be read on a computer ex111 or the like, and used. The software performs encoding or decoding processing. In addition, when the smart phone ex115 is attached with a camera, the dynamic image data acquired by the camera can also be transmitted. The dynamic map data at this time is encoded and processed by the LSI ex500 which the smart phone ex115 has.

Furthermore, the LSI ex500 can also be configured to download and activate the application software. At this time, the terminal first determines whether the terminal corresponds to the encoding mode of the content or whether it has the execution capability of the specific service. When the terminal does not have a coding mode corresponding to the content, or does not have the execution capability of the specific service, the terminal downloads the codec or the application software, and then acquires and plays the content.

Further, the present invention is not limited to the content supply system ex100 via the Internet ex101, and the video encoding device (image encoding device) or the moving image decoding device (image decoding) of each of the above embodiments may be incorporated in the digital broadcasting system. At least one of the devices). Since the multiplexed data in which the video and audio are multiplexed is carried by the satellite or the like to be transmitted and received by the radio wave for broadcasting, it is more suitable for the unicast configuration of the content supply system ex100. The difference in multicast, but there are still similar applications for encoding processing and decoding processing. [Hardware composition]

Figure 32 is a diagram showing the smartphone ex115. FIG. 33 is a diagram showing an example of the configuration of the smartphone ex115. The smartphone ex115 includes an antenna ex450 for transmitting and receiving radio waves between the base station ex110, a camera portion ex465 capable of capturing video and still images, and an image captured by the camera unit ex465 and received at the antenna ex450. The display unit ex458 of the decoded data such as the video. The smart phone ex115 further includes an operation unit ex466 such as a touch panel, an audio output unit ex457 for outputting a sound or an audio speaker, an audio input unit ex456 for inputting a sound microphone, and the like, and can store the captured image. Or static picture, recorded sound, received image or still picture, mailed data such as mail, or memory part ex467 of decoded data, and slot part ex464 as the interface between the face and SIMex468, the SIMex468 It is used for specific users and is authenticated by accessing various materials based on the Internet. Further, an external memory may be used instead of the memory portion ex467.

Further, the main control unit ex460 such as the display unit ex458 and the operation unit ex466 is integrally connected to the power supply circuit unit ex461, the operation input control unit ex462, the video signal processing unit ex455, the camera interface ex463, and the display via the bus bar ex470. The control unit ex459, the modulation/demodulation unit ex452, the multiplex/separation unit ex453, the audio signal processing unit ex454, the slot unit ex464, and the memory unit ex467.

When the power supply unit ex461 is set to the on state by the user's operation, the power supply unit ex115 starts the smart phone ex115 in an operable state by supplying power to each unit from the battery pack.

The smart phone ex115 performs processing such as call and data communication based on the control of the main control unit ex460 having a CPU, a ROM, and a RAM. At the time of the call, the audio signal processing unit ex454 converts the audio signal received by the voice input unit ex456 into a digital audio signal, and performs spreading processing by the modulation/demodulation unit ex452, and then transmits/receives the part After the digital analog conversion processing and the frequency conversion processing are performed, the transmission is performed through the antenna ex450. Further, the received data is amplified, frequency conversion processing, and analog-to-digital conversion processing are performed, and demodulation processing is performed by the modulation/demodulation unit ex452, and then converted into an analog sound signal by the audio signal processing unit ex454, and then the sound output unit is used. Ex457 will output it. In the data communication mode, the text, the static image, or the video data is sent to the main control unit ex460 through the operation input control unit ex462 by the operation of the operation unit ex466 or the like of the main unit, and the transmission and reception processing is performed in the same manner. . When the video, the still picture, or the video and audio are transmitted in the data communication mode, the video signal processing unit ex455 is stored in the memory unit ex467 by the moving picture coding method described in each of the above embodiments. The image signal or the image signal input from the camera unit ex465 is compression-encoded, and the encoded image data is sent to the multiplex/separation unit ex453. Further, the audio signal processing unit ex454 encodes the audio signal received by the audio input unit ex456 when the camera unit ex465 captures a video or a still picture, and sends the encoded audio data to the multiplex/separation unit ex453. The multiplexer/separation unit ex453 multiplexes the coded video data and the coded voice data in a predetermined manner, and uses a modulation/demodulation unit (modulation/demodulation circuit unit) ex452 and a transmission/reception unit. The ex451 performs modulation processing and conversion processing, and transmits it through the antenna ex450.

In the case where an image attached to an e-mail or a web chat or an image linked to a web page has been received, in order to decode the multiplexed data received through the antenna ex450, the multiplex/separation unit ex453 separates the multiplexed data. And dividing the multiplexed data into a bit stream of the bit stream and the sound data of the image data, and then supplying the encoded image data to the image signal processing unit ex455 through the sync bus ex470, and supplying the encoded sound data to the The audio signal processing unit ex454. The video signal processing unit ex455 decodes the video signal by the moving picture decoding method corresponding to the moving picture coding method described in each of the above embodiments, and displays the linked moving picture from the display unit ex458 via the display control unit ex459. The image or static image contained in the file. Further, the audio signal processing unit ex454 decodes the audio signal and outputs the sound from the audio output unit ex457. Furthermore, since real time streaming has become widespread, it is also possible to play sounds in places where the society is not suitable for sound generation according to the user's situation. Therefore, as an initial value, it is preferable to play only the video material without playing the audio signal. It is also possible to play the sound synchronously only when the user performs an operation such as clicking on the image data.

Here, although the smartphone ex115 has been described as an example, the following three types of assembly can be considered as the terminal: in addition to the transmission/reception type terminal having both the encoder and the decoder, A transmitting terminal having only an encoder, and a receiving terminal having only a decoder. In addition, in the digital broadcasting system, although it is configured to receive or transmit multiplexed data that has been multiplexed with sound data and the like in the video data, the multiplexed data may be in addition to the sound data. The multiplexed texts and the like associated with the images are multiplexed, and the image data itself can be received or transmitted instead of the multiplexed data.

In addition, although the main control unit ex460 including the CPU controls the encoding or decoding process, the terminal has a GPU. Therefore, it is also possible to use a memory that is common to both the CPU and the GPU, or to manage the address to be a memory that can be used in common, and to utilize the performance of the GPU to process a wider area together. This can shorten the encoding time, ensure immediacy, and achieve low latency. In particular, in the case where the CPU is not used, the GPU is used to perform motion search, deblock filter, SAO (Sample Adaptive Offset), conversion, quantization, and the like in units of pictures and the like. When it is processed, it is efficient. Industrial availability

The present disclosure can be utilized in, for example, televisions, digital video recorders, car navigation systems, mobile phones, digital cameras, digital cameras, video conferencing systems, or electronic mirrors.

10~23‧‧‧ Block

100‧‧‧ coding device

102‧‧‧ Division

104‧‧‧Subtraction Department

106‧‧‧Transition Department

108‧‧‧Quantity Department

110‧‧‧ Entropy Coding Department

112, 204‧‧‧ inverse quantization

114, 206‧‧‧ inverse conversion department

116, 208‧‧ Addition Department

118, 210‧‧‧ Block memory

120, 212‧‧‧Circuit Filtering Department

122, 214‧‧‧ box memory

124, 216‧‧‧ In-frame forecasting department

126, 218‧‧‧Inter-frame prediction department

128, 220‧‧‧Predictive Control Department

160, 260‧‧‧ processing circuit

162, 262‧‧‧ memory

200‧‧‧ decoding device

202‧‧‧ Entropy Decoding Department

Cur block‧‧‧ current block

Cur Pic‧‧‧ Current picture

Ex100‧‧‧Content Supply System

Ex101‧‧‧Internet

Ex102‧‧‧Internet Service Provider

Ex103‧‧‧Streaming server

Ex104‧‧‧Communication Network

Ex106, ex107, ex108, ex109, ex110‧‧‧ base station

Ex111‧‧‧ computer

Ex112‧‧‧game machine

Ex113‧‧‧ camera

Ex114‧‧‧Home appliances

Ex115‧‧‧Smart mobile phone

Ex116‧‧‧ satellite

Ex117‧‧ aircraft

Ex450‧‧‧Antenna

Ex451‧‧‧Transmission/receiving department

Ex452‧‧‧Modulation/Demodulation Department

Ex453‧‧‧Multiplex/Separation Department

Ex454‧‧‧Sound Signal Processing Department

Ex455‧‧‧Image Signal Processing Department

Ex456‧‧‧Sound Input Department

Ex457‧‧‧Sound Output Department

Ex458‧‧‧Display Department

Ex459‧‧‧Display Control Department

Ex460‧‧‧Main Control Department

Ex461‧‧‧Power Circuit Department

Ex462‧‧‧Operation Input Control Department

Ex463‧‧· Camera face

Ex464‧‧‧Slots

Ex465‧‧‧ camera department

Ex466‧‧‧Operation Department

Ex467‧‧‧ memory department

Ex468‧‧‧SIM

Ex470‧‧‧ busbar

MV0, MV1, MVx0, MVy0, MVx1, MVy1, v0, v1‧‧‧ motion vectors

Ref0, Ref1‧‧‧ reference picture

S101~S105, S111~S115, S201~S204, S211~S214, S202a, S202b, S212a, S212b, S202aa, S202ab, S212aa, S212ab, S301~S306‧‧

TD0, TD1‧‧‧ distance

Fig. 1 is a block diagram showing a functional configuration of an encoding apparatus according to a first embodiment. Fig. 2 is a view showing an example of block division in the first embodiment. Figure 3 is a table showing a conversion basis function corresponding to each conversion type. Fig. 4A is a view showing an example of the shape of a filter used in ALF. Fig. 4B is a view showing another example of the shape of the filter used in the ALF. Fig. 4C is a view showing another example of the shape of the filter used in the ALF. FIG. 5 is a diagram showing 67 in-frame prediction modes in the in-frame prediction. Fig. 6 is a diagram for explaining pattern matching (bidirectional matching) between two blocks along a motion trajectory. FIG. 7 is a diagram for explaining pattern matching (template matching) between a template in a current picture and a block in a reference picture. Fig. 8 is a view for explaining a model assuming constant-speed linear motion. Figure 9 is a diagram for explaining the derivation of motion vectors of sub-block units according to motion vectors of a plurality of adjacent blocks. Fig. 10 is a block diagram showing the functional configuration of a decoding apparatus according to the first embodiment. Figure 11 is a flow chart showing the dynamic compensation performed by other encoding devices that are the basis of the present disclosure. Figure 12 is a flow chart showing the dynamic compensation performed by other decoding devices that are the basis of the present disclosure. FIG. 13 is a view for explaining an example of a method of calculating an evaluation value. FIG. 14 is a view for explaining another example of the method of calculating the evaluation value. Fig. 15 is a flow chart showing an example of dynamic compensation performed by the coding apparatus in the second embodiment. Fig. 16 is a flow chart showing an example of dynamic compensation performed by the decoding device in the second embodiment. Fig. 17 is a flow chart showing another example of dynamic compensation performed by the encoding device in the second embodiment. Fig. 18 is a flowchart showing another example of dynamic compensation performed by the decoding device in the second embodiment. 19(a) to 19(c) are diagrams for explaining a method of extracting N motion vector predictor candidates from a plurality of candidate motion vectors in the second embodiment. Fig. 20 is a flowchart showing a method of selecting a motion vector predictor by the encoding device and the decoding device in the third embodiment. Fig. 21 is a flow chart showing an example of dynamic compensation performed by the coding apparatus in the fourth embodiment. Fig. 22 is a flowchart showing an example of dynamic compensation performed by the decoding device in the fourth embodiment. 23(a) and 23(b) are diagrams for explaining a method of capturing a motion vector predictor candidate in the fourth embodiment. Figs. 24(a) and (b) are diagrams showing an example of a common candidate list in the fourth embodiment. Fig. 25 is a block diagram showing an example of assembly of an encoding apparatus according to each embodiment. Fig. 26 is a block diagram showing an example of assembly of a decoding device according to each embodiment. Fig. 27 is a view showing the overall configuration of a content supply system that implements a content delivery service. Fig. 28 is a view showing an example of a coding structure in the case of adjustable coding. Fig. 29 is a view showing an example of a coding structure in the case of adjustable coding. 30 is a diagram showing an example of a display screen of a web page. 31 is a diagram showing an example of a display screen of a web page. 32 is a diagram showing an example of a smart phone. Fig. 33 is a block diagram showing a configuration example of a smart phone.

Claims (8)

An encoding device is an encoding device that encodes a moving image, and includes a processing circuit and a memory connected to the processing circuit, wherein the processing circuit uses the memory to generate a coding target region in the moving image Obtaining a plurality of motion vector vectors of each of the plurality of coded completion blocks corresponding to the block to obtain a plurality of candidate motion vectors, and extracting at least one motion vector predictor of the foregoing coding target block from the plurality of candidate motion vectors a candidate, and deriving a motion vector of the foregoing coding target block with reference to a reference picture included in the foregoing dynamic image, and predicting a motion vector predictor among the at least one motion vector prediction sub-candidate that has been extracted, and the derived Encoding the difference of the motion vector of the encoding target block, and dynamically using the derived motion vector of the encoding target block to dynamically compensate the coding target block, in the capturing of the at least one motion vector predictor candidate , encoding the mode information used to identify the capture method, and from the first capture method and In the second extraction method, for the encoding target block, the capturing method identified by the mode information is selected, and the at least one motion vector predictor candidate is captured according to the selected capturing method. The first extraction method is a method for extracting an evaluation result of each of the plurality of candidate motion vectors, wherein the plurality of candidate motion vectors do not use the image region of the encoding target block, but the foregoing The candidate motion vector of the reconstructed image of the coded completion region in the motion image, and the second capture method is a capture method according to a priority order, which is a priority order that has been previously defined for the plurality of candidate motion vectors. The encoding device of claim 1, wherein in the encoding of the mode information, the mode information is encoded in a header region of any one of a sequence layer, a picture layer, and a slice layer in the stream of the moving image. Where the mode information is information for identifying a method of capturing the blocks included in the foregoing layer. The encoding device of claim 1, wherein in the encoding of the foregoing mode information, the mode information is encoded for each block included in the dynamic image, wherein the mode information is used to identify the region. Information on the method of block extraction. A decoding device is a decoding device that decodes a coded moving image, and includes a processing circuit and a memory connected to the processing circuit, wherein the processing circuit uses the memory and is based on the dynamic image Decoding a motion vector of each of the plurality of decoding completion blocks corresponding to the target block to obtain a plurality of candidate motion vectors, and extracting at least one motion of the decoding target block from the plurality of candidate motion vectors a vector predictor candidate, and decoding difference information showing differences between the two motion vectors, and adding the at least one motion vector predictor candidate that has been captured to the difference indicated by the decoded difference information a motion vector predictor in which the motion vector of the decoding target block is derived, and the decoded target block is dynamically compensated by using the derived motion vector of the decoding target block, and the at least one motion vector prediction is performed. In the capture of the sub-candidate, the mode information used to identify the capture method is decoded, and from the first In the method and the second extraction method, for the decoding target block, a capture method identified by the decoded mode information is selected, and the at least one of the at least one is captured according to the selected extraction method. a motion vector predictor candidate, wherein the first method of capturing is a method of extracting an evaluation result according to each of the plurality of candidate motion vectors, wherein the plurality of candidate motion vectors do not use an image region of the encoding target block, Instead, the candidate motion vector of the reconstructed image of the coded completion region in the moving image is used, and the second capture method is a capture method according to a priority order in which the plurality of candidate motion vectors are specified in advance. Priority. The decoding device of claim 4, wherein in the decoding of the mode information, the mode information is from a header region of any one of a sequence layer, a picture layer, and a slice layer in the stream of the moving image. Decoding, wherein the mode information is information for identifying a method of capturing the blocks included in the aforementioned layer. The decoding device of claim 4, wherein in the decoding of the foregoing mode information, the mode information is decoded for each block included in the dynamic image, wherein the mode information is used to identify the region. Information on the method of block extraction. An encoding method is an encoding method for encoding a moving image, the encoding method is to obtain a complex number according to a motion vector of each of a plurality of encoding completion blocks corresponding to the encoding target block in the moving image. And selecting at least one motion vector predictor candidate of the foregoing coding target block from the plurality of candidate motion vectors, and deriving the foregoing coding target block by referring to a reference picture included in the dynamic image. a motion vector, and encoding, for the motion vector predictor among the at least one motion vector predictor candidate that has been captured, and the derived motion vector difference of the encoding target block, using the derived encoding object The motion vector of the block dynamically compensates the foregoing coding target block, and in the extraction of the at least one motion vector predictor candidate, the mode information for identifying the capture method is encoded, and from the first frame In the obtaining method and the second capturing method, for the encoding target block, the identification by the aforementioned mode information is selected. Extracting the method, and according to the selected extraction method, the at least one motion vector predictor candidate is captured, and the first capturing method is based on the evaluation result of each of the plurality of candidate motion vectors. a method, wherein the plurality of candidate motion vectors do not use the image region of the encoding target block, but a candidate motion vector of the reconstructed image of the encoded completion region in the moving image is used, and the second capturing method is According to the prioritized retrieval method, the priority order is a prioritized order of the plurality of candidate motion vectors. A decoding method is a decoding method for decoding an encoded moving image, the decoding method being based on a motion vector of each of a plurality of decoding completion blocks corresponding to a decoding target block in the foregoing moving image, Obtaining a plurality of candidate motion vectors, and extracting at least one motion vector predictor candidate of the decoding target block from the plurality of candidate motion vectors, and decoding differential information showing differences between the two motion vectors, And adding, to the difference indicated by the decoded difference information, a motion vector predictor among the at least one motion vector predictor candidate that has been captured, thereby extracting a motion vector of the decoding target block, and utilizing Deriving the motion vector of the decoding target block to dynamically compensate the decoding target block, and decoding the mode information for identifying the capturing method in the capturing of the at least one motion vector predicting sub-candidate And in the first extraction method and the second extraction method, for the decoding target block, the foregoing is selected by decoding Obtaining a method for capturing the at least one motion vector predictor candidate according to the selected method of capturing, the first capturing method is based on each of the plurality of candidate motion vectors a method for extracting the evaluation result, wherein the plurality of candidate motion vectors do not use the image region of the encoding target block, but a candidate motion vector of the reconstructed image of the encoded completion region in the moving image is used, The second extraction method is a method of capturing priorities according to a priority order, which is a predetermined priority order for the plurality of candidate motion vectors.