JP2021002876A - Decoding method and encoding method - Google Patents

Abstract

To provide a decoding method and a decoding device, capable of improving encoding efficiency.SOLUTION: A decoding method according to one embodiment includes receiving an input of encoded data including: a syntax that is called from a syntax at a level higher than a slice level and is used for weighted motion compensation and that describes a plurality of parameters obtained using a common index commonly used in a first reference image list and a second reference image list; and a syntax that is at a level higher than the slice level and that describes a syntax element including a maximum value that the common index can take. The decoding method further includes performing decoding processing, including the weighted motion compensation, using reference images and the plurality of parameters obtained using the common index with the maximum value as an upper limit.SELECTED DRAWING: Figure 1

Description

Embodiments of the present invention relate to decoding methods and decoding devices.

In recent years, an image coding method with significantly improved coding efficiency has been introduced by ITU-T (International).
In collaboration with the Telecommunication Union Telecommunication Standardization Sector) and ISO (International Organization for Standardization) / IEC (International Electrotechnical Commission), ITU-T REC. H. It is recommended as 264 and ISO / IEC 14496-10 (hereinafter referred to as "H.264").

H. 264 discloses an inter-prediction coding method that eliminates redundancy in the time direction and realizes high coding efficiency by performing fractional-accuracy motion compensation prediction using a coded image as a reference image. ing.

In addition, a method for encoding moving images including fade and dissolve effects with higher efficiency than the inter-prediction coding method in ISO / IEC MPEG (Moving Picture Experts Group) -1,2,4 has also been proposed. In this method, as a framework for predicting a change in brightness in the time direction, motion compensation prediction with fractional accuracy is performed for an input moving image having brightness and two color differences. Then, the predicted image is multiplied by the weighting coefficient and the offset is added by using the index indicating the combination including the reference image, the luminance and the weighting coefficient for each of the two color differences, and the brightness and the offset for each of the two color differences.

Japanese Unexamined Patent Publication No. 2004-7377

However, in the prior art as described above, in a bidirectional prediction slice in which bidirectional prediction can be selected, when weighted motion compensation is performed using two indexes having the same reference image but different reference image numbers. Indexes with the same value may be encoded twice, which may reduce the coding efficiency. An object to be solved by the present invention is to provide a decoding method and a decoding device capable of improving the coding efficiency.

The decoding method of the embodiment is a first syntax that is called from a higher syntax at a level higher than the slice level and is used for weighted motion compensation, and describes a plurality of parameters including a weighting coefficient specified by a reference number. The first syntax described above, the second syntax at a level higher than the slice level, and the second syntax describing the first syntax element including the maximum value that the reference number can take, and the unidirectional Upon receiving input of encoded data including a third syntax describing a second syntax element that specifies a slice type including a predicted slice and a bidirectional predicted slice, the first syntax element and the second syntax element are input from the coded data. The syntax element is reproduced, and the plurality of parameters are acquired from the reproduced first syntax element and the first syntax by the reference number up to the maximum value, and the reference image, the identified plurality of parameters, and the specified plurality of parameters are obtained. , Performs a decoding process including the weighted motion compensation using the slice type.

The block diagram which shows the example of the coding apparatus of 1st Embodiment. The explanatory view which shows the predicted coding order example of the pixel block in 1st Embodiment. The figure which shows the block size example of the coding tree block in 1st Embodiment. The figure which shows the specific example of the coding tree block of 1st Embodiment. The figure which shows the specific example of the coding tree block of 1st Embodiment. The figure which shows the specific example of the coding tree block of 1st Embodiment. The block diagram which shows the example of the prediction image generation part of 1st Embodiment. The figure which shows the example of the relationship of the motion vector of the motion compensation prediction in the bidirectional prediction of the first embodiment. The block diagram which shows the example of the multi-frame motion compensation part of 1st Embodiment. The explanatory view of the example of the fixed-point precision of the weighting coefficient in 1st Embodiment. The block diagram which shows the example of the index setting part of 1st Embodiment. The figure which shows the example of WP parameter information of 1st Embodiment. The figure which shows the example of WP parameter information of 1st Embodiment. The figure which shows the example of the coding order and the display order in the low delay coding structure of 1st Embodiment. The figure which shows the example of the relationship between the reference image and the reference number in the low delay coding structure of 1st Embodiment. The figure which shows the example of the coding order and the display order in the random access coding structure of 1st Embodiment. The figure which shows the example of the relationship between the reference image and the reference number in the random access coding structure of 1st Embodiment. The figure which shows the list number of the reference image of 1st Embodiment, and the example of the scan order of a reference number. The figure which showed the example of the WP parameter information after common list conversion in 1st Embodiment simplified. The figure which showed the example of the WP parameter information after common list conversion in 1st Embodiment simplified. The flowchart which shows the generation processing example of the index information of 1st Embodiment. The figure which shows the example of the syntax of 1st Embodiment. The figure which shows the example of the picture parameter set syntax of 1st Embodiment. The figure which shows the example of the slice header syntax of 1st Embodiment. The figure which shows the example of the pred weight table syntax of 1st Embodiment. The figure which shows the example of the sequence parameter set syntax of the modification example. The figure which shows the example of the adaptation parameter set syntax of the modification example. The figure which shows the example of the pred weight table syntax of a modification. The block diagram which shows the structural example of the index setting part of 2nd Embodiment. The flowchart which shows the generation processing example of the index information of 2nd Embodiment. The figure which shows the example of the pred weight table syntax of 2nd Embodiment. The block diagram which shows the structural example of the coding apparatus of 3rd Embodiment. The figure which shows the detailed example of the motion information memory of 3rd Embodiment. The figure which shows the example of the block position which derives the motion information candidate with respect to the coded pixel block of 3rd Embodiment. The figure which shows the example of the block position which derives the motion information candidate with respect to the coded pixel block of 3rd Embodiment. The figure which shows the example of the relationship between the pixel block position and the pixel block position index of a plurality of motion information candidates of 3rd Embodiment. The flowchart which shows the example of the storage process in MergeCandList of 3rd Embodiment. The figure which shows the example of the storage list of the motion information of 3rd Embodiment. The flowchart which shows the example of the storage method in the storage list of the motion information of 3rd Embodiment. The figure which shows the combination example of the bidirectional prediction of 3rd Embodiment. The figure which shows the example of the pred unit syntax of 3rd Embodiment. The flowchart which shows the other example of the storage process in MergeCandList of 3rd Embodiment. The block diagram which shows the structural example of the decoding apparatus of 4th Embodiment. The block diagram which shows the structural example of the index setting part of 4th Embodiment. The block diagram which shows the structural example of the index setting part of 5th Embodiment. The block diagram which shows the structural example of the decoding apparatus of 6th Embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The coding device and decoding device of each of the following embodiments are LSI (Large-Scale Integration) chips and DSPs (Digital).
It can be realized by hardware such as Signal Processor) or FPGA (Field Programmable Gate Array). Further, the encoding device and the decoding device of each of the following embodiments can be realized by causing a computer to execute a program, that is, by software. In the following description, the term "image" can be appropriately read as a term such as "video", "pixel", "image signal", "picture", or "image data".

(First Embodiment)
In the first embodiment, a coding device that encodes a moving image will be described.

FIG. 1 is a block diagram showing an example of the configuration of the coding device 100 of the first embodiment.

The coding device 100 divides each frame or each field constituting the input image into a plurality of pixel blocks, and uses the coding parameters input from the coding control unit 111 to obtain a prediction code for the divided pixel blocks. To generate a predicted image. Then, the coding apparatus 100 generates a prediction error by subtracting the input image divided into a plurality of pixel blocks and the prediction image, orthogonally transforms and quantizes the generated prediction error, and further performs entropy coding for coding. Generate and output data.

The coding device 100 selectively applies a plurality of prediction modes in which at least one of the block size of the pixel block and the method of generating the prediction image is different to perform prediction coding. There are roughly two types of prediction image generation methods: intra-prediction, which makes predictions within a coded frame, and inter-prediction, which makes motion compensation predictions using one or more reference frames that differ in time. Intra-prediction is also referred to as in-screen prediction or in-frame prediction, and inter-prediction is also referred to as inter-screen prediction, inter-frame prediction, motion compensation prediction, or the like.

FIG. 2 is an explanatory diagram showing an example of the predicted coding order of the pixel blocks in the first embodiment. In the example shown in FIG. 2, the coding apparatus 100 performs predictive coding from the upper left to the lower right of the pixel block, and in the frame f to be coded, the left side of the pixel block c to be coded and the left side of the pixel block c. The encoded pixel block p is located on the upper side. In the following, for the sake of simplicity of description, the coding apparatus 100 shall perform predictive coding in the order shown in FIG. 2, but the order of predictive coding is not limited to this.

A pixel block indicates a unit for processing an image, for example, an M × N size block (M and N are natural numbers), a coding tree block, a macro block, a sub block, or 1
Pixels etc. are applicable. In the following description, the pixel block is basically used in the meaning of a coding tree block, but it may be used in other meanings. For example, in the description of the prediction unit, the pixel block is used to mean the pixel block of the prediction unit. In addition, the block is sometimes called by a name such as a unit. For example, a coding block is called a coding unit.

FIG. 3A is a diagram showing an example of the block size of the coding tree block in the first embodiment. The coding tree block is typically a 64x64 pixel block as shown in FIG. 3A. However, the present invention is not limited to this, and may be a 32 × 32 pixel block, a 16 × 16 pixel block, an 8 × 8 pixel block, a 4 × 4 pixel block, or the like. Further, the coding tree block does not have to be a square, and may be, for example, a pixel block of M × N size (M ≠ N).

3B to 3D are diagrams showing specific examples of the coding tree block of the first embodiment. FIG. 3B shows a coding tree block with a block size of 64 × 64 (N = 32). N represents the size of the reference coding tree block, and the size when divided is N and the size when not divided is 2N. FIG. 3C shows a coding tree block obtained by dividing the coding tree block of FIG. 3B into quadtrees. The coding tree block has a quadtree structure, as shown in FIG. 3C. When the coding tree block is divided, the four pixel blocks after the division are numbered in the Z scan order as shown in FIG. 3C.

The coding tree block can be further divided into quadtrees within the number of one quadtree. As a result, the coding tree block can be divided hierarchically. In this case, the depth of division is defined by Depth. FIG. 3D shows one of the coding tree blocks obtained by dividing the coding tree block of FIG. 3B into quadtrees, and the block size is 32 × 32 (N = 16). The Depth of the coding tree block shown in FIG. 3B is 0, and the Depth of the coding tree block shown in FIG. 3D is 1. The coding tree block having the largest unit is called a large coding tree block, and the input image signal is encoded in the raster scan order in this unit.

In the following description, the coding target block or coding tree block of the input image may be referred to as a prediction target block or a prediction pixel block. The coding unit is not limited to the pixel block, and at least one of a frame, a field, a slice, a line, and a pixel can be used.

As shown in FIG. 1, the coding apparatus 100 predicts that the subtraction unit 101, the orthogonal conversion unit 102, the quantization unit 103, the inverse quantization unit 104, the inverse orthogonal conversion unit 105, and the addition unit 106. An image generation unit 107, an index setting unit 108, a motion evaluation unit 109, and a coding unit 110 are provided. The coding control unit 111 shown in FIG. 1 controls the coding device 100, and can be realized by, for example, a CPU (Central Processing Unit) or the like.

The subtraction unit 101 subtracts the corresponding prediction image from the input image divided into pixel blocks to obtain a prediction error. The subtraction unit 101 outputs a prediction error and inputs it to the orthogonal conversion unit 102.

The orthogonal transform unit 102 performs orthogonal transform such as discrete cosine transform (DCT) or discrete sine transform (DST) on the prediction error input from the subtraction unit 101 to obtain a conversion coefficient. The orthogonal conversion unit 102 outputs the conversion coefficient and inputs it to the quantization unit 103.

The quantization unit 103 performs a quantization process on the conversion coefficient input from the orthogonal conversion unit 102 to obtain a quantization conversion coefficient. Specifically, the quantization unit 103 performs quantization according to quantization information such as a quantization parameter and a quantization matrix specified by the coding control unit 111. More specifically, the quantization unit 103 divides the conversion coefficient by the quantization step size derived from the quantization information to obtain the quantization conversion coefficient. The quantization parameter indicates the fineness of quantization. The quantization matrix is used to weight the fineness of quantization for each component of the conversion factor. The quantization unit 103 outputs the quantization conversion coefficient and inputs it to the inverse quantization unit 104 and the coding unit 110.

The inverse quantization unit 104 performs an inverse quantization process on the quantization conversion coefficient input from the quantization unit 103 to obtain a restoration conversion coefficient. Specifically, the inverse quantization unit 104 performs inverse quantization according to the quantization information used in the quantization unit 103. More specifically, the inverse quantization unit 104 multiplies the quantization step size derived from the quantization information by the quantization conversion coefficient to obtain the restoration conversion coefficient. The quantization information used in the quantization unit 103 is loaded from an internal memory (not shown) of the coding control unit 111 and used. The inverse quantization unit 104 outputs the restoration conversion coefficient and inputs it to the inverse orthogonal conversion unit 105.

The inverse quadrature transforming unit 105 performs an inverse quadrature transform such as, for example, inverse discrete cosine transform (IDCT) or inverse discrete sine transform (IDST) with respect to the restoration transform coefficient input from the inverse quantized unit 104. Obtain the restoration prediction error. The inverse orthogonal conversion performed by the inverse orthogonal conversion unit 105 corresponds to the orthogonal conversion performed by the orthogonal conversion unit 102. The inverse orthogonal conversion unit 105 outputs the restoration prediction error and inputs it to the addition unit 106.

The addition unit 106 adds the restoration prediction error input from the inverse orthogonal conversion unit 105 and the corresponding prediction image to generate a locally decoded image. The addition unit 106 outputs a locally decoded image and inputs it to the prediction image generation unit 107.

The prediction image generation unit 107 stores the locally decoded image input from the addition unit 106 as a reference image in a memory (not shown in FIG. 1), outputs the reference image stored in the memory, and inputs the reference image to the motion evaluation unit 109. .. Further, the prediction image generation unit 107 performs weighted motion compensation prediction based on the motion information and the WP parameter information input from the motion evaluation unit 109, and generates a prediction image. The prediction image generation unit 107 outputs a prediction image and inputs it to the subtraction unit 101 and the addition unit 106.

FIG. 4 is a block diagram showing an example of the configuration of the prediction image generation unit 107 of the first embodiment. As shown in FIG. 4, the prediction image generation unit 107 includes a plurality of frame motion compensation unit 201, a memory 202, a unidirectional motion compensation unit 203, a prediction parameter control unit 204, a reference image selector 205, and a frame memory 206. And a reference image control unit 207.

The frame memory 206 stores the locally decoded image input from the addition unit 106 as the reference image under the control of the reference image control unit 207. The frame memory 206 has a plurality of memory sets FM1 to FMN (N ≧ 2) for temporarily holding the reference image.

The prediction parameter control unit 204 prepares a plurality of combinations of the reference image number and the prediction parameter as a table based on the movement information input from the motion evaluation unit 109. Here, the motion information refers to a motion vector indicating the amount of deviation of the motion used in the motion compensation prediction, a reference image number, information on a prediction mode such as unidirectional / bidirectional prediction, and the like. Prediction parameters refer to information about motion vectors and prediction modes. Then, the prediction parameter control unit 204 selects and outputs a combination of the reference image number and the prediction parameter used for generating the prediction image based on the input image, inputs the reference image number to the reference image selector 205, and inputs the prediction parameter. Is input to the unidirectional motion compensation unit 203.

The reference image selector 205 is a switch that switches which output terminal of the frame memories FM1 to FMN of the frame memory 206 is connected according to the reference image number input from the prediction parameter control unit 204. For example, if the reference image number is 0, the reference image selector 205 connects the output end of the FM1 to the output end of the reference image selector 205, and if the reference image number is N-1, the reference image selector 205 refers to the output end of the FMN. Connect to the output end of the image selector 205. The reference image selector 205 outputs a reference image stored in the frame memory to which the output end is connected among the frame memories FM1 to FMN of the frame memory 206, and outputs the unidirectional motion compensation unit 203 and the motion evaluation unit 109. Enter in.

The unidirectional prediction motion compensation unit 203 performs motion compensation prediction processing according to the prediction parameters input from the prediction parameter control unit 204 and the reference image input from the reference image selector 205, and generates a unidirectional prediction image.

FIG. 5 is a diagram showing an example of the relationship between the motion vectors of the motion compensation prediction in the bidirectional prediction of the first embodiment. In motion compensation prediction, interpolation processing is performed using the reference image, and a unidirectional prediction image is generated based on the amount of motion deviation from the pixel block of the coded target position between the created interpolation image and the input image. .. Here, the amount of deviation is a motion vector. As shown in FIG. 5, in the bidirectional prediction slice (B-slice), a prediction image is generated using two types of reference images and a set of motion vectors. As the interpolation process, an interpolation process having a 1/2 pixel accuracy, an interpolation process having a 1/4 pixel accuracy, or the like is used, and the value of the interpolated image is generated by performing the filtering process on the reference image. For example, H.A., which can perform interpolation processing up to 1/4 pixel accuracy for a luminance signal. In 264, the amount of deviation is expressed as four times the precision of the integer pixel.

The unidirectional prediction motion compensation unit 203 outputs the unidirectional prediction image and temporarily stores it in the memory 202. Here, when the motion information (prediction parameter) indicates bidirectional prediction, the multi-frame motion compensation unit 201 performs weighted prediction using two types of unidirectional prediction images, so that the unidirectional prediction motion compensation unit 203 Stores the first unidirectional prediction image in the memory 202, and directly outputs the second unidirectional prediction image to the multi-frame motion compensation unit 201. Here, the unidirectional prediction image corresponding to the first is referred to as the first prediction image, and the unidirectional prediction image corresponding to the second is referred to as the second prediction image.

In addition, two unidirectional motion compensation units 203 may be prepared, and each may generate two unidirectional prediction images. In this case, when the motion information (prediction parameter) indicates unidirectional prediction, if the unidirectional motion compensation unit 203 directly outputs the first unidirectional prediction image as the first prediction image to the plurality of frame motion compensation unit 201. Good.

The multi-frame motion compensation unit 201 uses the first prediction image input from the memory 202, the second prediction image input from the unidirectional prediction motion compensation unit 203, and the WP parameter information input from the motion evaluation unit 109. , Performs weighted prediction to generate a prediction image. The multi-frame motion compensation unit 201 outputs a predicted image and inputs it to the subtraction unit 101 and the addition unit 106.

FIG. 6 is a block diagram showing an example of the configuration of the multiple frame motion compensation unit 201 of the first embodiment. As shown in FIG. 6, the multi-frame motion compensation unit 201 includes a default motion compensation unit 301, a weighted motion compensation unit 302, a WP parameter control unit 303, and WP selectors 304 and 305.

The WP parameter control unit 303 outputs the WP application flag and weight information based on the WP parameter information input from the motion evaluation unit 109, and sets the WP application flag to the WP selector 30.
It is input to 4 and 305, and the weight information is input to the weighted motion compensation unit 302.

Here, the WP parameter information includes the fixed-point accuracy of the weighting factor, the first WP application flag corresponding to the first predicted image, the first weighting factor, and the first offset, and the second WP adaptation corresponding to the second predicted image. Contains information on flags, second weighting factors, and second offset. The WP application flag is a parameter that can be set for each corresponding reference image and signal component, and indicates whether or not to perform weighted motion compensation prediction. The weight information includes information on the fixed-point accuracy of the weighting factors, the first weighting factor, the first offset, the second weighting factor, and the second offset.

Specifically, when the WP parameter information is input from the motion evaluation unit 109, the WP parameter control unit 303 separates and outputs the WP parameter information into the first WP application flag, the second WP application flag, and the weight information. , The first WP application flag is input to the WP selector 304, the second WP application flag is input to the WP selector 305, and the weight information is input to the weighted motion compensation unit 302.

The WP selectors 304 and 305 switch the connection end of each predicted image based on the WP application flag input from the WP parameter control unit 303. The WP selectors 304 and 305 connect their output ends to the default motion compensation unit 301 when their respective WP application flags are 0. Then, the WP selectors 304 and 305 output the first predicted image and the second predicted image and input them to the default motion compensation unit 301. On the other hand, the WP selectors 304 and 305 connect their output ends to the weighted motion compensation unit 302 when their respective WP application flags are 1. Then, the WP selectors 304 and 305 output the first predicted image and the second predicted image and input them to the weighted motion compensation unit 302.

The default motion compensation unit 301 performs average value processing based on the two unidirectional prediction images (first prediction image and second prediction image) input from the WP selectors 304 and 305, and generates a prediction image. Specifically, when the first WP application flag and the second WP application flag are 0, the default motion compensation unit 301 performs the average value processing based on the mathematical formula (1).

P [x, y] = Clip1 ((PL0 [x, y] + PL1 [x, y] + offset2) >> (shift2)) ... (1)

Here, P [x, y] is a predicted image, PL0 [x, y] is a first predicted image, and PL1 [x, y] is a second predicted image. Offset2 and shift2 are parameters of the rounding process in the mean value processing, and are determined by the internal calculation accuracy of the first predicted image and the second predicted image. Assuming that the bit accuracy of the predicted image is L and the bit accuracy of the first predicted image and the second predicted image is M (L ≦ M), shift2 is formulated by the mathematical formula (2) and offset2 is formulated by the mathematical formula (3). Will be done.

shift2 = (ML + 1) ... (2)

offset2 = (1 << (shift2-1) ... (3)

For example, when the bit accuracy of the predicted image is 8 and the bit accuracy of the first predicted image and the second predicted image is 14, shift2 = 7 from the formula (2) and offset2 = (1 << from the formula (3). 6).

When the prediction mode indicated by the motion information (prediction parameter) is unidirectional prediction, the default motion compensation unit 301 uses only the first prediction image to obtain the final prediction image based on the mathematical formula (4). calculate.

P [x, y] = Clip1 ((PLX [x, y] + office1) >> (shift1)) ... (4)

Here, PLX [x, y] indicates a unidirectional prediction image (first prediction image), and X is an identifier indicating either 0 or 1 in the reference list. For example, when the reference list is 0, it is PL0 [x, y], and when the reference list is 1, it is PL1 [x, y]. Offset1 and shift1 are parameters of the rounding process and are determined by the internal calculation accuracy of the first predicted image. Assuming that the bit accuracy of the predicted image is L and the bit accuracy of the first predicted image is M, shift1 is formulated by the mathematical formula (5), and offset1 is formulated by the mathematical formula (6).

shift1 = (ML) ... (5)

offset1 = (1 << (shift1-1) ... (6)

For example, when the bit precision of the predicted image is 8 and the bit precision of the first predicted image is 14, shift1 = 6 from the mathematical formula (5) and offset1 = (1 << 5) from the mathematical formula (6).

The weighted motion compensation unit 302 is based on two unidirectional prediction images (first prediction image and second prediction image) input from the WP selectors 304 and 305 and weight information input from the WP parameter control unit 303. Perform weighted motion compensation. Specifically, when the first WP application flag and the second WP application flag are 1, the weighted motion compensation unit 302 performs weighting processing based on the mathematical formula (7).

P [x, y] = Clip1 (((PL0 [x, y] * w0C + PL1 [x, y] * w1C + (1 << logWDC)) >> (logWDC + 1)) + ((o0C + o1C + 1) >> 1)) ... (7)

Here, w0C represents a weighting coefficient corresponding to the first predicted image, w1C represents a weighting coefficient corresponding to the second predicted image, o0C represents an offset corresponding to the first predicted image, and o1C represents an offset corresponding to the second predicted image. .. Hereinafter, they will be referred to as a first weighting coefficient, a second weighting coefficient, a first offset, and a second offset, respectively. logWDC is a parameter indicating the fixed-point accuracy of each weighting factor. The variable C means a signal component. For example, in the case of a YUV spatial signal, the luminance signal is represented by C = Y, the Cr color difference signal is represented by C = Cr, and the Cb color difference component is represented by C = Cb.

When the calculation accuracy of the first prediction image and the second prediction image is different from that of the prediction image, the weighted motion compensation unit 302 rounds the logWDC, which is the fixed-point accuracy, by controlling it as in the mathematical formula (8). To realize.

logWD'C = logWDC + offset1 ... (8)

The rounding process can be realized by replacing the logWDC of the mathematical formula (7) with the logWD'C of the mathematical formula (8). For example, when the bit accuracy of the predicted image is 8 and the bit accuracy of the first predicted image and the second predicted image is 14, by resetting the logWDC, the calculation accuracy is the same as that of shift2 of the mathematical formula (1). It is possible to realize batch rounding processing.

When the prediction mode indicated by the motion information (prediction parameter) is unidirectional prediction, the weighted motion compensation unit 302 uses only the first prediction image and the final prediction image based on the mathematical formula (9). Is calculated.

P [x, y] = Clip1 ((PLX [x, y] * wXC + (1 << logWDC-1)) >> (logWDC)) ... (9)

Here, PLX [x, y] indicates a unidirectional prediction image (first prediction image), wXC indicates a weighting coefficient corresponding to unidirectional prediction, and X indicates either 0 or 1 in the reference list. It is an identifier to indicate. For example, when the reference list is 0, it is PL0 [x, y], w0C, and when the reference list is 1, it is PL1 [x, y], w1C.

When the calculation accuracy of the first prediction image and the second prediction image is different from that of the prediction image, the weighted motion compensation unit 302 sets the logWDC, which is the fixed-point accuracy, as in the mathematical formula (8) as in the case of bidirectional prediction. The rounding process is realized by controlling to.

The rounding process can be realized by replacing the logWDC of the mathematical formula (7) with the logWD'C of the mathematical formula (8). For example, when the bit accuracy of the predicted image is 8 and the bit accuracy of the first predicted image is 14, by resetting the logWDC, a batch rounding process with the same calculation accuracy as shift 1 of the mathematical formula (4) is realized. It becomes possible to do.

FIG. 7 is an explanatory diagram of an example of fixed-point accuracy of the weighting coefficient in the first embodiment, and is a diagram showing an example of a change between a moving image having a change in brightness in the time direction and a gradation value. In the example shown in FIG. 7, the frame to be encoded is Frame (t), the frame immediately before in time is Frame (t-1), and the frame one frame after in time is Frame (t + 1). As shown in FIG. 7, in a fade image that changes from white to black, the brightness (gradation value) of the image decreases with time. The weighting coefficient means the degree of change in FIG. 7, and as is clear from the formulas (7) and (9), it takes a value of 1.0 when there is no change in brightness. The fixed-point precision is a parameter that controls the step size corresponding to the decimal point of the weighting coefficient, and the weighting coefficient when there is no change in brightness is 1 << logWDC.

In the case of unidirectional prediction, various parameters (second WP adaptation flag, second weighting coefficient, and second offset information) corresponding to the second prediction image are not used, so they are set to predetermined initial values. It may have been.

Returning to FIG. 1, the motion evaluation unit 109 evaluates motion between a plurality of frames based on the input image and the reference image input from the prediction image generation unit 107, outputs motion information and WP parameter information, and outputs motion information. It is input to the prediction image generation unit 107 and the coding unit 110, and the WP parameter information is input to the prediction image generation unit 107 and the index setting unit 108.

The motion evaluation unit 109 calculates an error by calculating a difference value starting from a plurality of reference images corresponding to the same position as the input image of the prediction target pixel block, and shifts this position with fractional accuracy to minimize the error. Optimal motion information is calculated by a method such as block matching. In the case of bidirectional prediction, the motion evaluation unit 109 uses the motion information derived by the unidirectional prediction to perform block matching including the default motion compensation prediction as shown in the mathematical formulas (1) and (4). By doing so, the motion information of the two-way prediction is calculated.

At this time, the motion evaluation unit 109 can calculate the WP parameter information by performing block matching including the weighted motion compensation prediction as shown by the mathematical formulas (7) and (9). The WP parameter information may be calculated by using a method of calculating the weighting coefficient or offset using the brightness gradient of the input image, or a method of calculating the weighting coefficient or offset by accumulating the prediction error at the time of coding. Good. Further, as the WP parameter information, a fixed value predetermined for each encoding device may be used.

Here, with reference to FIG. 7, a method of calculating the weighting coefficient, the fixed-point accuracy of the weighting coefficient, and the offset from the moving image whose brightness changes with time will be described. As described above, in the fade image that changes from white to black as shown in FIG. 7, the brightness (gradation value) of the image decreases with time. The motion evaluation unit 109 can calculate the weighting coefficient by calculating this slope.

Further, the fixed-point accuracy of the weighting coefficient is information indicating the accuracy of the inclination, and the motion evaluation unit 109 can calculate an optimum value from the temporal distance of the reference image and the degree of change in the image brightness. For example, in FIG. 7, when the weighting coefficient between Frame (t-1) and Frame (t + 1) is 0.75 with decimal point accuracy, 3/4 can be expressed with 1/4 accuracy, so motion evaluation is performed. The unit 109 sets the fixed-point precision to 2 (1 << 2). Since the fixed-point precision value affects the code amount when the weighting coefficient is encoded, the optimum value may be selected in consideration of the code amount and the prediction accuracy. The fixed-point precision value may be a predetermined fixed value.

Further, when the slopes do not match, the motion evaluation unit 109 can calculate the offset value by obtaining the correction value (deviation amount) corresponding to the intercept of the linear function. For example, in FIG. 7, when the weighting coefficient between Frame (t-1) and Frame (t + 1) is 0.60 with a decimal point precision and the fixed-point precision is 1 (1 << 1), the weighting coefficient is 1. (That is, the decimal point precision of the weighting factor corresponds to 0.50) is likely to be set. In this case, since the decimal point accuracy of the weighting coefficient deviates from the optimum value of 0.60 by 0.10, the motion evaluation unit 109 calculates the correction value for this amount from the maximum value of the pixel and uses it as an offset value. Set. When the maximum value of the pixel is 255, the motion evaluation unit 109 may set a value such as 25 (255 × 0.1).

In the first embodiment, the motion evaluation unit 109 is illustrated as one function of the coding device 100, but the motion evaluation unit 109 is not an essential configuration of the coding device 100, and for example, the motion evaluation unit 109 is coded. It may be a device outside the conversion device 100. In this case, the motion information and the WP parameter information calculated by the motion evaluation unit 109 may be loaded into the coding device 100.

The index setting unit 108 receives the WP parameter information input from the motion evaluation unit 109, confirms the reference list (list number) and the reference image (reference number), outputs the index information, and outputs the index information to the index setting unit 108. input.

FIG. 8 is a block diagram showing an example of the configuration of the index setting unit 108 of the first embodiment. As shown in FIG. 8, the index setting unit 108 includes a reference image confirmation unit 401 and an index generation unit 402.

The reference image confirmation unit 401 receives the WP parameter information input from the motion evaluation unit 109, and confirms whether or not there is a WP parameter in which the reference numbers included in the two reference lists point to the same reference image. Then, the reference image confirmation unit 401 deletes the WP parameter pointing to the same reference image among the WP parameter information, outputs the deleted WP parameter information, and inputs it to the index generation unit 402.

The index generation unit 402 receives the WP parameter information from which the redundant WP parameters have been deleted from the reference image confirmation unit 401, maps the syntax elements described later, and generates the index information. The index generation unit 402 outputs index information and inputs it to the coding unit 110.

9A and 9B are diagrams showing an example of WP parameter information of the first embodiment. P-
An example of WP parameter information at the time of slice is as shown in FIG. 9A, and an example of WP parameter information at the time of B-slice is as shown in FIGS. 9A and 9B. The list number is an identifier indicating the prediction direction, and takes a value of 0 at the time of unidirectional prediction, and takes two values of 0 and 1 at the time of bidirectional prediction because two types of prediction can be used. The reference number is a value corresponding to 1 to N shown in the frame memory 206. Since the WP parameter information is held for each reference list and reference image, the information required at the time of B-slice is 2N, assuming that there are N reference images. The reference image confirmation unit 401 reconverts these WP parameter information and deletes redundant WP parameters.

FIG. 10 is a diagram showing an example of a coding order and a display order (POC: Pcity Order Count) in the low delay coding structure of the first embodiment, and B-slice that can be a reference image is rB-. An example of the coded structure used as a slice is shown. In the low delay coding structure, the image cannot be read ahead, so the coding order and the display order are the same. Here, it is considered that frames 0 to 3 are already encoded in the display order, and frame 4 is encoded.

FIG. 11 is a diagram showing an example of the relationship between the reference image and the reference number in the low delay coding structure of the first embodiment, and simplifies the relationship between the reference image and the reference number when the frame 4 is encoded. It is shown by the WP parameter information. In the example shown in FIG. 11, the order of the reference numbers in the reference list is the same in the list 0 and the list 1, which means that the reference images in the list 0 and the list 1 are the same.

FIG. 12 is a diagram showing an example of a coding order and a display order in the random access coding structure of the first embodiment. Here, frames 0, 2, 4, and 8 are already encoded in the display order. Consider encoding frame 1.

FIG. 13 is a diagram showing an example of the relationship between the reference image and the reference number in the random access coding structure of the first embodiment, and simplifies the relationship between the reference image and the reference number when the frame 1 is encoded. It is shown by the WP parameter information. In the example shown in FIG. 13, the order of the reference numbers for each reference list is different, but list 0 and list 1 include four common reference images, and list 0 and list 1 have different reference numbers. It means that the reference images are the same.

The reference image confirmation unit 401 rearranges the reference numbers to remove redundant WP parameters, that is, WP parameters in which the reference numbers included in the two reference lists point to the same reference image, and a common new index. Convert to.

FIG. 14 is a diagram showing an example of the list numbers of the reference images of the first embodiment and the scanning order of the reference numbers. The reference image confirmation unit 401 converts a two-dimensional reference list (two reference lists) into a one-dimensional common list (one common list) in the scanning order shown in FIG. Specifically, the reference image confirmation unit 401 rearranges the two reference lists into a common list by reading the WP parameter information shown in FIGS. 9A and 9B in the scan order shown in FIG. 14, and redundant WP parameters. To delete. Here, the reference image confirmation unit 401 reads the WP parameter information according to the pseudo code shown in the mathematical expression (10).

for (scan = 0; scan <= num_of_common_active_ref_minus1; scan ++) {
list = common_scan_list (scan)
ref_idx = common_scan_ref_idx (scan)
refPOC = RefPicOrderCount (list, ref_idx)
identical_ref_flag = false;
for (curridx = 0; curridx <= num_of_common_active_ref_minus1; curridx ++) {
if (refPOC == CommonRefPicOrderCount (curridx)) {
identical_ref_flag = true;
}
}
if (! identical_ref_flag)
InsertCommonRefPicOrderCount (scan, refPOC);
}… (10)

Here, common_scan_list () is a function that returns the list number corresponding to the scan number in FIG. common_scan_ref_idx () is a function that returns the reference number corresponding to the scan number in FIG. RefPicOrderCnt () is a function that returns the POC number corresponding to the list number and reference number. The POC number is a number indicating the display order of the reference images. However, if the POC number exceeds a predetermined maximum number, it is mapped to an initial value. For example, if the maximum value of the POC number is 255, the POC number corresponding to 256 is 0. CommonRefPicOrderCount () is a function that returns the POC number corresponding to the reference number of the common list (CommonList). InsertCommonRefPicOrderCount () is a function that inserts the POC number corresponding to the scan number into the common list (CommonList).

The scan order shown in FIG. 14 is an example, and any other scan order may be used as long as it is a predetermined scan order. Further, the pseudo code shown by the mathematical formula (10) is an example, and if the purpose of this processing can be realized, it is possible to add processing and reduce redundant processing.

15 and 16 are diagrams showing a simplified example of WP parameter information after common list conversion in the first embodiment. FIG. 15 shows a simplified WP parameter information of the common list obtained by converting the WP parameter information of the reference list shown in FIG. 11, and FIG. 16 shows a common list obtained by converting the WP parameter information of the reference list shown in FIG. The WP parameter information of is simplified and shown.

FIG. 17 is a flowchart showing an example of the index information generation process executed by the index setting unit 108 of the first embodiment.

When the WP parameter information is input, the reference image confirmation unit 401 branches the process according to the slice type (step S101).

When the slice type is a unidirectional prediction slice (P-slice) using only one reference list (No in step S101), the reference image confirmation unit 401 uses the WP parameter information shown in FIG. 9A as it is as an index generation unit. Output to 402. The index generation unit 402 maps the WP parameter information input from the reference image confirmation unit 401 to a predetermined syntax element described later, generates index information, and outputs the index information.

On the other hand, when the slice type is a bidirectional prediction slice (B-slice) using two reference lists (Yes in step S101), the reference image confirmation unit 401 initializes the variable scan to 0 (step S102). ). The variable scan indicates the scan number in FIG.

Subsequently, the reference image confirmation unit 401 uses the variable scan to acquire the variable list indicating the list number corresponding to the scan number (step S103), and acquires the variable refIdx indicating the reference number corresponding to the scan number. (Step S104).

Subsequently, the reference image confirmation unit 401 derives a variable refPOC indicating the POC number of the reference image corresponding to the list number indicated by the variable list and the reference number indicated by the variable refIdx (step S105).

Subsequently, the reference image confirmation unit 401 sets the flag identical_ref_flag to false (step S106) and sets the variable curridx to 0 (step S107). The flag identical_ref_flag indicates whether or not the same reference image exists in the common list. The variable curridx indicates the reference number of the common list.

Subsequently, the reference image confirmation unit 401 determines whether or not the POC number indicated by the derived variable refPOC and the POC number of the reference image corresponding to the reference number indicated by the variable curridx are the same (step S108).

When both POC numbers are the same (Yes in step S108), the reference image confirmation unit 401 sets the flag electrical_ref_flag to true (step S109). On the other hand, if both POC numbers are not the same (No in step S108), the process of step S109 is not performed.

Subsequently, the reference image confirmation unit 401 increments the variable curridx (step S110).

Subsequently, the reference image confirmation unit 401 determines whether the value of the variable curridx is larger than num_of_common_active_ref_minus1 which is the value obtained by subtracting 1 from the maximum number of the common list (step S111), and is between num_of_common_active_ref_minus1 and below (No in step S111). ), The processing of steps S108 to S110 is repeated.

When it becomes larger than num_of_common_active_ref_minus1 (Yes in step S111), the reference image confirmation unit 401 completes the confirmation of the common list and further confirms whether or not the flag electrical_ref_flag is false (step S112).

When the flag electrical_ref_flag is false (Yes in step S112), the reference image confirmation unit 401 determines that the common list does not include the same reference image as the reference image of the POC number indicated by the variable refPOC, and determines that the POC indicated by the variable refPOC is not included. The number is added to the common list (step S113). On the other hand, when the flag electrical_ref_flag is true (No in step S112), the reference image confirmation unit 401 determines that the common list contains the same reference image as the reference image of the POC number indicated by the variable refPOC, and processes in step S113. Do not do.

Subsequently, the reference image confirmation unit 401 increments the variable scan (step S114).

Subsequently, the reference image confirmation unit 401 determines whether the value of the variable scan is greater than num_of_common_active_ref_minus1 which is the value obtained by subtracting 1 from the maximum number of the common list (step S115), and if it is num_of_common_active_ref_minus1 or less (in step S115). No), the process returns to step S103. On the other hand, if it is larger than num_of_common_active_ref_minus1 (Yes in step S115), the reference image confirmation unit 401 outputs the WP parameter information after conversion to the common list to the index generation unit 402. The index generation unit 402 maps the WP parameter information input from the reference image confirmation unit 401 to a predetermined syntax element described later, generates index information, and outputs the index information.

Returning to FIG. 1, the coding unit 110 includes the quantization conversion coefficient input from the quantization unit 103, the motion information input from the motion evaluation unit 109, the index information input from the index setting unit 108, and the coding control. Entropy coding is performed on various coding parameters such as the quantization information specified by the unit 111, and coded data is generated. The entropy coding corresponds to, for example, Huffman coding or arithmetic coding.

The coding parameter is a parameter required for decoding such as prediction information indicating a prediction method, information on a quantization conversion coefficient, and information on quantization. For example, the coding control unit 111 has an internal memory (not shown), the coding parameters are held in the internal memory, and when encoding the pixel blocks, the coding parameters of the adjacent already encoded pixel blocks are used. Can be done. For example, H. In the intra-prediction of 264, the prediction information of the pixel block can be derived from the prediction information of the encoded adjacent block.

The coding unit 110 outputs the generated coded data according to an appropriate output timing managed by the coded control unit 111. In the output coded data, for example, various information is multiplexed by a multiplexing unit (not shown) and temporarily stored in an output buffer (not shown), and then, for example, a storage system (not shown) ( It is output to the storage media) or the transmission system (communication line).

FIG. 18 is a diagram showing an example of the syntax 500 used by the coding device 100 of the first embodiment. The syntax 500 shows the structure of the coded data generated by the coding device 100 encoding the input image (moving image data). When decoding the coded data, the decoding device described later performs the syntax interpretation of the moving image with reference to the same syntax structure as the syntax 500.

The syntax 500 includes three parts: a high level syntax 501, a slice level syntax 502, and a coding tree level syntax 503. High-level syntax 501 contains syntax information for layers above the slice. A slice refers to a rectangular area or a continuous area contained in a frame or field. The slice level syntax 502 contains the information required to decode each slice. The coding tree level syntax 503 contains information necessary to decode each coding tree (ie, each coding tree block). Each of these parts contains more detailed syntax.

High-level syntax 501 includes sequence and picture-level syntax such as sequence parameter set syntax 504, picture parameter set syntax 505, and adaptation parameter set syntax 506.

The slice level syntax 502 includes a slice header syntax 507, a pred weight table syntax 508, a slice data syntax 509, and the like. The pred weight table syntax 508 is called from the slice header syntax 507.

The coding tree level syntax 503 includes a coding tree unit syntax 510, a transform unit syntax 511, a prediction unit syntax 512, and the like. The coding tree unit Syntax 510 can have a quadtree structure. Specifically, the coding tree unit syntax 510 can be recursively called as a syntax element of the coding tree unit syntax 510. That is, one coding tree block can be subdivided into quadtrees. Further, the transform unit syntax 511 is included in the coding tree unit syntax 510. The transform unit syntax 511 is called in each coding tree unit syntax 510 at the end of the quadtree. The transform unit syntax 511 describes information related to inverse orthogonal transformation, quantization, and the like. Information about weighted motion compensation predictions may be described in these syntaxes.

FIG. 19 is a diagram showing an example of the picture parameter set syntax 505 of the first embodiment. The weighted_pred_flag is, for example, a syntax element indicating the validity or invalidity of the weighted compensation prediction of the first embodiment regarding P-slice. When weighted_pred_flag is 0, the weighted motion compensation prediction of the first embodiment in P-slice is invalid. Therefore, the WP application flag included in the WP parameter information is always set to 0, and the WP selectors 304 and 305 connect their respective output ends to the default motion compensation unit 301. On the other hand, when weighted_pred_flag is 1, the weighted motion compensation prediction of the first embodiment in P-slice is valid.

As another example, when weighted_pred_flag is 1, in the syntax of lower layers (slice header, coding tree block, transform unit, prediction unit, etc.), the first is for each local area inside the slice. One embodiment may specify the validity or invalidity of the weighted motion compensation prediction.

The weighted_bipred_idc is, for example, a syntax element indicating the validity or invalidity of the weighted compensation prediction of the first embodiment regarding B-slice. When weighted_bipred_idc is 0, the weighted motion compensation prediction of the first embodiment in B-slice becomes invalid. Therefore, the WP application flag included in the WP parameter information is always set to 0, and the WP selectors 304 and 305 connect their respective output ends to the default motion compensation unit 301. On the other hand, when weighted_bipred_idc is 1, the weighted motion compensation prediction of the first embodiment in B-slice is valid.

As another example, when weighted_bipred_idc is 1, in the syntax of lower layers (slice header, coding tree block, transform unit, etc.), for each local region inside the slice, the first embodiment The validity or invalidity of the weighted motion compensation prediction may be specified.

FIG. 20 is a diagram showing an example of the slice header syntax 507 of the first embodiment. slice_type indicates the slice type of the slice (I-slice, P-slice, B-slice, etc.). The pic_parameter_set_id is an identifier indicating which picture parameter set syntax 505 is to be referred to. num_ref_idx_active_override_flag is a flag indicating whether to update the number of valid reference images. When this flag is 1, num_ref_idx_l0_active_minus1 and num_ref_idx_l1_active_minus1 that define the number of reference images in the reference list can be used. pred_weight_table () is a function indicating the pred weight table syntax used for weighted motion compensation prediction, and when the above-mentioned weighted_pred_flag is 1 and P-slice, and when weighted_bipred_idc is 1 and B-slice, this function is Called.

FIG. 21 is a diagram showing an example of the pred weight table syntax 508 of the first embodiment. luma_log2_weight_denom represents the fixed-point accuracy of the weighting coefficient of the luminance signal in the slice, and is a value corresponding to the logWDC of the equation (7) or the equation (9). chroma_log2_weight_denom represents the fixed-point accuracy of the weighting coefficient of the color difference signal in the slice, and is a value corresponding to the logWDC of the formula (7) or the formula (9). chroma_format_idc is an identifier representing a color space, and MONO_IDX is a value indicating a monochrome image. num_ref_common_active_minus1 indicates the value obtained by subtracting 1 from the number of reference images included in the common list in the slice, and is used in the formula (10). The maximum value of this value is the sum of the maximum values of the two reference images in List 0 and List 1.

luma_weight_common_flag indicates the WP adaptation flag in the luminance signal. When this flag is 1, the weighted motion compensation prediction of the luminance signal of the first embodiment is effective in the entire area within the slice. chroma_weight_common_flag indicates the WP adaptation flag in the color difference signal. When this flag is 1, the weighted motion compensation prediction of the color difference signal of the first embodiment is effective in the entire area within the slice. luma_weight_common [i] is the weighting coefficient of the i-th corresponding luminance signal managed in the common list. luma_offset_common [i] is the offset of the i-th corresponding luminance signal managed in the common list. These are the values corresponding to w0C, w1C, o0C, and o1C of the mathematical formula (7) or the mathematical formula (9), respectively. However, C = Y.

chroma_weight_common [i] [j] is the weighting coefficient of the i-th corresponding color difference signal managed in the common list. chroma_offset_common [i] [j] is the offset of the i-th corresponding color difference signal managed in the common list. These are the values corresponding to w0C, w1C, o0C, and o1C of the mathematical formula (7) or the mathematical formula (9), respectively. However, C = Cr or Cb. j indicates a component of a color difference component. For example, in the case of a YUV4: 2: 0 signal, j = 0 is a Cr component and j = 1 is a Cb component.

As described above, in the first embodiment, when the index setting unit 108 has a combination including the same reference image in the two reference lists of the list 0 and the list 1 in the WP parameter information, the index setting unit 108 sets the two reference lists. By converting to a common list, duplicate redundant WP parameters are deleted from the WP parameter information and index information is generated. Therefore, according to the first embodiment, the code amount of the index information can be reduced.

In particular, since the weighting coefficient and the offset need to be encoded for each reference image, the amount of information signaled to the decoder increases as the number of reference images increases, but when the number of reference images increases, they are the same in the reference list. Since the number of cases where the reference image is referred to increases, a significant reduction in the amount of code can be expected by managing with a common list.

(Modified example of the first embodiment)
A modified example of the first embodiment will be described. In the modified example of the first embodiment, the syntax element used by the index generation unit 402 is different from the syntax element of the first embodiment.

FIG. 22 is a diagram showing an example of the sequence parameter set syntax 504 of the modified example of the first embodiment. profile_idc is an identifier indicating information about the profile of the encoded data. level_idc is an identifier that indicates information about the level of encoded data. seq_parameter_set_id is an identifier indicating which sequence parameter set syntax 504 is to be referred to. num_ref_frames is a variable that indicates the maximum number of reference images in a frame. The weighted_prediction_enabled_flag is, for example, a syntax element indicating the validity or invalidity of the weighted motion compensation prediction of the modified example with respect to the encoded data.

If weighted_prediction_enabled_flag is 0, the weighted motion compensation prediction of the variant in the encoded data is invalid. Therefore, the WP application flag included in the WP parameter information is always set to 0, and the WP selectors 304 and 305 connect their respective output ends to the default motion compensation unit 301. On the other hand, when weighted_prediction_enabled_flag is 1, the weighted motion compensation prediction of the modified example is valid in the entire coded data.

As another example, when weighted_prediction_enabled_flag is 1, in the syntax of lower layers (picture parameter set, adaptation parameter set, slice header, coding tree block, transform unit, prediction unit, etc.) The validity or invalidity of the weighted motion compensation prediction of the modified example may be specified for each local region inside the slice.

FIG. 23 is a diagram showing an example of the adaptation parameter set syntax 506 of the modified example of the first embodiment. The adaptation parameter set syntax 506 holds parameters that affect the entire coded frame and is independently encoded as a superordinate unit. For example, H. In 264, the syntax element corresponding to the high level syntax 501 is encoded as a NAL unit. Therefore, the adaptation parameter set syntax 506 differs from the syntax lower than the slice level syntax 502, which holds parameters of lower information represented by slices and pixel blocks, and a coding unit.

aps_id is an identifier indicating which adaptation parameter set syntax 506 is to be referred to. By referring to this identifier, the lower slice can refer to any encoded aps_id. In this case, for example, by adding the same aps_id to the slice header syntax 507 of FIG. 20, it is possible to read the parameters that affect the entire frame.

The aps_weighted_prediction_flag is, for example, a syntax element indicating the validity or invalidity of the weighted motion compensation prediction of the modified example regarding P-slice in the frame. When aps_weighted_prediction_flag is 0, the weighted motion compensation prediction of the P-slice variant in the frame is invalid. Therefore, the WP application flag included in the WP parameter information is always set to 0, and the WP selectors 304 and 305 connect their respective output ends to the default motion compensation unit 301. On the other hand, when aps_weighted_prediction_flag is 1, the weighted motion compensation prediction of the modified example is effective in the entire area within the frame.

As another example, when aps_weighted_prediction_flag is 1, the syntax of lower layers (slice header, coding tree block, transform unit, prediction unit, etc.) is transformed for each local area inside the slice. You may want to specify the validity or invalidity of the weighted motion compensation prediction of the example prediction.

aps_weighted_bipred_idx is, for example, a syntax element indicating the validity or invalidity of the weighted motion compensation prediction of the modified example regarding B-slice in the frame. If aps_weighted_bipred_idx is 0, the weighted motion compensation prediction of the P-slice variant in the frame is invalid. Therefore, the WP application flag included in the WP parameter information is always set to 0, and the WP selectors 304 and 305 connect their respective output ends to the default motion compensation unit 301. On the other hand, when aps_weighted_bipred_idx is 1, the weighted motion compensation prediction of the modified example is effective in the entire area within the frame.

As another example, when aps_weighted_bipred_idx is 1, the syntax of lower layers (slice header, coding tree block, transform unit, prediction unit, etc.) is transformed for each local area inside the slice. You may want to specify the validity or invalidity of the weighted motion compensation prediction of the example prediction.

pred_weight_table () is a function indicating the pred weight table syntax used for weighted motion compensation prediction, and this function is called when the above-mentioned aps_weighted_prediction_flag is 1 or aps_weighted_bipred_idx is 1.

FIG. 24 is a diagram showing an example of the pread weight table syntax 508 of the modified example of the first embodiment. In the modified example of the first embodiment, in the syntax structure of FIG. 18, the pread weight table syntax 508 is called from the adaptation parameter set syntax 506. The difference from the pred weight table syntax 508 in FIG. 21 is that num_ref_common_active_minus1 has been changed to MAX_COMMON_REF_MINUS1. Since the number of reference images is described by the slice header syntax 507, it cannot be referred to by the adaptation parameter set syntax 506, which is an upper layer. Therefore, for example, the value obtained by subtracting 1 from the value of num_ref_frames described in the sequence parameter set syntax 504 is set in MAX_COMMON_REF_MINUS1. In addition, a value that can take the maximum number of reference images may be set according to a predetermined profile or level. The other syntax elements are the same as those in FIG.

As described above, according to the modification of the first embodiment, the code of the WP parameter information when one frame is divided into a plurality of slices by adopting the structure in which the pred weight table syntax 508 is called from the adaptation parameter set syntax 506. The amount can be significantly reduced.

For example, the adaptation parameter set syntax 506 having three different types of WP parameter information to be used is encoded first, and the slice header syntax 507 calls the necessary WP parameter information using aps_id depending on the situation. , The code amount can be reduced as compared with the configuration in which the WP parameter information is always encoded by the slice header syntax 507.

(Second Embodiment)
The second embodiment will be described. In the coding device 600 of the second embodiment, the configuration of the index setting unit 608 is different from that of the coding device 100 of the first embodiment. In the following, the differences from the first embodiment will be mainly explained, and the components having the same functions as those of the first embodiment will be given the same names and codes as those of the first embodiment, and the explanations will be given. Omit.

FIG. 25 is a block diagram showing an example of the configuration of the index setting unit 608 of the second embodiment. As shown in FIG. 25, the index setting unit 608 includes a reuse determination unit 601 and an index generation unit 602.

The reuse determination unit 601 collates the reference number of the list 1 with the reference number of the list 0, and determines whether or not to reuse the WP parameter information of the list 0 as the WP parameter information of the list 1. When the same reference image is included between two reference lists and the corresponding WP parameter information values are the same, encoding the same information not only in list 0 but also in list 1 increases the amount of code. Invite. Therefore, the reuse determination unit 601 reuses the WP parameter information in Listing 0.

Specifically, the reuse determination unit 601 sets the reuse flag to 1 when reusing the WP parameter information in the list 0, and sets the reference number of the reference destination (list 0) as the reference information. On the other hand, the reuse determination unit 601 sets the reuse flag to 0 when the WP parameter information in the list 0 is not reused, and sets the WP parameter information corresponding to the reference number in the list 1 in the syntax.

The reuse determination unit 601 determines whether or not to reuse the WP parameter information in Listing 0 as the WP parameter information in Listing 1, for example, according to the pseudo code shown in the mathematical expression (11).

for (refIdx = 0; refidx <= num_of_active_ref_l1_minus1; refIdx ++) {
refPOC = RefPicOrderCnt (ListL1, refIdx)
refWP = RefWPTable (ListL1, refIdx)
reuse_wp_flag = false;
reuse_ref_idx = 0;
for (curridx = 0; curridx <= num_of_active_ref_l0_minus1; curridx ++) {
if (refPOC == RefPicOrderCnt (ListL0, curridx)
&& RefWPTable (ListL0, curridx, refWP)) {
reuse_wp_flag = true;
reuse_ref_idx = curridx;
}
}
}… (11)

Here, ListL0 indicates List 0, and List L1 indicates List 1. When RefWPTable () inputs the list number, reference number, and reference WP parameter, it flags whether the entered reference WP parameter matches the input list number and the WP parameter corresponding to the reference number. The function to return. When the value of the flag is 1, it means that both WP parameters match. In the pseudo code shown in the formula (11), the reference numbers in the list 1 are scanned in order, and when the reference images having the same POC number are present in the list 0, if the WP parameters are the same, the reuse flag is set. Set the indicated reuse_wp_flag to ture and the corresponding reference number in Listing 0 to reuse_ref_idx.

The reuse determination unit 601 outputs the WP parameter information (which has been set for reuse) after the reuse determination, and inputs it to the index generation unit 602.

The index generation unit 602 receives the WP parameter information after the reuse determination from the reuse determination unit 601 and maps it to the syntax element described later to generate the index information. The syntax element that the index generation unit 602 maps is different from that of the first embodiment. The index generation unit 602 outputs the index information and inputs it to the coding unit 110.

FIG. 26 is a flowchart showing an example of the index information generation process executed by the index setting unit 608 of the second embodiment.

When the WP parameter information is input, the reuse determination unit 601 branches the process according to the slice type (step S201).

When the slice type is a unidirectional prediction slice (P-slice) using only one reference list (No in step S201), the reuse determination unit 601 uses the WP parameter information shown in FIG. 9A as it is as an index generation unit. Output to 602. The index generation unit 602 maps the WP parameter information input from the reuse determination unit 601 to a predetermined syntax element described later, generates index information, and outputs the index information.

On the other hand, when the slice type is a bidirectional prediction slice (B-slice) using two reference lists (Yes in step S201), the reuse determination unit 601 initializes the variable refIdc to 0 (step S202). ). The variable refIdc indicates the reference number in Listing 1.

Subsequently, the reuse determination unit 601 derives the variable refPOC indicating the POC number of the reference image corresponding to the reference number indicated by the variable refIdx (step S203), and derives the WP parameter corresponding to the reference number indicated by the variable refIdx. (Step S204).

Subsequently, the reuse determination unit 601 sets the flag reuse_wp_flag to false (step S205) and sets the variable curridx to 0 (step S206). The flag reuse_wp_flag indicates whether or not the WP parameters of List 0 and List 1 are the same. The variable curridx indicates the reference number in Listing 0.

Subsequently, in the reuse determination unit 601, the POC number indicated by the derived variable refPOC and the POC number of the reference image corresponding to the reference number indicated by the variable curridx are the same, and the WP corresponding to the reference number indicated by the variable refIdx. It is determined whether the parameter and the WP parameter corresponding to the reference number indicated by the variable curridx are the same (step S207).

When both POC numbers and both WP parameters are the same (Yes in step S207), the reuse determination unit 601 sets the flag reuse_wp_flag to true (step S208) and substitutes the value of the variable curridx into the variable reuse_ref_idx (step S208). Step S209). On the other hand, when both POC numbers or both WP parameters are not the same (No in step S207), the processes of steps S208 and S209 are not performed. It should be noted that the configuration may be such that the matching of the POC numbers is not confirmed.

Subsequently, the reuse determination unit 601 increments the variable curridx (step S210).

Subsequently, the reuse determination unit 601 determines whether the value of the variable curridx is larger than num_of_active_ref_l0_minus1 which is the value obtained by subtracting 1 from the maximum number of list 0 (step S211), and is between num_of_active_ref_l0_minus1 and less (No in step S211). ), The processing of steps S207 to S210 is repeated.

When it becomes larger than num_of_active_ref_l0_minus1 (Yes in step S211), the reuse determination unit 601 completes the confirmation of the list 0 and increments the variable refIdx (step S212).

Subsequently, the reuse determination unit 601 determines whether the value of the variable refIdx is larger than num_of_active_ref_l1_minus1 which is the value obtained by subtracting 1 from the maximum number in the list 1 (step S213), and is between num_of_active_ref_l1_minus1 and below (No in step S213). ), The processing of steps S203 to S212 is repeated.

When it becomes larger than num_of_active_ref_l1_minus1 (Yes in step S213), the reuse determination unit 601 completes the confirmation of the list 1 and outputs the WP parameter information after the reuse determination to the index generation unit 602. The index generation unit 602 maps the WP parameter information input from the reuse determination unit 601 to a predetermined syntax element described later, generates index information, and outputs the index information.

FIG. 27 is a diagram showing an example of the pred weight table syntax 506 of the second embodiment. Among the syntax elements shown in FIG. 27, those having the symbols l0 and l1 correspond to lists 0 and 1, respectively. Further, the syntax elements having the same prefix as in FIG. 21 are used as the same variable except that the reference list is handled differently, although there is a difference between the list 0 and the list 1 instead of the common list. For example, luma_weight_l0 [i] is a syntax element representing the i-th weighting factor of the reference number in the reference list l0.

luma_log2_weight_denom and chroma_log2_weight_denom have the same configuration as that of the first embodiment. First, luma_weight_l0_flag, luma_weight_l0 [i], luma_offset_l0 [i], chroma_weight_l0_flag, chroma_weight_l0 [i] [j], and chroma_offset_l0 [i] [j] are defined for list 0 with the symbol l0. Next, the syntax elements for Listing 1 with the symbol l1 are defined.

Here, reuse_wp_flag is a syntax element indicating whether to reuse the WP parameter of l0 corresponding to List 0. If reuse_wp_flag is 1, reuse_ref_idx is encoded without the weighting factors and offsets being encoded. reuse_ref_idx indicates which reference number of l0 corresponding to Listing 0 to use the WP parameter. For example, when reuse_ref_idx is 1, the WP parameter corresponding to the reference number 1 in the list 0 is copied to the WP parameter corresponding to the reference number i in the list 1. When reuse_wp_flag is 0, luma_weight_l1_flag, luma_weight_l1 [i], luma_offset_l1 [i], chroma_weight_l1_flag, chroma_weight_l1 [i] [j], and chroma_offset_l1 [i] [j] are encoded, as in Listing 0.

As described above, in the second embodiment, when the index setting unit 608 has a combination in the WP parameter information that includes the same reference image in the two reference lists of list 0 and list 1, duplicate WP parameters are set. By reusing, duplicate redundant WP parameters are deleted from the WP parameter information and index information is generated. Therefore, according to the second embodiment, the code amount of the index information can be reduced.

In the second embodiment, the WP parameter information corresponding to the list 0 is encoded as before, and when encoding the WP parameter information corresponding to the list 1, it is checked whether or not the same reference image is referred to in the list 0. However, when the same reference image is referred to and the WP parameters are also the same, the flag for reusing the WP parameters in Listing 0 is encoded, and the information indicating which reference number in Listing 0 corresponds to is encoded. To become. Since this information is much less than the amount of information regarding the weighting coefficient and the offset, the amount of code for the WP parameter information can be significantly reduced as compared with the case where the list 0 and the list 1 are encoded separately. ..

Further, in the second embodiment, when the reference numbers included in the list 0 and the list 1 point to the same reference image and the WP parameters are different, it is possible to set different WP parameters. That is, when there is a combination pointing to the same reference image in the reference list, the same WP parameter is forcibly set in the first embodiment, and the same in the second embodiment only when the WP parameter is the same. Remove redundant representations of WP parameters.

(Third Embodiment)
The third embodiment will be described. FIG. 28 is a block diagram showing an example of the configuration of the coding device 700 of the third embodiment. The coding device 700 of the third embodiment is different from the coding device 100 of the first embodiment in that it further includes a motion information memory 701, a motion information acquisition unit 702, and a changeover switch 703. In the following, the differences from the first embodiment will be mainly explained, and the components having the same functions as those of the first embodiment will be given the same names and codes as those of the first embodiment, and the explanations will be given. Omit.

The motion information memory 701 temporarily stores the motion information applied to the pixel block whose coding has been completed as reference motion information. In the motion information memory 701, the motion information may be compressed by subsampling or the like to reduce the amount of information.

FIG. 29 is a diagram showing a detailed example of the motion information memory 701 of the third embodiment. As shown in FIG. 29, the motion information memory 701 is held in units of frames or slices, and the spatial direction reference motion information memory 701A that stores motion information on the same frame as reference motion information 710 and already encoded. Further has a time direction reference motion information memory 701B for storing the motion information of the frame at which the above is completed as the reference motion information 710. The time direction reference motion information memory 701B may have a plurality of reference frames depending on the number of reference frames used for prediction by the coded target frame.

The reference motion information 710 is held in the spatial direction reference motion information memory 701A and the time direction reference motion information memory 701B in predetermined area units (for example, 4 × 4 pixel block units). The reference motion information 710 further has information indicating whether the region is encoded by the inter-prediction described later or the intra-prediction described later. In addition, the pixel block (coding unit or prediction unit) is H.I. As in the skip mode defined by 264, the direct mode, or the merge mode described later, the value of the motion vector in the motion information is not encoded, and is inter-predicted using the motion information predicted from the encoded region. Even in this case, the motion information of the pixel block is retained as the reference motion information 710.

When the coding process of the frame or slice to be encoded is completed, the spatial direction reference motion information memory 701A of the frame is changed to the time direction reference motion information memory 701B used for the frame to be encoded next. To. At this time, in order to reduce the memory capacity of the time direction reference motion information memory 701B, the motion information may be compressed and the compressed motion information may be stored in the time direction reference motion information memory 701B.

The motion information acquisition unit 702 receives reference motion information from the motion information memory 701 and outputs motion information B used for the coded pixel block. The details of the motion information acquisition unit 702 will be described later.

The changeover switch 703 selects either motion information B output from the motion information acquisition unit 702 or motion information A output from the motion evaluation unit 109 according to the prediction mode information described later, and generates a prediction image as motion information. Output to unit 107. The motion information A output from the motion evaluation unit 109 encodes the difference information from the predicted motion vector acquired from the predicted motion vector acquisition unit (not shown) and the predicted motion vector acquisition position information. Such a prediction mode is hereinafter referred to as an inter mode. On the other hand, the motion information B output from the motion information acquisition unit 702 according to the prediction mode information is applied to the coded pixel block as it is by merging the motion information from the adjacent pixel blocks, and thus is related to other motion information. Coding of information (eg, motion vector difference information) is not required. Hereinafter, such a prediction mode will be referred to as a merge mode.

The prediction mode information follows the prediction mode controlled by the coding control unit 111, and includes the changeover information of the changeover switch 703.

The details of the motion information acquisition unit 702 will be described below.

The motion information acquisition unit 702 receives the reference motion information as an input and outputs the motion information B to the changeover switch 703. 30A and 30B are diagrams showing an example of a block position for deriving a motion information candidate for the coded pixel block of the third embodiment. A to E in FIG. 30A indicate spatially adjacent pixel blocks for deriving motion information candidates to the coded pixel block, and T in FIG. 30B indicates temporally adjacent pixel blocks for deriving motion information candidates. ..

FIG. 31 is a diagram showing an example of the relationship between the pixel block positions of the plurality of motion information candidates of the third embodiment and the pixel block position index (idx). The block positions of the motion information candidates are arranged in the order of spatially adjacent block positions (A to E) and temporally adjacent pixel block positions (T), and Available (inter-prediction is applied) in that order. ) If there is a pixel block, idx is sequentially assigned until the maximum number of motion information (N-1) stored in the merge mode storage list MergeCandList is reached.

FIG. 32 is a flowchart showing an example of the storage process in the MergeCandList of the third embodiment.

First, the motion information acquisition unit 702 initializes the number of stored numMergeCands in the MergeCandList to 0 (step S301).

Subsequently, the motion information acquisition unit 702 determines whether or not all the spatially adjacent blocks (for example, blocks A to E) are available (steps S302 and S303). In the case of Available (Yes in step S303), the motion information acquisition unit 702 stores the motion information of the space adjacent block in MergeCandList and increments numMergeCand (step S304). If it is not Available (No in step S303), the motion information acquisition unit 702 does not perform the process of step S304.

When the motion information acquisition unit 702 executes the processes of steps S303 and S304 for all the spatially adjacent blocks (step S305), the motion information acquisition unit 702 determines whether or not the temporally adjacent blocks (for example, the block T) are available. (Step S306).

In the case of Available (Yes in step S306), the motion information acquisition unit 702 stores the motion information of the time-adjacent block T in the MergeCandList and increments the numMergeCand (step S307). If it is not Available (No in step S306), the motion information acquisition unit 702 does not perform the process of step S307.

Subsequently, the motion information acquisition unit 702 deletes the duplicate motion information in the MergeCandList and decrements the numMergeCand by the deleted number (step S308). In order to determine whether or not there is duplicate motion information in MergeCandList, it is determined whether the two types of motion information (motion vector mv, reference number refIdx) and WP parameter information in MergeCandList match, and all the information is If they match, remove one from MergeCandList [].

FIG. 33 is a diagram showing an example of a storage list of motion information according to the third embodiment. Specifically, FIG. 33 is a diagram showing motion information (motion vector mv) corresponding to each idx according to the table shown in FIG. 31 with N = 5. , Reference number refIdx) storage list (MergeCandList [idx]) is shown as an example. In FIG. 33, idx = 0 is the reference motion information of the list 0 (motion vector mv0L0, reference number refIdx0L0), idx = 1 is the reference motion information of the list 1 (motion vector mv1L1, reference number refIdx1L1), idx = 2- Reference numeral 4 stores both reference motion information (motion vector mv2L0, mv2L1, reference number refIdx2L0, refIdx2L1) of both list 0 and list 1.

The coded pixel block selects one of the maximum N types of idx, derives the reference motion information of the pixel block corresponding to the selected idx from MergeCandList [idx], and outputs it as motion information B. When the reference motion information does not exist, the coded pixel block outputs the motion information having a zero vector as the predicted motion information candidate B. The selected idx information (syntax merge_idx described later) is included in the prediction mode information and is encoded by the coding unit 110.

The blocks that are spatially and temporally adjacent to each other are not limited to FIGS. 30A and 30B, and may be located at any position as long as they are already encoded regions. Further, as another example of the above list, the maximum number (N) and the order of storage in the MergeCandList are not limited to those described above, and any order / maximum number may be used.

If the coded slice is a B slice and the maximum number of stored pixels (N) is not reached even if the reference motion information of spatially and temporally adjacent pixel blocks is stored in MergeCandList [], MergeCandList at that point. The motion information of List 0 and List 1 stored in [] is combined with each other to generate a new bidirectional prediction and store it in MergeCandList [].

FIG. 34 is a flowchart showing an example of a method of storing motion information in the storage list of the third embodiment.

First, the motion information acquisition unit 702 sets the number of stored pixels in the MergeCandList from spatially and temporally adjacent pixel blocks to numInputMergeCand, initializes the variable numMergeCand indicating the current number of stored pixels with numInputMergeCand, and sets the variables combIdx and combCnt. Initialize at 0 (step S401).

Subsequently, the motion information acquisition unit 702 derives the variable l0CandIdx and the variable l1CandIdx from combIdx using the newly generated combination of bidirectional predictions (see FIG. 35) (step S402). The combination for deriving the variable l0CandIdx and the variable l1CandIdx from combIdx is not limited to the example shown in FIG. 36, and may be rearranged in any order as long as they do not overlap.

Subsequently, the motion information acquisition unit 702 sets the reference motion information (two types of list 0 and list 1 in the case of bidirectional prediction) stored in MergeCandList [l0CandIdx] in l0Cand. The reference number in l0Cand is called refIdxL0l0Cand, and the motion vector is called mvL0l0Cand. Similarly, the motion information acquisition unit 702 sets the reference motion information stored in the MergeCandList [l1CandIdx] in l1Cand. The reference number in l1Cand is referred to as refIdxL1l1Cand, and the motion vector is referred to as mvL1l1Cand (step S403). The combination of motion information for bidirectional prediction using l0Cand as the motion information in Listing 0 and L1Cand as the motion information in Listing 1 is called combCand.

Subsequently, the motion information acquisition unit 702 determines whether or not the blocks referred to by the l0Cand and the l1Cand are the same by using the conditional expressions (12) to (15) (step S404).

Are l0Cand and l1Cand Available? … (12)

RefPicOrderCnt (refIdxL0l0Cand, L0)! = RefPicOrderCnt (refIdxL1l1Cand, L1)… (13)

mvL0l0Cand! = mvL1l1Cand… (14)

WpParam (refIdxL0l0Cand, L0)! = WpParam (refIdxL1l1Cand, L1)… (15)

RefPicOrderCnt (refIdx, LX) is a function that derives the POC (Picture Order Count) of the reference frame corresponding to the reference number refIdx in the reference list X (X = 0, 1). WpParam (refIdx, LX) is a function for deriving the WP parameter information of the reference frame corresponding to the reference number refIdx in the reference list X (X = 0, 1). This function is Yes if the reference frame in Listing 0 and the reference frame in Listing 1 are not the same WP parameters (WP presence / absence WP_flag, weighting factor Weight, offset Offset), if they are exactly the same parameters. Returns No. As another example of WpParam (), it is not necessary to check the match of all parameters, and the judgment is made using only some data of the WP parameter, such as checking the match of only WP_flag and offset Offset with or without WP. You may.

When the conditional expression (12) is satisfied and any of the conditional expressions (13) to (15) is satisfied (Yes in step S404), the motion information acquisition unit 702 is the same as l0Cand and l1Cand in the mergeCandList []. It is determined whether or not there is a bidirectional prediction using the combination of (step S405).

If there is no same combination of l0Cand and l1Cand in mergeCandList [] (Yes in step S405), the motion information acquisition unit 702 adds the combCand to the end of the mergeCandList (step S406). Specifically, the motion information acquisition unit 702 substitutes combCand into mergeCandList [numMergeCand] and increments numMergeCand and combCnt.

Subsequently, the motion information acquisition unit 702 increments combIdx (step S407).

Subsequently, the motion information acquisition unit 702 determines whether or not to end the storage in the mergeCandList [] by using the conditional expressions (16) to (18) (step S408).

combIdx == numInputMergeCand * (numInputMergeCand? 1))… (16)

numMergeCand == maxNumMergeCand… (17)

combCnt == N… (18)

If any of the conditional expressions (16) to (18) is satisfied (Yes in step S408), the process ends, and if any of the conditional expressions (16) to (18) is satisfied (No in step S408). ), Return to step S402.

The above is an explanation of the processing when the coded slice is a B slice and the maximum number of stored pixels (N) is not reached even if the reference motion information of spatially and temporally adjacent pixel blocks is stored in MergeCandList []. Is. As a result, the motion information B is output from the motion information acquisition unit 702 to the changeover switch 703.

FIG. 36 is a diagram showing an example of the pred unit syntax 512 of the third embodiment. skip_flag is a flag indicating whether or not the prediction mode of the coding unit to which the prediction unit syntax belongs is the skip mode. When skip_flag is 1, it indicates that syntaxes other than prediction mode information (coding unit syntax, prediction unit syntax, transform unit syntax) are not encoded. When skip_flag is 0, it indicates that the prediction mode of the coding unit to which the prediction unit syntax belongs is not the skip mode.

Next, merge_flag, which is a flag indicating whether or not the coded pixel block (prediction unit) is in the merge mode, is encoded. Merge_flag indicates that the prediction unit is in merge mode when the value is 1, and indicates that the prediction unit uses intermode when the value is 0. In the case of Merge_flag, merge_idx, which is the information for specifying the pixel block position index (idx) described above, is encoded.

When Merge_flag is 1, prediction unit syntaxes other than merge_flag and merge_idx need not be encoded, and when Merge_flag is 0, it indicates that the prediction unit is in intermode.

As described above, in the third embodiment, when the weighted motion compensation and the merge mode are applied at the same time, the encoding device 700 stores the reference motion information of the pixel blocks adjacent in space and time in the merge mode storage list (MergeCandList). When deleting the duplicate motion information in), even if the motion vector and the reference number match, if the WP parameter information is different, the problem of being deleted from the MergeCandList is solved. In addition, if the maximum number of stored items (N) is not reached even if the items are stored in the MergeCandList, the motion information of the list 0 and the list 1 stored in the MergeCandList at that time is combined with each other to perform a new bidirectional prediction. This solves the problem that two types of motion information used for bidirectional prediction refer to the same block when they are generated and stored in MergeCandList [].

When two types of motion information used for bidirectional prediction refer to the same block, the predicted value by bidirectional prediction and the predicted value by unidirectional prediction match. In general, the prediction efficiency of bidirectional prediction is higher than the prediction efficiency of unidirectional prediction, so it is desirable that two types of motion information used for bidirectional prediction do not refer to the same block. When the coding device 700 determines whether or not two types of motion information used for bidirectional prediction refer to the same block, the motion vector, the reference frame number position (POC) derived from the reference number, and the like. In addition, the parameter of weighted motion compensation is introduced into the judgment item. As a result, the coding apparatus 700 determines that even if the motion vector and the reference frame number position (POC) are the same, the two types of motion information having different weighted motion compensation parameters do not refer to the same block. It is possible to improve the prediction efficiency of the motion information used for the two-way prediction stored in the storage list of the merge mode.

As another example of the storage process in the MergeCandList, when the motion information of the time adjacent block T is stored in the MergeCandList, the WP parameter in the reference frame of the time adjacent block T and the WP parameter in the reference frame of the coded pixel block are set. It may be determined whether they match, and only when the WP parameters match, they may be stored in the MergeCandList. This is because when the WP parameter in the reference frame of the time-adjacent block T and the WP parameter in the reference frame of the coded pixel block are different, it can be inferred that the correlation of motion information between the time-adjacent block T and the coded pixel block decreases. Is.

FIG. 37 is a flowchart showing another example of the storage process in the MergeCandList of the third embodiment. The flowchart shown in FIG. 38 is the same as the flowchart shown in FIG. 32 except for step S507.

As yet another example, when the WP parameter in the reference frame of the time-adjacent block T and the WP parameter in the reference frame of the coded pixel block are different, the coded pixel of the blocks spatially adjacent to the block T A block having the same WP parameters in the reference frame of the block may be replaced with the block T. At this time, the correlation of motion information does not decrease between the time-adjacent block T and the coded pixel block.

(Fourth Embodiment)
In the fourth embodiment, a decoding device that decodes the coded data encoded by the coding device of the first embodiment will be described.

FIG. 38 is a block diagram showing an example of the configuration of the decoding device 800 of the fourth embodiment.

The decoding device 800 decodes the encoded data stored in an input buffer (not shown) or the like into a decoded image, and outputs the encoded data as an output image to an output buffer (not shown). The coded data is output from, for example, the coding device 100 of FIG. 1, and is input to the decoding device 800 via a storage system, a transmission system, a buffer, or the like (not shown).

As shown in FIG. 38, the decoding device 800 includes a decoding unit 801, an inverse quantization unit 802, an inverse orthogonal conversion unit 803, an addition unit 804, a prediction image generation unit 805, and an index setting unit 806. Be prepared. The inverse quantization unit 802, the inverse orthogonal conversion unit 803, the addition unit 804, and the prediction image generation unit 805 are the inverse quantization unit 104, the inverse orthogonal conversion unit 105, the addition unit 106, and the prediction image generation unit 107 of FIG. 1, respectively. It is an element that is substantially the same as or similar to. The decoding control unit 807 shown in FIG. 38 controls the decoding device 800, and can be realized by, for example, a CPU or the like.

The decoding unit 801 performs decoding based on the syntax for each frame or field for decoding the encoded data. The decoding unit 801 sequentially entropy-decodes the code string of each syntax, and the motion information including the prediction mode, the motion vector, and the reference number, the index information for the weighted motion compensation prediction, and the code such as the quantization conversion coefficient. Reproduce the coding parameters of the block to be converted. The coding parameters are all parameters required for decoding, such as information on conversion coefficients and information on quantization other than the above. The decoding unit 801 outputs motion information, index information, and quantization conversion coefficient, inputs the quantization conversion coefficient to the inverse quantization unit 802, inputs index information to the index setting unit 806, and predicts motion information. Input to the generation unit 805.

The inverse quantization unit 802 performs an inverse quantization process on the quantization conversion coefficient input from the decoding unit 801 to obtain a restoration conversion coefficient. Specifically, the inverse quantization unit 802 performs inverse quantization according to the quantization information used in the decoding unit 801. More specifically, the inverse quantization unit 802 multiplies the quantization step size derived from the quantization information by the quantization conversion coefficient to obtain the restoration conversion coefficient. The inverse quantization unit 802 outputs the restoration conversion coefficient and inputs it to the inverse orthogonal conversion unit 803.

The inverse orthogonal conversion unit 803 performs an inverse orthogonal transformation corresponding to the orthogonal transformation performed on the coding side with respect to the restoration conversion coefficient input from the inverse quantization unit 802, and obtains a restoration prediction error. The inverse orthogonal conversion unit 803 outputs the restoration prediction error and inputs it to the addition unit 804.

The addition unit 804 adds the restoration prediction error input from the inverse orthogonal conversion unit 803 and the corresponding predicted image to generate a decoded image. The addition unit 804 outputs the decoded image and inputs it to the prediction image generation unit 805. Further, the addition unit 804 outputs the decoded image as an output image to the outside. The output image is then temporarily stored in an external output buffer or the like (not shown), and is transferred to a display device system such as a display or monitor (not shown) or a video device system according to the output timing managed by the decoding control unit 807, for example. It is output.

The index setting unit 806 receives the index information input from the decoding unit 801, confirms the reference list (list number) and the reference image (reference number), outputs the WP parameter information, and outputs the WP parameter information to the prediction image generation unit 805. input.

FIG. 39 is a block diagram showing an example of the configuration of the index setting unit 806 of the fourth embodiment. As shown in FIG. 39, the index setting unit 806 includes a reference image confirmation unit 901 and a WP parameter generation unit 902.

The reference image confirmation unit 901 receives the index information from the decoding unit 801 and confirms whether or not the reference numbers included in the two reference lists point to the same reference image.

Here, the reference number in the reference list is, for example, H. It has already been deciphered by the method specified in 264 and the like. Therefore, the reference image confirmation unit 901 has a H. According to the management of the Decoded Picture Buffer (DPB) specified in 264 and the like, it is possible to confirm whether or not there is a combination pointing to the same reference image by using the derived reference list and reference number. Regarding the control of DPB, H. The method described in 264 or the like may be used, or another method may be used. Here, it is sufficient that the reference list and the reference number are determined in advance by DPB control.

The reference image confirmation unit 901 creates a common reference list, and confirms the reference list and the reference number in the scan order shown in FIG. The common reference list is created according to the pseudo code shown in the formula (10).

The WP parameter generation unit 902 generates WP parameter information corresponding to the reference list and the reference number from the created common reference list based on the relationship between the reference list and the reference number confirmed by the reference image confirmation unit 901. Is output and input to the prediction image generation unit 805. Since the WP parameter information has already been described with reference to FIGS. 9A and 9B, the description thereof will be omitted.

The WP parameter generation unit 902 uses the common_scan_list () function and the common_scan_ref_idx () function according to the common list to draw a reference list and a reference number in the scan, and fills the missing places with the WP parameter information.

That is, the common list of FIGS. 15 and 16 is restored to FIGS. 11 and 13, respectively, and the index information corresponding to the common list is obtained by the WP parameters shown in FIGS. 9A and 9B based on these correspondences. Assign to information.

The scan order shown in FIG. 14 is an example, and any other scan order may be used as long as it is a predetermined scan order. Further, the pseudo code shown by the mathematical formula (10) is an example, and if the purpose of this processing can be realized, it is possible to add processing and reduce redundant processing.

Returning to FIG. 38, the prediction image generation unit 805 uses the motion information input from the decoding unit 801, the WP parameter information input from the index setting unit 806, and the decoded image input from the addition unit 804 to perform the prediction image. To generate.

Here, the details of the prediction image generation unit 805 will be described with reference to FIG. Like the prediction image generation unit 107, the prediction image generation unit 805 includes a plurality of frame motion compensation units 201, a memory 202, a unidirectional motion compensation unit 203, a prediction parameter control unit 204, a reference image selector 205, and a frame memory 206. And a reference image control unit 207.

The frame memory 206 stores the decoded image input from the addition unit 106 as the reference image under the control of the reference image control unit 207. The frame memory 206 has a plurality of memory sets FM1 to FMN (N ≧ 2) for temporarily holding the reference image.

The prediction parameter control unit 204 prepares a plurality of combinations of the reference image number and the prediction parameter as a table based on the motion information input from the decoding unit 801. Here, the motion information refers to a motion vector indicating the amount of motion deviation used in motion compensation prediction, a reference image number, information on a prediction mode such as unidirectional / bidirectional prediction, and the like. Prediction parameters refer to information about motion vectors and prediction modes. Then, the prediction parameter control unit 204 selects and outputs a combination of the reference image number and the prediction parameter used for generating the prediction image based on the motion information, inputs the reference image number to the reference image selector 205, and inputs the prediction parameter. Is input to the unidirectional motion compensation unit 203.

The reference image selector 205 is a switch that switches which output terminal of the frame memories FM1 to FMN of the frame memory 206 is connected according to the reference image number input from the prediction parameter control unit 204. For example, if the reference image number is 0, the reference image selector 205 connects the output end of the FM1 to the output end of the reference image selector 205, and if the reference image number is N-1, the reference image selector 205 refers to the output end of the FMN. Connect to the output end of the image selector 205. The reference image selector 205 outputs a reference image stored in the frame memory to which the output end is connected among the frame memories FM1 to FMN of the frame memory 206, and inputs the reference image to the unidirectional motion compensation unit 203. In the decoding device 800, since the reference image is not used except for the prediction image generation unit 805, it is not necessary to output the reference image to the outside of the prediction image generation unit 805.

The unidirectional prediction motion compensation unit 203 performs motion compensation prediction processing according to the prediction parameters input from the prediction parameter control unit 204 and the reference image input from the reference image selector 205, and generates a unidirectional prediction image. Since the motion compensation prediction has already been described with reference to FIG. 5, the description thereof will be omitted.

The unidirectional prediction motion compensation unit 203 outputs the unidirectional prediction image and temporarily stores it in the memory 202. Here, when the motion information (prediction parameter) indicates bidirectional prediction, the multi-frame motion compensation unit 201 performs weighted prediction using two types of unidirectional prediction images, so that the unidirectional prediction motion compensation unit 203 Stores the first unidirectional prediction image in the memory 202, and directly outputs the second unidirectional prediction image to the multi-frame motion compensation unit 201. Here, the unidirectional prediction image corresponding to the first is referred to as the first prediction image, and the unidirectional prediction image corresponding to the second is referred to as the second prediction image.

In addition, two unidirectional motion compensation units 203 may be prepared, and each may generate two unidirectional prediction images. In this case, when the motion information (prediction parameter) indicates unidirectional prediction, if the unidirectional motion compensation unit 203 directly outputs the first unidirectional prediction image as the first prediction image to the plurality of frame motion compensation unit 201. Good.

The multi-frame motion compensation unit 201 uses the first prediction image input from the memory 202, the second prediction image input from the unidirectional prediction motion compensation unit 203, and the WP parameter information input from the motion evaluation unit 109. , Performs weighted prediction to generate a prediction image. The multi-frame motion compensation unit 201 outputs a predicted image and inputs it to the addition unit 804.

Here, the details of the plurality of frame motion compensation units 201 will be described with reference to FIG. Like the predicted image generation unit 107, the multi-frame motion compensation unit 201 includes a default motion compensation unit 301, a weighted motion compensation unit 302, a WP parameter control unit 303, and WP selectors 304 and 305.

The WP parameter control unit 303 outputs the WP application flag and weight information based on the WP parameter information input from the index setting unit 806, inputs the WP application flag to the WP selectors 304 and 305, and weights the weight information. Input to the motion compensation unit 302.

Here, the WP parameter information includes the fixed-point accuracy of the weighting factor, the first WP application flag corresponding to the first predicted image, the first weighting factor, and the first offset, and the second WP adaptation corresponding to the second predicted image. Contains information on flags, second weighting factors, and second offset. The WP application flag is a parameter that can be set for each corresponding reference image and signal component, and indicates whether or not to perform weighted motion compensation prediction. The weight information includes information on the fixed-point accuracy of the weighting factors, the first weighting factor, the first offset, the second weighting factor, and the second offset. The WP parameter information represents the same information as in the first embodiment.

Specifically, when the WP parameter information is input from the index setting unit 806, the WP parameter control unit 303 separates and outputs the WP parameter information into the first WP application flag, the second WP application flag, and the weight information. , The first WP application flag is input to the WP selector 304, the second WP application flag is input to the WP selector 305, and the weight information is input to the weighted motion compensation unit 302.

The WP selectors 304 and 305 switch the connection end of each predicted image based on the WP application flag input from the WP parameter control unit 303. The WP selectors 304 and 305 connect their output ends to the default motion compensation unit 301 when their respective WP application flags are 0. Then, the WP selectors 304 and 305 output the first predicted image and the second predicted image and input them to the default motion compensation unit 301. On the other hand, the WP selectors 304 and 305 connect their output ends to the weighted motion compensation unit 302 when their respective WP application flags are 1. Then, the WP selectors 304 and 305 output the first predicted image and the second predicted image and input them to the weighted motion compensation unit 302.

The default motion compensation unit 301 performs average value processing based on the two unidirectional prediction images (first prediction image and second prediction image) input from the WP selectors 304 and 305, and generates a prediction image. Specifically, when the first WP application flag and the second WP application flag are 0, the default motion compensation unit 301 performs the average value processing based on the mathematical formula (1).

When the prediction mode indicated by the motion information (prediction parameter) is unidirectional prediction, the default motion compensation unit 301 uses only the first prediction image to obtain the final prediction image based on the mathematical formula (4). calculate.

The weighted motion compensation unit 302 is based on two unidirectional prediction images (first prediction image and second prediction image) input from the WP selectors 304 and 305 and weight information input from the WP parameter control unit 303. Perform weighted motion compensation. Specifically, when the first WP application flag and the second WP application flag are 1, the weighted motion compensation unit 302 performs weighting processing based on the mathematical formula (7).

When the calculation accuracy of the first prediction image and the second prediction image is different from that of the prediction image, the weighted motion compensation unit 302 rounds the logWDC, which is the fixed-point accuracy, by controlling it as in the mathematical formula (8). To realize.

Further, when the prediction mode indicated by the motion information (prediction parameter) is unidirectional prediction, the weighted motion compensation unit 302 uses only the first prediction image and the final prediction image based on the mathematical formula (9). Is calculated.

When the calculation accuracy of the first prediction image and the second prediction image is different from that of the prediction image, the weighted motion compensation unit 302 sets the logWDC, which is the fixed-point accuracy, as in the mathematical formula (8) as in the case of bidirectional prediction. The rounding process is realized by controlling to.

Since the fixed-point accuracy of the weighting coefficient has already been described with reference to FIG. 7, the description thereof will be omitted. In the case of unidirectional prediction, various parameters (second WP adaptation flag, second weighting coefficient, and second offset information) corresponding to the second prediction image are not used, so they are set to predetermined initial values. It may have been.

The decoding unit 801 uses the syntax 500 shown in FIG. The syntax 500 shows the structure of the coded data to be decoded by the decoding unit 801. Since the syntax 500 has already been described with reference to FIG. 18, the description thereof will be omitted. Further, since the picture parameter set syntax 505 has already been described with reference to FIG. 19, except that the coding is decoding, the description thereof will be omitted. Further, the slice header syntax 507 has already been described with reference to FIG. 20, except that the coding is decoding, and thus the description thereof will be omitted. Further, the pread weight table syntax 508 has already been described with reference to FIG. 21, except that the coding is decoding, and thus the description thereof will be omitted.

As described above, in the fourth embodiment, the decoding device 800 is a bidirectional prediction slice in which bidirectional prediction can be selected, and weighted motion using two indexes having the same reference image but different reference image numbers. By decoding the indexes having the same value twice when performing compensation, the problem that the coding efficiency is lowered is solved.

By rearranging the two reference lists possessed by the bidirectional prediction slice and the reference numbers defined for each list into a common list and a common reference number, a combination of reference numbers pointing to the same reference image can be created in the reference list. Since it is not included, the decoding device 800 can reduce the amount of redundant index code.

(Modified example of the fourth embodiment)
A modified example of the fourth embodiment will be described. In the modified example of the fourth embodiment, the syntax element used by the decoding unit 801 is different from the syntax element of the fourth embodiment. Since the sequence parameter set syntax 504 has already been described with reference to FIG. 22, except that the coding is decoding, the description thereof will be omitted. The adaptation parameter set syntax 506 has already been described with reference to FIG. 23, except that the coding is decoding, and thus the description thereof will be omitted. Since the pred weight table syntax 508 has already been described with reference to FIG. 24 except that the coding is decoding, the description thereof will be omitted.

As described above, according to the modified example of the fourth embodiment, the code of the WP parameter information when one frame is divided into a plurality of slices by adopting the structure in which the pred weight table syntax 508 is called from the adaptation parameter set syntax 506. The amount can be significantly reduced.

For example, the adaptation parameter set syntax 506 having three different types of WP parameter information to be used is first decoded, and the slice header syntax 507 calls the necessary WP parameter information using aps_id depending on the situation. , The code amount can be reduced as compared with the configuration in which the WP parameter information is always decoded by the slice header syntax 507.

(Fifth Embodiment)
In the fifth embodiment, a decoding device that decodes the coded data encoded by the coding device of the second embodiment will be described. In the decoding device 1000 of the fifth embodiment, the configuration of the index setting unit 1006 is different from that of the decoding device 800 of the fourth embodiment. In the following, the differences from the fourth embodiment will be mainly explained, and the components having the same functions as those of the fourth embodiment will be given the same names and codes as those of the fourth embodiment, and the explanations will be given. Omit.

FIG. 40 is a block diagram showing an example of the configuration of the index setting unit 1006 of the fifth embodiment. As shown in FIG. 40, the index setting unit 1006 includes a reuse determination unit 1001 and a WP parameter generation unit 1002.

The reuse determination unit 1001 checks the reuse flag and confirms whether or not to reuse the WP parameter in Listing 0. When reusing the WP parameter of the list 0, the WP parameter generation unit 1002 copies the WP parameter of the referenced list 0 to the WP parameter of the list 1 according to the information of which WP parameter of the list 0 to copy. The information regarding the reuse flag and the reference number indicating the reference destination is described in the syntax element, and the WP parameter generation unit 1002 restores the WP parameter information by interpreting the index information.

The reuse determination unit 1001 sets the list 0 according to, for example, the pseudo code shown in the mathematical expression (11).
Check if you want to reuse the WP parameters of.

Since the pred weight table syntax 508 has already been described with reference to FIG. 27 except that the coding is decoding, the description thereof will be omitted.

As described above, in the fifth embodiment, the decoding device 1000 is a bidirectional prediction slice in which bidirectional prediction can be selected, and weighted motion using two indexes having the same reference image but different reference image numbers. By decoding the indexes having the same value twice when performing compensation, the problem that the coding efficiency is lowered is solved.

The decoding device 1000 decodes the index of list 0 as before, and when decoding the index of list 1, confirms whether there is a combination of reference numbers pointing to the same reference image in the reference list, and has the same index. In the case of, by reusing the index used in List 1 with the index used in List 0, it is possible to avoid decoding the same index twice and reduce the amount of redundant index code.

(Sixth Embodiment)
In the sixth embodiment, a decoding device that decodes the coded data encoded by the coding device of the third embodiment will be described. FIG. 41 is a block diagram showing an example of the configuration of the decoding device 1100 according to the sixth embodiment. The decoding device 1100 of the sixth embodiment is different from the decoding device 800 of the fourth embodiment in that it further includes a motion information memory 1101, a motion information acquisition unit 1102, and a changeover switch 1103. In the following, the differences from the fourth embodiment will be mainly explained, and the components having the same functions as those of the fourth embodiment will be given the same names and codes as those of the fourth embodiment, and the explanations will be given. Omit.

The motion information memory 1101 temporarily stores the motion information applied to the pixel block for which decoding has been completed as reference motion information. In the motion information memory 1101, the motion information may be compressed by subsampling or the like to reduce the amount of information. As shown in FIG. 29, the motion information memory 1101 is held in units of frames or slices, and the spatial direction reference motion information memory 701A that stores motion information on the same frame as reference motion information 710, and decoding has already been completed. Further, the time direction reference motion information memory 701B for storing the motion information of the frame as the reference motion information 710 is provided. The time-direction reference motion information memory 701B may have a plurality of reference frames depending on the number of reference frames used for prediction by the decoding target frame.

The reference motion information 710 is held in the spatial direction reference motion information memory 701A and the time direction reference motion information memory 701B in predetermined area units (for example, 4 × 4 pixel block units). The reference motion information 710 further has information indicating whether the region is encoded by the inter-prediction described later or the intra-prediction described later. In addition, the pixel block (coding unit or prediction unit) is H.I. When the value of the motion vector in the motion information is not decoded and inter-predicted using the motion information predicted from the decoded area, as in the skip mode, the direct mode, or the merge mode described later in 264. Also, the motion information of the pixel block is held as the reference motion information 710.

When the decoding process of the frame or slice to be decoded is completed, the spatial direction reference motion information memory 701A of the frame is changed as the time direction reference motion information memory 701B used for the frame to be decoded next. At this time, in order to reduce the memory capacity of the time direction reference motion information memory 701B, the motion information may be compressed and the compressed motion information may be stored in the time direction reference motion information memory 701B.

The motion information acquisition unit 1102 receives reference motion information from the motion information memory 1101 and outputs motion information B used for the decoding pixel block. Since the operation of the motion information acquisition unit 1102 is the same as that of the motion information acquisition unit 702 of the third embodiment, the description thereof will be omitted.

The changeover switch 1103 selects either the motion information B output from the motion information acquisition unit 1102 or the motion information A output from the decoding unit 801 according to the prediction mode information described later, and uses the prediction image generation unit as the motion information. Output to 805. The motion information A output from the decoding unit 801 decodes the difference information from the predicted motion vector acquired from the predicted motion vector acquisition unit (not shown) and the predicted motion vector acquisition position information. Such a prediction mode is hereinafter referred to as an inter mode. On the other hand, the motion information B output from the motion information acquisition unit 1102 according to the prediction mode information merges the motion information from the adjacent pixel blocks and is applied as it is to the decoded pixel block. Therefore, information on other motion information. Decoding of (for example, motion vector difference information) is not necessary. Hereinafter, such a prediction mode will be referred to as a merge mode.

The prediction mode information follows the prediction mode controlled by the decoding control unit 807, and includes the switching information of the changeover switch 1103.

Since the pred unit syntax 512 has already been described with reference to FIG. 36 except that the coding is decoding, the description thereof will be omitted.

Since the effect of the sixth embodiment is the same as that of the third embodiment, the description thereof will be omitted.

As another example of the storage process in the MergeCandList, when the motion information of the time adjacent block T is stored in the MergeCandList, the WP parameter in the reference frame of the time adjacent block T and the WP parameter in the reference frame of the decoded pixel block are one. It may be determined whether or not it is done, and only when the WP parameters match, it may be stored in the MergeCandList. This is because when the WP parameter in the reference frame of the time-adjacent block T and the WP parameter in the reference frame of the decoded pixel block are different, it can be estimated that the correlation of motion information is reduced between the time-adjacent block T and the decoded pixel block. ..

As yet another example, when the WP parameter in the reference frame of the time-adjacent block T and the WP parameter in the reference frame of the decoding pixel block are different, the decoding pixel block among the blocks spatially adjacent to the block T A block having the same WP parameters in the reference frame may be replaced with a block T. At this time, the correlation of motion information does not decrease between the time-adjacent block T and the decoded pixel block.

(Modification example)
The first to third embodiments may be selectively used. That is, the information on which method to select is determined in advance, and when the bidirectional slice is selected, the information indicating whether to use the method of the first embodiment or the method of the second embodiment is used. You may switch. For example, it is possible to set a flag for switching between these methods in the slice header syntax shown in FIG. 20 and easily select the flag according to the coding situation. In addition, it may be determined in advance which method to use according to the configuration of a specific hardware. The prediction image generation unit and the index setting unit of the first to third embodiments are suitable for both hardware implementation and software implementation.

Further, the fourth to sixth embodiments may be selectively used. That is, the information on which method is to be selected is determined in advance, and when the bidirectional slice is selected, the information indicating whether to use the method of the fourth embodiment or the method of the fifth embodiment is used. You may switch. For example
It is also possible to set a flag for switching between these methods in the slice header syntax shown in FIG. 20 and easily select the flag according to the coding situation. In addition, it may be determined in advance which method to use according to the configuration of a specific hardware. The predicted image generation unit and the index setting unit of the fourth to sixth embodiments are suitable for both hardware implementation and software implementation.

Further, a syntax element not specified in the above embodiment may be inserted between the rows of the syntax table illustrated in the first to sixth embodiments, or a description regarding other conditional branches may be included. .. Further, the syntax table may be divided into a plurality of tables, or a plurality of syntax tables may be integrated. In addition, the terms of each of the illustrated syntactic elements can be changed arbitrarily.

Moreover, the modification of the first embodiment can be easily applied to the second embodiment. In this case, the maximum number of reference images corresponding to List 0 and List 1 may be set, or may be a predetermined maximum number.

Further, the modified example of the fourth embodiment can be easily applied to the fifth embodiment. In this case, the maximum number of reference images corresponding to List 0 and List 1 may be set, or may be a predetermined maximum number.

In the first to sixth embodiments, an example is described in which a frame is divided into rectangular blocks having a size of 16 × 16 pixels, and encoding / decoding is performed in order from the block on the upper left of the screen to the lower right (FIG. See 3A). However, the coding order and decoding order are not limited to this example. For example, coding and decoding may be performed in order from the lower right to the upper left, or coding and decoding may be performed so as to draw a spiral from the center of the screen toward the edge of the screen. Further, coding and decoding may be performed in order from the upper right to the lower left, or coding and decoding may be performed so as to draw a spiral from the edge of the screen toward the center of the screen. In this case, the positions of the adjacent pixel blocks that can be referred to change depending on the coding order, so the positions may be changed as appropriate.

In the first to sixth embodiments, the prediction target block sizes such as 4 × 4 pixel block, 8 × 8 pixel block, and 16 × 16 pixel block have been illustrated and described, but the prediction target block is a uniform block. It does not have to be a shape. For example, the prediction target block size may be a 16 × 8 pixel block, an 8 × 16 pixel block, an 8 × 4 pixel block, a 4 × 8 pixel block, or the like. Further, it is not necessary to unify all block sizes in one coding tree block, and a plurality of different block sizes may be mixed. When a plurality of different block sizes are mixed in one coding tree block, the amount of code for encoding or decoding the division information increases as the number of divisions increases. Therefore, it is desirable to select the block size in consideration of the balance between the code amount of the divided information and the quality of the locally decoded image or the decoded image.

In the first to sixth embodiments, for the sake of simplification, a comprehensive description of the color signal component is described without distinguishing between the luminance signal and the prediction process in the color difference signal. However, when the prediction process differs between the luminance signal and the color difference signal, the same or different prediction methods may be used. If different prediction methods are used for the luminance signal and the color difference signal, the prediction method selected for the luminance signal can be encoded or decoded in the same manner as the luminance signal.

In the first to sixth embodiments described above, for the sake of simplification, the luminance signal and the weighted motion compensation prediction processing in the color difference signal are not distinguished, and a comprehensive description of the color signal component is described. However, when the weighted motion compensation prediction processing differs between the luminance signal and the color difference signal, the same or different weighted motion compensation prediction processing may be used. If different weighted motion compensation prediction processes are used between the luminance signal and the color difference signal, the weighted motion compensation prediction process selected for the color difference signal can be encoded or decoded in the same manner as the luminance signal.

In the first to sixth embodiments, it is possible to insert a syntax element not specified in the present embodiment between the rows of the table shown in the syntax configuration, and other descriptions regarding conditional branching are included. It doesn't matter if it is. Alternatively, the syntax table can be divided and integrated into a plurality of tables. Moreover, it is not always necessary to use the same term, and it may be arbitrarily changed depending on the form to be used.

As described above, each embodiment realizes highly efficient weighted motion compensation prediction processing while solving the problem of encoding redundant information when performing weighted motion compensation prediction. Therefore, according to each embodiment, the coding efficiency is improved, and the subjective image quality is also improved.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

For example, it is also possible to provide a program that realizes the processing of each of the above embodiments by storing it in a computer-readable storage medium. The storage medium may be a storage medium such as a magnetic disk, an optical disk (CD-ROM, CD-R, DVD, etc.), a magneto-optical disk (MO, etc.), a semiconductor memory, etc., which can store a program and can be read by a computer. For example, the storage form may be any form.

Further, the program that realizes the processing of each of the above embodiments may be stored on a computer (server) connected to a network such as the Internet and downloaded to the computer (client) via the network.

100, 600, 700 Encoding device 101 Subtraction unit 102 Orthogonal conversion unit 103 Quantization unit 104 Inverse quantization unit 105 Inverse orthogonal conversion unit 106 Addition unit 107 Prediction image generation unit 108, 608 Index setting unit 109 Motion evaluation unit 110 Coding Unit 111 Coding control unit 201 Multiple frame motion compensation unit 202 Memory 203 Unidirectional motion compensation unit 204 Prediction parameter control unit 205 Reference image selector 206 Frame memory 207 Reference image control unit 301 Default motion compensation unit 302 Weighted motion compensation unit 303 WP Parameter control unit 304, 305 WP selector 401, 901 Reference image confirmation unit 402, 602 Index generation unit 601, 1001 Reuse judgment unit 701, 1101 Motion information memory 702, 1102 Motion information acquisition unit 703, 1103 Changeover switch 800, 1000, 1100 Decoding device 801 Decoding unit 802 Inverse quantization unit 803 Inverse orthogonal conversion unit 804 Addition unit 805 Prediction image generation unit 806, 1006 Index setting unit 807 Decoding control unit 902, 1002 WP parameter generation unit

Claims (2)

The first syntax, which is called from a higher syntax at a level higher than the slice level and is used for weighted motion compensation, and describes a plurality of parameters including a weighting coefficient specified by a reference number, and the first syntax.
The second syntax, which is a second syntax at a level higher than the slice level and describes the first syntax element including the maximum value that the reference number can take, and the second syntax.
A third syntax that describes a second syntax element indicating the validity or invalidity of the weighted motion compensation, and
Receives input of encoded data including
The first syntax element and the second syntax element are reproduced from the coded data, and the first syntax element and the second syntax element are reproduced.
From the reproduced first syntax element and the first syntax, the plurality of parameters are acquired by the reference number with the maximum value as the upper limit.
When the second syntax element indicates the effectiveness of the weighted motion compensation, a decoding process including the weighted motion compensation is executed using the reference image and the acquired plurality of parameters.
Decryption method. The first syntax, which is called from a higher syntax at a level higher than the slice level and is used for weighted motion compensation, and describes a plurality of parameters including a weighting coefficient specified by a reference number, and the first syntax.
The second syntax, which is a second syntax at a level higher than the slice level and describes the first syntax element including the maximum value that the reference number can take, and the second syntax.
A third syntax that describes a second syntax element indicating the validity or invalidity of the weighted motion compensation, and
Encode the encoded data, including
Coding method.