JP6105034B2 - Decoding method and decoding apparatus - Google Patents

Description

  Embodiments described herein relate generally to a decoding method and a decoding apparatus.

In recent years, an image encoding method that greatly improves encoding efficiency has been developed by ITU-T (International
In collaboration with Telecommunication Union Telecommunication Standardization Sector) and ISO (International Organization for Standardization) / IEC (International Electrotechnical Commission), ITU-T REC. H. H.264 and ISO / IEC 14496-10 (hereinafter referred to as “H.264”).

  H. H.264 discloses an inter-prediction coding scheme that eliminates temporal redundancy and realizes high coding efficiency by performing fractional accuracy motion compensated prediction using a coded image as a reference image. ing.

  Also, a scheme for encoding moving images including fade and dissolve effects with higher efficiency than the inter prediction encoding schemes in ISO / IEC MPEG (Moving Picture Experts Group) -1, 2, 4 has been proposed. In this method, as a framework for predicting a change in brightness in the time direction, fractional accuracy motion compensation prediction is performed on an input moving image having luminance and two color differences. Then, the prediction image is multiplied by the weighting factor using an index indicating a combination including the reference image, the weighting factor for each luminance and two color differences, and the offset for each luminance and two color differences, and the offset is added.

JP 2004-7377 A

  However, in the conventional technology as described above, when performing weighted motion compensation using two indexes having the same reference image but different reference image numbers in a bidirectional prediction slice in which bidirectional prediction can be selected, An index having the same value may be encoded twice, resulting in a decrease in encoding efficiency. The problem to be solved by the present invention is to provide a decoding method and a decoding apparatus capable of improving encoding efficiency.

The method of decoding embodiment is a than the slice level is called from a higher-level adaptation parameter set syntax Resid Ntakusu, spreads weight table syntax that describes a plurality parameters including weighting coefficient specified by the reference number and the a sheet Ntakusu of higher level than the slice level, the input of the coded data including a sequence parameter set syntax that describes a specific be cie Ntakusu elements the maximum number of the reference number possible values receiving the reproduced pre-carboxymethyl Ntakusu elements from a sequence parameter set syntax, from said a reproduced before carboxymethyl Ntakusu element spreads weight table syntax, specifies a plurality parameter by the reference number the maximum number as an upper limit , using the reference number Identifying a reference image from the first reference picture list and the second reference picture list, it executes a decoding process including a weighted motion compensation using the reference image and the plurality parameters identified.

The block diagram which shows the example of the encoding apparatus of 1st Embodiment. Explanatory drawing which shows the example of the prediction encoding order of the pixel block in 1st Embodiment. The figure which shows the block size example of the coding tree block in 1st Embodiment. The figure which shows the specific example of the coding tree block of 1st Embodiment. The figure which shows the specific example of the coding tree block of 1st Embodiment. The figure which shows the specific example of the coding tree block of 1st Embodiment. The block diagram which shows the example of the estimated image generation part of 1st Embodiment. The figure which shows the example of the relationship of the motion vector of the motion compensation prediction in the bidirectional | two-way prediction of 1st Embodiment. The block diagram which shows the example of the multi-frame motion compensation part of 1st Embodiment. Explanatory drawing of the example of the fixed point precision of the weighting coefficient in 1st Embodiment. The block diagram which shows the example of the index setting part of 1st Embodiment. The figure which shows the WP parameter information example of 1st Embodiment. The figure which shows the WP parameter information example of 1st Embodiment. The figure which shows the example of the encoding order in the low-delay encoding structure of 1st Embodiment, and a display order. The figure which shows the example of the relationship between the reference image in the low delay coding structure of 1st Embodiment, and a reference number. The figure which shows the example of the encoding order in the random access encoding structure of 1st Embodiment, and a display order. The figure which shows the example of the relationship between the reference image in the random access encoding structure of 1st Embodiment, and a reference number. The figure which shows the example of the scanning order of the list number of the reference image of 1st Embodiment, and a reference number. The figure which simplified and showed the example of WP parameter information after common list conversion in 1st Embodiment. The figure which simplified and showed the example of WP parameter information after common list conversion in 1st Embodiment. 5 is a flowchart illustrating an example of index information generation processing according to the first embodiment. The figure which shows the example of the syntax of 1st Embodiment. The figure which shows the example of the picture parameter set syntax of 1st Embodiment. The figure which shows the example of the slice header syntax of 1st Embodiment. The figure which shows the example of the tread weight table syntax of 1st Embodiment. The figure which shows the example of the sequence parameter set syntax of a modification. The figure which shows the example of the adaptation parameter set syntax of a modification. The figure which shows the example of the tread weight table syntax of a modification. The block diagram which shows the structural example of the index setting part of 2nd Embodiment. The flowchart which shows the production | generation process example of the index information of 2nd Embodiment. The figure which shows the example of the tread weight table syntax of 2nd Embodiment. The block diagram which shows the structural example of the encoding apparatus of 3rd Embodiment. The figure which shows the detailed example of the motion information memory of 3rd Embodiment. The figure which shows the example of the block position which derives | leads-out the motion information candidate with respect to the encoding pixel block of 3rd Embodiment. The figure which shows the example of the block position which derives | leads-out the motion information candidate with respect to the encoding pixel block of 3rd Embodiment. The figure which shows the example of the relationship between the pixel block position of the some motion information candidate of 3rd Embodiment, and a pixel block position index. The flowchart which shows the example of a storage process to MergeCandList of 3rd Embodiment. The figure which shows the example of the storage list | wrist of the motion information of 3rd Embodiment. The flowchart which shows the example of the storage method to the storage list of the motion information of 3rd Embodiment. The figure which shows the example of a combination of the bidirectional | two-way prediction of 3rd Embodiment. The figure which shows the example of the tread unit syntax of 3rd Embodiment. The flowchart which shows the other example of the storing process to MergeCandList of 3rd Embodiment. The block diagram which shows the structural example of the decoding apparatus of 4th Embodiment. The block diagram which shows the structural example of the index setting part of 4th Embodiment. The block diagram which shows the structural example of the index setting part of 5th Embodiment. The block diagram which shows the structural example of the decoding apparatus of 6th Embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In each of the following embodiments, an encoding device and a decoding device include an LSI (Large-Scale Integration) chip, a DSP (Digital
It can be realized by hardware such as a signal processor (FPGA) or a field programmable gate array (FPGA). In addition, the encoding device and the decoding device of each of the following embodiments can be realized by software by causing a computer to execute a program. In the following description, the term “image” can be appropriately replaced with a term such as “video”, “pixel”, “image signal”, “picture”, or “image data”.

(First embodiment)
In the first embodiment, an encoding apparatus that encodes a moving image will be described.

  FIG. 1 is a block diagram illustrating an example of a configuration of the encoding device 100 according to the first embodiment.

  The encoding apparatus 100 divides each frame or each field constituting the input image into a plurality of pixel blocks, and uses the encoding parameters input from the encoding control unit 111 to predict the encoded code for the divided pixel blocks. To generate a predicted image. The encoding apparatus 100 subtracts the input image divided into a plurality of pixel blocks and the prediction image to generate a prediction error, orthogonally transforms and quantizes the generated prediction error, and further performs entropy encoding to perform encoding. Generate and output data.

  The encoding apparatus 100 performs predictive encoding by selectively applying a plurality of prediction modes in which at least one of the block size of the pixel block and the prediction image generation method is different. The generation method of the prediction image is roughly classified into two types: intra prediction that performs prediction within the encoding target frame and inter prediction that performs motion compensation prediction using one or more reference frames that are temporally different. Note that intra prediction is also referred to as intra prediction or intra frame prediction, and inter prediction is also referred to as inter prediction, inter frame prediction, motion compensation prediction, or the like.

  FIG. 2 is an explanatory diagram illustrating an example of a predictive coding order of pixel blocks in the first embodiment. In the example illustrated in FIG. 2, the encoding device 100 performs predictive encoding from the upper left to the lower right of the pixel block, and the left side of the encoding target pixel block c in the encoding target frame f and The encoded pixel block p is located on the upper side. Hereinafter, for simplification of description, the encoding apparatus 100 performs predictive encoding in the order shown in FIG. 2, but the order of predictive encoding is not limited to this.

The pixel block indicates a unit for processing an image. For example, an M × N size block (M and N are natural numbers), a coding tree block, a macro block, a sub block, or 1
This applies to pixels. In the following description, the pixel block is basically used in the meaning of the coding tree block, but may be used in other meanings. For example, in the description of the prediction unit, a pixel block is used to mean a pixel block of the prediction unit. A block may be called by a name such as a unit. For example, a coding block is called a coding unit.

  FIG. 3A is a diagram illustrating an example of a block size of a coding tree block according to the first embodiment. The coding tree block is typically a 64 × 64 pixel block as shown in FIG. 3A. However, the present invention is not limited to this, and it may be a 32 × 32 pixel block, a 16 × 16 pixel block, an 8 × 8 pixel block, a 4 × 4 pixel block, or the like. Further, the coding tree block may not be a square, and may be, for example, an M × N size (M ≠ N) pixel block.

  3B to 3D are diagrams illustrating specific examples of the coding tree block according to the first embodiment. FIG. 3B shows a coding tree block having a block size of 64 × 64 (N = 32). N represents the size of the reference coding tree block. The size when divided is defined as N, and the size when not divided is defined as 2N. FIG. 3C shows a coding tree block obtained by dividing the coding tree block of FIG. 3B into quadtrees. The coding tree block has a quadtree structure as shown in FIG. 3C. When the coding tree block is divided, the four pixel blocks after the division are numbered in the Z-scan order as shown in FIG. 3C.

  The coding tree block can be further divided into quadtrees within one quadtree number. Thereby, a coding tree block can be divided | segmented hierarchically. In this case, the depth of division is defined by Depth. FIG. 3D shows one of the coding tree blocks obtained by dividing the coding tree block of FIG. 3B into quadtrees, and the block size is 32 × 32 (N = 16). The Depth of the coding tree block shown in FIG. 3B is 0, and the Depth of the coding tree block shown in FIG. The coding tree block having the largest unit is called a large coding tree block, and the input image signal is encoded in this unit in the raster scan order.

  In the following description, the encoding target block or coding tree block of the input image may be referred to as a prediction target block or a prediction pixel block. The encoding unit is not limited to a pixel block, and at least one of a frame, a field, a slice, a line, and a pixel can be used.

  As illustrated in FIG. 1, the encoding device 100 includes a subtraction unit 101, an orthogonal transformation unit 102, a quantization unit 103, an inverse quantization unit 104, an inverse orthogonal transformation unit 105, an addition unit 106, and a prediction The image generation unit 107, the index setting unit 108, the motion evaluation unit 109, and the encoding unit 110 are provided. Note that the encoding control unit 111 illustrated in FIG. 1 controls the encoding apparatus 100 and can be realized by, for example, a CPU (Central Processing Unit).

  The subtraction unit 101 subtracts the corresponding prediction image from the input image divided into pixel blocks to obtain a prediction error. The subtraction unit 101 outputs a prediction error and inputs it to the orthogonal transform unit 102.

  The orthogonal transform unit 102 performs orthogonal transform such as discrete cosine transform (DCT) or discrete sine transform (DST) on the prediction error input from the subtraction unit 101 to obtain transform coefficients. The orthogonal transform unit 102 outputs transform coefficients and inputs them to the quantization unit 103.

  The quantization unit 103 performs a quantization process on the transform coefficient input from the orthogonal transform unit 102 to obtain a quantized transform coefficient. Specifically, the quantization unit 103 performs quantization according to quantization information such as a quantization parameter or a quantization matrix specified by the encoding control unit 111. More specifically, the quantization unit 103 divides the transform coefficient by the quantization step size derived from the quantization information to obtain a quantized transform coefficient. The quantization parameter indicates the fineness of quantization. The quantization matrix is used for weighting the fineness of quantization for each component of the transform coefficient. The quantization unit 103 outputs the quantized transform coefficient and inputs it to the inverse quantization unit 104 and the encoding unit 110.

  The inverse quantization unit 104 performs inverse quantization processing on the quantized transform coefficient input from the quantization unit 103 to obtain a restored transform coefficient. Specifically, the inverse quantization unit 104 performs inverse quantization according to the quantization information used in the quantization unit 103. More specifically, the inverse quantization unit 104 multiplies the quantization transform coefficient by the quantization step size derived from the quantization information to obtain a restored transform coefficient. Note that the quantization information used in the quantization unit 103 is loaded from an internal memory (not shown) of the encoding control unit 111 and used. The inverse quantization unit 104 outputs the restored transform coefficient and inputs it to the inverse orthogonal transform unit 105.

  The inverse orthogonal transform unit 105 performs inverse orthogonal transform such as inverse discrete cosine transform (IDCT) or inverse discrete sine transform (IDST) on the restored transform coefficient input from the inverse quantization unit 104, Get the restoration prediction error. Note that the inverse orthogonal transform performed by the inverse orthogonal transform unit 105 corresponds to the orthogonal transform performed by the orthogonal transform unit 102. The inverse orthogonal transform unit 105 outputs the reconstruction prediction error and inputs it to the addition unit 106.

  The adding unit 106 adds the restored prediction error input from the inverse orthogonal transform unit 105 and the corresponding predicted image to generate a locally decoded image. The adding unit 106 outputs the local decoded image and inputs it to the predicted image generation unit 107.

  The predicted image generation unit 107 stores the locally decoded image input from the addition unit 106 as a reference image in a memory (not shown in FIG. 1), outputs the reference image stored in the memory, and inputs the reference image to the motion evaluation unit 109. . Further, the predicted image generation unit 107 performs weighted motion compensation prediction based on the motion information and WP parameter information input from the motion evaluation unit 109, and generates a predicted image. The predicted image generation unit 107 outputs a predicted image and inputs it to the subtracting unit 101 and the adding unit 106.

  FIG. 4 is a block diagram illustrating an example of the configuration of the predicted image generation unit 107 of the first embodiment. As shown in FIG. 4, the predicted image generation unit 107 includes a multi-frame motion compensation unit 201, a memory 202, a unidirectional motion compensation unit 203, a prediction parameter control unit 204, a reference image selector 205, and a frame memory 206. And a reference image control unit 207.

  The frame memory 206 stores the locally decoded image input from the addition unit 106 as a reference image under the control of the reference image control unit 207. The frame memory 206 has a plurality of memory sets FM1 to FMN (N ≧ 2) for temporarily storing reference images.

The prediction parameter control unit 204 prepares a plurality of combinations of reference image numbers and prediction parameters as a table based on the motion information input from the motion evaluation unit 109. Here, the motion information indicates a motion vector indicating a shift amount of motion used in motion compensation prediction, a reference image number, information on a prediction mode such as unidirectional / bidirectional prediction, and the like. The prediction parameter refers to information regarding a motion vector and a prediction mode. Then, the prediction parameter control unit 204 selects and outputs a combination of the reference image number and the prediction parameter used for generating the prediction image based on the input image, inputs the reference image number to the reference image selector 205, and outputs the prediction parameter. Is input to the unidirectional motion compensation unit 203.

  The reference image selector 205 is a switch for switching which output terminal of the frame memories FM1 to FMN included in the frame memory 206 is connected according to the reference image number input from the prediction parameter control unit 204. For example, if the reference image number is 0, the reference image selector 205 connects the output end of FM1 to the output end of the reference image selector 205. If the reference image number is N-1, the reference image selector 205 refers to the output end of the FMN. Connected to the output terminal of the image selector 205. The reference image selector 205 outputs the reference image stored in the frame memory to which the output terminal is connected among the frame memories FM1 to FMN included in the frame memory 206, and the unidirectional motion compensation unit 203 and the motion evaluation unit 109. Enter.

  The unidirectional prediction motion compensation unit 203 performs a motion compensation prediction process according to the prediction parameter input from the prediction parameter control unit 204 and the reference image input from the reference image selector 205, and generates a unidirectional prediction image.

  FIG. 5 is a diagram illustrating an example of a motion vector relationship of motion compensation prediction in bidirectional prediction according to the first embodiment. In motion compensated prediction, interpolation processing is performed using a reference image, and a unidirectional prediction image is generated based on the amount of motion deviation from the pixel block at the encoding target position between the generated interpolation image and the input image. . Here, the amount of deviation is a motion vector. As shown in FIG. 5, in a bi-directional prediction slice (B-slice), a prediction image is generated using a set of two types of reference images and motion vectors. As the interpolation process, an interpolation process with 1/2 pixel accuracy, an interpolation process with 1/4 pixel accuracy, or the like is used, and the value of the interpolation image is generated by performing the filtering process on the reference image. For example, H.P. capable of performing interpolation processing up to 1/4 pixel accuracy on a luminance signal. In H.264, the shift amount is expressed by four times the integer pixel accuracy.

  The unidirectional prediction motion compensation unit 203 outputs a unidirectional prediction image and temporarily stores it in the memory 202. Here, when the motion information (prediction parameter) indicates bidirectional prediction, the multi-frame motion compensation unit 201 performs weighted prediction using two types of unidirectional prediction images, and thus the unidirectional prediction motion compensation unit 203. Stores the unidirectional predicted image corresponding to the first in the memory 202, and directly outputs the unidirectional predicted image corresponding to the second to the multi-frame motion compensation unit 201. Here, the first unidirectional prediction image corresponding to the first is the first prediction image, and the second unidirectional prediction image is the second prediction image.

  Two unidirectional motion compensation units 203 may be prepared, and each may generate two unidirectional prediction images. In this case, when the motion information (prediction parameter) indicates unidirectional prediction, the unidirectional motion compensation unit 203 directly outputs the first unidirectional prediction image as the first prediction image to the multi-frame motion compensation unit 201. Good.

  The multi-frame motion compensation unit 201 uses the first prediction image input from the memory 202, the second prediction image input from the unidirectional prediction motion compensation unit 203, and the WP parameter information input from the motion evaluation unit 109. Then, a prediction image is generated by performing weighted prediction. The multi-frame motion compensation unit 201 outputs a prediction image and inputs the prediction image to the subtraction unit 101 and the addition unit 106.

  FIG. 6 is a block diagram illustrating an example of the configuration of the multi-frame motion compensation unit 201 according to the first embodiment. As shown in FIG. 6, the multi-frame motion compensation unit 201 includes a default motion compensation unit 301, a weighted motion compensation unit 302, a WP parameter control unit 303, and WP selectors 304 and 305.

The WP parameter control unit 303 outputs a WP application flag and weight information based on the WP parameter information input from the motion evaluation unit 109, and sets the WP application flag to the WP selector 30.
4 and 305, and weight information is input to the weighted motion compensation unit 302.

  Here, the WP parameter information includes the fixed-point precision of the weighting factor, the first WP application flag corresponding to the first predicted image, the first weighting factor, the first offset, and the second WP adaptation corresponding to the second predicted image. Contains information on flag, second weighting factor, and second offset. The WP application flag is a parameter that can be set for each corresponding reference image and signal component, and indicates whether to perform weighted motion compensation prediction. The weight information includes information on the fixed point precision of the weight coefficient, the first weight coefficient, the first offset, the second weight coefficient, and the second offset.

  Specifically, when WP parameter information is input from the motion evaluation unit 109, the WP parameter control unit 303 separates the WP parameter information into a first WP application flag, a second WP application flag, and weight information, and outputs them. The first WP application flag is input to the WP selector 304, the second WP application flag is input to the WP selector 305, and the weight information is input to the weighted motion compensation unit 302.

  The WP selectors 304 and 305 switch the connection end of each predicted image based on the WP application flag input from the WP parameter control unit 303. When each WP application flag is 0, the WP selectors 304 and 305 connect each output terminal to the default motion compensation unit 301. Then, the WP selectors 304 and 305 output the first predicted image and the second predicted image and input them to the default motion compensation unit 301. On the other hand, when each WP application flag is 1, the WP selectors 304 and 305 connect each output terminal to the weighted motion compensation unit 302. The WP selectors 304 and 305 output the first predicted image and the second predicted image, and input them to the weighted motion compensation unit 302.

  The default motion compensation unit 301 performs an average value process based on the two unidirectional predicted images (first predicted image and second predicted image) input from the WP selectors 304 and 305 to generate a predicted image. Specifically, when the first WP application flag and the second WP application flag are 0, the default motion compensation unit 301 performs average value processing based on Expression (1).

  P [x, y] = Clip1 ((PL0 [x, y] + PL1 [x, y] + offset2) >> (shift2)) (1)

  Here, P [x, y] is a predicted image, PL0 [x, y] is a first predicted image, and PL1 [x, y] is a second predicted image. offset2 and shift2 are parameters of the rounding process in the average value process, and are determined by the internal calculation accuracy of the first predicted image and the second predicted image. If the bit accuracy of the prediction image is L and the bit accuracy of the first prediction image and the second prediction image is M (L ≦ M), shift2 is formulated by Equation (2), and offset2 is formulated by Equation (3). Is done.

  shift2 = (ML + 1) (2)

  offset2 = (1 << (shift2-1) (3)

  For example, when the bit accuracy of the predicted image is 8 and the bit accuracy of the first predicted image and the second predicted image is 14, shift2 = 7 from Equation (2), and offset2 = (1 << from Equation (3). 6).

  When the prediction mode indicated by the motion information (prediction parameter) is unidirectional prediction, the default motion compensation unit 301 uses only the first prediction image and calculates a final prediction image based on Expression (4). calculate.

  P [x, y] = Clip1 ((PLX [x, y] + offset1) >> (shift1)) (4)

  Here, PLX [x, y] indicates a unidirectional prediction image (first prediction image), and X is an identifier indicating either 0 or 1 in the reference list. For example, when the reference list is 0, PL0 [x, y] is obtained, and when the reference list is 1, PL1 [x, y] is obtained. offset1 and shift1 are parameters of the rounding process, and are determined by the internal calculation accuracy of the first predicted image. When the bit accuracy of the predicted image is L and the bit accuracy of the first predicted image is M, shift1 is formulated by Equation (5), and offset1 is formulated by Equation (6).

  shift1 = (ML) (5)

  offset1 = (1 << (shift1-1) (6)

  For example, when the bit accuracy of the predicted image is 8 and the bit accuracy of the first predicted image is 14, shift1 = 6 from Equation (5) and offset1 = (1 << 5) from Equation (6).

  The weighted motion compensation unit 302 is based on the two unidirectional prediction images (first prediction image and second prediction image) input from the WP selectors 304 and 305 and the weight information input from the WP parameter control unit 303. Performs weighted motion compensation. Specifically, when the first WP application flag and the second WP application flag are 1, the weighted motion compensation unit 302 performs the weighting process based on Expression (7).

  P [x, y] = Clip1 (((PL0 [x, y] * w0C + PL1 [x, y] * w1C + (1 << logWDC)) >> (logWDC + 1)) + ((o0C + o1C + 1) >> 1)) (7)

  Here, w0C represents a weighting factor corresponding to the first predicted image, w1C represents a weighting factor corresponding to the second predicted image, o0C represents an offset corresponding to the first predicted image, and o1C represents an offset corresponding to the second predicted image. . Hereinafter, these are referred to as a first weighting factor, a second weighting factor, a first offset, and a second offset, respectively. logWDC is a parameter indicating the fixed-point precision of each weighting factor. The variable C means a signal component. For example, in the case of a YUV spatial signal, the luminance signal is C = Y, the Cr color difference signal is C = Cr, and the Cb color difference component is C = Cb.

  Note that the weighted motion compensation unit 302 performs rounding processing by controlling logWDC, which is fixed-point precision, as in Expression (8) when the calculation accuracy of the first predicted image, the second predicted image, and the predicted image is different. Is realized.

  logWD'C = logWDC + offset1 (8)

  The rounding process can be realized by replacing logWDC in Expression (7) with logWD′C in Expression (8). For example, when the bit accuracy of the predicted image is 8 and the bit accuracy of the first predicted image and the second predicted image is 14, by resetting logWDC, the calculation accuracy similar to shift2 in Equation (1) can be obtained. A batch rounding process can be realized.

  When the prediction mode indicated by the motion information (prediction parameter) is unidirectional prediction, the weighted motion compensation unit 302 uses only the first predicted image and uses the final predicted image based on Equation (9). Is calculated.

  P [x, y] = Clip1 ((PLX [x, y] * wXC + (1 << logWDC-1)) >> (logWDC)) (9)

  Here, PLX [x, y] indicates a unidirectional prediction image (first prediction image), wXC indicates a weighting factor corresponding to unidirectional prediction, and X indicates either 0 or 1 in the reference list. It is an identifier to indicate. For example, when the reference list is 0, PL0 [x, y] and w0C are obtained, and when the reference list is 1, PL1 [x, y] and w1C are obtained.

  Note that the weighted motion compensation unit 302, when the calculation accuracy of the first prediction image, the second prediction image, and the prediction image is different, uses the log WDC, which is a fixed-point accuracy, as in Expression (8) as in the bidirectional prediction. Rounding processing is realized by controlling to.

  The rounding process can be realized by replacing logWDC in Expression (7) with logWD′C in Expression (8). For example, when the bit accuracy of the predicted image is 8 and the bit accuracy of the first predicted image is 14, the round rounding process with the same calculation accuracy as that of shift1 of Expression (4) is realized by resetting logWDC It becomes possible to do.

  FIG. 7 is an explanatory diagram of an example of the fixed-point precision of the weighting coefficient in the first embodiment, and is a diagram illustrating an example of a change between a moving image having a brightness change in the time direction and a gradation value. In the example shown in FIG. 7, the encoding target frame is Frame (t), the previous frame in time is Frame (t−1), and the next frame in time is Frame (t + 1). As shown in FIG. 7, in a fade image that changes from white to black, the brightness (gradation value) of the image decreases with time. The weighting coefficient means the degree of change in FIG. 7 and takes a value of 1.0 when there is no change in brightness, as is clear from Equation (7) and Equation (9). The fixed-point precision is a parameter for controlling the step size corresponding to the decimal point of the weighting coefficient, and the weighting coefficient when there is no change in brightness is 1 << logWDC.

  In the case of unidirectional prediction, various parameters (second WP adaptive flag, second weighting factor, and second offset information) corresponding to the second predicted image are not used, and are set to predetermined initial values. May be.

  Returning to FIG. 1, the motion evaluation unit 109 performs motion evaluation between a plurality of frames based on the input image and the reference image input from the predicted image generation unit 107, outputs motion information and WP parameter information, and stores the motion information. The prediction image generation unit 107 and the encoding unit 110 are input, and the WP parameter information is input to the prediction image generation unit 107 and the index setting unit 108.

  For example, the motion evaluation unit 109 calculates an error by calculating a difference value starting from a plurality of reference images corresponding to the same position as the input image of the prediction target pixel block, shifts the position with a fractional accuracy, and minimizes the error. Optimal motion information is calculated by a technique such as block matching for searching for a block. In the case of bidirectional prediction, the motion evaluation unit 109 performs block matching including default motion compensation prediction as shown in Equation (1) and Equation (4) using motion information derived by unidirectional prediction. Thus, motion information for bidirectional prediction is calculated.

  At this time, the motion evaluation unit 109 can calculate WP parameter information by performing block matching including weighted motion compensated prediction as represented by Equation (7) and Equation (9). Note that the WP parameter information may be calculated using a method of calculating a weighting factor or an offset using the brightness gradient of the input image, or a method of calculating a weighting factor or an offset by accumulating prediction errors when encoded. Good. The WP parameter information may be a fixed value determined in advance for each encoding device.

  Here, a method for calculating a weighting factor, a fixed point precision of the weighting factor, and an offset from a moving image having a temporal change in brightness will be described with reference to FIG. As described above, in a fade image that changes from white to black as shown in FIG. 7, the brightness (tone value) of the image decreases with time. The motion evaluation unit 109 can calculate a weighting coefficient by calculating this inclination.

  The fixed-point precision of the weighting coefficient is information indicating the precision of the slope, and the motion evaluation unit 109 can calculate an optimum value from the temporal distance of the reference image and the degree of change in image brightness. For example, in FIG. 7, when the weighting coefficient between Frame (t-1) and Frame (t + 1) is 0.75 in decimal point precision, 3/4 can be expressed with 1/4 precision. The unit 109 sets the fixed point precision to 2 (1 << 2). Since the value of the fixed-point precision affects the code amount when the weighting coefficient is encoded, an optimal value may be selected in consideration of the code amount and the prediction accuracy. Note that the fixed-point precision value may be a predetermined fixed value.

  Further, when the inclinations do not match, the motion evaluation unit 109 can calculate the offset value by obtaining a correction value (deviation amount) corresponding to the intercept of the linear function. For example, in FIG. 7, when the weighting coefficient between Frame (t−1) and Frame (t + 1) is 0.60 in decimal point precision and the fixed point precision is 1 (1 << 1), the weighting coefficient is 1 (That is, it corresponds to the decimal point precision 0.50 of the weighting coefficient) is set. In this case, since the decimal point accuracy of the weighting coefficient is deviated from 0.60, which is the optimum value, by 0.10, the motion evaluation unit 109 calculates the correction value for this amount from the maximum value of the pixel and uses it as an offset value. Set. When the maximum value of the pixel is 255, the motion evaluation unit 109 may set a value such as 25 (255 × 0.1).

  In the first embodiment, the motion evaluation unit 109 is illustrated as a function of the encoding device 100. However, the motion evaluation unit 109 is not an essential component of the encoding device 100. For example, the motion evaluation unit 109 is encoded. It is good also as an apparatus outside the converter 100. In this case, the motion information and WP parameter information calculated by the motion evaluation unit 109 may be loaded into the encoding device 100.

  The index setting unit 108 receives the WP parameter information input from the motion evaluation unit 109, confirms the reference list (list number) and the reference image (reference number), outputs the index information, and sends it to the index setting unit 108. input.

  FIG. 8 is a block diagram illustrating an example of the configuration of the index setting unit 108 of the first embodiment. As illustrated in FIG. 8, the index setting unit 108 includes a reference image confirmation unit 401 and an index generation unit 402.

  The reference image confirmation unit 401 receives the WP parameter information input from the motion evaluation unit 109 and confirms whether there is a WP parameter in which the reference numbers included in the two reference lists indicate the same reference image. The reference image confirmation unit 401 deletes the WP parameter indicating the same reference image from the WP parameter information, outputs the WP parameter information after the deletion, and inputs the WP parameter information to the index generation unit 402.

  The index generation unit 402 receives the WP parameter information from which the redundant WP parameter has been deleted from the reference image confirmation unit 401, performs mapping on syntax elements described later, and generates index information. The index generation unit 402 outputs index information and inputs it to the encoding unit 110.

9A and 9B are diagrams illustrating examples of WP parameter information according to the first embodiment. P-
An example of the WP parameter information at the time of slice is as shown in FIG. 9A, and an example of the WP parameter information at the time of B-slice is as shown in FIGS. 9A and 9B. The list number is an identifier indicating the prediction direction, and takes a value of 0 when unidirectional prediction is used, and two values of 0 and 1 can be used when bidirectional prediction is used. The reference numbers are values corresponding to 1 to N indicated in the frame memory 206. Since the WP parameter information is held for each reference list and reference image, the information necessary for the B-slice is 2N when the number of reference images is N. The reference image confirmation unit 401 reconverts these WP parameter information and deletes redundant WP parameters.

  FIG. 10 is a diagram illustrating an example of a coding order and a display order (POC: Pcorder Order Count) in the low-delay coding structure according to the first embodiment. A B-slice that can be a reference image is represented by rB- The example of the encoding structure used as slice is shown. In the low-delay coding structure, since the image cannot be prefetched, the coding order and the display order are the same. Here, it is assumed that frames 0 to 3 are already encoded in the display order, and frame 4 is encoded.

  FIG. 11 is a diagram illustrating an example of a relationship between a reference image and a reference number in the low delay coding structure according to the first embodiment, and simplifies the relationship between the reference image and the reference number when the frame 4 is encoded. WP parameter information. In the example shown in FIG. 11, the order of reference numbers in the reference list is the same in list 0 and list 1, which means that the reference images in list 0 and list 1 are the same.

  FIG. 12 is a diagram illustrating an example of the encoding order and the display order in the random access encoding structure of the first embodiment. Here, frames 0, 2, 4, and 8 have been encoded in the display order. Consider encoding frame 1.

  FIG. 13 is a diagram illustrating an example of the relationship between the reference image and the reference number in the random access coding structure according to the first embodiment, and simplifies the relationship between the reference image and the reference number when encoding frame 1. WP parameter information. In the example illustrated in FIG. 13, the order of reference numbers for each reference list is different, but list 0 and list 1 include four common reference images. This means that the reference images are the same.

  The reference image confirmation unit 401 deletes redundant WP parameters, that is, WP parameters indicating the same reference images included in the two reference lists, so that the reference numbers are rearranged and a common new index Convert to

  FIG. 14 is a diagram illustrating an example of a reference image list number and a reference number scan order according to the first embodiment. The reference image confirmation unit 401 converts a two-dimensional reference list (two reference lists) into a one-dimensional common list (one common list) in the scan order shown in FIG. Specifically, the reference image confirmation unit 401 reads the WP parameter information shown in FIGS. 9A and 9B in the scan order shown in FIG. 14, rearranges the two reference lists into a common list, and redundant WP parameters. Is deleted. Here, the reference image confirmation unit 401 reads WP parameter information in accordance with the pseudo code shown in Equation (10).

for (scan = 0; scan <= num_of_common_active_ref_minus1; scan ++) {
list = common_scan_list (scan)
ref_idx = common_scan_ref_idx (scan)
refPOC = RefPicOrderCount (list, ref_idx)
identical_ref_flag = false;
for (curridx = 0; curridx <= num_of_common_active_ref_minus1; curridx ++) {
if (refPOC == CommonRefPicOrderCount (curridx)) {
identical_ref_flag = true;
}
}
if (! identical_ref_flag)
InsertCommonRefPicOrderCount (scan, refPOC);
} (10)

  Here, common_scan_list () is a function that returns a list number corresponding to the scan number of FIG. common_scan_ref_idx () is a function that returns a reference number corresponding to the scan number of FIG. RefPicOrderCnt () is a function that returns a POC number corresponding to a list number and a reference number. The POC number is a number indicating the display order of reference images. However, when the POC number exceeds a predetermined maximum number, the POC number is mapped to an initial value. For example, when the maximum POC number is 255, the POC number corresponding to 256 is 0. CommonRefPicOrderCount () is a function that returns a POC number corresponding to the reference number of the common list (CommonList). InsertCommonRefPicOrderCount () is a function for inserting a POC number corresponding to a scan number into a common list (CommonList).

  Note that the scan order shown in FIG. 14 is an example, and any other scan order may be used as long as it is a predetermined scan order. Further, the pseudo code shown in Equation (10) is an example, and if the purpose of this processing can be realized, it is possible to add processing and reduce redundant processing.

  FIGS. 15 and 16 are diagrams showing a simplified example of WP parameter information after common list conversion in the first embodiment. 15 shows simplified WP parameter information of the common list obtained by converting the WP parameter information of the reference list shown in FIG. 11, and FIG. 16 shows a common list obtained by converting the WP parameter information of the reference list shown in FIG. The WP parameter information is simply shown.

  FIG. 17 is a flowchart illustrating an example of index information generation processing executed by the index setting unit 108 of the first embodiment.

  When the WP parameter information is input, the reference image confirmation unit 401 branches the process according to the slice type (step S101).

  When the slice type is a unidirectional prediction slice (P-slice) using only one reference list (No in step S101), the reference image confirmation unit 401 directly uses the WP parameter information illustrated in FIG. 9A as an index generation unit. Output to 402. The index generation unit 402 generates index information by mapping the WP parameter information input from the reference image confirmation unit 401 to a predetermined syntax element to be described later, and outputs the index information.

  On the other hand, when the slice type is a bi-predictive slice (B-slice) using two reference lists (Yes in step S101), the reference image confirmation unit 401 initializes the variable scan to 0 (step S102). ). The variable scan indicates the scan number in FIG.

  Subsequently, the reference image confirmation unit 401 acquires a variable list indicating a list number corresponding to the scan number by using the variable scan (step S103), and acquires a variable refIdx indicating the reference number corresponding to the scan number. (Step S104).

Subsequently, the reference image confirmation unit 401 derives a variable refPOC indicating the POC number of the reference image corresponding to the list number indicated by the variable list and the reference number indicated by the variable refIdx (step S105).

  Subsequently, the reference image confirmation unit 401 sets the flag “identical_ref_flag” to “false” (step S106), and sets the variable “curridx” to “0” (step S107). The flag “identical_ref_flag” indicates whether or not the same reference image exists in the common list. The variable curridx indicates the reference number of the common list.

  Subsequently, the reference image confirmation unit 401 determines whether the POC number indicated by the derived variable refPOC is the same as the POC number of the reference image corresponding to the reference number indicated by the variable curridx (step S108).

  When both POC numbers are the same (Yes in Step S108), the reference image confirmation unit 401 sets the flag “identical_ref_flag” to true (Step S109). On the other hand, when both POC numbers are not the same (No in step S108), the process of step S109 is not performed.

  Subsequently, the reference image confirmation unit 401 increments the variable curridx (step S110).

  Subsequently, the reference image confirmation unit 401 determines whether or not the value of the variable curridx is larger than num_of_common_active_ref_minus1 that is a value obtained by subtracting 1 from the maximum number of common lists (step S111), and during num_of_common_active_ref_minus1 or less (No in step S111) ), The process of step S108 to step S110 is repeated.

  When it becomes larger than num_of_common_active_ref_minus1 (Yes in step S111), the reference image confirmation unit 401 completes confirmation of the common list, and further confirms whether the flag identification_ref_flag is false (step S112).

  When the flag “identical_ref_flag” is false (Yes in step S112), the reference image confirmation unit 401 determines that the common list does not include the same reference image as the reference image having the POC number indicated by the variable refPOC, and the POC indicated by the variable refPOC. The number is added to the common list (step S113). On the other hand, if the flag identifier_ref_flag is true (No in step S112), the reference image confirmation unit 401 determines that the same reference image as the reference image of the POC number indicated by the variable refPOC is included in the common list, and the process of step S113 Do not do.

  Subsequently, the reference image confirmation unit 401 increments the variable scan (step S114).

  Subsequently, the reference image confirmation unit 401 determines whether or not the value of the variable scan is larger than num_of_common_active_ref_minus1 that is a value obtained by subtracting 1 from the maximum number of common lists (step S115), and if it is less than or equal to num_of_common_active_ref_minus1 (in step S115) No), it returns to step S103. On the other hand, if it is larger than num_of_common_active_ref_minus1 (Yes in step S115), the reference image confirmation unit 401 outputs the WP parameter information converted into the common list to the index generation unit 402. The index generation unit 402 generates index information by mapping the WP parameter information input from the reference image confirmation unit 401 to a predetermined syntax element to be described later, and outputs the index information.

Returning to FIG. 1, the encoding unit 110 receives the quantized transform coefficient input from the quantization unit 103, the motion information input from the motion evaluation unit 109, the index information input from the index setting unit 108, and the encoding control. Entropy encoding is performed on various encoding parameters such as quantization information specified by the unit 111 to generate encoded data. Entropy coding corresponds to, for example, Huffman coding or arithmetic coding.

  The encoding parameter is a parameter required for decoding, such as prediction information indicating a prediction method, information on quantization transform coefficients, information on quantization, and the like. For example, the encoding control unit 111 has an internal memory (not shown), the encoding parameters are held in the internal memory, and the encoding parameters of the adjacent already encoded pixel block are used when encoding the pixel block. You can For example, H.M. In the H.264 intra prediction, the prediction information of the pixel block can be derived from the prediction information of the encoded adjacent block.

  The encoding unit 110 outputs the generated encoded data according to an appropriate output timing managed by the encoding control unit 111. The output encoded data is, for example, multiplexed with various information such as a multiplexing unit (not shown) and temporarily stored in an output buffer (not shown). (Storage medium) or transmission system (communication line).

  FIG. 18 is a diagram illustrating an example of the syntax 500 used by the encoding device 100 according to the first embodiment. A syntax 500 indicates a structure of encoded data generated by the encoding apparatus 100 by encoding an input image (moving image data). When decoding the encoded data, a decoding device described later refers to the same syntax structure as that of the syntax 500 and interprets the syntax of the moving image.

  The syntax 500 includes three parts: a high level syntax 501, a slice level syntax 502, and a coding tree level syntax 503. The high level syntax 501 includes syntax information of a layer higher than the slice. A slice refers to a rectangular area or a continuous area included in a frame or a field. The slice level syntax 502 includes information necessary for decoding each slice. The coding tree level syntax 503 includes information necessary for decoding each coding tree (ie, each coding tree block). Each of these parts includes more detailed syntax.

  The high level syntax 501 includes sequence and picture level syntaxes such as a sequence parameter set syntax 504, a picture parameter set syntax 505, and an adaptation parameter set syntax 506.

  The slice level syntax 502 includes a slice header syntax 507, a breadweight table syntax 508, a slice data syntax 509, and the like. The tread weight table syntax 508 is called from the slice header syntax 507.

The coding tree level syntax 503 includes a coding tree unit syntax 510, a transform unit syntax 511, a prediction unit syntax 512, and the like. The coding tree unit syntax 510 may have a quadtree structure. Specifically, the coding tree unit syntax 510 can be recursively called as a syntax element of the coding tree unit syntax 510. That is, one coding tree block can be subdivided with a quadtree. The coding tree unit syntax 510 includes a transform unit syntax 511. The transform unit syntax 511 is called in each coding tree unit syntax 510 at the extreme end of the quadtree. The transform unit syntax 511 describes information related to inverse orthogonal transformation and quantization. In these syntaxes, information related to weighted motion compensation prediction may be described.

  FIG. 19 is a diagram illustrating an example of the picture parameter set syntax 505 according to the first embodiment. The weighted_pred_flag is a syntax element indicating, for example, whether the weighted compensated prediction of the first embodiment related to P-slice is valid or invalid. When weighted_pred_flag is 0, the weighted motion compensation prediction of the first embodiment in the P-slice is invalid. Accordingly, the WP application flag included in the WP parameter information is always set to 0, and the WP selectors 304 and 305 connect the respective output terminals to the default motion compensation unit 301. On the other hand, when weighted_pred_flag is 1, the weighted motion compensation prediction of the first embodiment in the P-slice is valid.

  As another example, when weighted_pred_flag is 1, in the syntax of lower layers (slice header, coding tree block, transform unit, prediction unit, etc.) The validity or invalidity of the weighted motion compensated prediction of one embodiment may be specified.

  For example, weighted_bipred_idc is a syntax element indicating whether the weighted compensated prediction of the first embodiment related to B-slice is valid or invalid. When weighted_bipred_idc is 0, the weighted motion compensated prediction of the first embodiment in the B-slice is invalid. Accordingly, the WP application flag included in the WP parameter information is always set to 0, and the WP selectors 304 and 305 connect the respective output terminals to the default motion compensation unit 301. On the other hand, when weighted_bipred_idc is 1, the weighted motion compensated prediction of the first embodiment in the B-slice is valid.

  As another example, when weighted_bipred_idc is 1, in the syntax of lower layers (slice header, coding tree block, transform unit, etc.), the syntax of the first embodiment is determined for each local region inside the slice. The validity or invalidity of the weighted motion compensation prediction may be defined.

  FIG. 20 is a diagram illustrating an example of the slice header syntax 507 according to the first embodiment. slice_type indicates the slice type (I-slice, P-slice, B-slice, etc.) of the slice. pic_parameter_set_id is an identifier indicating which picture parameter set syntax 505 is to be referred to. num_ref_idx_active_override_flag is a flag indicating whether or not the number of valid reference images is updated. When this flag is 1, num_ref_idx_l0_active_minus1 and num_ref_idx_l1_active_minus1 that define the number of reference images in the reference list can be used. pred_weight_table () is a function indicating the predicate weight table syntax used for weighted motion compensation prediction. When the above weighted_pred_flag is 1 and P-slice, and when weighted_bipred_idc is 1 and B-slice, this function is Called.

  FIG. 21 is a diagram illustrating an example of the tread weight table syntax 508 according to the first embodiment. luma_log2_weight_denom represents the fixed-point precision of the weighting factor of the luminance signal in the slice, and is a value corresponding to the log WDC in Equation (7) or Equation (9). chroma_log2_weight_denom represents the fixed-point precision of the weight coefficient of the color difference signal in the slice, and is a value corresponding to the log WDC in Equation (7) or Equation (9). chroma_format_idc is an identifier representing a color space, and MONO_IDX is a value indicating a monochrome video. num_ref_common_active_minus1 indicates a value obtained by subtracting 1 from the number of reference images included in the common list in the slice, and is used in Expression (10). The maximum value of this value is a value obtained by adding the maximum values of the two reference image numbers of list 0 and list 1.

  luma_weight_common_flag indicates a WP adaptation flag in the luminance signal. When this flag is 1, the weighted motion compensated prediction of the luminance signal of the first embodiment is effective over the entire slice. chroma_weight_common_flag indicates a WP adaptation flag in the color difference signal. When this flag is 1, the weighted motion compensation prediction of the chrominance signal of the first embodiment is effective over the entire slice. luma_weight_common [i] is a weighting factor of the luminance signal corresponding to the i-th managed in the common list. luma_offset_common [i] is an offset of the luminance signal corresponding to the i-th managed in the common list. These are values corresponding to w0C, w1C, o0C, and o1C, respectively, of Equation (7) or Equation (9). However, C = Y.

  chroma_weight_common [i] [j] is a weight coefficient of the color difference signal corresponding to the i-th managed in the common list. chroma_offset_common [i] [j] is an offset of the color difference signal corresponding to the i-th managed in the common list. These are values corresponding to w0C, w1C, o0C, and o1C, respectively, of Equation (7) or Equation (9). However, C = Cr or Cb. j indicates a component of a color difference component. For example, in the case of a YUV 4: 2: 0 signal, j = 0 indicates a Cr component and j = 1 indicates a Cb component.

  As described above, in the first embodiment, the index setting unit 108 includes two reference lists when there is a combination including the same reference image in the two reference lists of the list 0 and the list 1 in the WP parameter information. By converting to a common list, redundant redundant WP parameters are deleted from the WP parameter information to generate index information. Therefore, according to the first embodiment, the code amount of the index information can be reduced.

  In particular, since the weighting factor and the offset need to be encoded for each reference image, the amount of information signaled to the decoder increases as the number of reference images increases. However, when the number of reference images increases, the same information is included in the reference list. Since the number of cases in which reference images are referred to increases, a large code amount reduction effect can be expected by managing the reference images.

(Modification of the first embodiment)
A modification of the first embodiment will be described. In the modification of the first embodiment, the syntax elements used by the index generation unit 402 are different from the syntax elements of the first embodiment.

  FIG. 22 is a diagram illustrating an example of a sequence parameter set syntax 504 according to a modification of the first embodiment. profile_idc is an identifier indicating information on a profile of encoded data. level_idc is an identifier indicating information on the level of encoded data. seq_parameter_set_id is an identifier indicating which sequence parameter set syntax 504 is to be referred to. num_ref_frames is a variable indicating the maximum number of reference images in a frame. The weighted_prediction_enabled_flag is a syntax element indicating, for example, whether the weighted motion compensated prediction of the modified example is valid or invalid with respect to the encoded data.

  When weighted_prediction_enabled_flag is 0, the weighted motion compensated prediction of the modified example in the encoded data is invalid. Accordingly, the WP application flag included in the WP parameter information is always set to 0, and the WP selectors 304 and 305 connect the respective output terminals to the default motion compensation unit 301. On the other hand, when weighted_prediction_enabled_flag is 1, the weighted motion compensation prediction of the modified example is valid over the entire encoded data.

As another example, when weighted_prediction_enabled_flag is 1, in the syntax of lower layers (picture parameter set, adaptation parameter set, slice header, coding tree block, transform unit, prediction unit, etc.) The validity or invalidity of the weighted motion compensated prediction of the modified example may be defined for each local region inside the slice.

  FIG. 23 is a diagram illustrating an example of the adaptation parameter set syntax 506 according to a modification of the first embodiment. The adaptation parameter set syntax 506 holds parameters that affect the entire encoded frame, and is encoded independently as an upper unit. For example, H.M. In H.264, syntax elements corresponding to the high-level syntax 501 are encoded as NAL units. For this reason, the adaptation parameter set syntax 506 is different in syntax and encoding unit from the lower level syntax than the slice level syntax 502 that holds parameters of lower information represented by slices and pixel blocks.

  aps_id is an identifier indicating which adaptation parameter set syntax 506 is to be referred to. By referencing this identifier, the lower slice can refer to any encoded aps_id. In this case, for example, by adding a similar aps_id to the slice header syntax 507 in FIG. 20, it is possible to read parameters that affect the entire frame.

  aps_weighted_prediction_flag is a syntax element indicating whether the weighted motion compensated prediction of the modified example related to the P-slice in the frame is valid or invalid. When aps_weighted_prediction_flag is 0, the weighted motion compensation prediction of the modified example with P-slice in the frame is invalid. Accordingly, the WP application flag included in the WP parameter information is always set to 0, and the WP selectors 304 and 305 connect the respective output terminals to the default motion compensation unit 301. On the other hand, when aps_weighted_prediction_flag is 1, the weighted motion compensated prediction of the modified example is valid throughout the frame.

  As another example, when aps_weighted_prediction_flag is 1, the syntax of lower layers (slice header, coding tree block, transform unit, prediction unit, etc.) is modified for each local region in the slice. The validity or invalidity of the weighted motion compensation prediction of the example prediction may be specified.

  aps_weighted_bipred_idx is a syntax element indicating whether the weighted motion compensation prediction of the modified example related to the B-slice in the frame is valid or invalid. When aps_weighted_bipred_idx is 0, the weighted motion compensation prediction of the modified example with P-slice in the frame is invalid. Accordingly, the WP application flag included in the WP parameter information is always set to 0, and the WP selectors 304 and 305 connect the respective output terminals to the default motion compensation unit 301. On the other hand, when aps_weighted_bipred_idx is 1, the weighted motion compensated prediction of the modified example is valid throughout the frame.

  As another example, when aps_weighted_bipred_idx is 1, the syntax of lower layers (slice header, coding tree block, transform unit, prediction unit, etc.) is modified for each local region inside the slice. The validity or invalidity of the weighted motion compensation prediction of the example prediction may be specified.

  pred_weight_table () is a function indicating a predicate weight table syntax used for weighted motion compensation prediction. This function is called when aps_weighted_prediction_flag is 1 or aps_weighted_bipred_idx is 1.

FIG. 24 is a diagram illustrating an example of a tread weight table syntax 508 according to a modification of the first embodiment. In the modification of the first embodiment, in the syntax structure of FIG. 18, the tread weight table syntax 508 is called from the adaptation parameter set syntax 506. A difference from the predicate weight table syntax 508 of FIG. 21 is that num_ref_common_active_minus1 is changed to MAX_COMMON_REF_MINUS1. Since the number of reference images is described in the slice header syntax 507, it cannot be referenced in the adaptation parameter set syntax 506 that is an upper layer. For this reason, for example, a value obtained by subtracting 1 from the value of num_ref_frames described in the sequence parameter set syntax 504 is set in MAX_COMMON_REF_MINUS1. Further, a value capable of obtaining the maximum number of reference images may be determined according to a predetermined profile or level. Other syntax elements are the same as those in FIG.

  As described above, according to the modification of the first embodiment, the WP parameter information code when one frame is divided into a plurality of slices is obtained by using the structure in which the preadweight table syntax 508 is called from the adaptation parameter set syntax 506. The amount can be greatly reduced.

  For example, the adaptation parameter set syntax 506 having three different types of WP parameter information to be used is encoded first, and the slice header syntax 507 calls a necessary WP parameter information using aps_id according to the situation. The amount of codes can be reduced as compared with the configuration in which the WP parameter information is always encoded with the slice header syntax 507.

(Second Embodiment)
A second embodiment will be described. In the encoding device 600 of the second embodiment, the configuration of the index setting unit 608 is different from that of the encoding device 100 of the first embodiment. In the following, differences from the first embodiment will be mainly described, and components having the same functions as those in the first embodiment will be given the same names and symbols as those in the first embodiment, and the description thereof will be made. Omitted.

  FIG. 25 is a block diagram illustrating an example of the configuration of the index setting unit 608 of the second embodiment. As shown in FIG. 25, the index setting unit 608 includes a reuse determination unit 601 and an index generation unit 602.

  The reuse determination unit 601 compares the reference number of list 1 with the reference number of list 0 and determines whether to reuse the WP parameter information of list 0 as the WP parameter information of list 1. When the same reference image is included between two reference lists and the values of the corresponding WP parameter information are the same, if the same information is encoded not only in list 0 but also in list 1, the amount of code increases. Invite. For this reason, the reuse determination unit 601 reuses the WP parameter information in the list 0.

  Specifically, the reuse determination unit 601 sets the reuse flag to 1 when reusing the WP parameter information in list 0, and sets the reference number of the reference destination (list 0) as reference information. On the other hand, the reuse determining unit 601 sets the reuse flag to 0 when not reusing the WP parameter information in list 0, and sets the WP parameter information corresponding to the reference number in list 1 in the syntax.

  For example, the reuse determination unit 601 determines whether to reuse the WP parameter information in the list 0 as the WP parameter information in the list 1 according to the pseudo code shown in the equation (11).

for (refIdx = 0; refidx <= num_of_active_ref_l1_minus1; refIdx ++) {
refPOC = RefPicOrderCnt (ListL1, refIdx)
refWP = RefWPTable (ListL1, refIdx)
reuse_wp_flag = false;
reuse_ref_idx = 0;
for (curridx = 0; curridx <= num_of_active_ref_l0_minus1; curridx ++) {
if (refPOC == RefPicOrderCnt (ListL0, curridx)
&& RefWPTable (ListL0, curridx, refWP)) {
reuse_wp_flag = true;
reuse_ref_idx = curridx;
}
}
} (11)

  Here, ListL0 indicates list 0, and ListL1 indicates list 1. When a list number, a reference number, and a reference WP parameter are input, RefWPTable () is a flag indicating whether or not the input reference WP parameter matches the WP parameter corresponding to the input list number and reference number. A function to return. A flag value of 1 indicates that both WP parameters match. In the pseudo code shown in Equation (11), when the reference numbers in list 1 are sequentially scanned and there are reference images with the same POC number in list 0, if the WP parameter is the same, the reuse flag is set. The reuse_wp_flag to be shown is set to “ture”, and the corresponding reference number in the list 0 is set to “reuse_ref_idx”.

  The reuse determination unit 601 outputs the WP parameter information after the reuse determination (reuse setting is performed) and inputs the WP parameter information to the index generation unit 602.

  The index generation unit 602 receives the WP parameter information after the reuse determination from the reuse determination unit 601 and performs mapping on syntax elements described later to generate index information. Note that the syntax elements to which the index generation unit 602 performs mapping are different from those in the first embodiment. The index generation unit 602 outputs index information and inputs it to the encoding unit 110.

  FIG. 26 is a flowchart illustrating an example of index information generation processing executed by the index setting unit 608 of the second embodiment.

  When the WP parameter information is input, the reuse determination unit 601 branches the process according to the slice type (step S201).

  When the slice type is a unidirectional prediction slice (P-slice) using only one reference list (No in step S201), the reuse determination unit 601 directly uses the WP parameter information illustrated in FIG. 9A as an index generation unit. Output to 602. The index generation unit 602 generates index information by mapping the WP parameter information input from the reuse determination unit 601 to a predetermined syntax element to be described later, and outputs the index information.

  On the other hand, when the slice type is a bi-directional prediction slice (B-slice) using two reference lists (Yes in step S201), the reuse determination unit 601 initializes the variable refIdc to 0 (step S202). ). The variable refIdc indicates the reference number of list 1.

  Subsequently, the reuse determination unit 601 derives a variable refPOC indicating the POC number of the reference image corresponding to the reference number indicated by the variable refIdx (step S203), and derives a WP parameter corresponding to the reference number indicated by the variable refIdx. (Step S204).

Subsequently, the reuse determination unit 601 sets the flag reuse_wp_flag to false (step S205) and sets the variable curridx to 0 (step S206). The flag reuse_wp_flag indicates whether or not the WP parameters of list 0 and list 1 are the same. The variable curridx indicates the reference number of list 0.

  Subsequently, the reuse determination unit 601 has the same POC number indicated by the derived variable refPOC and the POC number of the reference image corresponding to the reference number indicated by the variable curridx, and the WP corresponding to the reference number indicated by the variable refIdx. It is determined whether the parameter and the WP parameter corresponding to the reference number indicated by the variable curridx are the same (step S207).

  When both POC numbers and both WP parameters are the same (Yes in Step S207), the reuse determining unit 601 sets the flag reuse_wp_flag to true (Step S208) and substitutes the value of the variable curridx for the variable reuse_ref_idx ( Step S209). On the other hand, if both POC numbers or both WP parameters are not the same (No in step S207), the processes in steps S208 and S209 are not performed. A configuration in which the POC number match is not confirmed may be used.

  Subsequently, the reuse determination unit 601 increments the variable curridx (step S210).

  Subsequently, the reuse determination unit 601 determines whether or not the value of the variable curridx is larger than num_of_active_ref_l0_minus1 that is a value obtained by subtracting 1 from the maximum number in list 0 (step S211), and during num_of_active_ref_l0_minus1 or less (No in step S211) ), The process of step S207 to step S210 is repeated.

  When it becomes larger than num_of_active_ref_l0_minus1 (Yes in step S211), the reuse determination unit 601 completes the confirmation of the list 0 and increments the variable refIdx (step S212).

  Subsequently, the reuse determination unit 601 determines whether the value of the variable refIdx is larger than num_of_active_ref_l1_minus1 that is a value obtained by subtracting 1 from the maximum number in the list 1 (step S213), and during num_of_active_ref_l1_minus1 or less (No in step S213) ), The process of step S203 to step S212 is repeated.

  When it becomes larger than num_of_active_ref_l1_minus1 (Yes in step S213), the reuse determination unit 601 completes the confirmation of the list 1, and outputs the WP parameter information after the reuse determination to the index generation unit 602. The index generation unit 602 generates index information by mapping the WP parameter information input from the reuse determination unit 601 to a predetermined syntax element to be described later, and outputs the index information.

  FIG. 27 is a diagram illustrating an example of the tread weight table syntax 506 according to the second embodiment. Of the syntax elements shown in FIG. 27, those having symbols l0 and l1 correspond to lists 0 and 1, respectively. Also, syntax elements having the same prefix as in FIG. 21 are used as the same variable except that the list is different from the list 0 or list 1 but not the common list. For example, luma_weight_l0 [i] is a syntax element representing the i-th weight coefficient with the reference number in the reference list l0.

luma_log2_weight_denom and chroma_log2_weight_denom have the same configuration as in the first embodiment. First, luma_weight_l0_flag, luma_weight_l0 [i], luma_offset_l0 [i], chroma_weight_l0_flag, chroma_weight_l0 [i] [j], and chroma_offset_l0 [i] [j] are defined for the list 0 having the symbol l0. Next, a syntax element relating to the list 1 having the symbol l1 is defined.

  Here, reuse_wp_flag is a syntax element indicating whether to reuse the WP parameter of l0 corresponding to list 0. When reuse_wp_flag is 1, reuse_ref_idx is encoded without encoding the weighting coefficient and the offset. reuse_ref_idx indicates which WP parameter corresponding to which reference number of l0 corresponding to list 0 is used. For example, when reuse_ref_idx is 1, the WP parameter corresponding to reference number 1 in list 0 is copied to the WP parameter corresponding to reference number i in list 1. When reuse_wp_flag is 0, as in list 0, luma_weight_l1_flag, luma_weight_l1 [i], luma_offset_l1 [i], chroma_weight_l1_flag, chroma_weight_l1 [i] [j], and chroma_offset_l1 [i] [j] are encoded.

  As described above, in the second embodiment, the index setting unit 608, when there is a combination including the same reference image in the two reference lists of list 0 and list 1 in the WP parameter information, By reusing, redundant redundant WP parameters are deleted from the WP parameter information to generate index information. Therefore, according to the second embodiment, the code amount of the index information can be reduced.

  In the second embodiment, the WP parameter information corresponding to the list 0 is encoded as before, and when the WP parameter information corresponding to the list 1 is encoded, it is checked whether or not the same reference image is referred to in the list 0. If the same reference image is referenced and the WP parameters are the same, a flag for reusing the WP parameter in list 0 is encoded, and information indicating which reference number in list 0 corresponds is encoded. Turn into. Since these pieces of information are much smaller than the information about weighting factors and offsets, the amount of code of WP parameter information can be greatly reduced as compared with the case where List 0 and List 1 are encoded separately. .

  In the second embodiment, when the reference numbers included in the list 0 and the list 1 indicate the same reference image and the WP parameters are different, different WP parameters can be set. That is, when there is a combination indicating the same reference image in the reference list, in the first embodiment, the same WP parameter is forcibly set, and in the second embodiment, the same only when the WP parameter is the same. Remove redundant representation of WP parameters.

(Third embodiment)
A third embodiment will be described. FIG. 28 is a block diagram illustrating an example of the configuration of the encoding apparatus 700 according to the third embodiment. The encoding apparatus 700 of the third embodiment is different from the encoding apparatus 100 of the first embodiment in that it further includes a motion information memory 701, a motion information acquisition unit 702, and a changeover switch 703. In the following, differences from the first embodiment will be mainly described, and components having the same functions as those in the first embodiment will be given the same names and symbols as those in the first embodiment, and the description thereof will be made. Omitted.

  The motion information memory 701 temporarily stores the motion information applied to the pixel block that has been encoded as reference motion information. The motion information memory 701 may reduce the amount of information by performing compression processing such as sub-sampling on the motion information.

  FIG. 29 is a diagram illustrating a detailed example of the motion information memory 701 according to the third embodiment. As shown in FIG. 29, the motion information memory 701 is held in units of frames or slices, and stores spatial direction reference motion information memory 701A that stores motion information on the same frame as reference motion information 710, and has already been encoded. It further has a time direction reference motion information memory 701B for storing the motion information of the frame for which the processing is completed as reference motion information 710. A plurality of temporal direction reference motion information memories 701B may be provided depending on the number of reference frames used for prediction by the encoding target frame.

  The reference motion information 710 is held in the spatial direction reference motion information memory 701A and the temporal direction reference motion information memory 701B in a predetermined area unit (for example, 4 × 4 pixel block unit). The reference motion information 710 further includes information indicating whether the region is encoded by inter prediction described later or encoded by intra prediction described later. Also, the pixel block (coding unit or prediction unit) is H.264. As in skip mode, direct mode, or merge mode described later in H.264, motion vector values in motion information are not encoded, but are inter-predicted using motion information predicted from an encoded region. In this case, the motion information of the pixel block is held as the reference motion information 710.

  When the encoding process of the frame or slice to be encoded is completed, the handling of the spatial direction reference motion information memory 701A of the frame is changed as the temporal direction reference motion information memory 701B used for the frame to be encoded next. The At this time, in order to reduce the memory capacity of the time direction reference motion information memory 701B, the motion information may be compressed and the compressed motion information may be stored in the time direction reference motion information memory 701B.

  The motion information acquisition unit 702 receives the reference motion information from the motion information memory 701 and outputs motion information B used for the encoded pixel block. Details of the motion information acquisition unit 702 will be described later.

  The changeover switch 703 selects either motion information B output from the motion information acquisition unit 702 or motion information A output from the motion evaluation unit 109 according to prediction mode information described later, and generates a predicted image as motion information. Output to the unit 107. The motion information A output from the motion evaluation unit 109 encodes difference information with a predicted motion vector acquired from a predicted motion vector acquisition unit (not shown) and predicted motion vector acquisition position information. Such a prediction mode is hereinafter referred to as an inter mode. On the other hand, the motion information B output from the motion information acquisition unit 702 according to the prediction mode information is merged with the motion information from adjacent pixel blocks and applied to the encoded pixel block as it is. It is not necessary to encode information (for example, motion vector difference information). Hereinafter, such a prediction mode is referred to as a merge mode.

  The prediction mode information conforms to the prediction mode controlled by the encoding control unit 111 and includes switching information of the changeover switch 703.

  Details of the motion information acquisition unit 702 will be described below.

  The motion information acquisition unit 702 receives the reference motion information and outputs motion information B to the changeover switch 703. 30A and 30B are diagrams illustrating examples of block positions from which motion information candidates are derived with respect to the encoded pixel block according to the third embodiment. 30A indicate spatially adjacent pixel blocks for deriving motion information candidates to the encoded pixel block, and T in FIG. 30B indicates temporally adjacent pixel blocks for deriving motion information candidates. .

  FIG. 31 is a diagram illustrating an example of a relationship between a pixel block position and a pixel block position index (idx) of a plurality of motion information candidates according to the third embodiment. The block positions of motion information candidates are arranged in the order of spatially adjacent block positions (A to E) and temporally adjacent pixel block positions (T), and Available (inter prediction is applied). If there is a pixel block, idx is sequentially assigned until the maximum number (N−1) of motion information stored in the merge mode storage list MergeCandList is reached.

FIG. 32 is a flowchart illustrating an example of storage processing in the MergeCandList according to the third embodiment.

  First, the motion information acquisition unit 702 initializes the number NumMergeCand stored in the MergeCandList to 0 (step S301).

  Subsequently, the motion information acquisition unit 702 determines whether or not all space adjacent blocks (for example, blocks A to E) are available (steps S302 and S303). If it is “Available” (Yes in Step S303), the motion information acquisition unit 702 stores the motion information of the space adjacent block in the MergeCandList and increments numMergeCand (Step S304). When it is not Available (No in Step S303), the motion information acquisition unit 702 does not perform the process of Step S304.

  When the motion information acquisition unit 702 executes the processes of steps S303 and S304 for all spatial adjacent blocks (step S305), the motion information acquisition unit 702 determines whether the temporal adjacent block (for example, block T) is available. (Step S306).

  If it is Available (Yes in Step S306), the motion information acquisition unit 702 stores the motion information of the time adjacent block T in the MergeCandList and increments numMergeCand (Step S307). When it is not Available (No in Step S306), the motion information acquisition unit 702 does not perform the process of Step S307.

  Subsequently, the motion information acquisition unit 702 deletes the motion information that overlaps in the MergeCandList, and decrements numMergeCand by the number of deleted (step S308). In order to determine whether or not there is overlapping motion information in MergeCandList, it is determined whether two types of motion information (motion vector mv, reference number refIdx) and WP parameter information in MergeCandList match, and all information is If they match, delete one from MergeCandList [].

  FIG. 33 is a diagram showing an example of a motion information storage list according to the third embodiment. Specifically, assuming that N = 5, the motion information (motion vector mv corresponding to each idx) according to the table shown in FIG. , Reference number refIdx) is an example of a storage list (MergeCandList [idx]). In FIG. 33, idx = 0 is reference motion information of list 0 (motion vector mv0L0, reference number refIdx0L0), idx = 1 is reference motion information of list 1 (motion vector mv1L1, reference number refIdx1L1), idx = 2 to 2 4 stores reference motion information of both list 0 and list 1 (motion vectors mv2L0, mv2L1, reference numbers refIdx2L0, refIdx2L1).

  The encoded pixel block selects one of the maximum N types of idx, derives reference motion information of the pixel block corresponding to the selected idx from MergeCandList [idx], and outputs it as motion information B. The encoded pixel block outputs motion information having a zero vector as a predicted motion information candidate B when there is no reference motion information. Information on the selected idx (a syntax merge_idx described later) is included in the prediction mode information, and is encoded by the encoding unit 110.

  It should be noted that blocks adjacent in space and time are not limited to FIGS. 30A and 30B, and may be any positions as long as they are already encoded regions. As another example of the list, the maximum number (N) and order in the MergeCandList are not limited to the above, and any order / maximum number may be used.

If the encoded slice is a B slice and the reference motion information of spatially and temporally adjacent pixel blocks is stored in MergeCandList [] but does not reach the maximum storage number (N), then MergeCandList The bi-directional prediction is newly generated by combining the motion information of list 0 and list 1 stored in [] with each other, and stored in MergeCandList [].

  FIG. 34 is a flowchart illustrating an example of a method for storing motion information in a storage list according to the third embodiment.

  First, the motion information acquisition unit 702 sets the number stored in the MergeCandList from spatially and temporally adjacent pixel blocks as numInputMergeCand, initializes the variable numMergeCand indicating the current stored number with numInputMergeCand, and sets the variables combIdx and combCnt. It is initialized with 0 (step S401).

  Subsequently, the motion information acquisition unit 702 derives a variable l0CandIdx and a variable l1CandIdx from combIdx using a newly generated bidirectional prediction combination (see FIG. 35) (step S402). The combination for deriving the variable l0CandIdx and the variable l1CandIdx from the combIdx is not limited to the example shown in FIG. 36, and may be rearranged in an arbitrary order as long as they do not overlap.

  Subsequently, the motion information acquisition unit 702 sets the reference motion information (two types of list 0 and list 1 in the case of bidirectional prediction) stored in MergeCandList [l0CandIdx] to l0Cand. The reference number in l0Cand is referred to as refIdxL0l0Cand, and the motion vector is referred to as mvL0l0Cand. Similarly, the motion information acquisition unit 702 sets the reference motion information stored in MergeCandList [l1CandIdx] to l1Cand. The reference number in l1Cand is referred to as refIdxL1l1Cand, and the motion vector is referred to as mvL1l1Cand (step S403). A combination of motion information that performs bidirectional prediction using l0Cand as the motion information of list 0 and L1Cand as the motion information of list 1 is referred to as combCand.

  Subsequently, the motion information acquisition unit 702 determines whether the blocks referenced by l0Cand and l1Cand are the same using the conditional expressions (12) to (15) (step S404).

  Are l0Cand and l1Cand available? (12)

  RefPicOrderCnt (refIdxL0l0Cand, L0)! = RefPicOrderCnt (refIdxL1l1Cand, L1) (13)

  mvL0l0Cand! = mvL1l1Cand (14)

  WpParam (refIdxL0l0Cand, L0)! = WpParam (refIdxL1l1Cand, L1) (15)

  RefPicOrderCnt (refIdx, LX) is a function for deriving a POC (Picture Order Count) of a reference frame corresponding to the reference number refIdx in the reference list X (X = 0, 1). WpParam (refIdx, LX) is a function for deriving WP parameter information of the reference frame corresponding to the reference number refIdx in the reference list X (X = 0, 1). This function is Yes when the reference frame of list 0 and the reference frame of list 1 are not the same WP parameter (WP presence / absence WP_flag, weight coefficient Weight, offset Offset), and when the parameters are completely the same Returns No. As another example of WpParam (), it is not necessary to check the match of all parameters, but only using a part of the data of WP parameters, such as checking the match of WP presence / absence WP_flag and offset Offset only. May be.

  When the conditional expression (12) is satisfied and any one of the conditional expressions (13) to (15) is satisfied (Yes in step S404), the motion information acquisition unit 702 is the same as l0Cand and l1Cand in mergeCandList []. It is determined whether or not there is bidirectional prediction using the combination (step S405).

  If there is no identical combination of l0Cand and l1Cand in mergeCandList [] (Yes in step S405), the motion information acquisition unit 702 adds the combCand to the end of the mergeCandList (step S406). Specifically, the motion information acquisition unit 702 substitutes combCand into mergeCandList [numMergeCand], and increments numMergeCand and combCnt.

  Subsequently, the motion information acquisition unit 702 increments combIdx (step S407).

  Subsequently, the motion information acquisition unit 702 determines whether to end the storage in mergeCandList [] using the conditional expressions (16) to (18) (step S408).

  combIdx == numInputMergeCand * (numInputMergeCand? 1)) (16)

  numMergeCand == maxNumMergeCand (17)

  combCnt == N (18)

  If any of the conditional expressions (16) to (18) is satisfied (Yes in step S408), the process ends. If any of the conditional expressions (16) to (18) are satisfied (No in step S408). ), The process returns to step S402.

  The above is a description of the processing when the coded slice is a B slice and the reference motion information of spatially and temporally adjacent pixel blocks has not reached the maximum storage number (N) even if it is stored in MergeCandList []. It is. As described above, the motion information B is output from the motion information acquisition unit 702 to the changeover switch 703.

  FIG. 36 is a diagram illustrating an example of the tread unit syntax 512 according to the third embodiment. skip_flag is a flag indicating whether or not the prediction mode of the coding unit to which the prediction unit syntax belongs is the skip mode. When skip_flag is 1, it indicates that syntaxes other than prediction mode information (coding unit syntax, prediction unit syntax, transform unit syntax) are not encoded. When skip_flag is 0, it indicates that the prediction mode of the coding unit to which the prediction unit syntax belongs is not the skip mode.

  Next, merge_flag which is a flag indicating whether or not the encoded pixel block (prediction unit) is in the merge mode is encoded. Merge_flag indicates that the prediction unit is in the merge mode when the value is 1, and indicates that the prediction unit uses the inter mode when the value is 0. In the case of Merge_flag, merge_idx, which is information specifying the above-described pixel block position index (idx), is encoded.

  When Merge_flag is 1, it is not necessary to encode the prediction unit syntax other than merge_flag and merge_idx, and when Merge_flag is 0, it indicates that the prediction unit is in inter mode.

As described above, in the third embodiment, when applying weighted motion compensation and the merge mode at the same time, the encoding apparatus 700 uses the merge mode storage list (MergeCandList) for the reference motion information of pixel blocks adjacent in space and time. When the overlapping motion information in () is deleted, even if the motion vector and the reference number match, if the WP parameter information is different, the problem of being deleted from the MergeCandList is solved. If the maximum number (N) of storage is not reached even if it is stored in MergeCandList, the motion information of list 0 and list 1 stored in MergeCandList at that time is combined with each other and a new bidirectional prediction is performed. When generating and storing in MergeCandList [], the problem that two types of motion information used for bidirectional prediction refer to the same block is solved.

  When two types of motion information used for bidirectional prediction refer to the same block, the prediction value by bidirectional prediction and the prediction value by unidirectional prediction match. Since the prediction efficiency of bidirectional prediction is generally higher than the prediction efficiency of unidirectional prediction, it is desirable that the two types of motion information used for bidirectional prediction do not refer to the same block. When determining whether or not the two types of motion information used for bidirectional prediction refer to the same block, the encoding apparatus 700, in addition to the motion vector and the reference frame number position (POC) derived from the reference number, In addition, a weighted motion compensation parameter is introduced as a determination item. Accordingly, the encoding apparatus 700 determines that two types of motion information with different weighted motion compensation parameters do not refer to the same block even if the motion vector and the reference frame number position (POC) are the same. The prediction efficiency of motion information used for bidirectional prediction stored in the merge mode storage list can be improved.

  As another example of the storing process in the MergeCandList, when the motion information of the temporally adjacent block T is stored in the MergeCandList, the WP parameter in the reference frame of the temporally adjacent block T and the WP parameter in the reference frame of the encoded pixel block are It may be determined whether or not they match, and may be stored in the MergeCandList only when the WP parameters match. This is because when the WP parameter in the reference frame of the temporally adjacent block T and the WP parameter in the reference frame of the encoded pixel block are different, it can be assumed that the correlation of motion information decreases in the temporally adjacent block T and the encoded pixel block. It is.

  FIG. 37 is a flowchart illustrating another example of the storing process in the MergeCandList according to the third embodiment. The flowchart shown in FIG. 38 is the same as the flowchart shown in FIG. 32 except for step S507.

  As yet another example, if the WP parameter in the reference frame of the temporally adjacent block T and the WP parameter in the reference frame of the encoded pixel block are different, the encoded pixel of the blocks spatially adjacent to the block T A block having the same WP parameter in the reference frame of the block may be replaced with the block T. At this time, the correlation of motion information does not decrease between the temporally adjacent block T and the encoded pixel block.

(Fourth embodiment)
In the fourth embodiment, a decoding device that decodes encoded data encoded by the encoding device of the first embodiment will be described.

  FIG. 38 is a block diagram illustrating an example of the configuration of the decoding device 800 according to the fourth embodiment.

  The decoding device 800 decodes encoded data stored in an input buffer (not shown) into a decoded image and outputs the decoded image to an output buffer (not shown) as an output image. The encoded data is output from, for example, the encoding device 100 of FIG. 1 and the like, and is input to the decoding device 800 via a storage system, a transmission system, or a buffer (not shown).

As illustrated in FIG. 38, the decoding device 800 includes a decoding unit 801, an inverse quantization unit 802, an inverse orthogonal transform unit 803, an addition unit 804, a predicted image generation unit 805, and an index setting unit 806. Prepare. The inverse quantization unit 802, the inverse orthogonal transform unit 803, the addition unit 804, and the predicted image generation unit 805 are respectively the inverse quantization unit 104, the inverse orthogonal transform unit 105, the addition unit 106, the predicted image generation unit 107, FIG. Are substantially the same or similar elements. Note that the decoding control unit 807 shown in FIG. 38 controls the decoding device 800 and can be realized by, for example, a CPU.

  The decoding unit 801 performs decoding based on the syntax for each frame or field in order to decode the encoded data. The decoding unit 801 sequentially entropy-decodes the code string of each syntax, and includes motion information including a prediction mode, a motion vector, a reference number, and the like, index information for weighted motion compensation prediction, and codes such as quantization transform coefficients The encoding parameter of the conversion target block is reproduced. The encoding parameters are all parameters necessary for decoding such as information related to transform coefficients other than the above and information related to quantization. The decoding unit 801 outputs the motion information, the index information, and the quantized transform coefficient, inputs the quantized transform coefficient to the inverse quantization unit 802, inputs the index information to the index setting unit 806, and converts the motion information into the predicted image. Input to the generation unit 805.

  The inverse quantization unit 802 performs an inverse quantization process on the quantized transform coefficient input from the decoding unit 801 to obtain a restored transform coefficient. Specifically, the inverse quantization unit 802 performs inverse quantization according to the quantization information used in the decoding unit 801. More specifically, the inverse quantization unit 802 multiplies the quantized transform coefficient by the quantization step size derived from the quantization information to obtain a restored transform coefficient. The inverse quantization unit 802 outputs the restored transform coefficient and inputs it to the inverse orthogonal transform unit 803.

  The inverse orthogonal transform unit 803 performs inverse orthogonal transform corresponding to the orthogonal transform performed on the encoding side on the reconstructed transform coefficient input from the inverse quantization unit 802 to obtain a reconstructed prediction error. The inverse orthogonal transform unit 803 outputs the restoration prediction error and inputs it to the addition unit 804.

  The adding unit 804 adds the restored prediction error input from the inverse orthogonal transform unit 803 and the corresponding predicted image to generate a decoded image. The adding unit 804 outputs the decoded image and inputs the decoded image to the predicted image generation unit 805. The adding unit 804 outputs the decoded image to the outside as an output image. The output image is then temporarily stored in an external output buffer (not shown) or the like. For example, in accordance with the output timing managed by the decoding control unit 807, the output image is sent to a display device system or a video device system (not shown). Is output.

  The index setting unit 806 receives the index information input from the decoding unit 801, confirms the reference list (list number) and the reference image (reference number), outputs WP parameter information, and outputs the WP parameter information to the predicted image generation unit 805. input.

  FIG. 39 is a block diagram illustrating an example of the configuration of the index setting unit 806 according to the fourth embodiment. As shown in FIG. 39, the index setting unit 806 includes a reference image confirmation unit 901 and a WP parameter generation unit 902.

  The reference image confirmation unit 901 receives the index information from the decoding unit 801, and confirms whether the reference numbers included in the two reference lists indicate the same reference image.

  Here, the reference numbers in the reference list are H.264, for example. H.264 has already been decoded. For this reason, the reference image confirmation unit 901 uses the H.264 standard. According to the management of the Decoded Picture Buffer (DPB) defined by H.264 or the like, it is possible to check whether there is a combination pointing to the same reference image using the derived reference list and reference number. Note that the DPB control is described in H.264. The method described in H.264 or the like may be used, or another method may be used. Here, the reference list and the reference number may be determined in advance by DPB control.

The reference image confirmation unit 901 creates a common reference list, and confirms the reference list and the reference number in the scan order shown in FIG. The common reference list is created in accordance with the pseudo code shown in Equation (10).

  The WP parameter generation unit 902 generates WP parameter information corresponding to the reference list and the reference number from the created common reference list based on the relationship between the reference list and the reference number confirmed by the reference image confirmation unit 901. Output to the predicted image generation unit 805. Since the WP parameter information has already been described with reference to FIGS. 9A and 9B, description thereof will be omitted.

  The WP parameter generation unit 902 uses the common_scan_list () function and the common_scan_ref_idx () function according to the common list, subtracts the reference list and reference number by scanning, and supplements the WP parameter information in the missing place.

  That is, the common lists of FIGS. 15 and 16 are restored to FIGS. 11 and 13, respectively, and based on these correspondences, the index information corresponding to the common lists is changed to the WP parameters shown in FIGS. 9A and 9B. Assign to information.

  Note that the scan order shown in FIG. 14 is an example, and any other scan order may be used as long as it is a predetermined scan order. Further, the pseudo code shown in Equation (10) is an example, and if the purpose of this processing can be realized, it is possible to add processing and reduce redundant processing.

  Returning to FIG. 38, the predicted image generation unit 805 uses the motion information input from the decoding unit 801, the WP parameter information input from the index setting unit 806, and the decoded image input from the addition unit 804 to generate a predicted image. Is generated.

  Here, the details of the predicted image generation unit 805 will be described with reference to FIG. Like the predicted image generation unit 107, the predicted image generation unit 805 includes a multi-frame motion compensation unit 201, a memory 202, a unidirectional motion compensation unit 203, a prediction parameter control unit 204, a reference image selector 205, and a frame memory 206. And a reference image control unit 207.

  The frame memory 206 stores the decoded image input from the addition unit 106 as a reference image under the control of the reference image control unit 207. The frame memory 206 has a plurality of memory sets FM1 to FMN (N ≧ 2) for temporarily storing reference images.

  The prediction parameter control unit 204 prepares a plurality of combinations of reference image numbers and prediction parameters as a table based on the motion information input from the decoding unit 801. Here, the motion information indicates a motion vector indicating a shift amount of motion used in motion compensation prediction, a reference image number, information on a prediction mode such as unidirectional / bidirectional prediction, and the like. The prediction parameter refers to information regarding a motion vector and a prediction mode. Then, the prediction parameter control unit 204 selects and outputs a combination of a reference image number and a prediction parameter used for generating a prediction image based on the motion information, inputs the reference image number to the reference image selector 205, and outputs the prediction parameter. Is input to the unidirectional motion compensation unit 203.

The reference image selector 205 is a switch for switching which output terminal of the frame memories FM1 to FMN included in the frame memory 206 is connected according to the reference image number input from the prediction parameter control unit 204. For example, if the reference image number is 0, the reference image selector 205 connects the output end of FM1 to the output end of the reference image selector 205. If the reference image number is N-1, the reference image selector 205 refers to the output end of the FMN. Connected to the output terminal of the image selector 205. The reference image selector 205 outputs the reference image stored in the frame memory to which the output terminal is connected among the frame memories FM1 to FMN included in the frame memory 206, and inputs the reference image to the unidirectional motion compensation unit 203. Note that in the decoding device 800, since the reference image is not used except by the predicted image generation unit 805, the reference image does not have to be output to the outside of the predicted image generation unit 805.

  The unidirectional prediction motion compensation unit 203 performs a motion compensation prediction process according to the prediction parameter input from the prediction parameter control unit 204 and the reference image input from the reference image selector 205, and generates a unidirectional prediction image. Since motion compensation prediction has already been described with reference to FIG. 5, description thereof will be omitted.

  The unidirectional prediction motion compensation unit 203 outputs a unidirectional prediction image and temporarily stores it in the memory 202. Here, when the motion information (prediction parameter) indicates bidirectional prediction, the multi-frame motion compensation unit 201 performs weighted prediction using two types of unidirectional prediction images, and thus the unidirectional prediction motion compensation unit 203. Stores the unidirectional predicted image corresponding to the first in the memory 202, and directly outputs the unidirectional predicted image corresponding to the second to the multi-frame motion compensation unit 201. Here, the first unidirectional prediction image corresponding to the first is the first prediction image, and the second unidirectional prediction image is the second prediction image.

  Two unidirectional motion compensation units 203 may be prepared, and each may generate two unidirectional prediction images. In this case, when the motion information (prediction parameter) indicates unidirectional prediction, the unidirectional motion compensation unit 203 directly outputs the first unidirectional prediction image as the first prediction image to the multi-frame motion compensation unit 201. Good.

  The multi-frame motion compensation unit 201 uses the first prediction image input from the memory 202, the second prediction image input from the unidirectional prediction motion compensation unit 203, and the WP parameter information input from the motion evaluation unit 109. Then, a prediction image is generated by performing weighted prediction. The multi-frame motion compensation unit 201 outputs a prediction image and inputs the prediction image to the addition unit 804.

  Here, the details of the multi-frame motion compensation unit 201 will be described with reference to FIG. Similar to the predicted image generation unit 107, the multi-frame motion compensation unit 201 includes a default motion compensation unit 301, a weighted motion compensation unit 302, a WP parameter control unit 303, and WP selectors 304 and 305.

  The WP parameter control unit 303 outputs a WP application flag and weight information based on the WP parameter information input from the index setting unit 806, inputs the WP application flag to the WP selectors 304 and 305, and weights the weight information. Input to the motion compensation unit 302.

  Here, the WP parameter information includes the fixed-point precision of the weighting factor, the first WP application flag corresponding to the first predicted image, the first weighting factor, the first offset, and the second WP adaptation corresponding to the second predicted image. Contains information on flag, second weighting factor, and second offset. The WP application flag is a parameter that can be set for each corresponding reference image and signal component, and indicates whether to perform weighted motion compensation prediction. The weight information includes information on the fixed point precision of the weight coefficient, the first weight coefficient, the first offset, the second weight coefficient, and the second offset. The WP parameter information represents the same information as in the first embodiment.

  Specifically, when the WP parameter information is input from the index setting unit 806, the WP parameter control unit 303 separates the WP parameter information into a first WP application flag, a second WP application flag, and weight information, and outputs them. The first WP application flag is input to the WP selector 304, the second WP application flag is input to the WP selector 305, and the weight information is input to the weighted motion compensation unit 302.

  The WP selectors 304 and 305 switch the connection end of each predicted image based on the WP application flag input from the WP parameter control unit 303. When each WP application flag is 0, the WP selectors 304 and 305 connect each output terminal to the default motion compensation unit 301. Then, the WP selectors 304 and 305 output the first predicted image and the second predicted image and input them to the default motion compensation unit 301. On the other hand, when each WP application flag is 1, the WP selectors 304 and 305 connect each output terminal to the weighted motion compensation unit 302. The WP selectors 304 and 305 output the first predicted image and the second predicted image, and input them to the weighted motion compensation unit 302.

  The default motion compensation unit 301 performs an average value process based on the two unidirectional predicted images (first predicted image and second predicted image) input from the WP selectors 304 and 305 to generate a predicted image. Specifically, when the first WP application flag and the second WP application flag are 0, the default motion compensation unit 301 performs average value processing based on Expression (1).

  When the prediction mode indicated by the motion information (prediction parameter) is unidirectional prediction, the default motion compensation unit 301 uses only the first prediction image and calculates a final prediction image based on Expression (4). calculate.

  The weighted motion compensation unit 302 is based on the two unidirectional prediction images (first prediction image and second prediction image) input from the WP selectors 304 and 305 and the weight information input from the WP parameter control unit 303. Performs weighted motion compensation. Specifically, when the first WP application flag and the second WP application flag are 1, the weighted motion compensation unit 302 performs the weighting process based on Expression (7).

  Note that the weighted motion compensation unit 302 performs rounding processing by controlling logWDC, which is fixed-point precision, as in Expression (8) when the calculation accuracy of the first predicted image, the second predicted image, and the predicted image is different. Is realized.

  Also, when the prediction mode indicated by the motion information (prediction parameter) is unidirectional prediction, the weighted motion compensation unit 302 uses only the first prediction image and uses the final prediction image based on Equation (9). Is calculated.

  Note that the weighted motion compensation unit 302, when the calculation accuracy of the first prediction image, the second prediction image, and the prediction image is different, uses the log WDC, which is a fixed-point accuracy, as in Expression (8) as in the bidirectional prediction. Rounding processing is realized by controlling to.

  The fixed-point precision of the weighting factor has already been described with reference to FIG. In the case of unidirectional prediction, various parameters (second WP adaptive flag, second weighting factor, and second offset information) corresponding to the second predicted image are not used, and are set to predetermined initial values. May be.

  Decoding section 801 uses syntax 500 shown in FIG. A syntax 500 indicates a structure of encoded data to be decoded by the decoding unit 801. The syntax 500 has already been described with reference to FIG. Also, the picture parameter set syntax 505 has already been described with reference to FIG. 19 except that encoding is decoding, and thus description thereof will be omitted. Also, the slice header syntax 507 has already been described with reference to FIG. 20 except that encoding is decoding, and thus description thereof will be omitted. Also, the treadweight table syntax 508 has already been described with reference to FIG. 21 except that the encoding is decoding, and thus the description thereof will be omitted.

  As described above, in the fourth embodiment, the decoding apparatus 800 is a bi-directional prediction slice in which bi-directional prediction can be selected, and weighted motion using two indexes having the same reference image but different reference image numbers. When the compensation is performed, the index having the same value is decoded twice, thereby solving the problem that the coding efficiency is lowered.

  By rearranging the two reference lists possessed by the bidirectional prediction slice and the reference numbers determined for each list into the common list and the common reference number, a combination of reference numbers indicating the same reference image in the reference list is obtained. Since it is not included, the decoding apparatus 800 can reduce the code amount of the redundant index.

(Modification of the fourth embodiment)
A modification of the fourth embodiment will be described. In the modification of the fourth embodiment, the syntax element used by the decoding unit 801 is different from the syntax element of the fourth embodiment. Since the sequence parameter set syntax 504 has already been described with reference to FIG. 22 except that the encoding is decoding, the description thereof will be omitted. The adaptation parameter set syntax 506 has already been described with reference to FIG. 23 except that the encoding is decoding, and thus the description thereof will be omitted. The treadweight table syntax 508 has already been described with reference to FIG. 24 except that the encoding is decoding, and thus the description thereof will be omitted.

  As described above, according to the modified example of the fourth embodiment, the structure of calling the preadweight table syntax 508 from the adaptation parameter set syntax 506 makes it possible to code the WP parameter information when one frame is divided into a plurality of slices. The amount can be greatly reduced.

  For example, the adaptation parameter set syntax 506 having three different types of WP parameter information to be used is decoded first, and the slice header syntax 507 calls the necessary WP parameter information using aps_id according to the situation. The amount of codes can be reduced as compared with the configuration in which the WP parameter information is always decoded with the slice header syntax 507.

(Fifth embodiment)
In the fifth embodiment, a decoding device that decodes encoded data encoded by the encoding device of the second embodiment will be described. In the decoding device 1000 of the fifth embodiment, the configuration of the index setting unit 1006 is different from that of the decoding device 800 of the fourth embodiment. In the following, differences from the fourth embodiment will be mainly described, and components having the same functions as those in the fourth embodiment will be given the same names and symbols as those in the fourth embodiment, and the description thereof will be made. Omitted.

  FIG. 40 is a block diagram illustrating an example of the configuration of the index setting unit 1006 of the fifth embodiment. As shown in FIG. 40, the index setting unit 1006 includes a reuse determination unit 1001 and a WP parameter generation unit 1002.

  The reuse determination unit 1001 checks the reuse flag and confirms whether to reuse the WP parameter in list 0. When reusing the WP parameter in list 0, the WP parameter generation unit 1002 copies the WP parameter in list 0 as a reference destination to the WP parameter in list 1 according to information on which WP parameter in list 0 is to be copied. Information on the reuse flag and the reference number indicating the reference destination is described in the syntax element, and the WP parameter generation unit 1002 interprets the index information to restore the WP parameter information.

For example, the reuse determination unit 1001 can execute the list 0 according to the pseudo code shown in the mathematical formula (11).
Confirm whether to reuse the WP parameter.

  The treadweight table syntax 508 has already been described with reference to FIG. 27 except that the encoding is decoding, and thus the description thereof will be omitted.

  As described above, in the fifth embodiment, the decoding apparatus 1000 is a bi-directional prediction slice in which bi-directional prediction can be selected, and weighted motion using two indexes having the same reference image but different reference image numbers. When the compensation is performed, the index having the same value is decoded twice, thereby solving the problem that the coding efficiency is lowered.

  The decoding apparatus 1000 decodes the index of the list 0 as usual, and when decoding the index of the list 1, checks whether a combination of reference numbers indicating the same reference image exists in the reference list, and the index is the same. In this case, by reusing the index used in the list 1 with the index used in the list 0, it is possible to avoid decoding the same index twice and reduce the code amount of the redundant index.

(Sixth embodiment)
In the sixth embodiment, a decoding device that decodes encoded data encoded by the encoding device of the third embodiment will be described. FIG. 41 is a block diagram illustrating an example of the configuration of the decoding device 1100 according to the sixth embodiment. The decoding device 1100 of the sixth embodiment is different from the decoding device 800 of the fourth embodiment in that it further includes a motion information memory 1101, a motion information acquisition unit 1102, and a changeover switch 1103. In the following, differences from the fourth embodiment will be mainly described, and components having the same functions as those in the fourth embodiment will be given the same names and symbols as those in the fourth embodiment, and the description thereof will be made. Omitted.

  The motion information memory 1101 temporarily stores motion information applied to the decoded pixel block as reference motion information. The motion information memory 1101 may reduce the amount of information by performing compression processing such as sub-sampling on the motion information. As shown in FIG. 29, the motion information memory 1101 is held in units of frames or slices. The spatial direction reference motion information memory 701A stores motion information on the same frame as reference motion information 710, and decoding has already been completed. It further has a time direction reference motion information memory 701B for storing the motion information of the frame as reference motion information 710. A plurality of temporal direction reference motion information memories 701B may be provided according to the number of reference frames used for prediction by the decoding target frame.

  The reference motion information 710 is held in the spatial direction reference motion information memory 701A and the temporal direction reference motion information memory 701B in a predetermined area unit (for example, 4 × 4 pixel block unit). The reference motion information 710 further includes information indicating whether the region is encoded by inter prediction described later or encoded by intra prediction described later. Also, the pixel block (coding unit or prediction unit) is H.264. When the motion vector value in the motion information is not decoded and is inter-predicted using the motion information predicted from the decoded region, as in the skip mode, the direct mode or the merge mode described later in H.264 Also, the motion information of the pixel block is held as reference motion information 710.

  When the decoding process of the frame or slice to be decoded is completed, the handling of the spatial direction reference motion information memory 701A of the frame is changed as the time direction reference motion information memory 701B used for the frame to be decoded next. At this time, in order to reduce the memory capacity of the time direction reference motion information memory 701B, the motion information may be compressed and the compressed motion information may be stored in the time direction reference motion information memory 701B.

  The motion information acquisition unit 1102 receives the reference motion information from the motion information memory 1101 and outputs motion information B used for the decoded pixel block. Since the operation of the motion information acquisition unit 1102 is the same as that of the motion information acquisition unit 702 of the third embodiment, description thereof is omitted.

  The change-over switch 1103 selects either the motion information B output from the motion information acquisition unit 1102 or the motion information A output from the decoding unit 801 in accordance with prediction mode information described later, and generates a prediction image generation unit as motion information. Output to 805. The motion information A output from the decoding unit 801 decodes difference information from a prediction motion vector acquired from a prediction motion vector acquisition unit (not shown) and prediction motion vector acquisition position information. Such a prediction mode is hereinafter referred to as an inter mode. On the other hand, the motion information B output from the motion information acquisition unit 1102 according to the prediction mode information is merged with the motion information from the adjacent pixel blocks and applied as it is to the decoded pixel block. Decoding of (for example, motion vector difference information) is not necessary. Hereinafter, such a prediction mode is referred to as a merge mode.

  The prediction mode information conforms to the prediction mode controlled by the decoding control unit 807 and includes switching information of the changeover switch 1103.

  Since the tread unit syntax 512 has already been described with reference to FIG. 36 except that the encoding is decoding, the description thereof will be omitted.

  Since the effect of the sixth embodiment is the same as that of the third embodiment, the description thereof is omitted.

  As another example of the storing process in the MergeCandList, when the motion information of the temporally adjacent block T is stored in the MergeCandList, the WP parameter in the reference frame of the temporally adjacent block T and the WP parameter in the reference frame of the decoded pixel block are the same. Only if the WP parameters match, it may be stored in the MergeCandList. This is because when the WP parameter in the reference frame of the temporally adjacent block T and the WP parameter in the reference frame of the decoded pixel block are different, it can be estimated that the correlation of motion information decreases in the temporally adjacent block T and the decoded pixel block. .

  As yet another example, when the WP parameter in the reference frame of the temporally adjacent block T and the WP parameter in the reference frame of the decoded pixel block are different, of the blocks spatially adjacent to the block T, A block having the same WP parameter in the reference frame may be replaced with the block T. At this time, the correlation of motion information does not decrease between the temporally adjacent block T and the decoded pixel block.

(Modification)
Note that the first to third embodiments may be selectively used. That is, information on which method to select is determined in advance, and information indicating whether the method of the first embodiment or the method of the second embodiment is used when a bidirectional slice is selected, You may switch. For example, a flag for switching between these methods can be set in the slice header syntax shown in FIG. 20 and can be easily selected in accordance with the coding situation. Further, it may be determined in advance which method is used in accordance with a specific hardware configuration. The predicted image generation unit and the index setting unit of the first to third embodiments are suitable for both hardware implementation and software implementation.

Further, the fourth to sixth embodiments may be selectively used. That is, information on which method to select is determined in advance, and information indicating whether to use the method of the fourth embodiment or the method of the fifth embodiment when the bidirectional slice is selected, You may switch. For example,
A flag for switching between these methods can be set in the slice header syntax shown in FIG. 20 and can be easily selected in accordance with the coding situation. Further, it may be determined in advance which method is used in accordance with a specific hardware configuration. The predicted image generation unit and the index setting unit of the fourth to sixth embodiments are suitable for both hardware implementation and software implementation.

  In addition, syntax elements not defined in the above embodiment may be inserted between the rows of the syntax tables exemplified in the first to sixth embodiments, or descriptions related to other conditional branches may be included. . Further, the syntax table may be divided into a plurality of tables, or a plurality of syntax tables may be integrated. Moreover, the term of each illustrated syntax element can be changed arbitrarily.

  Moreover, the modification of 1st Embodiment is easily applicable to 2nd Embodiment. In this case, the maximum number of reference images corresponding to list 0 and list 1 may be set, or may be a predetermined maximum number.

  Further, the modification of the fourth embodiment can be easily applied to the fifth embodiment. In this case, the maximum number of reference images corresponding to list 0 and list 1 may be set, or may be a predetermined maximum number.

  In the first to sixth embodiments described above, an example is described in which a frame is divided into rectangular blocks having a size of 16 × 16 pixels, and encoding / decoding is sequentially performed from the upper left block to the lower right side of the screen (see FIG. See 3A). However, the encoding order and the decoding order are not limited to this example. For example, encoding and decoding may be performed in order from the lower right to the upper left, or encoding and decoding may be performed so as to draw a spiral from the center of the screen toward the screen end. Furthermore, encoding and decoding may be performed in order from the upper right to the lower left, or encoding and decoding may be performed so as to draw a spiral from the screen end toward the center of the screen. In this case, since the position of the adjacent pixel block that can be referred to changes depending on the encoding order, the position may be changed to a usable position as appropriate.

  In the first to sixth embodiments described above, the prediction target block size such as a 4 × 4 pixel block, an 8 × 8 pixel block, and a 16 × 16 pixel block has been described as an example, but the prediction target block is a uniform block. It does not have to be a shape. For example, the prediction target block size may be a 16 × 8 pixel block, an 8 × 16 pixel block, an 8 × 4 pixel block, a 4 × 8 pixel block, or the like. Moreover, it is not necessary to unify all the block sizes within one coding tree block, and a plurality of different block sizes may be mixed. When a plurality of different block sizes are mixed in one coding tree block, the code amount for encoding or decoding the division information increases as the number of divisions increases. Therefore, it is desirable to select the block size in consideration of the balance between the code amount of the division information and the quality of the locally decoded image or the decoded image.

  In the first to sixth embodiments, for the sake of simplification, a comprehensive description of the color signal component is described without distinguishing between the luminance signal and the prediction process for the color difference signal. However, when the prediction process is different between the luminance signal and the color difference signal, the same or different prediction methods may be used. If different prediction methods are used between the luminance signal and the color difference signal, the prediction method selected for the color difference signal can be encoded or decoded in the same manner as the luminance signal.

In the first to sixth embodiments, for the sake of simplification, the luminance signal and the weighted motion compensation prediction process for the chrominance signal are not distinguished from each other, and the comprehensive explanation is described regarding the color signal component. However, when the weighted motion compensation prediction process is different between the luminance signal and the color difference signal, the same or different weighted motion compensation prediction process may be used. If a weighted motion compensation prediction process that is different between the luminance signal and the color difference signal is used, the weighted motion compensation prediction process selected for the color difference signal can be encoded or decoded in the same manner as the luminance signal.

  In the first to sixth embodiments, syntax elements not defined in this embodiment can be inserted between the rows of the table shown in the syntax configuration, and other conditional branch descriptions are included. It does not matter. Alternatively, the syntax table can be divided and integrated into a plurality of tables. Moreover, it is not always necessary to use the same term, and it may be arbitrarily changed depending on the form to be used.

  As described above, each embodiment realizes a highly efficient weighted motion compensation prediction process while solving the problem of encoding redundant information when performing weighted motion compensation prediction. Therefore, according to each embodiment, the encoding efficiency is improved, and the subjective image quality is also improved.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  For example, it is possible to provide a program that realizes the processing of each of the above embodiments by storing it in a computer-readable storage medium. The storage medium may be a computer-readable storage medium such as a magnetic disk, optical disk (CD-ROM, CD-R, DVD, etc.), magneto-optical disk (MO, etc.), semiconductor memory, etc. For example, the storage format may be any form.

  Further, the program for realizing the processing of each of the above embodiments may be stored on a computer (server) connected to a network such as the Internet and downloaded to the computer (client) via the network.

100, 600, 700 Encoding device 101 Subtraction unit 102 Orthogonal transformation unit 103 Quantization unit 104 Inverse quantization unit 105 Inverse orthogonal transformation unit 106 Addition unit 107 Predictive image generation unit 108, 608 Index setting unit 109 Motion evaluation unit 110 Coding Unit 111 encoding control unit 201 multi-frame motion compensation unit 202 memory 203 unidirectional motion compensation unit 204 prediction parameter control unit 205 reference image selector 206 frame memory 207 reference image control unit 301 default motion compensation unit 302 weighted motion compensation unit 303 WP Parameter control unit 304, 305 WP selector 401, 901 Reference image confirmation unit 402, 602 Index generation unit 601, 1001 Reuse determination unit 701, 1101 Motion information memory 702, 1102 Motion information acquisition unit 703, 103 selector switch 800,1000,1100 decoder 801 decoding unit 802 inverse quantization unit 803 inverse orthogonal transform unit 804 adding unit 805 predicted image generation unit 806,1006 index setting unit 807 the decoding control unit 902,1002 WP parameter generator

Claims (2)

A slice level than called from the adaptation parameter set syntax of higher levels Resid Ntakusu, and spreads the weight table syntax that describes a plurality parameters including weighting coefficient specified by reference numbers,
A sheet Ntakusu level higher than the slice level, receives the encoded data comprising, a sequence parameter set syntax that describes a specific be cie Ntakusu elements the maximum number of the reference number possible values ,
To play the previous carboxymethyl Ntakusu element from the sequence parameter set syntax,
And a regenerated before carboxymethyl Ntakusu element and the spreads weight table syntax, specifies a plurality parameter by the reference number the maximum number as an upper limit,
A reference image is identified from the first reference image list and the second reference image list using the reference number ,
Performing a decoding process including weighted motion compensation using the identified reference image and the plurality of parameters;
Decrypt method. A slice level than called from the adaptation parameter set syntax of higher levels Resid Ntakusu, and spreads the weight table syntax that describes a plurality parameters including weighting coefficient specified by reference numbers,
A sheet Ntakusu level higher than the slice level, receives an input of coded data including a sequence parameter set syntax that describes a specific be cie Ntakusu elements the maximum number of the reference number possible values An input section;
A reproducing unit for reproducing the pre-carboxymethyl Ntakusu element from the sequence parameter set syntax,
And a regenerated before carboxymethyl Ntakusu element and the spreads weight table syntax, specifies a plurality parameter by the reference number the maximum number as an upper limit, using the reference number, the first reference picture list and the second reference A specifying unit for specifying a reference image from the image list ;
A decoding unit that executes a decoding process including weighted motion compensation using the identified reference image and the plurality of parameters.
Decrypt apparatus.