JP7000498B2 - Storage device, transmitter and coding method - Google Patents

Description

Embodiments of the present invention relate to a data structure of coded data, a storage device, a transmission device, and a coding method.

In recent years, an image coding method with significantly improved coding efficiency has been developed in collaboration with ITU-T (International Telecommunication Union Telecommunication Standardization Sector) and ISO (International Organization for Standardization) / IEC (International Electrotechnical Commission). TREC. H. It is recommended as 264 and ISO / IEC 14496-10 (hereinafter referred to as "H.264").

H. 264 discloses an inter-prediction coding method that eliminates time-direction redundancy and realizes high coding efficiency by performing fractionally accurate motion compensation prediction using a coded image as a reference image. ing.

In addition, a method for encoding moving images including fade and dissolve effects with higher efficiency than the inter-prediction coding method in ISO / IEC MPEG (Moving Picture Experts Group) -1, 2, and 4 has been proposed. In this method, as a framework for predicting a change in brightness in the time direction, motion compensation prediction with fractional accuracy is performed for an input moving image having luminance and two color differences. Then, the predicted image is multiplied by the weighting coefficient and the offset is added by using the index showing the combination including the reference image, the luminance and the weighting coefficient for each of the two color differences, and the offset for each of the luminance and the two color differences.

Japanese Unexamined Patent Publication No. 2004-7377

However, in the conventional technique as described above, since the index is coded as a direct value, the coding efficiency is lowered. An object to be solved by the present invention is to provide an electronic device, a decoding method and a program capable of improving the coding efficiency.

The data structure of the coded data of the embodiment is used in an electronic device including a hardware circuit and a storage unit, and is stored in the storage unit, and is used in a process of decoding a moving image by the hardware circuit. The first syntax, which is a data structure and includes data indicating the validity or invalidity of the weighted motion compensation prediction, and the data indicating the validity or invalidity of the weighted motion compensation prediction indicate the validity of the weighted motion compensation prediction. In this case, the second syntax includes the second syntax read from the storage unit by the hardware circuit, and the second syntax includes data in which the first fixed-point precision of the weighting coefficient of the brightness is encoded and the first fixed-point precision. The data is data in which the first difference value is encoded, which is the same value as the difference between the accuracy and the second fixed-point precision of the weight coefficient of the color difference, and the first fixed-point accuracy and the first difference value are By adding, the data used for the process of determining the second fixed-point precision and the same value as the value obtained by shifting "1" to the left by the number of bits determined by the first fixed-point precision. The difference between the first reference value, which is the same value as the value of the weight coefficient of the brightness when there is no change in the pixel value in the brightness between the reference image and the target image, and the weight coefficient of the brightness. The second difference value of the brightness weighting coefficient, which is the same value, is encoded data, and the first reference value and the second difference value are added to obtain the brightness weighting coefficient. The data used for the determination process and the value obtained by shifting "1" to the left by the number of bits determined by the second fixed-point precision, and the reference image and the target image are the same. The third difference value, which is the difference between the second reference value, which is the same value as the weight coefficient value of the color difference when there is no change in the pixel value due to the color difference between the two, is encoded data. The data used in the process of determining the weighting coefficient of the color difference by adding the difference value and the second reference value is included.

The block diagram which shows the example of the coding apparatus of 1st Embodiment. The explanatory view which shows the predicted coding order example of the pixel block in 1st Embodiment. The figure which shows the block size example of the coding tree block in 1st Embodiment. The figure which shows the specific example of the coding tree block of 1st Embodiment. The figure which shows the specific example of the coding tree block of 1st Embodiment. The figure which shows the specific example of the coding tree block of 1st Embodiment. The block diagram which shows the example of the prediction image generation part of 1st Embodiment. The figure which shows the example of the relationship of the motion vector of the motion compensation prediction in the bidirectional prediction of the first embodiment. The block diagram which shows the example of the multi-frame motion compensation part of 1st Embodiment. The explanatory view of the example of the fixed-point precision of the weighting factor in 1st Embodiment. The figure which shows the example of WP parameter information of 1st Embodiment. The figure which shows the example of WP parameter information of 1st Embodiment. The figure which shows the example of the syntax of 1st Embodiment. The figure which shows the example of the picture parameter set syntax of 1st Embodiment. The figure which shows the example of the slice header syntax of 1st Embodiment. The figure which shows the example of the pred weight table syntax of 1st Embodiment. The figure which shows the example of the syntax composition which explicitly showed the prediction method of 1st Embodiment. The flowchart which shows the prediction processing example of the fixed-point accuracy of 1st Embodiment. The flowchart which shows the restoration processing example of the fixed point precision of 1st Embodiment. The flowchart which shows the prediction processing example of the weighting coefficient of 1st Embodiment. The flowchart which shows the restoration processing example of the weighting coefficient of 1st Embodiment. The flowchart which shows the other example of the prediction processing of the weighting coefficient of 1st Embodiment. The flowchart which shows the other example of the restoration process of the weighting coefficient of 1st Embodiment. The flowchart which shows the prediction processing example of the color difference signal of 1st Embodiment. The flowchart which shows the restoration processing example of the color difference signal of 1st Embodiment. The flowchart which shows the other example of the prediction processing example of the weighting coefficient of 1st Embodiment. The flowchart which shows the other example of the restoration processing example of the weighting coefficient of 1st Embodiment. The block diagram which shows the structural example of the decoding apparatus of 2nd Embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The coding device and decoding device of each of the following embodiments can be realized by hardware such as an LSI (Large-Scale Integration) chip, a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array). Further, the coding device and the decoding device of each of the following embodiments can be realized by having a computer execute a program, that is, by software. In the following description, the term "image" can be appropriately read as a term such as "video", "pixel", "image signal", "picture", or "image data".

(First Embodiment)
In the first embodiment, a coding device for coding a moving image will be described.

FIG. 1 is a block diagram showing an example of the configuration of the coding device 100 of the first embodiment.

The coding device 100 divides each frame or each field constituting the input image into a plurality of pixel blocks, and uses a coding parameter input from the coding control unit 111 to predict a code for the divided pixel blocks. And generate a predicted image. Then, the coding apparatus 100 generates a prediction error by subtracting the input image divided into a plurality of pixel blocks and the prediction image, orthogonally transforms and quantifies the generated prediction error, and further performs entropy coding for coding. Generate and output data.

The coding device 100 selectively applies a plurality of prediction modes in which at least one of the block size of the pixel block and the method of generating the prediction image is different to perform prediction coding. There are roughly two types of prediction image generation methods: intra-prediction, which makes predictions within a coded frame, and inter-prediction, which makes motion compensation predictions using one or more reference frames that differ in time. Intra prediction is also referred to as in-screen prediction or in-frame prediction, and inter-screen prediction is also referred to as inter-screen prediction, inter-frame prediction, motion compensation prediction, or the like.

FIG. 2 is an explanatory diagram showing an example of the predicted coding order of the pixel blocks in the first embodiment. In the example shown in FIG. 2, the coding apparatus 100 performs predictive coding from the upper left to the lower right of the pixel block, and in the frame f to be coded, the left side of the pixel block c to be coded and the left side of the pixel block c. The coded pixel block p is located on the upper side. In the following, for the sake of simplicity of description, the coding apparatus 100 shall perform predictive coding in the order shown in FIG. 2, but the order of predictive coding is not limited to this.

The pixel block indicates a unit for processing an image, and corresponds to, for example, an M × N size block (M and N are natural numbers), a coding tree block, a macro block, a sub block, or one pixel. In the following description, the pixel block is basically used in the meaning of the coding tree block, but it may be used in other meanings. For example, in the description of the prediction unit, the pixel block is used to mean the pixel block of the prediction unit. In addition, the block may be called by a name such as a unit. For example, a coding block is called a coding unit.

FIG. 3A is a diagram showing an example of the block size of the coding tree block in the first embodiment. The coding tree block is typically a 64x64 pixel block as shown in FIG. 3A. However, the present invention is not limited to this, and may be a 32 × 32 pixel block, a 16 × 16 pixel block, an 8 × 8 pixel block, a 4 × 4 pixel block, or the like. Further, the coding tree block does not have to be a square, and may be, for example, a pixel block of M × N size (M ≠ N).

3B to 3D are diagrams showing specific examples of the coding tree block of the first embodiment. FIG. 3B shows a coding tree block with a block size of 64 × 64 (N = 32). N represents the size of the reference coding tree block, and the size when divided is N and the size when not divided is 2N. FIG. 3C shows a coding tree block obtained by dividing the coding tree block of FIG. 3B into quadtrees. The coding tree block has a quadtree structure, as shown in FIG. 3C. When the coding tree block is divided, the four pixel blocks after the division are numbered in the Z scan order as shown in FIG. 3C.

The coding tree block can be further divided into quadtrees within the number of one quadtree. This makes it possible to divide the coding tree block hierarchically. In this case, the depth of division is defined by Depth. FIG. 3D shows one of the coding tree blocks obtained by dividing the coding tree block of FIG. 3B into quadtrees, and the block size is 32 × 32 (N = 16). The Depth of the coding tree block shown in FIG. 3B is 0, and the Depth of the coding tree block shown in FIG. 3D is 1. The coding tree block having the largest unit is called a large coding tree block, and the input image signal is encoded in the raster scan order in this unit.

In the following description, the coding target block or coding tree block of the input image may be referred to as a prediction target block or a prediction pixel block. The coding unit is not limited to the pixel block, and at least one of a frame, a field, a slice, a line, and a pixel can be used.

As shown in FIG. 1, the coding apparatus 100 predicts that the subtraction unit 101, the orthogonal transformation unit 102, the quantization unit 103, the inverse quantization unit 104, the inverse orthogonal transformation unit 105, and the addition unit 106. An image generation unit 107, an index setting unit 108, a motion evaluation unit 109, and a coding unit 110 are provided. The coding control unit 111 shown in FIG. 1 controls the coding device 100, and can be realized by, for example, a CPU (Central Processing Unit).

The subtraction unit 101 subtracts the corresponding prediction image from the input image divided into pixel blocks to obtain a prediction error. The subtraction unit 101 outputs a prediction error and inputs it to the orthogonal transformation unit 102.

The orthogonal transformation unit 102 performs an orthogonal transformation such as a discrete cosine transform (DCT) or a discrete sine transform (DST) with respect to the prediction error input from the subtraction unit 101 to obtain a conversion coefficient. The orthogonal transformation unit 102 outputs the conversion coefficient and inputs it to the quantization unit 103.

The quantization unit 103 performs a quantization process on the conversion coefficient input from the orthogonal transformation unit 102 to obtain a quantization conversion coefficient. Specifically, the quantization unit 103 performs the quantization according to the quantization information such as the quantization parameter and the quantization matrix specified by the coding control unit 111. More specifically, the quantization unit 103 divides the conversion coefficient by the quantization step size derived from the quantization information to obtain the quantization conversion coefficient. The quantization parameter indicates the fineness of the quantization. The quantization matrix is used to weight the fineness of the quantization for each component of the conversion factor. The quantization unit 103 outputs the quantization conversion coefficient and inputs it to the inverse quantization unit 104 and the coding unit 110.

The inverse quantization unit 104 performs an inverse quantization process on the quantization conversion coefficient input from the quantization unit 103, and obtains a restoration conversion coefficient. Specifically, the inverse quantization unit 104 performs inverse quantization according to the quantization information used in the quantization unit 103. More specifically, the inverse quantization unit 104 multiplies the quantization step size derived by the quantization information by the quantization conversion coefficient to obtain the restoration conversion coefficient. The quantization information used in the quantization unit 103 is loaded from an internal memory (not shown) of the coding control unit 111 and used. The inverse quantization unit 104 outputs the restoration conversion coefficient and inputs it to the inverse orthogonal transformation unit 105.

The inverse orthogonal transform unit 105 performs an inverse orthogonal transform such as, for example, an inverse discrete cosine transform (IDCT) or an inverse discrete sine transform (IDST) with respect to the restoration transform coefficient input from the inverse quantize unit 104. Obtain the restoration prediction error. The inverse orthogonal transformation performed by the inverse orthogonal transformation unit 105 corresponds to the orthogonal transformation performed by the orthogonal transformation unit 102. The inverse orthogonal transformation unit 105 outputs the restoration prediction error and inputs it to the addition unit 106.

The addition unit 106 adds the restoration prediction error input from the inverse orthogonal transformation unit 105 and the corresponding prediction image to generate a locally decoded image. The addition unit 106 outputs a locally decoded image and inputs it to the prediction image generation unit 107.

The prediction image generation unit 107 stores the locally decoded image input from the addition unit 106 as a reference image in a memory (not shown in FIG. 1), outputs the reference image stored in the memory, and inputs the reference image to the motion evaluation unit 109. .. Further, the prediction image generation unit 107 performs weighted motion compensation prediction based on the motion information and WP parameter information input from the motion evaluation unit 109, and generates a prediction image. The prediction image generation unit 107 outputs a prediction image and inputs it to the subtraction unit 101 and the addition unit 106.

FIG. 4 is a block diagram showing an example of the configuration of the predicted image generation unit 107 of the first embodiment. As shown in FIG. 4, the prediction image generation unit 107 includes a plurality of frame motion compensation unit 201, a memory 202, a unidirectional motion compensation unit 203, a prediction parameter control unit 204, a reference image selector 205, and a frame memory 206. And a reference image control unit 207.

The frame memory 206 stores the locally decoded image input from the addition unit 106 as the reference image under the control of the reference image control unit 207. The frame memory 206 has a plurality of memory sets FM1 to FMN (N ≧ 2) for temporarily holding the reference image.

The prediction parameter control unit 204 prepares a plurality of combinations of the reference image number and the prediction parameter as a table based on the motion information input from the motion evaluation unit 109. Here, the motion information refers to a motion vector indicating the amount of motion deviation used in motion compensation prediction, a reference image number, information on a prediction mode such as unidirectional / bidirectional prediction, and the like. Prediction parameters refer to information about motion vectors and prediction modes. Then, the prediction parameter control unit 204 selects and outputs a combination of the reference image number and the prediction parameter used for generating the prediction image based on the input image, inputs the reference image number to the reference image selector 205, and inputs the reference image number to the prediction parameter. Is input to the unidirectional motion compensation unit 203.

The reference image selector 205 is a switch for switching which output terminal of the frame memories FM1 to FMN of the frame memory 206 is connected according to the reference image number input from the prediction parameter control unit 204. For example, if the reference image number is 0, the reference image selector 205 connects the output end of the FM1 to the output end of the reference image selector 205, and if the reference image number is N-1, the output end of the FMN is referred to. Connect to the output end of the image selector 205. The reference image selector 205 outputs a reference image stored in the frame memory to which the output end is connected among the frame memories FM1 to FMN of the frame memory 206, and outputs the unidirectional motion compensation unit 203 and the motion evaluation unit 109. Enter in.

The unidirectional prediction motion compensation unit 203 performs motion compensation prediction processing according to the prediction parameter input from the prediction parameter control unit 204 and the reference image input from the reference image selector 205, and generates a unidirectional prediction image.

FIG. 5 is a diagram showing an example of the relationship between motion vectors of motion compensation prediction in the bidirectional prediction of the first embodiment. In motion compensation prediction, interpolation processing is performed using the reference image, and a unidirectional prediction image is generated based on the amount of motion deviation from the pixel block of the coded target position between the created interpolated image and the input image. .. Here, the amount of deviation is a motion vector. As shown in FIG. 5, in a bidirectional predicted slice (B-slice), a predicted image is generated using two types of reference images and a set of motion vectors. As the interpolation process, an interpolation process having a 1/2 pixel accuracy, an interpolation process having a 1/4 pixel accuracy, or the like is used, and the value of the interpolated image is generated by performing the filtering process on the reference image. For example, H.I. In 264, the amount of deviation is expressed by four times the precision of the integer pixel.

The unidirectional prediction motion compensation unit 203 outputs the unidirectional prediction image and temporarily stores it in the memory 202. Here, when the motion information (prediction parameter) indicates bidirectional prediction, the multi-frame motion compensation unit 201 performs weighted prediction using two types of unidirectional prediction images, so that the unidirectional prediction motion compensation unit 203 Stores the first unidirectional predicted image in the memory 202, and directly outputs the second unidirectional predicted image to the multi-frame motion compensating unit 201. Here, the unidirectional prediction image corresponding to the first is referred to as a first prediction image, and the unidirectional prediction image corresponding to the second is referred to as a second prediction image.

It should be noted that two unidirectional motion compensation units 203 may be prepared, and each may generate two unidirectional prediction images. In this case, when the motion information (prediction parameter) indicates unidirectional prediction, the unidirectional motion compensation unit 203 may directly output the first unidirectional prediction image as the first prediction image to the multi-frame motion compensation unit 201. good.

The multi-frame motion compensation unit 201 uses the first prediction image input from the memory 202, the second prediction image input from the unidirectional prediction motion compensation unit 203, and the WP parameter information input from the motion evaluation unit 109. , Performs weighted prediction to generate a prediction image. The multi-frame motion compensation unit 201 outputs a predicted image and inputs it to the subtraction unit 101 and the addition unit 106.

FIG. 6 is a block diagram showing an example of the configuration of the multi-frame motion compensation unit 201 of the first embodiment. As shown in FIG. 6, the multi-frame motion compensation unit 201 includes a default motion compensation unit 301, a weighted motion compensation unit 302, a WP parameter control unit 303, and WP selectors 304 and 305.

The WP parameter control unit 303 outputs the WP application flag and weight information based on the WP parameter information input from the motion evaluation unit 109, inputs the WP application flag to the WP selectors 304 and 305, and weights the weight information. Input to the motion compensation unit 302.

Here, the WP parameter information includes the fixed-point accuracy of the weighting factor, the first WP application flag corresponding to the first predicted image, the first weighting factor, and the first offset, and the second WP adaptation corresponding to the second predicted image. Contains information on flags, second weighting factors, and second offset. The WP application flag is a parameter that can be set for each applicable reference image and signal component, and indicates whether or not to perform weighted motion compensation prediction. The weighting information includes information on the fixed-point accuracy of the weighting factors, the first weighting factor, the first offset, the second weighting factor, and the second offset.

Specifically, when the WP parameter information is input from the motion evaluation unit 109, the WP parameter control unit 303 separates and outputs the WP parameter information into the first WP application flag, the second WP application flag, and the weight information. , The first WP application flag is input to the WP selector 304, the second WP application flag is input to the WP selector 305, and the weight information is input to the weighted motion compensation unit 302.

The WP selectors 304 and 305 switch the connection end of each predicted image based on the WP application flag input from the WP parameter control unit 303. When each WP application flag is 0, the WP selectors 304 and 305 connect their output ends to the default motion compensation unit 301. Then, the WP selectors 304 and 305 output the first predicted image and the second predicted image and input them to the default motion compensation unit 301. On the other hand, the WP selectors 304 and 305 connect their respective output ends to the weighted motion compensation unit 302 when each WP application flag is 1. Then, the WP selectors 304 and 305 output the first predicted image and the second predicted image and input them to the weighted motion compensation unit 302.

The default motion compensation unit 301 performs average value processing based on the two unidirectional predicted images (first predicted image and second predicted image) input from the WP selectors 304 and 305, and generates a predicted image. Specifically, when the first WP application flag and the second WP application flag are 0, the default motion compensation unit 301 performs the average value processing based on the mathematical formula (1).

P [x, y] = Clip1 ((PL0 [x, y] + PL1 [x, y] + offset2) >> (shift2)) ... (1)

Here, P [x, y] is a predicted image, PL0 [x, y] is a first predicted image, and PL1 [x, y] is a second predicted image. Offset2 and shift2 are parameters of the rounding process in the mean value process, and are determined by the internal calculation accuracy of the first predicted image and the second predicted image. Assuming that the bit precision of the predicted image is L and the bit precision of the first predicted image and the second predicted image is M (L ≦ M), shift2 is formulated by the mathematical formula (2) and offset2 is formulated by the mathematical formula (3). Will be done.

shift2 = (ML + 1) ... (2)

offset2 = (1 << (shift2-1) ... (3)

For example, when the bit precision of the predicted image is 8 and the bit precision of the first predicted image and the second predicted image is 14, shift2 = 7 from the formula (2) and offset2 = (1 << from the formula (3). 6).

When the prediction mode indicated by the motion information (prediction parameter) is unidirectional prediction, the default motion compensation unit 301 uses only the first prediction image and obtains the final prediction image based on the mathematical formula (4). calculate.

P [x, y] = Clip1 ((PLX [x, y] + office1) >> (shift1)) ... (4)

Here, PLX [x, y] indicates a unidirectional predicted image (first predicted image), and X is an identifier indicating either 0 or 1 in the reference list. For example, when the reference list is 0, it is PL0 [x, y], and when the reference list is 1, it is PL1 [x, y]. Offset1 and shift1 are parameters of the rounding process and are determined by the internal calculation accuracy of the first predicted image. Assuming that the bit precision of the predicted image is L and the bit precision of the first predicted image is M, shift1 is formulated by the mathematical formula (5), and offset1 is formulated by the mathematical formula (6).

shift1 = (ML) ... (5)

offset1 = (1 << (shift1-1) ... (6)

For example, when the bit precision of the predicted image is 8 and the bit precision of the first predicted image is 14, shift1 = 6 from the mathematical formula (5) and offset1 = (1 << 5) from the mathematical formula (6).

The weighted motion compensation unit 302 is based on the two unidirectional prediction images (first prediction image and second prediction image) input from the WP selectors 304 and 305 and the weight information input from the WP parameter control unit 303. Perform weighted motion compensation. Specifically, when the first WP application flag and the second WP application flag are 1, the weighted motion compensation unit 302 performs weighting processing based on the mathematical formula (7).

P [x, y] = Clip1 (((PL0 [x, y] * w0C + PL1 [x, y] * w1C + (1 << logWDC)) >> (logWDC + 1)) + ((o0C + o1C + 1) >> 1)) ... (7)

Here, w0C represents a weighting coefficient corresponding to the first predicted image, w1C represents a weighting coefficient corresponding to the second predicted image, o0C represents an offset corresponding to the first predicted image, and o1C represents an offset corresponding to the second predicted image. .. Hereinafter, they are referred to as a first weighting coefficient, a second weighting coefficient, a first offset, and a second offset, respectively. logWDC is a parameter indicating the fixed-point accuracy of each weighting factor. The variable C means a signal component. For example, in the case of a YUV spatial signal, the luminance signal is represented as C = Y, the Cr color difference signal is represented as C = Cr, and the Cb color difference component is represented as C = Cb.

When the calculated accuracy of the first predicted image and the second predicted image is different from that of the predicted image, the weighted motion compensating unit 302 rounds the logWDC, which is the fixed-point accuracy, by controlling it as in the mathematical formula (8). To realize.

logWD'C = logWDC + offset1 ... (8)

The rounding process can be realized by replacing the logWDC of the mathematical formula (7) with the logWD'C of the mathematical formula (8). For example, when the bit accuracy of the predicted image is 8 and the bit precision of the first predicted image and the second predicted image is 14, by resetting the logWDC, the calculation accuracy is the same as that of shift2 of the mathematical formula (1). It is possible to realize batch rounding processing.

When the prediction mode indicated by the motion information (prediction parameter) is unidirectional prediction, the weighted motion compensation unit 302 uses only the first prediction image and the final prediction image based on the mathematical formula (9). Is calculated.

P [x, y] = Clip1 ((PLX [x, y] * wXC + (1 << logWDC-1)) >> (logWDC)) ... (9)

Here, PLX [x, y] indicates a unidirectional prediction image (first prediction image), wXC indicates a weighting coefficient corresponding to unidirectional prediction, and X indicates either 0 or 1 in the reference list. It is an identifier to indicate. For example, when the reference list is 0, it is PL0 [x, y], w0C, and when the reference list is 1, it is PL1 [x, y], w1C.

In addition, when the calculation accuracy of the first predicted image and the second predicted image is different from that of the predicted image, the weighted motion compensating unit 302 uses the fixed-point accuracy logWDC as in the mathematical formula (8) as in the case of bidirectional prediction. The rounding process is realized by controlling to.

The rounding process can be realized by replacing the logWDC of the mathematical formula (7) with the logWD'C of the mathematical formula (8). For example, when the bit precision of the predicted image is 8 and the bit precision of the first predicted image is 14, by resetting the logWDC, a batch rounding process with the same calculation precision as shift 1 of the formula (4) is realized. It becomes possible to do.

FIG. 7 is an explanatory diagram of an example of the fixed-point accuracy of the weighting coefficient in the first embodiment, and is a diagram showing an example of a change between a moving image having a change in brightness in the time direction and a gradation value. In the example shown in FIG. 7, the frame to be encoded is Frame (t), the frame immediately before in time is Frame (t-1), and the frame one after in time is Frame (t + 1). As shown in FIG. 7, in a fade image that changes from white to black, the brightness (gradation value) of the image decreases with time. The weighting coefficient means the degree of change in FIG. 7, and as is clear from the formulas (7) and (9), it takes a value of 1.0 when there is no change in lightness. The fixed-point precision is a parameter that controls the step size corresponding to the decimal point of the weighting factor, and the weighting factor when there is no change in brightness is 1 << logWDC.

In the case of unidirectional prediction, various parameters (second WP adaptation flag, second weighting coefficient, and second offset information) corresponding to the second prediction image are not used, so they are set to predetermined initial values. It may have been done.

Returning to FIG. 1, the motion evaluation unit 109 performs motion evaluation between a plurality of frames based on the input image and the reference image input from the predicted image generation unit 107, outputs motion information and WP parameter information, and outputs motion information. It is input to the prediction image generation unit 107 and the coding unit 110, and the WP parameter information is input to the prediction image generation unit 107 and the index setting unit 108.

The motion evaluation unit 109 calculates an error by calculating a difference value starting from a plurality of reference images corresponding to the same position as the input image of the prediction target pixel block, shifts this position with fractional accuracy, and minimizes the error. Optimal motion information is calculated by a method such as block matching. In the case of bidirectional prediction, the motion evaluation unit 109 uses the motion information derived by the unidirectional prediction to perform block matching including the default motion compensation prediction as shown in the mathematical formulas (1) and (4). By doing so, the motion information of the two-way prediction is calculated.

At this time, the motion evaluation unit 109 can calculate the WP parameter information by performing block matching including the weighted motion compensation prediction as shown by the mathematical expressions (7) and (9). In addition, in the calculation of the WP parameter information, a method of calculating the weighting coefficient and the offset using the brightness gradient of the input image, a method of calculating the weighting coefficient and the offset by accumulating the prediction error at the time of encoding, etc. may be used. good. Further, as the WP parameter information, a fixed value predetermined for each coding device may be used.

Here, with reference to FIG. 7, a method of calculating the weighting coefficient, the fixed-point accuracy of the weighting coefficient, and the offset from the moving image whose brightness changes with time will be described. As described above, in a fade image that changes from white to black as shown in FIG. 7, the brightness (gradation value) of the image decreases with time. The motion evaluation unit 109 can calculate the weighting coefficient by calculating this inclination.

Further, the fixed-point accuracy of the weighting coefficient is information indicating the accuracy of this inclination, and the motion evaluation unit 109 can calculate an optimum value from the temporal distance of the reference image and the degree of change in the image brightness. For example, in FIG. 7, when the weighting coefficient between Frame (t-1) and Frame (t + 1) is 0.75 with decimal point accuracy, 3/4 can be expressed with 1/4 accuracy, so motion evaluation is performed. The unit 109 sets the fixed-point precision to 2 (1 << 2). Since the value of fixed-point precision affects the code amount when the weighting factor is encoded, the optimum value may be selected in consideration of the code amount and the prediction accuracy. The fixed-point precision value may be a predetermined fixed value.

Further, when the slopes do not match, the motion evaluation unit 109 can calculate the offset value by obtaining the correction value (deviation amount) corresponding to the intercept of the linear function. For example, in FIG. 7, when the weighting factor between Frame (t-1) and Frame (t + 1) is 0.60 with a decimal point precision and the fixed-point precision is 1 (1 << 1), the weighting factor is 1. (That is, the decimal point precision of the weighting factor corresponds to 0.50) is likely to be set. In this case, since the decimal point accuracy of the weighting factor deviates from the optimum value of 0.60 by 0.10, the motion evaluation unit 109 calculates the correction value for this amount from the maximum value of the pixel and uses it as an offset value. Set. When the maximum value of the pixel is 255, the motion evaluation unit 109 may set a value such as 25 (255 × 0.1).

In the first embodiment, the motion evaluation unit 109 is illustrated as one function of the coding device 100, but the motion evaluation unit 109 is not an essential configuration of the coding device 100, and for example, the motion evaluation unit 109 is coded. It may be a device outside the conversion device 100. In this case, the motion information and the WP parameter information calculated by the motion evaluation unit 109 may be loaded into the coding device 100.

The index setting unit 108 receives the WP parameter information input from the motion evaluation unit 109, confirms the reference list (list number) and the reference image (reference number), outputs the index information, and outputs the index information to the coding unit 110. input. The index setting unit 108 maps the WP parameter information input from the motion evaluation unit 109 to a syntax element described later to generate index information.

8A and 8B are diagrams showing an example of WP parameter information of the first embodiment. An example of WP parameter information at the time of P-slice is as shown in FIG. 8A, and an example of WP parameter information at the time of B-slice is as shown in FIGS. 8A and 8B. The list number is an identifier indicating the prediction direction, and takes a value of 0 at the time of unidirectional prediction, and takes two values of 0 and 1 at the time of bidirectional prediction because two types of prediction can be used. The reference number is a value corresponding to 1 to N shown in the frame memory 206. Since the WP parameter information is held for each reference list and reference image, the information required at the time of B-slice is 2N when the number of reference images is N.

Returning to FIG. 1, the coding unit 110 includes a quantization conversion coefficient input from the quantization unit 103, motion information input from the motion evaluation unit 109, index information input from the index setting unit 108, and coding control. Coding processing is performed on various coding parameters such as the quantization information specified by the unit 111, and the coding data is generated. The coding process corresponds to, for example, Huffman coding or arithmetic coding.

The coding parameter is a parameter required for decoding such as prediction information indicating a prediction method, information on a quantization conversion coefficient, and information on quantization. For example, the coding control unit 111 has an internal memory (not shown), the coding parameters are held in the internal memory, and when coding the pixel block, the coding parameters of the adjacent already coded pixel blocks are used. Can be done. For example, H. In the intra-prediction of 264, the prediction information of the pixel block can be derived from the prediction information of the encoded adjacent block.

The coding unit 110 outputs the generated coded data according to an appropriate output timing managed by the coded control unit 111. In the output coded data, for example, various information is multiplexed by a multiplexing unit (not shown) and temporarily stored in an output buffer (not shown), and then, for example, a storage system (not shown) ( It is output to the storage media) or the transmission system (communication line).

The coding unit 110 includes an entropy coding unit 110A and an index reconstruction unit 110B.

The entropy coding unit 110A performs coding processing such as variable length coding and arithmetic coding on the input information. For example, H. In 264, context-adaptive variable-length coding (CAVLC) and context-adaptive arithmetic coding (CABAC: Adapt-based Adaptive Binary Coding) are used.

In order to reduce the code length of the syntax element of the index information input from the index setting unit 108, the index reconstruction unit 110B performs prediction processing according to the characteristics of the parameter of the syntax element, and the value of the syntax element as it is (direct). The value) and the difference value of the predicted value are calculated and output to the entropy coding unit 110A. Specific examples of the prediction process will be described later.

FIG. 9 is a diagram showing an example of the syntax 500 used by the coding device 100 of the first embodiment. The syntax 500 shows the structure of the coded data generated by coding the input image (moving image data) by the coding device 100. When decoding the coded data, the decoding device described later performs the syntax interpretation of the moving image with reference to the same syntax structure as the syntax 500.

The syntax 500 includes three parts: high level syntax 501, slice level syntax 502 and coding tree level syntax 503. High-level syntax 501 contains syntax information for layers above the slice. Slice refers to a rectangular or continuous area contained in a frame or field. The slice level syntax 502 contains the information required to decode each slice. The coding tree level syntax 503 contains the information necessary to decode each coding tree (ie, each coding tree block). Each of these parts contains more detailed syntax.

High-level syntax 501 includes sequence and picture-level syntax such as sequence parameter set syntax 504, picture parameter set syntax 505, and adaptation parameter set syntax 506.

The slice level syntax 502 includes a slice header syntax 507, a pred weight table syntax 508, a slice data syntax 509, and the like. The pred weight table syntax 508 is called from the slice header syntax 507.

The coding tree level syntax 503 includes a coding tree unit syntax 510, a transform unit syntax 511, a prediction unit syntax 512, and the like. The coding tree unit Syntax 510 can have a quadtree structure. Specifically, the coding tree unit syntax 510 can be recursively called as a syntax element of the coding tree unit syntax 510. That is, one coding tree block can be subdivided into quadtrees. Further, the transform unit syntax 511 is included in the coding tree unit syntax 510. The transform unit syntax 511 is called in each coding tree unit syntax 510 at the end of the quadtree. The transform unit syntax 511 describes information related to inverse orthogonal transformation, quantization, and the like. Information about weighted motion compensation predictions may be described in these syntaxes.

FIG. 10 is a diagram showing an example of the picture parameter set syntax 505 of the first embodiment. The weighted_pred_flag is, for example, a syntax element indicating the validity or invalidity of the weighted compensation prediction of the first embodiment regarding P-slice. When weighted_pred_flag is 0, the weighted motion compensation prediction of the first embodiment in P-slice becomes invalid. Therefore, the WP application flag included in the WP parameter information is always set to 0, and the WP selectors 304 and 305 connect their respective output ends to the default motion compensation unit 301. On the other hand, when weighted_pred_flag is 1, the weighted motion compensation prediction of the first embodiment in P-slice is valid.

As another example, when weighted_pred_flag is 1, in the syntax of lower layers (slice header, coding tree block, transform unit, prediction unit, etc.), the first is for each local area inside the slice. One embodiment may specify the validity or invalidity of the weighted motion compensation prediction.

The weighted_bipred_idc is, for example, a syntax element indicating the validity or invalidity of the weighted compensation prediction of the first embodiment regarding B-slice. When weighted_bipred_idc is 0, the weighted motion compensation prediction of the first embodiment in B-slice becomes invalid. Therefore, the WP application flag included in the WP parameter information is always set to 0, and the WP selectors 304 and 305 connect their respective output ends to the default motion compensation unit 301. On the other hand, when weighted_bipred_idc is 1, the weighted motion compensation prediction of the first embodiment in B-slice is valid.

As another example, when weighted_bipred_idc is 1, in the syntax of lower layers (slice header, coding tree block, transform unit, etc.), for each local region inside the slice, the first embodiment The validity or invalidity of the weighted motion compensation prediction may be specified.

FIG. 11 is a diagram showing an example of the slice header syntax 507 of the first embodiment. slice_type indicates the slice type of the slice (I-slice, P-slice, B-slice, etc.). The pic_parameter_set_id is an identifier indicating which picture parameter set syntax 505 is to be referred to. num_ref_idx_active_override_flag is a flag indicating whether to update the number of valid reference images. If this flag is 1, num_ref_idx_l0_active_minus1 and num_ref_idx_l1_active_minus1 that define the number of reference images in the reference list can be used. pred_weight_table () is a function indicating the pred weight table syntax used for weighted motion compensation prediction, and this function is used when the above-mentioned weighted_pred_flag is 1 and P-slice, and when weighted_bipred_idc is 1 and B-slice. Called.

FIG. 12 is a diagram showing an example of the pred weight table syntax 508 of the first embodiment. luma_log2_weight_denom represents the fixed-point accuracy of the weighting factor of the luminance signal in the slice, and is a value corresponding to the logWDC of the equation (7) or the equation (9). chroma_log2_weight_denom represents the fixed-point accuracy of the weighting factor of the color difference signal in the slice, and is a value corresponding to the logWDC of the equation (7) or the equation (9). chroma_format_idc is an identifier representing a color space, and MONO_IDX is a value indicating a monochrome image. num_ref_common_active_minus1 shows the number of reference images in the common list in the slice minus one.

luma_weight_l0_flag and luma_weight_l1_flag indicate WP adaptation flags in the luminance signals corresponding to List 0 and List 1, respectively. When this flag is 1, the weighted motion compensation prediction of the luminance signal of the first embodiment is valid in the entire area of the slice. chroma_weight_l0_flag and chroma_weight_l1_flag indicate WP adaptation flags in the color difference signals corresponding to List 0 and List 1, respectively. When this flag is 1, the weighted motion compensation prediction of the color difference signal of the first embodiment is valid in the entire area of the slice. luma_weight_l0 [i] and luma_weight_l1 [i] are the i-th corresponding luminance signal weighting coefficients managed in List 0 and List 1, respectively. luma_offset_l0 [i] and luma_offset_l1 [i] are the offsets of the i-th corresponding luminance signals managed in List 0 and List 1, respectively. These are the values corresponding to w0C, w1C, o0C, and o1C of the formula (7) or the formula (9), respectively. However, C = Y.

chroma_weight_l0 [i] [j] and chroma_weight_l1 [i] [j] are the i-th corresponding color difference signal weighting coefficients managed in List 0 and List 1, respectively. chroma_offset_l0 [i] [j] and chroma_offset_l1 [i] [j] are the offsets of the i-th corresponding color difference signals managed in List 0 and List 1, respectively. These are the values corresponding to w0C, w1C, o0C, and o1C of the formula (7) or the formula (9), respectively. However, C = Cr or Cb. j indicates a component of the color difference component, and for example, in the case of a YUV4: 2: 0 signal, j = 0 is a Cr component and j = 1 is a Cb component.

Here, the details of the prediction method of each syntax element related to the weighted prediction in the syntax configuration will be described. The prediction of the syntax element is performed by the index reconstruction unit 110B. FIG. 13 is a diagram showing an example of a syntax configuration that explicitly shows the prediction method of the first embodiment. In the example shown in FIG. 13, the syntax elements into which the prediction is introduced are prefixed with delta, but these syntax configurations basically have the same components as the syntax configuration shown in FIG.

First, a prediction method between the signals of luma_log2_weight_denom and chroma_log2_weight_denom, which indicate the fixed-point accuracy of the weighting factor, will be described. The index reconstruction unit 110B performs prediction processing between the signals of luma_log2_weight_denom and chroma_log2_weight_denom using the formula (10), and performs restoration processing using the formula (11). Here, as shown in FIGS. 12 and 13, since luma_log2_weight_denom is defined first, chroma_log2_weight_denom is predicted from the value of luma_log2_weight_denom.

delta_chroma_log2_weight_denom = (chroma_log2_weight_denom --luma_log2_weight_denom)… (10)

chroma_log2_weight_denom = (luma_log2_weight_denom + delta_chroma_log2_weight_denom)… (11)

FIG. 14 is a flowchart showing an example of the prediction process of chroma_log2_weight_denom of the first embodiment.

First, the index reconstruction unit 110B derives the luma_log2_weight_denom set in the index information as a predicted value (step S101).

Subsequently, the index reconstruction unit 110B subtracts luma_log2_weight_denom from chroma_log2_weight_denom (step S102), and sets the difference value as delta_chroma_log2_weight_denom in the index information (step S103).

FIG. 15 is a flowchart showing an example of the restoration process of chroma_log2_weight_denom of the first embodiment.

First, the index reconstruction unit 110B derives luma_log2_weight_denom already set in the index information as a predicted value (step S201).

Subsequently, the index reconstruction unit 110B adds luma_log2_weight_denom to delta_chroma_log2_weight_denom (step S202), and sets the added value as chroma_log2_weight_denom in the index information (step S203).

Since the fade effect generally has few cases of different temporal changes depending on the color space, the fixed-point accuracy for each signal component has a strong correlation between the luminance component and the color difference component. Therefore, by making predictions in the color space in this way, the amount of information indicating fixed-point accuracy can be reduced.

Although the luminance component is subtracted from the luminance component in the mathematical formula (10), the luminance component may be subtracted from the luminance component. In this case, the formula (11) may be modified according to the formula (10).

Next, a method for predicting luma_weight_lx [i] and chroma_weight_lx [i] [j], which represent the weight coefficients of the luminance and color difference signals, respectively, will be described. Here, x is an identifier indicating 0 or 1. The values of luma_weight_lx [i] and chroma_weight_lx [i] [j] increase or decrease according to the values of luma_log2_weight_denom and chroma_log2_weight_denom, respectively. For example, if the value of luma_log2_weight_denom is 3, then luma_weight_lx [i] is (1 << 3) assuming that there is no change in brightness. On the other hand, when the value of luma_log2_weight_denom is 5, luma_weight_lx [i] is (1 << 5) assuming that there is no change in brightness.

Therefore, the index reconstruction unit 110B performs prediction processing using the weighting coefficient (default value) when there is no change in brightness. Specifically, the index reconstruction unit 110B performs the prediction process of luma_weight_lx [i] using the mathematical formulas (12) to (13), and performs the restoration process using the mathematical formula (14). Similarly, the index reconstruction unit 110B performs the prediction processing of chroma_weight_lx [i] using the mathematical formulas (15) to (16), and performs the restoration processing using the mathematical formula (17).

delta_luma_weight_lx [i] = (luma_weight_lx [i] --default_luma_weight_lx)… (12)

default_luma_weight_lx = (1 << luma_log2_weight_denom)… (13)

luma_weight_lx [i] = (default_luma_weight_lx + delta_luma_weight_lx [i])… (14)

delta_chroma_weight_lx [i] [j] = (chroma_weight_lx [i] [j] --default_chroma_weight_lx)… (15)

default_chroma_weight_lx = (1 << chroma_log2_weight_denom)… (16)

chroma_weight_lx [i] [j] = (default_chroma_weight_lx + delta_chroma_weight_lx [i] [j])… (17)

Here, default_luma_weight_lx and default_chroma_weight_lx are default values in which there is no change in brightness in the luminance component and the color difference component, respectively.

FIG. 16 is a flowchart showing an example of the prediction process of luma_weight_lx [i] of the first embodiment.

First, the index reconstruction unit 110B derives the luma_log2_weight_denom set in the index information (step S301), and calculates default_luma_weight_lx as a predicted value (step S302).

Subsequently, the index reconstruction unit 110B subtracts default_luma_weight_lx from luma_weight_lx [i] (step S303), and sets the difference value as delta_luma_weight_lx [i] in the index information (step S304).

The prediction process can be applied to luma_weight_lx [i] by repeating this process for the number of reference images.

FIG. 17 is a flowchart showing an example of the restoration process of luma_weight_lx [i] of the first embodiment.

First, the index reconstruction unit 110B derives delta_luma_weight_lx [i] already set in the index information (step S401), and calculates default_luma_weight_lx as a predicted value (step S402).

Subsequently, the index reconstruction unit 110B adds delta_luma_weight_lx [i] to default_luma_weight_lx (step S403), and sets the added value as luma_weight_lx [i] in the index information (step S404).

Although the flowchart for the luminance component is shown here, the prediction process and the restoration process can be similarly realized for the color difference component (chroma_weight_lx [i] [j]).

An image containing a fade effect is often faded at a specific fade change point, and the other images are normal natural images or images without a fade effect. In this case, the weighting factor is often taken when there is no change in brightness. Therefore, the sign amount of the weighting coefficient can be reduced by deriving the initial value when there is no change in brightness from the fixed-point accuracy and using it as the predicted value.

The predicted values of the luminance and color difference signal weighting coefficients (luma_weight_lx [i] and chroma_weight_lx [i] [j]) may be derived with different reference numbers or different POC numbers. In this case, assuming that the reference number closest to the coded slice is base_idx, the index reconstruction unit 110B performs the prediction process of luma_weight_lx [i] using the mathematical formula (18), and uses the mathematical formula (19). , Perform the restoration process. Similarly, the index reconstruction unit 110B performs the prediction process of chroma_weight_lx [i] [j] using the mathematical expression (20), and performs the restoration process using the mathematical expression (21).

delta_luma_weight_lx [i] = (luma_weight_lx [i] --luma_weight_lx [base_idx])… (18)

luma_weight_lx [i] = (delta_luma_weight_lx [i] + luma_weight_lx [base_idx])… (19)

delta_chroma_weight_lx [i] [j] = (chroma_weight_lx [i] [j] --chroma_weight_lx [base_idx] [j])… (20)

chroma_weight_lx [i] [j] = (delta_chroma_weight_lx [i] [j] + chroma_weight_lx [base_idx] [j])… (21)

Here, in the mathematical formulas (18) and (20), i ≠ base_idx. Since the weighting coefficient of the reference number indicated by base_idx cannot be used in the formulas (18) and (20), the formulas (12) to (13) and (15) to (16) may be used.

FIG. 18 is a flowchart showing another example of the prediction process of luma_weight_lx [i] of the first embodiment.

First, the index reconstruction unit 110B sets a baseidx indicating a reference reference number (step S501). Here, the value of baseidx is assumed to be 0.

Subsequently, the index reconstruction unit 110B derives luma_weight_lx [baseidx] as a predicted value from the index information based on the baseidx (step S502). The index information luma_weight_lx [baseidx] indicated by baseidx is, for example, encoded by a direct value without prediction.

Subsequently, the index reconstruction unit 110B subtracts luma_weight_lx [baseidx] from luma_weight_lx [i] (step S503), and sets the difference value as delta_luma_weight_lx [i] in the index information (step S504).

By repeating this process for the number of reference images, the prediction process can be applied to luma_weight_lx [i] other than baseidx.

FIG. 19 is a flowchart showing another example of the restoration process of luma_weight_lx [i] of the first embodiment.

First, the index reconstruction unit 110B sets a baseidx indicating a reference number as a reference (step S601). Here, the value of baseidx is assumed to be 0.

Subsequently, the index reconstruction unit 110B derives luma_weight_lx [baseidx] as a predicted value from the index information based on the baseidx (step S602). The luma_weight_lx [baseidx] of the index information indicated by baseidx is, for example, encoded or decoded by a direct value without prediction.

Subsequently, the index reconstruction unit 110B adds delta_luma_weight_lx [i] to luma_weight_lx [baseidx] (step S603), and sets the added value as luma_weight_lx [i] in the index information (step S604).

Although the flowchart for the luminance component is shown here, the prediction process and the restoration process can be similarly realized for the color difference component (chroma_weight_lx [i] [j]). Further, here, as an example, the prediction method and the restoration method of luma_weight_lx [i] have been described, but the prediction and restoration of luma_offset_lx [i] can be similarly performed.

Further, the predicted values of the luminance and color difference signal weighting coefficients (luma_weight_lx [i] and chroma_weight_lx [i] [j]) may be derived using the distance from the reference slice to be encoded. In this case, the index reconstruction unit 110B performs the prediction process of luma_weight_lx [i] using the mathematical expression (22), and performs the restoration process using the mathematical expression (23). Similarly, the index reconstruction unit 110B performs the prediction process of chroma_weight_lx [i] [j] using the mathematical expression (24), and performs the restoration process using the mathematical expression (25).

delta_luma_weight_lx [i] = (luma_weight_lx [i] --luma_weight_lx [i-1])… (22)

luma_weight_lx [i] = (delta_luma_weight_lx [i] + luma_weight_lx [i-1])… (23)

delta_chroma_weight_lx [i] [j] = (chroma_weight_lx [i] [j] --chroma_weight_lx [i-1] [j])… (24)

chroma_weight_lx [i] [j] = (delta_chroma_weight_lx [i] [j] + chroma_weight_lx [i-1] [j])… (25)

Here, in the mathematical formulas (22) and (24), i ≠ 0.

Since the prediction process and the decoding process are equivalent to introducing the i-1st value (i ≠ 0) into baseidx in the flowcharts of FIGS. 18 and 19, the description thereof will be omitted. Although the flowchart for the luminance component is shown here, the prediction process and the restoration process can be similarly realized for the color difference component (chroma_weight_lx [i] [j]). Further, here, as an example, the prediction method and the restoration method of luma_weight_lx [i] have been described, but the prediction and restoration of luma_offset_lx [i] can be similarly performed.

As the reference slice to which the coded target slice can be referred, a slice that is close in time distance or spatial distance to the coded target slice is often set from the viewpoint of coding efficiency. Here, since the luminance changes of the slices that are continuous over time and distance have a high correlation, the weighting coefficient and the offset have a high correlation over time and distance. Therefore, the code amount can be efficiently reduced by predicting the weighting coefficient and the offset value of the reference slice which are different in time by using the weighting coefficient and the offset value of the reference slice as the reference. Since spatially identical reference slices often have the same weighting factor and offset value, the amount of code can be reduced by introducing prediction for the same reason.

Next, a method for predicting chroma_offset_lx [i] [j], which represents the offset of the color difference signal, will be described. In the YUV color space, the color difference component expresses the color by the amount of deviation from the median. Therefore, the amount of change from the change in brightness in consideration of the median can be used as the predicted value by using the weighting coefficient. Specifically, the index reconstruction unit 110B performs the prediction processing of chroma_offset_lx [i] [j] using the mathematical formulas (26) to (27), and performs the restoration processing using the mathematical formula (28).

delta_chroma_offset_lx [i] [j] = (chroma_offset_lx [i] [j] + ((MED * chroma_weight_lx [i] [j]) >> chroma_log2_weight_denom) --MED)… (26)

MED = (MaxChromaValue >> 1)… (27)

Here, MaxChromaValue indicates the maximum brightness at which a color difference signal can be obtained. For example, for an 8-bit signal, MaxChromaValue is 255 and MED is 128.

chroma_offset_lx [i] [j] = (delta_chroma_offset_lx [i] [j]-((MED * chroma_weight_lx [i] [j]) >> chroma_log2_weight_denom) + MED)… (28)

FIG. 20 is a flowchart showing an example of the prediction process of chroma_offset_lx [i] [j] of the first embodiment.

First, the index reconstruction unit 110B derives the chroma_log2_weight_denom set in the index information (step S701).

Subsequently, the index reconstruction unit 110B derives chroma_offset_lx [i] [j] set in the index information (step S702).

Subsequently, the index reconstruction unit 110B derives an intermediate value of the maximum value (maximum signal) of the color difference signal (step S703).

Subsequently, the index reconstruction unit 110B derives delta_chroma_offsett_lx [i] [j] and sets it in the index information (step S704).

FIG. 21 is a flowchart showing an example of the restoration process of chroma_offset_lx [i] [j] of the first embodiment.

First, the index reconstruction unit 110B derives the chroma_log2_weight_denom already set in the index information (step S801).

Subsequently, the index reconstruction unit 110B derives chroma_offset_lx [i] [j] set in the index information (step S802).

Subsequently, the index reconstruction unit 110B derives an intermediate value of the maximum value (maximum signal) of the color difference signal (step S803).

Subsequently, the index reconstruction unit 110B derives chroma_offsett_lx [i] [j] and sets it in the index information (step S804).

By using the signal characteristics of the color difference signal and introducing a predicted value that takes into account the amount of deviation from the median, the code amount of the offset value of the color difference signal is reduced compared to the case where the offset value is encoded as it is. can.

Next, H. The method of deriving the predicted value of the weighting coefficient and the fixed-point accuracy using the WP parameter deriving method of the implicit weighted prediction defined in 264 and the like will be described. H. In the implicit weighted prediction of 264, the weighting factor is derived (the offset is 0) according to the time distance between the reference slices (the time ratio of the POC number). For the temporal distance between reference slices, the distance between the coded slice and the reference slice is derived based on the POC number, and the weighting factor is determined based on the distance ratio. At this time, the fixed-point precision is set to a fixed value of 5.

For example, H. In 264, the weighting coefficient is derived according to the pseudo code shown in the equation (29).

td = Clip3 (-128, 127, POCA --POCB)
tb = Clip3 (-128, 127, POCT --POCA)
tx = (td! = 0)? ((16384 + abs (td / 2)) / td): (0)
DistScaleFactor = Clip3 (-1024, 1023, (tb * tx + 32) >> 6)
implicit_luma_weight_l0 [i] = 64-(DistScaleFactor >> 2)
implicit_luma_weight_l1 [i] = DistScaleFactor >> 2… (29)

Here, POCA indicates the POC number of the reference image A corresponding to the list 1, POCB indicates the POC number of the reference image B corresponding to the list 0, and POCT indicates the POC number of the prediction target image. Clip3 (L, M, N) is a function that performs clip processing so that the last argument N does not exceed the range of the minimum value L and the maximum value M indicated by the first two. The abs () function is a function that returns the absolute value of the argument. td and tb represent the time ratio, td is the difference between the POC number of the reference image corresponding to List 1 and the POC number of the reference image corresponding to List 0, and tb is the POC number and the list of the predicted image. The difference in the POC numbers of the reference images corresponding to 0 is shown. These values derive the scaling variable DistScaleFactor for the distance of the weighting factors. Based on the DistScaleFactor, the weighting factors (implicit_luma_weight_l0 [i], implicit_luma_weight_l1 [i]) corresponding to List 0 and List 1 are derived. The same applies to the color difference signal. The index reconstruction unit 110B predicts the fixed-point accuracy by the mathematical formula (30) using the fixed-point accuracy implicit_log2_weight_denom derived here.

delta_luma_log2_weight_denom = (luma_log2_weight_denom --implicit_log2_weight_denom)… (30)

The fixed-point accuracy of the color difference signal can also be predicted by the mathematical formula (30). This value is restored by the formula (31).

luma_log2_weight_denom = (delta_luma_log2_weight_denom + implicit_log2_weight_denom)… (31)

The fixed-point accuracy of the color difference signal can also be restored by the same method as in the equation (31).

Next, a prediction formula for the weighting factor will be described. Assuming that the implicit weighting factor is implicit_luma_weight_lx [i], the index reconstruction unit 110B predicts the weighting factor luma_weight_lx [i] by the formula (32) and restores it by the formula (33).

if (luma_log2_weight_denom> = implicit_log2_weight_denom) {
norm_denom = (luma_log2_weight_denom --implicit_log2_weight_denom)
delta_luma_weight_lx [i] = (luma_weight_lx [i]-(implicit_luma_weight_lx [i] << norm_denom))
}
else {
norm_denom = (implicit_log2_weight_denom --luma_log2_weight_denom)
delta_luma_weight_lx [i] = (luma_weight_lx [i]-(implicit_luma_weight_lx [i] >> norm_denom))
}… (32)

Here, the index reconstruction unit 110B corrects the weighting coefficient and uses it for the prediction depending on whether it is larger or smaller than the fixed-point accuracy of the implicit weighted prediction.

if (luma_log2_weight_denom> = implicit_log2_weight_denom) {
norm_denom = (luma_log2_weight_denom --implicit_log2_weight_denom)
luma_weight_lx [i] = (delta_luma_weight_lx [i] + (implicit_luma_weight_lx [i] << norm_denom))
}
else {
norm_denom = (implicit_log2_weight_denom --luma_log2_weight_denom)
luma_weight_lx [i] = (delta_luma_weight_lx [i] + (implicit_luma_weight_lx [i] >> norm_denom))
}… (33)

Although the formula (32) shows an example of the weighting coefficient of the luminance component, the predicted value can be derived by using the same method for the color difference component.

FIG. 22 is a flowchart showing another example of the prediction process of luma_weight_lx [i] of the first embodiment.

First, the index reconstruction unit 110B derives the luma_log2_weight_denom set in the index information (step S901).

Subsequently, the index reconstruction unit 110B is subjected to H. Implicit_log2_weight_denom and implicit_luma_weight_lx [i] are derived according to the method of deriving the implicit weighted prediction of 264 (steps S902 and S903).

Subsequently, the index reconstruction unit 110B determines whether or not luma_log2_weight_denom is implicit_log2_weight_denom or more (step S904).

When luma_log2_weight_denom is implicit_log2_weight_denom or more (Yes in step S904), the index reconstruction unit 110B subtracts implicit_log2_weight_denom from luma_log2_weight_denom (step S905), derives the value obtained by subtracting implicit_luma_weight_lx [i], and shifts to the left. Step S906).

On the other hand, when luma_log2_weight_denom is not greater than or equal to implicit_log2_weight_denom (No in step S904), the index reconstruction unit 110B subtracts luma_log2_weight_denom from implicit_log2_weight_denom (step S907), derives the value obtained by subtracting implicit_luma_weight_lx [i], and shifts to the right. (Step S908).

Subsequently, the index reconstruction unit 110B subtracts the predicted value derived from luma_weight_lx [i] (step S909), and sets the subtracted value (difference value) in the index information (step S910).

FIG. 23 is a flowchart showing another example of the restoration process of luma_weight_lx [i] of the first embodiment.

First, the index reconstruction unit 110B derives the luma_log2_weight_denom already set in the index information (step S1001).

Subsequently, the index reconstruction unit 110B is subjected to H. Implicit_log2_weight_denom and implicit_luma_weight_lx [i] are derived according to the method of deriving the implicit weighted prediction of 264 (steps S1002 and S1003).

Subsequently, the index reconstruction unit 110B determines whether or not luma_log2_weight_denom is implicit_log2_weight_denom or more (step S1004).

When luma_log2_weight_denom is implicit_log2_weight_denom or more (Yes in step S1004), the index reconstruction unit 110B subtracts implicit_log2_weight_denom from luma_log2_weight_denom (step S1005), derives the value obtained by subtracting implicit_luma_weight_lx [i], and shifts to the left. Step S1006).

On the other hand, when luma_log2_weight_denom is not greater than or equal to implicit_log2_weight_denom (No in step S1004), the index reconstruction unit 110B subtracts luma_log2_weight_denom from implicit_log2_weight_denom (step S1007), derives the value obtained by subtracting implicit_luma_weight_lx [i], and shifts to the right. (Step S1008).

Subsequently, the index reconstruction unit 110B adds the predicted value derived to delta_luma_weight_lx [i] (step S1009), and sets the added value in the index information (step S1010).

It should be noted that the plurality of prediction methods described above can be used not only individually but also in combination. For example, by combining formulas (10), formulas (12) to (13), formulas (15) to (16), and formulas (26) to (27), the sign amount of the syntax element in the index information Can be efficiently reduced.

As described above, in the first embodiment, the index setting unit 108 outputs the index information in which the WP parameter information is mapped to the corresponding syntax configuration, and the index reconstruction unit 110B expresses the redundant representation of the syntax element in the slice. Predict based on the information encoded within. Therefore, according to the first embodiment, the amount of code can be reduced as compared with the case where the syntax element is coded as it is (with a direct value).

Here, based on the definition order (coding order) of the syntax elements used in the coded slice, the predicted value is derived from the already coded syntax elements as an in-screen correlation, and there is no change in brightness. By deriving the predicted value from the default value assuming the above, it is possible to make a prediction that makes the best use of the characteristics of the syntax element, and as a result, the overhead required for coding the syntax element can be reduced.

Note that a syntax element not specified in this embodiment may be inserted between the rows of the syntax table illustrated in FIGS. 10 to 13 of the first embodiment, and a description of other conditional branches is included. You may. Further, the syntax table may be divided into a plurality of tables, or a plurality of syntax tables may be integrated. In addition, the terms of each of the illustrated syntactic elements can be changed arbitrarily.

As described above, the coding apparatus 100 of the first embodiment solves the problem that the coding efficiency is lowered by removing the spatial redundancy by utilizing the correlation of the parameters of the information to be coded. The coding device 100 can reduce the amount of coding as compared with the conventional configuration in which the syntax element used in the weighted motion compensation prediction is coded as it is (with a direct value).

(Second Embodiment)
In the second embodiment, a decoding device that decodes the coded data encoded by the coding device of the first embodiment will be described.

FIG. 24 is a block diagram showing an example of the configuration of the decoding device 800 of the second embodiment.

The decoding device 800 decodes the coded data stored in an input buffer (not shown) or the like into a decoded image, and outputs the coded data as an output image to an output buffer (not shown). The coded data is output from, for example, the coding device 100 of FIG. 1, and is input to the decoding device 800 via a storage system, a transmission system, a buffer, or the like (not shown).

As shown in FIG. 24, the decoding device 800 includes a decoding unit 801, an inverse quantization unit 802, an inverse orthogonal transformation unit 803, an addition unit 804, a prediction image generation unit 805, and an index setting unit 806. Be prepared. The inverse quantization unit 802, the inverse orthogonal transformation unit 803, the addition unit 804, and the prediction image generation unit 805 are the inverse quantization unit 104, the inverse orthogonal transformation unit 105, the addition unit 106, and the prediction image generation unit 107, respectively, in FIG. It is an element that is substantially the same as or similar to. The decoding control unit 807 shown in FIG. 24 controls the decoding device 800, and can be realized by, for example, a CPU or the like.

The decoding unit 801 performs decoding based on the syntax for each frame or field for decoding the coded data. The decoding unit 801 includes an entropy decoding unit 801A and an index reconstruction unit 801B.

The entropy decoding unit 801A sequentially entropy-decodes the code string of each syntax, and motion information including a prediction mode, a motion vector, a reference number, etc., index information for weighted motion compensation prediction, a quantization conversion coefficient, and the like. Reproduce the coding parameters of the coded block. Here, the coding parameters are all parameters required for decoding, such as information on conversion coefficients and information on quantization, in addition to the above.

Specifically, the entropy decoding unit 801A has a function of performing decoding processing such as variable length decoding processing and arithmetic decoding processing on the input coded data. For example, H. In 264, context-adaptive variable-length coding (CAVLC: Context-based Adaptive Variable Lens Coding) and context-adaptive arithmetic coding (CABAC: Adapt-based Adaptive Binary Coding) are used. These processes are also called decoding processes.

The index reconstruction unit 801B restores the decoded index information and reconstructs the index information. Specifically, the index reconstruction unit 801B performs prediction processing according to the characteristics of the parameters of the syntax element, restores the syntax element, and indexes in order to reduce the code length of the syntax element of the decoded index information. Reconstruct the information. Specific examples of the prediction process will be described later.

The decoding unit 801 outputs motion information, index information, and quantization conversion coefficient, inputs the quantization conversion coefficient to the inverse quantization unit 802, inputs index information to the index setting unit 806, and predicts motion information. Input to the generation unit 805.

The inverse quantization unit 802 performs an inverse quantization process on the quantization conversion coefficient input from the decoding unit 801 to obtain a restoration conversion coefficient. Specifically, the dequantization unit 802 performs dequantization according to the quantization information used in the decoding unit 801. More specifically, the inverse quantization unit 802 multiplies the quantization step size derived by the quantization information by the quantization conversion coefficient to obtain the restoration conversion coefficient. The inverse quantization unit 802 outputs the restoration conversion coefficient and inputs it to the inverse orthogonal transformation unit 803.

The inverse orthogonal transformation unit 803 performs an inverse orthogonal transformation corresponding to the orthogonal transformation performed on the coding side with respect to the restoration transformation coefficient input from the inverse quantization unit 802, and obtains a restoration prediction error. The inverse orthogonal transformation unit 803 outputs the restoration prediction error and inputs it to the addition unit 804.

The addition unit 804 adds the restoration prediction error input from the inverse orthogonal transformation unit 803 and the corresponding predicted image to generate a decoded image. The addition unit 804 outputs the decoded image and inputs it to the prediction image generation unit 805. Further, the addition unit 804 outputs the decoded image as an output image to the outside. The output image is then temporarily stored in an external output buffer or the like (not shown), and is transferred to a display device system such as a display or monitor (not shown) or a video device system according to the output timing managed by the decoding control unit 807, for example. It is output.

The index setting unit 806 receives the index information input from the decoding unit 801, converts it into WP parameter information, outputs it, and inputs it to the prediction image generation unit 805. Specifically, the index setting unit 806 receives the index information reconstructed by the entropy decoding unit 801A and the index reconstructing unit 801B. Then, the index setting unit 806 confirms the list of reference images and the reference number, converts them into WP parameter information, and outputs the converted WP parameter information to the prediction image generation unit 805. Since the WP parameter information has already been described with reference to FIGS. 8A and 8B, the description thereof will be omitted.

The predicted image generation unit 805 generates a predicted image 815 using the motion information input from the decoding unit 801, the WP parameter information input from the index setting unit 806, and the decoded image input from the addition unit 804.

Here, the details of the predicted image generation unit 805 will be described with reference to FIG. 4. Like the predicted image generation unit 107, the predicted image generation unit 805 includes a multi-frame motion compensation unit 201, a memory 202, a unidirectional motion compensation unit 203, a prediction parameter control unit 204, a reference image selector 205, and a frame memory 206. And a reference image control unit 207.

The frame memory 206 stores the decoded image input from the addition unit 106 as the reference image under the control of the reference image control unit 207. The frame memory 206 has a plurality of memory sets FM1 to FMN (N ≧ 2) for temporarily holding the reference image.

The prediction parameter control unit 204 prepares a plurality of combinations of the reference image number and the prediction parameter as a table based on the motion information input from the decoding unit 801. Here, the motion information refers to a motion vector indicating the amount of motion deviation used in motion compensation prediction, a reference image number, information on a prediction mode such as unidirectional / bidirectional prediction, and the like. Prediction parameters refer to information about motion vectors and prediction modes. Then, the prediction parameter control unit 204 selects and outputs a combination of the reference image number and the prediction parameter used for generating the prediction image based on the motion information, inputs the reference image number to the reference image selector 205, and inputs the reference image number to the prediction parameter. Is input to the unidirectional motion compensation unit 203.

The reference image selector 205 is a switch for switching which output terminal of the frame memories FM1 to FMN of the frame memory 206 is connected according to the reference image number input from the prediction parameter control unit 204. For example, if the reference image number is 0, the reference image selector 205 connects the output end of the FM1 to the output end of the reference image selector 205, and if the reference image number is N-1, the output end of the FMN is referred to. Connect to the output end of the image selector 205. The reference image selector 205 outputs a reference image stored in the frame memory to which the output end is connected among the frame memories FM1 to FMN of the frame memory 206, and inputs the reference image to the unidirectional motion compensation unit 203. In the decoding device 800, since the reference image is not used except for the prediction image generation unit 805, it is not necessary to output the reference image to the outside of the prediction image generation unit 805.

The unidirectional prediction motion compensation unit 203 performs motion compensation prediction processing according to the prediction parameter input from the prediction parameter control unit 204 and the reference image input from the reference image selector 205, and generates a unidirectional prediction image. Since the motion compensation prediction has already been described with reference to FIG. 5, the description thereof will be omitted.

The unidirectional prediction motion compensation unit 203 outputs the unidirectional prediction image and temporarily stores it in the memory 202. Here, when the motion information (prediction parameter) indicates bidirectional prediction, the multi-frame motion compensation unit 201 performs weighted prediction using two types of unidirectional prediction images, so that the unidirectional prediction motion compensation unit 203 Stores the first unidirectional predicted image in the memory 202, and directly outputs the second unidirectional predicted image to the multi-frame motion compensating unit 201. Here, the unidirectional prediction image corresponding to the first is referred to as a first prediction image, and the unidirectional prediction image corresponding to the second is referred to as a second prediction image.

It should be noted that two unidirectional motion compensation units 203 may be prepared, and each may generate two unidirectional prediction images. In this case, when the motion information (prediction parameter) indicates unidirectional prediction, the unidirectional motion compensation unit 203 may directly output the first unidirectional prediction image as the first prediction image to the multi-frame motion compensation unit 201. good.

The multi-frame motion compensation unit 201 uses the first prediction image input from the memory 202, the second prediction image input from the unidirectional prediction motion compensation unit 203, and the WP parameter information input from the motion evaluation unit 109. , Performs weighted prediction to generate a prediction image. The multi-frame motion compensation unit 201 outputs a predicted image and inputs it to the addition unit 804.

Here, the details of the multi-frame motion compensation unit 201 will be described with reference to FIG. Like the predicted image generation unit 107, the multi-frame motion compensation unit 201 includes a default motion compensation unit 301, a weighted motion compensation unit 302, a WP parameter control unit 303, and WP selectors 304 and 305.

The WP parameter control unit 303 outputs the WP application flag and weight information based on the WP parameter information input from the index setting unit 806, inputs the WP application flag to the WP selectors 304 and 305, and weights the weight information. Input to the motion compensation unit 302.

Here, the WP parameter information includes the fixed-point accuracy of the weighting factor, the first WP application flag corresponding to the first predicted image, the first weighting factor, and the first offset, and the second WP adaptation corresponding to the second predicted image. Contains information on flags, second weighting factors, and second offset. The WP application flag is a parameter that can be set for each applicable reference image and signal component, and indicates whether or not to perform weighted motion compensation prediction. The weighting information includes information on the fixed-point accuracy of the weighting factors, the first weighting factor, the first offset, the second weighting factor, and the second offset. The WP parameter information represents the same information as in the first embodiment.

Specifically, when the WP parameter information is input from the index setting unit 806, the WP parameter control unit 303 separates and outputs the WP parameter information into the first WP application flag, the second WP application flag, and the weight information. , The first WP application flag is input to the WP selector 304, the second WP application flag is input to the WP selector 305, and the weight information is input to the weighted motion compensation unit 302.

The WP selectors 304 and 305 switch the connection end of each predicted image based on the WP application flag input from the WP parameter control unit 303. When each WP application flag is 0, the WP selectors 304 and 305 connect their output ends to the default motion compensation unit 301. Then, the WP selectors 304 and 305 output the first predicted image and the second predicted image and input them to the default motion compensation unit 301. On the other hand, the WP selectors 304 and 305 connect their respective output ends to the weighted motion compensation unit 302 when each WP application flag is 1. Then, the WP selectors 304 and 305 output the first predicted image and the second predicted image and input them to the weighted motion compensation unit 302.

The default motion compensation unit 301 performs average value processing based on the two unidirectional predicted images (first predicted image and second predicted image) input from the WP selectors 304 and 305, and generates a predicted image. Specifically, when the first WP application flag and the second WP application flag are 0, the default motion compensation unit 301 performs the average value processing based on the mathematical formula (1).

When the prediction mode indicated by the motion information (prediction parameter) is unidirectional prediction, the default motion compensation unit 301 uses only the first prediction image and obtains the final prediction image based on the mathematical formula (4). calculate.

The weighted motion compensation unit 302 is based on the two unidirectional prediction images (first prediction image and second prediction image) input from the WP selectors 304 and 305 and the weight information input from the WP parameter control unit 303. Perform weighted motion compensation. Specifically, when the first WP application flag and the second WP application flag are 1, the weighted motion compensation unit 302 performs weighting processing based on the mathematical formula (7).

When the calculated accuracy of the first predicted image and the second predicted image is different from that of the predicted image, the weighted motion compensating unit 302 rounds the logWDC, which is the fixed-point accuracy, by controlling it as in the mathematical formula (8). To realize.

Further, when the prediction mode indicated by the motion information (prediction parameter) is unidirectional prediction, the weighted motion compensation unit 302 uses only the first prediction image and the final prediction image based on the mathematical formula (9). Is calculated.

In addition, when the calculation accuracy of the first predicted image and the second predicted image is different from that of the predicted image, the weighted motion compensating unit 302 uses the fixed-point accuracy logWDC as in the mathematical formula (8) as in the case of bidirectional prediction. The rounding process is realized by controlling to.

Since the fixed-point accuracy of the weighting factor has already been described with reference to FIG. 7, the description thereof will be omitted. In the case of unidirectional prediction, various parameters (second WP adaptation flag, second weighting coefficient, and second offset information) corresponding to the second prediction image are not used, so they are set to predetermined initial values. It may have been done.

The decoding unit 801 uses the syntax 500 shown in FIG. The syntax 500 shows the structure of the coded data to be decoded by the decoding unit 801. Since the syntax 500 has already been described with reference to FIG. 9, the description thereof will be omitted. Further, since the picture parameter set syntax 505 has already been described with reference to FIG. 10 except that the coding is decoding, the description thereof will be omitted. Further, the slice header syntax 507 has already been described with reference to FIG. 11 except that the coding is decoding, and thus the description thereof will be omitted. Further, the pred weight table syntax 508 has already been described with reference to FIG. 12, except that the coding is decoding, and thus the description thereof will be omitted.

Here, the details of the prediction method of each syntax element related to the weighted prediction in the syntax configuration will be described. The prediction of the syntax element is performed by the index reconstruction unit 801B. The syntax configuration that explicitly shows the prediction method of the second embodiment is the same as that of the second embodiment, and is as shown in FIG.

For the prediction method between the signals of luma_log2_weight_denom and chroma_log2_weight_denom indicating the fixed-point accuracy of the weighting coefficient, the restoration process is performed using the mathematical formula (11). The details of the restoration process are as shown in FIG.

For the prediction methods of luma_weight_lx [i] and chroma_weight_lx [i] [j] representing the luminance and color difference signal weighting coefficients, restoration processing is performed using mathematical formulas (14) and (17). The details of the restoration process are as shown in FIG.

For prediction methods for deriving the predicted values of the luminance and color difference signal weighting coefficients (luma_weight_lx [i] and chroma_weight_lx [i] [j]) with different reference numbers or different POC numbers, formulas (19) and (21) are used. The restoration process is performed using. The details of the restoration process are as shown in FIG.

For the prediction method for deriving the predicted values of the luminance and color difference signal weighting coefficients (luma_weight_lx [i] and chroma_weight_lx [i] [j]) using the distance from the reference slice to be encoded, refer to the equation (23) and The restoration process is performed using (25). The details of the restoration process are equivalent to introducing the i-1st value (i ≠ 0) into baseidx in the flowchart of FIG.

H. For the method of deriving the predicted value of the weighting coefficient and the fixed-point accuracy using the WP parameter deriving method of the implicit weighted prediction defined in 264 and the like, the formulas (31) and (33) are used. The restoration process is performed. The details of the restoration process are as shown in FIG.

It should be noted that the plurality of prediction methods described above can be used not only individually but also in combination. For example, by combining the mathematical formula (11), the mathematical formula (14), the mathematical formula (17), and the mathematical formula (28), it is possible to efficiently reduce the sign amount of the syntax element in the index information.

As described above, in the second embodiment, the decoding device 800 solves the problem that the coding efficiency is lowered by removing the spatial redundancy by utilizing the correlation of the parameters of the information to be coded. The decoding device 800 can reduce the amount of coding as compared with the conventional configuration in which the syntax element used in the weighted motion compensation prediction is encoded as it is (with a direct value).

(Modification example)
In the first to second embodiments, an example is described in which a frame is divided into rectangular blocks having a size of 16 × 16 pixels, and encoding / decoding is performed in order from the block on the upper left of the screen to the lower right (FIG.). See 2A). However, the coding order and the decoding order are not limited to this example. For example, coding and decoding may be performed in order from the lower right to the upper left, or coding and decoding may be performed so as to draw a spiral from the center of the screen toward the edge of the screen. Further, coding and decoding may be performed in order from the upper right to the lower left, or coding and decoding may be performed so as to draw a spiral from the edge of the screen toward the center of the screen. In this case, the positions of the adjacent pixel blocks that can be referred to change depending on the coding order, so the positions may be changed as appropriate.

In the first to second embodiments, the prediction target block sizes such as 4 × 4 pixel block, 8 × 8 pixel block, and 16 × 16 pixel block have been illustrated as examples, but the prediction target block is a uniform block. It does not have to be a shape. For example, the prediction target block size may be a 16 × 8 pixel block, an 8 × 16 pixel block, an 8 × 4 pixel block, a 4 × 8 pixel block, or the like. Further, it is not necessary to unify all block sizes in one coding tree block, and a plurality of different block sizes may be mixed. When a plurality of different block sizes are mixed in one coding tree block, the amount of code for encoding or decoding the division information increases as the number of divisions increases. Therefore, it is desirable to select the block size in consideration of the balance between the code amount of the divided information and the quality of the locally decoded image or the decoded image.

In the first and second embodiments described above, for the sake of simplification, a comprehensive description of the color signal component is described without distinguishing between the luminance signal and the prediction process in the color difference signal. However, when the prediction process differs between the luminance signal and the color difference signal, the same or different prediction methods may be used. If different prediction methods are used for the luminance signal and the color difference signal, the prediction method selected for the luminance signal can be encoded or decoded in the same manner as the luminance signal.

In the first and second embodiments described above, for the sake of simplicity, the luminance signal and the weighted motion compensation prediction process in the color difference signal are not distinguished, and a comprehensive description of the color signal component is described. However, when the weighted motion compensation prediction process differs between the luminance signal and the color difference signal, the same or different weighted motion compensation prediction process may be used. If different weighted motion compensation prediction processes are used between the luminance signal and the color difference signal, the weighted motion compensation prediction process selected for the color difference signal can be encoded or decoded in the same manner as the luminance signal.

In the first and second embodiments, a syntax element not specified in the present embodiment may be inserted between the rows of the table shown in the syntax configuration, and other descriptions regarding conditional branching are included. It doesn't matter if it is. Alternatively, the syntax table can be divided and integrated into a plurality of tables. Further, it is not always necessary to use the same term, and it may be arbitrarily changed depending on the form to be used.

As described above, each embodiment realizes highly efficient weighted motion compensation prediction processing while solving the problem of encoding redundant information of the syntax configuration when performing weighted motion compensation prediction. Therefore, according to each embodiment, the coding efficiency is improved, and the subjective image quality is also improved.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and variations thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

For example, it is also possible to store and provide a program that realizes the processing of each of the above embodiments in a computer-readable storage medium. The storage medium may be a storage medium such as a magnetic disk, an optical disk (CD-ROM, CD-R, DVD, etc.), a magneto-optical disk (MO, etc.), a semiconductor memory, etc., which can store a program and can be read by a computer. For example, the storage form may be any form.

Further, the program that realizes the processing of each of the above embodiments may be stored on a computer (server) connected to a network such as the Internet and downloaded to the computer (client) via the network.

100 Coding device 101 Subtraction unit 102 Orthogonal transformation unit 103 Quantization unit 104 Inverse quantization unit 105 Inverse orthogonal transformation unit 106 Addition unit 107 Prediction image generation unit 108 Index setting unit 109 Motion evaluation unit 110 Coding unit 110A Entropy coding unit 110B Index reconstruction unit 111 Coding control unit 201 Multiple frame motion compensation unit 202 Memory 203 Unidirectional motion compensation unit 204 Prediction parameter control unit 205 Reference image selector 206 Frame memory 207 Reference image control unit 301 Default motion compensation unit 302 Weighted motion Compensation unit 303 WP parameter control unit 304, 305 WP selector 800 Decoding device 801 Decoding unit 801A Entropy decoding unit 801B Index reconstruction unit 802 Inverse quantization unit 803 Inverse orthogonal transformation unit 804 Addition unit 805 Prediction image generation unit 806 Index setting unit 807 Decoding control unit

Claims (3)

Data representing the first fixed-point accuracy of the luminance weighting factor,
Data representing the first difference value, which is the same value as the difference between the first fixed-point accuracy and the second fixed-point accuracy of the weight coefficient of the color difference, and is the data representing the first fixed-point accuracy and the first difference value. And the data used in the process of determining the second fixed-point accuracy by adding
When the value is the same as the value obtained by shifting "1" to the left by the number of bits determined by the first fixed-point precision, and there is no change in the pixel value in the brightness between the reference image and the target image. Data representing the second difference value of the brightness weighting coefficient, which is the same value as the difference between the first reference value, which is the same value as the brightness weighting coefficient value, and the brightness weighting coefficient. The data used in the process of determining the weighting coefficient of the brightness by adding the first reference value and the second difference value, and
It is the same value as the value obtained by shifting "1" to the left by the number of bits determined by the second fixed-point precision, and there is no change in the pixel value due to the color difference between the reference image and the target image. The data representing the third difference value which is the same value as the difference between the second reference value which is the same value as the value of the weight coefficient of the color difference and the weight coefficient of the color difference in the case, and the third difference value. And the data used in the process of determining the weight coefficient of the color difference by adding the second reference value and the data.
A coding unit that generates coded data that encodes each of
A storage unit for storing the coded data and
A storage device equipped with . Data representing the first fixed-point accuracy of the luminance weighting factor,
Data representing the first difference value, which is the same value as the difference between the first fixed-point accuracy and the second fixed-point accuracy of the weight coefficient of the color difference, and is the data representing the first fixed-point accuracy and the first difference value. And the data used in the process of determining the second fixed-point accuracy by adding
When the value is the same as the value obtained by shifting "1" to the left by the number of bits determined by the first fixed-point precision, and there is no change in the pixel value in the brightness between the reference image and the target image. Data representing the second difference value of the brightness weighting coefficient, which is the same value as the difference between the first reference value, which is the same value as the brightness weighting coefficient value, and the brightness weighting coefficient. The data used in the process of determining the weighting coefficient of the brightness by adding the first reference value and the second difference value, and
It is the same value as the value obtained by shifting "1" to the left by the number of bits determined by the second fixed-point precision, and there is no change in the pixel value due to the color difference between the reference image and the target image. The data representing the third difference value which is the same value as the difference between the second reference value which is the same value as the value of the weight coefficient of the color difference and the weight coefficient of the color difference in the case, and the third difference value. And the data used in the process of determining the weight coefficient of the color difference by adding the second reference value and the data.
A coding unit that generates coded data that encodes each of
A transmission unit that transmits the coded data and
A transmitter equipped with . A step of setting an index for a reference image, a luminance weighting factor, a first fixed-point precision of the luminance weighting factor, a color difference weighting factor, and a second fixed-point precision of the color difference weighting factor.
The step of encoding the first fixed-point precision and
A step of encoding a first difference value, which is the same value as the difference between the first fixed-point precision and the second fixed-point precision.
When the value is the same as the value obtained by shifting "1" to the left by the number of bits determined by the first fixed-point precision, and there is no change in the pixel value in the brightness between the reference image and the target image. A step of deriving a first reference value, which is the same value as the value of the weighting coefficient of the brightness, based on the first fixed-point accuracy.
A step of encoding a second difference value of the luminance weighting coefficient, which is the same value as the difference between the first reference value and the luminance weighting coefficient.
It is the same value as the value obtained by shifting "1" to the left by the number of bits determined by the second fixed-point precision, and there is no change in the pixel value due to the color difference between the reference image and the target image. A step of deriving a second reference value, which is the same value as the value of the weight coefficient of the color difference in the case, based on the second fixed-point accuracy.
A step of encoding a third difference value, which is the same value as the difference between the color difference weighting coefficient and the second reference value.
By subtracting the value obtained by multiplying the median by the weighting factor of the color difference and shifting to the right by the number of bits determined by the second fixed-point precision, from the median of the maximum pixel value. The step of deriving the third reference value, which is the same value as the obtained value,
A step of encoding a fourth difference value, which is the same value as the difference between the color difference offset and the third reference value, and
Coding method including.

Images