JP2022065225A - Lic unit, image decoding device, and image coding device - Google Patents

Abstract

To solve a problem in which the division used in image coding and decoding processing refers to a table, but the memory usage of the table is large, and also, when linear approximation is performed by changing division to multiplication using the table, the bit width required for multiplication becomes large and the circuit scale increases.SOLUTION: The index and scale shift value of a table are derived from the features derived using the adjacent pixels of a target block and a reference block, and the value of the table referenced using the index is shifted by using the scale shift value to derive the scale factor. This reduces the memory required for the table by storing only some elements using the periodicity of the table and deriving the elements that are not stored from some elements.SELECTED DRAWING: Figure 13

Description

Embodiments of the present invention relate to a predictive image generator, a moving image decoding device, and a moving image coding device.

In order to efficiently transmit or record a moving image, a moving image coding device that generates coded data by encoding the moving image, and a moving image that generates a decoded image by decoding the coded data. An image decoding device is used.

Specific examples of the moving image coding method include H.264 / AVC and HEVC (High-Efficiency Video Coding) method.

In such a moving image coding method, the image (picture) constituting the moving image is a tile group obtained by dividing the image, a tile obtained by dividing the tile group, and a tile by dividing the tile. The obtained coding tree unit (CTU: Coding Tree Unit), the coding unit obtained by dividing the coding tree unit (sometimes referred to as a coding unit (CU)), and the coding unit. It is managed by a hierarchical structure consisting of a transformation unit (TU: Transform Unit) obtained by dividing the image, and is encoded / decoded for each CU.

Further, in such a moving image coding method, a predicted image is usually generated based on a locally decoded image obtained by encoding / decoding an input image, and the predicted image is obtained from the input image (original image). The prediction error obtained by subtraction (sometimes referred to as a "difference image" or "residual image") is encoded. Examples of the method for generating the predicted image include inter-screen prediction (inter-prediction) and in-screen prediction (intra-prediction).

Further, Non-Patent Document 1 is mentioned as a technique for coding and decoding moving images in recent years.

"Versatile Video Coding (Draft 4)", JVET-M1001, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29/WG 11, 2019-01-09 17:06 : 06 "CE10: Low pipeline latency LIC (test 10.5.2)", JVET-M0087, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29/WG 11, 2019-01 -09

However, in image coding and image decoding processing, division is used in LIC prediction, CCLM prediction, motion vector scaling, and the like. In Non-Patent Documents 1 and 2, addition / subtraction multiplication, shift operation and table reference are used in these processes, but there is a problem that the memory usage of this table is large. Further, when linear approximation is performed by changing division to multiplication using this table, there is a problem that the bit width required for multiplication becomes large and the circuit scale increases.

The LIC unit according to one aspect of the present invention is an LIC unit that generates a predicted image by LIC prediction, and is an adjacent pixel (first adjacent pixel) of a target block and an adjacent pixel (second adjacent pixel) of a reference block. ), A LIC parameter derivation unit that derives the scaling factor scale and offset offset (LIC parameter) using a table, and a LIC filter unit that generates a predicted image using the image of the reference block and the LIC parameter. The parameter derivation unit uses the first adjacent pixel and the second adjacent pixel, and the feature amount a1 (first feature amount) of the adjacent pixel of the target block and the feature amount a2 (second feature amount) of the adjacent pixel of the reference block. The feature amount) is derived, the reference value InvDiv of the above table is derived using the second feature amount a2, and the value obtained by multiplying the first feature amount a1 and the above table reference value InvDiv is the first shift value (expC). , ShiftA) to derive the scaling factor scale by shifting to the right, and the LIC parameter derivation unit is characterized by deriving the offset offset using the first adjacent pixel, the second adjacent pixel, and the scaling factor. And.

According to one aspect of the present invention, in the moving image coding / decoding process, it is possible to suppress an increase in memory bandwidth while suppressing a deterioration in performance.

It is a schematic diagram which shows the structure of the image transmission system which concerns on this embodiment. It is a figure which showed the structure of the transmission device equipped with the moving image coding device which concerns on this embodiment, and the receiving device equipped with moving image decoding device. (a) shows a transmitting device equipped with a moving image coding device, and (b) shows a receiving device equipped with a moving image decoding device. It is a figure which showed the structure of the recording apparatus which carried out the moving image coding apparatus which concerns on this embodiment, and the reproduction apparatus which provided with moving image decoding apparatus. (a) shows a recording device equipped with a moving image coding device, and (b) shows a reproducing device equipped with a moving image decoding device. It is a figure which shows the hierarchical structure of the data of a coded stream. It is a figure which shows the division example of CTU. It is a schematic diagram which shows the structure of the moving image decoding apparatus. It is a schematic diagram which shows the structure of the inter prediction parameter derivation part. It is a schematic diagram which shows the structure of a LIC part. It is a figure explaining the target pixel and the reference pixel to be referenced in LIC prediction. It is a figure explaining the derivation of the scaling factor of the LIC prediction of this invention. It is another figure explaining the derivation of the scaling factor of the LIC prediction of this invention. It is a figure explaining the division table of this invention. It is a figure which shows the data flow of the LIC parameter derivation part of this invention. It is a block diagram which shows the structure of a moving image coding apparatus. It is a schematic diagram which shows the structure of the inter prediction parameter derivation part. It is a schematic diagram which shows the structure of the inter prediction image generation part. It is a flowchart explaining the schematic operation of the moving image decoding apparatus.

(First Embodiment)
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a schematic diagram showing a configuration of an image transmission system 1 according to the present embodiment.

The image transmission system 1 is a system that transmits a coded stream in which a coded image is encoded, decodes the transmitted coded stream, and displays an image. The image transmission system 1 includes a moving image coding device (image coding device) 11, a network 21, a moving image decoding device (image decoding device) 31, and a moving image display device (image display device) 41. ..

An image is input to the moving image coding device 11.

The network 21 transmits the coded stream Te generated by the moving image coding device 11 to the moving image decoding device 31. The network 21 is an Internet (Internet), a wide area network (WAN: Wide Area Network), a small network (LAN: Local Area Network), or a combination thereof. The network 21 is not necessarily limited to a two-way communication network, but may be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. Further, the network 21 may be replaced with a storage medium on which a coded stream Te such as a DVD (Digital Versatile Disc: registered trademark) or BD (Blue-ray Disc: registered trademark) is recorded.

The moving image decoding device 31 decodes each of the coded streams Te transmitted by the network 21 and generates one or a plurality of decoded images.

The moving image display device 41 displays all or a part of one or a plurality of decoded images generated by the moving image decoding device 31. The moving image display device 41 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. Examples of the display form include stationary, mobile, and HMD. If the moving image decoding device 31 has high processing power, it displays an image with high image quality, and if it has only lower processing power, it displays an image that does not require high processing power and display power. do.

<Operator>
The operators used herein are described below.

>> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR, | = is an OR assignment operator, and || is a logical sum.

x? y: z is a ternary operator that takes y when x is true (other than 0) and z when x is false (0).

Clip3 (a, b, c) is a function that clips c to a value greater than or equal to a and less than or equal to b. Is a function that returns c (where a <= b).

Clip1 (x) is a function that clips x to a value of 0 or more (1 << bitDepth) -1 or less, and is a function expressed by the following formula. Here, bitDepth is a value indicating the number of pixel bits of luminance or color difference.

Clip1 (x) = Clip3 (0, (1 << bitDepth) -1, x)
abs (a) is a function that returns the absolute value of a.

Int (a) is a function that returns an integer value of a.

floor (a) is a function that returns the largest integer less than or equal to a.

ceil (a) is a function that returns the smallest integer greater than or equal to a.

a / d represents the division of a by d (rounded down to the nearest whole number).

a ^ b represents a to the bth power.

sign (x) is a function that returns 0 when x is 0, 1 when x is positive, and -1 when x is negative.

<Structure of coded stream Te>
Prior to the detailed description of the moving image coding device 11 and the moving image decoding device 31 according to the present embodiment, the data of the coded stream Te generated by the moving image coding device 11 and decoded by the moving image decoding device 31. The structure will be described.

FIG. 4 is a diagram showing a hierarchical structure of data in the coded stream Te. The coded stream Te typically includes a sequence and a plurality of pictures constituting the sequence. In FIGS. 4 (a) to 4 (f), a coded video sequence that defines the sequence SEQ, a coded picture that defines the picture PICT, a coded tile group that defines the tile group, and a coded tile that defines the tile, respectively. , The coded tree unit included in the coded tile, and the coded unit included in the coded tree unit.

(Coded video sequence)
The coded video sequence defines a set of data that the moving image decoding device 31 refers to in order to decode the sequence SEQ to be processed. As shown in FIG. 4A, the sequence SEQ includes a video parameter set (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and an additional extension. Information SEI (Supplemental Enhancement Information) is included.

A video parameter set VPS is a set of coding parameters common to a plurality of moving images in a moving image composed of a plurality of layers, and a set of coding parameters related to the multiple layers included in the moving image and individual layers. The set is defined.

The sequence parameter set SPS defines a set of coding parameters that the moving image decoding device 31 refers to in order to decode the target sequence. For example, the width and height of the picture are specified. There may be a plurality of SPS. In that case, select one of multiple SPS from PPS.

The picture parameter set PPS defines a set of coding parameters that the moving image decoding device 31 refers to in order to decode each picture in the target sequence. For example, it includes a reference value of the quantization width used for decoding a picture (pic_init_qp_minus26) and a flag (weighted_pred_flag) indicating the application of weighted prediction. There may be a plurality of PPS. In that case, select one of a plurality of PPSs from each picture in the target sequence.

(Encoded picture)
The coded picture defines a set of data referred to by the moving image decoding device 31 in order to decode the picture PICT to be processed. The picture PICT includes tile groups, as shown in Figure 4 (b).

(Coded tile group)
The coded tile group defines a set of data referred to by the moving image decoding device 31 in order to decode the tile group to be processed. The tile group contains the tile group header and the tile group data as shown in FIG. 4 (c).

The tile group header includes a set of coding parameters referred to by the moving image decoding device 31 to determine the decoding method of the target tile group. Tile_group_type), which specifies the tile group type, is an example of the encoding parameters included in the tile group header.

The tile group types are (1) I tile group that uses only intra prediction for coding, (2) P tile group that uses unidirectional prediction or intra prediction for coding, and (3) coding. Examples thereof include a B tile group using unidirectional prediction, bidirectional prediction, or intra prediction. Note that the inter-prediction is not limited to single prediction and bi-prediction, and a prediction image may be generated using more reference pictures. Hereinafter, when referred to as P and B tile groups, they refer to tile groups including blocks in which inter-prediction can be used.

The tile group header may include a reference (pic_parameter_set_id) to the picture parameter set PPS. The tile group data defines a set (tiles, segments) of data referred to by the moving image decoding device 31.

(Coded tile)
The coded tile defines a set of data referenced by the moving image decoding device 31. The tile contains the CTU as shown in Figure 4 (d). A CTU is a block of fixed size (for example, 64x64) that constitutes a tile group, and is sometimes called a maximum coding unit (LCU).

(Coded tree unit)
FIG. 4 (e) defines a set of data referred to by the moving image decoding device 31 in order to decode the CTU to be processed. CTU is the basis of coding processing by recursive quadtree division (QT (Quad Tree) division), binary tree division (BT (Binary Tree) division) or ternary tree division (TT (Ternary Tree) division). It is divided into a coding unit CU, which is a typical unit. The BT division and the TT division are collectively called a multi-tree division (MT (Multi Tree) division). A node with a tree structure obtained by recursive quadtree division is called a coding node. The intermediate nodes of the quadtree, binary, and ternary tree are coded nodes, and the CTU itself is defined as the highest level coded node.

FIG. 5 is a diagram showing an example of CTU division. The CU is the terminal node of the encoding node and is not further divided. CU is the basic unit of coding processing (a). In the case of quadtree division, the coded node is divided into four coded nodes (b). A vertical ternary split (c), a horizontal ternary split (d), a vertical ternary split (e), and a horizontal ternary split (f).

If the CTU size is 64x64 pixels, the CU size is 64x64 pixels, 64x32 pixels, 32x64 pixels, 32x32 pixels, 64x16 pixels, 16x64 pixels, 32x16 pixels, 16x32 pixels, 16x16 pixels, 64x8 pixels, 8x64 pixels. , 32x8 pixels, 8x32 pixels, 16x8 pixels, 8x16 pixels, 8x8 pixels, 64x4 pixels, 4x64 pixels, 32x4 pixels, 4x32 pixels, 16x4 pixels, 4x16 pixels, 8x4 pixels, 4x8 pixels, and 4x4 pixels. ..

(Encoding unit)
As shown in FIG. 4 (f), a set of data referred to by the moving image decoding device 31 in order to decode the coding unit to be processed is defined. Specifically, the CU is composed of a CU header CUH, a prediction parameter, a conversion parameter, a quantization conversion coefficient, and the like. The prediction mode etc. are specified in the CU header.

The prediction process may be performed in CU units or in sub-CU units that are further divided CUs. If the size of the CU and the sub CU are equal, there is only one sub CU in the CU. If the CU is larger than the size of the sub CU, the CU is split into sub CUs. For example, if the CU is 8x8 and the sub CU is 4x4, the CU is divided into four sub-CUs consisting of two horizontal divisions and two vertical divisions.

There are two types of prediction (prediction mode): intra prediction and inter prediction. Intra prediction refers to prediction within the same picture, and inter prediction refers to prediction processing performed between pictures different from each other (for example, between display times and between layer images).

The conversion / quantization process is performed in CU units, but the quantization conversion coefficient may be entropy-coded in subblock units such as 4x4.

(Prediction parameter)
The prediction image is derived by the prediction parameters associated with the block. Prediction parameters include intra-prediction and inter-prediction prediction parameters.

Hereinafter, the prediction parameters of the inter-prediction will be described. The inter-prediction parameters are composed of the prediction list usage flags predFlagL0 and predFlagL1, the reference picture indexes refIdxL0 and refIdxL1, the motion vectors mvL0 and mvL1, and the LIC usage flags licFlag. predFlagL0 and predFlagL1 are flags indicating whether or not a reference picture list called an L0 list and an L1 list is used, respectively, and when the value is 1, the corresponding reference picture list is used. Further, licFlag is a flag indicating whether or not the LIC processing described later is performed, and when the value is 1, the LIC processing is performed. If licFlag is not set in the target block, the value of the licFlag is treated as 0. In the present specification, when "a flag indicating whether or not it is XX" is described, it is assumed that a flag other than 0 (for example, 1) is XX, 0 is not XX, and logical negation, logical product, etc. Treat 1 as true and 0 as false (same below). However, in an actual device or method, other values can be used as true values and false values.

The syntax elements for deriving the inter-prediction parameters include, for example, affine flag affinity_flag, merge flag merge_flag, merge index merge_idx, inter-prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, difference vector mvdLX, motion vector accuracy. There is a mode amvr_mode.

(Reference picture list)
The reference picture list is a list of reference pictures stored in the reference picture memory 306.

(Merge prediction and AMVP)
Prediction parameter decoding (encoding) methods include a merge mode and an AMVP (Adavanced Motion Vector Prediction) mode, and merge_flag is a flag for identifying these. The merge prediction mode is a mode in which predFlagLX (or inter_pred_idc), refIdxLX, and mvLX are not included in the coded data and are derived from the prediction parameters of the neighboring blocks that have already been processed. AMVP mode is a mode that includes inter_pred_idc, refIdxLX, and mvLX in the coded data. Note that mvLX is encoded as mvp_LX_idx, mvdLX, and amvr_mode that identify the prediction vectors mvpLX. In addition to the merge prediction mode, there may be an affine prediction mode identified by the affine flag affine_flag. As one form of the merge prediction mode, there may be a skip mode identified by the skip flag skip_flag. The skip mode is a mode in which the prediction parameters are derived by the same method as the merge prediction mode, and the prediction error (residual image) is not included in the coded data. In other words, when the skip flag skip_flag is 1, only the syntax related to the merge prediction mode such as the skip_flag and merge_idx is included with respect to the target CU, and the motion vector and the like are not included in the coded data.

inter_pred_idc is a value indicating the type and number of reference pictures, and takes one of PRED_L0, PRED_L1, and PRED_BI. PRED_L0 and PRED_L1 indicate a simple prediction using one reference picture managed by the L0 list and the L1 list, respectively. PRED_BI shows a bipredictive BiPred that uses two reference pictures managed by the L0 list and the L1 list.

merge_idx is an index indicating which of the prediction parameter candidates (merge candidates) derived from the processed block is used as the prediction parameter of the target block.

(Motion vector)
mvLX indicates the amount of shift between blocks on two different pictures. mvLX is encoded as mvpLX, mvdLX.

(Inter prediction identifier inter_pred_idc and prediction list usage flag predFlagLX)
The relationship between inter_pred_idc and predFlagL0 and predFlagL1 is as follows and can be converted to each other.

inter_pred_idc = (predFlagL1 << 1) + predFlagL0
predFlagL0 = inter_pred_idc & 1
predFlagL1 = inter_pred_idc >> 1
As the inter-prediction parameter, the prediction list use flag may be used, or the inter-prediction identifier may be used. Further, the determination using the prediction list utilization flag may be replaced with the determination using the inter-prediction identifier. On the contrary, the determination using the inter-prediction identifier may be replaced with the determination using the prediction list utilization flag.

(Configuration of moving image decoding device)
The configuration of the moving image decoding device 31 (FIG. 6) according to the present embodiment will be described.

The moving image decoding device 31 includes an entropy decoding unit 301, a parameter decoding unit (predicted image decoding device) 302, a loop filter 305, a reference picture memory 306, a predicted parameter memory 307, a predicted image generation unit (predicted image generator) 308, and a reverse. It includes a quantization / inverse conversion unit 311, an addition unit 312, and a prediction parameter derivation unit 320. In addition, in accordance with the moving image coding device 11 described later, there is also a configuration in which the moving image decoding device 31 does not include the loop filter 305.

The parameter decoding unit 302 further includes a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (predictive mode decoding unit), and the CU decoding unit 3022 further includes a TU decoding unit 3024. These may be collectively called a decoding module. The header decoding unit 3020 decodes the parameter set information such as SPS and PPS and the tile group header (tile group information) from the coded data. The CT information decoding unit 3021 decodes the CT from the encoded data. The CU decoding unit 3022 decodes the CU from the encoded data. The TU decoding unit 3024 decodes the QP update information (quantization correction value) and the quantization prediction error (residual_coding) from the coded data when the TU contains a prediction error.

The TU decoding unit 3024 decodes the QP update information (quantization correction value) and the quantization prediction error (residual_coding) from the coded data when the mode is other than the skip mode (skip_mode == 0). More specifically, when skip_mode == 0, the TU decoding unit 3024 decodes the flag cu_cbp indicating whether or not the target block contains a quantization prediction error from the encoded data, and cu_cbp is 1. In some cases, the quantization prediction error is decoded. If cu_cbp does not exist in the coded data, it is derived as 0.

Further, the prediction parameter derivation unit 320 includes an inter-prediction parameter derivation unit 303 and an intra-prediction parameter derivation unit 304 (not shown). The prediction image generation unit 308 includes an inter prediction image generation unit 309 and an intra prediction image generation unit 310.

In the following, examples of using CTU and CU as the processing unit will be described, but the processing is not limited to this example, and processing may be performed in sub-CU units. Alternatively, CTU and CU may be read as blocks, sub-CUs may be read as sub-blocks, and processing may be performed in units of blocks or sub-blocks.

The entropy decoding unit 301 performs entropy decoding on the coded stream Te input from the outside, and decodes each code (syntax element). For entropy coding, a method of variable-length coding of syntax elements using a context (probability model) adaptively selected according to the type of syntax element and the surrounding situation, a predetermined table, or There is a method of variable-length coding a syntax element using a calculation formula. The former CABAC (Context Adaptive Binary Arithmetic Coding) stores the CABAC state of the context (the type of dominant symbol (0 or 1) and the probability state index pStateIdx that specifies the probability) in memory. The entropy decoding unit 301 initializes all CABAC states at the beginning of the segment (tile, CTU row). The entropy decoding unit 301 converts the syntax element into a binary string (Bin String) and decodes each bit of the Bin String. When using a context, the context index ctxInc is derived for each bit of the syntax element, the bits are decoded using the context, and then the CABAC state of the context is updated. Bits that do not use context are decoded with equal probability (EP, bypass), and ctxInc derivation and CABAC state update are omitted.

The parameter decoding unit 302 decodes the code by the entropy decoding unit 301 and outputs it to the prediction parameter derivation unit 320. The prediction parameter derivation unit 320 derives prediction parameters such as predMode, merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX, amvr_mode, and licFlag from the decoded code. The control of which code is decoded is performed based on the instruction of the parameter decoding unit 302.

(Basic flow)
FIG. 17 is a flowchart illustrating a schematic operation of the moving image decoding device 31.

(S1100: Parameter set information decoding) The header decoding unit 3020 decodes the parameter set information such as SPS and PPS from the coded data.

(S1200: Tile group information decoding) The header decoding unit 3020 decodes the tile group header (tile group information) from the encoded data.

Hereinafter, the moving image decoding device 31 derives the decoded image of each CTU by repeating the processes of S1300 to S5000 for each CTU included in the target picture.

(S1300: CTU information decoding) The CT information decoding unit 3021 decodes the CTU from the encoded data.

(S1400: CT information decoding) The CT information decoding unit 3021 decodes the CT from the encoded data.

(S1500: CU decoding) The CU decoding unit 3022 executes S1510 and S1520 to decode the CU from the coded data.

(S1510: CU information decoding) The CU decoding unit 3022 decodes CU information, prediction information, TU division flag split_transform_flag, CU residual flags cbf_cb, cbf_cr, cbf_luma, etc. from the coded data.

(S1520: TU information decoding) The TU decoding unit 3024 decodes the QP update information (quantization correction value) and the quantization prediction error (residual_coding) from the coded data when the TU contains a prediction error. The QP update information is a difference value from the quantized parameter predicted value qPpred, which is the predicted value of the quantized parameter QP.

(S2000: Predicted image generation) The predicted image generation unit 308 generates a predicted image based on the predicted information for each block included in the target CU.

(S3000: Inverse quantization / inverse conversion) The inverse quantization / inverse conversion unit 311 executes the inverse quantization / inverse conversion process for each TU included in the target CU.

(S4000: Decoded image generation) The adder 312 decodes the target CU by adding the predicted image supplied by the predicted image generation unit 308 and the prediction error supplied by the inverse quantization / inverse conversion unit 311. Generate an image.

(S5000: Loop filter) The loop filter 305 applies a loop filter such as a deblocking filter, SAO, or ALF to the decoded image to generate a decoded image.

(Structure of inter-prediction parameter derivation part)
The inter-prediction parameter derivation unit 303 derives the inter-prediction parameter by referring to the prediction parameter stored in the prediction parameter memory 307 (108) based on the code input from the entropy decoding unit 301 or the coding parameter determination unit 110. do. Further, the inter-prediction parameter derivation unit 303 outputs the inter-prediction parameter to the prediction image generation unit 308 (101) and stores it in the prediction parameter memory 307 (108). The inter-prediction parameter derivation unit 303 and its internal elements AMVP prediction parameter derivation unit 3032, merge prediction parameter derivation unit 3036, affine prediction unit 30372, and MMVD prediction unit 30373 are common to the motion image coding device and the motion image decoding device. These may be collectively referred to as a motion vector derivation unit (motion vector derivation device).

When merge_flag is 1, that is, the merge prediction mode is indicated, merge_idx is decoded and output to the merge prediction parameter derivation unit 3036.

When merge_flag is 0, that is, it indicates AMVP mode, for example, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX, amvr_mode are decoded as AMVP parameters. The AMVP parameter derivation unit 3032 derives mvpLX from mvp_LX_idx.

(MV addition part)
In the MV addition unit 3038, the derived mvpLX and mvdLX are added to derive mvLX.

(Merge prediction)
FIG. 7B is a schematic diagram showing the configuration of the merge prediction parameter derivation unit 3036 according to the present embodiment. The merge prediction parameter derivation unit 3036 includes a merge candidate derivation unit 30361 and a merge candidate selection unit 30362. The merge candidate derivation unit 30361 includes a spatial merge candidate derivation unit 30364 and a time merge candidate derivation unit 30365. The merge candidate to be stored in the list may be a label indicating the merge candidate (for example, A1, B1, B0, A0, B2 of the spatial merge candidate, Col of the time merge candidate). Based on the label of the merge candidate selected based on merge_idx, the inter-prediction parameter (motion vector mvLXN) corresponding to the label may be referenced. The merge candidates include predFlagLX, mvLX, and refIdxLX.

The merge candidate derivation unit 30361 derives the merge candidate by using the motion vector of the decoded adjacent block and the reference picture index as they are.

(AMVP)
FIG. 7C is a schematic diagram showing the configuration of the AMVP parameter derivation unit 3032 according to the present embodiment. The AMVP parameter derivation unit 3032 includes a vector candidate derivation unit 3033 and a vector candidate selection unit 3034. The vector candidate derivation unit 3033 derives the prediction vector candidate from the decoded adjacent block mvLX stored in the prediction parameter memory 307 based on refIdxLX, and stores the prediction vector candidate in the prediction vector candidate list mvpListLX [].

The vector candidate selection unit 3034 selects the motion vector mvpListLX [mvp_LX_idx] indicated by mvp_LX_idx among the predicted vector candidates of mvpListLX [] as mvpLX. The vector candidate selection unit 3034 outputs the selected mvpLX to the MV addition unit 3038.

(MV addition part)
The MV addition unit 3038 calculates mvLX by adding the mvpLX input from the AMVP parameter derivation unit 3032 and the decoded difference vector mvdLX. The addition unit 3038 outputs the calculated mvLX to the prediction image generation unit 308 and the prediction parameter memory 307.

mvLX [0] = mvpLX [0] + mvdLX [0]
mvLX [1] = mvpLX [1] + mvdLX [1]
amvr_mode is a syntax for switching the accuracy of the motion vector derived in the AMVP mode. For example, in amvr_mode = 0, 1, 2, 1/4 pixel, 1 pixel, and 4 pixel accuracy are switched.

When the motion vector accuracy is 1/16 accuracy, it is derived from amvr_mode as shown below to change the motion vector difference with 1/4, 1, 4 pixel accuracy to the motion vector difference with 1/16 pixel accuracy. It may be dequantized using MvShift (= 1 << amvr_mode).

MvdLX [0] = MvdLX [0] << (MvShift + 2)
MvdLX [1] = MvdLX [1] << (MvShift + 2)
(Motion vector scaling)
The method of deriving the scaling of the motion vector will be described. A motion vector Mv (reference motion vector), a picture PicMv containing a block with Mv, a reference picture PicMvRef of Mv, a motion vector sMv after scaling, a picture CurPic containing a block with sMv, and a reference picture CurPicRef referenced by sMv. The sMv derivation function MvScale (Mv, PicMv, PicMvRef, CurPic, CurPicRef) is derived from the following equation.

sMv = MvScale (Mv, PicMv, PicMvRef, CurPic, CurPicRef)
= Clip3 (-R1, R1-1, sign (distScaleFactor * Mv) * ((abs (distScaleFactor * Mv) + round1-1) >> shift1))
distScaleFactor = Clip3 (-R2, R2-1, (tb * tx + round2) >> shift2)
tx = (KN_MVSCALE + abs (td) >> 1) / td
td = DiffPicOrderCnt (PicMv, PicMvRef)
tb = DiffPicOrderCnt (CurPic, CurPicRef)
Here, round1, round2, shift1, and shift2 are round values and shift values for performing division using the reciprocal, for example, round1 = 1 << (shift1-1), round2 = 1 << (shift2-12-1). ), Shift1 = 8, shift2 = 6, etc. DiffPicOrderCnt (Pic1, Pic2) is a function that returns the difference between the time information of Pic1 and Pic2 (for example, POC). R1 and R2 limit the range in order to perform processing with limited accuracy. For example, R1 = 32768 and R2 = 4096. KN_MVSCALE is 1 << (shift1 + shift2), for example 16384. 2 ^ (shift1 + shift2) = 2 ^ (8 + 6) = 2 ^ 14 = 16384
Further, the scaling function MvScale (Mv, PicMv, PicMvRef, CurPic, CurPicRef) may be expressed by the following equation.

MvScale (Mv, PicMv, PicMvRef, CurPic, CurPicRef) =
Mv * DiffPicOrderCnt (CurPic, CurPicRef) / DiffPicOrderCnt (PicMv, PicMvRef)
That is, Mv may be scaled according to the ratio between the time information difference between CurPic and CurPicRef and the time information difference between PicMv and PicMvRef.

(Modified example of tx derivation method)
In the modified example, a method of deriving tx that scales by a method different from the above will be described.
More specifically, a method for deriving tx used for scaling will be described when the motion vector Mv is scaled to derive the motion vector sMv using the above two POC differences td and tb.

td = DiffPicOrderCnt (PicMv, PicMvRef) (MVSCALE-1)
tb = DiffPicOrderCnt (CurPic, CurPicRef)
sMv = Clip3 (-R1, R1-1, sign (distScaleFactor * Mv) * ((abs (distScaleFactor * Mv) + round1-1) >> shift1))
distScaleFactor = Clip3 (-R2, R2-1, (tb * tx + round2) >> shift2)
tx = (KN_MVSCALE + abs (td) >> 1) / td
(Modification example 1 of MV scaling value derivation method)
Instead of dividing td in the expression (MVSCALE-1), tx may be derived by table reference. More specifically, tx may be derived by the following equation using MvDivTableFull [k] (0 <= k <= 128).

tx = td> 0? MvDivTableFull [abs (td)]: -MvDivTableFull [abs (td)]
MVDivTableFull [k] = (k == 0)? 0: (KN_MVSCALE + abs (k) >> 1) / k (k = 0..128)
MvDivTable [] is a table (reciprocal, reciprocal table) for executing division by table reference, and the value held in the table is the derived value (value corresponding to the reciprocal of the divisor). That is, the divisor and the derived value are stored in association with each other in the table.

(Modification 2 of the MV scaling value derivation method)
In this processing example, the memory is reduced by storing only a part of the elements of the division table and deriving the elements that are not stored from some elements by using the periodicity of the division table. Here, the division table is a table that stores the derived value val obtained by dividing a predetermined constant of power of 2 (for example, KN_MVSCALE) by the absolute value tdAbs of the divisor td.

val = (KN_MVSCALE + tdAbs >> 1) / tdAbs
The MV scaling unit 3030 derives tx using some elements (table elements) of the division table MvDivTableFull. MvDivTableFull is a virtual table consisting of 2 ^ FN elements. Then, the MV scaling unit 3030 sets the values (inverse values) of 2 ^ FN-2 ^ N elements other than some sections from the table MvDivTable (number of elements 2 ^ N) that stores a part of MvDivTableFull as follows. Derived by processing. When the index using the reciprocal table as a reference is idx, the desired val is derived by sampling the index idx by k and then multiplying the value of the reciprocal table by 1 / k.

val = MvDivTable [idx / k] / k = MvDivTable [idx >> sc] >> sc
k = 2 ^ (sc)
Specifically, it will be described with reference to FIG. FIG. 12 is a diagram illustrating an example of deriving an inverse value other than the storage table from the inverse value of the storage table. In this example, N = 4. First, as shown in Fig. 12 (a), the MV scaling unit 3030 stores the value in the MvDivTable section Gr5 [16..31] (that is, [2 ^ N .. (2 ^ (N + 1) -1). In)]), the value of MvDivTable is used as it is as val. Here k = 1.

In the next interval Gr6 [32..63] (that is, [2 ^ (N + 1) .. (2 ^ (N + 2) -1)]) after the part, the value of MvDivTable is set to 1 as val. Derivate the value multiplied by / 2 (k = 2). Also, since Gr6 has twice the width of Gr5, the same value is repeatedly derived twice. Furthermore, in the next interval Gr7 [64-127] (that is, [2 ^ (N + 2) .. (2 ^ (N + 3) -1)]), the value of MvDivTable multiplied by 1/4 as val. Is derived (k = 4). Also, since Gr7 has four times the width of Gr5, the same value is repeatedly derived four times each. Hereinafter, the same process is repeated.

On the other hand, the MV scaling unit 3030 is MvDivTable [] as a val in the section Gr4 [8..15] (that is, [2 ^ (N-1) .. (2 ^ N-1)]) before the partial section. Derivation of the value obtained by doubling the value of (k = 1/2). Also, since Gr4 has half the width of Gr5, the elements of the table are thinned out to 2: 1 to derive. Furthermore, in the previous section Gr3 [4..7] (that is, [2 ^ (N-2) .. (2 ^ (N-1) -1)]), the value of MvDivTable [] was multiplied by 4 as val. Derive the value (k = 1/4). Also, since Gr3 has a width 1/4 times that of Gr5, the elements of the table are thinned out to 4: 1 to derive. Similarly, in the interval Gr2 [2..3] (that is, [2 ^ (N-3) .. (2 ^ (N-2) -1)]), the value of MvDivTable [] is multiplied by 8 as val. Is thinned out to 8: 1 and derived (k = 1/8). Similarly, in the interval Gr1 [1] (that is, [2 ^ (N-4) .. (2 ^ (N-3) -1)]), the value obtained by multiplying the value of MvDivTable [] by 16 as val is 16 Derived by thinning out to 1 (k = 1/16).

FIG. 12 (b) is a diagram summarizing these relationships. In FIG. 12 (b), the absolute value tdAbs of td, which is the difference between the POCs of the target picture and the reference picture, is regarded as the divisor a2. idx is the index of MvDivTable [].

The MV scaling unit 3030 derives the index idx for referencing the division table MvDivTable and the value v from tdAbs, and normalizes v by shifting with sc to derive the val corresponding to tdAbs.

sc = floor (log2 (tdAbs))
normDiff = (tdAbs << N) >> sc
idx = normDiff & ((1 << N) -1)
MSB = 1 << (PREC-1)
sc = sc + (normDiff! = 0)
v = (MvDivTable [idx] | MSB)
val = v << (14-(PREC-1) --sc)
When the number of elements (2 ^ N) of the division table MvDivTable is 16 (N = 4) and the accuracy of the division table PREC = 4, an example of MvDivTable [1 << N] is shown.

MvDivTable [1 << N] = {0, 7, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0}
The MV scaling unit 3030 derives the scale shift value sc from the logarithmic value of tdAbs. In addition, normDiff is derived by shifting the tdAbs to the right by the sc bit from the value obtained by shifting the tdAbs to the left by N bits. Further, idx may be derived by subtracting a predetermined eigenvalue (16, 2 ^ N when N = 4) from normDiff.

idx = normDiff --2 ^ N
This is equivalent to the process of extracting the lower (N-1) bits of normDiff (normDiff & ((1 << N) -1)). The value v is derived by adding 2 ^ (PREC-1) to MvDivTable [idx].

More specifically, the MV scaling unit 303 may be derived by the following processing. The MV scaling unit 303 derives sc from the logarithm of 2 of tdAbs, shifts tdAbs to the left by N, and shifts to the right by sc to derive normDiff. The index idx used to refer to the division table is derived from the least significant bit of normDiff. Further, the fixed value MSB is added to derive the value v. By normalizing v by shifting with sc (v << (11 --sc)), the value txAbs corresponding to val is derived. Since the derived txAbs is the absolute value of tx, the sign of td is added to derive tx. The scale shift value sc is basically a logarithm of 2 of tdAbs, but since floor () is used in the following processing, it is modified by normDiff! = 0.

sc = floorLog2 (tdAbs)
normDiff = (tdAbs << N) >> sc
idx = normDiff & ((1 << N) ―― 1))
MSB = 1 << (PREC-1)
sc = sc + (normDiff! = 0)
v = MvDivTable [idx] | MSB
txAbs = v << (11 --sc)
tx = ((td <0)? -txAbs: txAbs)
Here, floorLog2 (x) may be derived by the following equation.

floorLog2 (x) = floor (log2 (x)) = (x <= 0)? -1: 31 --clz (x)
However, it is assumed that the argument x in the above clz function has a width of 32 bits. The clz function can realize a function equivalent to floor (Log2 (x)).

When N = 4 and PREC = 4, the following reciprocal table is used.

MvDivTable [16] = {0, 7, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0}
The MV scaling unit 303 of the modified example uses the following reciprocal table when N = 5 and PREC = 5.

MvDivTable [32] = {0, 15, 14, 13, 12, 12, 11, 10, 10, 9, 8, 8, 7, 7, 6, 6, 5, 5, 4, 4, 4, 3, 3, 3, 2, 2, 2, 1, 1, 1, 0, 0}
In addition, (v << (10-sc)) is used in deriving txAbs of the above pseudo code.

txAbs = v << (10 --sc)
Note that txAbs may be derived by the following equation using (shift1 + shift2)-(PREC-1), that is, 14- (PREC-1).

txAbs = v << (14-(PREC-1) --sc)
In this modification, when the motion vector is scaled by the reciprocal of the difference value td of POC, the size of the value is reduced by the right shift of the shift value sc derived from the absolute value tdAbs of td (tdAbs << N). ) >> By referring to the division table MvDivTable by the index idx derived from sc (MvDivTable [idx]), it has the effect of scaling the motion vector using the MvDivTable with a small table size.

(Derivation of adjacent motion vector)
The merge prediction unit 30374 and the sub-block prediction unit 30372 may derive the motion information of the adjacent block as follows.

The upper left position (xCb, yCb) of the target picture, the width bW and the height bH of the target block are input to the merge prediction unit 30374 and the subblock prediction unit 30372, and the adjacent blocks (A0, A1, B0, B1, B2) Available flags availableFlagA0, availableFlagA1, availableFlagB0, availableFlagB1 and availableFlagB2, reference picture index refIdxLXA0, refIdxLXA1, refIdxLXB0, refIdxLXB1 and refIdxLXB2, predictive list usage flags predFlagLXA0, predFlagLXA1, predFlagB0 mvLXB2 is output.

The merge prediction unit 30374 and the sub-block prediction unit 30372 determine the availability of the adjacent block N in the order of A1, B1, B0, A0, B2.

If adjacent blocks are available, set availableFlagN to 1 and set mvLXN, refIdxLXN, predFlagLXN to MvLX [xNbN] [yNbN], RefIdxLX [xNbN] [yNbN], PredFlagLX [xNbN] [yNbN]. If not, set availableFlagN to 0 and mvLXN, refIdxLXN, predFlagLXN to 0.

(Inter prediction image generation unit 309)
When predMode indicates the inter-prediction mode, the inter-prediction image generation unit 309 generates a block or sub-block prediction image by inter-prediction using the inter-prediction parameter input from the inter-prediction parameter derivation unit 303 and the read reference picture. do.

FIG. 16 is a schematic diagram showing the configuration of the inter-predictive image generation unit 309 included in the predictive image generation unit 308 according to the present embodiment. The inter-predicted image generation unit 309 includes a motion compensation unit (predictive image generation device) 3091 and a compositing unit 3095.

(Motion compensation)
The motion compensation unit 3091 (interpolated image generation unit 3091) is a reference specified by refIdxLX from the reference picture memory 306 based on the inter-prediction parameters (predFlagLX, refIdxLX, mvLX) input from the inter-prediction parameter derivation unit 303. An interpolated image (motion compensation image) is generated by reading out a block in the picture RefPicLX (refImg) at a position shifted by mvLX from the position of the target block as a starting point. Here, when the accuracy of mvLX is not an integer accuracy, an interpolated image is generated by applying a filter called a motion compensation filter for generating pixels at decimal positions.

First, the motion compensation unit 3091 derives the integer position (xInt, yInt) and the phase (xFrac, yFrac) corresponding to the coordinates (x, y) in the prediction block by the following equations.

xInt = xPb + (mvLX [0] >> (log2 (MVPREC))) + x
xFrac = mvLX [0] & (MVPREC-1)
yInt = yPb + (mvLX [1] >> (log2 (MVPREC))) + y
yFrac = mvLX [1] & (MVPREC-1)
Here, (xPb, yPb) is the upper left coordinate of the bW * bH size block, x = 0… bW-1, y = 0… bH-1, and MVPREC is the accuracy of mvLX (1 / MVPREC pixel accuracy). ) Is shown. For example, MVPREC = 16.

The motion compensation unit 3091 derives a temporary image temp [] [] by performing horizontal interpolation processing on the reference picture refImg using an interpolation filter. The following Σ is the sum of k = 0..NTAP-1 for k, shift1 is the normalization parameter that adjusts the range of values, offset1 = 1 << (shift1-1).

temp [x] [y] = (ΣmcFilter [xFrac] [k] * refImg [xInt + k-NTAP / 2 + 1] [yInt] + offset1) >> shift1
Subsequently, the motion compensation unit 3091 derives the interpolated image Pred [] [] by performing vertical interpolation processing on temp [] []. The following Σ is the sum of k = 0..NTAP-1 with respect to k, and shift2 is the normalization parameter that adjusts the range of values, offset2 = 1 << (shift2-1).

Pred [x] [y] = (ΣmcFilter [yFrac] [k] * temp [x] [y + k-NTAP / 2 + 1] + offset2) >> shift2 In the case of dual prediction, the above Pred [ ] [] Is derived for each L0 list and L1 list (called PredL0 [] [] and PredL1 [] []), and Pred [] [] is generated from PredL0 [] [] and PredL1 [] [].

(Synthesis part)
The compositing unit 3095 refers to the interpolated image supplied from the motion compensation unit 3091, the inter-predicted parameters supplied from the inter-predicted parameter derivation unit 303, and the intra-predicted image supplied from the intra-predicted image generation unit 310. A predicted image is generated, and the generated predicted image is supplied to the addition unit 312.

The synthesis unit 3095 includes a combined intra / inter synthesis unit 30951, a Triangle synthesis unit 30952, an OBMC unit 30953, a BIO unit 30954, and a LIC unit 30955.

(Combined intra / inter compositing process)
The Combined intra / inter combination unit 30951 generates a prediction image by using a unidirectional prediction image in AMVP, a prediction image in the skip mode and the merge prediction mode, and an intra prediction image in combination.

(Triangle composition process)
The Triangle synthesizer 30952 generates a prediction image using the Triangle prediction.

(OBMC processing)
The OBMC unit 30953 generates a predicted image using OBMC (Overlapped block motion compensation) processing.

(BIO processing)
The BIO unit 30954 generates a predicted image by performing BIO (Bi-directional optical flow) processing. In the BIO process, a predicted image is generated with reference to PredL0 and PredL1 and the gradient correction term.

(LIC processing)
When the value of the LIC utilization flag licFlag is 1, the LIC unit 30955 predicts the pixel value of the target block based on the pixel value of the same color component already decoded by performing LIC (Local Illumination Compensation) processing. .. Specifically, it is a method of generating a predicted image of a target block by using a linear model based on a decoded image of the same color component. FIG. 8 is a block diagram showing an example of the configuration of the LIC unit 30955. The LIC unit 30955 includes a LIC parameter derivation unit (parameter derivation unit) 309552 and a LIC filter unit 309553.

Below, the LIC parameter derivation unit 309552 derives the scaling factor scale and offset offset in the linear model scale * x + offset, and the LIC filter unit 309553 uses the predicted images predLXL [x] [y], predLXcb [x] [y] and The process of updating predLXcr [x] [y] will be described.

In general, the LIC section 30955 includes the upper left position (xCb, yCb) of the target block, width bW and height bH, motion vector mvLX with 1/16 pixel accuracy of luminance, and motion vector mvCLX with 1/32 pixel accuracy of color difference. Using the reference picture refPicLXL, refPicLXCb and refPicLXCr, and the interpolated images predLXL, predLXCb and predLXCr generated by the motion compensation unit 3091 as input values, the updated predicted images predLXL, predLXCb and predLXCr which are output values are derived.

The LIC parameter derivation unit 309552 derives an output value using (xCb, yCb), bW and bH, mvLX, refPicLX, and the color component index cIdx as input values in the process of deriving adjacent pixels of the target block and the reference block. The output value is sumR of the adjacent pixel values of the reference block, sumC of the adjacent pixel values of the target block, sum RR of the squared pixel values of the reference block, and the adjacent pixel value of the target block and the adjacent pixel value of the reference block. The sum of products sumRC and the logarithmic cntShift with the number of adjacent pixels at the base of 2.

Further, the LIC parameter derivation unit 309552 derives an output value using the above sumR, sumC, sumRR, sumRC, cntShift, and the color component index cIdx as input values in the process of deriving the linear model for updating the predicted image. The output values are scale and offset offset.

In addition, the LIC filter unit 309553 updates the predicted image by the following formula. Here, x takes a value from 0 to bW-1, and y takes a value from 0 to bH-1.

predLX [x] [y] = Clip1 (((scale * predLX [x] [y]) >> shiftS) + offset) (LIC-1)
Here, shiftS = 5 may be set.

For the luminance component, (xCb, yCb), bW and bH, mvLX, refPicLXL, cIdx = 0 are input for the above, and scale and offset are output.

For the color difference component, enter the following instead of (xCb, yCb), bW, bH.

(xCbC, yCbC) = (xCb >> 1, yCb >> 1)
bWC = bW >> 1
bHC = bH >> 1
Also, enter mvCLX instead of mvLX. Instead of refPicLXL, refPicLXcb and cIdx = 1 are input for the Cb component, refPicLXcr and cIdx = 2 are input for the Cr component, and the scale and offset of the Cb component and the scale and offset of the Cr component are output.

Next, a process in which the LIC parameter derivation unit 309552 derives sumR, sumC, sumRR, sumRC, and contShift using (xCb, yCb), bW and bH, mvLX, refPicLX, and cIdx as input values will be described.

The LIC parameter derivation unit 309552 derives the following.
-Integer-precision motion vector of the target block intMV:
intMv [0] = (cIdx == 0)? ((mvLX [0] + (1 << 3)) >> 4): ((mvLX [0] + (1 << 4)) >> 5)
intMv [1] = (cIdx == 0)? ((mvLX [1] + (1 << 3)) >> 4): ((mvLX [1] + (1 << 4)) >> 5)
-Reference block position (xInt [x], yInt [y]):
xInt [x] = Clip3 (0, pic_width_in_luma_samples --1, xCb + intMv [0] + x)
yInt [y] = Clip3 (0, pic_height_in_luma_samples --1, yCb + intMv [1] + y)
Here, x takes a value from -1 to bW-1, and y takes a value from -1 to bH-1.
-Number of pixel bits bitDepth:
bitDepth = (cIdx == 0)? BitDepthY: BitDepthC
Here, BitDepthY is the bit depth of the luminance component, and BitDepthC is the bit depth of the color difference component, for example, 10 bits.
-Variables precShift, minDim and numSteps:
precShift = Max (0, bitDepth --12)
minDim = Min (bW, bH)
numSteps = (minDim> 8)? (minDim >> 1): minDim
-Target block positions xCbY and yCbY:
(xCbY, yCbY) = (cIdx == 0)? (xCb, yCb): (xCb << 1, yCb << 1)
FIG. 9 illustrates the relationship between the target block, the reference block, and the adjacent pixel referenced by the LIC parameter derivation unit 309552.

In addition, the LIC parameter derivation unit 309552 sets 1 to the availability availL of the left adjacent pixel if the adjacent block containing (xCbY-1, yCbY) is available, and sets 0 to availL otherwise. do.

In addition, the LIC parameter derivation unit 309552 sets 1 to the availability availT of the upper adjacent pixel if the adjacent block containing (xCbY, yCbY-1) is available, and sets 0 to availT otherwise. do.

Here, if both availL and availT are FALSE, then sumR = sumC = sumRR = sumRC = contShift = 0. Otherwise, the LIC parameter derivation unit 309552 derives scale and offset by the following procedure.

(Step 1) When availL is TRUE, the LIC parameter derivation unit 309552 sets (currPic [xCb + x] [yCb + y] >> precShift) to the left adjacent pixel C [x] [y] of the target block. do. Here, x takes -1 and y takes a value from 0 to bH-1. When availT is TRUE, the LIC unit 30955 sets (currPic [xCb + x] [yCb + y] >> precShift) to the upper adjacent pixels C [x] [y] of the target block. Here, x takes a value from 0 to bW-1, and y takes -1. currPic indicates the target picture.

(Step 2) When availL is TRUE, the LIC parameter derivation unit 309552 is set to the left adjacent pixel R [x] [y] of the reference block (refPicLX [xInt [x]] [yInt [y]] >> precShift). To set. Here, x takes -1 and y takes a value from 0 to bH-1. If availT is TRUE, set (refPicLX [xInt [x]] [yInt [y]] >> precShift) to the upper adjacent pixel R [x] [y] of the reference block. Here, x takes a value from 0 to bW-1, and y takes -1. refPicLX indicates a reference picture.

As shown in FIG. 9A, in (step 1) and (step 2), the L-shaped area outside the target block and the reference block is set as adjacent pixels. In addition to this, as shown in FIG. 9B, 2 * bH pixels (all or part) on the left and lower left of the target block and the reference block may be set as adjacent pixels. Alternatively, as shown in FIG. 9 (c), 2 * bW pixels (all or part) above and on the upper right of the target block and the reference block may be set as adjacent pixels. Hereinafter, the process will be described using the adjacent pixels set in (step 1) and (step 2).

(Step 3) The LIC parameter derivation unit 309552 derives the variables sL [k] and sT [k] by the following equations. Where k takes a value from 0 to numSteps-1.
sL [k] = (k * bH) >> Log2 (numSteps)
sT [k] = (k * bW) >> Log2 (numSteps)
(Step 4) The LIC parameter derivation unit 309552 derives the variables sumR, sumC, sumRR and sumRC by the following equations.

（ステップ５）LICパラメータ導出部309552が予測画像を更新するための線形モデルに係る値を導出する工程について説明する。
cntShift = (availL && availT) ? (Log2(numSteps)+1) : Log2(numSteps)
以下の工程において、LICパラメータ導出部309552は、上記sumR、sumC、sumRR、sumRC、contShiftおよびcIdxを入力値として、対象ブロックの隣接画素の特徴量a1、参照ブロックの隣接画素の特徴量a2を導出する。
・除算テーブルLICDivTable[k]を以下のように設定する。ここで、kは0から63までの値をとる。
LICDivTableFull[0] = 0
LICDivTableFull[k] = ((1 << 15) + (k >> 1)) / k
・変数precShiftおよびcropShift:
precShift = Max(0, bitDepth - 12)
cropShift = Max(0, bitDepth - precShift + cntShift - 15)
ここで、LICパラメータ導出部309552は、cntShiftの値が0であればscale=32、offset=0とする。そうでなければ、以下の手順でa1、a2を導出する。
sumRR = ((sumRR + (sumRR >> 7)) >> (2 * cropShift)) << cntShift
sumRC = ((sumRC + (sumRR >> 7)) >> (2*cropShift)) << cntShift
sumRsumR = (sumR >> cropShift) * (sumR >> cropShift)
sumRsumC = (sumR >> cropShift) * (sumC >> cropShift)
a1 = sumRC - sumRsumC
a2 = sumRR - sumRsumR
（ステップ６）LICパラメータ導出部309552は、上記のa1、a2を用いて以下の手順でスケーリングファクタscaleを導出する。
shifta1 = Max(0, Ceil(Log2(Abs(a2))) - 18)
shifta2 = Max(0, Ceil(Log2(Abs(a2))) - 6)
shifta = shifta2 + 10 - shifta1
a1 = a1 >> shifta1
a2 = Clip3(0, 63, a2 >> shifta2)
LICパラメータ導出部309552は、a1、a2を用いてスケーリングファクタscaleを導出する。
scale = Clip3(0, 128, (a1 * LICDivTableFull[a2]) >> shifta)
（ステップ７）LICパラメータ導出部309552は、以下の手順でオフセットoffsetを導出する。
sumR = sumR << precShift
sumC = sumC << precShift
maxb = (1 << (bitDepth - 1)) - 1
minb = -1 - maxb
shiftS = 5
offset = Clip3(minb, maxb, (sumC - ((scale * sumR) >> shiftS)+(1 << (cntShift - 1)) >> cntShift)
LICフィルタ部309553は、scale、offsetを用いて予測画像を以下の式によって更新する。ここで、xは0からbW-1までの値をとり、yは0からbH-1までの値をとる。
predLX[x][y] = Clip1(((scale * predLX[x][y]) >> shiftS) + offset)
（scale、offsetの導出方法の変形例）
変形例では（ステップ４）～（ステップ５）とは異なる方法でスケーリングファクタscaleおよびオフセットoffsetを導出する。

(Step 5) The step in which the LIC parameter derivation unit 309552 derives the value related to the linear model for updating the predicted image will be described.
cntShift = (availL && availT)? (Log2 (numSteps) +1): Log2 (numSteps)
In the following steps, the LIC parameter derivation unit 309552 derives the feature amount a1 of the adjacent pixel of the target block and the feature amount a2 of the adjacent pixel of the reference block using the above sumR, sumC, sumRR, sumRC, contShift and cIdx as input values. do.
-Set the division table LICDivTable [k] as follows. Here, k takes a value from 0 to 63.
LICDivTableFull [0] = 0
LICDivTableFull [k] = ((1 << 15) + (k >> 1)) / k
-Variables precShift and cropShift:
precShift = Max (0, bitDepth --12)
cropShift = Max (0, bitDepth --precShift + cntShift --15)
Here, the LIC parameter derivation unit 309552 sets scale = 32 and offset = 0 if the value of cntShift is 0. If not, a1 and a2 are derived by the following procedure.
sumRR = ((sumRR + (sumRR >> 7)) >> (2 * cropShift)) << cntShift
sumRC = ((sumRC + (sumRR >> 7)) >> (2 * cropShift)) << cntShift
sumRsumR = (sumR >> cropShift) * (sumR >> cropShift)
sumRsumC = (sumR >> cropShift) * (sumC >> cropShift)
a1 = sumRC --sumRsumC
a2 = sumRR --sumRsumR
(Step 6) The LIC parameter derivation unit 309552 derives the scaling factor scale by the following procedure using the above a1 and a2.
shifta1 = Max (0, Ceil (Log2 (Abs (a2))) --18)
shifta2 = Max (0, Ceil (Log2 (Abs (a2))) -6)
shifta = shifta2 + 10 --shifta1
a1 = a1 >> shift a1
a2 = Clip3 (0, 63, a2 >> shift a2)
The LIC parameter derivation unit 309552 derives the scaling factor scale using a1 and a2.
scale = Clip3 (0, 128, (a1 * LICDivTableFull [a2]) >> shifta)
(Step 7) The LIC parameter derivation unit 309552 derives the offset offset by the following procedure.
sumR = sumR << precShift
sumC = sumC << precShift
maxb = (1 << (bitDepth ―― 1)) ―― 1
minb = -1 --maxb
shiftS = 5
offset = Clip3 (minb, maxb, (sumC-((scale * sumR) >> shiftS) + (1 << (cntShift-1)) >> cntShift)
The LIC filter unit 309553 updates the predicted image by the following formula using scale and offset. Here, x takes a value from 0 to bW-1, and y takes a value from 0 to bH-1.
predLX [x] [y] = Clip1 (((scale * predLX [x] [y]) >> shiftS) + offset)
(Modified example of derivation method of scale and offset)
In the modified example, the scaling factor scale and the offset offset are derived by a method different from (step 4) to (step 5).

(LIC parameter derivation unit 309552)
(How to derive scale)
In the derivation of scale and offset, the LIC parameter derivation unit 309552 has a point (x1, y1) where the pixel value becomes the maximum (Ref_MAX) and a point (x2) where the pixel value becomes the maximum (Ref_MAX) from the adjacent pixels R [] [] of the reference block. , y2) is derived. Next, let the pixel values of (x1, y1) and (x2, y2) on the adjacent pixels C [] [] of the target block be Cur_MAX and Cur_MIN, respectively. Here, for R [] [] and C [] [], all the adjacent pixels shown in FIG. 9 may be used, or the positions indicated by sL [k] and sT [k] in (step 3) may be used. Pixel values may be used. Then, as shown in Fig. 10 (a), R [x] [y] and C [x] [y] (-1 <= x <bW, -1 <= y <bH) are on the x-axis and y-axis, respectively. Find the straight line connecting (Ref_MAX, Cur_MAX) and (Ref_MIN, Cur_MIN) on the graph. The variables a1 and a2 for deriving the scale and offset of this straight line are derived by the following equations.

a1 = Cur_MAX --Cur_MIN (SCALE-1)
a2 = Ref_MAX --Ref_MIN
When there are a plurality of points having pixel values Ref_MAX, Ref_MIN, Cur_MAX, and Cur_MIN, for example, the average value of the plurality of points may be (x1, y1), (x2, y2).

The reference pixels (Ref_MAX, Ref_MIN) and the target pixels (Cur_MAX, Cur_MIN) when a1 and a2 are obtained are not limited to the above. For example, there are the following four modifications.

When obtaining the offset, the following derivation method may be used in addition to the method (step 7). For example, when Ref_MIN and Cur_MIN are used as the coordinates of a point through which a straight line passes, offset may be derived by the following equation.

offset = Clip3 (minb, maxb, Cur_MIN-(scale * Ref_MIN))
Alternatively, when Ref_MAX and Cur_MAX are used as the coordinates of the points through which the straight line passes, the offset may be derived by the following equation.

offset = Clip3 (minb, maxb, Cur_MAX-(scale * Ref_MAX))
It may also be as follows.

offset = Clip3 (minb, maxb, Cur_AVE-(scale * Ref_AVE))
Here, Cur_AVE and Ref_AVE are the average values of C [] [] and R [] [], respectively.

(Modification example 1 of scale derivation method)
In the first modification, as a representative point used for deriving a1 and a2, the second largest pixel value Ref_MAX2 and its coordinates (x3, y3) and the second smallest pixel value from the adjacent pixel R [] [] of the reference block. Derive Ref_MIN2 and its coordinates (x4, y4). Next, let the pixel values of (x3, y3) and (x4, y4) on the adjacent pixels C [] [] of the target block be Cur_MAX2 and Cur_MIN2, respectively.

Next, Ref_MAX2, Ref_MIN2, Cur_MAX2, and Cur_MIN2 are substituted as Ref_MAX, Ref_MIN, Cur_MAX, and Cur_MIN.

Ref_MAX = Ref_MAX2
Ref_MIN = Ref_MIN2
Cur_MAX = Cur_MAX2
Cur_MIN = Cur_MIN2
Substitute these into (SCALE-1) and derive a1 and a2. When there are multiple points with pixel values Ref_MAX, Ref_MIN, Cur_MAX, and Cur_MIN, for example, the average value of the multiple points is set as (x1, y1), (x2, y2), (x3, y3), (x4, y4). May be good.

By not using the maximum and minimum values, there is an effect of reducing the performance deterioration when the maximum and minimum values are outliers.

(Variation example 2 of the scale derivation method)
In the second modification, the maximum pixel value Ref_MAX and its coordinates (x1, y1) and the minimum pixel value Ref_MIN and its coordinates (x2) from the adjacent pixels R [] [] of the reference block as representative points used for deriving a1 and a2. , y2) The second largest pixel value Ref_MAX2 and its coordinates (x3, y3) are derived, and the second smallest pixel value Ref_MIN2 and its coordinates (x4, y4) are derived. Next, the pixel values of (x1, y1), (x2, y2), (x3, y3), and (x4, y4) on the adjacent pixels C [] [] of the target block are set to Cur_MAX, Cur_MIN, Cur_MAX2, and Cur_MIN2, respectively. do.

Substitute the average value of Ref_MAX and Ref_MAX2, the average value of Ref_MIN and Ref_MIN2, the average value of Cur_MAX and Cur_MAX2, and the average value of Cur_MIN and Cur_MIN2 as Ref_MAX, Ref_MIN, Cur_MAX, and Cur_MIN. The updated (Ref_MAX, Cur_MAX) and (Ref_MIN, Cur_MIN) are the points indicated by "X" in FIG. 10 (b).

Ref_MAX = (Ref_MAX + Ref_MAX2 + 1) >> 1
Ref_MIN = (Ref_MIN + Ref_MIN2 + 1) >> 1
Cur_MAX = (Cur_MAX + Cur_MAX2 + 1) >> 1
Cur_MIN = (Cur_MIN + Cur_MIN2 + 1) >> 1
Substituting these into (SCALE-1), a1 and a2 for deriving the scale and offset of this straight line are derived. When there are multiple points with pixel values Ref_MAX, Ref_MIN, Cur_MAX, and Cur_MIN, for example, the average value of the multiple points is set as (x1, y1), (x2, y2), (x3, y3), (x4, y4). May be good.

By using the average value, there is an effect of reducing the performance deterioration when the maximum value and the minimum value are outliers.

(Variation example 3 of the scale derivation method)
In the third modification, the maximum pixel value Ref_MAX and the minimum pixel value Ref_MIN are derived from the adjacent pixels R [] [] of the reference block as representative points used for deriving a1 and a2. Next, the maximum pixel value Cur_MAX and the minimum pixel value Cur_MIN are derived from the adjacent pixels C [] [] of the target block.

Next, find the straight line connecting (Ref_MIN, Cur_MIN) and (Ref_MAX, Cur_MAX). (Ref_MIN, Cur_MIN) and (Ref_MAX, Cur_MAX) are the points indicated by "X" in FIG. 11 (a). The variables a1 and a2 for deriving the scale and offset of this straight line can be derived by (SCALE-1).

(Variation example 4 of the scale derivation method)
In the modified example 4, as the first representative points used for deriving a1 and a2, the maximum pixel value Ref_MAX and its coordinates (x1, y1) and the minimum pixel value Ref_MIN and their coordinates from the adjacent pixels R [] [] of the reference block. (x2, y2) The second largest pixel value Ref_MAX2 and its coordinates (x3, y3) are derived, and the second smallest pixel value Ref_MIN2 and its coordinates (x4, y4) are derived. Next, the pixel values of (x1, y1), (x2, y2), (x3, y3), and (x4, y4) on the adjacent pixels C [] [] of the target block are set to Cur_MAX, Cur_MIN, Cur_MAX2, and Cur_MIN2, respectively. do.

Substitute the average value of Ref_MAX and Ref_MAX2, the average value of Ref_MIN and Ref_MIN2, the average value of Cur_MAX and Cur_MAX2, and the average value of Cur_MIN and Cur_MIN2 as Ref_MAX, Ref_MIN, Cur_MAX, and Cur_MIN.

Ref_MAX = (Ref_MAX + Ref_MAX2 + 1) >> 1
Ref_MIN = (Ref_MIN + Ref_MIN2 + 1) >> 1
Cur_MAX = (Cur_MAX + Cur_MAX2 + 1) >> 1
Cur_MIN = (Cur_MIN + Cur_MIN2 + 1) >> 1
These are the points Xmax = (Ref_MAX, Cur_MAX) and Xmin = (Ref_MIN, Cur_MIN) indicated by the white circles in FIG. 11 (b).

Next, the average value Ref_AVE of all pixels of R [] [] and the average value Cur_AVE of all pixels of C [] [] are derived.

Ref_AVE = ΣR [x] [y]
Cur_AVE = ΣC [x] [y]
Where Σ is the sum of (x, y). This is the point Xave = (Ref_AVE, Cur_AVE) indicated by the white circle in Fig. 11 (b).

Furthermore, the variables a1_1 and a2_1 for deriving scale1 of the straight line passing through Xmax and Xave are derived by (SCALE-1). Similarly, the variables a1_2 and a2_2 for deriving scale2 of the straight line passing through Xmin and Xave are derived by (SCALE-1).

(Step 6) Or the following (a modified example of the LIC parameter derivation method) is used to derive scale1 and scale2 from a1_1, a2_1 and a1_2, and a2_2. Then, the average value of scale1 and scale2 is set as scale, and the intercept of the straight line of the slope scale passing through Xave is set as offset.

scale = (scale1 + scale2 + 1) >> 1
offset = Cur_AVE --scale * Ref_AVE
The modified example 4 is more robust to outliers than the modified examples 1 to 3, and the performance is improved.

(LIC filter section 309553)
The LIC filter unit 309553 uses the reference image predLX [] [] as an input signal and outputs the updated predLX [] [] using scale, offset, and shiftS.

predLX [] [] = ((scale * predLX [] []) >> shiftS) + offset (LIC-2)
ShiftS will be described below.

(Modified example of LIC parameter derivation method)
In this processing example, the number of elements in the division table is reduced to reduce the memory. Since LICDivTableFull has a kind of repeating structure, the table size can be reduced. Derivation of LIC parameters (scaling factor scale, offset offset) using a relatively small division table LICDivTable. LICDivTable is a table corresponding to a part of LICDivTableFull and consists of 2 ^ N elements. Then, the LIC parameter derivation unit 309552 derives the element InvDiv of LICDivTableFull from the elements of the decimal LICDivTable.

FIG. 12 is a diagram illustrating an example of deriving InvDiv. In this example, N = 4, and the LICDivTable contains 16 elements. As shown in FIG. 12 (a), the LIC parameter derivation unit 309552 is the reciprocal of the divisor in the interval Gr5 [16-31] (that is, [2 ^ N .. (2 ^ (N + 1) -1)]). Store each value InvDiv for in LICDivTable [].

In the next interval Gr6 [32..63] (that is, [2 ^ (N + 1) .. (2 ^ (N + 2) -1)]), InvDiv is the value obtained by multiplying the value of LICDivTable by 1/2. Derived as. Also, since Gr6 has twice the width of Gr5, the same value is repeatedly derived twice. Furthermore, in the next interval Gr7 [64..127] (that is, [2 ^ (N + 2) .. (2 ^ (N + 3) -1)]), the value of LICDivTable is multiplied by 1/4. Is derived as InvDiv. Also, since Gr7 has four times the width of Gr5, the same value is repeatedly derived four times each. Hereinafter, the same process is repeated.

On the other hand, the LIC parameter derivation unit 309552 doubled the value of LICDivTable [] in the interval Gr4 [8..15] (that is, [2 ^ (N-1) .. (2 ^ N-1)]). Derived the value as InvDiv. Also, since Gr4 has half the width of Gr5, the elements of the table are thinned out to 2: 1 to derive. Furthermore, in the interval Gr3 [4..7] (that is, [2 ^ (N-2) .. (2 ^ (N-1) -1)]), the value of LICDivTable [] is multiplied by 4 and the value is InvDiv. Derived as. Also, since Gr3 has a width 1/4 times that of Gr5, the elements of the table are thinned out to 4: 1 to derive. Similarly, in the interval Gr2 [2..3] (that is, [2 ^ (N-3) .. (2 ^ (N-2) -1)]), the value of LICDivTable [] is multiplied by 8 each. , 8: 1 thinned out to derive. Similarly, in the interval Gr1 [1] (ie [2 ^ (N-4) .. (2 ^ (N-3) -1)]), the value of LICDivTable [] is multiplied by 16 to 16: 1. It is thinned out to and derived.

FIG. 12 (b) is a diagram summarizing these relationships. a2 is a divisor, idx is an index of LICDivTable [], and InvDiv = LICDivTable [idx].

The LIC parameter derivation unit 309552 derives the index idx for referencing the division table LICDivTable and the value InvDiv required for deriving the reciprocal of a2 from the divisor a2 derived in (step 5).

sc = floor (log2 (a2))
normDiff = (a2 << N) >> sc
idx = normDiff & ((1 << N) ―― 1)
MSB = 1 << (PREC -1)
InvDiv = (LICDivTable [idx] | MSB)
When the number of elements (2 ^ N) of the division table is 16 (N = 4) and the accuracy of the division table is PREC = 4, an example of LICDivTble [1 << N] is shown.

LICDivTable [1 << N] = {0, 7, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0}
The LIC parameter derivation unit 309552 derives the scale shift value sc from the logarithmic value of a2. In addition, normDiff is derived by shifting a2 to the left by N bits and shifting it to the right by sc bits. Also, idx is derived by subtracting a predetermined eigenvalue (16, 2 ^ N when N = 4) from normDiff. This is equivalent to the process of extracting the lower (N-1) bits of normDiff (normDiff & ((1 << N) -1)). InvDiv is derived by adding 2 ^ (PREC-1) to LICDivTable [idx].

In this modification, by referring to the division table LICDivTable (LICDivTable [idx]) with the index reduced by the shift, the number of elements of the LICDivTable is reduced and the table size is reduced. When N = 4, the LICDivTable in (step 5) stores 64 elements, but the LICDivTable in the modified example only needs to store 16 elements.

Next, the LIC parameter derivation unit 309552 derives the reciprocal of the divisor from InvDiv by the following method.

The LIC parameter derivation unit 309552 derives the variables expC and shiftA for adjusting the bit width of scale, the division number a1 and the larger value a10 of 0 from the division number a1 derived in (step 5).

a10 = Max (0, a1)
expC = floor (log2 (a10))
sc1 = sc + ((idx! = 0)? 1: 0)
shiftA = (PREC-1) + sc1 --expC
Negative slopes rarely occur in LIC. Here, a10 is set to 0 or a positive number in order to simplify the code processing.

The LIC parameter derivation unit 309552 may derive the scale using the following equation.

add1 = 1 << (shiftA + expC) >> 1
scale = ((a10 * InvDiv) + add1) >> (shiftA + expC)
In this case, the maximum bit width required for multiplication of a10 * InvDiv is expC + PREC. It is necessary to increase the scale of the multiplication circuit according to the maximum value that expC can take. In order to suppress this, the LIC parameter derivation unit 309552 may derive the scale by the following method as a configuration different from the above.

add = (1 << expC) >> 1
add1 = (1 << shiftA) >> 1
tmp = ((a10 << LSHIFT) + add) >> expC
scale = ((tmp * InvDiv) + add1) >> shiftA
LSHIFT is a parameter that indicates the accuracy of scale.

The above tmp suppresses this. First, shift a10 to the left by the LSHIFT bit, and then shift the expC bit to the right to obtain tmp. The value of tmp is the value of the upper LSHIFT bit of a10 if a10> = 1 << (LSHIFT-1), otherwise the lower bits are padded with 0s so that tmp becomes the LSHIFT bit. It becomes a value. As a result, the multiplication tmp * InvDiv at the time of deriving the scale always has the effect of suppressing the multiplication to a relatively small scale of LSHIFT * PREC bits or less. The value of LSHIFT is, for example, 5.

Note that a10 may be derived as follows instead of a10 = Max (0, a1).

a10 = abs (a1)
However, in this case, it is necessary to include the sign derivation in the scale derivation. for example
scale = (((tmp * InvDiv) + add1) >> shiftA) * sign (a1)
And.

The LIC parameter derivation unit 309552 sets shiftS.

shiftS = LSHIFT
Summarizing the above algorithms, scale can be derived by the following processing.

LICDivTable [1 << N] = {0, 7, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0}
sc = floorLog2 (a2)
normDiff = (a2 << N) >> sc
idx = normDiff & ((1 << N) ―― 1))
MSB = 1 << (PREC-1)
InvDiv = LICDivTable [idx] | MSB
sc1 = sc + (idx! = 0? 1: 0)
a10 = Max (0, a1)
expC = floorLog2 (a10)
add = 1 << expC >> 1
add1 = 1 << shiftA >> 1
shiftA = (PREC-1) + sc1 --expC
tmp = ((a10 << LSHIFT) + add) >> expC
scale = Clip3 (0, 1 << (LSHIFT + 2), ((tmp * InvDiv) + add1) >> shiftA)
shiftS = LSHIFT
The upper limit of clipping of scale may be 1 << (LSHIFT + 1) instead of 1 << (LSHIFT + 2).

scale = Clip3 (0, 1 << (LSHIFT + 1), ((tmp * InvDiv) + add1) >> shiftA)
Alternatively, it is appropriate that the upper limit of clipping is 2 to the power of -1. In this case, the bit width required for multiplication using scale is reduced by 1 bit as compared with the case where the upper limit of clipping is set to the exponent power of 2. For example, (1 << (LSHIFT + 2)) -1 may be used. When LSHIFT = 5, the maximum value of scale is 127 (6 bits), which can be reduced by 1 bit compared to the case where 128 (7 bits) is the maximum value.

scale = Clip3 (0, (1 << (LSHIFT + 2))-1, ((tmp * InvDiv) + add1) >> shiftA)
Further, the upper limit of clipping may be (1 << (LSHIFT + 1)) -1. In this case, the maximum value of scale is 63 (5 bits), and the bit width required for multiplication using scale can be reduced by 2 bits.

scale = Clip3 (0, (1 << (LSHIFT + 1))-1, ((tmp * InvDiv) + add1) >> shiftA)
floorLog2 may be derived by the following formula.

floorLog2 (x) = floor (log2 (x)) = (x <= 0)? -1: 31 --clz (x)
However, it is assumed that the argument x in the above clz function has a width of 32 bits.

The above LICDivTable was for N = 4 and PREC = 4, but the following table may be used for N = 4 and PREC = 5.

LICDivTable [1 << N] = {0,14,12,10, 9, 8, 7, 6, 5, 4, 3, 2, 2, 1, 1, 0}
Further, in the case of N = 4 and PREC = 6, the following table may be used.

LICDivTable [1 << N] = {0,28,24,21,19,16,14,12,10, 8, 7, 5, 4, 3, 2, 1}
Since the values of these LICDivTables are approximate values, tables that are not exactly the above values may be used.

Further, scale, shiftS, and offset may be set to predetermined values based on the value of the divisor a2. For example, if a2 is not a positive number, do the following:

scale = 1 << LSHIFT
shiftS = LSHIFT
Further, when a2 is equal to a1, the inclination is obviously 45 degrees, so the following may be used.

scale = 1 << LSHIFT
shiftS = LSHIFT
In these cases, offset may be derived from the equation described in the section (method of deriving scale), or may be a fixed value (for example, 0).

FIG. 13 is a data flow diagram showing the operation of the LIC parameter derivation unit 309552 of (a modified example of the LIC parameter derivation method). The LIC parameter derivation unit 309552 derives the scaling factor scale of a1 / a2 by taking a2 and a1 as inputs. Further, shiftS, which is the accuracy of scale, may be derived. Internally, the LIC parameter derivation unit 309552 may derive the shift values expC and shiftA. Here, a1 is a numerator and a2 is a denominator, and a1 may be called a numerator and a2 may be called a denom.

In addition, (variation example of the LIC parameter derivation method) may be combined with (variation example of the derivation example of scale and offset) to derive scale and offset.

(Weight prediction)
In weight prediction, a predicted image of a block is generated by multiplying PredLX by a weighting factor. When predFlagL0 or predFlagL1 is 1 (single prediction) and weight prediction is not used, the following formula processing is performed to match PredLX (LX is L0 or L1) to the number of pixel bits bitDepth.

Pred [x] [y] = Clip3 (0, (1 << bitDepth) -1, (PredLX [x] [y] + offset1) >> shift1)
Here, shift1 = 14-bitDepth, offset1 = 1 << (shift1-1).
If predFlagL0 and predFlagL1 are both 1 (bipred BiPred) and weight prediction is not used, the following formula is processed by averaging PredL0 and PredL1 to match the number of pixel bits.

Pred [x] [y] = Clip3 (0, (1 << bitDepth) -1, (PredL0 [x] [y] + PredL1 [x] [y] + offset2) >> shift2)
Here, shift2 = 15-bitDepth, offset2 = 1 << (shift2-1).

Further, in the case of simple prediction and weight prediction, the weight prediction unit 3094 derives the weight prediction coefficient w0 and the offset o0 from the coded data, and processes the following equation.

Pred [x] [y] = Clip3 (0, (1 << bitDepth) -1, ((PredLX [x] [y] * w0 + 2 ^ (log2WD-1)) >> log2WD) + o0)
Here, log2WD is a variable indicating a predetermined shift amount.

Further, in the case of double prediction BiPred and weight prediction, the weight prediction unit 3094 derives the weight prediction coefficients w0, w1, o0, and o1 from the coded data, and processes the following equation.

Pred [x] [y] = Clip3 (0, (1 << bitDepth) -1, (PredL0 [x] [y] * w0 + PredL1 [x] [y] * w1 + ((o0 + o1 + 1) <<log2WD))>> (log2WD + 1))
Then, the predicted image of the generated block is output to the addition unit 312.

The inverse quantization / inverse conversion unit 311 inversely quantizes the quantization conversion coefficient input from the entropy decoding unit 301 to obtain the conversion coefficient. This quantization transform coefficient is a coefficient obtained by performing frequency conversion such as DCT (Discrete Cosine Transform) and DST (Discrete Sine Transform) on the prediction error in the coding process. Is. The inverse quantization / inverse conversion unit 311 performs inverse frequency conversion such as inverse DCT and inverse DST on the obtained conversion coefficient, and calculates a prediction error. The inverse quantization / inverse conversion unit 311 outputs the prediction error to the addition unit 312. The inverse quantization / inverse conversion unit 311 sets all the prediction errors of the target block to 0 when skip_flag is 1 or cu_cbp is 0.

The addition unit 312 adds the prediction image of the block input from the prediction image generation unit 308 and the prediction error input from the inverse quantization / inverse conversion unit 311 for each pixel to generate a decoded image of the block. The addition unit 312 stores the decoded image of the block in the reference picture memory 306 and outputs it to the loop filter 305.

(Configuration of moving image coding device)
Next, the configuration of the moving image coding device 11 according to the present embodiment will be described. FIG. 14 is a block diagram showing the configuration of the moving image coding device 11 according to the present embodiment. The moving image coding device 11 includes a prediction image generation unit 101, a subtraction unit 102, a conversion / quantization unit 103, an inverse quantization / inverse conversion unit 105, an addition unit 106, a loop filter 107, and a prediction parameter memory (prediction parameter storage unit). , Frame memory) 108, reference picture memory (reference image storage unit, frame memory) 109, coding parameter determination unit 110, prediction parameter derivation unit 120, and entropy coding unit 104.

The prediction image generation unit 101 generates a prediction image for each CU, which is a region in which each picture of the image is divided. The predicted image generation unit 101 has the same operation as the predicted image generation unit 308 described above, and the description thereof will be omitted.

The subtraction unit 102 subtracts the pixel value of the predicted image of the block input from the prediction image generation unit 101 from the pixel value of the image to generate a prediction error. The subtraction unit 102 outputs the prediction error to the conversion / quantization unit 103.

The conversion / quantization unit 103 calculates the conversion coefficient by frequency conversion for the prediction error input from the subtraction unit 102, and derives the quantization conversion coefficient by quantization. The conversion / quantization unit 103 outputs the quantization conversion coefficient to the entropy coding unit 104 and the inverse quantization / inverse conversion unit 105.

The inverse quantization / inverse conversion unit 105 is the same as the inverse quantization / inverse conversion unit 311 (FIG. 6) in the moving image decoding device 31, and the description thereof will be omitted. The calculated prediction error is output to the addition unit 106.

A quantization conversion coefficient is input to the entropy coding unit 104 from the conversion / quantization unit 103, and a coding parameter is input from the coding parameter determination unit 110. Coding parameters include, for example, codes such as refIdxLX, mvp_LX_idx, mvdLX, predMode, and merge_idx.

The entropy coding unit 104 entropy-codes the division information, the prediction parameter, the quantization conversion coefficient, and the like to generate a coded stream Te, and outputs the coded stream Te.

The prediction parameter derivation unit 120 is a means including the inter prediction parameter derivation unit 303 and the intra prediction parameter derivation unit 304, and derives the inter prediction parameter and the intra prediction parameter from the coding parameter input from the coding parameter determination unit 110. .. The derived parameters are output to the prediction image generation unit 101 and the prediction parameter memory 108. The parameter coding unit 111 includes a header coding unit 1110, a CT information coding unit 1111, and a CU coding unit 1112 (prediction mode coding unit). The CU coding unit 1112 further includes a TU coding unit 1114.

Hereinafter, the schematic operation of each module will be described. The prediction parameter derivation unit 120 performs coding processing of parameters such as header information, division information, prediction information, and quantization conversion coefficient.

The CT information coding unit 1111 encodes QT, MT (BT, TT) division information and the like.

The CU coding unit 1112 encodes CU information, prediction information, split_transform_flag, cbf_cb, cbf_cr, cbf_luma and the like.

The TU coding unit 1114 encodes the QP update information (quantization correction value) and the quantization prediction error (residual_coding) when the TU contains a prediction error.

The CT information coding unit 1111 and the CU coding unit 1112 entropy-code the syntax elements such as inter-prediction parameters (predMode, merge_flag, merge_idx, inter_pred_idc, refIdxLX, mvp_LX_idx, mvdLX), intra-prediction parameters, and quantization conversion coefficients. Supply to unit 104.

The coding parameter determination unit 110 selects one set from a plurality of sets of coding parameters. The coding parameter is the above-mentioned QT, BT or TT division information, prediction parameter, or a parameter to be coded generated in connection with the prediction parameter. The prediction parameter derivation unit 120 derives the prediction parameter from the parameter determined by the coding parameter determination unit 110 and outputs the prediction parameter to the prediction image generation unit 101. The prediction image generation unit 101 generates a prediction image using these prediction parameters.

The coding parameter determination unit 110 calculates an RD cost value indicating the magnitude of the amount of information and the coding error for each of the plurality of sets. The RD cost value is, for example, the sum of the code amount and the value obtained by multiplying the squared error by the coefficient λ. The code amount is the amount of information of the coded stream Te obtained by entropy-coding the quantization error and the coding parameter. The square error is the sum of squares of the prediction error calculated by the subtraction unit 102. The coefficient λ is a real number greater than the preset zero. The coding parameter determination unit 110 selects the set of coding parameters that minimizes the calculated cost value. As a result, the entropy coding unit 104 outputs the selected set of coding parameters as the coding stream Te. The prediction parameters derived by the prediction parameter derivation unit 303 are stored in the prediction parameter memory 108.

The addition unit 106 adds the pixel value of the predicted image of the block input from the prediction image generation unit 101 and the prediction error input from the inverse quantization / inverse conversion unit 105 for each pixel to generate a decoded image. The addition unit 106 stores the generated decoded image in the reference picture memory 109.

The loop filter 107 applies a deblocking filter, SAO, and ALF to the decoded image generated by the addition unit 106. The loop filter 107 does not necessarily have to include the above three types of filters, and may be configured with only a deblocking filter, for example.

The prediction parameter memory 108 stores the prediction parameters generated by the coding parameter determination unit 110 at positions predetermined for each target picture and CU.

The reference picture memory 109 stores the decoded image generated by the loop filter 107 at a predetermined position for each target picture and CU.

A part of the moving image coding device 11 and the moving image decoding device 31 in the above-described embodiment, for example, the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the prediction image generation unit 308, and the inverse quantization / reverse. Conversion unit 311, Addition unit 312, Prediction parameter derivation unit 320, Prediction image generation unit 101, Subtraction unit 102, Conversion / quantization unit 103, Entropy coding unit 104, Inverse quantization / inverse conversion unit 105, Loop filter 107, The coding parameter determination unit 110, the parameter coding unit 111, and the prediction parameter derivation unit 120 may be realized by a computer. In that case, a program for realizing this control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer system and executed. The "computer system" referred to here is a computer system built in either the moving image coding device 11 or the moving image decoding device 31, and includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a medium that dynamically holds a program for a short time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In that case, a program may be held for a certain period of time, such as a volatile memory inside a computer system serving as a server or a client. Further, the above-mentioned program may be for realizing a part of the above-mentioned functions, and may be further realized for realizing the above-mentioned functions in combination with a program already recorded in the computer system.

Further, a part or all of the moving image coding device 11 and the moving image decoding device 31 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the moving image coding device 11 and the moving image decoding device 31 may be individually made into a processor, or a part or all of them may be integrated into a processor. Further, the method of making an integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, when an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology, an integrated circuit based on this technology may be used.

Although one embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to the above, and various design changes and the like are made without departing from the gist of the present invention. It is possible to do.

The LIC unit according to one aspect of the present application has the following features.

It is an LIC part that generates a predicted image by LIC prediction, and uses an adjacent pixel of the target block (first adjacent pixel), an adjacent pixel of the reference block (second adjacent pixel), and a scaling factor scale using a table. The LIC parameter derivation unit for deriving the offset offset (LIC parameter) and the LIC filter unit for generating the predicted image using the reference block image and the LIC parameter are provided, and the LIC parameter derivation unit includes the first adjacent pixel and the LIC parameter. Using the second adjacent pixel, the feature amount a1 (first feature amount) of the adjacent pixel of the target block and the feature amount a2 (second feature amount) of the adjacent pixel of the reference block are derived, and the second feature is derived. The scaling factor is derived by deriving the reference value InvDiv of the above table using the quantity a2, and shifting the value obtained by multiplying the first feature quantity a1 and the above table reference value InvDiv to the right by the first shift value (expC, shiftA). The scale is derived, and the LIC parameter derivation unit is characterized in that an offset offset is derived using a first adjacent pixel, a second adjacent pixel, and a scaling factor.

The LIC parameter derivation unit derives the scale shift value sc from the second feature quantity a2, and refers to the table by the index idx obtained by shifting the second feature quantity a2 to the right by the scale shift value sc. It is a feature.

The LIC parameter derivation unit is characterized in that the first shift value (expC, shiftA) is derived from the first feature amount a1 and the scale shift value sc.

The first shift value includes the values of the second shift value (expC) and the third shift value (shiftA), and the second shift value (expC) is derived from the first feature quantity a1 and is the third. The shift value (shiftA) of is derived by using the scale shift value sc and the second shift value (expC).

The LIC parameter derivation unit shifts the first feature amount to the left by a predetermined shift value (LSHIFT), then multiplies the value shifted to the right by the second shift value (expC) by the table reference value InvDiv, and the third It is characterized by deriving the scaling factor scale by shifting to the right with the shift value (shiftA) of.

The LIC parameter derivation unit is characterized in that the scaling factor scale is clipped.

The upper limit of clipping is derived based on a predetermined shift value (LSHIFT).

The LIC parameter derivation unit derives the maximum pixel value (Ref_MAX) coordinates (x1, y1) and the minimum pixel value (Ref_MIN) coordinates (x2, y2) from the second adjacent pixel, and Ref_MAX, Ref_MIN, the first The first feature quantity a1 and the second feature quantity a2 are derived using the pixel value (Cur_MAX) of the coordinates (x1, y1) of the adjacent pixels and the pixel value (Cur_MIN) of the coordinates (x2, y2). It is characterized by that.

The LIC parameter derivation unit derives the coordinates (x3, y3) of the second largest pixel value (Ref_MAX2) and the coordinates (x4, y4) of the second smallest pixel value (Ref_MIN2) from the second adjacent pixel. Ref_MAX2, Ref_MIN2, the pixel value (Cur_MAX2) of the coordinates (x3, y3) of the first adjacent pixel, and the pixel value (Cur_MIN2) of the coordinates (x4, y4) are used for the first feature quantity a1, the second. It is characterized by deriving the feature quantity a2.

The LIC parameter derivation unit has the maximum pixel value (Ref_MAX) coordinates (x1, y1), the minimum pixel value (Ref_MIN) coordinates (x2, y2), and the second largest pixel value (Ref_MAX2) from the second adjacent pixel. Coordinates (x3, y3), the coordinates (x4, y4) of the second smallest pixel value (Ref_MIN2) are derived, and the pixels of Ref_MAX, Ref_MIN, Ref_MAX2, Ref_MIN2, and the coordinates (x1, y1) of the first adjacent pixel. Value (Cur_MAX), pixel value (Cur_MIN) of coordinates (x2, y2), pixel value (Cur_MAX2) of coordinates (x3, y3) of the first adjacent pixel, and pixel value (Cur_MIN2) of coordinates (x4, y4) ) Is used to derive the first feature amount a1 and the second feature amount a2.

In the division part that derives the reciprocal of the divisor td using a table, the shift value sc is derived from the absolute value tdAbs of the divisor td, and the absolute value tdAbs of the divisor td is shifted to the right by the sc bit (tdAbs << N) >> sc. The index idx is derived by modifying the value normDiff using the number N of elements in the table MvDivTable, and the value of the table MvDivTable referenced by the index idx, the reciprocal precision PREC, and the reciprocal based on the shift value sc. It is characterized by deriving the absolute value and deriving the reciprocal based on the sign of the divisor.

In a motion vector derivation device that derives the motion vector of the target picture (first motion vector) with respect to the first reference picture by scaling the motion vector (second motion vector) of the target picture with respect to the second reference picture. , The difference value of the POC of the target picture and the first reference picture is used as a divisor, and the inverse of the difference value is derived by the dividing unit, and the difference value of the POC of the second motion vector, the target picture and the second reference picture is derived. , And the first motion vector is derived by multiplying the inverse number.

[Application example]
The moving image coding device 11 and the moving image decoding device 31 described above can be mounted on and used in various devices that transmit, receive, record, and reproduce moving images. The moving image may be a natural moving image captured by a camera or the like, or an artificial moving image (including CG and GUI) generated by a computer or the like.

First, it will be described with reference to FIG. 2 that the above-mentioned moving image coding device 11 and moving image decoding device 31 can be used for transmission and reception of moving images.

FIG. 2A is a block diagram showing the configuration of the transmission device PROD_A equipped with the moving image coding device 11. As shown in the figure, the transmitter PROD_A has a coding unit PROD_A1 that obtains coded data by encoding a moving image, and a modulation signal by modulating a carrier wave with the coded data obtained by the coding unit PROD_A1. It includes a modulation unit PROD_A2 to obtain and a transmission unit PROD_A3 to transmit the modulation signal obtained by the modulation unit PROD_A2. The moving image coding device 11 described above is used as the coding unit PROD_A1.

The transmitter PROD_A has a camera PROD_A4 for capturing a moving image, a recording medium PROD_A5 for recording a moving image, an input terminal PROD_A6 for inputting a moving image from the outside, and a video input terminal PROD_A6 as a source of the moving image to be input to the coding unit PROD_A1. , An image processing unit A7 for generating or processing an image may be further provided. In the figure, the configuration in which the transmitter PROD_A is provided with all of these is illustrated, but some of them may be omitted.

The recording medium PROD_A5 may be a recording of an unencoded moving image, or a moving image encoded by a recording coding method different from the transmission coding method. It may be a thing. In the latter case, a decoding unit (not shown) that decodes the coded data read from the recording medium PROD_A5 according to the coding method for recording may be interposed between the recording medium PROD_A5 and the coding unit PROD_A1.

FIG. 2B is a block diagram showing the configuration of the receiving device PROD_B equipped with the moving image decoding device 31. As shown in the figure, the receiving device PROD_B is obtained by a receiving unit PROD_B1 that receives a modulated signal, a demodulation unit PROD_B2 that obtains coded data by demodulating the modulated signal received by the receiving unit PROD_B1, and a demodulation unit PROD_B2. It includes a decoding unit PROD_B3 that obtains a moving image by decoding the coded data. The moving image decoding device 31 described above is used as the decoding unit PROD_B3.

The receiving device PROD_B is a display PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording a moving image, and an output terminal for outputting the moving image to the outside as a supply destination of the moving image output by the decoding unit PROD_B3. It may be further equipped with PROD_B6. In the figure, the configuration in which the receiving device PROD_B includes all of them is illustrated, but some of them may be omitted.

The recording medium PROD_B5 may be used for recording an unencoded moving image, or may be encoded by a recording coding method different from the transmission coding method. You may. In the latter case, it is advisable to interpose a coding unit (not shown) that encodes the moving image acquired from the decoding unit PROD_B3 according to the coding method for recording between the decoding unit PROD_B3 and the recording medium PROD_B5.

The transmission medium for transmitting the modulated signal may be wireless or wired. Further, the transmission mode for transmitting the modulated signal may be broadcasting (here, a transmission mode in which the destination is not specified in advance) or communication (here, transmission in which the destination is specified in advance). It may refer to an aspect). That is, the transmission of the modulated signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

For example, a broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) of terrestrial digital broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by radio broadcasting. Further, a broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by cable broadcasting.

In addition, servers (workstations, etc.) / clients (television receivers, personal computers, smartphones, etc.) such as VOD (Video On Demand) services and video sharing services using the Internet are transmitters that send and receive modulated signals via communication. This is an example of PROD_A / receiver PROD_B (usually, in LAN, either wireless or wired is used as a transmission medium, and in WAN, wired is used as a transmission medium). Here, personal computers include desktop PCs, laptop PCs, and tablet PCs. Smartphones also include multifunctional mobile phone terminals.

The client of the video sharing service has a function of decoding the coded data downloaded from the server and displaying it on the display, as well as a function of encoding the moving image captured by the camera and uploading it to the server. That is, the client of the video sharing service functions as both the transmitting device PROD_A and the receiving device PROD_B.

Next, it will be described with reference to FIG. 3 that the above-mentioned moving image coding device 11 and moving image decoding device 31 can be used for recording and reproducing moving images.

FIG. 3A is a block diagram showing the configuration of the recording device PROD_C equipped with the above-mentioned moving image coding device 11. As shown in the figure, the recording device PROD_C has a coding unit PROD_C1 that obtains coded data by encoding a moving image and a writing unit PROD_C2 that writes the coded data obtained by the coding unit PROD_C1 to the recording medium PROD_M. And have. The moving image coding device 11 described above is used as the coding unit PROD_C1.

The recording medium PROD_M may be of a type built in the recording device PROD_C, such as (1) HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) SD memory. It may be of a type connected to the recording device PROD_C, such as a card or USB (Universal Serial Bus) flash memory, and (3) DVD (Digital Versatile Disc: registered trademark) or BD (Blu-ray). It may be loaded into a drive device (not shown) built in the recording device PROD_C, such as Disc (registered trademark).

Further, the recording device PROD_C has a camera PROD_C3 that captures a moving image, an input terminal PROD_C4 for inputting a moving image from the outside, and a reception for receiving the moving image as a source of the moving image to be input to the coding unit PROD_C1. The unit PROD_C5 and the image processing unit PROD_C6 for generating or processing an image may be further provided. In the figure, the configuration in which the recording device PROD_C is provided with all of these is illustrated, but some of them may be omitted.

The receiving unit PROD_C5 may receive an unencoded moving image, or receives coded data encoded by a coding method for transmission different from the coding method for recording. It may be something to do. In the latter case, a transmission decoding unit (not shown) that decodes the coded data encoded by the transmission coding method may be interposed between the receiving unit PROD_C5 and the coding unit PROD_C1.

Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, an HDD (Hard Disk Drive) recorder, and the like (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is the main source of moving images). .. In addition, a camcorder (in this case, the camera PROD_C3 is the main source of the moving image), a personal computer (in this case, the receiving unit PROD_C5 or the image processing unit C6 is the main source of the moving image), a smartphone (this). In this case, the camera PROD_C3 or the receiver PROD_C5 is the main source of moving images) is also an example of such a recording device PROD_C.

FIG. 3B is a block showing the configuration of the reproduction device PROD_D equipped with the above-mentioned moving image decoding device 31. As shown in the figure, the reproduction device PROD_D includes a reading unit PROD_D1 that reads the coded data written in the recording medium PROD_M, and a decoding unit PROD_D2 that obtains a moving image by decoding the coded data read by the reading unit PROD_D1. , Is equipped. The moving image decoding device 31 described above is used as the decoding unit PROD_D2.

The recording medium PROD_M may be of a type built in the playback device PROD_D, such as (1) HDD or SSD, or (2) such as an SD memory card or USB flash memory. It may be of a type connected to the playback device PROD_D, or may be loaded into a drive device (not shown) built in the playback device PROD_D, such as (3) DVD or BD. good.

Further, the playback device PROD_D has a display PROD_D3 for displaying a moving image, an output terminal PROD_D4 for outputting the moving image to the outside, and a transmitting unit for transmitting the moving image as a supply destination of the moving image output by the decoding unit PROD_D2. It may also have PROD_D5. In the figure, the configuration in which the reproduction device PROD_D is provided with all of these is illustrated, but some of them may be omitted.

The transmission unit PROD_D5 may transmit an unencoded moving image, or transmits coded data encoded by a coding method for transmission different from the coding method for recording. It may be something to do. In the latter case, it is preferable to interpose a coding unit (not shown) that encodes the moving image by a coding method for transmission between the decoding unit PROD_D2 and the transmission unit PROD_D5.

Examples of such a playback device PROD_D include a DVD player, a BD player, an HDD player, and the like (in this case, the output terminal PROD_D4 to which a television receiver or the like is connected is the main supply destination of the moving image). .. In addition, a television receiver (in this case, display PROD_D3 is the main supply destination of moving images) and digital signage (also called electronic signage or electronic bulletin board, etc., and display PROD_D3 or transmitter PROD_D5 is the main supply destination of moving images. (First), desktop PC (in this case, output terminal PROD_D4 or transmitter PROD_D5 is the main supply destination of moving images), laptop or tablet PC (in this case, display PROD_D3 or transmitter PROD_D5 is video) An example of such a playback device PROD_D is a smartphone (in this case, the display PROD_D3 or the transmitter PROD_D5 is the main supply destination of the moving image), which is the main supply destination of the image.

(Hardware realization and software realization)
Further, each block of the moving image decoding device 31 and the moving image coding device 11 described above may be realized by hardware by a logic circuit formed on an integrated circuit (IC chip), or may be realized by a CPU (Central Processing). It may be realized by software using Unit).

In the latter case, each of the above devices is a CPU that executes an instruction of a program that realizes each function, a ROM (Read Only Memory) that stores the above program, a RAM (Random Access Memory) that expands the above program, the above program, and various types. It is equipped with a storage device (recording medium) such as a memory for storing data. Then, an object of the embodiment of the present invention is a recording in which the program code (execution format program, intermediate code program, source program) of the control program of each of the above-mentioned devices, which is software for realizing the above-mentioned function, is readablely recorded by a computer. It can also be achieved by supplying the medium to each of the above devices and having the computer (or CPU or MPU) read and execute the program code recorded on the recording medium.

Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic discs such as floppy (registered trademark) discs / hard disks, and CD-ROMs (Compact Disc Read-Only Memory) / MO discs (Magneto-Optical discs). ) / MD (Mini Disc) / DVD (Digital Versatile Disc: registered trademark) / CD-R (CD Recordable) / Blu-ray Disc (registered trademark) and other optical discs, IC cards (memory cards) (Including) / Optical cards and other cards, mask ROM / EPROM (Erasable Programmable Read-Only Memory) / EEPROM (Electrically Erasable and Programmable Read-Only Memory: registered trademark) / Flash ROM and other semiconductor memories, or PLD ( Logic circuits such as Programmable logic device) and FPGA (Field Programmable Gate Array) can be used.

Further, each of the above devices may be configured to be connectable to a communication network, and the above program code may be supplied via the communication network. This communication network is not particularly limited as long as it can transmit the program code. For example, Internet, Intranet, Extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community Antenna television / Cable Television) communication network, Virtual Private network (Virtual Private). Network), telephone line network, mobile communication network, satellite communication network, etc. can be used. Further, the transmission medium constituting this communication network may be any medium as long as it can transmit the program code, and is not limited to a specific configuration or type. For example, even wired such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared ray such as IrDA (Infrared Data Association) and remote control. , BlueTooth (registered trademark), IEEE802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance: registered trademark), mobile phone network, satellite line, terrestrial digital broadcasting network, etc. It can also be used wirelessly. The embodiment of the present invention can also be realized in the form of a computer data signal embedded in a carrier wave, in which the program code is embodied by electronic transmission.

The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the claims. That is, an embodiment obtained by combining technical means appropriately modified within the scope of the claims is also included in the technical scope of the present invention.

The embodiment of the present invention is suitably applied to a moving image decoding device that decodes coded data in which image data is encoded, and a moving image coding device that generates coded data in which image data is encoded. be able to. Further, it can be suitably applied to the data structure of the coded data generated by the moving image coding device and referenced by the moving image decoding device.

31 Image decoder
301 Entropy Decoding Unit
302 Parameter decoder
3020 Header decoder
303 Inter prediction parameter derivation part
308 Prediction image generator
309 Inter prediction image generator
311 Inverse quantization / inverse conversion unit
312 Addition part
11 Image coding device
101 Predictive image generator
102 Subtractor
103 Conversion / Quantization Department
104 Entropy coding unit
105 Inverse quantization / inverse conversion unit
107 Loop filter
110 Coded parameter determination unit
111 Parameter encoding unit
1110 Header encoding
1111 CT information coding unit
1112 CU coding unit (prediction mode coding unit)
1114 TU coder
120 Prediction parameter derivation unit

Claims (12)

It is a LIC part that generates a predicted image by LIC prediction,
Adjacent pixels of the target block (first adjacent pixel), adjacent pixels of the reference block (second adjacent pixel), and a LIC parameter derivation unit that derives the scaling factor scale and offset offset (LIC parameter) using a table. ,
It is equipped with a reference block image and a LIC filter unit that generates a predicted image using LIC parameters.
The LIC parameter derivation unit uses the first adjacent pixel and the second adjacent pixel, and uses the feature amount a1 (first feature amount) of the adjacent pixel of the target block and the feature amount a2 (first feature amount) of the adjacent pixel of the reference block. Derivation of the feature amount of 2),
Using the second feature amount a2, the reference value InvDiv in the above table is derived.
The scaling factor scale is derived by right-shifting the value obtained by multiplying the first feature quantity a1 by the above table reference value InvDiv by the first shift value (expC, shiftA).
The LIC parameter derivation unit is a LIC unit characterized by deriving an offset offset using a first adjacent pixel, a second adjacent pixel, and a scaling factor. The above LIC parameter derivation unit
The scale shift value sc is derived from the second feature quantity a2, and the scale shift value sc is derived.
The LIC portion according to claim 1, wherein the above table is referred to by the index idx obtained by right-shifting the second feature amount a2 by the scale shift value sc. The LIC unit according to claim 1 or 2, wherein the LIC parameter derivation unit derives the first shift value (expC, shiftA) from the first feature amount a1 and the scale shift value sc. The first shift value includes the values of the second shift value (expC) and the third shift value (shiftA), and the second shift value (expC) is derived from the first feature quantity a1 and is the third. The LIC portion according to any one of claims 1 to 3, wherein the shift value (shiftA) is derived by using the scale shift value sc and the second shift value (expC). The LIC parameter derivation unit shifts the first feature amount to the left by a predetermined shift value (LSHIFT), then multiplies the value shifted to the right by the second shift value (expC) by the table reference value InvDiv, and the third The LIC portion according to any one of claims 1 to 4, wherein the scaling factor scale is derived by right-shifting with the shift value (shiftA) of. The LIC unit according to claim 5, wherein the LIC parameter derivation unit clips the scaling factor scale according to claim 5. The LIC portion according to claim 6, wherein the upper limit of clipping is derived based on a predetermined shift value (LSHIFT). The LIC parameter derivation unit derives the maximum pixel value (Ref_MAX) coordinates (x1, y1) and the minimum pixel value (Ref_MIN) coordinates (x2, y2) from the second adjacent pixel, and Ref_MAX, Ref_MIN, the first The first feature quantity a1 and the second feature quantity a2 are derived using the pixel value (Cur_MAX) of the coordinates (x1, y1) of the adjacent pixels and the pixel value (Cur_MIN) of the coordinates (x2, y2). The LIC unit according to claim 1, wherein the LIC unit is characterized by the above. The LIC parameter derivation unit derives the coordinates (x3, y3) of the second largest pixel value (Ref_MAX2) and the coordinates (x4, y4) of the second smallest pixel value (Ref_MIN2) from the second adjacent pixel. Ref_MAX2, Ref_MIN2, the pixel value (Cur_MAX2) of the coordinates (x3, y3) of the first adjacent pixel, and the pixel value (Cur_MIN2) of the coordinates (x4, y4) are used for the first feature quantity a1, the second. The LIC unit according to claim 1, wherein the feature amount a2 is derived. The LIC parameter derivation unit has the maximum pixel value (Ref_MAX) coordinates (x1, y1), the minimum pixel value (Ref_MIN) coordinates (x2, y2), and the second largest pixel value (Ref_MAX2) from the second adjacent pixel. Coordinates (x3, y3), the coordinates (x4, y4) of the second smallest pixel value (Ref_MIN2) are derived, and the pixels of Ref_MAX, Ref_MIN, Ref_MAX2, Ref_MIN2, and the coordinates (x1, y1) of the first adjacent pixel. Value (Cur_MAX), pixel value (Cur_MIN) of coordinates (x2, y2), pixel value (Cur_MAX2) of coordinates (x3, y3) of the first adjacent pixel, and pixel value (Cur_MIN2) of coordinates (x4, y4) The LIC unit according to claim 1, wherein the first feature amount a1 and the second feature amount a2 are derived using). An image decoding device that decodes an image by adding a residual to a predicted image derived from the LIC unit according to any one of claims 1 to 10. An image coding device that derives and encodes a residual from a difference between a predicted image derived from the LIC unit according to any one of claims 1 to 10 and an input image.

Images