JP2023177425A - Moving image decoding device, moving image encoding device, and angle mode derivation device - Google Patents

Abstract

To dispense with a search for converting the gradient of a pixel value to an intra direction prediction mode, the search having interfered with concurrent processing in conventional decoder-side intra-mode derivation (DIMD) prediction.SOLUTION: A moving image decoding device comprises: a gradient derivation unit which derives the gradient of a pixel value in a block adjacent to the top and the left of an object block; an angle mode derivation unit which derives an angle mode for deriving an angle mode corresponding to the gradient; an angle mode selection unit which selects the derived angle mode, and a temporal prediction image derivation unit which derives a temporal prediction image based on the angle mode. The device refers to a table including the angle mode corresponding to the gradient and thereby derives the angle mode.SELECTED DRAWING: Figure 6

Description

Embodiments of the present invention relate to a moving image decoding device, a moving image encoding device, and an angular mode deriving device.

In order to efficiently transmit or record moving images, a moving image encoding device generates encoded data by encoding a moving image, and a moving image generates a decoded image by decoding the encoded data. An image decoding device is used.

Specific video encoding methods include, for example, methods proposed in H.264/AVC and HEVC (High-Efficiency Video Coding).

In such a video encoding method, the images (pictures) that make up a video are divided into slices obtained by dividing the image and coding tree units (CTUs) obtained by dividing the slices. ), a coding unit obtained by dividing a coding tree unit (sometimes called a coding unit (CU)), and a transform unit (TU: obtained by partitioning a coding unit). It is managed by a hierarchical structure consisting of CUs (Transform Units), and is encoded/decoded for each CU.

In addition, in such a video encoding method, a predicted image is usually generated based on a locally decoded image obtained by encoding/decoding the input image, and the predicted image is generated from the input image (original image). The prediction error obtained by subtraction (sometimes referred to as a "difference image" or "residual image") is encoded. Methods for generating predicted images include inter-screen prediction (inter prediction) and intra-screen prediction (intra prediction).

Furthermore, non-patent document 1 can be cited as a recent technique for video encoding and decoding. Non-Patent Document 1 discloses decoder-side intra mode derivation (DIMD) prediction in which a decoder derives a predicted image by deriving an intra direction prediction mode number using pixels in an adjacent region. ing.

M. Abdoli, T. Guionnet, E. Mora, et. al, “Non-CE3: Decoder-side Intra Mode Derivation with Prediction Fusion Using Planar”, JVET-O0449, Gothenburg, July 2019.

In decoder-side intra mode derivation as in Non-Patent Document 1, the intra direction prediction mode in the target block is estimated using the gradient of pixel values in the reference area. At this time, an array search is performed to convert the direction of the gradient to the intra direction prediction mode, resulting in an increase in the amount of processing.

An object of the present invention is to perform suitable intra-mode derivation on the decoder side without increasing the amount of processing for deriving the intra-mode from the gradient.

In order to solve the above problems, a moving image decoding device according to one aspect of the present invention includes:
A gradient derivation unit that derives the gradient of pixel values in blocks adjacent to the top and left of the target block, an angular mode derivation unit that derives an angular mode corresponding to the gradient, and an angular mode selection unit that selects the derived angular mode. and a temporary predicted image derivation unit that derives a temporary predicted image based on the angle mode.
A moving image decoding device and an angular mode deriving device comprising: a moving image decoding device and an angular mode deriving device, each of which is characterized by deriving an angular mode corresponding to a gradient based on table reference.

According to one aspect of the present invention, suitable intra prediction can be performed without increasing the amount of calculation for intra mode derivation on the decoder side.

1 is a schematic diagram showing the configuration of an image transmission system according to the present embodiment. FIG. 3 is a diagram showing a hierarchical structure of data of an encoded stream. It is a schematic diagram showing the types (mode numbers) of intra prediction modes. 1 is a schematic diagram showing the configuration of a moving image decoding device. FIG. 2 is a diagram showing the configuration of an intra predicted image generation section. FIG. 3 is a diagram showing details of a DIMD prediction unit. This is an example of DIMD syntax. This is an example of a spatial filter. FIG. 3 is a diagram illustrating an example of gradient derivation target pixels. FIG. 3 is a diagram showing the relationship between a gradient and a region. FIG. 1 is a block diagram showing the configuration of a video encoding device.

(First embodiment)
Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a schematic diagram showing the configuration of an image transmission system 1 according to this embodiment.

The image transmission system 1 is a system that transmits an encoded stream obtained by encoding an image to be encoded, decodes the transmitted encoded stream, and displays the image. The image transmission system 1 includes a video encoding device (image encoding device) 11, a network 21, a video decoding device (image decoding device) 31, and a video display device (image display device) 41. .

An image T is input to the moving image encoding device 11.

The network 21 transmits the encoded stream Te generated by the video encoding device 11 to the video decoding device 31. The network 21 is the Internet, a wide area network (WAN), a local area network (LAN), or a combination thereof. The network 21 is not necessarily a bidirectional communication network, but may be a unidirectional communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. Further, the network 21 may be replaced by a storage medium on which the encoded stream Te is recorded, such as a DVD (Digital Versatile Disc: registered trademark) or a BD (Blu-ray Disc: registered trademark).

The video decoding device 31 decodes each encoded stream Te transmitted by the network 21, and generates one or more decoded images Td.

The moving image display device 41 displays all or part of one or more decoded images Td generated by the moving image decoding device 31. The moving image display device 41 includes a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. Display formats include stationary, mobile, HMD, etc. Further, when the video decoding device 31 has high processing capacity, it displays a high quality image, and when it has only a lower processing capacity, it displays an image that does not require high processing capacity or display capacity. .

<Operator>
The operators used in this specification are described below.

>> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR
, ^ is bitwise XOR, |= is OR assignment operator, ! is logical negation (NOT), && is logical product (AND)
, || indicates logical sum (OR).

x?y:z is a ternary operator that takes y if x is true (other than 0) and z if x is false (0).

Clip3(a,b,c) is a function that clips c to a value greater than or equal to a and less than or equal to b, and returns a if c<a, returns b if c>b, and otherwise is a function that returns c (where a<=b).

Clip1Y(c) is an operator set to a=0 and b=(1<<BitDepthY)-1 in Clip3(a,b,c). BitDepthY is the brightness bit depth.

abs(a) is a function that returns the absolute value of a.

Int(a) is a function that returns the integer value of a.

Floor(a) is a function that returns the largest integer less than or equal to a.

Ceil(a) is a function that returns the smallest integer greater than or equal to a.

a/d represents the division of a by d (rounding down to the nearest whole number).

Min(a,b) is a function that returns the smaller value of a and b.

<Structure of encoded stream Te>
Prior to a detailed explanation of the video encoding device 11 and the video decoding device 31 according to the present embodiment, data of the encoded stream Te generated by the video encoding device 11 and decoded by the video decoding device 31 will be described. Explain the structure.

FIG. 2 is a diagram showing the hierarchical structure of data in the encoded stream Te. The encoded stream Te exemplarily includes a sequence and a plurality of pictures that constitute the sequence. FIG. 2 shows an encoded video sequence that defines the sequence SEQ, an encoded picture that defines the picture PICT, an encoded slice that defines the slice S, encoded slice data that defines the slice data, and encoded slice data. A diagram illustrating included encoding tree units and encoding units included in the encoding tree units is shown.

(encoded video sequence)
In the encoded video sequence, a set of data that the moving image decoding device 31 refers to in order to decode the sequence SEQ to be processed is defined. As shown in the encoded video sequence of FIG. 2, the sequence SEQ includes a video parameter set VPS (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and , Supplemental Enhancement Information (SEI) is included.

Video parameter set VPS is a set of encoding parameters common to multiple video images and encoding parameters related to multiple layers and individual layers included in the video image. A set is defined.

The sequence parameter set SPS defines a set of encoding parameters that the video decoding device 31 refers to in order to decode the target sequence. For example, the width and height of the picture are defined. Note that a plurality of SPSs may exist. In that case, select one of the multiple SPSs from the PPSs.

The picture parameter set PPS defines a set of encoding parameters that the video decoding device 31 refers to in order to decode each picture in the target sequence. For example, it includes a reference value for the quantization width used in picture decoding (pic_init_qp_minus26) and a flag indicating application of weighted prediction (weighted_pred_flag). Note that multiple PPSs may exist. In that case, one of the plurality of PPSs is selected from each picture in the target sequence.

(encoded picture)
In the encoded picture, a set of data that the video decoding device 31 refers to in order to decode the picture PICT to be processed is defined. Picture PICT includes slice 0 to slice NS-1 (NS is the total number of slices included in picture PICT), as shown in the encoded picture in Figure 2.
.

Note that hereinafter, if there is no need to distinguish each of slices 0 to NS-1, the subscripts of the symbols may be omitted in the description. Further, the same applies to other data included in the encoded stream Te described below and having subscripts attached thereto.

(encoded slice)
In the encoded slice, a set of data that the video decoding device 31 refers to in order to decode the slice S to be processed is defined. The slice includes a slice header and slice data, as shown in the encoded slice of FIG. 2.

The slice header includes a group of encoding parameters that the video decoding device 31 refers to in order to determine the decoding method for the target slice. Slice type designation information (slice_type) that designates the slice type is an example of an encoding parameter included in the slice header.

Slice types that can be specified by the slice type designation information include (1) an I slice that uses only intra prediction during encoding, (2) a P slice that uses unidirectional prediction or intra prediction during encoding, (3) Examples include B slices that use unidirectional prediction, bidirectional prediction, or intra prediction during encoding. Note that inter prediction is not limited to uni-prediction or bi-prediction, and a predicted image may be generated using more reference pictures. Below, P
, B slice refers to a slice that includes blocks for which inter prediction can be used.

Note that the slice header may include a reference (pic_parameter_set_id) to the picture parameter set PPS.

(encoded slice data)
The encoded slice data defines a set of data that the video decoding device 31 refers to in order to decode the slice data to be processed. The slice data includes a CTU, as shown in the encoded slice header of FIG. 2. A CTU is a block of fixed size (for example, 64x64) that constitutes a slice, and is also called a largest coding unit (LCU).

(encoding tree unit)
The encoding tree unit in FIG. 2 includes a video decoding device 31 for decoding the CTU to be processed.
The set of data referenced by is defined. CTU is the basic encoding process using recursive quad tree partitioning (QT (Quad Tree) partitioning), binary tree partitioning (BT (Binary Tree) partitioning), or ternary tree partitioning (TT (Ternary Tree) partitioning). It is divided into encoding units CU, which are standard units. The combination of BT partitioning and TT partitioning is called multi-tree partitioning (MT (Multi Tree) partitioning). A tree-structured node obtained by recursive quadtree partitioning is called a coding node. An intermediate node of a quadtree, a binary tree, and a tertiary tree is a coding node, and the CTU itself is defined as the topmost coding node.

(encoding unit)
As shown in the encoding unit of FIG. 2, a set of data that the video decoding device 31 refers to in order to decode the encoding unit to be processed is defined. Specifically, the CU includes a CU header CUH, prediction parameters, transformation parameters, quantized transformation coefficients, and the like. The prediction mode etc. are defined in the CU header.

The prediction process may be performed on a CU basis or on a sub-CU basis, which is obtained by further dividing a CU. If the sizes of the CU and sub-CU are equal, there is one sub-CU in the CU. If the CU is larger than the sub-CU size, the CU is divided into sub-CUs. For example, if the CU is 8x8 and the sub-CU is 4x4, the CU is divided into four sub-CUs, consisting of two horizontal divisions and two vertical divisions.

There are two types of prediction (prediction modes): intra prediction and inter prediction. Intra prediction is prediction within the same picture, and inter prediction refers to prediction processing performed between mutually different pictures (for example, between display times, between layer images).

Although the transform/quantization process is performed in units of CUs, the quantized transform coefficients may be entropy encoded in units of subblocks such as 4x4.

(prediction parameter)
A predicted image is derived by prediction parameters associated with a block. The prediction parameters include intra prediction and inter prediction parameters.

The prediction parameters for intra prediction will be explained below. The intra prediction parameters are composed of a luminance prediction mode IntraPredModeY and a color difference prediction mode IntraPredModeC. FIG. 3 is a schematic diagram showing types (mode numbers) of intra prediction modes. As shown in the figure, there are, for example, 67 types (0 to 66) of intra prediction modes. For example, planar prediction (0), DC prediction (1), and Angular prediction (2 to 66). In addition, linear model (LM) prediction such as color component linear model (CCLM) prediction or multimode linear model (MMLM) prediction may be used. Furthermore, an LM mode may be added for color difference.

(Configuration of moving image decoding device)
The configuration of the moving image decoding device 31 (FIG. 4) according to this embodiment will be explained.

The video decoding device 31 includes an entropy decoding section 301, a parameter decoding section (predicted image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a predicted image generation section (predicted image generation device) 308, and an inverse It is configured to include a quantization/inverse transformation section 311 and an addition section 312. Note that there is also a configuration in which the loop filter 305 is not included in the video decoding device 31 in accordance with the video encoding device 11 described later.

Further, the parameter decoding unit 302 is configured to include an inter prediction parameter decoding unit 303 and an intra prediction parameter decoding unit 304 (not shown). The predicted image generation section 308 includes an inter predicted image generation section 309 and an intra predicted image generation section 310.

Furthermore, although an example will be described below in which CTUs and CUs are used as processing units, the processing is not limited to this example, and processing may be performed in sub-CU units. Alternatively, CTU and CU may be read as blocks and sub-CUs as sub-blocks, and processing may be performed in units of blocks or sub-blocks.

The entropy decoding unit 301 performs entropy decoding on the encoded stream Te input from the outside, and separates and decodes individual codes (syntax elements). Entropy encoding has two methods: variable-length encoding of syntax elements using a context (probabilistic model) adaptively selected according to the type of syntax element and surrounding situation, and a method of encoding syntax elements with variable length using a predetermined table or There is a method of variable length encoding of syntax elements using calculation formulas. The former, CABAC (Context Adaptive Binary Arithmetic Coding), stores in memory an updated probability model for each encoded or decoded picture (slice). And P picture,
Alternatively, as the initial state of the B-picture context, a picture probability model using quantization parameters of the same slice type and slice level is set from among the probability models stored in the memory. This initial state is used for encoding and decoding processing. The separated codes include prediction information for generating a predicted image, prediction errors for generating a difference image, and the like.

Entropy decoding section 301 outputs the separated codes to parameter decoding section 302. Control of which code to decode is performed based on instructions from parameter decoding section 302.

(Configuration of intra prediction parameter decoding unit 304)
The intra prediction parameter decoding unit 304 decodes the intra prediction parameter, for example, the intra prediction mode IntraPredMode, based on the code input from the entropy decoding unit 301, with reference to the prediction parameters stored in the prediction parameter memory 307. The intra prediction parameter decoding unit 304 outputs the decoded intra prediction parameters to the predicted image generation unit 308 and stores them in the prediction parameter memory 307. The intra prediction parameter decoding unit 304 may derive different intra prediction modes for luminance and color difference.

Intra prediction parameter decoding section 304 decodes syntax elements related to intra prediction as shown in FIG.

The loop filter 305 is a filter provided in the encoding loop, and is a filter that removes block distortion and ringing distortion and improves image quality. The loop filter 305 applies filters such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the addition unit 312.

The reference picture memory 306 stores the decoded image of the CU generated by the addition unit 312 at a predetermined position for each target picture and target CU.

The prediction parameter memory 307 stores prediction parameters in a predetermined position for each CTU or CU to be decoded. Specifically, the prediction parameter memory 307 stores the parameters decoded by the parameter decoding unit 302 and the prediction mode predMode separated by the entropy decoding unit 301.

A prediction mode predMode, prediction parameters, etc. are input to the predicted image generation unit 308. The predicted image generation unit 308 also reads a reference picture from the reference picture memory 306. The predicted image generation unit 308 generates a predicted image of a block or subblock using the prediction parameter and the read reference picture (reference picture block) in the prediction mode indicated by the prediction mode predMode. Here, the reference picture block is a set of pixels on a reference picture (usually called a block because it is rectangular), and is an area to be referenced to generate a predicted image.

(Intra predicted image generation unit 310)
When the prediction mode predMode indicates an intra prediction mode, the intra prediction image generation unit 310 performs intra prediction using the intra prediction parameters input from the intra prediction parameter decoding unit 304 and the reference pixels read from the reference picture memory 306.

Specifically, the intra predicted image generation unit 310 reads adjacent blocks on the target picture within a predetermined range from the target block from the reference picture memory 306. The predetermined range is adjacent blocks to the left, upper left, upper, and upper right of the target block, and the reference area differs depending on the intra prediction mode.

The intra predicted image generation unit 310 generates a predicted image of the target block by referring to the read decoded pixel value and the prediction mode indicated by IntraPredMode. The intra predicted image generation unit 310 outputs the generated predicted image of the block to the addition unit 312.

Generation of predicted images based on intra prediction mode will be described below. In planar prediction, DC prediction, and Angular prediction, a decoded peripheral area adjacent to (close to) the prediction target block is set as the reference area R. Then, a predicted image is generated by extrapolating pixels on the reference area R in a specific direction. For example, the reference region R may be set as an L-shaped region including the left and above (or further upper left, upper right, and lower left) of the prediction target block.

(Details of predicted image generation unit)
Next, details of the configuration of the intra predicted image generation unit 310 will be described using FIG. 5. The intra predicted image generation unit 310 includes a reference sample filter unit 3103 (second reference image setting unit), a prediction unit 3104
, and a predicted image correction unit 3105 (predicted image correction unit, filter switching unit, weighting coefficient changing unit).

Based on each reference pixel (reference image) on the reference area R, the filtered reference image generated by applying the reference pixel filter (first filter), and the intra prediction mode, the prediction unit 3104 makes a tentative prediction of the prediction target block. An image (pre-correction predicted image) is generated and output to the predicted image correction unit 3105. The predicted image correction unit 3105 corrects the temporary predicted image according to the intra prediction mode, generates a predicted image (corrected predicted image), and outputs it.

Each unit included in the intra predicted image generation unit 310 will be described below.

(Reference sample filter section 3103)
The reference sample filter unit 3103 derives a reference sample s[x][y] at each position (x, y) on the reference area R by referring to the reference image. Further, the reference sample filter unit 3103 applies a reference pixel filter (first filter) to the reference sample s[x][y] according to the intra prediction mode, and applies a reference pixel filter (first filter) to each position (x, Update the reference sample s[x][y] of y) (derive the filtered reference image s[x][y]). Specifically, a low-pass filter is applied to the reference image at position (x, y) and its surroundings, and a filtered reference image is derived. Note that it is not necessary to apply a low-pass filter to all intra prediction modes, and a low-pass filter may be applied to some intra prediction modes. Note that the filter applied to the reference image on the reference area R in the reference sample filter unit 3103 is referred to as a "reference pixel filter (first filter)," whereas the filter applied to the reference image on the reference area R is referred to as a "reference pixel filter (first filter)," whereas the filter applied to the reference image on the reference area R is referred to as a "reference pixel filter (first filter)." The filter to be corrected is called a "position-dependent filter (second filter)."

(Configuration of intra prediction unit 3104)
The intra prediction unit 3104 generates a temporary predicted image (tentative predicted pixel value, pre-correction predicted image) of the prediction target block based on the intra prediction mode, reference image, and filtered reference pixel value, and sends it to the predicted image correction unit 3105. Output. The prediction unit 3104 internally includes a planar prediction unit 31041, a DC prediction unit 31042, an Angular prediction unit 31043, a LM prediction unit 31044, an MIP (Matrix-based Intra Prediction) unit 31045, and a DIMD prediction unit 31046. The prediction unit 3104 selects a specific prediction unit according to the intra prediction mode, and inputs the reference image and the filtered reference image. The relationship between the intra prediction mode and the corresponding prediction unit is as follows.
・Planar prediction ・・・・Planar prediction part 31041
・DC prediction ...DC prediction section 31042
・Angular prediction ・・・Angular prediction section 31043
・LM prediction...LM prediction section 31044
・Matrix intra prediction・・MIP part 31045
・DIMD prediction・・・DIMD prediction section 31046
(Planar prediction)
The planar prediction unit 31041 generates a temporary predicted image by linearly adding reference samples s[x][y] according to the distance between the prediction target pixel position and the reference pixel position, and outputs it to the predicted image correction unit 3105.

(DC prediction)
The DC prediction unit 31042 derives a DC predicted value corresponding to the average value of the reference samples s[x][y], and outputs a temporary predicted image q[x][y] whose pixel value is the DC predicted value.

(Angular prediction)
The Angular prediction unit 31043 generates a temporary predicted image q[x][y] using the reference sample s[x][y] in the prediction direction (reference direction) indicated by the intra prediction mode, and sends it to the predicted image correction unit 3105. Output.

(LM prediction)
The LM prediction unit 31044 predicts the chrominance pixel value based on the luminance pixel value. Specifically, this method uses a linear model to generate a predicted image of a color difference image (Cb, Cr) based on a decoded luminance image. CCLM (Cross-Component Linear Model prediction) is one of the LM predictions.
) There is a prediction. CCLM prediction is a prediction method that uses a linear model to predict color difference from luminance for one block.

(matrix intra prediction)
The MIP unit 31045 generates a tentative predicted image q[x][y] by a product-sum operation of the reference sample s[x][y] derived from the adjacent region and the weight matrix, and outputs it to the predicted image correction unit 3105.

(DIMD prediction)
The DIMD prediction unit 31046 is a prediction method that generates a predicted image using an intra prediction mode that is not explicitly signaled. The intra prediction parameter decoding unit 304 derives an intra prediction mode suitable for the target block using the information of the adjacent region, and the DIMD prediction unit 31046 generates a predicted image using this intra prediction mode. Details will be described later.

(Configuration of predicted image correction unit 3105)
The predicted image correction unit 3105 corrects the tentative predicted image output from the prediction unit 3104 according to the intra prediction mode. Specifically, the predicted image correction unit 3105 derives a position-dependent weighting coefficient for each pixel of the tentative predicted image, according to the reference region R and the position of the target predicted pixel. Then, by performing weighted addition (weighted average) of the reference sample s[][] and the provisional prediction image q[x][y], the prediction image (corrected prediction image) Pred[][] is obtained by correcting the provisional prediction image. Derive. Note that in some intra prediction modes, the predicted image correction unit 3105 may set the tentative predicted image q[x][y] as a predicted image without correcting it.

(Example 1)
FIG. 6 shows the configuration of the DIMD prediction unit 31046 in this embodiment. The DIMD prediction unit 31046 includes a gradient derivation unit 310461, an angular mode derivation unit 310462, an angular mode selection unit 310463, and a tentative predicted image generation unit 310464.

FIG. 7 shows an example of the syntax of encoded data regarding DIMD. The intra prediction parameter decoding unit 304 decodes a flag dimd_flag indicating whether to use DIMD for each block from the encoded data. When dimd_flag is 1, it is not necessary to decode the subsequent luminance intra prediction mode and syntax elements regarding MPM.

When dimd_flag is 1, the DIMD prediction unit 31046 derives an angle indicating the texture direction in the adjacent area using the pixel value. Then, a tentative predicted image is generated using the intra prediction mode corresponding to that angle. For example, (1) the gradient direction of the pixel value is derived for a pixel at a predetermined position in the adjacent region. (2) Convert the derived gradient direction to the corresponding direction prediction mode (Angular prediction mode). (3) Create a histogram of the obtained prediction direction for each predetermined pixel in the adjacent region. (4) Select a mode of prediction of the mode or a plurality of prediction modes including the mode of the mode from the histogram, and generate a tentative predicted image using the prediction mode. Below, the processing in each part of the DIMD prediction unit 31046 shown in FIG. 6 will be explained in more detail.

(1) Gradient derivation unit (image-based angle derivation unit)
The gradient deriving unit 310461 derives an angle (angle information) indicating the texture direction based on the image data of the adjacent area (of the target block). The angle information may be a value representing an angle with 1/36 precision, or may be any other value. The gradient deriving unit 310461 derives gradients (for example, Dx, Dy) in two or more specific directions, and derives the direction of the gradient (angle information) from the relationship between the gradients Dx, Dy.

A spatial filter may be used to derive the gradient. As the spatial filter, a 3x3 pixel Sobel filter corresponding to the horizontal and vertical directions may be used, for example, as shown in FIGS. 8(a) and (b). Gradients are derived for points Pn[0][0] (n=0..N-1) at N gradient derivation target positions in the adjacent region. FIG. 9(a) shows an example of the position of a gradient derivation target pixel in an 8x8 pixel target block. Further, FIG. 9(b) shows an example of the position of the gradient derivation target pixel in the target block of 4x4 pixels. The shaded pixels in the area adjacent to the target block are the gradient derivation target pixels. In this way, the number of gradient derivation target pixels, the position pattern, and the reference range of the spatial filter may be changed depending on information such as the size of the target block and the prediction mode of blocks included in the adjacent region.

Specifically, the gradient deriving unit 310461 derives the horizontal and vertical gradients Dx and Dy for each point Pn using the following equations.
Dx = Pn[-1][-1] + 2*Pn[-1][0] + Pn[-1][1] - Pn[1][-1] - 2*Pn[1][0] - Pn[1][1]
Dy = - Pn[-1][-1] - 2*Pn[0][-1] - Pn[1][-1] + Pn[-1][1] + 2*Pn[0][1 ] + Pn[1][1]
Here, Pn[x][y] is a pixel value at a pixel expressed in relative coordinates from point Pn ([0][0]) at the gradient derivation target position. The filters shown in FIGS. 8(c) and (d), which are obtained by inverting the filters shown in FIGS. 8(a) and (b) horizontally or vertically, may also be used. In that case, derive Dx and Dy using the following equations.
Dx = - Pn[-1][-1] - 2*Pn[-1][0] - Pn[-1][1] + Pn[1][-1] + 2*Pn[1][0 ] + Pn[1][1]
Dy = Pn[-1][-1] + 2*Pn[0][-1] + Pn[1][-1] - Pn[-1][1] - 2*Pn[0][1] - Pn[1][1]
The method for deriving the gradient is not limited to this, and other methods (filter, calculation formula, table, etc.) may be used. For example, a Prewitt filter or a Scharr filter may be used instead of the Sobel filter, or the filter size may be set to 2x2 or 5x5. The gradient deriving unit 310461 derives Dx and Dy as follows using a Prewitt filter.
Dx = Pn[-1][-1] + Pn[-1][0] + Pn[-1][1] - Pn[1][-1] - Pn[1][0] - Pn[1 ][1]
Dy = - Pn[-1][-1] - Pn[0][-1] - Pn[1][-1] + Pn[-1][1] + Pn[0][1] + Pn[ 1][1]
The following equation is an example of deriving Dx and Dy using a Scharr filter.
Dx = 3*Pn[-1][-1]+10*Pn[-1][0]+3*Pn[-1][1] -3*Pn[1][-1]-10*Pn [1][0]-3*Pn[1][1]
Dy = -3*Pn[-1][-1]-10*Pn[0][-1]-3*Pn[1][-1] +3*Pn[-1][1]+10* Pn[0][1]+3*Pn[1][1]
The method for deriving the gradient may be changed for each block. For example, a Sobel filter is used for a target block of 4x4 pixels, and a Scharr filter is used for a block larger than 4x4. In this way, by using a filter that is easier to calculate in small blocks, it is possible to suppress an increase in the amount of calculation in small blocks.

The gradient derivation method may be changed for each position of the gradient derivation symmetric pixel. For example, a Sobel filter is used for the gradient derivation target pixel in the upper or left adjacent region, and a Scharr filter is used for the gradient derivation target pixel in the upper left adjacent region.

The gradient deriving unit 310461 derives angle information consisting of a quadrant (hereinafter referred to as region) of the angle of the texture of the target block and an angle within the quadrant, based on the sign and magnitude relationship of Dx and Dy. Derive. The regions make it possible to share processing in directions with rotational symmetry or line symmetry. However, the angle information is not limited to the angle within the area and quadrant. For example, only the angle may be used as information, and the area may be derived as necessary. Additionally, in this embodiment, the intra direction prediction mode derived below is limited to the direction from the bottom left to the top right (2 to 66 in FIG. 3), and the intra direction prediction mode for directions with 180 degree rotational symmetry is the same. to be treated.

FIG. 10(a) is a table showing the signs (signx, signy) of Dx and Dy, the magnitude relationship (xgty), and the relationship between regions (Ra to Rd are constants representing region numbers). FIG. 10(b) shows quadrants indicated by regions Ra to Rd. The gradient deriving unit 310461 derives signx, signy, and xgty as follows.
absx = abs(Dx)
absy = abs(Dy)
signx = Dx < 0 ? 1 : 0
signy = Dy < 0 ? 1 : 0
xgty = absx > absy ? 1 : 0
Here, the inequality signs (>, <) may be replaced with equal signs (>=, <=). The area indicates a rough angle and can be derived only from the signs signx and signy of Dx and Dy and the magnitude relationship xgty.

The gradient derivation unit derives a region from 310461, the signs signx, signy, and the magnitude relationship xgty using calculations and table references. The gradient deriving unit may refer to the table in FIG. 10(a) and derive the corresponding region.

The gradient deriving unit 310461 may derive the region using a logical formula as follows.
region = xgty ? ( (signx^signy) ? 1 : 0 ) : ( (signx^signy) ? 2 : 3)
Here, ^ indicates XOR (exclusive OR). Indicates region as a value from 0 to 3. {Ra,Rb,Rc,Rd} = {0,1,2,3}. Note that the method of assigning region values is not limited to the above.

The gradient deriving unit 310461 may derive the region using another logical formula and addition/multiplication as follows.
region = 2*(!xgty) + (signx^signy^!xgty)
Here, the symbol ! means logical negation.

(2) Angle mode derivation section
The angular mode deriving unit 310462 derives an angular mode (a prediction mode corresponding to the slope, for example, an intra prediction mode) based on the slope information of the point Pn.

The angle mode derivation unit 310462 derives the slope iRatio (=absy÷absx) based on the absolute values absx, absy of the slope. Here, an integer representing the ratio in 1/R_UNIT increments is used as iRatio.
iRatio = int(R_UNIT*absy/absx) ≒ ratio*R_UNIT
R_UNIT uses an exponential power of 2 (1<<shiftR), for example, 65536 (shiftR=16).

The angular mode deriving unit 310462 derives idx from iRatio, refers to a lookup table (LUT) using the idx, and derives an angular mode difference value mode_delta that is difference information from the reference direction.
idx = iRatio >> 10
mode_delta = LUT[idx]
where LUT[N_LUT] = {0,1,1,2,2,3,3,4, 4,4,5,5,5,5,6,6, 6,6,7,7,7, 7,8,8, 8,8,9,9,9,9,10,10, 10,10,11,11,11,11,12,12, 12,12,12,13,13,13, 13,13, 13,14,14,14,14,14,14,15, 15,15,15,15,15,16,16,16, 16};
Here, N_LUT indicates the number of elements of LUT.
N_LUT = (1 << (shiftR - shiftI)) + 1
Note that when iRatio is approximately 0, that is, idx==0, it can be easily determined that the gradient is horizontal or vertical, and the difference from the reference direction is 0. Therefore, the first element 0 of the LUT may be omitted, and the LUT may be configured with a power of 2 elements of N_LUT'= (1 << (shiftR - shiftI)). The lookup table LUT' in this case is as follows.
LUT'[N_LUT'] = {1,1,2,2,3,3,4,4, 4,5,5,5,5,6,6,6, 6,7,7,7,7, 8,8,8, 8,9,9,9,9,10,10,10, 10,11,11,11,11,12,12,12, 12,12,13,13,13,13, 13,13, 14,14,14,14,14,14,15,15, 15,15,15,15,16,16,16,16};
At this time, the angular mode derivation unit derives mode_delta using the following equation.
mode_delta = idx ? LUT'[idx-1] : 0
In this way, since the number of elements N_LUT' is a power of 2, memory usage efficiency can be improved.

The angle mode derivation unit 310462 derives the angle mode modeVal by adding or subtracting mode_delta from the reference direction base_mode[region].
base_mode[4] = {18, 18, 50, 50}
direction[4] = {-1, 1, -1, 1}
modeVal = base_mode[region] + direction[region] * mode_delta
base_mode value of 18 is horizontal intra-directional prediction (HOR), 50 is vertical intra-directional prediction (VER)
means.

Note that the method for deriving modeVal is not limited to this. The angular mode deriving unit 310462 may use a rounding operation as follows when deriving idx.
idx = (iRatio + roundI) >> shiftI
Here, roundI=1<<(shiftI-1), shiftI is a predetermined constant 10, etc.

In the angle mode derivation unit 310462, the number of LUT elements N_LUT is 17 (shiftR-shiftI=4), 33 (shiftR-shiftI=5), 129 (shiftR- shiftI=7),
Other numbers such as 257 (shiftR-shiftI=8) may also be used.

(3) Gradient mode selection unit The angle mode selection unit 310463 selects a representative value dimdModeVal (dimdModeVal0, dimdModeVal0, dimdModeVal1,…). The representative value of the angle mode in this embodiment is the estimated value of the directionality of the texture pattern of the target block. Here, the representative value dimdModeVal is derived from the mode derived using a histogram. Convert the angular mode value modeValPn obtained for each point Pn into a histogram, and select the first mode dimdModeVal0 and the second mode dimdModeVal1, which are the most frequent mode and the next most frequent mode in the histogram, respectively. It is derived by

Note that the method for deriving dimdModeVal0 or dimdModeVal1 is not limited to the histogram. For example, the angle mode selection unit 310463 may set the average value of modeValPn to dimdModeVal0 or dimdModelVal1.

The angle mode selection unit 310463 sets a predetermined intra prediction mode to dimdModeVal2 as the third mode. Here, dimdModeVal2=0 (Planar), but the present invention is not limited to this. Other modes may be adaptively set, or the third mode may not be used.

The angular mode selection unit 310463 also derives weights corresponding to the representative values of each angular mode. For example, the weight w2 of the third mode is set to 21, and the remainder is distributed to weights w0 and w1, respectively, according to the ratio of the frequencies of the first and second modes in the histogram. Note that the total weight is 64. The derivation of the weights is not limited to this, and the weights w0, w1, and w2 of the first, second, and third modes may be adaptively changed. For example, w2 may be increased or decreased depending on the number of the first mode or the second mode, or the frequency or ratio thereof. Note that the angle mode selection unit sets the corresponding weight value to 0 for any of the first to third modes when the mode is not used.

As described above, the angle mode selection unit 310463 selects the intra prediction mode (representative value of the angle mode) estimated from the gradient, and outputs it together with the weight corresponding to each intra prediction mode.

Note that the configurations of the gradient deriving section, the angular mode deriving section, and the angular mode selecting section can also be used to derive information other than the intra prediction mode. For example, if the angular mode selection section selects the transformation matrix of the residual using the directionality of the texture pattern derived by the gradient derivation section and the angular mode derivation section, the residual in the region having the directionality can be A matrix suitable for difference conversion can be selected, and an improvement in coding efficiency can be expected.

(4) Temporary predicted image generation unit The temporary predicted image generation unit 310464 generates a temporary predicted image using one or more input intra prediction modes. When there is one intra prediction mode, an intra prediction image according to the prediction mode is generated and output as a tentative predicted image q[x][y]. When there are multiple intra prediction modes, a predicted image (pred0, pred1, pred2) according to each prediction mode is generated. A plurality of predicted images are combined using the corresponding weights (w0, w1, w2) and output as a tentative predicted image q[x][y]. The tentative predicted image q[x][y] is derived as follows.
q[x][y] = (w0 * pred0[x][y] + w1 * pred1[x][y] + w2 * pred2[x][y]) >> 6
However, if the frequency of the second mode is 0 or if it is not the direction prediction mode (such as DC mode), the predicted image pred0 in the first prediction mode is set as the temporary predicted image.

The inverse quantization/inverse transform unit 311 inversely quantizes the quantized transform coefficients input from the entropy decoding unit 301 to obtain transform coefficients. This quantized transform coefficient is a coefficient obtained by performing frequency transform such as DCT (Discrete Cosine Transform) or DST (Discrete Sine Transform) on the prediction error and quantizing it in the encoding process. It is. The inverse quantization/inverse transform unit 311 performs inverse frequency transform such as inverse DCT and inverse DST on the transform coefficients to calculate a prediction error. The inverse quantization/inverse transformation section 311 outputs the prediction error to the addition section 312.

The addition unit 312 adds the predicted image of the block input from the predicted image generation unit 308 and the prediction error input from the inverse quantization/inverse transformation unit 311 for each pixel to generate a decoded image of the block.
The adding unit 312 stores the decoded image of the block in the reference picture memory 306 and also outputs it to the loop filter 305.

(Configuration of video encoding device)
Next, the configuration of the video encoding device 11 according to this embodiment will be explained. FIG. 11 is a block diagram showing the configuration of the video encoding device 11 according to this embodiment. The video encoding device 11 includes a predicted image generation section 101, a subtraction section 102, a transformation/quantization section 103, an inverse quantization/inverse transformation section 105, an addition section 106, a loop filter 107, a prediction parameter memory (prediction parameter storage section , frame memory) 108, reference picture memory (reference image storage unit, frame memory) 109, encoding parameter determination unit 110, parameter encoding unit 111, and entropy encoding unit 104.

The predicted image generation unit 101 generates a predicted image for each CU, which is a region obtained by dividing each picture of the image T. The predicted image generation unit 101 operates in the same manner as the predicted image generation unit 308 already described, and the explanation will be omitted.

The subtraction unit 102 subtracts the pixel value of the predicted image of the block input from the predicted image generation unit 101 from the pixel value of the image T to generate a prediction error. Subtraction section 102 outputs the prediction error to conversion/quantization section 103.

The conversion/quantization unit 103 calculates a conversion coefficient by frequency conversion for the prediction error input from the subtraction unit 102, and derives a quantized conversion coefficient by quantization. The conversion/quantization unit 103
The quantized transform coefficients are output to entropy encoding section 104 and inverse quantization/inverse transform section 105.

The inverse quantization/inverse transformation unit 105 is the inverse quantization/inverse transformation unit 311 (FIG. 4) in the video decoding device 31.
is the same as , and the explanation will be omitted. The calculated prediction error is output to addition section 106.

Entropy encoding section 104 receives quantized transform coefficients from transform/quantization section 103 and inputs encoding parameters from parameter encoding section 111. Entropy encoding section 104 performs entropy encoding on division information, prediction parameters, quantization transform coefficients, etc., generates encoded stream Te, and outputs the encoded stream Te.

The parameter encoding unit 111 includes a header encoding unit 1110, a CT information encoding unit 1111, and a CU (not shown).
It includes an encoding section 1112 (prediction mode encoding section), an inter prediction parameter encoding section 112, and an intra prediction parameter encoding section 113. CU encoding section 1112 further includes a TU encoding section 1114.

(Configuration of intra prediction parameter encoding unit 113)
Intra prediction parameter encoding section 113 encodes IntraPredMode etc. input from encoding parameter determining section 110. Intra prediction parameter encoding section 113 includes a configuration that is partially the same as the configuration in which intra prediction parameter decoding section 304 derives intra prediction parameters.

Adding section 106 adds the pixel value of the predicted image of the block inputted from predicted image generation section 101 and the prediction error inputted from inverse quantization/inverse transformation section 105 for each pixel to generate a decoded image. Adding unit 106 stores the generated decoded image in reference picture memory 109.

The loop filter 107 applies a deblocking filter, SAO, and ALF to the decoded image generated by the addition unit 106. Note that the loop filter 107 does not necessarily need to include the above three types of filters, and may have a configuration including only a deblocking filter, for example.

The prediction parameter memory 108 stores the prediction parameters generated by the encoding parameter determination unit 110 at predetermined positions for each target picture and CU.

Reference picture memory 109 stores the decoded image generated by loop filter 107 at a predetermined position for each target picture and CU.

Encoding parameter determining section 110 selects one set from among multiple sets of encoding parameters. The encoding parameter is the above-mentioned QT, BT or TT division information, a prediction parameter, or a parameter to be encoded that is generated in relation to these. Predicted image generation section 101 generates a predicted image using these encoding parameters.

Encoding parameter determination section 110 calculates an RD cost value indicating the amount of information and encoding error for each of the plurality of sets. Encoding parameter determining section 110 selects a set of encoding parameters that minimizes the calculated cost value. Thereby, entropy encoding section 104 outputs the selected set of encoding parameters as encoded stream Te. Encoding parameter determining section 110 stores the determined encoding parameters in prediction parameter memory 108.

Note that some of the video encoding device 11 and video decoding device 31 in the embodiment described above, such as the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the predicted image generation unit 308, and the dequantization/inverse Transform unit 311, addition unit 312, predicted image generation unit 101, subtraction unit 102, transformation/quantization unit 103, entropy encoding unit 104, inverse quantization/inverse transformation unit 105, loop filter 107, encoding parameter determination unit 110 , the parameter encoding unit 111 may be realized by a computer. In that case, a program for realizing this control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed. Note that the "computer system" herein refers to a computer system built into either the video encoding device 11 or the video decoding device 31, and includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. Furthermore, a "computer-readable recording medium" refers to a medium that dynamically stores a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, it may also include a device that retains a program for a certain period of time, such as a volatile memory inside a computer system that is a server or a client. Further, the above-mentioned program may be one for realizing a part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

Furthermore, part or all of the video encoding device 11 and the video decoding device 31 in the embodiments described above may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the moving image encoding device 11 and the moving image decoding device 31 may be made into a processor individually, or some or all of them may be integrated into a processor. Further, the method of circuit integration is not limited to LSI, but may be implemented using a dedicated circuit or a general-purpose processor. Further, if an integrated circuit technology that replaces LSI emerges due to advances in semiconductor technology, an integrated circuit based on this technology may be used.

Although one embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to that described above, and various design changes etc. may be made without departing from the gist of the present invention. It is possible to

Embodiments of the present invention are suitably applied to a video decoding device that decodes encoded data obtained by encoding image data, and a video encoding device that generates encoded data obtained by encoding image data. be able to. Further, the present invention can be suitably applied to the data structure of encoded data generated by a video encoding device and referenced by a video decoding device.

31 Image decoding device
301 Entropy decoding section
302 Parameter decoding section
303 Inter prediction parameter decoding unit
304 Intra prediction parameter decoding unit
308 Predicted image generation unit
309 Inter predicted image generation unit
310 Intra predicted image generation unit
31046 DIMD prediction section
311 Inverse quantization/inverse transformation section
312 Addition section
11 Image encoding device
101 Predicted image generation unit
102 Subtraction section
103 Conversion/quantization section
104 Entropy encoder
105 Inverse quantization/inverse transformation section
107 Loop filter
110 Encoding parameter determination unit
111 Parameter encoding section
112 Inter prediction parameter encoder
113 Intra prediction parameter encoding unit
1110 Header encoder
1111 CT information encoder
1112 CU encoding unit (prediction mode encoding unit)
1114 TU encoding section

Claims (5)

A gradient derivation unit that derives the gradient of pixel values in blocks adjacent to the top and left of the target block, an angular mode derivation unit that derives an angular mode corresponding to the gradient, and an angular mode selection unit that selects the derived angular mode. and a temporary predicted image derivation unit that derives a temporary predicted image based on the angle mode.
A moving image decoding device and an angular mode deriving device comprising:
A moving image decoding device and an angular mode deriving device characterized in that decoding is performed based on a table including angular modes corresponding to the gradients. The image decoding device and angular mode deriving device according to claim 1, wherein the table includes difference information from a reference direction of an intra direction prediction mode corresponding to a gradient. 3. The moving image decoding device and angular mode deriving device according to claim 2, wherein the difference information is a positive integer. The moving image decoding device and the angular mode deriving device according to claim 1, wherein the angular mode selection unit selects the angular mode using an average value of the derived angular modes. A gradient derivation unit that derives the gradient of pixel values in blocks adjacent to the top and left of the target block, an angular mode derivation unit that derives an angular mode corresponding to the gradient, and an angular mode selection unit that selects the derived angular mode. and a temporary predicted image derivation unit that derives a temporary predicted image based on the angle mode.
A video encoding device and an angular mode deriving device comprising:
A video encoding device and an angular mode deriving device characterized in that the derivation is performed based on a table including angular modes corresponding to the gradients.

Images