JP2020025150A - Image filter device - Google Patents

Abstract

To achieve processing that makes use of both characteristics by removing redundant parts or sharing some information in a combination of PO and SAO to improve encoding efficiency more than before while suppressing an increase in a processing amount.SOLUTION: The image filter device for adding a first offset to each pixel value of a coding unit, is configured to: set a category indicating a change pattern of a pixel value in a two-dimensional direction with respect to a target pixel and a class indicating an amount of change in the pixel value in the two-dimensional direction; use a pixel adjacent to the target pixel as a reference pixel when adding an offset obtained by referring to the category and the class to the target pixel value from offsets decoded from encoded data to set categories and classes; and switch four pixels adjacent to the target pixel in the horizontal and vertical directions and four pixels adjacent to the target pixel in the upper left and lower right and lower left and upper right directions for the reference pixel.SELECTED DRAWING: Figure 25

Description

  Embodiments of the present invention relate to an image filter device.

  In order to efficiently transmit or record a moving image, an image encoding device that generates encoded data by encoding a moving image, and an image decoding that generates a decoded image by decoding the encoded data The device is used.

  Specific examples of the moving image coding method include, for example, methods proposed in H.264 / AVC and HEVC (High-Efficiency Video Coding).

In such a moving image coding method, an image (picture) constituting a moving image includes a slice obtained by dividing the image, and a coding unit (coding unit (Coding Unit) obtained by dividing the slice. : CU)), and is managed by a hierarchical structure including a prediction unit (PU) and a transform unit (TU), which are blocks obtained by dividing the coding unit. Decrypted.

  In such a moving picture coding method, a predicted image is usually generated based on a locally decoded image obtained by encoding / decoding an input image, and the predicted image is converted from the input image (original image). A prediction residual obtained by subtraction (sometimes called a “difference image” or a “residual image”) is encoded. As a method for generating a predicted image, there are an inter-screen prediction (inter prediction) and an intra-screen prediction (intra prediction). Furthermore, it has become common to apply a loop filter to a decoded image for the purpose of removing visual distortion.

  Also, Non-Patent Document 1 is a recent moving image encoding and decoding technique.

Further, in recent years, a peak SAO of Non-Patent Document 2 has been proposed as a loop filter in addition to a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF).

  As described above, although there are a plurality of loop filters, each having different effects, some of them have overlapping effects, and thus a method of combining loop filters needs to be devised.

"Algorithm Description of Joint Exploration Test Model 4", JVET-D1001, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 15-21 October 2016 "Peak Sample Adaptive Offset", JVET-D0133, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 15-21 October 2016

The types of loop filters applied to the decoded image are increasing, such as deblocking filter, SAO, ALF, and peak SAO (PO). In particular, the offset type EO of SAO uses the change of the pixel value in the one-dimensional direction. Although there is a difference in using a change in the pixel value in the two-dimensional direction of the image, both are similar in that a change in the pixel value between the target pixel and the reference pixel is used. Therefore, although there is a redundant portion or sharing of derived information can improve the coding efficiency of the SAO and the PO together, these points have not been sufficiently considered.

Therefore, the present invention realizes processing utilizing both characteristics by removing redundant parts in the combination of PO and SAO, or sharing some information, and performs encoding while suppressing an increase in processing amount. The purpose is to improve efficiency more than before.

  An embodiment of the present application is a first image filter device that adds a first offset to each pixel value of a plurality of coding units that form a decoded image and is generated by adding a residual image and a predicted image. A first setting unit that sets a first category indicating a two-dimensional direction pixel value change pattern and a first class indicating a two-dimensional direction pixel value change amount for the target pixel; And a first adder that adds a first offset obtained by referring to the first category and the first class to the target pixel value from among the first offsets decoded from the encoded data. The setting unit uses a pixel adjacent to the target pixel as a reference pixel in order to set a first category and a first class, and the reference pixel includes four pixels adjacent to the target pixel in the horizontal and vertical directions. , Upper left lower right, lower left right Switching the direction of the adjacent four pixels. Accordingly, since the reference pixel is switched in the horizontal, vertical, or oblique direction, the optimal reference direction can be selected, and the pixel value can be corrected by offset addition.

  An embodiment of the present application provides a second image filter device that adds an offset to each pixel value of a coding unit using a change in a pixel value in a one-dimensional direction with respect to an output image of the first filter device. A second class indicating a direction to refer to the target pixel in a one-dimensional direction, a second category indicating a change pattern of a pixel value in the one-dimensional direction, and a range of the target pixel value. And a second setting unit that sets a band position indicating the second offset, obtained from the second offset decoded from the encoded data, by referring to the second category, as the target pixel value. A second adding unit for adding the first offset without adding the second offset to the pixel to which the first offset has been added by the first adding unit. Part did not add the first offset Adding the second offset with respect to iodine. Thereby, even when the addition of the offset is unnecessary from the change in the pixel value in the two-dimensional direction, the offset can be appropriately added even when the addition of the offset is necessary from the change in the pixel value in the one-dimensional direction, Further, it is possible to avoid performing the offset addition processing twice.

  An embodiment of the present application provides a second image filter device that adds an offset to each pixel value of a coding unit using a change in pixel value in a one-dimensional direction with respect to an input image of the first filter device. A second class indicating a direction to refer to the target pixel in a one-dimensional direction, a second category indicating a change pattern of a pixel value in the one-dimensional direction, and a range of the target pixel value. And a second setting unit that sets a band position indicating the second offset, obtained from the second offset decoded from the encoded data, by referring to the second category, as the target pixel value. A second adder for adding, wherein the first adder of the first filter device adds the first offset to the pixel added with the second offset by the second adder, In the second adder, the second offset Not adding the first offset for pixels not calculated. This makes it possible to add an offset in consideration of a change in the pixel value in the two-dimensional direction to a pixel that needs to be added with an offset from a change in the pixel value in the one-dimensional direction.

An embodiment of the present application provides a second image filter device that adds an offset to each pixel value of a coding unit using a change in a pixel value in a one-dimensional direction with respect to an output image of the first filter device. A first type using a second class indicating a direction to refer to the target pixel in a one-dimensional direction, and a second type using a second category indicating a change pattern of a pixel value in the one-dimensional direction. A second setting unit for setting a second type using a band position indicating a range of the target pixel value, and a second category or band position from a second offset decoded from encoded data. A second adding unit that adds the second offset obtained by reference to the target pixel value, wherein the second adding unit is configured to add the first offset to the pixel added by the first adding unit. Off, using the second type Performs addition of Tsu bets, for pixels that have not adding the first offset in the first addition unit image filter device which is characterized in that the addition of the offset using the first type. This makes it possible to avoid redundant processing by combining filter processes having different image quality improvement effects.

  An embodiment of the present application provides a first image filter device that adds a first offset to each pixel value of a plurality of coding units that constitute a locally decoded image and is generated by adding a residual image and a prediction image. A first setting unit that sets a first category indicating a two-dimensional pixel value change pattern for the target pixel, and a first class indicating a two-dimensional direction pixel value change amount, A first adder for adding a first offset obtained by referring to the first category and the first class to the target pixel value from among the first offsets calculated by the first offset calculator; The setting unit uses a pixel adjacent to the target pixel as a reference pixel to set a first category and a first class, and the reference pixel is adjacent to the target pixel in the horizontal and vertical directions. Pixel and target pixel Image filter apparatus comprising a first image filter device characterized by switching the four adjacent pixels in the lower right upper direction. Accordingly, since the reference pixel is switched in the horizontal, vertical, or oblique direction, the optimal reference direction can be selected, and the pixel value can be corrected by offset addition.

  An embodiment of the present application provides a second image filter device that adds an offset to each pixel value of a coding unit using a change in a pixel value in a one-dimensional direction with respect to an output image of the first filter device. A second class indicating a direction to refer to the target pixel in a one-dimensional direction, a second category indicating a change pattern of a pixel value in the one-dimensional direction, and a range of the target pixel value. A second setting unit that sets a band position indicating a second offset, and a second offset obtained by referring to the second category from among the second offsets calculated by the second offset calculating unit. And a second adding unit that adds the first offset to the pixel to which the first offset has been added by the first adding unit. Addition unit does not add the first offset Adding the second offset to the pixel. Thereby, even when the addition of the offset is unnecessary from the change in the pixel value in the two-dimensional direction, the offset can be appropriately added even when the addition of the offset is necessary from the change in the pixel value in the one-dimensional direction, Further, it is possible to avoid performing the offset addition processing twice.

  An embodiment of the present application provides a second image filter device that adds an offset to each pixel value of a coding unit using a change in pixel value in a one-dimensional direction with respect to an input image of the first filter device. A second class indicating a direction to refer to the target pixel in a one-dimensional direction, a second category indicating a change pattern of a pixel value in the one-dimensional direction, and a range of the target pixel value. A second setting unit that sets a band position indicating a second offset, and a second offset obtained by referring to the second category from among the second offsets calculated by the second offset calculating unit. And a second adder for adding the first offset to the pixel to which the second offset has been added by the second adder. , The second offset in the second adder Image filter apparatus characterized by not adding the first offset for pixels which are not added. This makes it possible to add an offset in consideration of a change in the pixel value in the two-dimensional direction to a pixel that needs to be added with an offset from a change in the pixel value in the one-dimensional direction.

An embodiment of the present application provides a second image filter device that adds an offset to each pixel value of a coding unit using a change in a pixel value in a one-dimensional direction with respect to an output image of the first filter device. A first type using a second class indicating a direction to refer to the target pixel in a one-dimensional direction, and a second type using a second category indicating a change pattern of a pixel value in the one-dimensional direction. A second setting unit that sets a second type using a band position indicating a range of a target pixel value, and a second category or a band position from the second offset calculated by the second offset calculating unit. And a second adder that adds a second offset obtained by referring to the above to the target pixel value, wherein the second adder is configured to add a first offset to the pixel added by the first adder. Uses the second type Performs addition of the set, for pixels that have not adding the first offset in a first adding unit for adding an offset using the first type. This makes it possible to avoid redundant processing by combining filter processes having different image quality improvement effects.

  The embodiment of the present application is a combination of a filtering process by adding an offset calculated using a change in a two-dimensional pixel value and a filtering process by adding an offset calculated using two changes in a one-dimensional pixel value. Expressed by An image filter that adds an offset to each pixel value of a coding unit by using a change in pixel values in two one-dimensional directions provides two classes indicating different one-dimensional directions for a target pixel, and directions indicated by the classes. And a setting unit for setting a band position indicating a range of a target pixel value, and a setting unit for setting a band position indicating a range of the target pixel value, and a category obtained by referring to the category from among the offsets decoded from the encoded data. An adding unit that adds two offsets to the target pixel value, and the adding unit adds an average of the two offsets to the target pixel. Accordingly, by expressing by a combination of two one-dimensional direction filters, redundant processing is eliminated, and a pattern of change in pixel value that cannot be expressed by only a change in pixel value in one-dimensional direction is supported. And more efficient filtering can be realized.

  According to the embodiment of the present application, information on a reference pixel position of a filter (second filter) using a change in pixel value in a one-dimensional direction and a filter (first filter) using a change in pixel value in a two-dimensional direction To share. When applying the second filter after the first filter, the reference pixel position of the second filter is determined with reference to the reference pixel position used in the first filter. When applying the first filter after the second filter, the reference pixel position of the first filter is determined with reference to the reference pixel position used in the second filter. Thereby, the derivation processing of the reference pixel position can be shared, and redundant processing can be reduced.

The embodiment of the present application encodes the maximum number of the first filter class and the normalization coefficient in a picture parameter set or a slice header, and uses a parameter closely related to a change amount of a pixel value at a CTU level. Estimate the maximum number of classes and the normalization factor. The quantization width QP and the CU size are used as parameters closely related to the amount of change in the pixel value. This eliminates the need to encode the maximum number of classes of the first filter and the normalization coefficient for each CTU, so that fine offset adjustment can be performed while reducing the code amount.

  In the embodiment of the present application, the offset of the second filtering process having a small value is set to be equal to or less than the maximum value of the offset of the first filtering process. In addition, for the binary processing of the offset in the first processing, a coding method in which the code amount is difficult to increase with a large input value is selected. Select a conversion method. Thereby, the code amount of the offset used in the second processing can be reduced more than the code amount used in the first processing.

According to the embodiment of the present invention, by removing redundant portions in the combination of PO and SAO, or by sharing some information, a process that further utilizes both characteristics is realized, thereby increasing the amount of processing. It is possible to improve the coding efficiency as compared with the related art while suppressing the noise.

1 is a schematic diagram illustrating a configuration of an image transmission system according to a first embodiment. FIG. 3 is a diagram illustrating a hierarchical structure of data of an encoded stream according to the first embodiment. It is a figure showing a pattern of PU division mode. (A) to (h) show the partition shapes when the PU division mode is 2Nx2N, 2NxN, 2NxnU, 2NxnD, Nx2N, nLx2N, nRx2N, and NxN, respectively. FIG. 3 is a conceptual diagram illustrating an example of a reference picture and a reference picture list. FIG. 2 is a block diagram illustrating a configuration of an image decoding device according to Embodiment 1. FIG. 2 is a block diagram illustrating a configuration of an image encoding device according to Embodiment 1. FIG. 7 is a diagram illustrating a relationship between a target pixel and a reference pixel for each EO class of SAO. FIG. 1 is a diagram illustrating a configuration of a transmission device including an image encoding device according to a first embodiment and a configuration of a reception device including an image decoding device. (A) shows a transmitting device equipped with an image encoding device, and (b) shows a receiving device equipped with an image decoding device. FIG. 2 is a diagram illustrating a configuration of a recording device including the image encoding device according to the first embodiment and a playback device including the image decoding device. (A) shows a recording device equipped with an image encoding device, and (b) shows a reproducing device equipped with an image decoding device. FIG. 6 is a block diagram of the SAO unit 314 shown in FIG. 11 is a flowchart showing the operation of the category setting unit shown in FIG. FIG. 7 is a diagram illustrating a magnitude relationship between pixel values of a target pixel and a reference pixel for each SAO category. FIG. 4 is a diagram illustrating a relationship between a target pixel and a reference pixel of the PO according to the first embodiment. FIG. 4 is a diagram illustrating a relationship between a target pixel and a reference pixel of the PO according to the first embodiment. FIG. 18 is a diagram illustrating a relationship between a target pixel and a reference pixel in EO and PO of SAO according to the fifth embodiment. It is a figure explaining the band of BO of SAO. FIG. 7 is a block diagram of a SAO_E unit 115 shown in FIG. 18 is a flowchart illustrating the operation of the offset calculation unit in FIG. 18 is a flowchart showing the operation of the offset information selection unit in FIG. It is a block diagram of a PO part. 21 is a flowchart showing the operation of the PO category / PO class setting unit in FIG. 20. It is a block diagram of a PO_E part. 23 is a flowchart showing the operation of the PO offset calculator of FIG. 10 is a one-dimensional EO combination table according to the second embodiment. 21 is another example of a flowchart showing the operation of the PO category / PO class setting unit in FIG. 20. 21 is a flowchart illustrating an operation of a PO category / PO class setting unit in Modification Example 1 of FIG. 20. 13 is a flowchart illustrating the operation of the SAO unit in Modification 1. FIG. 21 is a block diagram of a SAO unit and a PO unit on the decoding device side in Modification 2. FIG. 18 is a block diagram of an SAO_E unit and a PO_E unit on the encoding device side in Modification 2. 13 is a flowchart illustrating an operation of a category setting unit according to a second modification. 15 is a flowchart illustrating the operation of a PO category / PO class setting unit according to a second modification. 14 is a table showing a relationship between a combination of one-dimensional EO and pcat according to the third embodiment. 15 is a table showing (α, β) used for calculating Cmax in the sixth embodiment. 9 is a flowchart illustrating an operation of a category setting unit according to the second embodiment. 9 is a flowchart illustrating an operation of an offset information selection unit according to the second embodiment. 21 is an example of a variable-length code table used for offset binaryization in a seventh embodiment. 13 is a flowchart illustrating operations of a PO unit and a SAO unit according to the third embodiment. 15 is a flowchart illustrating operations of a PO unit and a SAO unit according to the fourth embodiment. 15 is a flowchart illustrating an operation of a PO category / PO class setting unit according to the sixth embodiment.

[Embodiment 1]
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic diagram illustrating a configuration of an image transmission system 1 according to the present embodiment.

  The image transmission system 1 is a system that transmits a code obtained by encoding an encoding target image, decodes the transmitted code, and displays the image. The image transmission system 1 includes an image encoding device 11, a network 21, an image decoding device 31, and an image display device 41.

  An image T indicating an image of a single layer or a plurality of layers is input to the image encoding device 11. The layer is a concept used to distinguish a plurality of pictures when there is one or more pictures constituting a certain time. For example, when the same picture is encoded by a plurality of layers having different image qualities and resolutions, scalable encoding is performed, and when pictures of different viewpoints are encoded by a plurality of layers, view scalable encoding is performed. When prediction (inter-layer prediction, inter-view prediction) is performed between pictures of a plurality of layers, coding efficiency is greatly improved. Also, in the case where prediction is not performed (simultaneous cast), encoded data can be collected.

The network 21 transmits the encoded stream Te generated by the image encoding device 11 to the image decoding device 31. The network 21 is the Internet (internet), a wide area network (WAN: Wide Area Network), a small network (LAN: Local Area Network), or a combination thereof. The network 21 is not limited to a two-way communication network, but may be a one-way communication network for transmitting broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. Further, the network 21 may be replaced with a storage medium that records the encoded stream Te such as a DVD (Digital Versatile Disc) and a BD (Blu-ray Disc: registered trademark).

  The image decoding device 31 decodes each of the encoded streams Te transmitted by the network 21, and generates one or a plurality of decoded images Td respectively decoded.

  The image display device 41 displays all or a part of one or a plurality of decoded images Td generated by the image decoding device 31. The image display device 41 includes a display device such as a liquid crystal display and an organic EL (Electro-luminescence) display. Also, in the spatial scalable coding and the SNR scalable coding, when the image decoding device 31 and the image display device 41 have a high processing capability, a high quality enhancement layer image is displayed, and only a lower processing capability is provided. Displays a base layer image that does not require higher processing capability and display capability than the enhancement layer.

<Operator>
The operators used in this specification are described below.

>> is right bit shift, << is left bit shift, & is bitwise AND, | is bitwise OR
, | = Is a sum operation (OR) with another condition.

x? y: z is a ternary operator that takes y when x is true (other than 0) and z when x is false (0).

  Clip3 (a, b, c) is a function that clips c to a value greater than or equal to a and less than or equal to b, returning a if c <a, returning b if c> b, otherwise Is a function that returns c (where a <= b).

<Structure of encoded stream Te>
Prior to the detailed description of the image encoding device 11 and the image decoding device 31 according to the present embodiment, a data structure of an encoded stream Te generated by the image encoding device 11 and decoded by the image decoding device 31 will be described. .

FIG. 2 is a diagram showing a hierarchical structure of data in the encoded stream Te. The coded stream Te illustratively includes a sequence and a plurality of pictures constituting the sequence. 2A to 2F show an encoded video sequence defining a sequence SEQ, an encoded picture defining a picture PICT, an encoded slice defining a slice S, and an encoded slice defining slice data, respectively. FIG. 3 is a diagram illustrating data, a coding tree unit included in the coding slice data, and a coding unit (Coding Unit; CU) included in the coding tree unit.

(Encoded video sequence)
In the encoded video sequence, a set of data referred to by the image decoding device 31 to decode the sequence SEQ to be processed is defined. As shown in FIG. 2A, the sequence SEQ includes a video parameter set (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and an addition. Contains extended information SEI (Supplemental Enhancement Information). Here, the value shown after # indicates the layer ID. FIG. 2 shows an example in which coded data of # 0 and # 1, that is, layer 0 and layer 1 are present, but the type of layer and the number of layers do not depend on this.

The video parameter set VPS includes, in a moving image composed of a plurality of layers, a set of encoding parameters common to a plurality of moving images and a plurality of layers included in the moving image and encoding parameters related to individual layers. Sets are defined.

In the sequence parameter set SPS, a set of encoding parameters referred to by the image decoding device 31 to decode the target sequence is defined. For example, the width and height of a picture are defined. Note that a plurality of SPSs may exist. In that case, one of the plurality of SPSs is selected from the PPS.

In the picture parameter set PPS, a set of encoding parameters referred to by the image decoding device 31 to decode each picture in the target sequence is defined. For example, a reference value (pic_init_qp_minus26) of a quantization width used for decoding a picture and a flag (weighted_pred_flag) indicating application of weighted prediction are included. Note that a plurality of PPSs may exist. In that case, any one of the plurality of PPSs is selected from each picture in the target sequence.

(Encoded picture)
In the coded picture, a set of data referred to by the image decoding device 31 for decoding the picture PICT to be processed is defined. As shown in FIG. 2B, the picture PICT includes slices S0 to SNS _-1 (NS is the total number of slices included in the picture PICT).

In the following, when it is not necessary to distinguish each of the slices S0 to SNS _-1 , the description may be omitted with suffixes of the reference numerals. The same applies to other data included in the coded stream Te described below and to which subscripts are added.

(Coding slice)
In the coded slice, a set of data referred to by the image decoding device 31 for decoding the slice S to be processed is defined. The slice S includes a slice header SH and slice data SDATA, as shown in FIG.

  The slice header SH includes an encoding parameter group referred to by the image decoding device 31 to determine a decoding method for the target slice. The slice type designation information (slice_type) that designates the slice type is an example of an encoding parameter included in the slice header SH.

  The slice types that can be specified by the slice type specification information include (1) an I slice using only intra prediction at the time of encoding, (2) a P slice using unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding.

  Note that the slice header SH may include a reference (pic_parameter_set_id) to a picture parameter set PPS included in the encoded video sequence.

(Encoded slice data)
In the encoded slice data, a set of data referred to by the image decoding device 31 for decoding the slice data SDATA to be processed is defined. The slice data SDATA includes a coding tree unit (CTU: Coding Tree Unit) as shown in FIG. The CTU is a fixed-size (for example, 64 × 64) rectangle that forms a slice, and may be referred to as a maximum coding unit (LCU).

(Coding tree unit)
As shown in FIG. 2E, a set of data referred to by the image decoding device 31 for decoding the coding tree unit to be processed is defined. The coding tree unit is divided by recursive quadtree division (QT division) or binary tree division (BT division). A node having a tree structure obtained by recursive quadtree division or binary tree division is referred to as a coding node (CN). An intermediate node of the quadtree and the binary tree is a coding tree (CT: Coding Tree), and the coding tree unit itself is defined as a top coding tree.

The CTU includes a QT split flag (cu_split_flag) indicating whether or not to perform QT split, and a BT split mode (split_bt_mode) indicating a split method of the BT split. When cu_split_flag is 1, it is divided into four coding nodes CN. When cu_split_flag is 0, the coding node CN is not divided and has one coding unit (CU: Coding Unit) as a node. On the other hand, when split_bt_mode is 2, the image is horizontally divided into two encoding nodes CN. When split_bt_mode is 1, the image is vertically divided into two coding nodes CN. When split_bt_mode is 0, the coding node CN is not split and has one coding unit CU as a node. The coding unit CU is a terminal node of the coding tree and is not further divided. The encoding unit CU is a basic unit of the encoding process.

The size of the coding unit that can be taken when the size of the coding tree unit CTU is 64x64 pixels is, for example, 64x64 pixels, 64x32 pixels, 32x64 pixels, 32x32 pixels, 64x16 pixels, 16x64 pixels, 32x16 pixels, 16x32 pixels, 16x16 pixels, One of 64 × 8 pixels, 8 × 64 pixels, 32 × 8 pixels, 8 × 32 pixels, 16 × 8 pixels, 8 × 16 pixels, and 8 × 8 pixels. However, other sizes can be used depending on restrictions on the number of times and combinations of division, the size of the coding unit, and the like.

(Encoding unit)
As shown in (f) of FIG. 2, a set of data referred to by the image decoding device 31 for decoding the encoding unit to be processed is defined. Specifically, the coding unit includes a prediction tree, a transform tree, and a CU header CUH. The CU header specifies a prediction mode, a division method (PU division mode), and the like.

  In the prediction tree, prediction information (a reference picture index, a motion vector, and the like) of each prediction unit (PU) obtained by dividing the coding unit into one or a plurality is defined. In other words, a prediction unit is one or more non-overlapping regions that make up a coding unit. Further, the prediction tree includes one or more prediction units obtained by the above-described division. Hereinafter, a prediction unit obtained by further dividing the prediction unit is referred to as a “sub-block”. The sub-block is composed of a plurality of pixels. If the size of the prediction unit is equal to the size of the sub-block, there is one sub-block in the prediction unit. If the prediction unit is larger than the size of the sub-block, the prediction unit is divided into sub-blocks. For example, when the prediction unit is 8 × 8 and the sub-block is 4 × 4, the prediction unit is divided into four sub-blocks, which are divided into two horizontally and two vertically.

  The prediction process may be performed for each prediction unit (sub-block).

  Broadly speaking, there are two types of divisions in the prediction tree: intra prediction and inter prediction. Intra prediction refers to prediction within the same picture, and inter prediction refers to prediction processing performed between different pictures (for example, between display times and between layer images).

  In the case of intra prediction, the division method includes 2Nx2N (the same size as the coding unit) and NxN.

In the case of inter prediction, the division method is a PU division mode (part_mode) of the encoded data.
2Nx2N (same size as encoding unit), 2NxN, 2NxnU, 2NxnD, Nx2N
, NLx2N, nRx2N, and NxN. Note that 2NxN and Nx2N indicate a 1: 1 symmetric division,
2NxnU, 2NxnD and nLx2N, nRx2N show asymmetric splits of 1: 3, 3: 1. The PUs included in the CU are represented as PU0, PU1, PU2, and PU3 in order.

FIGS. 3A to 3H specifically illustrate the shape of the partition (the position of the boundary of the PU division) in each PU division mode. FIG. 3A shows a 2Nx2N partition, and FIGS. 3B, 3C, and 3D show 2NxN, 2NxnU, and 2NxnD partitions (horizontally long partitions), respectively. (E), (f), and (g) show partitions (vertical partitions) in the case of Nx2N, nLx2N, and nRx2N, respectively, and (h) shows NxN
Indicates the partition of Note that the horizontally long partition and the vertically long partition are generically called a rectangular partition, and 2Nx2N and NxN are generically called a square partition.

  In the transform tree, the coding unit is divided into one or a plurality of transform units, and the position and size of each transform unit are defined. In other words, a transform unit is one or more non-overlapping regions that make up a coding unit. Further, the transform tree includes one or a plurality of transform units obtained by the above-described division.

  The division in the transform tree includes one in which an area having the same size as the encoding unit is allocated as the transform unit, and one in which the recursive quadtree division is performed similarly to the above-described CU division.

  The conversion process is performed for each conversion unit.

(Forecast parameters)
A prediction image of a prediction unit (PU: Prediction Unit) is derived by a prediction parameter attached to the PU. The prediction parameter includes a prediction parameter for intra prediction or a prediction parameter for inter prediction. Hereinafter, the prediction parameters of the inter prediction (inter prediction parameters) will be described. The inter prediction parameter includes a prediction list use flag predFlagL0, predFlagL1, a reference picture index refIdxL0, refIdxL1, and a motion vector mvL0, mvL1. The prediction list use flags predFlagL0 and predFlagL1 are flags indicating whether reference picture lists called L0 list and L1 list are used, respectively. When the value is 1, the corresponding reference picture list is used. Note that in this specification, when a “flag indicating whether it is XX” is described, a flag other than 0 (for example, 1) is XX, 0 is not XX, and logical negation, logical product, etc. Treat 1 as true and 0 as false (the same applies hereinafter). However, other values can be used as a true value and a false value in an actual device or method.

  Syntax elements for deriving the inter prediction parameters included in the encoded data include, for example, PU partition mode part_mode, merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, There is a difference vector mvdLX.

(Reference picture list)
The reference picture list is a list including reference pictures stored in the reference picture memory 306. FIG. 4 is a conceptual diagram illustrating an example of a reference picture and a reference picture list. In FIG. 4A, a rectangle is a picture, an arrow is a picture reference relationship, a horizontal axis is time, and I, P, and B in the rectangle are intra pictures, uni-prediction pictures, bi-prediction pictures, and numbers in the rectangles, respectively. Indicates the decoding order. As shown in the figure, the decoding order of the pictures is I0, P1, B2, B3, B4, and the display order is I0, B3, B2, B4, P1. FIG. 4B shows an example of the reference picture list. The reference picture list is a list representing reference picture candidates, and one picture (slice) may have one or more reference picture lists. In the illustrated example, the target picture B3 has two reference picture lists, an L0 list RefPicList0 and an L1 list RefPicList1. When the current picture is B3, the reference pictures are I0, P1, and B2, and the reference picture has these pictures as elements. In each prediction unit, which picture in the reference picture list RefPicListX is actually referred to is specified by the reference picture index refIdxLX. The figure shows an example in which reference pictures P1 and B2 are referenced by refIdxL0 and refIdxL1.

(Merge prediction and AMVP prediction)
The prediction parameter decoding (encoding) method includes a merge prediction (merge) mode and an AMVP (Adaptive Motion Vector Prediction) mode. A merge flag merge_flag is a flag for identifying these. The merge prediction mode is a mode in which a prediction list use flag predFlagLX (or an inter prediction identifier inter_pred_idc), a reference picture index refIdxLX, and a motion vector mvLX are not included in encoded data but are derived from prediction parameters of already processed neighboring PUs. , AMVP mode is a mode in which an inter prediction identifier inter_pred_idc, a reference picture index refIdxLX, and a motion vector mvLX are included in encoded data. The motion vector mvLX is encoded as a prediction vector index mvp_LX_idx for identifying the prediction vector mvpLX and a difference vector mvdLX.

The inter prediction identifier inter_pred_idc is a value indicating the type and number of reference pictures, and takes one of PRED_L0, PRED_L1, and PRED_BI. PRED_L0 and PRED_L1 indicate that a reference picture managed by a reference picture list of an L0 list and an L1 list is used, respectively, and indicate that one reference picture is used (uni-prediction). PRED_BI indicates that two reference pictures are used (bi-prediction BiPred), and uses reference pictures managed in an L0 list and an L1 list. The prediction vector index mvp_LX_idx is an index indicating a prediction vector, and the reference picture index refIdxLX is an index indicating a reference picture managed in a reference picture list. Note that LX is a description method used when L0 prediction and L1 prediction are not distinguished, and the parameters for the L0 list and the parameters for the L1 list are distinguished by replacing LX with L0 and L1.

The merge index merge_idx is an index indicating which of the prediction parameter candidates (merge candidates) derived from the processed PU is to be used as the prediction parameter of the decoding target PU.

(Motion vector)
The motion vector mvLX indicates a shift amount between blocks on two different pictures. The prediction vector and the difference vector related to the motion vector mvLX are called a prediction vector mvpLX and a difference vector mvdLX, respectively.

(Inter prediction identifier inter_pred_idc and prediction list use flag predFlagLX)
The relationship between the inter prediction identifier inter_pred_idc and the prediction list use flags predFlagL0 and predFlagL1 is as follows, and can be mutually converted.

inter_pred_idc = (predFlagL1 << 1) + predFlagL0
predFlagL0 = inter_pred_idc & 1
predFlagL1 = inter_pred_idc >> 1
As the inter prediction parameter, a prediction list use flag or an inter prediction identifier may be used. Further, the determination using the prediction list use flag may be replaced with a determination using an inter prediction identifier. Conversely, the determination using the inter prediction identifier may be replaced with a determination using a prediction list use flag.

(Judgment of bi-prediction biPred)
The flag biPred as to whether it is bi-prediction BiPred can be derived depending on whether both of the two prediction list use flags are “1”. For example, it can be derived by the following equation.

biPred = (predFlagL0 == 1 && predFlagL1 == 1)
The flag biPred can also be derived based on whether or not the inter prediction identifier is a value indicating that two prediction lists (reference pictures) are used. For example, it can be derived by the following equation.

biPred = (inter_pred_idc == PRED_BI)? 1: 0
The above equation can also be expressed by the following equation.

biPred = (inter_pred_idc == PRED_BI)
Note that a value of 3, for example, can be used for PRED_BI.

(Configuration of image decoding device)
Next, the configuration of the image decoding device 31 according to the present embodiment will be described. FIG. 5 is a block diagram illustrating a configuration of the image decoding device 31 according to the present embodiment. The image decoding device 31 includes an entropy decoding unit 301, a prediction parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit (prediction image generation device) 308, and an inverse. It is configured to include a quantization / inverse DCT unit 311 and an addition unit 312.

  Further, the prediction parameter decoding unit 302 includes an inter prediction parameter decoding unit 303 and an intra prediction parameter decoding unit 304. The predicted image generation unit 308 includes an inter predicted image generation unit 309 and an intra predicted image generation unit 310.

  The entropy decoding unit 301 performs entropy decoding on the coded stream Te input from the outside to separate and decode individual codes (syntax elements). The separated codes include prediction information for generating a prediction image, residual information for generating a difference image, and the like.

Entropy decoding section 301 outputs a part of the separated code to prediction parameter decoding section 302. The part of the separated code is, for example, a prediction mode predMode, a PU division mode part_mode, a merge flag merge_flag, a merge index merge_idx, an inter prediction identifier inter_pred_idc, a reference picture index refIdxLX, a prediction vector index mvp_LX_idx, and a difference vector mvdLX. The control of which code is to be decoded is performed by the prediction parameter decoding unit 30.
2 is performed based on the instruction. Entropy decoding section 301 outputs the quantized coefficient to inverse quantization / inverse DCT section 311. These quantized coefficients are coefficients obtained by performing DCT (Discrete Cosine Transform) on the residual signal in the encoding process and quantizing the result.

  The inter prediction parameter decoding unit 303 decodes the inter prediction parameters based on the code input from the entropy decoding unit 301 with reference to the prediction parameters stored in the prediction parameter memory 307.

  The inter prediction parameter decoding unit 303 outputs the decoded inter prediction parameter to the prediction image generation unit 308, and stores the decoded inter prediction parameter in the prediction parameter memory 307. Details of the inter prediction parameter decoding unit 303 will be described later.

Intra prediction parameter decoding section 304 decodes the intra prediction parameters with reference to the prediction parameters stored in prediction parameter memory 307 based on the code input from entropy decoding section 301. The intra prediction parameter is a parameter used in a process of predicting a CU in one picture, for example, an intra prediction mode IntraPredMode.
The intra prediction parameter decoding unit 304 outputs the decoded intra prediction parameters to the prediction image generation unit 308, and stores the decoded intra prediction parameters in the prediction parameter memory 307.

  The intra prediction parameter decoding unit 304 may derive different intra prediction modes for luminance and chrominance. In this case, the intra prediction parameter decoding unit 304 decodes the luminance prediction mode IntraPredModeY as the luminance prediction parameter and the chrominance prediction mode IntraPredModeC as the chrominance prediction parameter. The luminance prediction mode IntraPredModeY is 35 modes, and corresponds to planar prediction (0), DC prediction (1), and direction prediction (2-34). The color difference prediction mode IntraPredModeC uses one of planar prediction (0), DC prediction (1), direction prediction (2-34), and LM mode (35). The intra prediction parameter decoding unit 304 decodes a flag indicating whether IntraPredModeC is the same mode as the luminance mode, and if the flag indicates that the mode is the same as the luminance mode, assigns IntraPredModeY to IntraPredModeC, and sets the flag to If the mode is different from the mode, the planar prediction (0), the DC prediction (1), the directional prediction (2-34), and the LM mode (35) may be decoded as IntraPredModeC.

  The loop filter 305 applies a filter such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the adding unit 312.

  The reference picture memory 306 stores the decoded image of the CU generated by the adding unit 312 at a position predetermined for each decoding target picture and CU.

  The prediction parameter memory 307 stores the prediction parameters at predetermined positions for each of a decoding target picture and a prediction unit (or a sub-block, a fixed-size block, or a pixel). Specifically, the prediction parameter memory 307 stores the inter prediction parameter decoded by the inter prediction parameter decoding unit 303, the intra prediction parameter decoded by the intra prediction parameter decoding unit 304, and the prediction mode predMode separated by the entropy decoding unit 301. . The stored inter prediction parameters include, for example, a prediction list use flag predFlagLX (inter prediction identifier inter_pred_idc), a reference picture index refIdxLX, and a motion vector mvLX.

  The prediction mode generation unit 308 receives the prediction mode predMode input from the entropy decoding unit 301 and receives the prediction parameters from the prediction parameter decoding unit 302. Further, the predicted image generation unit 308 reads a reference picture from the reference picture memory 306. The prediction image generation unit 308 generates a PU prediction image in the prediction mode indicated by the prediction mode predMode, using the input prediction parameters and the read reference pictures.

  Here, when the prediction mode predMode indicates the inter prediction mode, the inter prediction image generation unit 309 performs a PU prediction image by the inter prediction using the inter prediction parameter input from the inter prediction parameter decoding unit 303 and the read reference picture. Generate

  The inter-prediction image generation unit 309 generates a motion vector for a reference picture list (L0 list or L1 list) in which the prediction list use flag predFlagLX is 1 from a reference picture indicated by a reference picture index refIdxLX with reference to a decoding target PU. The reference picture block at the position indicated by mvLX is read from the reference picture memory 306. The inter-prediction image generation unit 309 performs prediction based on the read reference picture block to generate a PU prediction image. The inter prediction image generation unit 309 outputs the generated PU prediction image to the addition unit 312.

  When the prediction mode predMode indicates the intra prediction mode, the intra prediction image generation unit 310 performs the intra prediction using the intra prediction parameters input from the intra prediction parameter decoding unit 304 and the read reference pictures. Specifically, the intra-prediction image generation unit 310 reads, from the reference picture memory 306, an adjacent PU that is a decoding target picture and is within a predetermined range from the decoding target PU among the already decoded PUs. The predetermined range is, for example, one of the left, upper left, upper, and upper right adjacent PUs when the decoding target PU sequentially moves in a so-called raster scan order, and differs depending on the intra prediction mode. The raster scan order is an order in which each row is sequentially moved from the left end to the right end in each picture from the upper end to the lower end.

The intra prediction image generation unit 310 performs prediction on the read adjacent PU in the prediction mode indicated by the intra prediction mode IntraPredMode, and generates a prediction image of the PU. The intra prediction image generation unit 310 outputs the generated prediction image of the PU to the addition unit 312.

When the intra prediction parameter decoding unit 304 derives different intra prediction modes for luminance and chrominance, the intra prediction image generation unit 310 performs planar prediction (0), DC prediction (1), and direction according to the luminance prediction mode IntraPredModeY. A prediction image of a luminance PU is generated by one of the predictions (2 to 34), and the planar prediction (0), the DC prediction (1), the direction prediction (2 to 34), and the LM mode are performed according to the color difference prediction mode IntraPredModeC. PU of color difference by any of (35)
Is generated.

The inverse quantization / inverse DCT section 311 inversely quantizes the quantization coefficient input from the entropy decoding section 301 to obtain a DCT coefficient. The inverse quantization / inverse DCT unit 311 performs an inverse DCT (Inverse Discrete Cosine Transform) on the obtained DCT coefficient, and calculates a residual signal. The inverse quantization / inverse DCT section 311 outputs the calculated residual signal to the addition section 312.

The addition unit 312 adds, for each pixel, the PU prediction image input from the inter prediction image generation unit 309 or the intra prediction image generation unit 310 and the residual signal input from the inverse quantization / inverse DCT unit 311. Generate a PU decoded image. The addition unit 312 outputs the generated decoded image of the PU to at least one of the deblocking filter 313, the SAO (sample adaptive offset) unit 314, and the ALF 315 for each picture.

The deblocking filter 313 performs the deblocking process on the CU boundary when the difference between the pre-deblocking pixel values of the pixels adjacent to each other via the CU boundary is smaller than a predetermined threshold. The image near the CU boundary is smoothed. The image that has been subjected to the deblocking process by the deblocking filter 313 is output to the SAO unit 314 as a deblocked decoded image. The pre-deblocking pixel value is a pixel value in the image output from the adding unit 312.

The SAO unit 314 generates an offset-filtered decoded image by performing, for each predetermined unit, an offset filter process using an offset decoded from the encoded data Te on the decoded image before the offset filter. Note that the decoded image before the offset filter may be an image output from the adding unit 312 or a deblocked decoded image output from the deblocking filter 313.

  The ALF 315 generates an ALF-completed decoded image by performing an adaptive filtering process on the pre-ALF decoded image using the ALF parameter ALFP decoded from the encoded data Te. The ALF-decoded image is output to the outside as a decoded image Td, and is stored in the reference picture memory 306 in association with the POC information decoded from the encoded data Te by the entropy decoding unit 301. Since the specific configuration of the ALF 315 will be described later, the description thereof is omitted here. The pre-ALF decoded image may be an image output from the adding unit 312, a deblocked decoded image, or an offset-filtered decoded image.

(Configuration of image coding device)
Next, the configuration of the image encoding device 11 according to the present embodiment will be described. FIG. 6 is a block diagram illustrating a configuration of the image encoding device 11 according to the present embodiment. The image encoding device 11 includes a predicted image generation unit 101, a subtraction unit 102, a DCT / quantization unit 103, and an entropy encoding unit 1.
04, inverse quantization / inverse DCT section 105, adder section 106, loop filter 107, prediction parameter memory (prediction parameter storage section, frame memory) 108, reference picture memory (reference image storage section, frame memory) 109, encoding parameter It is configured to include a determination unit 110 and a prediction parameter encoding unit 111. The prediction parameter coding unit 111 includes an inter prediction parameter coding unit 112 and an intra prediction parameter coding unit 113.

For each picture of the image T, the predicted image generation unit 101 generates a predicted image P of the prediction unit PU for each coding unit CU that is a region obtained by dividing the picture. Here, the predicted image generation unit 101 reads a decoded block from the reference picture memory 109 based on the prediction parameters input from the prediction parameter encoding unit 111. The prediction parameter input from the prediction parameter coding unit 111 is, for example, a motion vector in the case of inter prediction. The predicted image generation unit 101 reads a block at a position on the reference image indicated by the motion vector starting from the target PU. In the case of intra prediction, the prediction parameter is, for example, an intra prediction mode. The pixel value of the adjacent PU used in the intra prediction mode is read from the reference picture memory 109, and a predicted image P of the PU is generated. The predicted image generation unit 101 generates a PU predicted image P using one of a plurality of prediction methods for the read reference picture block. The predicted image generation unit 101 outputs the generated PU predicted image P to the subtraction unit 102.

  The operation of the predicted image generation unit 101 is the same as the operation of the predicted image generation unit 308 described above.

The predicted image generating unit 101 generates a PU predicted image P based on the pixel values of the reference block read from the reference picture memory using the parameters input from the prediction parameter encoding unit. The predicted image generated by the predicted image generation unit 101 is output to the subtraction unit 102 and the addition unit 106.

  The subtraction unit 102 generates a residual signal by subtracting the signal value of the predicted image P of the PU input from the predicted image generation unit 101 from the pixel value of the corresponding PU of the image T. Subtraction section 102 outputs the generated residual signal to DCT / quantization section 103.

The DCT / quantization section 103 performs DCT on the residual signal input from the subtraction section 102 and calculates a DCT coefficient. The DCT / quantization unit 103 quantizes the calculated DCT coefficient to obtain a quantization coefficient. DCT / quantization section 103 outputs the obtained quantization coefficient to entropy encoding section 104 and inverse quantization / inverse DCT section 105.

A quantization coefficient is input from the DCT / quantization unit 103 to the entropy encoding unit 104,
An encoding parameter is input from the prediction parameter encoding unit 111. The input coding parameters include codes such as a reference picture index refIdxLX, a prediction vector index mvp_LX_idx, a difference vector mvdLX, a prediction mode predMode, and a merge index merge_idx.

  The entropy coding unit 104 generates a coded stream Te by entropy coding the input quantization coefficients and coding parameters, and outputs the generated coded stream Te to the outside.

The inverse quantization / inverse DCT section 105 inversely quantizes the quantization coefficient input from the DCT / quantization section 103 to obtain a DCT coefficient. The inverse quantization / inverse DCT section 105 performs an inverse DCT on the obtained DCT coefficient to calculate a residual signal. The inverse quantization / inverse DCT section 105 adds the calculated residual signal to the addition section 10
6 is output.

The addition unit 106 adds the signal value of the PU predicted image P input from the predicted image generation unit 101 and the signal value of the residual signal input from the inverse quantization / inverse DCT unit 105 for each pixel, and performs decoding. Generate an image. The adding unit 106 stores the generated decoded image in the reference picture memory 109.

The loop filter 107 applies a deblocking filter 114, a SAO_E (sample adaptive offset encoder) unit 115, and an adaptive loop filter (ALF) 116 to the decoded image generated by the adding unit 106.

  The prediction parameter memory 108 stores the prediction parameter generated by the coding parameter determination unit 110 at a position predetermined for each picture to be coded and each CU.

  The reference picture memory 109 stores the decoded image generated by the loop filter 107 at a predetermined position for each picture to be encoded and each CU.

  The coding parameter determination unit 110 selects one set from a plurality of sets of coding parameters. The coding parameter is the above-described prediction parameter or a parameter to be coded that is generated in association with the prediction parameter. The predicted image generation unit 101 generates a PU predicted image P using each of these sets of encoding parameters.

  The encoding parameter determination unit 110 calculates a magnitude of the information amount and a cost value indicating an encoding error for each of the plurality of sets. The cost value is, for example, a sum of a code amount and a value obtained by multiplying a square error by a coefficient λ. The code amount is the information amount of the coded stream Te obtained by entropy coding the quantization error and the coding parameter. The square error is the sum of the squares of the residual values of the residual signals calculated by the subtraction unit 102 between pixels. The coefficient λ is a real number larger than a preset zero. The coding parameter determination unit 110 selects a set of coding parameters that minimizes the calculated cost value. As a result, the entropy encoding unit 104 outputs the selected set of encoding parameters to the outside as the encoded stream Te, and does not output the unselected set of encoding parameters. The coding parameter determination unit 110 stores the determined coding parameter in the prediction parameter memory 108.

  The prediction parameter coding unit 111 derives a coding format from the parameters input from the coding parameter determination unit 110 and outputs the format to the entropy coding unit 104. Deriving a format for encoding is, for example, deriving a difference vector from a motion vector and a prediction vector. Further, the prediction parameter coding unit 111 derives parameters necessary for generating a predicted image from the parameters input from the coding parameter determination unit 110, and outputs the parameters to the predicted image generation unit 101. The parameter required to generate the predicted image is, for example, a motion vector in sub-block units.

  The inter prediction parameter coding unit 112 derives an inter prediction parameter such as a difference vector based on the prediction parameter input from the coding parameter determination unit 110. The inter prediction parameter encoding unit 112 derives parameters necessary for generating a predicted image to be output to the predicted image generation unit 101, and the inter prediction parameter decoding unit 303 (see FIG. 6 and the like) derives the inter prediction parameters. Some configurations are the same as the configurations to be performed. The configuration of the inter prediction parameter coding unit 112 will be described later.

The intra prediction parameter coding unit 113 derives a coding format (for example, mpm_idx, rem_intra_luma_pred_mode, etc.) from the intra prediction mode IntraPredMode input from the coding parameter determination unit 110. (SAO processing)
Next, the SAO (sample adaptive offset) section 314 will be described. SAO is a filter applied after the deblocking filter, and has an effect of removing ringing distortion and quantization distortion. SAO is a process on a CTU basis, and is a filter that classifies pixel values into several categories and adds / subtracts an offset on a pixel basis for each category. SAO has two types of offset types, edge offset (EO) and band offset (BO), and the category classification method of the pixel values in the CTU is determined by the offset type. EO classifies pixel values into categories according to the magnitude relationship between a target pixel and an adjacent pixel (reference pixel). BO classifies pixel values into categories according to the magnitude of the pixel value of the target pixel. Both EO and BO add the offset determined for each category to the decoded pixel value (the pixel value of the image input to the SAO unit 314). To the SAO unit 314, a decoded image obtained by adding the prediction image and the prediction error, or a decoded image further applied with a deblocking filter is input, but hereinafter the input images to the SAO unit 314 are all `` decoded images '', The pixel values of the input image are called “decoded pixel values”.

  FIG. 10 is a block diagram of the SAO unit 314. The SAO unit 314 includes a category setting unit 1001, an offset information storage unit 1002, and an offset addition unit 1003. The category setting unit 1001 receives the offset type (type), the class (class), or the band position (band_position) decoded by the entropy decoding unit 301 and the decoded image rec. The category setting unit 1001 sets a category using these. If the offset type is 0, do not perform SAO (do not add offset). If the offset type is 1, execute BO, and if it is 2, execute EO.

  The operation of the category setting unit 1001 will be described with reference to the flowchart in FIG. The category setting unit 1001 checks the offset type in step S1101, and if EO, proceeds to step S1102, and if BO, proceeds to step S1105. S1102 to S1104 are EO processing, and S1105 to S1106 are BO processing. If the offset type is EO, the category setting unit 1001 refers to the class in S1102, and sets adjacent pixels of the target pixel X as reference pixels a and b.

FIG. 7 shows a method of setting a reference pixel according to the EO class. The class corresponds to the direction in which the reference pixel is viewed from the target pixel and is used for selecting the reference pixel. If class = 0, set the reference pixels a and b to the left and right pixels of the target pixel X.If class = 1, set a and b to the upper and lower pixels of the target pixel X.If class = 2, set The upper left and lower right pixels of the target pixel X are set to a and b. If class = 3, the lower left and upper right pixels of the target pixel X are set to a and b.

  Next, in step S1103, the category setting unit 1001 sets the difference sign (rec [X] -rec [a]) and sign (rec [X] -rec [b) between the target pixel X and the two reference pixels a and b. ]) Is used to derive edgeIdx.

edgeIdx = sign (rec [X] -rec [a]) + sign (rec [X] -rec [b]) + 2
sign (x) = 1 (x> 0)
= 0 (x = 0)
= -1 (x <0)
Here, rec [x] represents the decoded pixel value of pixel x.
Next, the category setting unit 1001 calculates a category cat from edgeIdx in S1104.

cat = (edgeIdx == 2)? 0: edgeIdx + 1 (edgeIdx <= 2)
= edgeIdx (edgeIdx> 2)
FIG. 12 shows the magnitude relationship between the target pixel value X indicated by the category cat and the reference pixels a and b when class = 0. The black circle is the target pixel X, the white circle to the left of X is the reference pixel a, the white circle to the right of X is the reference pixel b, and the vertical direction indicates the magnitude relationship of the pixel values.

If the offset type is BO, the category setting unit 1001 divides the pixel values of 0 to 2 ^N -1 into bands as shown in FIG. 16 in S1105. If N = 8, the pixel values range from 0 to 255, and each band consists of eight consecutive pixel values. If N = 10, the pixel values range from 0 to 1023, and each band consists of 32 consecutive pixel values.

In S1106, the category setting unit 1001 sets four consecutive bands from the band indicated by the band position (band_position) decoded by the entropy decoding unit 301 to 0 to 3 of the category cat. If there is a category including the pixel value rec [X] of the target pixel, the category is set to the target pixel X.

The offset information storage unit 1002 stores the offset (offset) decoded by the entropy decoding unit 301.
) Is stored.

The offset addition unit 1003 reads the offset offset [cat] from the offset information storage unit 1002 using the category cat derived by the category setting unit 1001. Assuming that the decoded pixel value is rec [X], the output pixel value recsao [X] of the offset adding unit 1003 is
recsao [X] = rec [X] + offset [cat]
It is.

(SAO in encoder)
FIG. 17 is a block diagram of the SAO_E unit 115 of FIG. 6 for explaining the operation of SAO in the image encoding device. SAO_E section 115 includes SAO information setting section 1701 and SAO section 314. The SAO unit 314 is shown in FIG.
The description is omitted. The SAO information setting section 1701 includes an offset calculation section 1702 and an offset information selection section 1703.

  First, the offset information calculation unit 1702 will be described. The offset information calculation unit 1702 calculates the offset of each class of EO (0 to 3) and BO. If the offset type is EO, the category of each pixel is set for each class, and the offset for each set category is calculated. A specific operation will be described with reference to FIG.

In step S1801, the offset calculation unit 1702 determines the class (class), count [class] [cat], and SAD
Initialize [class] [cat]. Here, count [class] [cat] is a variable that counts the number of pixels for each combination of (class, cat) in the CTU, and SAD [class] [cat] is decoding for each combination of (class, cat) in the CTU. This variable stores the sum of absolute differences between the pixel value and the original pixel.

The offset calculation unit 1702 calculates the offset of EO in S1802 and S1803. In S1702, the reference pixels a and b indicated by the class are set for the target pixel X, and a category cat is derived. Increment count [class] [cat] by 1 for the current class. In the current class, the absolute difference between the decoded pixel value and the original pixel is added to SAD [class] [cat].

SAD [class] [cat] + = | rec (X) -org [X] |
The above processing is performed on all the pixels in the CTU.

  The offset calculation unit 1702 calculates the offset offset [class] [cat] for the categories 1 to 4 using the following formula in S1803, and increments the class by one.

offset [class] [cat] = SAD [class] [cat] / count [class] [cat]
The offset of category 0 is set to 0.

If class = 4 in S1804 (all EOs have been completed), the offset calculation unit 1702 proceeds to S1805, otherwise repeats S1802 and S1803.

The offset calculation unit 1702 calculates the offset of BO in S1805 and S1806. In step S1805, the band i to which the decoded pixel value rec [X] of the target pixel belongs is obtained, and the difference sum SAD [class] [i] of the band i is rec [X].
And the difference between org [X]. Also, the count of band i is incremented by one.

i = Rec [X] >> 5
SAD [class] [i] + = (rec [X] -org [X])
count [class] [i] ++
The offset calculation unit 1702 calculates the offset for each of the 32 bands i (i = 0 to 31) in S1806.

offset [class] [i] = SAD [class] [i] / count [class] [i]
The offset of each category (cat = 1 to 4) of EO (class = 0 to 3) and BO (class = 4
) Was calculated for each band (i = 0 to 31).

The offset information selection unit 1703 uses the offset calculated by the offset calculation unit 1702 to select an offset type (EO / BO), a class for EO, and a band position for BO.
FIG. 19 is a flowchart showing the operation of the offset information selection unit 1703.

In step S1901, the offset information selection unit 1703 initializes the class class (0 to 3) and the absolute difference sum SAD [class]. The absolute difference sum SAD [class] is a variable that stores an absolute difference sum between a pixel value to which SAO is applied (to which an offset is added) and an original pixel value.

  In S1902, the absolute difference sum SAD [class] sets the reference pixels a and b indicated by the class, derives the category for each pixel of the CTU in the above-described manner, and uses the offset assigned to each category. Calculate the absolute difference sum SAD.

SAD [class] + = | rec [X] + offset [class] [cat] -org [X] |
The offset of category 0 is set to 0. rec [X] + offset [class] [cat] is the value of SAO when (class, cat) is combined, and the absolute difference between this and the original pixel value org [X] is added to the SAD for each class. When the processing of all the pixels in the CTU is completed, the class is incremented by one.

  In step S1903, the offset information selection unit 1703 checks whether the class is equal to 4. If the class is not equal to 4, the process of S1902 is performed until the class becomes 4. If the class is equal to 4, it means that the processing of all classes of EO has been completed, and the process proceeds to S1904.

  In step S1904, the offset information selection unit 1703 obtains i that minimizes SAD [i] (i = 0 to 3), and sets this as the class of EO.

SAD _EO = min (SAD [i]), class = i that satisfies
In step S1905, the offset information selection unit 1703 calculates, for each band of the CTU, the absolute difference sum between the pixel value to which SAO has been applied (to which the offset has been added) and the original pixel value. Specifically, using the band i of the target pixel X and the offset offset [class] [i] of each band obtained by the offset calculation unit 1702, a pixel value to which SAO of pixels belonging to four consecutive bands is applied Calculate the absolute difference sum of rec [X] + oft and original pixel value org [X]. here,
i = rec [X] >> 5
oft = offset [class = 4] [j] (j <= i <= j + 3)
= 0 (otherwise)
SAD [i] = | rec [X] + oft-org [X] |
And Then, after the above processing is performed on all the pixels of the CTU, j is incremented by one.

In step S1906, the offset information selection unit 1703 checks whether i <29. If i <32, there are bands for which processing has not been completed, and S1905 is repeated until j = 32. i = 32
In the case of, since the processing has been completed for all the bands, the process proceeds to S1907.

  In step S1907, the offset information selection unit 1703 sets j (j = 0 to 31) at which SAD [j] is minimum to the band position (band_position) of the BO of the current CTU.

SAD _BO = min (SAD [j]), band_position = j that satisfies the following
In step S1908, the offset information selection unit 1703 compares the magnitude relationship between SAO _EO and SAO _BO . If the SAO _BO is large, the process advances to S1909 to set the offset type (type) of the current CTU to EO. Otherwise, the process proceeds to S1910, where the offset type (type) of the current CTU is set to BO.

  The above is the processing performed by the offset information selection unit 1703.

The offset information (offset, offset type, class, band position) calculated by SAO information setting section 1701 is input to SAO section 314. The operation of SAO section 314 is the same as described above, and description thereof will be omitted.

(PO processing)
Although the SAO (especially EO) described above is a filter using pixel values in one-dimensional (up, down, left, right, and oblique) directions, a filter using two-dimensional pixel values is applied before this SAO. be able to. This filter is called peak SAO (peak SAO: PO). FIG. 20 shows a block diagram of the PO and SAO. The PO unit 2001 outputs the PO offset information (Cmax) decoded by the entropy decoding unit 301.
, NF, offset, etc.) and the decoded image, perform the PO processing, and output the resulting image to the SAO unit 314.

The PO unit 2001 includes a PO category / PO class setting unit 2002, an offset information storage unit 2003, and a PO offset adding unit 2004. The PO category / PO class setting unit 2002 sets the PO class using the PO category calculated for each pixel from the pixel value of the decoded image. The offset information storage unit 2003 stores the PO offset information decoded by the entropy decoding unit 301. PO offset addition section 2004 adds the offset specified by the PO class and the PO category to the decoded image, and outputs the result to SAO section 314.

The operation of the PO category / PO class setting unit 2002 will be described with reference to the flowchart of FIG. In S2101, the PO category / PO class setting unit 2002 sets a reference pixel of the target pixel. FIG. 13 shows the positional relationship between the target pixel X and the reference pixels c0 to c3. In S2102, the PO category / PO class setting unit 2002 checks the magnitude relationship between the reference pixel ci (i = 0 to 3) of the target pixel X and the target pixel X for each pixel X in the CTU according to the following equation.

Three
M _L = Σlgr (rec (ci) -rec (X))
i = 0
Three
M _S = Σsml (rec (ci) -rec (X))
i = 0
M = max (M _L , M _S )
here
lgr (x) = (x> 0)? 1: 0
sml (x) = (x <0)? 1: 0
It is.

  The PO category / PO class setting unit 2002 sets the category pcat in S2103.

pcat = 0 (M = 3)
1 (M = 4)
2 (otherwise)
The PO category / PO class setting unit 2002 calculates the PO class cid in S2104.

Three
cid = Σ | rec (ci) -rec (X) | / (M * (1 << NF)
i = 0
Here Σ, if the M = M _L sums of i satisfying lgr (rec (ci) -rec ( X)) = 1, M =
For M _S computing the sum of i satisfying sml (rec (ci) -rec ( X)) = 1. NF is a normalization coefficient decoded by the entropy decoding unit 301.

  In S2105, the PO category / PO class setting unit 2002 determines the PO class c_id as follows.

c_id = cid (cid <Cmax)
= Cmax (otherwise)
Here, Cmax is the maximum value of the PO class decoded by the entropy decoding unit 301, and prevents the number of PO classes from becoming too large.

The PO offset addition unit 2004 determines an offset to be applied using the PO category and the PO class of the target pixel calculated by the PO category / PO class setting unit 2002, and adds the offset to the target pixel value rec [X]. , And calculates a filtered target pixel value recpo [X].

recpo [X] = rec [X] + offset [c_id] [pcat]
(PO in the encoding device)
The operation of the PO_E unit that executes the PO processing in the encoding device will be described with reference to the block diagram of FIG. The PO_E unit 2201 includes a PO offset calculation unit 2202 that calculates an offset for each PO category and PO class, and a PO unit 2001. Since the PO unit 2001 is the same as described above, the description is omitted. The offset information (Cmax, NF, offset) calculated by the PO_E unit 2201 is output to the entropy encoding unit 104, and the filtered image is output to the SAO_E unit 115.

  The operation of the PO offset calculator 2202 will be described with reference to the flowchart in FIG.

In step S2301, the PO offset calculation unit 2202 calculates a variable count [c_id] [pcat] for counting the number of pixels for each combination of the PO class c_id and the PO category pcat, and an absolute difference between the decoded pixel value of the target pixel and the original pixel value. Initialize SAD [c_id] [pcat] that stores the sum.

In step S2302, the PO offset calculation unit 2202 sets the reference pixels c0 to c3 of the target pixel, and derives the PO category in the same manner as in FIG. The above processing is performed on all pixels of the CTU, and the distribution of the absolute difference between the decoded pixel value rec [X] of the target pixel X and the original pixel value org [X] is obtained for each PO category.

In step S2303, the PO offset calculation unit 2202 derives the normalization coefficient NF of the target pixel X and the maximum value Cmax of the PO class from the maximum value of the distribution obtained in step S2302. For example, if the maximum value of the distribution is Pmax,
Cmax = Pmax / 2
NF = Ceil (log2 (Pmax / Cmax))
Or using the mode Pfreq of the distribution
Cmax = min (Pmax / 2, Pfreq)
It may be.

In step S2304, the PO offset calculation unit 2202 extracts the PO category and the PO class of the target pixel X by the method described with reference to the flowchart of FIG. 21 and adds the absolute difference sum SAD [c_id] [pcat] between each PO category and the PO class. , The absolute difference between the decoded pixel value of the target pixel X and the original pixel value is added.

In S2305, the PO offset calculating unit 2202 calculates an offset for each PO category and PO class.

offset [c_id] [pcat] = SAD [c_id] [pcat] / count [c_id] [pcat]
Through the above processing, the filter processing can be performed using the change in the pixel value in the two-dimensional direction, which cannot be considered in SAO.

The above description is an example of using both PO and SAO. PO uses the change in the pixel value in the two-dimensional direction of the image, and the offset type EO of SAO uses the change in the pixel value in the one-dimensional direction. There is a difference, but both use the change in the pixel value between the target pixel and the reference pixel. It is similar in terms of use, and by changing the combination method of PO and SAO, processing that further utilizes both characteristics can be performed. In one embodiment of the present invention, a reference pixel of PO is selected from two types shown in FIG. 14 (a) horizontal / vertical direction and (b) diagonal direction using a flag for each CTU. The operation of the PO category / PO class setting unit 2002 of the PO unit 2001 for realizing this processing will be described with reference to the flowchart of FIG.

FIG. 25 is obtained by replacing S2101 in FIG. 21 with S25010 to S25012, and the other steps are not changed. The PO category / PO class setting unit 2002 checks the flag (flag) decoded by the entropy decoding unit 301 in S25010. If the flag is 0, the process advances to step S25011 to set the reference pixel ci (i = 0 to 3) of the target pixel X to horizontal and vertical pixels shown in FIG. If not, the process advances to step S25012 to set the reference pixel ci (i = 0 to 3) of the target pixel X to a diagonal pixel shown in FIG. 14B.

In the encoding device, in the flowchart of FIG. 23 showing the operation of the offset calculation unit 2202 of the PO_E unit 2201, the number of selectable reference pixels increases to two types, that is, a horizontal / vertical direction and an oblique direction. To S2305. Then, the absolute difference sum SAD [c = id] [pcat] obtained in S2304 is compared with the two types of reference pixels in FIG. 14, and the smaller reference pixel is adopted. A flag indicating whether the absolute difference sum SAD is smaller is a flag indicating whether (a) or (b) in FIG. 14 and an offset, Cmax, and NF calculated using this reference pixel are output to the entropy encoder 104. I do.

In this way, by setting the reference pixel position of the PO in either the horizontal / vertical direction or the diagonal direction using the flag, the optimal reference direction is selected while considering the change in the pixel value in the diagonal direction, The pixel value can be corrected by offset addition.
[Modification 1]
As another embodiment of the present invention, an example in which the processing of the offset type EO of the SAO is changed by the processing of the PO will be described. Specifically, the processing of the offset type EO of the SAO is changed depending on whether the pixel is the pixel to which the offset has been added by the PO or the pixel to which the offset has not been added. In the above PO configuration, when the PO category is 2, no offset is added. That is, it is not necessary to add an offset when checking a change in the two-dimensional pixel value between the target pixel and the reference pixel. However, in the EO of SAO described above, a change in the pixel value of the target pixel and the reference pixel in the one-dimensional direction may be checked, and an offset may be added. However, it is appropriate to not perform at least the SAO EO processing for pixels with a PO category other than 2 because the PO has already performed the offset processing. Do).

The operation of the PO category / PO class setting unit 2002 in the PO unit 2001 of the first modification will be described with reference to FIG. FIG. 26 adds S26030 to S26032 after S2103 in FIG. 21, and the other steps are not changed. In S26030, the PO category / PO class setting unit 2002 refers to the PO category pcat set in S2103, and determines whether pcat is 2 or not. If pcat is 2 (a category in which no offset is added in the PO section), the flow advances to S26031 to set the mask of the target pixel X to 1. If pcat is not 2 (the category to which the offset is added in the PO section), the process proceeds to S26032, and the mask of the target pixel X is set to 0. mask is a variable assigned to each pixel in the CTU.

Next, the SAO process performed after the PO will be described with reference to the flowchart in FIG. FIG.
S27040 to S27041 are added after S1104 in FIG. 11, and the other steps are the same. In S27040, the category setting unit 1001 of the SAO unit 314 sets the mask of the target pixel X set in the PO unit 2001.
Is checked, and if it is not 1 (the offset is added in the PO section), the process proceeds to S27041, where the category cat of the EO which is a one-dimensional SAO is set to 0 of the category to which the offset is not added (EO is not performed).

  By the above operation, only the pixels for which no offset was added at the time of the PO, which is the two-dimensional offset addition filter, are candidates for the offset addition of the EO, which is the one-dimensional offset addition filter. If the addition of an offset is not necessary, the offset can be appropriately added when the offset needs to be added from the change in the pixel value in the one-dimensional direction, and the similar processing of PO and EO is doubled. Implementation can be avoided.

The switching of the reference pixel position described in the first embodiment can be applied to the first modification. In this case, S2101 in FIG. 26 may be replaced with S25010 to S25012 in FIG.
Other steps are the same.
[Modification 2]
As another embodiment of the present invention, an example in which the execution order of PO and SAO is changed will be described. FIG. 28 is a block diagram on the decoding device side, and FIG. 29 is a block diagram on the encoding device side. Both have the same components as those in the second embodiment, but the execution order of PO and SAO is different. In the second modification, a pixel whose SAO offset type is EO and an offset is added (a pixel whose pixel category cat is other than 0, that is, a pixel whose category is a pixel category cat to which an offset is added) is assigned to a subsequent PO. It is assumed that the pixel to which the offset is added is a candidate. That is, the EO performs a correction in consideration of the change in the pixel value in the two-dimensional direction with respect to the pixel in consideration of the change in the pixel value in the one-dimensional direction.

  The operation of the category setting unit 1001 of the SAO unit 314 according to Modification 2 will be described with reference to the flowchart of FIG. FIG. 30 is obtained by adding S30000 to S30002 after S1104 in FIG. 11, and adding S30060 after S1106 in FIG. 11, and the other steps are the same.

In S30000, the category setting unit 1001 of the SAO unit 314 checks whether the category cat set in S1104 is 0. If the category cat is 0 (no offset addition by EO), the process advances to S30001 to set the mask of the target pixel to 0. If the category cat is not 0 (the offset is added by EO), the process advances to step S30002 to set the mask of the target pixel to 1. The category setting unit 1001 determines
Then, the mask of the target pixel is set to 0. mask is a variable assigned to each pixel in the CTU.

Next, the operation of the PO category / PO class setting section 2002 of the PO section 1001 will be described with reference to FIG.
FIG. 31 is obtained by adding S31030 and S31031 after S2103 in FIG. 21, and the other steps are the same. In S31030, the PO category / PO class setting unit 2002 checks a mask indicating whether or not the offset has been added by EO. If mask is 0 (no offset is added by EO), the process advances to step S31031 to set the PO category pcat to 0 of the category to which no offset is added. As a result, pixels for which no offset has been added in EO do not add the offset for PO.

The switching of the reference pixel position described in the second embodiment can be applied to the first modification. In this case, S2101 in FIG. 31 may be replaced with S25010 to S25012 in FIG.
Other steps are the same.
[Embodiment 2]
In the second embodiment of the present application, the PO pixel classification (PO category classification) is realized by a combination of two one-dimensional EOs. By implementing a combination of two EOs, redundant processing that occurs when processing both PO and EO is eliminated, and changes in pixel values that cannot be expressed only by changes in pixel values in the one-dimensional direction , And more efficient filter processing can be realized. Specifically, EO categories in two directions among the directions (horizontal direction, vertical direction, upper left to lower right, upper right to lower left) corresponding to the four classes shown in FIG. The category is derived as category 2, and a PO category classification is derived based on the two EO categories.

FIG. 24A shows the value of M when two categories of EO are combined (EO category 1 and EO category 2). By using FIG. 24A, a temporary variable M for deriving the PO category can be derived from the EO category 1 and the EO category 2.

M = max (Σlgr (rec (X) -rec (ci)), Σsml (rec (X) -rec (ci)))
Here, M of a pattern that does not occur is set to 0.

For example, EO category 1 in FIG. 24A is a category obtained from reference pixels in class 0 (reference pixels are horizontal) in FIG. 7, and EO category 2 is class 2 in FIG. 7 (reference pixels are from upper left to lower right). ) Is a category obtained from the reference pixel. For example, if EO category 1 is 2 and EO category 2 is 1, M is derived as 3.

FIG. 24B is a table showing the relationship between M of the PO category (derived from the equation of the first embodiment) and a combination of two EO categories (EO category 1 and EO category 2). For example, M = 3 is 2
One category corresponds to 0 and 3, or 1 and 2. M = 4 corresponds to two categories of 1 and 1, or 4 and 4. M = 0 to 2 is a change pattern of a pixel value which is not subjected to offset addition in PO, but is a change pattern of a pixel value which is a target of offset addition as one-dimensional EO. Do. In this way, by setting two directions (EO classes) for each CTU and obtaining categories in two directions for each pixel, processing that conventionally required categorization by both EO of PO and SAO can be performed by SAO Can be processed only by EO categorization.

  The block diagram of the second embodiment is the same as the SAO unit 314 of FIG. 10 of the first embodiment and the SAO_E unit 115 of FIG. 17 except that the operations of the category setting unit 1001, offset adding unit 1003, and offset information selecting unit 1703 are different. different. The operations of the category setting unit 1001 and the offset information selection unit 1703 will be described with reference to the flowcharts of FIGS.

The category setting unit 1001 in FIG. 34 repeats the processing of S1102 to S1104 twice to derive two categories from reference pixels corresponding to two directions (classes). Other steps are the same as those in FIG. 11 in which one category is derived from reference pixels in one direction (class). The category setting unit 1001 counts the number of processes in S1102 to S1104 in S34040, terminates the process if it is twice, returns to S1102 if it is once, and continues the process. Note that it is not necessary to perform the two category derivation in a loop process, and a configuration in which the category derivation process is performed twice may be employed.

  The offset adding unit 1003 reads from the offset information storage unit the two classes (class1, class2) calculated by the category setting unit 1001 and the two offsets corresponding to the categories (cat1, cat2), and reads the pixel value Rec [ X].

recsao [X] = rec [X] + (offset [class1] [cat1] + offset [class2] [cat2]) / 2
(cat1 ≠ 0 and cat2 ≠ 0)
= rec [X] + offset [class1] [cat1] (cat2 = 0)
= rec [X] + offset [class2] [cat2] (cat1 = 0)
In the above expression, the average value of the two offsets is added, but a weighted operation of the two offsets may be used. However, the weight w1 of the class class1 encoded first is made larger than the weight w2 of the class2.

recsao [X] = rec [X] + w1 * offset [cat1] + w2 * offset [cat2] (cat1 ≠ 0 and cat2 ≠ 0)
Here, w1> = w2 and w1 + w2 = 1.

Note that the two directions (classes) used in the category setting unit 1001 may be encoded / decoded on a CTU basis. To select two directions (classes) from the four directions (classes), 2-bit syntax is required. It should be noted that a restriction may be imposed that only two directions (classes) are selected as selectable directions when the directions are orthogonal. In this case, the possible reference pixel positions are limited to two patterns in the horizontal and vertical directions and two oblique directions, but the number of bits representing the class can be reduced by one bit.

As described above, in the second embodiment of the present invention, the processing corresponding to the PO category classification is realized by a combination of two one-dimensional EO category classifications. By implementing a combination of two EOs, redundant processing that occurs when processing both the PO category classification and the EO category classification is avoided, and it cannot be expressed only by a change in the pixel value in the one-dimensional direction. It is possible to cope with a pattern of a change in pixel value, and more efficient filtering can be realized.
[Embodiment 3]
In the third embodiment of the present application, PO is extended to express the one-dimensional EO combination of the second embodiment. By expanding and expressing PO, redundant processing that occurs when processing both PO and EO can be eliminated, and it is possible to deal with changes in pixel values in the one-dimensional direction that could not be expressed with conventional PO. And more efficient filter processing can be realized.

The block diagram of the third embodiment is the same as the PO unit 2001 of FIG. 20 of the first embodiment and the PO_E unit 2201 of FIG. 22, but the operation of the PO category / PO class setting unit 2002 is different.
FIG. 32 shows the correspondence between the PO category pcat and the EO category. In FIG. 32, M of PO
Is 0 to 2, the processing is performed according to the SAO EO category. If the M of PO is 3, PO is performed, but the offset is calculated on the encoding device side according to the EO category, and the PO offset addition unit 2004 uses the offset for addition processing. When M of PO is 4, PO is performed, but the offset is adjusted to the EO category, and an offset different from M = 3 is calculated on the encoding device side, and is used in the addition process by the PO offset addition unit 2004. . If the characteristics are similar, such as EO categories 1 and 4, or 2 and 3, the offset may be the same value.

Further, since the expanded PO also includes the EO function of the SAO, the SAO processed after the PO may implement only the function of the BO as shown in FIG. In this configuration, as shown in FIG. 37, first, the PO unit 2001 (or the PO_E unit 2201) performs the above-described PO category classification and offset addition processing, and the SAO unit 314 (or the SAO_E unit 115) removes the EO from the BO. Perform category classification and offset addition processing. Since the offset type of SAO is only BO, for example, the category setting unit 1001 in FIG. 11 performs only the processing of S1105 and S1106. The offset calculation unit 1702 in FIG. 18 performs only the processing in S1801, S1805, and S1806. The offset information selection unit 1703 in FIG. 19 performs only the processes in S1901 and S1905 to S1907.
Therefore, the offset information is the offset and the band position (band_position). Also, since the offset type is only BO, there is no need to encode the offset type (type).

As described above, the third embodiment of the present application realizes the category classification processing corresponding to the one-dimensional EO combination of the second embodiment by extending PO. By extending and realizing PO, redundant category classification processing that occurs when processing PO and EO is eliminated, and it is possible to cope with changes in pixel values in the one-dimensional direction that cannot be expressed by conventional PO. And more efficient filtering can be realized.
[Embodiment 4]
In the fourth embodiment of the present invention, since the EO of PO and SAO have different image quality improvement effects such as a ringing removal effect and the BO of SAO has a different image quality improvement effect of a false contour prevention effect, the SAO when PO is used (an offset is added by PO) is Select BO.

The block diagram of the fourth embodiment is the same as the PO unit 2001 of FIG. 20 of the first embodiment (the PO category / PO class setting unit 2002 operates in FIG. 26) and the PO_E unit 2201 of FIG. In step S3802, it is determined whether the offset has been added to all the pixels of the CTU. Specifically, when mask = 1, the offset is not added by PO, so if the mask of all the pixels of the CTU is 1, it means that the CTU has not added the offset. . Mask of all pixels of CTU with S3802
If is 1, the subsequent SAO selects and processes the appropriate type from EO and BO as before. Otherwise, the subsequent SAO is a BO only process. If the SAO is only BO, it is not necessary to encode the offset type of the SAO.

Note that if the encoding device encodes the offset type (BO) when at least one pixel in the CTU has a mask of 0 in the PO, the decoding device does not need S3802 and S3804 in FIG. This is exactly the same operation as the conventional operation of processing PO and SAO in order.
[Embodiment 5]
The fifth embodiment of the present application relates to the addition of an oblique direction pixel (FIG. 14B) to the reference pixel position of the PO described in the first embodiment. Since the EO of the PO and the SAO both need to appropriately calculate the reference pixel position, the redundant calculation process can be reduced by sharing the information of the reference pixel position between the PO and the EO.

  When processing SAO after PO (FIGS. 20 and 22), the reference pixel position of EO is determined using the reference pixel position calculated by PO. If the flag indicating the reference direction used in PO is POflag and the 1-bit class information used in EO is EOflag, the class class of EO can be calculated as follows.

class = (POflag << 1) + EOflag
In this case, information for indicating the EO class can be reduced by one bit.

When processing PO after SAO (FIGS. 28 and 29), the reference pixel position of PO is determined as shown in FIG. 15A using the reference pixel position calculated by EO of SAO. That is, when the EO class is 0 or 1, POflag = 0 (the reference pixel position is in the horizontal / vertical direction), and when the EO class is 2 or 3, POflag = 1 (the reference pixel position is in the oblique direction). . In this case, the information indicating the reference pixel position of the PO can be deleted.

When the reference pixel position of the EO of the PO and the SAO is common, the use of the reference pixel position having the effect of correction in the first processing is also used in the subsequent processing, so that there is also an effect that correction with higher accuracy can be performed.
[Modification 3]
The above is a case where the EO reference pixel positions of the PO and the SAO are common, but the third modification uses different reference pixel positions for the PO and the EO.

  When processing SAO after PO (FIGS. 20 and 22), the reference pixel position of EO is determined using the reference pixel position calculated by PO. If the flag indicating the reference direction used in PO is POflag and the 1-bit class information used in EO is EOflag, the class class of EO can be calculated as follows.

class = ((1-POflag) << 1) + EOflag
In this case, information for indicating the EO class can be reduced by one bit.

When processing PO after SAO (FIGS. 28 and 29), the reference pixel position of PO is determined as shown in FIG. 15B using the reference pixel position calculated by EO of SAO. In other words, if the EO class is 0 or 1, POflag = 1 (the reference pixel position is oblique), and if the EO class is 2 or 3, POflag = 0 (the reference pixel position is horizontal / vertical). . In this case, the information indicating the reference pixel position of the PO can be deleted.

When the reference pixel positions of EO of PO and SAO are different, subsequent processing can be performed using a change in the pixel value in the direction that could not be corrected in the first processing, so that there is an effect of improving coding efficiency.
[Embodiment 6]
The sixth embodiment of the present invention describes the estimation of the maximum number Cmax of PO classes notified by PO and the normalization coefficient NF used for class calculation. Since the number of offsets to be coded is limited by Cmax, and the granularity of the change amount of the corresponding pixel value can be changed by one offset by NF, the optimum offset can be adjusted by notifying Cmax and NF for each CTU. However, when Cmax and NF are encoded for each CTU, there is a problem that the code amount increases. Therefore, Cmax and NF are encoded using a picture parameter set, a slice header, and the like.At the CTU level, the Cmax and NF are changed using parameters that are closely related to the amount of change in the pixel value, thereby reducing the amount of code and increasing the fine offset. Can be adjusted.

  Hereinafter, an example in which the quantization width QP is used as a parameter closely related to the amount of change in the pixel value will be described.

  When QP is large, ringing distortion and quantization distortion increase in the decoded image, so it is necessary to increase the number of offsets and finely adjust the pixel value. Therefore, when QP is large, Cmax is increased, NF is reduced, and the number of offsets is increased, so that the accuracy of the change amount of the pixel value corresponding to one offset is reduced.

The operation of the PO category / PO class setting unit 2202 will be described with reference to the flowchart in FIG. FIG.
In FIG. 21, S39030 is added after S2103 in FIG. 21, and the other steps are the same. The PO category / PO class setting unit 2002 estimates Cmax and NF in C390 units in S39030. Specifically, the average value DIFF_avg of the absolute difference between the target pixel X and the decoded pixel values of the reference pixels c0 to c3 is calculated for each PO category, and using the (α, β) shown in FIG. calculate.

Cmax = (Cmax_init * α) >> β
Here, Cmax_init is a value of Cmax notified by a picture parameter set or a slice header. Avg in FIG. 33 is the average value DIFF_avg of the absolute differences. For example, when QP is smaller than THQ and avg is between THD1 and THD2, (α, β) = (3, 1), and Cmax = Cmax_init * 3/2. Further, NF is calculated by the following formula using the maximum value DIFF_max of the absolute difference between the decoded pixel value of the target pixel X and the reference pixels c0 to c3 for each PO category.

NF = Ceil (log2 (DIFF_max / DIFF_avg))
In the above description, the quantization width QP is used as a parameter closely related to the amount of change in the pixel value. However, the size of the CU may be used. An area having a large CU size has a flat texture, and a pseudo contour is likely to occur. Since a slight change in the pixel value is noticeable, the same control as when QP is large can be performed.

Further, in the above, Cmax and NF are estimated for each CTU. The code amount can also be reduced. If Cmax and NF of the i-th CTU are Cmax [i] and NF [i],
dCmax = Cmax [i] -Cmax [i-1] (i> 0)
dNF = NF [i] -NF [i-1]
And encodes dCmax and dNF.

On the encoding device side, in the PO offset calculation unit 2202, the derivation of Cmax and NF in S2303 shown in FIG. 23 is replaced with S39030 in FIG. 39 described above.

As described above, in the sixth embodiment, at the CTU level, Cmax and NF are changed using a parameter closely related to the amount of change in pixel value, or the difference value between the immediately preceding Cmax and NF is encoded. Thus, fine offset adjustment can be performed while reducing the code amount.

Note that the PO_E unit 2201 outputs Cmax_init and NF_init, which are the initial values of Cmax and NF, to the entropy coding unit 104, and the entropy coding unit 104 codes these into a picture parameter set and a slice header. The PO unit 2001 passes the Cmax_init and NF_init decoded by the entropy decoding unit 301 to the PO category / PO class setting unit 2002.
[Embodiment 7]
Embodiment 7 of the present application describes an offset entropy encoding method. In the present invention, PO and SAO encode and decode offsets, respectively. However, the difference between the original pixel value and the pixel value after the filter processing is smaller than the difference between the pixel value before the filter processing and the second processing (the processing 2) by the offset addition in the first processing (the processing 1). May be equal to or less than the first offset. Therefore, the code length of the offset of the process 2 can be reduced using the offset of the process 1.

In the entropy decoding unit 301 of FIG. 5 or the entropy encoding unit 104 of FIG. 6, there is a binaryization unit (not shown) that performs binaryization of the offset value. An example in which a TR (Truncated Rice) code is used as a binaryization method performed here will be described. The maximum value Omax1 that the offset of process 1 can take is
Omax1 = (1 << (min (N, 10) -5))-1
It is represented by Here, N is the gradation of the pixel, and in the case of 8 (0 to 255), Omax1 = 7, and a value of 0 to 7 can be used as the offset. FIG. 36A shows a TR code table when Omax1 = 7. For example, when the maximum value of the offset in the process 1 is 5, the maximum value of the offset in the process 2 is 5 or less, so in process 2, Omax2 = 5, and the TR code table at this time is shown in FIG. . Therefore, when the offset becomes equal to Omax2 in the process 2, the code amount can be reduced by one bit compared to when encoding using the TR code table used in the process 1. In the seventh embodiment, the case where the process 1 is PO and the process 2 is SAO, and vice versa can be applied.
[Modification 4]
In the seventh embodiment, the offset of the process 2 is limited to the maximum value of the offset of the process 1 or less. In the fourth modification, the limitation of the offset of the process 2 is set to be equal to or less than 最大 of the maximum value of the offset of the process 1. FIG. 36C shows a TR code table of the offset in the process 2 in the case of the modification 4. For example, when the maximum value of the offset in the process 1 is 5, the maximum value of the offset in the process 2 is equal to or less than 1/2 of 5, and therefore, in the process 2, Omax2 = 2.
[Modification 5]
In the seventh embodiment, the TR code (RiceParam = 0) is used for the binarization of the offset between the processing 1 and the processing 2. In the modified example 5, in the processing 1 in which the offset value is larger than that in the processing 2, the TR code (RiceParam = 1) is used, in which the code length is hardly increased even when the input value is increased.
Reduce the code amount of the offset. FIG. 36D shows a TR code table (RiceParam = 1). Comparing the code table of FIG. 36 (a) with the code table of FIG. 36 (d), when the input is 0, the code amount allocation of the code table of FIG. 36 (a) is small. The code amount allocation of the code table d) is small. Therefore, in the binaryization of the offset, the code table of FIG. 36D is used in the processing 1 and the code table of FIG. 36A is used in the processing 2, so that the code amount when the input is large can be reduced. .

  As described above, in the seventh embodiment of the present application, in the entropy encoding of the offset, the code length of the offset can be reduced by utilizing the fact that the offset of the second process is smaller than the offset of the first process. it can.

[Others]
Note that a part of the image encoding device 11 and the image decoding device 31 in the above-described embodiment, for example, the entropy decoding unit 301, the prediction parameter decoding unit 302, the loop filter 305, the predicted image generation unit 308, the inverse quantization / inverse DCT Unit 311, addition unit 312, predicted image generation unit 10
1, the subtraction unit 102, the DCT / quantization unit 103, the entropy encoding unit 104, the inverse quantization / inverse DCT unit 105, the loop filter 107, the encoding parameter determination unit 110, the prediction parameter encoding unit 111, and each unit The blocks may be realized by a computer. In that case, a program for realizing this control function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read and executed by a computer system. Note that the “computer system” here is a computer system built in either the image encoding device 11 or the image decoding device 31 and includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in a computer system. Further, the "computer-readable recording medium" is a medium that holds the program dynamically for a short time, such as a communication line for transmitting the program through a network such as the Internet or a communication line such as a telephone line, In this case, a program holding a program for a certain period of time, such as a volatile memory in a computer system serving as a server or a client, may be included. Further, the program may be for realizing a part of the functions described above, or may be for realizing the functions described above in combination with a program already recorded in a computer system.

  Further, part or all of the image encoding device 11 and the image decoding device 31 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the image encoding device 11 and the image decoding device 31 may be individually implemented as a processor, or a part or all thereof may be integrated and implemented as a processor. The method of circuit integration is not limited to an LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, in the case where a technology for forming an integrated circuit that replaces the LSI appears due to the progress of the semiconductor technology, an integrated circuit based on the technology may be used.

(Application example)
The above-described image encoding device 11 and image decoding device 31 can be used by being mounted on various devices that transmit, receive, record, and reproduce moving images. Note that the moving image may be a natural moving image captured by a camera or the like, or may be an artificial moving image (including CG and GUI) generated by a computer or the like.

  First, the fact that the above-described image encoding device 11 and image decoding device 31 can be used for transmitting and receiving moving images will be described with reference to FIG.

FIG. 8A is a block diagram illustrating a configuration of a transmission device PROD_A on which the image encoding device 11 is mounted. As shown in FIG. 8A, the transmission device PROD_A modulates a carrier with an encoding unit PROD_A1 that obtains encoded data by encoding a moving image and encoded data obtained by the encoding unit PROD_A1. And a transmitting section PROD_A3 for transmitting the modulated signal obtained by the modulating section PROD_A2. The above-described image encoding device 11 is used as the encoding unit PROD_A1.

The transmission device PROD_A is a camera PROD_A4 that captures a moving image, a recording medium PROD_A5 that records the moving image, an input terminal PROD_A6 for externally inputting the moving image, as a supply source of the moving image input to the encoding unit PROD_A1, and , An image processing unit A7 for generating or processing an image. FIG. 8A illustrates a configuration in which all of them are included in the transmission device PROD_A, but some of them may be omitted.

Note that the recording medium PROD_A5 may be a recording of a moving image that is not encoded, or may record a moving image encoded by a recording encoding method different from the transmission encoding method. It may be something. In the latter case, between the recording medium PROD_A5 and the encoding unit PROD_A1, a decoding unit that decodes the encoded data read from the recording medium PROD_A5 according to a recording encoding method (
(Not shown) may be interposed.

  FIG. 8B is a block diagram illustrating a configuration of the receiving device PROD_B on which the image decoding device 31 is mounted. As shown in FIG. 8B, the receiving device PROD_B includes a receiving unit PROD_B1 that receives a modulated signal, a demodulating unit PROD_B2 that obtains encoded data by demodulating the modulated signal received by the receiving unit PROD_B1, A decoding unit PROD_B3 that obtains a moving image by decoding the encoded data obtained by the unit PROD_B2. The above-described image decoding device 31 is used as the decoding unit PROD_B3.

The receiving device PROD_B has a display PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal for outputting the moving image to the outside, as a supply destination of the moving image output by the decoding unit PROD_B3. PROD_B6 may be further provided. FIG. 8B illustrates the configuration in which the receiving device PROD_B includes all of them, but a part of the configuration may be omitted.

Note that the recording medium PROD_B5 may be for recording a moving image that is not encoded, or may be encoded using a recording encoding method different from the transmission encoding method. You may. In the latter case, an encoding unit (not shown) that encodes the moving image obtained from the decoding unit PROD_B3 according to the encoding method for recording may be interposed between the decoding unit PROD_B3 and the recording medium PROD_B5.

  The transmission medium for transmitting the modulated signal may be wireless or wired. The transmission mode for transmitting the modulated signal may be broadcast (here, a transmission mode in which the transmission destination is not specified in advance), or communication (here, transmission in which the transmission destination is specified in advance). (Which refers to an embodiment). That is, the transmission of the modulated signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

  For example, a terrestrial digital broadcasting station (such as a broadcasting facility) / receiving station (such as a television receiver) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by wireless broadcasting. A broadcasting station (broadcasting facility or the like) / receiving station (television receiver or the like) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by cable broadcasting.

  In addition, servers (workstations, etc.) / Clients (television receivers, personal computers, smartphones, etc.) such as VOD (Video On Demand) services and video sharing services using the Internet are transmission devices that transmit and receive modulated signals by communication. This is an example of PROD_A / reception device PROD_B (usually, either a wireless or wired transmission medium is used in a LAN, and a wired transmission medium is used in a WAN). Here, the personal computer includes a desktop PC, a laptop PC, and a tablet PC. In addition, the smartphone includes a multifunctional mobile phone terminal.

The client of the moving image sharing service has a function of decoding encoded data downloaded from the server and displaying the encoded data on a display, and a function of encoding a moving image captured by a camera and uploading the encoded moving image to the server. That is, the client of the moving image sharing service functions as both the transmitting device PROD_A and the receiving device PROD_B.

  Next, the fact that the above-described image encoding device 11 and image decoding device 31 can be used for recording and reproduction of a moving image will be described with reference to FIG.

FIG. 9A is a block diagram illustrating a configuration of a recording device PROD_C in which the above-described image encoding device 11 is mounted. As illustrated in FIG. 9A, the recording device PROD_C includes an encoding unit PROD_C1 that acquires encoded data by encoding a moving image, and encoded data obtained by the encoding unit PROD_C1 on a recording medium PROD_M. A writing unit PROD_C2 for writing. The above-described image encoding device 11 is used as the encoding unit PROD_C1.

The recording medium PROD_M may be (1) a type built in the recording device PROD_C such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), or (2) an SD memory. It may be of a type connected to the recording device PROD_C, such as a card or a USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray Disc: registered) Such as a trademark, for example, may be loaded in a drive device (not shown) built in the recording device PROD_C.

In addition, the recording device PROD_C includes a camera PROD_C3 for capturing a moving image, an input terminal PROD_C4 for externally inputting a moving image, and a reception terminal for receiving the moving image, as a supply source of the moving image to be input to the encoding unit PROD_C1. A unit PROD_C5 and an image processing unit PROD_C6 for generating or processing an image may be further provided. FIG. 9A illustrates an example of a configuration in which the recording device PROD_C includes all of them, but some of them may be omitted.

The receiving unit PROD_C5 may receive an uncoded moving image, or may receive coded data coded by a transmission coding method different from the recording coding method. May be used. In the latter case, a transmission decoding unit (not shown) for decoding encoded data encoded by the transmission encoding method may be interposed between the receiving unit PROD_C5 and the encoding unit PROD_C1.

Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, an HDD (Hard Disk Drive) recorder, and the like (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is a main source of moving images). . In addition, a camcorder (in this case, the camera PROD_C3 is a main source of moving images), a personal computer (in this case, the receiving unit PROD_C5 or the image processing unit C6 is a main source of moving images), and a smartphone (this If the camera PROD_C3
Or the receiving unit PROD_C5 is a main source of moving images).
This is an example.

FIG. 9B is a block diagram illustrating a configuration of a playback device PROD_D including the above-described image decoding device 31. As shown in FIG. 9B, the playback device PROD_D reads a moving image by decoding a coded data read by the reading unit PROD_D1 and a reading unit PROD_D1 that reads coded data written on the recording medium PROD_M. And a decoding unit PROD_D2 for obtaining the same. The above-described image decoding device 31 is used as the decoding unit PROD_D2.

The recording medium PROD_M may be (1) a type built in the playback device PROD_D, such as an HDD or SSD, or (2) a type such as an SD memory card or a USB flash memory. It may be a type connected to the playback device PROD_D, or (3) a device such as a DVD or a BD that is loaded into a drive device (not shown) built in the playback device PROD_D. Good.

Also, the playback device PROD_D includes a display PROD_D3 for displaying a moving image, an output terminal PROD_D4 for outputting the moving image to the outside, and a transmitting unit for transmitting the moving image, as a supply destination of the moving image output by the decoding unit PROD_D2. PROD_D5 may be further provided. In FIG. 9B,
Although the configuration in which the playback device PROD_D includes all of them is illustrated, a part of the configuration may be omitted.

The transmitting unit PROD_D5 may transmit an uncoded moving image, or may transmit coded data coded by a transmission coding method different from the recording coding method. May be used. In the latter case, an encoding unit (not shown) for encoding a moving image using a transmission encoding method may be interposed between the decoding unit PROD_D2 and the transmission unit PROD_D5.

Such a playback device PROD_D includes, for example, a DVD player, a BD player, an HDD player, and the like (in this case, an output terminal PROD_D4 to which a television receiver or the like is connected is a main destination of a moving image). . In addition, a television receiver (in this case, the display PROD_D3 is a main supply destination of the moving image), a digital signage (also referred to as an electronic signboard or an electronic bulletin board, etc.) Desktop PC (in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is the main supply destination of moving images), laptop or tablet PC (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a moving image A main supply destination of an image), a smartphone (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a main supply destination of a moving image) are also examples of such a playback device PROD_D.

(Hardware realization and software realization)
Also, each block of the image decoding device 31 and the image encoding device 11 described above may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip),
(Central Processing Unit).

In the latter case, each device includes a CPU that executes instructions of a program for realizing each function, a ROM (Read Only Memory) storing the program, and a RAM (Random) that expands the program.
Access Memory), and a storage device (recording medium) such as a memory for storing the program and various data. An object of an embodiment of the present invention is to record a program code (executable program, intermediate code program, source program) of a control program of each device, which is software for realizing the above-described functions, in a computer-readable manner. The present invention can also be achieved by supplying a medium to each of the above-described devices, and reading out and executing a program code recorded on a recording medium by a computer (or CPU or MPU).

Examples of the recording medium include tapes such as a magnetic tape and a cassette tape; magnetic disks such as a floppy (registered trademark) disk / hard disk; and CD-ROMs (Compact Disc Read-Only Memory) / MO disks (Magneto-Optical discs). ) / MD (Mini Disc) / DVD (Digital Versatile Disc) / CD-R (CD Recordable) / Blu-ray Disc (Blu-ray Disc: registered trademark) and other discs including optical discs, IC cards (including memory cards) / Electronically Erasable and Programmable Read-Only Memory (EEPROM) / Electrically Erasable and Programmable Read-Only Memory (registered trademark) / flash ROM and other semiconductor memories or PLDs (Programmable logic devices) ) Or logic circuits such as an FPGA (Field Programmable Gate Array).

Further, the respective devices may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. This communication network is not particularly limited as long as it can transmit a program code. For example, the Internet, intranet,
Extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community Antenna television / Cable Television) communication network, virtual private network (Virtual Private Network), telephone line network , A mobile communication network, a satellite communication network and the like can be used. Also, the transmission medium constituting this communication network may be any medium capable of transmitting the program code, and is not limited to a specific configuration or type. For example, even if a cable such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, etc., infrared rays such as IrDA (Infrared Data Association) and remote control are used. , BlueTooth (registered trademark), IEEE 802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance: registered trademark), mobile phone network, satellite line, terrestrial digital broadcasting network, etc. It can also be used wirelessly. Note that the embodiment of the present invention can also be realized in the form of a computer data signal embedded in a carrier wave, in which the program code is embodied by electronic transmission.

[Appendix]
Embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately changed within the scope of the claims are also included in the technical scope of the present invention.

  An embodiment of the present application is a first image filter device that adds a first offset to each pixel value of a plurality of coding units that form a decoded image and is generated by adding a residual image and a predicted image. A first setting unit that sets a first category indicating a two-dimensional direction pixel value change pattern and a first class indicating a two-dimensional direction pixel value change amount for the target pixel; And a first adder that adds a first offset obtained by referring to the first category and the first class to the target pixel value from among the first offsets decoded from the encoded data. The setting unit uses a pixel adjacent to the target pixel as a reference pixel in order to set a first category and a first class, and the reference pixel includes four pixels adjacent to the target pixel in the horizontal and vertical directions. , Upper left lower right, lower left right Switching the direction of the adjacent four pixels. Accordingly, since the reference pixel is switched in the horizontal, vertical, or oblique direction, the optimal reference direction can be selected, and the pixel value can be corrected by offset addition.

  An embodiment of the present application provides a second image filter device that adds an offset to each pixel value of a coding unit using a change in a pixel value in a one-dimensional direction with respect to an output image of the first filter device. A second class indicating a direction to refer to the target pixel in a one-dimensional direction, a second category indicating a change pattern of a pixel value in the one-dimensional direction, and a range of the target pixel value. And a second setting unit that sets a band position indicating the second offset, obtained from the second offset decoded from the encoded data, by referring to the second category, as the target pixel value. A second adding unit for adding the first offset without adding the second offset to the pixel to which the first offset has been added by the first adding unit. Part did not add the first offset Adding the second offset with respect to iodine. Thereby, even when the addition of the offset is unnecessary from the change in the pixel value in the two-dimensional direction, the offset can be appropriately added even when the addition of the offset is necessary from the change in the pixel value in the one-dimensional direction, Further, it is possible to avoid performing the offset addition processing twice.

An embodiment of the present application provides a second image filter device that adds an offset to each pixel value of a coding unit using a change in pixel value in a one-dimensional direction with respect to an input image of the first filter device. A second class indicating a direction to refer to the target pixel in a one-dimensional direction, a second category indicating a change pattern of a pixel value in the one-dimensional direction, and a range of the target pixel value. And a second setting unit that sets a band position indicating the second offset, obtained from the second offset decoded from the encoded data, by referring to the second category, as the target pixel value. A second adder for adding, wherein the first adder of the first filter device adds the first offset to the pixel added with the second offset by the second adder, In the second adder, the second offset Not adding the first offset for pixels not calculated. This makes it possible to add an offset in consideration of a change in the pixel value in the two-dimensional direction to a pixel that needs to be added with an offset from a change in the pixel value in the one-dimensional direction.

  An embodiment of the present application provides a second image filter device that adds an offset to each pixel value of a coding unit using a change in a pixel value in a one-dimensional direction with respect to an output image of the first filter device. A first type using a second class indicating a direction to refer to the target pixel in a one-dimensional direction, and a second type using a second category indicating a change pattern of a pixel value in the one-dimensional direction. A second setting unit for setting a second type using a band position indicating a range of the target pixel value, and a second category or band position from a second offset decoded from encoded data. A second adding unit that adds the second offset obtained by reference to the target pixel value, wherein the second adding unit is configured to add the first offset to the pixel added by the first adding unit. Off, using the second type Performs addition of Tsu bets, for pixels that have not adding the first offset in the first addition unit image filter device which is characterized in that the addition of the offset using the first type. This makes it possible to avoid redundant processing by combining filter processes having different image quality improvement effects.

  An embodiment of the present application provides a first image filter device that adds a first offset to each pixel value of a plurality of coding units that constitute a locally decoded image and is generated by adding a residual image and a prediction image. A first setting unit that sets a first category indicating a two-dimensional pixel value change pattern for the target pixel, and a first class indicating a two-dimensional direction pixel value change amount, A first adder for adding a first offset obtained by referring to the first category and the first class to the target pixel value from among the first offsets calculated by the first offset calculator; The setting unit uses a pixel adjacent to the target pixel as a reference pixel to set a first category and a first class, and the reference pixel is adjacent to the target pixel in the horizontal and vertical directions. Pixel and target pixel Image filter apparatus comprising a first image filter device characterized by switching the four adjacent pixels in the lower right upper direction. Accordingly, since the reference pixel is switched in the horizontal, vertical, or oblique direction, the optimal reference direction can be selected, and the pixel value can be corrected by offset addition.

  An embodiment of the present application provides a second image filter device that adds an offset to each pixel value of a coding unit using a change in a pixel value in a one-dimensional direction with respect to an output image of the first filter device. A second class indicating a direction to refer to the target pixel in a one-dimensional direction, a second category indicating a change pattern of a pixel value in the one-dimensional direction, and a range of the target pixel value. A second setting unit that sets a band position indicating a second offset, and a second offset obtained by referring to the second category from among the second offsets calculated by the second offset calculating unit. And a second adding unit that adds the first offset to the pixel to which the first offset has been added by the first adding unit. Addition unit does not add the first offset Adding the second offset to the pixel. Thereby, even when the addition of the offset is unnecessary from the change in the pixel value in the two-dimensional direction, the offset can be appropriately added even when the addition of the offset is necessary from the change in the pixel value in the one-dimensional direction, Further, it is possible to avoid performing the offset addition processing twice.

An embodiment of the present application provides a second image filter device that adds an offset to each pixel value of a coding unit using a change in pixel value in a one-dimensional direction with respect to an input image of the first filter device. A second class indicating a direction to refer to the target pixel in a one-dimensional direction, a second category indicating a change pattern of a pixel value in the one-dimensional direction, and a range of the target pixel value. A second setting unit that sets a band position indicating a second offset, and a second offset obtained by referring to the second category from among the second offsets calculated by the second offset calculating unit. And a second adder for adding the first offset to the pixel to which the second offset has been added by the second adder. , The second offset in the second adder Image filter apparatus characterized by not adding the first offset for pixels which are not added. This makes it possible to add an offset in consideration of a change in the pixel value in the two-dimensional direction to a pixel that needs to be added with an offset from a change in the pixel value in the one-dimensional direction.

  An embodiment of the present application provides a second image filter device that adds an offset to each pixel value of a coding unit using a change in pixel value in a one-dimensional direction with respect to an output image of the first filter device. A first type using a second class indicating a direction to refer to the target pixel in a one-dimensional direction, and a second type using a second category indicating a change pattern of a pixel value in the one-dimensional direction. A second setting unit that sets a second type using a band position indicating a range of target pixel values, and the second category or band position from the second offset calculated by the second offset calculating unit. And a second adder that adds a second offset obtained by referring to the above to the target pixel value, wherein the second adder is configured to add a first offset to the pixel added by the first adder. Uses the second type Performs addition of the set, for pixels that have not adding the first offset in a first adding unit for adding an offset using the first type. This makes it possible to avoid redundant processing by combining filter processes having different image quality improvement effects.

  The embodiment of the present application is a combination of a filtering process by adding an offset calculated using a change in a two-dimensional pixel value and a filtering process by adding an offset calculated using two changes in a one-dimensional pixel value. Expressed by An image filter that adds an offset to each pixel value of a coding unit by using a change in pixel values in two one-dimensional directions provides two classes indicating different one-dimensional directions for a target pixel, and directions indicated by the classes. And a setting unit for setting a band position indicating a range of a target pixel value, and a setting unit for setting a band position indicating a range of the target pixel value, and a category obtained by referring to the category from among the offsets decoded from the encoded data. An adding unit that adds two offsets to the target pixel value, and the adding unit adds an average of the two offsets to the target pixel. Accordingly, by expressing by a combination of two one-dimensional direction filters, redundant processing is eliminated, and a pattern of change in pixel value that cannot be expressed by only a change in pixel value in one-dimensional direction is supported. And more efficient filtering can be realized.

  According to the embodiment of the present application, information on a reference pixel position of a filter (second filter) using a change in pixel value in a one-dimensional direction and a filter (first filter) using a change in pixel value in a two-dimensional direction To share. When applying the second filter after the first filter, the reference pixel position of the second filter is determined with reference to the reference pixel position used in the first filter. When applying the first filter after the second filter, the reference pixel position of the first filter is determined with reference to the reference pixel position used in the second filter. Thereby, the derivation processing of the reference pixel position can be shared, and redundant processing can be reduced.

The embodiment of the present application encodes the maximum number of the first filter class and the normalization coefficient in a picture parameter set or a slice header, and uses a parameter closely related to a change amount of a pixel value at a CTU level. Estimate the maximum number of classes and the normalization factor. The quantization width QP and the CU size are used as parameters closely related to the amount of change in the pixel value. This eliminates the need to encode the maximum number of classes of the first filter and the normalization coefficient for each CTU, so that fine offset adjustment can be performed while reducing the code amount.

In the embodiment of the present application, the offset of the second filtering process having a small value is set to be equal to or less than the maximum value of the offset of the first filtering process. In addition, for the binary processing of the offset in the first processing, a coding method in which the code amount is difficult to increase with a large input value is selected, and in the binary processing of the offset in the second processing, the coding method is likely to reduce the code amount with a small input value. Select a conversion method. Thereby, the code amount of the offset used in the second processing can be reduced more than the code amount used in the first processing.

  Embodiments of the present invention can be suitably applied to an image decoding device that decodes encoded data obtained by encoding image data, and an image encoding device that generates encoded data obtained by encoding image data. it can. Further, the present invention can be suitably applied to the data structure of encoded data generated by an image encoding device and referenced by the image decoding device.

11 Image Encoding Device 31 Image Decoding Device 314 SAO Unit 115 SAO_E Unit 2001 PO Unit 2201 PO_E Unit

Claims (8)

A first image filter device configured to add a first offset to each pixel value of a plurality of coding units forming a decoded image, which is generated by adding a residual image and a prediction image,
A first setting unit that sets a first category indicating a two-dimensional direction pixel value change pattern and a first class indicating a two-dimensional direction pixel value change amount for the target pixel;
A first adder that adds a first offset obtained by referring to the first category and the first class to the target pixel value from among the first offsets decoded from the encoded data; Prepared,
The setting unit uses a pixel adjacent to the target pixel as a reference pixel to set a first category and a first class,
An image filter comprising a first image filter device, wherein the reference pixel switches between four adjacent pixels in the horizontal and vertical directions with respect to the target pixel and four adjacent pixels in the upper left and lower right and lower left and upper right directions with respect to the target pixel. apparatus. In claim 1,
A second image filter device that adds an offset to each pixel value of a coding unit using a change in a pixel value in a one-dimensional direction with respect to an output image of the first filter device,
The second image filter device indicates a second class indicating a direction to refer to the target pixel in a one-dimensional direction, a second category indicating a change pattern of pixel values in a one-dimensional direction, and a range of the target pixel value. A second setting unit for setting a band position;
A second adder that adds a second offset obtained by referring to the second category to the target pixel value from among the second offsets decoded from the encoded data,
The second adder did not add the second offset to the pixel to which the first offset was added in the first adder, and did not add the first offset in the first adder. An image filter device, wherein a second offset is added to a pixel. In claim 1,
A second image filter device that adds an offset to each pixel value of a coding unit using a change in a pixel value in a one-dimensional direction with respect to an input image of the first filter device,
The second image filter device indicates a second class indicating a direction to refer to the target pixel in a one-dimensional direction, a second category indicating a change pattern of pixel values in a one-dimensional direction, and a range of the target pixel value. A second setting unit for setting a band position;
A second adder that adds a second offset obtained by referring to the second category to the target pixel value from among the second offsets decoded from the encoded data,
The first addition unit of the first filter device adds the first offset to the pixel to which the second offset has been added by the second addition unit, and adds the second offset to the pixel by the second addition unit. An image filter device, wherein a first offset is not added to a pixel that has not been added. In claim 1,
A second image filter device that adds an offset to each pixel value of a coding unit using a change in a pixel value in a one-dimensional direction with respect to an output image of the first filter device,
The second image filter device includes a first type using a second class indicating a direction in which a target pixel is referred to in a one-dimensional direction and a second type using a second category indicating a change pattern of pixel values in a one-dimensional direction. A second setting unit that sets a second type using a band position indicating a range of pixel values;
A second adding unit that adds a second offset obtained by referring to the second category or the band position to the target pixel value from among the second offsets decoded from the encoded data,
The second addition unit adds an offset using the second type to the pixel to which the first offset has been added in the first addition unit, and adds the first offset in the first addition unit. An image filter apparatus, wherein an offset using the first type is added to a missing pixel. A first image filter device configured to add a first offset to each pixel value of a plurality of coding units included in a locally decoded image generated by adding a residual image and a prediction image,
A first setting unit that sets a first category indicating a two-dimensional direction pixel value change pattern and a first class indicating a two-dimensional direction pixel value change amount for the target pixel;
A first adder for adding a first offset obtained by referring to the first category and the first class to the target pixel value from among the first offsets calculated by the first offset calculator; With
The setting unit uses a pixel adjacent to the target pixel as a reference pixel to set a first category and a first class,
An image filter comprising a first image filter device, wherein the reference pixel switches between four adjacent pixels in the horizontal and vertical directions with respect to the target pixel and four adjacent pixels in the upper left and lower right and lower left and upper right directions with respect to the target pixel. apparatus. In claim 5,
A second image filter device that adds an offset to each pixel value of a coding unit using a change in a pixel value in a one-dimensional direction with respect to an output image of the first filter device,
The second image filter device indicates a second class indicating a direction to refer to the target pixel in a one-dimensional direction, a second category indicating a change pattern of pixel values in a one-dimensional direction, and a range of the target pixel value. A second setting unit for setting a band position;
A second adder that adds a second offset obtained by referring to the second category to the target pixel value from among the second offsets calculated by the second offset calculator;
The second adder did not add the second offset to the pixel to which the first offset was added in the first adder, and did not add the first offset in the first adder. An image filter device, wherein a second offset is added to a pixel. In claim 5,
A second image filter device that adds an offset to each pixel value of a coding unit using a change in a pixel value in a one-dimensional direction with respect to an input image of the first filter device,
The second image filter device indicates a second class indicating a direction to refer to the target pixel in a one-dimensional direction, a second category indicating a change pattern of pixel values in a one-dimensional direction, and a range of the target pixel value. A second setting unit for setting a band position;
A second adder that adds a second offset obtained by referring to the second category to the target pixel value from among the second offsets calculated by the second offset calculator;
The first addition unit of the first filter device adds the first offset to the pixel to which the second offset has been added by the second addition unit, and adds the second offset to the pixel by the second addition unit. An image filter device, wherein a first offset is not added to a pixel that has not been added. In claim 5,
A second image filter device that adds an offset to each pixel value of a coding unit using a change in pixel value in a one-dimensional direction with respect to an output image of the first filter device,
The second image filter device includes a first type using a second class indicating a direction in which a target pixel is referred to in a one-dimensional direction and a second type using a second category indicating a change pattern of pixel values in a one-dimensional direction. A second setting unit that sets a second type using a band position indicating a range of pixel values;
A second adder for adding a second offset obtained by referring to the second category or band position from the second offset calculated by the second offset calculator to the target pixel value. ,
The second addition unit adds an offset using the second type to the pixel to which the first offset has been added in the first addition unit, and adds the first offset in the first addition unit. An image filter apparatus, wherein an offset using the first type is added to a missing pixel.

Images