JP2021057620A - Image filter device, image decoding device, image encoding device, and data structure - Google Patents

Abstract

To provide an image filter device capable of achieving a high filter effect according to the characteristics of the sequence and region.SOLUTION: An adaptive offset filter 60 of a moving image decoding device 1 includes: band classification derivation means that classifies pixel values into one or more bands according to the pixel values in a unit area based on band setting information and derives of the band index indicating the band; offset selection means that selects one of the offsets according to the band index; and filter means that adds the selected offset to the pixel value.SELECTED DRAWING: Figure 12

Description

The present invention relates to an image filter device that filters images. The present invention also relates to a coding device including such an image filter and a decoding device. It also relates to the data structure of the coded data decoded by such a decoding device.

In order to efficiently transmit or record a moving image, a moving image coding device (encoding device) that generates coded data by encoding the moving image and decoding by decoding the coded data. A moving image decoding device (decoding device) that generates an image is used. Specific examples of the moving image coding method include the method adopted in HEVC.

In such a coding method, the image (picture) constituting the moving image is a slice obtained by dividing the image, a maximum coding unit (LCU: Largest Coding Unit, a tree block) obtained by dividing the slice. A hierarchical structure consisting of a coding unit (CU: Coding Unit, also called a coding node) obtained by dividing the maximum coding unit, and blocks and partitions obtained by dividing the coding unit. It is managed by, and is often encoded with a block as the smallest unit.

Further, in such a coding method, a predicted image is usually generated based on a locally decoded image obtained by encoding and decoding the input image, and the difference data between the predicted image and the input image is encoded. Will be done. Further, as a method of generating a prediction image, methods called inter-screen prediction (inter-screen prediction) and in-screen prediction (intra-prediction) are known.

Further, Non-Patent Document 1 discloses an adaptive offset filter (also referred to as "adaptive offset filter" or "SAO (Sample Adaptive Offset)") that adds an offset according to pixel classification for each pixel.

Further, Patent Document 1 describes a technique for finely classifying the band offset (BO) of SAO near the center of the range.

H.265 JP 2013-236358

The methods described in Non-Patent Document 1 and Patent Document 1 described above have a problem that different image characteristics are not utilized for each sequence or color component.

In particular, Non-Patent Document 1 has a problem that the width of the BO offset range cannot be adjusted, and Patent Document 1 has a problem that the adjustment of the BO offset range is limited to the vicinity of the center of the range. Further, there is a problem that the code amount of the information indicating the position of the band offset becomes large.

In order to solve the above problems, the image filter device according to the present invention is an image filter device that adds an offset to each pixel value of an input image composed of a plurality of unit areas, and is a pixel in the unit area. The pixel value is classified into one or more bands based on the band setting information according to the value, and one of the offsets is selected according to the band classification derivation means for deriving the band index indicating the band and the band index. The band classification derivation means is provided with an offset selection means for performing an image and a filter means for adding the selected offset to the pixel value, and the band classification / derivation means sets band setting information according to information in a unit area.

The band classification derivation means is characterized in that band setting information is set according to a color component. The image filter device means is characterized in that band setting information for each color component is set by the band classification derivation means. ..

The band classification derivation means is characterized in that band setting information is set according to at least one of at least one of a maximum value, a minimum value, and an average value of pixel values in a unit area.

The band classification derivation means sets the maximum value, the minimum value, and the average value of the pixel values of the unit area to the pixels having the lower predetermined width of the unit area and the pixels having the right side predetermined width (that is, the lower predetermined pixels of the unit area). The band setting information is derived by using (pixels other than the predetermined pixel on the right side).

The image filter device is provided with adaptive offset information copying means for copying already decoded offset information, and the adaptive offset information copying means is characterized in that it includes band setting information in addition to offset information.

The band setting information is characterized by being the width of the band and the head position of the band.

The band classification derivation means derives based on the head position of the first band and the head position of the second band, and the offset selection means is either or both of the first band and the second band. It is characterized in that the offset is selected based on one of them.

The head position of the first band and the head position of the band are shifted by half the bandwidth.

The band classification derivation means is characterized in that it selects whether to use the first band or the first band and the second band according to the range of the unit region.

The adaptive offset information copying means is characterized in that, in addition to the offset information, band number selection information of whether to use the first band or to use the first band and the second band is copied.

The image filter device is characterized in that the number of bands is further decoded.

The image filter device is characterized in that the number of offsets is further decoded.

An image decoding device including the above image filter device.

An image coding device including the above-mentioned image filter device.

A data structure of encoded data in which the predicted residual, which is the difference between the predicted image and the original image, is decoded by the image decoding device, and the pixel value obtained by decoding the predicted residual is banded. A data structure of encoded data, characterized in that it contains band setting information, which is a parameter used for classifying into.

As described above, by setting the band setting information according to the information of the unit area, there is an effect that the filter processing more adapted to the sequence or the area becomes possible.

It is a block diagram which shows the structure of the adaptive offset filter which concerns on embodiment of this invention. It is a figure which shows the hierarchical structure of the data of the coded stream which concerns on this embodiment. It is a block diagram which shows the structure of the moving image decoding apparatus which concerns on embodiment. It is a figure which shows the syntax included in the sequence parameter set SPS of the coded data which concerns on embodiment. It is a figure which shows the syntax slice_header () in an embodiment. It is a figure which shows the syntax coding_tree_unit () in an embodiment. It is a figure which shows the syntax sao_unit () in an embodiment. It is a figure which shows the syntax sao_offset () in an embodiment. It is a figure which shows the covering range of a band, and the pixel value distribution. It is a figure explaining the method for deriving the real offset SaoOffsetVal. It is a figure which shows the band setting information and the example of a band which concerns on embodiment. It is a figure which shows the band setting information and the example of a band which concerns on embodiment. It is a figure which shows the band setting information and the example of a band which concerns on embodiment. This is an example of changing the bandShift value for each color component. The syntax table when the bandwidth information sao_add_shift, which is the band setting information, is included in the slice data is shown. The syntax table when the bandwidth information sao_add_shift which is the band setting information is included in the CTU data is shown. It is a figure which shows the example of setting NUM_SAO_BO_CLASSES in the embodiment. It is a figure which shows the syntax slice_header () including num_sao_bo_classes in the embodiment. It is a figure which shows the example of setting sao_offset_num in the embodiment. It is a figure which shows the syntax slice_header () including sao_offset_num in embodiment. It is a figure explaining the head position of a band when a plurality of bands are used, and the band using the head position. It is a block diagram which shows the structure of the moving image coding apparatus which concerns on embodiment. It is a block diagram which shows the structure of the adaptive offset filter provided in the moving image coding apparatus which concerns on embodiment. It is a figure for demonstrating that a moving image decoding apparatus and a moving image coding apparatus can be used for transmission / reception of a moving image, and (a) is a block diagram which showed the structure of the transmitting apparatus equipped with the moving image coding apparatus. (B) is a block diagram showing a configuration of a receiving device equipped with a moving image decoding device. It is a figure for demonstrating that a moving image decoding apparatus and a moving image coding apparatus can be used for recording and reproducing a moving image, and (a) shows the structure of the recording apparatus equipped with the moving image coding apparatus 2. FIG. 3B is a block diagram showing a configuration of a playback device equipped with a moving image decoding device.

[Embodiment 1]
<Structure of coded stream Te>
Prior to the detailed description of the moving image coding device 11 and the moving image decoding device 31 according to the present embodiment, the data of the coded stream Te generated by the moving image coding device 11 and decoded by the moving image decoding device 31. The structure will be described.

FIG. 2 is a diagram showing a hierarchical structure of data in the coded stream Te. The coded stream Te typically includes a sequence and a plurality of pictures that make up the sequence. In FIGS. 2 (a) to 2 (f), a coded video sequence that defines the sequence SEQ, a coded picture that defines the picture PICT, a coded slice that defines the slice S, and a coded slice that defines the slice data, respectively. It is a figure which shows the coding unit (CU) included in the data, the coding slice data, and the coding tree unit.

(Coded video sequence)
The coded video sequence defines a set of data that the moving image decoding device 31 refers to in order to decode the sequence SEQ to be processed. As shown in FIG. 2A, the sequence SEQ includes a video parameter set VPS (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and Includes Supplemental Enhancement Information (SEI). Here, the value shown after # indicates the layer ID.

The video parameter set VPS is a set of coding parameters common to a plurality of moving images in a moving image composed of a plurality of layers, and coding related to the plurality of layers included in the moving image and individual layers. A set of parameters is specified.

The sequence parameter set SPS defines a set of coding parameters that the moving image decoding device 31 refers to in order to decode the target sequence. For example, the width and height of the picture are specified. There may be a plurality of SPSs. In that case, select one of multiple SPSs from PPS.

The picture parameter set PPS defines a set of coding parameters that the moving image decoding device 31 refers to in order to decode each picture in the target sequence. For example, a reference value of the quantization width used for decoding a picture (pic_init_qp_minus26) and a flag indicating the application of weighted prediction (weighted_pred_flag) are included. There may be a plurality of PPSs.
In that case, one of a plurality of PPSs is selected from each slice header in the target sequence.

(Coded picture)
The coded picture defines a set of data referred to by the moving image decoding device 31 in order to decode the picture PICT to be processed. _{The picture PICT contains slices S0 to S NS-1} as shown in FIG. 2 (b) (NS is the total number of slices contained in the picture PICT).

In the following, _{if it is not necessary to distinguish between slices S0 to S NS-1} , the subscripts of the symbols may be omitted. The same applies to the data included in the coded stream Te described below and with subscripts.

(Coded slice)
The coded slice defines a set of data referred to by the moving image decoding device 31 in order to decode the slice S to be processed. As shown in (c) of FIG. 2, the slice S includes the slice header SH and the slice data SDATA.

In the slice header SH, the moving image decoding device 31 is used to determine the decoding method of the target slice.
Contains a set of coding parameters referenced by. The slice type specification information (slice_type) that specifies the slice type is an example of the coding parameter included in the slice header SH.

The slice types that can be specified by the slice type specification information include (1) I slices that use only intra-prediction during coding, and (2) P-slices that use unidirectional prediction or intra-prediction during coding. (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of coding can be mentioned. Note that the inter-prediction is not limited to single prediction and bi-prediction, and a prediction image may be generated using more reference pictures. Below, P
, B slice refers to a slice containing a block for which inter-prediction can be used.

Further, the slice header SH includes a SAO parameter SAOP referred to by an adaptive offset filter included in the moving image decoding device 31. The details of the SAO parameter SAOP will be described later.

The slice header SH may include a reference (pic_parameter_set_id) to the picture parameter set PPS included in the coded video sequence.

The slice data defines a set of data referred to by the moving image decoding device 31 in order to decode the slice data SDATA to be processed. The slice data SDATA includes a coding tree unit (CTU: Coding Tree Unit, CTU block).

(Coded tree unit)
FIG. 2D defines a set of data referred to by the moving image decoding device 31 in order to decode the coded tree unit CTU. As shown in (d) of FIG. 2, the CTU includes a CTU header and a coding unit (CU: Coding Unit, CU block) CU.

The CTU header contains coding parameters referenced by the moving image decoding device 31 to determine the decoding method of the target CTU. Specifically, as shown in FIG. 2D, the division information SP_CU that specifies the division pattern of the target CTU to each CU, the quantization parameter difference Δqp (qp_delta) that specifies the size of the quantization step, And the SAO parameter SAOP referenced by the adaptive offset filter is included.

Tree block division information SP_CU is information representing a coded tree for dividing a CTU. Specifically, the shape and size of each CU included in the target CTU and its position within the target CTU are specified. Information.

The quantization parameter difference Δqp is the difference qp-qp between the quantization parameter qp in the target tree block and the quantization parameter qp'in the CTU encoded immediately before the target CTU.
'.

Further, the SAO parameter SAOP is a parameter referred to by the adaptive offset filter included in the moving image decoding device 31. The details of the SAO parameter SAOP will be described later.

In FIG. 2 (e), the moving image decoding device 3 is used to decode the coded tree unit to be processed.
A set of data referenced by 1 is specified. The coding tree unit is divided into a coding unit (CU: Coding Unit), which is a basic unit of coding processing, by recursive quadtree division (QT division) or binary tree division (BT division). .. The tree structure obtained by recursive quadtree division or binary tree division is called a coding tree (CT: Coding Tree), and the node of the tree structure is called a coding node (CN: Coding Node). The intermediate node of the quadtree and the binary tree is a coding node, and the coding tree unit itself is also defined as the highest-level coding node.

The CT includes, as CT information, a QT division flag (cu_split_flag) indicating whether or not to perform QT division, and a BT division mode (split_bt_mode) indicating a division method of BT division. cu_split_flag and / or split_bt_mode is transmitted for each coding node CN. When cu_split_flag is 1, the coding node CN is divided into four coding node CNs. When cu_split_flag is 0 and split_bt_mode is 1, the coding node CN is horizontally divided into two coding nodes CN, and when split_bt_mode is 2, the coding node CN becomes two coding nodes CN. When it is vertically divided and split_bt_mode is 0, the coding node CN is not divided and has one coding unit CU as a node. The coding unit CU is a terminal node (leaf node) of the coding node and is not further divided.

When the size of the coding tree unit CTU is 64x64 pixels, the size of the coding unit is 64x64 pixels, 64x32 pixels, 32x64 pixels, 32x32 pixels, 64x16 pixels, 16x64 pixels, 32x16.
Pixels, 16x32 pixels, 16x16 pixels, 64x8 pixels, 8x64 pixels, 32x8 pixels, 8x32 pixels, 16x8 pixels, 8x16 pixels, 8x8 pixels, 64x4 pixels, 4x64 pixels, 32x4 pixels, 4x32 pixels, 16x4 pixels, 4x16 pixels, 8x4 pixels, It can take either 4x8 pixels or 4x4 pixels.

(Code-coding unit)
FIG. 2F defines a set of data referred to by the moving image decoding device 31 in order to decode the coding unit to be processed. Specifically, the coding unit is composed of a prediction tree, a conversion tree, and a CU header CUH. The CU header defines the prediction mode, division method (PU division mode), and the like.

In the prediction tree, prediction parameters (reference picture index, motion vector, etc.) of each prediction unit (PU) obtained by dividing the coding unit into one or a plurality are defined. In other words, the prediction unit is one or more non-overlapping regions that make up the coding unit. The prediction tree also includes one or more prediction units obtained by the above division. In the following, the prediction unit obtained by further dividing the prediction unit will be referred to as a “sub-block”. The subblock is composed of a plurality of pixels. If the size of the prediction unit and the subblock are equal, there is only one subblock in the prediction unit. If the prediction unit is larger than the size of the subblock, the prediction unit is divided into subblocks. For example, when the prediction unit is 8x8 and the subblock is 4x4, the prediction unit is divided into four subblocks consisting of two horizontal divisions and two vertical divisions.

The prediction process may be performed for each prediction unit (subblock).

Roughly speaking, there are two types of predictions in the prediction tree: intra-prediction and inter-prediction. Intra-prediction is prediction within the same picture, and inter-prediction refers to prediction processing performed between pictures different from each other (for example, between display times and between layer images).

In the case of intra-prediction, there are two division methods: 2Nx2N (same size as the coding unit) and NxN.

In the case of inter-prediction, the division method is the PU division mode (part_mode) of the coded data.
2Nx2N (same size as the coding unit), 2NxN, 2NxnU, 2NxnD, Nx2N
, NLx2N, nRx2N, and NxN. Note that 2NxN and Nx2N show a 1: 1 symmetric division.
2NxnU, 2NxnD and nLx2N, nRx2N show 1: 3, 3: 1 asymmetric division. The PUs included in the CU are expressed as PU0, PU1, PU2, and PU3 in that order.

Further, in the conversion tree, the coding unit is divided into one or a plurality of conversion units TU, and the position and size of each conversion unit are defined. In other words, the conversion unit is one or more non-overlapping regions that make up the coding unit. The conversion tree also includes one or more conversion units obtained from the above divisions.

There are two types of division in the conversion tree: one that allocates an area of the same size as the coding unit as the conversion unit, and one that recursively divides into quadtrees as in the above-mentioned division of CU.

The conversion process is performed for each conversion unit.

(Prediction parameter)
The prediction image of the prediction unit (PU) is derived by the prediction parameters associated with the PU. The prediction parameters include prediction parameters for intra-prediction and prediction parameters for inter-prediction. Hereinafter, the prediction parameters of the inter-prediction (inter-prediction parameters) will be described. The inter-prediction parameter is composed of the prediction list usage flags predFlagL0 and predFlagL1, the reference picture indexes refIdxL0 and refIdxL1, and the motion vectors mvL0 and mvL1. The prediction list usage flags predFlagL0 and predFlagL1 are flags indicating whether or not reference picture lists called L0 list and L1 list are used, respectively, and when the value is 1, the corresponding reference picture list is used. In the present specification, when "a flag indicating whether or not it is XX" is described, it is assumed that the flag other than 0 (for example, 1) is XX, 0 is not XX, and logical negation, logical product, etc. Treat 1 as true and 0 as false (same below). However, in an actual device or method, other values can be used as true values and false values.

The syntax elements for deriving the inter-prediction parameters contained in the coded data include, for example, PU split mode part_mode, merge flag merge_flag, merge index merge_idx, inter-prediction identifier inter_pred_idc, reference picture index ref_idx_lX (refIdxLX), prediction vector. It has an index mvp_lX_idx and a difference vector mvdLX.

(Sequence parameter set SPS)
FIG. 4 shows a part of the syntax elements included in the sequence parameter set SPS (denoted as seq_parameter_set_rbsp () in FIG. 4) of the coded data according to the present embodiment.

profile_idc and level_idc indicate the profile and level to which the bitstream fits, respectively. seq_parameter_set_id indicates the identifier of the parameter set. chroma_format_idc indicates the color format of the target sequence. pic_width_in_luma_samples and pic_height_in_luma_samples indicate the vertical or horizontal length of the picture size included in the target sequence in units of luminance pixels, respectively. sample_adaptive_offset_enabled_flag indicates whether to apply the adaptive offset filter when decoding the target sequence. Adaptive_loop_filter_enabled_flag indicates whether to apply an adaptive filter (ALF) when decoding the target sequence.

(SHSAO parameter SAOP_S)
FIG. 5 is a diagram showing a syntax element of the SHSAO parameter SAOP_S included as a part of the slice header SH (denoted as slice_header () in FIG. 5) of the coded data according to the present embodiment. The SHSAO parameter SAOP_S is information indicating on / off of the parameters of the adaptive offset filter processing, and when slice_sao_luma_flag is 1, the adaptive offset filter is applied to the luminance (Y: color component cIdx == 0). If it is on or 0, it is turned off. If slice_sao_chroma_flag is 1, the adaptive offset filter is turned on for the color difference (Cb: color component cIdx == 1 and Cr: color component cIdx == 2), and if it is 0, it is turned off. It is called the parameter CTUSAO parameter SAOP_C of the adaptive offset filter applied to each CTU.

(CTUSAO parameter SAOP_C)
FIG. 6 is a diagram showing each syntax element of the CTUSAO parameter SAOP_C included as a part of CTU data (denoted as coding_tree_unit () in FIG. 6) of the coded data according to the present embodiment. The CTUSAO parameter SAOP_C generally includes CTUSAO unit information (sao_unit ()) and CTUSAO offset information (sao_offset ()) as syntax elements.

(CTUSAO unit)
FIG. 7 is a diagram showing the syntax of the CTUSAO unit (denoted as sao_unit () in FIG. 7) of the coded data according to the present embodiment. The CTUSAO unit represents a processing unit of SAO, is a syntax associated with one or more coding units CU, and is located near the coding data of the corresponding tree unit. The area on the image that the CTUSAO unit processes is the area indicated by the corresponding CTU. The CTUSAO unit generally includes CTUSAO offset information (sao_offset ()) and CTUSAO offset information merge control information (sao_merge_left_flag, sao_merge_up_flag) as syntax elements. sao_merge_left_flag is a flag indicating whether to duplicate the CTUSAO offset information of the target CTU from the left adjacent CTU of the target CTU. sao_merge_up_flag is a flag indicating whether to duplicate the CTUSAO offset information of the target CTU from the CTU above the target CTU.

(CTUSAO offset information)
FIG. 8 is a diagram showing the syntax of CTUSAO offset information (denoted as sao_offset () in FIG. 8) of coded data in the prior art. CTUSAO offset information includes an index indicating the offset type (offset type, sao_type_idx), an index indicating the position of the band to which the offset is applied within the pixel value range (band offset position, sao_band_position), and an offset value (sao_offset) as syntax elements. including. sao_offset is
It consists of the absolute value of the offset, sao_offset_abs, and the sign of the offset, sao_offset_sign. sao_band_position is used when the type of SAO is band offset (BO). The offset value in the coded data may be encoded as a predicted residual for deriving an offset value using a quantized value or a predicted value such as linear prediction. sao_type_idx and sao_offset will be described later.

The Descriptor ue (v) shown in FIGS. 4 to 8 indicates that the syntax associated with this descriptor is an unsigned numerical value, and the value is variable-length coded. There is. se (v) indicates that the syntax associated with this descriptor is a signed number, which is variable-length coded separately for the sign and the absolute value. ae (v) shows that the syntax associated with this descriptor is variable-length coded using arithmetic coding. u (n) indicates that the syntax associated with this descriptor is an unsigned number and that n-bit fixed-length coding is used. f (n) indicates that a fixed value bit pattern is used.

Next, the offset type sao_type_idx and the offset sao_offset will be described in more detail.

(Sao_type_idx)
sao_type_idx is an identifier that represents the type of adaptive offset filter. sao_type_idx
Is assigned to a predetermined unit (for example, CTU) for each color component. Therefore,
The type value in each of the given units is represented using sao_type_idx [component] [rx] [ry] and the subscript. The subscript component (cIdx) is an index indicating any of luminance, color difference Cr, and color difference Cb, and rx and ry indicate the x and y coordinates of a predetermined unit (for example, SAO unit or CTU) of interest, respectively. In this embodiment, sao_type_idx [component] [rx] [ry] may be simply referred to as sao_type_idx or offset type. Even if it is simply called sao_type_idx, the required individual values can be referred to as sao_type_idx [component] [rx] [ry].

The offset type is roughly divided into two types called edge offset processing (EO) and band offset processing (BO). The edge offset process is a process of adding any of a plurality of offsets to the pixel value of the pixel to be processed according to the difference between the pixel value of the pixel to be processed and the pixel value of the peripheral pixels thereof. Further, the band offset process refers to a process of adding any of a plurality of offsets to the pixel value of the pixel to be processed according to the value of each pixel to be processed.

In the present embodiment, sao_type_idx takes an integer value from 0 to 5, and what kind of filter processing is performed on the image before the offset filter (deblocked decoded image P_DB or locally decoded image P), or a filter. Indicates whether to perform processing. Specifically, it is as follows.
-Sao_type_idx = 0 indicates that the offset filter processing is not performed on the image before the offset filter in the target CTU.
-Sao_type_idx = 1 to 4 indicates that edge offset processing (EO) is performed on the image before the offset filter in the target CTU.
-Sao_type_idx = 5 indicates that band offset processing (BO) is performed on the image before the offset filter in the target CTU.

Note that a number of offset types other than those described above may be used for edge offset and band offset. When the number of offset types is different from that of the present embodiment, the range of values of sao_type_idx changes according to the total number of offset types. For example, if the total number of offset types is 4, including the case where offset processing is not performed, sao_type_idx takes an integer value from 0 to 3.

When band offset processing is performed, sao_band_position that specifies the left end position of the pixel value band to be offset processing is encoded.

(SaoOffsetVal)
sao_offset_abs [component] [rx] [ry] [i] and sao_offset_sign [component] [rx] [ry] [i] are added to each pixel included in the target tree block in the adaptive offset filter processing according to the present embodiment. Offset value to be SaoOffsetVal [component] [rx] [ry] [i] is a parameter used to derive. In this embodiment, sao_offset_abs [component] [rx] [ry] [i], sao_offset_sign [component] [rx] [ry] [i] may be simply referred to as sao_offset_abs, sao_offset_sign or offset value. Even when simply called sao_offset_abs and sao_offset_sign, the required individual values can be referred to as sao_offset [component] [rx] [ry] [i] and sao_offset_sign [component] [rx] [ry] [i]. Is.

sao_offset_abs [component] [rx] [ry] [i] and sao_offset_sign [component] [rx] [ry] [i] are specified by the index i in addition to the above arguments component, rx, ry. Here, the index i is an index for designating a class (candidate for a pixel array to be referred to) corresponding to the offset. The index i takes a value of 0 to N-1 if there are N classes belonging to the offset type to be applied.

In the processing of the adaptive offset filter in the present embodiment, the absolute value of the offset sao_offset_abs and the sign sao_offset_sign are included in the coded data, and the moving image decoding device 31 includes the offset value SaoOffsetVal (actually) actually used from the values of sao_offset_abs and sao_offset_sign. (Also called offset) is derived and used.

SaoOffsetVal [cIdx] [rx] [ry] [i + 1] = (1-2 * sao_offset_sign [cIdx] [rx] [ry] [i]) * sao_offset_abs [cIdx] [rx] [ry] [i] <<log2OffsetScale
Here, log2OffsetScale is a quantization parameter used when quantizationing an offset, and a value such as 0, 1, 2 is usually used.

In addition, sao_offset_sign is encoded as a parameter for deriving the sign of the offset value to be added to each derived pixel. sao_offset_sign encodes when the offset value to be added is other than 0.

Next, the range of values of sao_offset will be described.

If the offset type sao_type_idx is not 5, that is, in the EO offset sao_offset, the range is
From 0 to (1 << (Min (bitDepth, bitDepthTh)-bitDepthD))-1
Is. Here, bitDepth is a value representing the bit depth of the pixel value to which the offset is applied, bitDepthTh is the lower limit bit depth for quantizing the offset value, and bitDepthD is a value for adjusting the number of bits in the offset value range. Also, << represents a left shift, and Min () represents a function that selects the smallest argument.

On the other hand, when the offset type sao_type_idx is 5, that is, in the band offset offset sao_offset, the range is.
-(1 << (Min (bitDepth, bitDepthTh) -bitDepthD)) to (1 << (Min (bitDepth, bitDepthTh) -bitDepthD) -1
Is. From the above, for example, when bitDepth = 8, bitDepthTh = 10, and bitDepthD = 5, the range of the offset sao_offset is 0 to 7 in the case of edge offset and -7 to 7 in the case of band offset.

(Video decoding device 31)
FIG. 4 shows the moving image decoding device (image decoding device) 31 of the present invention.
The moving image decoding device 31 includes an entropy decoding unit 301, a prediction parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, and a prediction image generation unit (prediction image generation device) 308. It is composed of an inverse quantization / inverse conversion unit 311 and an addition unit 312.

Further, the prediction parameter decoding unit 302 includes an inter prediction parameter decoding unit 303 and an intra prediction parameter decoding unit 304. The prediction image generation unit 308 includes an inter-prediction image generation unit 309 and an intra-prediction image generation unit 310.

The loop filter 305 also includes at least one of a deblocking filter 3051, an adaptive offset (SAO) 3052, and an adaptive loop filter (ALF) 3053.

In the following, an example in which CTU, CU, PU, and TU are used as the processing unit will be described, but the present invention is not limited to this example, and processing may be performed in CU units instead of TU or PU units. Or CTU, CU, P
U and TU may be read as blocks and processed in block units.

The entropy decoding unit 301 performs entropy decoding on the coded stream Te input from the outside, separates and decodes each code (syntax element). The separated codes include prediction parameters for generating a prediction image, residual information for generating a difference image, and the like.

The entropy decoding unit 301 outputs a part of the separated codes to the prediction parameter decoding unit 302. The entropy decoding unit 301 outputs the quantization conversion coefficient to the inverse quantization / inverse conversion unit 311. This quantization transform coefficient is used for the residual signal in the coding process, such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), and KLT (Karyhnen Loeve Transform). ) Etc. is a coefficient obtained by performing frequency conversion and quantization. In addition, the entropy decoding unit 301 decodes the SAO parameter SAOP from the coded stream and supplies them to the adaptive offset 3052. The SAO parameter SAOP contains at least offset information.

The inter-prediction parameter decoding unit 303 decodes the inter-prediction parameter with reference to the prediction parameter stored in the prediction parameter memory 307 based on the code input from the entropy decoding unit 301. Further, the inter-prediction parameter decoding unit 303 outputs the decoded inter-prediction parameter to the prediction image generation unit 308, and stores the decoded inter-prediction parameter in the prediction parameter memory 307.

The intra prediction parameter decoding unit 304 decodes the intra prediction parameter with reference to the prediction parameter stored in the prediction parameter memory 307 based on the code input from the entropy decoding unit 301. The intra prediction parameter is a parameter used in the process of predicting the CU in one picture, for example, the intrapred mode. The intra prediction parameter decoding unit 304 outputs the decoded intra prediction parameter to the prediction image generation unit 308, and stores the decoded intra prediction parameter in the prediction parameter memory 307.

The loop filter 305 applies filters such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the addition unit 312. Details of the loop filter 305 will be described later.

The reference picture memory 306 stores the decoded image of the CU generated by the addition unit 312 at a position predetermined for each of the picture to be decoded and the CTU or CU.

The prediction parameter memory 307 stores the prediction parameters at predetermined positions for each picture and prediction unit (or subblock, fixed size block, pixel) to be decoded.

The prediction mode predMode input from the entropy decoding unit 301 is input to the prediction image generation unit 308, and the prediction parameters are input from the prediction parameter decoding unit 302. Further, the prediction image generation unit 308 reads the reference picture from the reference picture memory 306. The prediction image generation unit 308 generates a prediction image of a PU (block) or a subblock using the input prediction parameter and the read reference picture (reference picture block) in the prediction mode indicated by the prediction mode predMode, and the addition unit. Output to 312.

When the prediction mode predMode indicates the inter-prediction mode, the inter-prediction image generation unit 309 blocks or blocks by inter-prediction using the inter-prediction parameter input from the inter-prediction parameter decoding unit 303 and the read reference picture (reference picture block). Generate a predicted image of the subblock.

When the prediction mode predMode indicates the intra prediction mode, the intra prediction image generation unit 310 performs intra prediction using the intra prediction parameters input from the intra prediction parameter decoding unit 304 and the reference pixels read out.

The inverse quantization / inverse conversion unit 311 inversely quantizes the quantization conversion coefficient input from the entropy decoding unit 301 to obtain the conversion coefficient. The inverse quantization / inverse conversion unit 311 performs inverse frequency conversion such as inverse DCT, inverse DST, and inverse KLT on the obtained conversion coefficient, and calculates a predicted residual signal. The inverse quantization / inverse conversion unit 311 outputs the calculated residual signal to the addition unit 312.

The addition unit 312 adds the prediction image of the block input from the inter-prediction image generation unit 309 or the prediction image generation unit 308 and the residual signal input from the inverse quantization / inverse conversion unit 311 for each pixel to block. Generate a decoded image of. The addition unit 312 outputs the decoded image of the generated block to the loop filter 305.

In the loop filter 305, the deblocking filter 3051 has block distortion when the difference between the pixel values of the pixels adjacent to each other via the block boundary in the locally decoded image P or the CU boundary is within a predetermined range. Then it is determined. Then, by performing deblocking processing on the block boundary or the CU boundary in the locally decoded image P, the image in the vicinity of the block boundary or the CU boundary is smoothed. The image subjected to the deblocking process by the deblocking filter 3051 is the deblocked decoded image P_DB. The locally decoded image P or the deblocked decoded image P_DB is output to the adaptive offset filter 3052. The input of the adaptive offset filter 3052 is the deblocked decoded image P_DB when the deblocking filter is on, and the locally decoded image P when the deblocking filter is off.

The adaptive offset filter 3052 generates an offset filtered image P_OF by performing an offset filter process using the offset decoded from the coded data with the CTU as a processing unit on the input image of the filter. The generated offset filtered decoded image P_OF is supplied to the adaptive filter 3053. Details of the adaptive offset filter 3052 will be described later.

The adaptive filter 3053 uses the filter parameter FP (filter set number, filter coefficient group, area designation information, and on / off information) decoded from the coded data for the offset filtered decoded image P_OF supplied from the adaptive offset filter 3052. The filtered decoded image P_FL is generated by performing the filtering process using. The filtered decoded image P_FL is output to the outside as a decoded moving image Td, and the entropy decoding unit 301
It is stored in the prediction parameter memory 307 in association with the POC designation information decoded from the coded data by.

(Details of Adaptive Offset Filter 3052)
Next, the details of the adaptive offset filter 3052 will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of an adaptive offset filter 3052. As shown in FIG. 1, the adaptive offset filter 3052 includes a SAO parameter decoding unit 61, an adaptive filter unit 62, an offset setting unit 63, and a SAO parameter storage unit 64.
(SAO parameter decoding unit 61)
The SAO parameter decoding unit 61 (offset information decoding means) extracts the offset type sao_type_idx, the band offset position sao_band_position, the offset sao_offset_abs, and the offset code sao_offset_sign from the input SAO parameter SAOP. The band offset position sao_band_position indicates the position of the band of the number of bands NUM_SAO_BO_CLASSES, and it is preferable to decode the binary strings having different lengths depending on the NUM_SAO_BO_CLASSES. For example, a binary string of binary length of log2 (NUM_SAO_BO_CLASSES) length may be decrypted. The offset type sao_type_idx and band offset position sao_band_position are output to the characteristic value calculation unit 621, and the offset type sao_type_idx, offset sao_offset_abs and offset code sao_offset_sign are the offset setting unit 63 and the SAO parameter storage unit 6.
It is output to 4. Depending on whether or not the merge control information (sao_merge_left_flag, sao_merge_up_flag) of CTUSAO offset information, which is the SAO parameter SAOP for the target CTU, indicates that it is a merge, the offset type sao_type_idx, band offset position sao_band_position, offset sao_offset_abs, and offset code The method of extracting sao_offset_sign is different. When the merge control information indicates that the merge is performed, the parameters that have already been decoded are read out from the SAO parameter storage unit 64 and extracted. Specifically, when sao_merge_left_flag is 1, the parameters of the CTU adjacent to the left of the target CTU are extracted. When sao_merge_up_flag is 1, the parameters of the CTU adjacent to the target CTU are extracted. On the contrary, when the merge control information does not indicate that it is a merge, the parameter is extracted from the input SAO parameter SAOP. In the present specification, when the merge control information indicates that it is a merge, the band setting information is also copied in addition to the above parameters. The band setting information is a parameter used to classify the pixel values into bands. Specifically, the band start position bandStartPos and the bandwidth information (for example, bandW and bandShift) are copied as the band setting information.

More specifically, when sao_merge_up_flag = 1,
sao_offset_abs [cIdx] [rx] [ry] [i] = sao_offset_abs [cIdx] [rx] [ry-1] [i]
sao_offset_sign [cIdx] [rx] [ry] [i] = sao_offset_sign [cIdx] [rx] [ry-1] [i]
sao_type_idx [cIdx] [rx] [ry] [i] = sao_type_idx [cIdx] [rx] [ry-1] [i]
bandStartPos [cIdx] [rx] [ry] = bandStartPos [cIdx] [rx] [ry-1] [i]
bandShift [cIdx] [rx] [ry] = bandShift [cIdx] [rx] [ry-1] [i]
Derived by. If sao_merge_left_flag = 1,
sao_offset_abs [cIdx] [rx] [ry] [i] = sao_offset_abs [cIdx] [rx-1] [ry] [i]
sao_offset_sign [cIdx] [rx] [ry] [i] = sao_offset_sign [cIdx] [rx-1] [ry] [i]
sao_type_idx [cIdx] [rx] [ry] [i] = sao_type_idx [cIdx] [rx-1] [ry] [i]
bandStartPos [cIdx] [rx] [ry] = bandStartPos [cIdx] [rx-1] [ry] [i]
bandShift [cIdx] [rx] [ry] = bandShift [cIdx] [rx-1] [ry] [i]
Derived by. Here, rx and ry are the upper left coordinates of the target CTU. The bandwidth information to be copied is not limited to the number of bandwidth shifts, bandShift, and may be bandwidth bandW. Further, the shift value addShift (for example, bandShift = BASE_SHIFT + addShift) for deriving the number of bandwidth shifts bandShift may be used.

bandW [cIdx] [rx] [ry] = bandW [cIdx] [rx] [ry-1] [i]
bandW [cIdx] [rx] [ry] = bandW [cIdx] [rx-1] [ry] [i]
addShift [cIdx] [rx] [ry] = addShift [cIdx] [rx] [ry-1] [i]
addShift [cIdx] [rx] [ry] = addShift [cIdx] [rx-1] [ry] [i]
Further, the SAO parameter decoding unit 61 may copy the band number selection information dblBandFlag as in the following pseudo code when sao_merge_up_flag = 1 or sao_merge_left_flag = 1.

dblBandFlag [cIdx] [rx] [ry] = dblBandFlag [cIdx] [rx] [ry-1] [i]
dblBandFlag [cIdx] [rx] [ry] = dblBandFlag [cIdx] [rx-1] [ry] [i]
When storing the band start position bandStartPos, in order to reduce the memory size required for storage in the parameter storage unit 64, the round processing described later may be performed in which the information of the lower bits is discarded. Round processing is effective in reducing line memory.

Further, as described below, round processing may be performed when reading the band start position bandStartPos from the SAO parameter storage unit 64.

bandStartPos [cIdx] [rx] [ry] = ((bandStartPos [cIdx] [rx] [ry-1] [i]) >> shift) << shift
bandShift [cIdx] [rx] [ry] = (bandShift [cIdx] [rx-1] [ry] [i]) >> shift) << shift
Further, the offset may be added at the time of round as follows.

bandStartPos [cIdx] [rx] [ry] = ((bandStartPos [cIdx] [rx] [ry-1] [i] + offset) >> shift) << shift
bandShift [cIdx] [rx] [ry] = (bandShift [cIdx] [rx-1] [ry] [i] + offset) >> shift) << shift
For example offset = 1 << (shift-1)
In the band offset, when the band setting information changes, the band classified even with the same pixel value changes, so the position of the band to which the offset information is applied also changes. As described above, the SAO parameter decoding unit 61 (adaptive offset information copying means) for copying the already decoded offset information is provided, and the adaptive offset information copying means copies the band setting information in addition to the offset information. Even when the band setting information changes for each unit area such as CTU, there is an effect that the offset information is applied to the expected band.

The band setting information may be any one or more of the band width, the band head position, and the band number selection information.

(Offset setting unit 63)
The offset setting unit 63 derives the actual offset SaoOffsetVal from the input offset type sao_type_idx, offset sao_offset, and offset code sao_offset_sign, and outputs the actual offset SaoOffsetVal to the offset allocation unit 624.

In addition, when deriving the actual offset SaoOffsetVal, inverse quantization is performed according to the bit depth of the pixel value. For example, if the offset value of the adaptive offset filter for an image with a bit depth of 12 bits is quantized and encoded to the equivalent of a bit depth of 10 bits, SaoOffsetVal is shifted left by 2 bits to derive the actual offset value. .. The method for deriving the specific actual offset SaoOffsetVal will be described later.
(SAO parameter storage unit 64)
The SAO parameter storage unit 64 stores the offset type sao_type_idx, the band offset position sao_band_position, the offset sao_offset_abs, and the offset code sao_offset_sign extracted by the SAO parameter decoding unit 61. The actual offset (SaoOffsetVal) may be stored instead of the absolute value and the sign.
(Adaptive filter unit 62)
As shown in FIG. 1, the adaptive filter unit 62 includes a characteristic value calculation unit 621, an offset allocation unit 624, and an offset application unit 625.

(Characteristic value calculation unit 621)
The characteristic value calculation unit 621 calculates the characteristic value corresponding to the offset type sao_type_idx based on the pixel value of the input image. The calculation of the characteristic value is executed at each pixel position. If sao_type_idx is 0, the characteristic value is not calculated. When sao_type_idx is 1 to 4 (EO), the characteristic value edgeIdx (class) representing the edge classification is calculated. The characteristic value edgeIdx is set based on the pixel value change between the target pixel and the adjacent pixel in a specific direction according to sao_type_idx. When sao_type_idx is 5 (BO), the characteristic value bandIdx indicating the size of the pixel value is calculated. Characteristic value b
andIdx is calculated as a quantized value of the pixel value of the target pixel.

In the case of band offset, the characteristic value calculation unit 621 includes pixel value recPicture [x] [y], band start position bandStartPos which is band setting information (band classification information), bandwidth information (bandwidth shift value bandShift and band). The band index bandIdx, which is a characteristic value, is calculated using the width bandW) and the band offset position sao_band_position, which is a parameter transmitted as encoded data.

bandIdx = ((recPicture [xSi] [ySj] -bandStartPos-sao_band_position) * bandW) >> bandShift
Here, recPicture [x] [y] is the pixel value of the input image of the adaptive filter. Bandwidth information bandW is (1 << bitDepth) >> bandShift. Here, bandStartPos = 0 and bandShift = 5 may be set. Further, clipping may be performed, or a predetermined value (for example, 1) may be added.

bandIdx = Clip3 (0, NUM_SAO_BO_CLASSES-1, bandIdx) +1
Further, the characteristic value calculation unit 621 may derive bandIdx using the table bandTable. Specifically, the bandTable may be derived from the band offset position sao_band_position by the following processing.

Set all elements of bandTable [] to 0
for (k = 0; k <sao_offset_num; k ++) bandTable [(k + sao_band_position) & (NUM_SAO_BO_CLASSES-1)] = k + 1
Here, the actual offset number sao_offset_num may be, for example, 4, and the band number NUM_SAO_BO_CLASSES may be, for example, 32. For example, if sao_band_position is 1, bandTable [] will have the following values. That is, a bandTable is a table in which sao_band_position to sao_band_position + 3 is 1,2,3,4, and the others are 0, and the offset of SaoOffsetVal [cIdx] [rx] [ry] [bandIdx] is set at bandIdx = 0. By using, the offset other than the band selected from 1 to 4 is 0.
Can be.

bandTable [0] = 0
bandTable [1] = 1 // bandTable [(0 + 1) & 31]
bandTable [2] = 2
bandTable [3] = 3
bandTable [4] = 4
bandTable [5] = 0
..
bandTable [31] = 0
Subsequently, the characteristic value calculation unit 621 may derive the band start position bandStartPos of the band setting information by the following equation.

bandIdx = bandTable [(recPicture [xSi] [ySj]-bandStartPos)] >> bandShift
The number of sao_offset_num bands (4 in this case) is encoded from the position specified by the band start position bandStartPos and the band offset position sao_band_position. Here, the bandwidth bandW is (1 << bitDepth) << bandShift, and in the example of bitDepth = 10 and bandShift = 5, 1024 pixel values from 0 to 1023 are set for each bandW (= 32). Divide into 1 << bandShift (= 32) bands. The band index bandIdx is derived from the pixel values recPicture [x] [y] by subtracting the position derived by bandStartPos and sao_band_position and shifting to the right by bandShift (dividing by bandW) as described above. Further, as described above, the difference between the pixel values recPicture [x] [y] and bandStartPos may be shifted to the right by bandShift.

(Pixel value distribution and band)
The band offset divides the pixel values above the band start position bandStartPos into NUM_SAO_BO_CLASSES bands with a bandwidth bandW. The range covered by this band is also called the support band range rangeBand. FIG. 9 is a diagram showing a band coverage range and a pixel value distribution. As shown in the figure, when the bandstart position bandStartPos is fixed to 0 and the pixel value bit range range (0 .. (1 << bitDepth) -1) is divided into NUM_SAO_BO_CLASSES (bandShift = log2 (NUM_SAO_BO_CLASSES)). Then, it can be seen that the support band range rangeBand is larger and the bandW is coarser than the pixel value distribution, especially in the color difference components of Cb and Cr. In this case, even if the band offset position is further selected by the band offset position sao_band_position, a band that does not actually occur is often specified, and the number of bits of sao_band_position is wasted. Moreover, since bandW is coarse, detailed processing cannot be realized. FIG. 11 is a diagram showing a pixel value distribution when the band start position bandStartPos is other than 0.

(Band setting information)
FIG. 12 shows band setting information and an example of the band according to the embodiment.
FIG. 12A is an example of the band start position bandStartPos = 0 and the bandwidth shift value bandShift = 5. (b) (c) (d) is the band start position bandStartPos = (1 << (bitDepth-1))-((1 << (bitDepth-1)) >> addShift), bandwidth shift value bandShift = 5 In + addShift, this is an example of addShift = 1, 2, 3. At this time, the bandwidth becomes (1 << bitDepth) >> (5 + addShift), and different bandwidths can be used depending on the value of addShift. The smaller the bandwidth, the finer the classification according to the pixel value value, and the more detailed offset can be added, but the range of the pixel value covered by one band (offset) becomes smaller. Further, as the bandwidth becomes smaller, the range covered as a whole becomes smaller even if the band offset position sao_band_position is applied, so it is advisable to set the band start position bandStartPos to apply. The maximum number of bands NUM_SAO_BO_CLASSES is 32 examples regardless of (a), (b), (c), and (d).

As described above, it is preferable to adaptively set the band start position bandStartPos and the bandwidth shift value bandShift, which are the band setting information.

FIG. 13 shows another example of the band setting information and the band according to the embodiment. Unlike FIG. 12, the bandwidth is not smaller than (1 << bitDepth) >> 6. The maximum number of bands NUM_SAO_BO_CLASSES is (a), (b), (c), and (d), which are examples of 32, 32, 16, and 8, respectively.

(Band setting information for each color component)
FIG. 14 shows an example in which the bandShift value is changed for each color component. In this example (a), bandShift = 5 is set for luminance (Y), bandShift = 7 for color difference (Cb), and bandShift = 7 for color difference (Cr). When bitDepth = 10, the bandwidths are 32, 8 and 8, respectively, and a smaller bandwidth is used in the color difference image than in the luminance image. In terms of color difference, the dynamic of the image is smaller than that of the luminance, so such a setting is appropriate. In (b), examples of 5, 7, and 6 are shown for Y, Cb, and Cr, respectively. The bandShift value set for each color component is not limited to this.

Also, determine the band start position for each color component according to the band start position bandStartPos = (1 << (bitDepth-1))-((1 << (bitDepth-1)) >> (bandShift-5)). May be good.

(Encoding of band setting information for each color component)
The dynamic range differs depending on the sequence or picture, the RGB or YUV color space, or the YUV color space such as BT.709, BT.2020, or BT.2100. Therefore, it is also appropriate to encode the band setting information of the color component with a slice header or the like instead of fixing it. FIG. 15 shows a syntax table when the bandwidth information sao_add_shift, which is the band setting information, is included in the slice data. The syntax table may include part of the band setting information. In this example, the SAO parameter decoding unit 61 reads the band setting information sao_add_shift [cIdx] from the coded data for each color component cIdx (= 0,1,2) in slice units and adds them to a predetermined band shift value. The band shift value is derived as follows.

bandShift = BASE_SHIFT + sao_add_shift [cIdx]
Here, the predetermined band shift value BASE_SHIFT may be set to 5, but the present invention is not limited to this. It may be set depending on the number of bands NUM_SAO_BO_CLASSES as follows.

BASE_SHIFT = log2 (NUM_SAO_BO_CLASSES)
FIG. 16 shows a syntax table when the bandwidth information sao_add_shift, which is the band setting information, is included in the CTU data. In this case as well, the SAO parameter decoding unit 61
In CTU units, the band setting information sao_add_shift [cIdx] is read from the coded data for each color component cIdx (= 0,1,2) and added to the predetermined band shift value to derive the band shift value.

The number of bands NUM_SAO_BO_CLASSES may be changed according to bandShift. For example, as shown below, a configuration in which the number of bands NUM_SAO_BO_CLASSES decreases as bandShift increases is preferable.

NUM_SAO_BO_CLASSES = 32 >> (bandShift -BASE_SHIFT)
(Derivation of distribution information of unit area)
The characteristic value calculation unit 621 may derive the distribution information of the unit region (CTU in this case) of the input image and determine the band setting information. The distribution information is one of the minimum value minCTU, re-approximate maxCTU, mean value aveCTU, median medCTU, most frequent modeCTU, 1/4 quantile hige1CTU, and 3/4 quantile hige3CTU of the pixel value in the unit area. There may be more than one. For example, the maximum value maxCTU, the minimum value minCTU, and the average value aveCTU may be derived by the following pseudo code.

minCTU = 1 << bitDepth
maxCTU = 0
aveCTU = 0
for (y = yCTU; y <yCTU + ctuHeight; y ++)
{
for (x = xCTU; x <xCTU + ctuWidth; x ++)
{
if (minCTU> recPicture [x] [y]) minCTU = recPicture [x] [y]
if (maxCTU <recPicture [x] [y]) maxCTU = recPicture [x] [y]
aveCTU + = recPicture [x] [y]
}
}
aveCTU = aveCTU >> log2 (ctuHeight * ctuWidth)
Here, xCTU and yCTU are the upper left coordinates of the target unit area, and ctuWidth and ctuHeight are the width and height of the target unit area.

In the band classification derivation means (characteristic value calculation unit 621), the distribution information of the unit area (for example, the maximum value, the minimum value, and the average value of the pixel values) is set to the left predetermined width (for example, ctuWidth-guardSize) of the unit area. Derived using the pixels in the range of and the pixels in the upper predetermined height (for example, ctuHeight-guardSize) range (that is, the pixels excluding the pixels in the right predetermined range on the right side of the unit area and the pixels in the lower predetermined range). You may. If the range to be excluded is set to guardSize, it can be derived with the following pseudo code.

The exclusion range considers that when the deblocking filter is performed, the pixel values on the right side and the lower side of the CTU are not fixed until the subsequent deblocking filter processing is performed. By providing an exclusion range, minCTU and maxCTU can be calculated without waiting for the subsequent deblocking filter processing. The guardSize may be set to 4 in consideration of the deblocking filter processing. Further, the guardSize may be 4 when the deblocking filter processing is performed, and 0 in other cases.

minCTU = 1 << bitDepth
maxCTU = 0
aveCTU = 0
for (y = yCTU; y <yCTU + ctuHeight-guardSize; y ++)
{
for (x = xCTU; x <xCTU + ctuWidth-guardSize; x ++)
{
if (minCTU> recPicture [x] [y]) minCTU = recPicture [x] [y]
if (maxCTU <recPicture [x] [y]) maxCTU = recPicture [x] [y]
aveCTU + = recPicture [x] [y]
}
}
aveCTU = aveCTU >> log2 ((ctuHeight-guardSize) * (ctuWidth-guardSize))
(Distribution information and round of band start position bandStartPos)
The characteristic value calculation unit 621 may perform a round with respect to the derived minimum value minCTU. For example, the following equation may be used so as to limit the amount of information to the range of K bit (for example, K = 8).

minCTU = (minCTU >> (bitDepth-K)) << (bitDepth-K)
Here, you may shift after adding an appropriate offset offset at the round of minCTU. offset = 1 << (1-shift), where shift = bitDepth-K may be used.

As another example, the characteristic value calculation unit 621 may perform a round so as to be a multiple of the bandwidth bandW with respect to the derived minimum value minCTU.

minCTU = (minCTU / bandW) * bandW
This can also be derived by the following shift operation when bandW is a power of 2.

minCTU = (minCTU >> bandShift) << bandShift
Or
minCTU = (minCTU >> (BASE_SHIFT + addShift)) << (BASE_SHIFT + addShift)
Here, addShift may decode sao_add_shift [cIdx] from the encoded data, or may derive it from the pixel value of the unit region as described in the following (input image-dependent band setting information 1). Good. For sao_add_shift [cIdx], the two components cIdx = 1 and 2 of the color difference may have the same sao_add_shift. That is, you may decode sao_add_shift [1] and set it to sao_add_shift [2].

Similarly, the band start position bandStartPos may be rounded.

bandStartPos = (bandStartPos >> (bitDepth-8)) << (bitDepth-8)
bandStartPos = (bandStartPos / bandW) * bandW
bandStartPos = (bandStartPos >> bandShift) << bandShift
This round of minimum minCTU, round of band start position bandStartPos, is also applicable to other embodiments herein.

(Band setting information 1 depending on the input image)
The characteristic value calculation unit 621 (band classification derivation means) may set band setting information according to at least one of at least the maximum value, the minimum value, and the average value of the pixel values in the unit region.

For example, the characteristic value calculation unit 621 may derive the band start position bandStartPos by the following formula using the minimum value minCTU. As mentioned above, minCTU may make rounds.

bandStartPos = minCTU
Further, the characteristic value calculation unit 621 (band classification derivation means) may set the bandwidth according to the dynamic range rangeCTU = maxCTU --minCTU of the unit region. For example, it may be derived by the following equation.

addShift = 0
if (1 <<(bitDepth-1)> rangeCTU) addShift = 1
if (1 <<(bitDepth-2)> rangeCTU) addShift = 2
bandShift = BASE_SHIFT + addShift
Further, as described above, the band start position bandStartPos may be derived by the following equation.

bandStartPos = (1 << (bitDepth-1))-((1 << (bitDepth-1)) >> (bandShift-BASE_SHIFT))
In this case, the band start position bandStartPos is derived according to the dynamic range of the unit region.

(Band setting information 2 depending on the input image)
The characteristic value calculation unit 621 (band classification derivation means) may set the band start position bandStartPos according to the dynamic range rangeCTU = maxCTU --minCTU of the unit region. For example, bandStartPos may be derived using the following pseudo code.

rangeBand = (1 << channelBitDepth) >>addShift;
if (rangeBand> rangeCTU)
{
bandStartPos = minCTU;
}
else else
{
bandStartPos = (minCTU + maxCTU + 1) >>1;
}
Here, in a certain addShift, if the band support width rangeBand is larger than the dynamic range rangeCTU, the minimum value minCTU of the unit area is set to bandStartPos, and the rangeCTU.
In the following cases, set the average value of the maximum value and the minimum value in bandStartPos. Instead of the average value of the maximum value and the minimum value, the average value aveCTU of the unit area may be used (input image-dependent band setting information 3).
The characteristic value calculation unit 621 (band classification derivation means) may set the band start position bandStartPos according to the dynamic range rangeCTU = maxCTU --minCTU of the unit region. For example, bandStartPos may be derived using the following pseudo code.

rangeBand = (1 << channelBitDepth) >>addShift;
if (rangeBand> rangeCTU)
{
bandStartPos = minCTU;
}
else else
{
bandStartPos = ((minCTU + maxCTU + 1) >> 1)-(rangeBand >>1);
}
Here, in a certain addShift, if the band support width rangeBand is larger than the dynamic range rangeCTU, the minimum value minCTU of the unit area is set to bandStartPos, and the rangeCTU.
In the following cases, set bandStartPos to the value obtained by subtracting 1/2 of rangeBand from the average value of the maximum and minimum values. Instead of the average value of the maximum value and the minimum value, the average value aveCTU of the unit area may be used.

(Band setting information 4 depending on the input image)
Further, the characteristic value calculation unit 621 (band classification derivation means) may derive the band start position bandStartPos by the following equation according to the dynamic range of the unit region. Here, addShiftFirst (for example, 2 or 3) is assigned to shift, and it is determined whether the rangeBand at the shift value is at least a predetermined time (here, rangeTH / 4) of the dynamic range (maxCTU --minCTU). If YES (if the rangeBand is large enough compared to the dynamic range of the unit area), select that shift. If NO (other than that), shift is subtracted by 1, and the same process is repeated until shift becomes 0 or the determination becomes YES.

for (shift = addShiftFirst; shift> = 0; shift--)
{
rangeBand = (1 << channelBitDepth) >>shift;
if (4 * rangeBand> = rangeTH * (maxCTU --minCTU))
{
break;
}
}
addShift = shift
bandShift = BASE_SHIFT + addShift
bandStartPos = minCTU
(Band setting information 5 depending on the input image)
As a modification, the search may be performed from the one with the larger shift value. Here, the shift value is searched from 0 to addShiftLast-1, and if the rangBand is sufficiently smaller than the dynamic range, that shift value is selected as the addShift value.

for (shift = 0; shift <addShiftLast; shift ++)
{
rangeBand = (1 << channelBitDepth) >>shift;
if (4 * rangeBand <rangeTH * (maxCTU --minCTU))
{
break;
}
}
addShift = shift
bandShift = BASE_SHIFT + addShift
bandStartPos = minCTU
As described above, the adaptive offset filter provided with the characteristic value calculation unit 621 (band classification derivation means) that sets the band setting information according to at least one of the maximum value, the minimum value, and the average value of the pixel values in the unit area. If this is the case, the offset band width and band offset position can be appropriately set by using different distribution information (range or distribution) for each unit area. That is, it is possible to perform suitable filtering for differences in images that differ by sequence, by picture, and by region.

(Configuration without band offset position sao_band_position)
As described above, in the configuration in which the characteristic value calculation unit 621 derives the band setting information (here, in particular, the band start position bandStartPos) based on the distribution information of the unit region, the characteristic value calculation unit 621 is the encoded data. It is not necessary to use the band offset position sao_band_position which is a parameter of. In this case, the SAO parameter decoding unit 61 does not decode the band offset position sao_band_position.

At this time, the characteristic value calculation unit 621 may derive bandIdx by the following formula.

bandIdx = ((recPicture [x] [y] --bandStartPos) >> bandShift
bandIdx = Clip3 (0, NUM_SAO_BO_CLASSES --1, bandIdx) + 1
(The number of bands NUM_SAO_BO_CLASSES is different)
A configuration that changes the maximum number of bands NUM_SAO_BO_CLASSES is preferable. FIG. 17 shows a configuration in which the luminance (cIdx = 0) has a NUM_SAO_BO_CLASSES of 32 and the color difference (cIdx! = 0) has an offset number of 16. In particular, as described above, in the configuration in which the characteristic value calculation unit 621 derives the band setting information (particularly the band start position bandStartPos) based on the distribution information of the unit region, the number of offsets to be included in the coded data sao_offset_num. It is preferable to change the configuration. For example, the brightness can be set to 32 and the color difference can be set to 16, but the present invention is not limited to this.

NUM_SAO_BO_CLASSES = 32 (cIdx == 0)
NUM_SAO_BO_CLASSES = 16 (cIdx! = 0, etc. 1, 2)
FIG. 18 shows the syntax slice_header () including num_sao_bo_classes in the embodiment. Num_sao_bo_classes [cIdx] for each decrypted cIdx indicates NUM_SAO_BO_CLASSES. The NUM_SAO_BO_CLASSES of the two color difference components cIdx = 1 and 2 may be the same. That is, num_sao_bo_classes [1] may be decrypted and set to num_sao_bo_classes [2].

By changing NUM_SAO_BO_CLASSES, the binary length of band_posision can be shortened, which reduces the offset overhead and improves the coding efficiency.

(Configuration with different offset number sao_offset_num)
It is preferable to change the number of offsets sao_offset_num included in the coded data. FIG. 19 shows a configuration in which the number of offsets of the luminance (cIdx = 0) is 4, and the number of offsets of the color difference (cIdx! = 0) is 2. In particular, as described above, in the configuration in which the characteristic value calculation unit 621 derives the band setting information (particularly the band start position bandStartPos) based on the distribution information of the unit region, the number of offsets to be included in the coded data sao_offset_num. It is preferable to change the configuration.

For example, the luminance can be set to 4 and the color difference can be set to 2, but the present invention is not limited to this.

sao_offset_num = 4 (cIdx == 0)
sao_offset_num = 2 (cIdx! = 0, etc. 1, 2)
FIG. 20 is a diagram showing the syntax slice_header () including sao_offset_num in the embodiment. The two components cIdx = 1 and 2 of the color difference may have the same sao_offset_num.
That is, sao_offset_num [1] may be decoded and set to sao_offset_num [2].

By changing sao_offset_num, the offset overhead is reduced and the coding efficiency is improved.

(Multiple band configuration)
In one embodiment, the characteristic value calculation unit 621 (band classification derivation means) derives bandIdx based on the head position of the first band and the head position of the second band.

FIG. 21 is a diagram illustrating a band head position and a band using the head position. As shown in (a) of the figure, the first start position bandStartPos0 derives the first band index bandIdx0 (0 to 15 in (a)), and the second start position bandStartPos1 derives the second start position band index. bandIdx1 (16 to 31 in (a)) is derived.

bandIdx0 = ((recPicture [x] [y] --bandStartPos0) >> bandShift)
bandIdx1 = ((recPicture [x] [y] --bandStartPos1) >> bandShift)) + bandIdxOffset1
Further, a clip may be used.

bandIdx0 = Clip3 (0, NUM_SAO_BO_CLASSES --1, (recPicture [x] [y] --bandStartPos0) >> bandShift)
bandIdx1 = Clip3 (0, NUM_SAO_BO_CLASSES --1, ((recPicture [x] [y] --bandStartPos1) >> bandShift) + bandIdxOffset1)
As shown in FIG. 21, by using band classifications that are displaced from each other, it is possible to perform band classifications that match the coding noise in a more coded state even for pixel values in the same range.

Here, it is preferable that the band start positions bandStartPos0 and bandStartPos1 are offset by half the bandwidth bandW ((1 << bitDepth >> bandShift)).

That is, bandStartPos1 = bandStartPos0 + (bandW >> 1)
As shown in FIG. 21 (c), the classification based on the head positions of a plurality of bands is not limited to two, but 4
It may be one. In the four cases, it is preferable to derive from the following equation.

bandIdx0 = (recPicture [x] [y]-bandStartPos0) >> bandShift
bandIdx1 = (((recPicture [x] [y] -bandStartPos1) >> bandShift) + bandIdxOffset1
bandIdx2 = (((recPicture [x] [y] -bandStartPos2) >> bandShift) + bandIdxOffset2
bandIdx3 = (((recPicture [x] [y] -bandStartPos3) >> bandShift) + bandIdxOffset3
In FIG. 21 (c), bandIdxOffset1 = 4, bandIdxOffset2 = 8, bandIdxOffset3 = 12 is an example. BandStartPos0, bandStartPos1, bandStartPos2, and bandStartPos3 are preferably offset by 1/4 of the bandwidth bandW ((1 << bitDepth >> bandShift)).

bandStartPos1 = bandStartPos0 + (bandW >> 2)
bandStartPos2 = bandStartPos1 + (bandW >> 2)
bandStartPos3 = bandStartPos2 + (bandW >> 2)
The offset allocation unit 624 (offset selection means) may select an offset based on both or one of the first band and the second band. That is, the offset allocation unit 624 derives Offset0 from SaoBandOffsetVal [cIdx] [rx] [ry] [bandIdx0] using the first band bandIdx0, and uses the second band bandIdx1 to derive Offset0, and uses the second band bandIdx1 to derive SaoBandOffsetVal [cIdx] [rx]. Offset1 may be derived from [ry] [bandIdx1]. The offset application unit 625 may select and add one of a plurality of offsets. For example, the processing when Offset0 and Offset1 are selected is as follows.

saoPicture [x] [y] = Clip3 (0, (1 << bitDepth) -1, recPicture [x] [y] + Offset0)
saoPicture [x] [y] = Clip3 (0, (1 << bitDepth) -1, recPicture [x] [y] + Offset1)
Further, the offset application unit 625 may add two offsets.

For example, the addition of two offsets is expressed by the following equation.

saoPicture [x] [y] = Clip3 (0, (1 << bitDepth) -1, recPicture [x] [y] + Offset0 + Offset1)
When there are multiple bands, SaoBandOffsetVal [cIdx] [rx] [ry] [bandIdx2] Offset2 is derived from the third band bandIdx2, and the fourth band bandIdx3 is used to derive SaoBandOffsetVal [cIdx] [rx. ] [ry] Offset3 may be derived from [bandIdx3].
The offset application unit 625 may select and add one of a plurality of offsets. For example, the processing when selecting Offset0, Offset1, Offset2, and Offset3 is as follows.

saoPicture [x] [y] = Clip3 (0, (1 << bitDepth) -1, recPicture [x] [y] + Offset0)
saoPicture [x] [y] = Clip3 (0, (1 << bitDepth) -1, recPicture [x] [y] + Offset1)
saoPicture [x] [y] = Clip3 (0, (1 << bitDepth) -1, recPicture [x] [y] + Offset2)
saoPicture [x] [y] = Clip3 (0, (1 << bitDepth) -1, recPicture [x] [y] + Offset3)
Further, the offset application unit 625 may add a plurality of offsets in the following calculation.

saoPicture [x] [y] = Clip3 (0, (1 << bitDepth) -1, recPicture [x] [y] + Offset0 + Offset1 + Offset2 + Offset3)
The characteristic value calculation unit 621 may set whether to use one band or two or more bands according to the range of the unit area. For example, bandIdxTmp when the maximum value maxCTU of the unit area is classified at a certain start point bandStartPos0 as shown in the following pseudo code.
If is less than 1/2 of the maximum number of bands NUM_SAO_BO_CLASSES, the flag dblBandFlag indicating that two or more bands are used may be set to 1.

bandIdxTmp = (maxCTU-bandStartPos0) >> (bitDepth-BASE_SHIFT-addShift)
if (bandIdxTmp <NUM_SAO_BO_CLASSES / 2)
{
dblBandFlag = 1
}
Further, when the bandIdxTmp is smaller than 1/4 of the maximum number of bands NUM_SAO_BO_CLASSES, the characteristic value calculation unit 621 may set dblBandFlag to 2 and use 4 bands.

bandIdxTmp = (maxCTU-bandStartPos0) >> (bitDepth-BASE_SHIFT-addShift)
if (bandIdxTmp <NUM_SAO_BO_CLASSES / 4)
{
dblBandFlag = 2
}
Further, the offset allocation unit 624 (offset selection means) derives the offset using one band when the dblBandFlag is 0 (for example, band0) and two bands when the dblBandFlag is 0, and further, The offset application unit 625 may perform addition based on the offset derived in one or two bands. When dblBandFlag is 2, four offsets may be derived and added using four bands (band0, band1, band2, band3).

Further, in addition to the offset information, the adaptive offset information copying means SAOs the band number selection information dblBandFlag of whether to use the first band or the first band and the second band.
It may be stored in the parameter storage unit 64 and copied according to the merge control information.

(Offset allocation unit 624)
The offset allocation unit 624 sets an offset to be allocated to each pixel from the actual offset SaoOffsetVal input from the offset setting unit 63 and the characteristic values edgeIdx and bandIdx input from the characteristic value calculation unit 621. Specifically, if sao_type_idx indicates an edge offset, SaoOffsetVal [cIdx] [rx] [ry] [edgeIdx] is assigned as the offset, and if sao_type_idx indicates a band offset, SaoOffsetVal [cIdx] [ Allocate rx] [ry] [bandIdx]. Here, cIdx means the color component index, cIdx = 0 means the luminance component (Y), cIdx = 1 means the Cb component (U) of the color difference, and cIdx = 2 means the Cr component (V) of the color difference. rx and ry indicate the coordinates of the target CTU, respectively.

(Offset application unit 625)
The offset application unit 625 adds the offset set by the offset allocation unit 624 to each pixel to the pixel value of the input image (deblocked decoded image P_DB or locally decoded image P) of the offset application unit 625. It is set as the pixel value of the filtered decoded image P_FL.

saoPicture [x] [y] = Clip3 (0, (1 << bitDepth) -1, recPicture [x] [y] + SaoOffsetVal [cIdx] [rx] [ry] [edgeIdx or bandIdx])
Here, saoPicture is the filtered decoded image P_FL, and recPicture is the input image to the offset application unit 625.

(Method of deriving the offset in the offset setting unit 63)
Next, the details of the method of deriving the offset in the offset setting unit 63 will be described with reference to FIG. As shown in FIG. 10, the corresponding real offset SaoOffsetV is derived from the four offset sao_offset_abs of i = 0.3 by the following equation.
SaoOffsetVal [cIdx] [rx] [ry] [i + 1] =
offsetSign [i] * sao_offset_abs [Idx] [rx] [ry] [i] << log2OffsetScale
Here, the actual offset SaoOffsetVal [cIdx] [rx] [ry] [0] means the filter is off and is always 0. The offset sao_offset_abs [cIdx] [rx] [ry] [i] of i = 0.3 corresponds to the real offset SaoOffsetVal [cIdx] [rx] [ry] [i] of i = 1.0.4. However, when the filter off is processed by another method, the offset of i = 0.3 may correspond to the actual offset of i = 0.3 as it is. Hereinafter, the same applies when the number of offsets is different.

In FIG. 10, << represents a left shift (the same applies to other figures). bitDepth means the bit depth of the pixel value, and log2OffsetScale means the bit precision of sao_offset_abs. OffsetSign indicates the sign of the actual offset. Here, it means that the offset of i = 0,1 is positive and the offset of i = 2,3 is negative as shown in the following equation. In the case of edge offset, the code may be explicitly encoded as in the case of band offset.

offsetSign [i] = {1, 1, -1, -1}
In the case of band offset, the sign offsetSign [i] uses sao_offset_sign [i] in the SAO parameter SAOP. That is, offsetSign [i] = sao_offset_sign [i] (moving image coding device 2).
Next, the configuration of the moving image coding device 11 will be described. As an example, the configuration of the moving image coding device 11 will be described below with reference to FIGS. 22 and 24. In the moving image coding device 2, 11 is a prediction image generation unit 101, a subtraction unit 102, a conversion / quantization unit 103, an entropy coding unit 104, an inverse quantization / inverse conversion unit 105, an addition unit 106, and a loop filter 107. , Prediction parameter memory (prediction parameter storage unit, frame memory) 108, reference picture memory (reference image storage unit, frame memory) 109, coding parameter determination unit 110, and prediction parameter coding unit 111. The prediction parameter coding unit 111 includes an inter prediction parameter coding unit 112 and an intra prediction parameter coding unit 113.

For each picture of the image T, the prediction image generation unit 101 generates a prediction image P of the prediction unit block for each coding unit CU which is a region in which the pictures are divided. The prediction image generation unit 101 has the same operation as the prediction image generation unit 308 described above.

The subtraction unit 102 subtracts the signal value of the predicted image P of the block input from the predicted image generation unit 101 from the pixel value of the corresponding block position of the image T to generate a residual signal. The subtraction unit 102 outputs the generated residual signal to the conversion / quantization unit 103.

The conversion / quantization unit 103 performs frequency conversion on the predicted residual signal input from the subtraction unit 102, and calculates the conversion coefficient. The conversion / quantization unit 103 quantizes the calculated conversion coefficient to obtain the quantization conversion coefficient. The conversion / quantization unit 103 outputs the obtained quantization conversion coefficient to the entropy coding unit 104 and the inverse quantization / inverse conversion unit 105.

A quantization conversion coefficient is input to the entropy coding unit 104 from the conversion / quantization unit 103, and a prediction parameter is input from the prediction parameter coding unit 111. The entropy coding unit 104 entropy-codes the input division information, prediction parameters, quantization conversion coefficient, etc. to generate a coded stream Te, and outputs the generated coded stream Te to the outside. Further, the entropy coding unit 104 variable-length encodes the SAO parameter SAOP supplied from the adaptive offset filter 1072 and includes it in the coded stream. The SAO parameter SAOP contains at least offset information.

The inverse quantization / inverse conversion unit 105 is the inverse quantization / inverse conversion unit 311 (FIG. 4) in the moving image decoding device 31.
), And the quantization conversion coefficient input from the conversion / quantization unit 103 is inversely quantized to obtain the conversion coefficient. The inverse quantization / inverse conversion unit 105 performs inverse conversion on the obtained conversion coefficient and calculates a residual signal. The inverse quantization / inverse conversion unit 105 outputs the calculated residual signal to the addition unit 106.

The addition unit 106 adds the signal value of the prediction image P of the block input from the prediction image generation unit 101 and the signal value of the residual signal input from the inverse quantization / inverse conversion unit 105 for each pixel, and decodes the signal. Generate an image. The addition unit 106 stores the generated decoded image in the reference picture memory 109.

The loop filter 107 applies a deblocking filter 1071, an adaptive offset (SAO) 1072, and an adaptive filter (ALF) 1073 to the decoded image generated by the addition unit 106.

The prediction parameter memory 108 stores the prediction parameters generated by the coding parameter determination unit 110 at a predetermined position for each of the picture to be coded and the CU.

The reference picture memory 109 stores the decoded image generated by the loop filter 107 at a predetermined position for each of the picture to be encoded and the CU.

The coding parameter determination unit 110 selects one set from the plurality of sets of coding parameters. The coding parameter is the above-mentioned QT or BT division parameter, prediction parameter, or a parameter to be coded generated in relation to these. The prediction image generation unit 101 generates a prediction image P of the block using each of these sets of coding parameters.

The coding parameter determination unit 110 calculates an RD cost value indicating the magnitude of the amount of information and the coding error for each of the plurality of sets. The RD cost value is, for example, the sum of the code amount and the squared error multiplied by the coefficient λ. The code amount is the amount of information of the coded stream Te obtained by entropy-coding the quantization residuals and the coding parameters. The squared error is the sum of the squared values of the residual values of the residual signal calculated by the subtracting unit 102 between the pixels. The coefficient λ is a real number greater than the preset zero. The coding parameter determination unit 110 selects the set of coding parameters that minimizes the calculated RD cost value. As a result, the entropy coding unit 104 outputs the selected set of coding parameters as a coding stream Te to the outside.
Do not output a set of unselected coding parameters. Coding parameter determination unit 110
Stores the determined coding parameter in the prediction parameter memory 108.

The prediction parameter coding unit 111 derives a format for coding from the parameters input from the coding parameter determination unit 110, and outputs the encoding format to the entropy coding unit 104. Derivation of the form for coding is, for example, deriving a difference vector from a motion vector and a prediction vector. Further, the prediction parameter coding unit 111 derives the parameters necessary for generating the prediction image from the parameters input from the coding parameter determination unit 110, and outputs the parameters to the prediction image generation unit 101.

The inter-prediction parameter coding unit 112 derives an inter-prediction parameter such as a difference vector based on the prediction parameter input from the coding parameter determination unit 110.

Further, the intra prediction parameter coding unit 113 has a configuration in which the intra prediction parameter decoding unit 304 derives the intra prediction parameter as a configuration for deriving the prediction parameter necessary for generating the prediction image to be output to the prediction image generation unit 101. Includes some identical configurations.

The deblocking filter 1071 determines the block boundary in the locally decoded image P, or the block boundary in the locally decoded image P, when the difference between the pixel values of the pixels adjacent to each other via the CU boundary is smaller than a predetermined threshold value. Alternatively, deblocking processing is performed on the CU boundary. Then, the image in the vicinity of the block boundary or the CU boundary is smoothed. The image deblocked by the deblocking filter 1071 is output to the adaptive offset filter 1072 as a deblocked decoded image P_DB.

The adaptive offset filter 1072 generates an offset filtered decoded image P_OF by performing an adaptive offset filter process on the input image. The generated offset filtered decoded image P_OF is supplied to the adaptive filter 1073. The specific configuration of the adaptive offset filter 80 will be described later.

The adaptive filter 1073 generates a filtered decoded image P_FL by subjecting the offset filtered decoded image P_OF supplied from the adaptive offset filter 1072 to an adaptive filtering process. The filtered decoded image P_FL filtered by the adaptive filter 1073 is stored in the reference picture memory 109. The filter coefficient used by the adaptive filter 1073 is defined so that the error between the filtered decoded image P_FL and the encoded image T becomes smaller, and the filter coefficient determined in this way is the filter parameter FP. Is encoded and transmitted to the moving image decoding device 11.

(Adaptive offset filter 1072)
Next, the adaptive offset filter 1072 will be described with reference to FIG. FIG. 23 is a block diagram showing the configuration of the adaptive offset filter 1072. As shown in FIG. 23, the adaptive offset filter 1072 includes a SAO parameter setting unit 81 and a SAO parameter numbering unit 83.
, SAO parameter storage unit 84, and adaptive filter unit 62.

(SAO parameter setting unit 81)
The SAO parameter setting unit 81 determines the target processing unit (for example, CTU) based on the input moving image T and the input image (deblocked decoded image P_DB or locally decoded image P) of the adaptive filter unit 62. The SAO offset information to be applied to each processing unit is set and output. SAO offset information includes offset type sao_type_idx and offset SaoOffsetVal. If the offset type sao_type_idx is a band offset, the band offset position sao_band_position is also set and output. The value of each parameter of the SAO offset information may be set so as to further improve the coding efficiency.

The SAO parameter setting unit 81 outputs the SAO parameter SAOP including the SAO offset information to the SAO parameter numbering unit 83.

(SAO Parameter No. 83)
The SAO parameter numbering unit 83 encodes the SAO parameter SAOP set by the SAO parameter setting unit 81. More specifically, the coded data of the SAO offset information is generated for each processing unit (for example, CTU) by variable length coding using an arithmetic code, and is stored for each processing unit in the slice data SDATA. When the processing unit is CTU unit, if the set parameter is equal to the already encoded CTU parameter, the merge control information (sao_merge_left_flag, sao_merge_up_flag) with a value indicating that merge is used is encoded. , Offset type sao_type_idx and offset sao_offset are not encoded. On the contrary, when the parameter of the target CTU is not equal to the parameter of the adjacent CTU, the merge control information in which the value indicating that the merge is not used is set is encoded, and the offset type sao_type_idx and the offset sao_offset are encoded. More specifically, sao_merge_left_flag = 1 when the parameter is equal to the CTU adjacent to the left of the target CTU, sao_merge_up_flag = 1 when the parameter is equal to the CTU adjacent above the target CTU, and other cases. In this case, the merge control information is set to 0. The encoded parameters are stored in the SAO parameter storage unit 84.

(SAO parameter storage unit 84)
The SAO parameter storage unit 84 stores the SAO parameter SAOP encoded by the SAO parameter coding unit 83. Further, the SAO parameter storage unit 84 may store offset setting information (for example, offset start position, bandwidth information) and unit area distribution information (for example, minimum value minCTU, maximum value maxCTU) in addition to offset information. Good. Further, the band number selection information dblBandFlag indicating whether to use a plurality of bands may be stored.

(Example of realization by software)
A part of the moving image coding device 11 and the moving image decoding device 31 in the above-described embodiment, for example, the entropy decoding unit 301, the prediction parameter decoding unit 302, the loop filter 305, the prediction image generation unit 308, and the inverse quantization. Inverse conversion unit 311, addition unit 312, prediction image generation unit 101, subtraction unit 102, conversion / quantization unit 103, entropy coding unit 104, inverse quantization / inverse conversion unit 105, loop filter 107, coding parameter determination unit 110, the prediction parameter coding unit 111 may be realized by a computer. In that case, the program for realizing this control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed. The "computer system" referred to here is a computer system built into either the moving image coding device 11 or the moving image decoding device 31, and includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built in a computer system. Furthermore, a "computer-readable recording medium" is a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In that case, a program may be held for a certain period of time, such as a volatile memory inside a computer system serving as a server or a client. Further, the above-mentioned program may be a program for realizing a part of the above-mentioned functions, and may be a program for realizing the above-mentioned functions in combination with a program already recorded in the computer system.

Further, a part or all of the moving image coding device 11 and the moving image decoding device 31 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the moving image coding device 11 and the moving image decoding device 31 may be individually converted into a processor, or a part or all of them may be integrated into a processor. Further, the method of making an integrated circuit is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, when an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology, an integrated circuit based on this technology may be used.

Although one embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to the above, and various design changes and the like are made without departing from the gist of the present invention. It is possible to do.

[Application example]
The moving image coding device 11 and the moving image decoding device 31 described above can be mounted on and used in various devices that transmit, receive, record, and reproduce moving images. The moving image may be a natural moving image captured by a camera or the like, or an artificial moving image (including CG and GUI) generated by a computer or the like.

First, it will be described with reference to FIG. 24 that the moving image coding device 11 and the moving image decoding device 31 described above can be used for transmitting and receiving moving images.

FIG. 24A is a block diagram showing a configuration of a transmission device PROD_A equipped with a moving image coding device 11. As shown in FIG. 24A, the transmitter PROD_A modulates the carrier wave with the coding unit PROD_A1 that obtains the coded data by encoding the moving image and the coded data obtained by the coding unit PROD_A1. It includes a modulation unit PROD_A2 that obtains a modulation signal, and a transmission unit PROD_A3 that transmits the modulation signal obtained by the modulation unit PROD_A2. The moving image coding device 11 described above is used as the coding unit PROD_A1.

The transmitter PROD_A has a camera PROD_A4 for capturing a moving image, a recording medium PROD_A5 for recording a moving image, an input terminal PROD_A6 for inputting a moving image from the outside, and a moving image as a source of the moving image to be input to the coding unit PROD_A1. , An image processing unit PRED_A7 for generating or processing an image may be further provided. In FIG. 24A, a configuration in which the transmitting device PROD_A includes all of them is illustrated, but some of them may be omitted.

The recording medium PROD_A5 may be a recording of an unencoded moving image, or a moving image encoded by a recording coding method different from the transmission coding method. It may be a thing. In the latter case, a decoding unit (decoding unit) that decodes the coded data read from the recording medium PROD_A5 between the recording medium PROD_A5 and the coding unit PROD_A1 according to the coding method for recording.
(Not shown) may be used.

FIG. 25B is a block diagram showing the configuration of the receiving device PROD_B equipped with the moving image decoding device 31. As shown in FIG. 24 (b), the receiving device PROD_B is a receiving unit PROD_ that receives the modulated signal.
B1, the demodulation unit PROD_B2 that obtains the coded data by demodulating the modulated signal received by the reception unit PROD_B1, and the decoding unit PROD_B3 that obtains the moving image by decoding the coded data obtained by the demodulation unit PROD_B2. I have. The moving image decoding device 31 described above is used as the decoding unit PROD_B3.

The receiving device PROD_B serves as a supply destination of the moving image output by the decoding unit PROD_B3, a display PROD_B4 for displaying the moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal for outputting the moving image to the outside. It may also have PROD_B6. In FIG. 25 (b), a configuration in which the receiving device PROD_B includes all of these is illustrated, but some of them may be omitted.

The recording medium PROD_B5 may be used for recording an unencoded moving image, or may be encoded by a recording coding method different from the transmission coding method. You may. In the latter case, a coding unit (not shown) that encodes the moving image acquired from the decoding unit PROD_B3 according to the recording coding method may be interposed between the decoding unit PROD_B3 and the recording medium PROD_B5.

The transmission medium for transmitting the modulated signal may be wireless or wired. Further, the transmission mode for transmitting the modulated signal may be broadcasting (here, a transmission mode in which the destination is not specified in advance) or communication (here, transmission in which the destination is specified in advance). Refers to an aspect). That is, the transmission of the modulated signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

For example, a broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) of terrestrial digital broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives modulated signals by radio broadcasting. Further, a broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives modulated signals by wired broadcasting.

In addition, servers (workstations, etc.) / clients (television receivers, personal computers, smartphones, etc.) for VOD (Video On Demand) services and video sharing services using the Internet are transmitters that send and receive modulated signals via communication. This is an example of PROD_A / receiver PROD_B (usually, in LAN, either wireless or wired is used as a transmission medium, and in WAN, wired is used as a transmission medium). Here, personal computers include desktop PCs, laptop PCs, and tablet PCs. Smartphones also include multifunctional mobile phone terminals.

The client of the video sharing service has a function of decoding the encoded data downloaded from the server and displaying it on the display, as well as a function of encoding the moving image captured by the camera and uploading it to the server. That is, the client of the video sharing service functions as both the transmitting device PROD_A and the receiving device PROD_B.

Next, it will be described with reference to FIG. 25 that the moving image coding device 11 and the moving image decoding device 31 described above can be used for recording and reproducing a moving image.

FIG. 25A is a block diagram showing a configuration of a recording device PROD_C equipped with the above-mentioned moving image coding device 11. As shown in FIG. 25 (a), the recording device PROD_C writes the coding unit PROD_C1 that obtains the coded data by encoding the moving image and the coded data obtained by the coding unit PROD_C1 to the recording medium PROD_M. It has a writing unit PROD_C2. The moving image coding device 11 described above is used as the coding unit PROD_C1.

The recording medium PROD_M may be of a type built in the recording device PROD_C, such as (1) HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) SD memory. It may be of a type that is connected to the recording device PROD_C, such as a card or USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray (registered trademark)). ) Disc: A registered trademark) may be loaded into a drive device (not shown) built in the recording device PROD_C.

Further, the recording device PROD_C has a camera PROD_C3 that captures a moving image, an input terminal PROD_C4 for inputting a moving image from the outside, and a reception for receiving the moving image as a source of the moving image to be input to the coding unit PROD_C1. The unit PROD_C5 and the image processing unit PROD_C6 for generating or processing an image may be further provided. In FIG. 25 (a), a configuration in which the recording device PROD_C is provided with all of these is illustrated, but some of them may be omitted.

The receiving unit PROD_C5 may receive an unencoded moving image, or receives coded data encoded by a transmission coding method different from the recording coding method. It may be something to do. In the latter case, it is preferable to interpose a transmission decoding unit (not shown) for decoding the coded data encoded by the transmission coding method between the receiving unit PROD_C5 and the coding unit PROD_C1.

Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, an HDD (Hard Disk Drive) recorder, and the like (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is the main source of moving images). .. In addition, a camcorder (in this case, the camera PROD_C3 is the main source of moving images), a personal computer (in this case, the receiving unit PROD_C5 or the image processing unit C6 is the main source of moving images), and a smartphone (this If the camera PROD_C3
Or the receiver PROD_C5 is the main source of moving images), etc., such as the recording device PROD_C.
This is an example.

FIG. 25B is a block showing the configuration of the reproduction device PROD_D equipped with the above-mentioned moving image decoding device 31. As shown in FIG. 25 (b), the reproduction device PROD_D obtains a moving image by decoding the reading unit PROD_D1 that reads the coded data written in the recording medium PROD_M and the coded data read by the reading unit PROD_D1. It has a decoding unit PROD_D2. The moving image decoding device 31 described above is used as the decoding unit PROD_D2.

The recording medium PROD_M may be of a type built into the playback device PROD_D, such as (1) HDD or SSD, or (2) such as an SD memory card or USB flash memory. It may be of a type connected to the playback device PROD_D, or may be loaded into a drive device (not shown) built in the playback device PROD_D, such as (3) DVD or BD. Good.

Further, the playback device PROD_D has a display PROD_D3 for displaying the moving image, an output terminal PROD_D4 for outputting the moving image to the outside, and a transmitting unit for transmitting the moving image as a supply destination of the moving image output by the decoding unit PROD_D2. It may also have PROD_D5. In FIG. 25 (b), the configuration in which the reproduction device PROD_D is provided with all of these is illustrated, but some of them may be omitted.

The transmission unit PROD_D5 may transmit an unencoded moving image, or transmits coded data encoded by a transmission coding method different from the recording coding method. It may be something to do. In the latter case, it is preferable to interpose a coding unit (not shown) that encodes the moving image by a coding method for transmission between the decoding unit PROD_D2 and the transmitting unit PROD_D5.

Examples of such a playback device PROD_D include a DVD player, a BD player, an HDD player, and the like (in this case, the output terminal PROD_D4 to which a television receiver or the like is connected is the main supply destination of moving images). .. In addition, a television receiver (in this case, display PROD_D3 is the main supply destination of moving images) and digital signage (also called electronic signage or electronic bulletin board, etc., and display PROD_D3 or transmitter PROD_D5 is the main supply destination of moving images. (First), desktop PC (in this case, output terminal PROD_D4 or transmitter PROD_D5 is the main supply destination of moving images), laptop or tablet PC (in this case, display PROD_D3 or transmitter PROD_D5 is video) An example of such a playback device PROD_D is a smartphone (in this case, the display PROD_D3 or the transmitter PROD_D5 is the main supply destination of the moving image), which is the main supply destination of the image.

(Hardware realization and software realization)
Further, each block of the moving image decoding device 31 and the moving image coding device 11 described above may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or may be realized by a CPU (Central Processing). It may be realized by software using Unit).

In the latter case, each of the above devices is a CPU that executes instructions of a program that realizes each function, a ROM (Read Only Memory) that stores the above program, and a RAM (Random) that expands the above program.
Access Memory), a storage device (recording medium) such as a memory for storing the above programs and various data. Then, an object of the embodiment of the present invention is a record in which the program code (execution format program, intermediate code program, source program) of the control program of each of the above devices, which is software for realizing the above-mentioned functions, is recorded readable by a computer. It can also be achieved by supplying the medium to each of the above devices and having the computer (or CPU or MPU) read and execute the program code recorded on the recording medium.

Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic discs such as floppy (registered trademark) discs / hard disks, and CD-ROMs (Compact Disc Read-Only Memory) / MO discs (Magneto-Optical discs). ) / MD (Mini Disc) / DVD (Digital Versatile Disc) / CD-R (CD Recordable) / Blu-ray Disc (registered trademark) Discs including optical discs, IC cards (memory) Cards such as (including cards) / optical cards, mask ROM / EPROM (Erasable Programmable Read-Only Memory) / EEPROM (Electrically Erasable and Programmable Read-Only Memory: registered trademark) / semiconductor memory such as flash ROM, or PLD Logic circuits such as (Programmable logic device) and FPGA (Field Programmable Gate Array) can be used.

Further, each of the above devices may be configured to be connectable to a communication network, and the above program code may be supplied via the communication network. This communication network is not particularly limited as long as it can transmit the program code. For example, Internet, Intranet, Extranet, LAN (Local Area Network), ISDN (Integrated Services Digital)
Network), VAN (Value-Added Network), CATV (Community Antenna television / Cable)
Television) Communication network, Virtual Private Network, telephone line network, mobile communication network, satellite communication network, etc. can be used. Further, the transmission medium constituting this communication network may be any medium as long as it can transmit the program code, and is not limited to a specific configuration or type. For example, IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber)
Even for wired lines such as Line), infrared rays such as IrDA (Infrared Data Association) and remote controls, BlueTooth (registered trademark), IEEE802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (registered trademark) ) (Digital Living Network Alliance: registered trademark)
, Mobile phone network, satellite line, terrestrial digital broadcasting network, etc. can also be used wirelessly. The embodiment of the present invention can also be realized in the form of a computer data signal embedded in a carrier wave, in which the program code is embodied by electronic transmission.

The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the claims. That is, an embodiment obtained by combining technical means appropriately modified within the scope of the claims is also included in the technical scope of the present invention.

The embodiment of the present invention is suitably applied to a moving image decoding device that decodes coded data in which image data is encoded, and a moving image coding device that generates coded data in which image data is encoded. be able to. Further, it can be suitably applied to the data structure of the coded data generated by the moving image coding device and referenced by the moving image decoding device.

31 Moving image decoding device (image decoding device)
11 Video coding device (image coding device)
301 Entropy decoding unit (decoding means)
308 Prediction image generation unit 309 Inter prediction image generation unit 310 Intra prediction image generation unit 104 Entropy coding unit (encoding means)
108 Prediction image generation unit 109 Inter-prediction image generation unit 110 Intra-prediction image generation unit 3052 Adaptive offset filter (image filter device)
61 SAO parameter decoding unit (decoding means, offset decoding means, adaptive offset information copying means)
63 Offset setting unit (offset selection means)
1072 Adaptive offset filter (image filter device)
621 Characteristic value calculation unit (band classification means)
624 Offset allocation unit 625 Offset application unit (filter means)

Claims (16)

An image filter device that adds an offset to each pixel value of an input image composed of a plurality of unit areas.
The pixel value is classified into one or more bands based on the band setting information according to the pixel value in the unit region, and the band classification derivation means for deriving the band index indicating the band and the band index are described. Offset selection means to select one of the offsets,
A filter means for adding the selected offset to the pixel value is provided.
The band classification derivation means is an image filter device characterized in that band setting information is set according to information in a unit area. The image filter device according to claim 1, wherein the band classification derivation means sets band setting information according to a color component. The image filter device according to claim 1, wherein the image filter device means sets band setting information for each color component by the band classification derivation means. The image filter device according to claim 1, wherein the band classification derivation means sets band setting information according to at least one of a maximum value, a minimum value, and an average value of pixel values in a unit region. .. The band classification derivation means derives the band setting information from the maximum value, the minimum value, and the average value of the pixel values in the unit area by using the pixels having a predetermined width on the lower side and the pixels having a predetermined width on the right side of the unit area. The image filter device according to claim 4. The image filter device includes an adaptive offset information copying means for copying the already decoded offset information.
The image filter device according to claim 1, wherein the adaptive offset information copying means includes band setting information in addition to the offset information. The image filter device according to claim 1, wherein the band setting information is a band width and a band head position. The band classification derivation means derives based on the head position of the first band and the head position of the second band, and the offset selection means is either or both of the first band and the second band. The image filter device according to claim 1, wherein the offset is selected based on one of the two. The image filter device according to claim 8, wherein the head position of the first band and the head position of the band are shifted by half the bandwidth. The image according to claim 9, wherein the band classification derivation means selects whether to use the first band or the first band and the second band according to the range of the unit region. Filter device. 10. The adaptive offset information copying means according to claim 10, wherein, in addition to the offset information, the band number selection information of whether to use the first band or to use the first band and the second band is copied. The image filter device described. The image filter device according to claim 1, wherein the image filter device further decodes the number of bands. The image filter device according to claim 1, wherein the image filter device further decodes the number of offsets. An image decoding device including the image filter device according to any one of claims 1 to 13. An image coding device including the image filter device according to any one of claims 1 to 13. It is a data structure of coded data in which the predicted residual, which is the difference between the predicted image and the original image, is encoded by the image decoding device.
A data structure of coded data including band setting information which is a parameter used for classifying pixel values obtained by decoding the predicted residuals into bands.

Images