WO2013046990A1 - Offset decoding apparatus, offset encoding apparatus, image filter apparatus, and data structure - Google Patents

Abstract

An adaptive offset filter (60) adds an offset to each of the pixel values of an input image constituted by a plurality of unit areas. If the size of a unit area for which an offset is determined is smaller than a predetermined size, the adaptive offset filter (60) determines, from among selectable offsets the number of which is limited, an offset to be added to the pixel values of the pixels included in the unit area, and adds the determined offset thereto.

Description

Offset decoding apparatus, offset encoding apparatus, image filter apparatus, and data structure

The present invention relates to an image filter device that performs image filtering. The present invention also relates to an offset decoding apparatus that decodes an offset referred to by an image filter, and an offset encoding apparatus that encodes an offset referred to by an image filter. The present invention also relates to the data structure of encoded data.

In order to efficiently transmit or record a moving image, a moving image encoding device (encoding device) that generates encoded data by encoding the moving image, and decoding by decoding the encoded data A video decoding device (decoding device) that generates an image is used. As a specific moving picture encoding method, for example, H.264 is used. H.264 / MPEG-4. A method used in AVC, a method used in KTA software, which is a joint development codec in VCEG (Video Coding Expert Group), and a method used in TMuC (Test Model Under Consulation) software, which is the successor codec And a method employed in HM (HEVC TestModel) software.

In such an encoding method, an image (picture) constituting a moving image is a slice obtained by dividing the image, a maximum coding unit obtained by dividing the slice (LCU: Largegest Coding Unit, tree block) Hierarchical structure consisting of coding units (CU: Coding 単 位 Unit, also called coding node) obtained by dividing the maximum coding unit, and blocks and partitions obtained by dividing the coding unit In many cases, the block is encoded as a minimum unit.

In such an encoding method, a predicted image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and difference data between the predicted image and the input image is encoded. It becomes. As methods for generating a predicted image, methods called inter-screen prediction (inter prediction) and intra-screen prediction (intra prediction) are known.

In intra prediction, predicted images in a corresponding frame are sequentially generated based on a locally decoded image in the same frame. Specifically, in intra prediction, usually, one prediction direction is selected from prediction directions included in a predetermined prediction direction (prediction mode) group for each prediction unit (for example, block), and A prediction pixel value on the prediction target region is generated by extrapolating the pixel value of the reference pixel in the locally decoded image in the selected prediction direction. Also, in inter prediction, by applying motion compensation using a motion vector to a reference image in a reference frame (decoded image) in which the entire frame is decoded, a predicted image in a prediction target frame is converted into a prediction unit ( For example, it is generated for each block).

Non-Patent Document 1 and Non-Patent Document 2 describe an adaptive loop filter (“", which is a subsequent stage of a deblocking filter that reduces block distortion of a decoded image and performs filter processing using adaptively determined filter coefficients. An adaptive offset filter (also referred to as an “adaptive offset filter”) introduced before the “adaptive filter” is disclosed. This adaptive offset filter adds an adaptively set offset to each pixel value of an image output from the deblocking filter.

It is possible to more effectively suppress block distortion by providing such an adaptive offset filter.

However, since the adaptive offset filter has many types of offset (type, class), a large memory size is required.

The present invention has been made in view of the above problems, and is to realize an image filter device and the like that can reduce block distortion while suppressing an increase in memory size.

In order to solve the above problems, an image filter device according to the present invention is an image filter device that adds an offset selected from a plurality of offsets to each pixel value of an input image composed of a plurality of unit regions. The offset determining means for determining the offset to be added to the pixel value of the pixel included in each unit area for each unit area, and the offset determined by the offset determining means as the pixel value of the pixel included in the unit area Filter means for adding, wherein the offset determining means can select when the size of the unit area for which the offset is determined is smaller than a predetermined size than when the size of the unit area is equal to or larger than the predetermined size Determines the offset to be added to the pixel value of the pixel included in the unit area from the set of offsets with a limited number of offsets It is characterized in Rukoto.

According to the above configuration, when the size of the unit area for which the offset is determined is smaller than a predetermined size, from the set of offsets that are more limited than when the size of the unit area is equal to or larger than the predetermined size, The offset to be added is determined.

When the size of the unit area is small, the number of pixels included in the unit area is small, and the pixels are likely to have approximate values. Therefore, when the size of the unit area is small, even if the number of selectable offsets is limited, the influence on the image after the offset application is small. Further, by limiting the number of offsets that can be selected, the required memory capacity can be reduced.

Therefore, according to the above configuration, it is possible to reduce the amount of memory used while reducing the effect on the image after the offset application. Also, since the number of selectable offsets is reduced, the amount of codes can be reduced, and the coding efficiency is improved.

In order to solve the above problems, an image filter device according to the present invention is an image filter device that adds an offset to each pixel value of an input image composed of a plurality of unit regions, and includes pixels included in each unit region. Offset determining means for determining the offset to be added to the pixel value for each unit region, and filter means for adding the offset determined by the offset determining means to the pixel value of the pixel included in the unit region, The filter means is characterized in that, when the size of the unit area for which the offset is determined is smaller than a predetermined size, an offset with coarser accuracy is added than when the size of the unit area is equal to or larger than the predetermined size. Yes.

According to the above configuration, when the size of the unit area for which the offset is to be determined is smaller than a predetermined size, an offset having a coarser accuracy is added than when the size of the unit area is equal to or larger than the predetermined size.

When the size of the unit area is small, the number of pixels included in the unit area is small, and the pixels are likely to have approximate values. Therefore, when the size of the unit region is small, the influence on the quantization error is small even if the offset accuracy is rough. Therefore, the influence on the image after applying the offset is small. In addition, it is possible to reduce the required memory capacity by increasing the accuracy of the offset.

Therefore, according to the above configuration, it is possible to reduce the amount of memory used while reducing the effect on the image after the offset application. Further, since the offset accuracy becomes coarse, the amount of codes can be reduced, and the coding efficiency is improved.

In order to solve the above problems, an image filter device according to the present invention is an image filter device that applies an adaptive offset (SAO: Sample ： Adaptive Offset) to each pixel value of an input image composed of a plurality of unit regions. , An offset determining unit that determines for each unit region the type of offset to be added to the pixel value of the pixel included in each unit region, and an offset according to the type of offset determined by the offset determining unit in the unit region Filter means for adding to the pixel values of the included pixels, the offset determining means for the target pixel to which the offset is to be added, for the pixels whose pixel values are near the maximum value and the minimum value, The type is determined to be band offset (BO), and the edge offset (EO) is determined for pixels in other ranges. It is characterized.

According to the above configuration, when adaptive offset is applied, a band offset is applied to pixels in the vicinity of the maximum and minimum pixel values, and an edge offset is applied to pixels in other value ranges. Is done.

In the high pixel value region and the low pixel value region, the pixel value tends to affect the error more easily than the edge. Therefore, the above configuration increases the error correction efficiency and improves the encoding efficiency. .

Also, if band offset and edge offset are used together in one type, the number of types of adaptive offset can be reduced, and the amount of memory used and the amount of processing can be reduced.

In the above configuration, the band offset refers to an offset process in which one of a plurality of offsets is added to the pixel value of the processing target pixel according to the size of the pixel value of the processing target pixel ( The same applies below). In the above configuration, the edge offset is a plurality of offsets in the pixel value of the processing target pixel according to the difference between the pixel value of the processing target pixel and the pixel values of the pixels around the processing target pixel. This is offset processing for adding any of the above (same below).

In order to solve the above-described problems, an image filter device according to the present invention is an image filter device that applies an adaptive offset (SAO: Sample Adaptive Offset) to an input image, and a class for applying an edge offset (EO). Class classification means for performing edge determination to determine only the pixels existing in the horizontal direction of the target pixel, and when applying edge offset (EO), the class classification means classifies the class. And a filter means for adding a corresponding offset.

According to the above configuration, since only the pixels in the horizontal direction are referred to for edge determination, the required memory amount can be reduced as compared with the case of referring to the pixels in the upward direction. In addition, since the upper boundary determination is not necessary, the processing amount can also be reduced.

In order to solve the above problems, an image filter device according to the present invention is an image filter device that applies an adaptive offset (SAO: Sample ： Adaptive Offset) to an input image, and the pixel value indicating a color difference has a maximum value and a minimum value. Class classification means for determining a class when band offset (BO) is applied by subdividing the division width of the class around the median value, which is the median value, than other value ranges, and band offset (BO) Is applied, a filter means for adding an offset corresponding to the class classified by the class classification means is provided.

According to the above configuration, the pixel value indicating the color difference is subdivided from the division value of the class around the median value that is the median value of the maximum value and the minimum value, and the band offset (BO) is reduced. Determine the class to apply.

When the pixel value of the color difference is the median value, the pixel is achromatic. Achromatic errors are easily noticeable by humans, and the subjective image quality deteriorates. Therefore, if the class width near the median is subdivided as in the above configuration, the offset can be set finely for the pixels near the median. Thereby, subjective image quality can be improved.

In order to solve the above problems, an image filter device according to the present invention is an image filter device that applies an adaptive offset (SAO: Sample ： Adaptive Offset) to an input image, and the pixel value indicating a color difference has a maximum value and a minimum value. It is characterized by comprising filter means for adding an offset with higher accuracy to the pixels near the median value, which is the median value, than the pixels in other value ranges.

According to the above configuration, the pixel value indicating the color difference adds an offset with higher accuracy than the pixels in the other value ranges to the pixels in the vicinity of the median value that is the median value of the maximum value and the minimum value.

When the pixel value of the color difference is the median value, the pixel is achromatic. Achromatic errors are easily noticeable by humans, and the subjective image quality deteriorates. Therefore, as in the above configuration, if the accuracy of the offset to be added is improved in the vicinity of the median value, the offset can be finely added to the pixels in the vicinity of the median value. Thereby, subjective image quality can be improved.

In order to solve the above problem, an offset decoding apparatus according to the present invention is an offset decoding apparatus that decodes each offset referenced by an image filter that adds an offset to each pixel value of an input image, and each offset residual Offset residual decoding means for decoding the encoded data from the encoded data, prediction value deriving means for deriving the predicted value of each offset from a decoded offset or a predetermined value, and each offset for deriving the predicted value And an offset calculating means for calculating from the predicted value derived by the means and the offset residual decoded by the offset residual decoding means.

According to the above configuration, since the offset is decoded from the residual, it is possible to reduce the code amount as compared with the case where the offset is encoded as it is. In addition, since the prediction value for obtaining the residual is derived from the decoded offset or a predetermined value, by using only the decoded offset, the code amount of the difference data is made as compared with the case of encoding as it is. It can be prevented from becoming large.

For example, “0” can be given as a predetermined value.

In order to solve the above problems, an offset encoding apparatus according to the present invention is an offset encoding apparatus that encodes each offset referenced by an image filter that adds an offset to each pixel value of an input image, An offset for calculating an offset residual from each offset and the predicted value derived by the predicted value deriving means, and a predicted value deriving means for deriving the predicted value of the offset from an encoded offset or a predetermined value The apparatus is characterized by comprising: a residual calculation means; and an offset residual encoding means for encoding the offset residual calculated by the offset residual calculation means.

According to the above configuration, since the offset is decoded from the residual, it is possible to reduce the code amount as compared with the case where the offset is encoded as it is. In addition, since the prediction value for obtaining the residual is derived from the decoded offset or a predetermined value, by using only the decoded offset, the code amount of the difference data is made as compared with the case of encoding as it is. It can be prevented from becoming large.

In order to solve the above problems, the data structure of encoded data according to the present invention is encoded data data referred to by an image filter that adds an offset to each pixel value of an input image composed of a plurality of unit areas. A prediction value derivation information indicating whether the prediction value is derived from a decoded offset or a predetermined value, and the image filter includes prediction value derivation information included in the encoded data The prediction value is derived and the offset is decoded with reference to FIG.

According to the above configuration, whether the predicted value is a decoded offset or a predetermined value can be determined from the predicted value derivation information.

As described above, the image filter device according to the present invention includes an offset determination unit that determines an offset to be added to the pixel value of a pixel included in each unit region for each unit region, and an offset determined by the offset determination unit. Filter means for adding to the pixel values of the pixels included in the unit area, and the offset determining means, when the size of the unit area for which the offset is determined is smaller than a predetermined size, the size of the unit area In this configuration, the offset to be added to the pixel value of the pixel included in the unit area is determined from the set of offsets in which the number of offsets that can be selected is limited as compared with the case where is larger than a predetermined size.

This has the effect of reducing the amount of memory used while reducing the effect on the image after applying the offset. In addition, since the number of selectable offsets is reduced, the amount of codes can be reduced, and the encoding efficiency is improved.

Also, the offset decoding apparatus according to the present invention derives an offset residual decoding means for decoding each offset residual from encoded data, and a predicted value of each offset from a decoded offset or a predetermined value. A prediction value deriving unit that performs the calculation, and an offset calculation unit that calculates each offset from the prediction value derived by the prediction value deriving unit and the offset residual decoded by the offset residual decoding unit. It is.

Thereby, since the offset is decoded from the residual, the code amount can be reduced as compared with the case where the offset is encoded as it is. In addition, since the prediction value for obtaining the residual is derived from the decoded offset or a predetermined value, by using only the decoded offset, the code amount of the difference data is made as compared with the case of encoding as it is. There exists an effect that it can prevent becoming large.

In addition, the offset encoding apparatus according to the present invention derives a predicted value of each offset from a coded offset or a predetermined value, a predicted value deriving unit, and each offset and the predicted value deriving unit. An offset residual calculating unit that calculates an offset residual from the predicted value, and an offset residual encoding unit that encodes the offset residual calculated by the offset residual calculating unit. is there.

Thereby, since the offset is decoded from the residual, the code amount can be reduced as compared with the case where the offset is encoded as it is. In addition, since the prediction value for obtaining the residual is derived from the decoded offset or a predetermined value, by using only the decoded offset, the code amount of the difference data is made as compared with the case of encoding as it is. There exists an effect that it can prevent becoming large.

It is a block diagram which shows the structure of the adaptive offset filter which concerns on embodiment of this invention. FIG. 6 is a diagram illustrating a data configuration of encoded data generated by the video encoding device according to the embodiment and decoded by the video decoding device, where (a) to (d) are a picture layer and a slice layer, respectively. FIG. 4B is a diagram illustrating a configuration of QAOU information related to a QAOU of a leaf, and FIG. 5F is a diagram illustrating a configuration of QAOU information related to a QAOU of a leaf. It is a figure which shows the syntax of the offset information OI of the coding data which concerns on the said embodiment. It is a figure which shows the aspect of the division | segmentation of the offset unit which concerns on the said embodiment, (a) is a figure which shows the case where sao_curr_depth = 0, (b) is a figure which shows the case where sao_curr_depth = 1, (c) (A) is a figure which shows the case where sao_curr_depth = 2, (d) is a figure which shows the case where sao_curr_depth = 3, (e) is a figure which shows the case where sao_curr_depth = 4. It is a block diagram which shows the structure of the moving image decoding apparatus which concerns on the said embodiment. It is a figure for demonstrating the said embodiment, Comprising: (a) is a figure which shows each QAOMU of the division | segmentation depth 3 which comprises the object process unit, and the QAOU index allocated to each QAOMU, (b) FIG. 10 is a diagram showing an offset type associated with each of QAOU indexes 0 to 9 and an offset for each class selectable in each offset type. It is a flowchart which shows the flow of a process by the adaptive offset filter process part which concerns on the said embodiment. It is a figure which shows the conversion table which the offset information decoding part which concerns on the said embodiment uses. FIG. 4 is a diagram for explaining offset processing by an adaptive offset filter according to the embodiment, and FIGS. 4A to 4D are diagrams illustrating pixels referred to when sao_type_idx = 1 to 4, respectively. FIG. It is a figure for demonstrating the offset process by the adaptive offset filter which concerns on the said embodiment, Comprising: (a) is the magnitude relationship between the pixel value pix [x] of the process target pixel x, and the pixel value of the pixel a or b (B) is a graph showing the magnitude relationship between the pixel value of the pixel x to be processed and the pixel values of the pixels a and b, and a graph showing the value of the function Sign according to the magnitude relationship. And (c) is a diagram showing the correspondence between each graph shown in (b) and class_idx, and (d) to (f) are shown in FIG. It is a figure which shows the conversion table showing the conversion from EggeType to class_idx. It is a figure for demonstrating the offset process by the adaptive offset filter which concerns on the said embodiment, Comprising: (a) is a figure which shows a class classification | category when sao_type_idx = 5 schematically, (b) sao_type_idx = 6 It is a figure which shows roughly the class classification | category when it is. It is a figure for demonstrating the offset process by the adaptive offset filter which concerns on the said embodiment, Comprising: (a) is a figure which shows roughly the class classification | category when the depth of the hierarchy of QAOU of a process target is smaller than a threshold value. Yes, (b) is a diagram schematically showing class classification when the depth of the hierarchy of the QAOU to be processed is greater than or equal to a threshold value. In the said embodiment, it is a figure which shows an example of the class classification in case the band offset is designated, (a) shows an example of the class classification in case the depth of the hierarchy of QAOU of a process target is smaller than a threshold value. (B) is a figure which shows an example of the class classification | category when the depth of the hierarchy of QAOU of a process target is more than a threshold value. It is a block diagram which shows the structure of the moving image encoder which concerns on the said embodiment. It is a block diagram which shows the structure of the adaptive offset filter with which the moving image encoder which concerns on the said embodiment is provided. It is a flowchart which shows the flow of a process by the offset calculation part of the adaptive offset filter with which the moving image encoder which concerns on the said embodiment is provided. It is a flowchart which shows the flow of a process by the offset information selection part of the adaptive offset filter with which the moving image encoder which concerns on the said embodiment is provided. In the said embodiment, it is a figure which shows an example of the QAOMU number attached | subjected to QAOMU contained in the process unit of object, Comprising: (a) is a figure which shows the QAOMU number attached | subjected to QAOMU whose division depth is 0. (B) is a diagram showing a QAOMU number assigned to a QAOMU having a division depth of 1, and (c) is a diagram showing a QAOMU number attached to a QAOMU having a division depth of 2. (D) is a figure which shows the QAOMU number attached | subjected to QAOMU with a division | segmentation depth of 3, (e) is a figure which shows the QAOMU number attached | subjected to QAOMU with a division | segmentation depth of 4. FIG. In the said embodiment, it is a figure which shows the outline | summary which calculates the square error about each offset type about QAOU whose QAOU index is "x". It is a figure for demonstrating the process by the offset information selection part of the adaptive offset filter with which the moving image encoder which concerns on the said embodiment is provided, Comprising: (a) is the aspect of a division | segmentation in case the division | segmentation depth is 0 and 1 (B) is a diagram showing an aspect of division when the division depth is 1, and (c) is a diagram showing an aspect of division when the division depth is 2. (D) is a figure which shows an example of the division | segmentation determined by the offset information selection part. FIG. 6 is a diagram for explaining a configuration for switching between EO and BO according to pixel values in the embodiment according to the present invention, and (a) is a diagram for explaining an outline of a configuration for switching between EO and BO according to pixel values. (B) is a diagram for explaining specific switching values, and (c) is a diagram showing the contents of the list memory stored in the offset information storage unit 621. In embodiment which concerns on this invention, it is a figure for demonstrating the case where the type of EO is limited to a horizontal direction, (a) is a figure for demonstrating the outline | summary of an effect, (b) is a horizontal direction. It is a figure for demonstrating the position of the pixel of (b), (c) is a figure for demonstrating the state different from (b), (d) is a figure for demonstrating the effect of (c). . In the said embodiment, it is a figure for demonstrating the case where the type of EO is limited to a horizontal direction, (a) And (b) is a figure which shows the case where the pixel to refer exists in an asymmetrical position, ( c) is a diagram showing an outline of a horizontal edge. In embodiment which concerns on this invention, it is a figure which shows the outline | summary about the case where the precision of offset is improved. FIG. 4 is a diagram showing an outline when classifying is subdivided in the above embodiment, (a) is a diagram showing a conversion table, and (b) to (d) are diagrams for explaining class division. is there. In the embodiment according to the present invention, it is a diagram for explaining the case of performing the class classification according to the color difference, (a) is a diagram for explaining the class classification in consideration of the achromatic pixel value, (B) is a figure for demonstrating a class classification asymmetrical in two range which pinches | interposes an achromatic color pixel value, (c) and (d) explain a class classification different for every color channel (Cr or Cb). (E) is a figure for demonstrating the case where the class classification of BO is made into one. In the embodiment according to the present invention, it is a block diagram showing a configuration of an offset information decoding unit. In the said embodiment, it is a figure which shows the outline | summary in case there exists a class without the pixel classified. In the said embodiment, it is a figure which shows the syntax in the case of using a prediction candidate flag. It is a figure for demonstrating that a moving image decoding apparatus and a moving image coding apparatus can be utilized for transmission / reception of a moving image, (a) is the block diagram which showed the structure of the transmission device carrying a moving image coding apparatus (B) is a block diagram showing a configuration of a receiving apparatus equipped with a moving picture decoding apparatus. It is a figure for demonstrating that a moving image decoding apparatus and a moving image encoding apparatus can be utilized for recording and reproduction | regeneration of a moving image, (a) shows the structure of the recording device carrying the moving image encoding apparatus 2. FIG. 8B is a block diagram illustrating a configuration of a playback device equipped with a video decoding device. It is a block diagram which shows the structure of the offset information decoding part which concerns on the said embodiment. (A) is a block diagram which shows the structure of the use offset type selection part which concerns on the said embodiment, (b) is a block diagram which shows the structure of another use offset type selection part. It is a block diagram which shows the structure of the class classification | category part which concerns on the said embodiment. It is a figure which shows the syntax of offset information and QAOU information which concerns on the said embodiment, (a) is a figure which shows the syntax of offset information, (b) is a figure which shows the syntax of QAOU information, (c () Is a diagram showing the syntax of the entire adaptive offset filter that calls the syntax shown in (a) and (b).

[Embodiment 1]
(Encoded data # 1)
Prior to detailed description of the video encoding device 2 and the video decoding device 1 according to the present embodiment, the encoded data # 1 generated by the video encoding device 2 and decoded by the video decoding device 1 is described. The data structure will be described.

FIG. 2 is a diagram showing a data structure of encoded data # 1. The encoded data # 1 exemplarily includes a sequence and a plurality of pictures constituting the sequence.

FIG. 2 shows the hierarchical structure below the picture layer in the encoded data # 1. 2A to 2D are included in the picture layer that defines the picture PICT, the slice layer that defines the slice S, the tree block layer that defines the tree block TBLK, and the tree block TBLK, respectively. It is a figure which shows the CU layer which prescribes | regulates a coding unit (Coding | union Unit; CU).

(Picture layer)
In the picture layer, a set of data referred to by the video decoding device 1 for decoding a picture PICT to be processed (hereinafter also referred to as a target picture) is defined. As shown in FIG. 2A, the picture PICT includes a picture header PH and slices S1 to SNS (NS is the total number of slices included in the picture PICT).

In the following description, if it is not necessary to distinguish each of the slices S1 to SNS, the subscripts may be omitted. The same applies to other data with subscripts included in encoded data # 1 described below.

The picture header PH includes a coding parameter group referred to by the video decoding device 1 in order to determine a decoding method of the target picture. For example, the encoding mode information (entropy_coding_mode_flag) indicating the variable length encoding mode used in encoding by the moving image encoding device 2 is an example of an encoding parameter included in the picture header PH.

When “entropy_coding_mode_flag” is “0”, the picture PICT is encoded by CAVLC (Context-based “Adaptive” Variable “Length” Coding). When “entropy_coding_mode_flag” is “1”, the picture PICT is encoded by CABAC (Context-based Adaptive Binary Arithmetic Coding).

Note that the picture PICT may include a picture parameter set (PPS: Picture Parameter Set) in addition to the picture header PH.

(Slice layer)
In the slice layer, a set of data referred to by the video decoding device 1 for decoding the slice S to be processed (also referred to as a target slice) is defined. As shown in FIG. 2B, the slice S includes a slice header SH and a sequence of tree blocks TBLK1 to TBLKNC (NC is the total number of tree blocks included in the slice S).

The slice header SH includes a coding parameter group that the moving image decoding apparatus 1 refers to in order to determine a decoding method of the target slice. Slice type designation information (slice_type) for designating a slice type is an example of an encoding parameter included in the slice header SH.

As slice types that can be specified by the slice type specification information, (1) I slice using only intra prediction at the time of encoding, (2) P slice using unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding may be used.

Further, the slice header SH includes a filter parameter FP that is referred to by an adaptive filter provided in the video decoding device 1. The filter parameter FP may be included in the picture header PH or the picture parameter set PPS.

(Tree block layer)
In the tree block layer, a set of data referred to by the video decoding device 1 for decoding a processing target tree block TBLK (hereinafter also referred to as a target tree block) is defined. Note that a tree block may be referred to as a maximum coding unit (LCU).

The tree block TBLK includes a tree block header TBLKH and coding unit information CU1 to CUNL (NL is the total number of coding unit information included in the tree block TBLK). Here, first, a relationship between the tree block TBLK and the coding unit information CU will be described as follows.

The tree block TBLK is divided into units for specifying a block size for each process of intra prediction or inter prediction and conversion.

The above unit of the tree block TBLK is divided by recursive quadtree partitioning. The tree structure obtained by this recursive quadtree partitioning is hereinafter referred to as a coding tree.

Hereinafter, a unit corresponding to a leaf that is a node at the end of the coding tree is referred to as a coding node. In addition, since the encoding node is a basic unit of the encoding process, hereinafter, the encoding node is also referred to as an encoding unit (CU).

That is, coding unit information (hereinafter referred to as CU information) CU1 to CUNL is information corresponding to each coding node (coding unit) obtained by recursively dividing the tree block TBLK into quadtrees.

Also, the root of the coding tree is associated with the tree block TBLK. In other words, the tree block TBLK is associated with the highest node of the tree structure of the quadtree partition that recursively includes a plurality of encoding nodes.

Note that the size of the image area corresponding to each coding node is half the size of the coding node to which the coding node directly belongs (that is, the unit of the node one level higher than the coding node). .

Also, the size that each coding node can take depends on the size designation information of the coding node and the maximum hierarchy depth (maximum hierarchical depth) included in the sequence parameter set SPS of the coded data # 1. For example, when the size of the tree block TBLK is 64 × 64 pixels and the maximum hierarchical depth is 3, the encoding nodes in the hierarchy below the tree block TBLK have four sizes, that is, 64 × 64. Any of pixel, 32 × 32 pixel, 16 × 16 pixel, and 8 × 8 pixel can be taken.

As the block structure, the slice S is divided to form a tree block TBLK, and the tree block TBLK is divided to form a coding unit CU.

(Tree block header)
The tree block header TBLKH includes an encoding parameter referred to by the video decoding device 1 in order to determine a decoding method of the target tree block. Specifically, as shown in FIG. 2C, tree block division information SP_TBLK that designates a division pattern of the target tree block into each CU, and a quantization parameter difference that designates the size of the quantization step. Δqp (qp_delta) is included.

The tree block division information SP_TBLK is information representing a coding tree for dividing the tree block. Specifically, the shape and size of each CU included in the target tree block, and the position in the target tree block Is information to specify.

Note that the tree block division information SP_TBLK may not explicitly include the shape or size of the CU. For example, the tree block division information SP_TBLK may be a set of flags (split_coding_unit_flag) indicating whether or not the entire target tree block or a partial area of the tree block is divided into four. In this case, the shape and size of each CU can be specified by using the shape and size of the tree block together.

Further, the quantization parameter difference Δqp is a difference qp−qp ′ between the quantization parameter qp in the target tree block and the quantization parameter qp ′ in the tree block encoded immediately before the target tree block.

(CU layer)
In the CU layer, a set of data referred to by the video decoding device 1 for decoding a CU to be processed (hereinafter also referred to as a target CU) is defined.

Here, before explaining the specific contents of the data included in the CU information CU, the tree structure of the data included in the CU will be described. The coding node is the root of a prediction tree (PT) and a transformation tree (TT). The prediction tree and the conversion tree are described as follows.

In the prediction tree, the encoding node is divided into one or a plurality of prediction blocks, and the position and size of each prediction block are defined. In other words, the prediction block is one or a plurality of non-overlapping areas constituting the encoding node. The prediction tree includes one or a plurality of prediction blocks obtained by the above division.

Prediction processing is performed for each prediction block. Hereinafter, a prediction block that is a unit of prediction is also referred to as a prediction unit (PU).

There are roughly two types of division in the prediction tree: intra prediction and inter prediction.

In the case of intra prediction, there are 2N × 2N (the same size as the encoding node) and N × N division methods.

In the case of inter prediction, there are 2N × 2N (the same size as the encoding node), 2N × N, N × 2N, N × N, and the like.

Also, in the transform tree, the encoding node is divided into one or a plurality of transform blocks, and the position and size of each transform block are defined. In other words, the transform block is one or a plurality of non-overlapping areas constituting the encoding node. The conversion tree includes one or a plurality of conversion blocks obtained by the above division.

The division in the transformation tree includes the allocation of an area having the same size as the encoding node as the transformation block and the recursive quadtree division similar to the above-described division of the tree block.

Conversion processing is performed for each conversion block. Hereinafter, the transform block which is a unit of transform is also referred to as a transform unit (TU).

(Data structure of CU information)
Next, specific contents of data included in the CU information CU will be described with reference to FIG. As shown in FIG. 2D, the CU information CU specifically includes a skip flag SKIP, PT information PTI, and TT information TTI.

The skip flag SKIP is a flag indicating whether or not the skip mode is applied to the target CU. When the value of the skip flag SKIP is 1, that is, when the skip mode is applied to the target CU, PT information PTI and TT information TTI in the CU information CU are omitted. Note that the skip flag SKIP is omitted for the I slice.

PT information PTI is information regarding the PT included in the CU. In other words, the PT information PTI is a set of information regarding each of one or a plurality of PUs included in the PT, and is referred to when the moving image decoding apparatus 1 generates a predicted image. As shown in FIG. 2D, the PT information PTI includes prediction type information PType and prediction information PInfo.

Prediction type information PType is information that specifies whether intra prediction or inter prediction is used as a prediction image generation method for the target PU.

The prediction information PInfo is composed of intra prediction information or inter prediction information depending on which prediction method is specified by the prediction type information PType. Hereinafter, a PU to which intra prediction is applied is also referred to as an intra PU, and a PU to which inter prediction is applied is also referred to as an inter PU.

Also, the prediction information PInfo includes information specifying the shape, size, and position of the target PU. As described above, the generation of the predicted image is performed in units of PU. Details of the prediction information PInfo will be described later.

TT information TTI is information related to TT included in the CU. In other words, the TT information TTI is a set of information regarding each of one or a plurality of TUs included in the TT, and is referred to when the moving image decoding apparatus 1 decodes residual data. Hereinafter, a TU may be referred to as a block.

As shown in FIG. 2 (d), the TT information TTI includes TT division information SP_TT that designates a division pattern for each transform block of the target CU, and quantized prediction residuals QD1 to QDNT (NT is a target CU). The total number of blocks contained in).

TT division information SP_TT is information for determining the shape and size of each TU included in the target CU and the position in the target CU. For example, the TT division information SP_TT can be generated from information (split_transform_unit_flag) indicating whether or not a target node is to be divided and information (trafoDepth) indicating the division depth. The TT partition information SP_TT is basically encoded for each node of the quadtree, but depends on restrictions on the transform size (maximum transform size, minimum transform size, maximum hierarchy depth of the quadtree). May be omitted and estimated.

Each quantization prediction residual QD is encoded data generated by the moving image encoding apparatus 2 performing the following processes 1 to 3 on a target block that is a processing target block.

Process 1: Perform orthogonal transform (DCT transform (Discrete Cosine Transform) or DST transform (Discrete sine Transform)) on the prediction residual obtained by subtracting the predicted image from the encoding target image;
Process 2: Quantize the transform coefficient obtained in Process 1;
Process 3: Variable length coding is performed on the transform coefficient quantized in Process 2;
The quantization parameter qp described above represents the magnitude of the quantization step QP used when the moving image coding apparatus 2 quantizes the transform coefficient (QP = 2 ^{qp / 6} ).

(Prediction information PInfo)
As described above, there are two types of prediction information PInfo: inter prediction information and intra prediction information.

The inter prediction information includes an encoding parameter that is referred to when the video decoding device 1 generates an inter predicted image by inter prediction. More specifically, the inter prediction information includes inter PU division information that specifies a division pattern of the target CU into each inter PU, and inter prediction parameters for each inter PU.

The inter prediction parameters include a reference image index, an estimated motion vector index, and a motion vector residual.

On the other hand, the intra prediction information includes an encoding parameter that is referred to when the video decoding device 1 generates an intra predicted image by intra prediction. More specifically, the intra prediction information includes intra PU division information that specifies a division pattern of the target CU into each intra PU, and intra prediction parameters for each intra PU. The intra prediction parameter is a parameter for designating an intra prediction method (prediction mode) for each intra PU.

(Offset unit)
In the present embodiment, each picture or each slice is recursively divided into a plurality of offset units (unit area, also referred to as QAOU: Quad Adaptive Offset Unit) by a quadtree structure. Here, QAOU is a processing unit of offset filter processing by the adaptive offset filter according to the present embodiment.

As shown in FIGS. 2E and 2F, the QAOU information that is information about each QAOU includes “sao_split_flag” that indicates whether or not the own (own QAOU) is further divided. More specifically, “sao_split_flag” is specified by arguments (sao_curr_depth, ys, xs) described later, and is also expressed as “sao_split_flag [sao_curr_depth] [ys] [xs]”.

When “sao_split_flag” included in a certain QAOU indicates that the QAOU is further divided (that is, when the QAOU is not a leaf), as shown in FIG. The QAOU information related to the QAOU includes QAOU information related to each of a plurality of QAOUs included in the QAOU.

On the other hand, when “sao_split_flag” included in a certain QAOU indicates that the QAOU is not further divided (that is, when the QAOU is a leaf), it is shown in FIG. Thus, the QAOU information related to the QAOU includes offset information OI related to the QAOU. Further, as shown in FIG. 2F, the offset information OI includes offset type designation information OTI for designating an offset type and an offset group determined according to the offset type. Furthermore, as shown in FIG. 2F, the offset group includes a plurality of offsets.

Note that the offset in the encoded data is a quantized value. Further, the offset in the encoded data may be a prediction residual obtained by using some prediction, for example, linear prediction.

Next, offset information OI will be described with reference to FIG. FIG. 3A is a diagram illustrating the syntax of the offset information OI (denoted as “sao_offset_param ()” in FIG. 3A).

As shown in FIG. 3A, the offset information OI includes a parameter “sao_type_idx [sao_curr_depth] [ys] [xs]”. When the parameter “sao_type_idx [sao_curr_depth] [ys] [xs]” is not “0”, the offset information OI includes the parameter “sao_offset [sao_curr_depth] [ys] [xs] [i]”.

(Sao_curr_depth, ys, xs)
“Sao_type_idx” and “sao_offset” arguments “sao_curr_depth” are parameters indicating the QAOU division depth, and “ys” and “xs” are the position of the QAOU (or QAOMU described later) in the y direction and It is a parameter for representing the position in the x direction.

The mode of QAOU division according to the value of “sao_curr_depth” will be described with reference to FIG. FIG. 4 is a diagram illustrating a QAOU division mode according to the value of “sao_curr_depth”. FIG. 4A illustrates a division mode when “sao_curr_depth” = 0, and FIG. 4B illustrates “sao_curr_depth” = 1. (C) shows a division mode when “sao_curr_depth” = 2, (d) shows a division mode when “sao_curr_depth” = 3, and (e) shows “ The division mode when sao_curr_depth ”= 4 is shown. As shown in FIGS. 4A to 4E, each QAOU is specified by “sao_curr_depth” and (xs, ys).

As shown in FIG. 4A, when “sao_curr_depth” = 0, “xs” and “ys” are each “0”, and as shown in FIG. 4B, “sao_curr_depth” = When it is 1, “xs” and “ys” can take values of “0” and “1”, respectively. Further, as shown in FIG. 4C, when “sao_curr_depth” = 2, “xs” and “ys” are values of “0”, “1”, “2”, and “3”, respectively. Can take. In general, for a given “sao_curr_depth”, “xs” and “ys” can take values of 0 to 2 × “sao_curr_depth” −1.

FIG. 3B is a diagram showing the syntax of the QAOU information (described as “sao_split_param ()” in FIG. 3B) as described above. As shown in the syntax of FIG. 3B, if the division depth “sao_curr_depth” is smaller than the maximum value set by the predetermined “saoMaxDepth”, whether or not the QAOU is further divided is selected by the parameter “sao_split_flag” Is done. When splitting, “sao_split_param ()” of the next hierarchical depth is recursively called. When the division depth reaches the maximum value ("sao_curr_depth" is not smaller than "saoMaxDepth"), "sao_split_flag [sao_curr_depth] [ys] [xs]" is set to "0", and further divisions Not done.

(Sao_type_idx)
“Sao_type_idx [sao_curr_depth] [ys] [xs]” corresponds to the above-described offset type designation information OTI, and is a parameter for designating an offset type for each QAOU. Hereinafter, “sao_type_idx” may be simply referred to as an offset type.

In this embodiment, “sao_type_idx [sao_curr_depth] [ys] [xs]” takes an integer value from “0” to “6”. “Sao_type_idx [sao_curr_depth] [ys] [xs]” = 0 indicates that the offset filter processing is not performed on the pre-offset filter image (for example, a deblocked decoded image P_DB described later) in the target QAOU. “Sao_type_idx [sao_curr_depth] [ys] [xs]” = 1 to 4 indicates that edge offset processing (EO) is performed on the pre-offset filter image in the target QAOU, and “sao_type_idx [sao_curr_depth] [ys ] [xs] ”= 5 or 6 indicates that band offset processing (BO) is performed on the pre-offset filter image in the target QAOU. The band offset process refers to a process of adding any of a plurality of offsets to the pixel value of the processing target pixel according to the size of the pixel value of the processing target pixel. In addition, the edge offset processing means that any one of a plurality of offsets is added to the pixel value of the processing target pixel according to the difference between the pixel value of the processing target pixel and the pixel values of the pixels around the processing target pixel. Refers to the process of adding. Specific contents of the edge offset process and the band offset process will be described later.

(Sao_offset)
“Sao_offset [sao_curr_depth] [ys] [xs] [i]” represents a specific value of the offset added to each pixel included in the target QAOU in the offset filter processing by the adaptive offset filter according to the present embodiment. It is a parameter. In the present embodiment, “sao_offset” may be simply referred to as an offset.

“Sao_offset [sao_curr_depth] [ys] [xs] [i]” is specified by the index “i” in addition to the arguments “sao_curr_depth”, “ys” and “xs”. Here, the index “i” is an index for designating a class, and is also expressed as “class_idx”. Index “i” is “sao_type_idx [sao_curr_depth] [ys] [xs]” when the value is “1 to 4” (that is, an edge offset), and “i” = 0-4 Takes an integer value. Further, when the value of “sao_type_idx [sao_curr_depth] [ys] [xs]” is “5 or 6” (that is, in the case of a band offset), “i” is an integer value of 0 to 16. As will be described later, in any case, “i” = 0 represents that no offset is added.

As will be described later, the adaptive offset filter according to the present embodiment classifies the target pixel included in the target QAOU into one of the plurality of classes, and classifies the target pixel with respect to the target pixel. Add the offset of.

Further, the descriptor (descriptor) “ue (v)” shown in FIG. 3 indicates that the syntax associated with this descriptor is an unsigned numerical value, and the value is variable-length encoded. “Se (v)” means that the syntax associated with this descriptor is a signed numerical value, which means that the variable length coding is performed by dividing it into a sign and an absolute value, and “ae (v)” This means that the syntax associated with this descriptor is variable length coded using arithmetic coding. “|” Indicates that any encoding method is selectively used.

FIG. 35 shows another example of the syntax of offset information and QAOU information.

FIG. 35 (a) shows the syntax of offset information. Although the configuration is the same as that of FIG. 3A, the argument “sao_offset_param ()” and the subscripts of the arrays “sao_split_flag”, “sao_type_idx”, and “sao_offset” are values that represent color components. component ”has been added. Thereby, different QAOU division can be performed for each color component such as luminance and color difference, and different offsets can be applied.

FIG. 35 (b) shows the syntax of QAOU information. Similar to FIG. 35A, the syntax is obtained by adding the color component “component” as an argument to FIG. 3B.

FIG. 35 (c) shows the syntax of the entire adaptive offset filter that calls the syntaxes of FIGS. 35 (a) and 35 (b). The parameter “sao_flag” is a flag for selecting whether or not to apply the adaptive offset filter. Only when the flag is true, the parameter related to the subsequent adaptive offset filter is used. When the flag is true, the syntax “sao_split_param ()” and “sao_offset_param ()” are called for each color component, specifying the highest layer. Since it is the highest hierarchy, the arguments given to each syntax are sao_curr_depth = 0, ys = 0, and xs = 0. The value of “component” is 0 for luminance (Y), 1 for color difference (Cb), and 2 for color difference (Cr) to distinguish each color component. The value of component may be another value as long as the color component to be processed can be distinguished.

For color difference (Cb) and color difference (Cr), select whether or not to apply an adaptive offset filter using the corresponding flags “sao_flag_cb” and “sao_flag_cr”. QAOU information and offset information corresponding to the component are not stored.

Since the argument “component” is added to the syntax shown in FIG. 35, the argument [sao_curr_depth] [ys] [xs] is changed to [sao_curr_depth] [ys] [xs] [component] in the description using FIG. Replace with and process. The same applies to the following description.

(Moving picture decoding apparatus 1)
Next, the moving picture decoding apparatus 1 according to the present embodiment will be described with reference to FIG. 1 and FIGS. The moving picture decoding apparatus 1 includes H. H.264 / MPEG-4. A method adopted in AVC, a method adopted in KTA software, which is a codec for joint development in VCEG (Video Coding Expert Group), and a method adopted in TMuC (Test Model under Consideration) software, which is the successor codec And the technology employed in HM (HEVC TestModel) software.

FIG. 5 is a block diagram showing a configuration of the moving picture decoding apparatus 1. As illustrated in FIG. 5, the video decoding device 1 includes a variable length code decoding unit 13, a motion vector restoration unit 14, a buffer memory 15, an inter prediction image generation unit 16, an intra prediction image generation unit 17, and a prediction method determination unit 18. , An inverse quantization / inverse transform unit 19, an adder 20, a deblocking filter 41, an adaptive filter 50, and an adaptive offset filter 60. The moving picture decoding apparatus 1 is an apparatus for generating moving picture # 2 by decoding encoded data # 1.

The variable length code decoding unit 13 decodes the prediction parameter PP related to each partition from the encoded data # 1. That is, for the inter prediction partition, the reference image index RI, the estimated motion vector index PMVI, and the motion vector residual MVD are decoded from the encoded data # 1, and these are supplied to the motion vector restoration unit 14. On the other hand, with respect to the intra prediction partition, (1) size designation information for designating the size of the partition and (2) prediction index designation information for designating the prediction index are decoded from the encoded data # 1, and this is decoded into the intra prediction image. This is supplied to the generation unit 17. In addition, the variable length code decoding unit 13 decodes the CU information from the encoded data, and supplies this to the prediction method determination unit 18 (not shown). Furthermore, the variable length code decoding unit 13 decodes the quantization prediction residual QD for each block and the quantization parameter difference Δqp for the tree block including the block from the encoded data # 1, and dequantizes and decodes them. This is supplied to the inverse conversion unit 19. Further, the variable length code decoding unit 13 extracts QAOU information from the encoded data # 1 and supplies the extracted QAOU information to the adaptive offset filter 60.

The motion vector restoration unit 14 restores the motion vector mv related to each inter prediction partition from the motion vector residual MVD related to that partition and the restored motion vector mv ′ related to another partition. Specifically, (1) the estimated motion vector pmv is derived from the restored motion vector mv ′ according to the estimation method specified by the estimated motion vector index PMVI, and (2) the derived estimated motion vector pmv and the motion vector remaining are derived. The motion vector mv is obtained by adding the difference MVD. It should be noted that the restored motion vector mv ′ relating to other partitions can be read from the buffer memory 15. The motion vector restoration unit 14 supplies the restored motion vector mv together with the corresponding reference image index RI to the inter predicted image generation unit 16. For the inter prediction partition that performs bi-directional prediction (weighted prediction), the restored two motion vectors mv1 and mv2 are supplied to the inter predicted image generation unit 16 together with the corresponding reference image indexes RI1 and RI2.

The inter prediction image generation unit 16 generates a motion compensation image mc related to each inter prediction partition. Specifically, using the motion vector mv supplied from the motion vector restoration unit 14, the motion compensated image mc from the filtered decoded image P_FL ′ designated by the reference image index RI also supplied from the motion vector restoration unit 14 is used. Is generated. Here, the filtered decoded image P_FL ′ is subjected to deblocking processing by the deblocking filter 41, offset filtering processing by the adaptive offset filter 60, and adaptive filtering processing by the adaptive filter 50 on the decoded image P that has already been decoded. It is an image obtained by applying. The inter predicted image generation unit 16 can read out the pixel value of each pixel constituting the filtered decoded image P_FL ′ from the buffer memory 15. The motion compensation image mc generated by the inter prediction image generation unit 16 is supplied to the prediction method determination unit 18 as an inter prediction image Pred_Inter. For the inter prediction partition that performs bi-directional prediction (weighted prediction), (1) a motion compensated image mc1 is generated from the filtered decoded image P_FL1 ′ specified by the reference image index RI1 using the motion vector mv1. (2) A motion compensated image mc2 is generated from the filtered decoded image P_FL2 ′ specified by the reference image index RI2 using the motion vector mv2, and (3) weighting between the motion compensated image mc1 and the motion compensated image mc2 An inter predicted image Pred_Inter is generated by adding an offset value to the average.

The intra predicted image generation unit 17 generates a predicted image Pred_Intra related to each intra prediction partition. Specifically, first, the prediction mode decoded from the encoded data # 1 is referred to, and the prediction mode is assigned to the target partition in, for example, raster scan order. Subsequently, a predicted image Pred_Intra is generated from the decoded image P according to the prediction method indicated by the prediction mode. The intra predicted image Pred_Intra generated by the intra predicted image generation unit 17 is supplied to the prediction method determination unit 18.

In addition, the intra predicted image generation unit 17 supplies the adaptive filter 50 with intra coding mode information IEM, which is information indicating the size of the target partition and the prediction mode assigned to the target partition.

The prediction method determination unit 18 determines whether each partition is an inter prediction partition for performing inter prediction or an intra prediction partition for performing intra prediction based on the CU information. In the former case, the inter predicted image Pred_Inter generated by the inter predicted image generation unit 16 is supplied to the adder 20 as the predicted image Pred. In the latter case, the inter predicted image generation unit 17 generates the inter predicted image Pred_Inter. The intra predicted image Pred_Intra that has been processed is supplied to the adder 20 as the predicted image Pred.

The inverse quantization / inverse transform unit 19 (1) inversely quantizes the quantized prediction residual QD, (2) performs inverse DCT (Discrete Cosine Transform) transform on the DCT coefficient obtained by the inverse quantization, and (3) The prediction residual D obtained by the inverse DCT transform is supplied to the adder 20. When the quantization prediction residual QD is inversely quantized, the inverse quantization / inverse transform unit 19 derives the quantization step QP from the quantization parameter difference Δqp supplied from the variable length code decoding unit 13. The quantization parameter qp can be derived by adding the quantization parameter difference Δqp to the quantization parameter qp ′ relating to the tree block that has just undergone inverse quantization / inverse DCT transformation, and the quantization step QP is performed from the quantization step qp to QP. = 2 ^{pq / 6} . The generation of the prediction residual D by the inverse quantization / inverse transform unit 19 is performed in units of blocks (transform units).

The adder 20 generates the decoded image P by adding the prediction image Pred supplied from the prediction method determination unit 18 and the prediction residual D supplied from the inverse quantization / inverse transformation unit 19.

The deblocking filter 41 determines that the block boundary in the decoded image P or the block boundary in the decoded image P when the difference between the pixel values of pixels adjacent to each other via the block boundary in the decoded image P or the CU boundary is smaller than a predetermined threshold value. By performing deblocking processing on the CU boundary, an image near the block boundary or the vicinity of the CU boundary is smoothed. The image subjected to the deblocking process by the deblocking filter 41 is output to the adaptive offset filter 60 as a deblocked decoded image P_DB.

The adaptive offset filter 60 performs an offset filter process using an offset decoded from the encoded data # 1 on the deblocked decoded image P_DB supplied from the deblocking filter 41, using QAOU as a processing unit. An offset filtered decoded image P_OF is generated. The generated offset filtered decoded image P_OF is supplied to the adaptive offset filter 60. Since a specific configuration of the adaptive offset filter 60 will be described later, description thereof is omitted here.

The adaptive filter 50 performs a filtering process using the filter parameter FP decoded from the encoded data # 1 on the offset filtered decoded image P_OF supplied from the adaptive offset filter 60, thereby obtaining a filtered decoded image P_FL. Is generated. The image that has been filtered by the adaptive filter 50 is output to the outside as a filtered decoded image P_FL, and is associated with the POC designation information decoded from the encoded data by the variable length code decoding unit 13 in the buffer memory 15. Stored.

(Adaptive offset filter 60)
Next, details of the adaptive offset filter 60 will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of the adaptive offset filter 60. As shown in FIG. 1, the adaptive offset filter 60 includes an adaptive offset filter information decoding unit 61 and an adaptive offset filter processing unit 62.

Further, as shown in FIG. 1, the adaptive offset filter information decoding unit 61 includes an offset information decoding unit 611 and a QAOU structure decoding unit 612.

The offset information decoding unit 611 refers to the QAOU information included in the encoded data # 1, and decodes the offset information OI included in the QAOU information. Also, each value of “sao_type_idx [sao_curr_depth] [ys] [xs] [component]” and “sao_offset [sao_curr_depth] [ys] [xs] [i]” obtained by decoding the offset information OI is respectively Are supplied to the offset information storage unit 621 in association with the arguments (sao_curr_depth, ys, xs) and (sao_curr_depth, ys, xs, i).

More specifically, the offset information decoding unit 611 decodes the code from the encoded data # 1, converts the decoded code into a value of “sao_type_idx”, and supplies the value to the offset information storage unit 621 in association with the argument. Here, the offset information decoding unit 611 changes the code decoding method and the conversion from the code to the value of “sao_type_idx” according to the depth of the QAOU hierarchy to be processed. Conditions such as the depth of the QAOU hierarchy are called parameter conditions. Decoding of offset information according to general parameter conditions will be described later with reference to FIGS. 32 and 33.

The code decoding method may be performed using different maximum values when the depth of the QAOU hierarchy to be processed is smaller than the threshold and when the depth of the QAOU hierarchy to be processed is greater than or equal to the threshold. However, the maximum value may be used only when the depth of the QAOU hierarchy to be processed is equal to or greater than a threshold value. Further, different binarization may be used when the depth of the QAOU hierarchy to be processed is smaller than the threshold and when the depth of the QAOU hierarchy to be processed is greater than or equal to the threshold. Further, different contexts may be used depending on whether the depth of the QAOU hierarchy to be processed is smaller than the threshold value and the depth of the QAOU hierarchy to be processed is greater than or equal to the threshold value.

For example, when the depth of the QAOU layer to be processed is smaller than the threshold, the code is decoded by variable length coding (ue (v)), and when the depth of the QAOU layer to be processed is equal to or greater than the threshold The code may be decoded by truncated coding (te (v)) according to the number of offset types. If the number of offset types is a power of 2, the code may be decoded by fixed length coding. If the number of offset types is four, it can be expressed in 2 bits, and can be fixed-length encoded in 2 bits.

Also, the offset information decoding unit 611 converts the decoded code into the value “sao_type_idx” using the conversion table 801 and the conversion table 802 as shown in FIG. The conversion table 801 in FIG. 8A shows two conversion patterns. These two types of conversion patterns are properly used depending on the depth of the QAOU hierarchy to be processed. That is, the offset information decoding unit 611 uses the conversion table 801A when the depth of the QAOU hierarchy to be processed is smaller than the threshold value, and uses the conversion table when the depth of the QAOU hierarchy to be processed is equal to or greater than the threshold value. 801B is used.

In the conversion table 801A of FIG. 8A, the code “1” is associated with the offset type “EO_0” (“sao_type_idx” = 1), and the code “2” is associated with the offset type “EO_1” (“sao_type_idx”). “= 2) is associated, and the offset type“ EO_2 ”(“ sao_type_idx ”= 3) is associated with the code“ 3 ”. The same applies to the codes 4 to 6 below.

Also, in the conversion table 801B of FIG. 8A, the code “1” is associated with the offset type “EO — 0” (“sao_type_idx” = 1), and the code “2” is associated with the offset type “BO — 0” ( “Sao_type_idx” = 5) is associated, and the offset type “BO_1” (“sao_type_idx” = 6) is associated with the code “3”. In the conversion table 801B, the codes 4 to 6 are not used.

As described above, the conversion table 801A includes all the offset types used for the adaptive offset (SAO), whereas the conversion table 801B includes a part of the offset types used for the adaptive offset (SAO). Only type is included. Therefore, if the depth of the QAOU hierarchy to be processed is greater than or equal to the threshold value, only some offset types can be used.

This is because, in a deep hierarchy, since the area of the QAOU is small, the characteristics of the pixel values in the QAOU are nearly uniform, and an appropriate offset can be derived without using many types. In addition, since the number of offset types to be used can be reduced, the required amount of memory can be reduced and the encoding efficiency can be improved by shortening the data length of the encoded data indicating the offset type. be able to.

Note that if the correspondence between the code and the offset type in the conversion table 801B is the same as the correspondence order of the code and the offset type in the conversion table 801A, only the conversion table 801A may be used.

8B shows various examples of conversion tables similar to the conversion table 801B that can be used in place of the conversion table 801B. As with the conversion table 801B, the type of each conversion table is limited. A blank indicates that the corresponding code is not used.

The conversion table 802A is an example using only an edge offset and does not include a band offset. The number of edge offsets is usually four, and the number of band offsets is usually sixteen. Thus, the number of edge offsets is smaller than the number of band offsets. For this reason, by limiting the use of the band offset, the amount of memory used for holding the offset can be reduced. In particular, when the hierarchy is deep, the amount of memory increases because the offset increases. In addition, when the hierarchy is deep, the selectivity of the band offset decreases. Therefore, by using the conversion table 802A having only the edge offset when the hierarchy is deep and using the conversion table (for example, 801A) having the edge offset and the band offset when the hierarchy is shallow, using the conversion table 802A as the parameter condition, The amount of memory can be reduced without reducing the efficiency. Further, since it is possible to omit the cost calculation of unnecessary options in the encoding device, the processing amount can be reduced.

The conversion table 802B is an example using only a band offset and does not include an edge offset. The band offset is characterized in that the amount of calculation is smaller than that of the edge offset, and a line memory or the like for holding the reference pixel is unnecessary because the pixels around the target pixel are not used. Therefore, the above-described effects can be obtained by using the conversion table 802B according to the parameter conditions. The conversion table 802C is an example in which only one band offset “BO — 0” (“sao_type_idx” = 5) is used.

Also, the conversion table 802D is an example using one edge offset and two band offsets as in the conversion table 801B. Specifically, the offset type “EO — 0” (“sao_type_idx” = 1), “BO — 0” (“sao_type_idx” = 5), “BO — 1” (“sao_type_idx” = 6) is used, and the band offset type is preferentially short. This is an example in which codes (small code numbers) are associated. When the hierarchy is shallow, the band offset selection rate is higher than when the hierarchy is deep. Therefore, with the hierarchy as a parameter condition, the conversion table 802D is used when the hierarchy is shallow, and a frequently used type is associated with a short code. Thus, encoding efficiency can be improved.

In addition to the example illustrated in FIG. 8, it is possible to change the association between the code and the offset type according to the parameter condition. An offset type different from any edge offset and band offset of the conversion table 801A can be used alone or in combination with other offset types depending on conditions such as hierarchical depth. Specific examples thereof include an offset type having both EO and BO characteristics, an edge offset that detects an edge at a horizontal sample position different from the conventional edge offset “EO_0”, A band offset by band allocation different from the band offsets “BO — 0” and “BO — 1” can be mentioned.

The QAOU structure decoding unit 612 determines the QAOU partition structure by decoding “sao_split_flag [sao_curr_depth] [ys] [xs]” included in the QAOU information, and indicates the determined QAOU partition structure. The structure information is supplied to the offset information storage unit 621.

Further, an offset attribute setting unit 613 as shown below may be provided. The offset attribute setting unit 613 determines an offset shift value. The offset of the encoded data is encoded with an offset bit depth (also referred to as SAO_DEPTH) that is less accurate than the pixel bit depth (also referred to as PIC_DEPTH). That is, the offset in the encoded data is quantized. The shift value indicates a bit shift amount necessary for performing inverse quantization. Also, the offset attribute setting unit 613 determines the offset bit depth and the offset value range. Here, the bit depth of the offset is determined from a pixel bit depth (also referred to as PIC_DEPTH) (not shown) input to the offset attribute setting unit 613. The pixel bit depth indicates the range of pixel values constituting the input image of the adaptive offset filter 60 in terms of bit width. When the pixel bit depth is N bits, the pixel value ranges from 0 to 2N−1. .

The SAO bit depth and shift value use the following equations, but other values may be used according to the parameter conditions, as will be described later.

SAO_DEPTH = MIN (PIC_DEPTH, 10)
Shift value = PIC_DEPTH-MIN (PIC_DEPTH, 10)
The quantization offset value range is determined by the following equation.

As shown in FIG. 1, the adaptive offset filter processing unit 62 includes an offset information storage unit 621, a QAOU control unit 622, an offset type derivation unit 623, a class classification unit 624, an offset derivation unit 625, and an offset addition unit 626. It is the composition which includes.

The offset information storage unit 621 is designated for each QAOU based on the QAOU structure information, “sao_type_idx [sao_curr_depth] [ys] [xs]”, and “sao_offset [sao_curr_depth] [ys] [xs] [i]”. It manages and stores an offset type and a specific value of an offset relating to each class selectable in the offset type, and has a map memory and a list memory.

The map memory and list memory will be described with reference to FIG. FIG. 6 is a diagram illustrating an example stored in the map memory and the list memory. FIG. 6A is a diagram for explaining an example of the QAOU index stored in the map memory, and FIG. FIG. 4 is a diagram for explaining an example of information stored in a list memory.

The map memory 601 stores a QAOU index, which will be described later, assigned to each offset minimum unit (also referred to as QAOMU: Quad-Adaptive-Offset-Minimum Unit) determined according to the division depth. FIG. 6A shows each QAOMU having a division depth of 3 that constitutes a target processing unit (for example, LCU) and the QAOU index assigned to each QAOMU. In FIG. 6A, indexes 0 to 9 are simply assigned to QAOUs without considering the QAOU division depth. In the example shown in FIG. 6A, the QAOU specified by QAOU index = I is expressed as QAOUI. Further, the thin line in FIG. 6A indicates the QAOMU boundary, and the thick line indicates the QAOU boundary.

As shown in FIG. 6A, QAOU0 is composed of four QAOMUs, and 0 is assigned to these four QAOMUs as QAOU indexes. On the other hand, QAOU3 is composed of one QAOMU, and 3 is assigned to this QAOMU as a QAOU index. In this way, the map memory stores the QAOU index assigned to each QAOMU.

The list memory 602 stores, for each QAOU index, an offset type associated with the QAOU index and a specific value of an offset for each class selectable in the offset type in association with each other. .

Specifically, description will be made with reference to FIG. FIG. 6B shows an offset type associated with each of the QAOU indexes 0 to 9 and an offset for each class selectable in each offset type. “Xxx” in FIG. 6B represents specific numerical values that may be different from each other.

In addition, “BO_1” in FIG. 6B represents an offset type specified by “sao_type_idx” = 5. “EO_1” represents an offset type designated by “sao_type_idx” = 1. In this way, the edge offset that is the offset type specified by “sao_type_idx” = 1, 2, 3, 4 is also expressed as “EO_1, 2, 3, 4”, and “sao_type_idx” = 5,6. The band offsets that are offset types are also referred to as “BO_1, 2”.

As shown in FIG. 6B, when the offset type is a band offset, the offset stored in the list memory with respect to the offset type is 16 offsets 1 to 16 in total. Here, the offset 1 to the offset 16 are “sao_offset [sao_curr_depth] [ys] [xs] [1] when the value of“ sao_type_idx [sao_curr_depth] [ys] [xs] ”is“ 5 ”or“ 6 ”. ] ”To“ sao_offset [sao_curr_depth] [ys] [xs] [16] ”.

On the other hand, when the offset type is an edge offset, the offset stored in the list memory for the offset type is a total of four offsets 1 to 4. Here, the offsets 1 to 4 are “sao_type_idx [sao_curr_depth] [ys] [xs]” when the value is “1, 2, 3, 4” or “sao_offset [sao_curr_depth] [ys] [ys] [ xs] [1] ”to“ sao_offset [sao_curr_depth] [ys] [xs] [4] ”. In the case of an edge offset, nothing is stored in the offsets 5 to 16.

Each QAOMU is assigned a QAOMU number, and the QAOMU numbers can be distinguished from each other by the QAOMU number. Hereinafter, a QAOMU whose QAOMU number is N _Q will also be referred to as QAOMUN _Q.

In the present embodiment, the accuracy of the offset may be varied depending on the QAOU hierarchy to be processed. Depending on the accuracy of the offset, the shift value used for inverse quantization also differs. In this case as well, the offset accuracy and shift value are derived using the pixel bit depth PIC_DEPTH.

As an example of the accuracy of the offset, for example, the accuracy of the offset and the shift value when the depth of the QAOU hierarchy is smaller than the threshold,
SAO_DEPTH = MIN (PIC_DEPTH, 10)
Shift value = PIC_DEPTH-MIN (PIC_DEPTH, 10)
The accuracy of the offset when the depth of the QAOU hierarchy is greater than or equal to the threshold value,
SAO_DEPTH = MIN (PIC_DEPTH, 8)
Shift value = PIC_DEPTH-MIN (PIC_DEPTH, 8)
In this way, the offset accuracy can be made different.

In general, the accuracy of offset (offset bit depth, SAO_DEPTH) and the pixel bit depth (PIC_DEPTH) in which the range of pixel values constituting an input image is expressed in bit width are closely related to each other in terms of quantization error. is there. The bit depth of the output image of the adaptive offset filter 60 is the pixel bit depth PIC_DEPTH, and SAO_DEPTH is the bit depth of the offset added to the pixel. Therefore, even if an offset with an accuracy exceeding the pixel bit depth is used, the offset is discarded in the output process, and it does not make sense to set SAO_DEPTH to exceed PIC_DEPTH. On the other hand, when SAO_DEPTH is smaller than PIC_DEPTH, only the correction that is coarser than the accuracy (PIC_DEPTH) that can correct the input image by the filter can be performed, so that the filter effect is reduced.

Therefore, by limiting the quantized offset to an offset value range that can be expressed with a constant bit width, the bit width for storing the offset quantized by the offset information storage unit 621 can be limited. Thereby, compared with the case where it does not restrict | limit, the effect of memory size reduction can be acquired. On the other hand, if the offset value range is excessively narrowed, the offset reduces the effect of correcting the distortion of the decoded image, and the distortion of the decoded image cannot be removed even by the offset addition process. It will decline.

Therefore, by setting an optimum range of the offset value range so that the encoding efficiency does not decrease, it is possible to maintain the filter effect while reducing the amount of memory used.

When the maximum bit length representing the value of the offset value range is CLIP_BIT and the offset value range is ^defined as -2 ^CLIP_BIT-1 to 2 ^CLIP_BIT-1 -1 by CLIP_BIT = SAO_DEPTH-K, the inventor's experiment According to the above, it has been found that when K = 4, there is no reduction in coding efficiency even if the offset range is limited by the offset value range.

When the pixel bit depth = 8, the offset bit depth SAO_DEPTH is 8 and CLIP_BIT = 8−K = 4. The fact that one offset can be stored in 4 bits means that one offset can be packed and stored in one byte by software that handles 8 bit bytes as a unit, and the memory size can be reduced easily. Become.

If the accuracy of the set offset varies depending on the QAOU hierarchy to be processed, the offset information storage unit 621 switches and secures the unit size of the offset storage area according to the depth of the QAOU hierarchy to be processed. Yes.

Specifically, when the depth of the QAOU hierarchy to be processed is smaller than the threshold, the offset accuracy is n _A bits (for example, n _A = 8 or 6), and the depth of the QAOU hierarchy to be processed is greater than or equal to the threshold. Is set to n _B bits (n _A > n _B , for example, n _B = n _A −2), the offset information storage unit 621 determines that the depth of the processing target QAOU is greater than the threshold value. An area for storing an offset when it is small is secured in n _A bit units, and an area for storing an offset when the depth of the QAOU hierarchy to be processed is equal to or greater than a threshold is secured in n _B bit units.

When the depth of the QAOU layer to be processed is equal to or greater than the threshold, when performing offset read / write to the offset information storage unit 621, (n _A −n _B ) bits are rounded down and read _A configuration in which (n _A −n _B ) bits are rounded up and input / output is preferable. This is because it is not necessary to perform processing in consideration of the difference in offset accuracy in other modules.

Also, the offset information storage unit 621 switches the area of the list memory to be secured according to the required number of classes when the required number of classes changes according to the depth of the QAOU hierarchy to be processed. For example, as will be described later, the band offset is classified into 16 classes when the depth of the QAOU hierarchy to be processed is smaller than the threshold, and is classified into 8 classes when the depth of the QAOU hierarchy to be processed is equal to or greater than the threshold. Consider the case. In this case, the memory capacity necessary for securing the class when the depth of the QAOU hierarchy to be processed is equal to or greater than the threshold is half that when the depth of the QAOU hierarchy to be processed is smaller than the threshold. Therefore, the offset information storage unit 621 switches the size of the list memory to be secured when the depth of the QAOU hierarchy to be processed is equal to or greater than the threshold to half that when the depth of the QAOU hierarchy to be processed is smaller than the threshold. .

The QAOU control unit 622 controls each unit included in the adaptive offset filter processing unit 62. Further, the QAOU control unit 622 refers to the QAOU structure information, divides the deblocked decoded image P_DB into one or a plurality of QAOUs, and scans each QAOU in a predetermined order. In addition, the QAOMU number representing the target QAOMU to be processed is supplied to the offset type deriving unit 623.

The offset type deriving unit 623 derives an offset type specified by the QAOMU number supplied from the QAOU control unit 622 with reference to the map memory and the list memory of the offset information storage unit 621. Further, the derived offset type is supplied to the class classification unit 624.

The class classification unit 624 classifies each pixel included in the target QAOU into any of a plurality of classes that can be selected in the offset type supplied from the offset type deriving unit 623. In addition, the offset type and a class index indicating a class into which each pixel is classified are supplied to the offset deriving unit 625. The specific classification process performed by the class classification unit 624 will be described later, and will not be described here.

The offset deriving unit 625 refers to the list memory of the offset information storage unit 621 and derives an offset specified by the offset type and class index supplied from the class classification unit 624 for each pixel included in the target QAOU. Further, an offset reverse shift unit (not shown) that shifts the offset to the left by the shift value set by the offset attribute setting unit 613 is provided. The offset inverse shift unit performs inverse quantization of the offset so that the bit depth of the offset matches the pixel bit depth. By performing such inverse quantization, it is possible to add the pixel value and the offset at the same bit depth in the adding process of the offset adding unit 626 described later. The offset obtained by inverse quantization for each pixel is supplied to the offset adding unit 626.

The offset adding unit 626 adds the offset supplied from the offset deriving unit 625 to each pixel of the deblocked decoded image P_DB in the target QAOU. The offset addition unit 626 outputs an image obtained by performing processing on all the QAOUs included in the deblocked decoded image P_DB as an offset filtered decoded image P_OF.

Next, the flow of processing in the adaptive offset filter processing unit 62 will be described with reference to FIG. FIG. 7 is a flowchart showing the flow of processing by the adaptive offset filter processing unit 62.

(Step S101)
First, the QAOU control unit 622 acquires QAOU structure information from the offset information storage unit 621.

(Step S102)
Subsequently, the QAOU control unit 622 starts a first loop using the QAOMU number of the target QAOMU to be processed as a loop variable.

(Step S103)
The QAOU control unit 622 supplies the QAOMU number to the offset type deriving unit 623. Based on the control by the QAOU control unit 622, the offset type deriving unit 623 reads the offset type specified by the QAOMU number supplied from the QAOU control unit 622 from the map memory and the list memory of the offset information storage unit 621. Further, the offset type deriving unit 623 supplies the read offset type to the class classification unit 624.

(Step S104)
Subsequently, the QAOU control unit 622 starts a second loop using the pixel number of each pixel included in the target QAOMU as a loop variable. Here, the pixel number is for identifying pixels included in the target QAOMU from each other. For example, the pixel number is assigned to each pixel included in the target QAOMU in a predetermined scan order. Further, instead of such a pixel number, the x coordinate and y coordinate of each pixel included in the target QAOMU may be used as a loop variable.

(Step S105)
Subsequently, the class classification unit 624 classifies the pixel to be processed into any of a plurality of classes that can be selected in the offset type supplied from the offset type deriving unit 623 based on the control by the QAOU control unit 622. Further, the offset type and the class index indicating the class into which the pixel to be processed is classified are supplied to the offset deriving unit 625.

(Step S106)
Subsequently, the offset deriving unit 625 reads the offset to be added to the pixel to be processed from the offset information storage unit 621 based on the control by the QAOU control unit 622. That is, the offset specified by the offset type and class index supplied from the class classification unit 624 is read.

(Step S107)
Subsequently, based on the control by the QAOU control unit 622, the offset addition unit 626 adds the offset supplied from the offset deriving unit 625 to the pixel value of the processing target pixel of the deblocked decoded image P_DB.

(Step S108)
This step is the end of the second loop.

(Step S109)
This step is the end of the first loop.

In step S103, when the offset type read by the offset type deriving unit 623 is “0” (“sao_type_idx” = 0), the QAOU control unit 622 sets an offset for each pixel of the QAOMU to be processed. The offset adder 626 is controlled not to add.

In step S105, when the target pixel is classified into class 0 (“class_idx” = 0), the offset adding unit 626 is controlled so that no offset is added to the target pixel.

Next, the classification process by the class classification unit 624 will be described with reference to FIGS. In the following, the case where conditions such as the depth of the QAOU hierarchy are used as parameter conditions will be described, but the configuration of the class classification unit corresponding to general parameter conditions will be described later with reference to FIG.

FIGS. 9A and 9B are diagrams for explaining offset processing by the adaptive offset filter 60. FIG. 9A shows pixels referred to when “sao_type_idx” = 1, and FIG. 9B shows when “sao_type_idx” = 2. (C) shows a pixel referred to when “sao_type_idx” = 3, and (d) shows a pixel referred to when “sao_type_idx” = 4.

FIG. 10 is a diagram for explaining the offset processing by the adaptive offset filter 60. FIG. 10A shows the magnitude relationship between the pixel value pix [x] of the processing target pixel x and the pixel value of the pixel a or b. (B) is a graph showing the magnitude relationship between the pixel value of the processing target pixel x and the pixel values of the pixels a and b, and the magnitude relationship thereof. (C) shows the correspondence between each graph shown in (b) and class_idx, and (d) to (f) show conversion tables representing conversion from EgdeType to class_idx. Is shown.

FIG. 11 is a diagram for explaining offset processing by the adaptive offset filter 60. FIG. 11A schematically shows the class classification when “sao_type_idx” = 5, and FIG. 11B shows “sao_type_idx” = FIG. 6 schematically shows the class classification when the number is 6. FIG.

FIG. 12 is a diagram for explaining offset processing by the adaptive offset filter 60. FIG. 12A schematically shows class classification when the depth of the QAOU hierarchy to be processed is smaller than a threshold, and (b) ) Schematically shows the class classification when the depth of the QAOU hierarchy to be processed is greater than or equal to a threshold value.

FIG. 13 is a diagram illustrating an example of class classification when a band offset is specified. FIG. 13A illustrates an example of class classification when the depth of the QAOU hierarchy to be processed is smaller than a threshold. b) shows an example of class classification when the depth of the QAOU to be processed is greater than or equal to a threshold value.

(When the offset type is 1 to 4 (edge offset))
When the offset type supplied from the offset type deriving unit 623 is any one of 1 to 4, the class classification unit 624 determines whether or not an edge exists in the vicinity of the pixel to be processed and if an edge exists. Determines the type of edge, and classifies the pixel to be processed into one of a plurality of classes according to the determination result.

More specifically, first, the class classification unit 624 determines the pixel value pix [x] of the processing target pixel x and the pixel values of two pixels a and b that are adjacent to the processing target pixel or share a vertex. Sign of difference from pix [a] and pix [b] Sign (pix [x] −pix [a]), and Sign (pix [x] −pix [b])
Is calculated. Where Sign (z) is
Sign (z) = + 1 (when z> 0)
Sign (z) = 0 (when z = 0)
Sign (z) =-1 (when z <0)
It is a function that takes each value of. Further, which pixel is used as the pixel a and the pixel b depends specifically on the offset type and is determined as follows.

When offset type = 1 (“sao_type_idx” = 1) As shown in FIG. 9A, a pixel adjacent to the left side of the processing target pixel x is defined as a pixel a, and a pixel adjacent to the right side of the processing target pixel is defined as a pixel Let b.

When offset type = 2 (“sao_type_idx” = 2) As shown in FIG. 9B, the pixel adjacent to the upper side of the processing target pixel x is defined as a pixel a, and the pixel adjacent to the lower side of the processing target pixel is Let it be pixel b.

When offset type = 3 (“sao_type_idx” = 3) As shown in FIG. 9C, the pixel sharing the upper left vertex of the processing target pixel x is defined as a pixel a, and the lower right vertex of the processing target pixel is defined as the pixel a. Let the pixel to share be the pixel b.

When offset type = 4 (“sao_type_idx” = 4) As shown in FIG. 9D, the pixel that shares the lower left vertex of the processing target pixel x is defined as pixel a, and the upper right vertex of the processing target pixel is shared A pixel to be processed is a pixel b.

Therefore, as shown in FIG. 10A, in the magnitude relationship between the pixel value pix [x] and the pixel value of the pixel a or b, the pixel value pix [x] is more than the pixel value of the pixel a or b. Is smaller, Sign (pix [x] −pix [a]) = − 1, the same is Sign (pix [x] −pix [a]) = 0, and the larger is Sign (pix [x] −pix [a]). a]) = 1. In FIG. 10A, a black circle with pix [x] indicates a pixel value of the processing target pixel x, and a black circle without pix [x] indicates the processing target pixel a or The pixel value of b is shown. Further, the vertical direction in FIG. 10A indicates the magnitude of the pixel value.

Subsequently, the class classification unit 624 calculates EgdeType by the following formula (1-1) based on Sign (pix [x] −pix [a]) and Sign (pix [x] −pix [b]). To derive.

EgdeType = Sign (pix [x] −pix [a]) + Sign (pix [x] −pix [b]) + 2 (1-1)
As a result, as shown in FIG. 10B, when both the pixel values of the pixels a and b are larger than the pixel value pix [x], EgdeType = 0. Further, if one of the pixels a and b is larger than the pixel value pix [x] and the other pixel value is the same, EgdeType = 1. Further, when one of the pixels a and b is smaller than the pixel value pix [x] and the other pixel value is the same, EgdeType = 3.
Further, when both the pixel values of the pixels a and b are smaller than the pixel value pix [x], EgdeType = 4. Further, when one of the pixels a and b is small and the other pixel value is large with respect to the pixel value pix [x], or any of the pixels a and b with respect to the pixel value pix [x]. If the pixel values of are also the same, EgdeType = 2.

In FIG. 10B, the black circle at the center of each graph indicates the pixel value of the processing target pixel x, and the black circles at both ends indicate the pixel values of the pixels a and b. Further, the vertical direction in FIG. 10B indicates the magnitude of the pixel value.

Subsequently, the class classification unit 624 derives the class index (class_idx) of the class to which the processing target pixel x should belong based on the derived EgdeType as follows.

class_idx = EoTbl [EdgeType]
Here, EoTbl [EdgeType] is a conversion table used to derive class_idx from EdgeType. Specific examples of the conversion table EoTbl are shown in FIGS.

The conversion table 1001X shown in FIG. 10 (d) is a conversion table when the conversion table used is not changed according to the depth of the QAOU hierarchy to be processed.

Also, the conversion table 1001A shown in FIG. 10 (e) and the conversion table 1001B shown in FIG. 10 (f) are conversion tables for changing the conversion table used according to the depth of the QAOU to be processed. When the depth of the QAOU hierarchy to be processed is smaller than the threshold, the conversion table 1001A is used, and when the depth of the QAOU hierarchy to be processed is equal to or larger than the threshold, the conversion table 1001B is used.

When the conversion table to be used is not changed depending on the depth of the QAOU hierarchy to be processed, as shown in the conversion table 1001X, the class classification unit 624 does not have an edge in the region including the processing target pixel x, the pixel a, and the pixel b. (Hereinafter also referred to as a flat case), that is, when EdgeType = 2, the processing target pixel x is classified into class 0 (“class_idx” = 0). Also, EdgeType = 0, 1, 3, 4 are classified into class_idx = 1, 2, 3, 4 respectively.

In addition, when the conversion table to be used is not changed depending on the depth of the QAOU hierarchy to be processed, and when the depth of the QAOU hierarchy to be processed is smaller than the threshold, the class classification unit 624 performs processing as shown in the conversion table 1001A. When an edge does not exist in the area composed of the pixel x, the pixel a, and the pixel b (hereinafter also referred to as a flat case), that is, when EdgeType = 2, the processing target pixel x is set to class 0 (“class_idx” = 0). Classify. Further, EdgeType = 0, 1, 3, and 4 are classified into class_idx = 1, 3, 4, and 2, respectively. In addition, when the depth of the QAOU hierarchy to be processed is equal to or greater than the threshold, the class classification unit 624 does not have an edge in the region including the processing target pixel x, the pixel a, and the pixel b as shown in the conversion table 1001B ( Hereinafter, it is also referred to as a flat case), that is, when EdgeType = 2, the processing target pixel x is classified into class 0 (“class_idx” = 0). Also, EdgeType = 0 and 1 are classified as class_idx = 1, and EdgeType = 3 and 4 are classified as class_idx = 2. Therefore, in the conversion table 1001B, one class_idx corresponds to a plurality of EdgeTypes.

Note that the conversion table 1001A may be the same as the conversion table 1001X. However, if the conversion table 1001A is the same as the conversion table 1001X, a different class_idx is assigned to the same EdgeType with the conversion S table 1001B, and processing changes depending on the hierarchy. For example, when EdgeType = 3, if the hierarchy depth is smaller than the threshold, class_idx = 4, and if the hierarchy depth is greater than or equal to the threshold, class_idx = 2, the process changes depending on the hierarchy.

Therefore, by making the conversion table 1001A as shown in FIG. 10E, the processing is not changed between the conversion table 1001B, that is, between hierarchies.

(When the offset type is 5 to 6 (band offset))
When the offset type supplied from the offset type deriving unit 623 is 5 or 6, the class classification unit 624 sets a plurality of pixel values of the processing target pixel according to the pixel value pix [x] of the processing target pixel x. Classify one of the classes.

When the offset type = 5 (sao_type_idx = 5) The class classification unit 624 determines that the pixel value pix [x] of the processing target pixel x is
(Max × 1/4) ≦ pix [x] ≦ (max × 3/4)
Is satisfied, the processing target pixel is classified into a class other than class 0. That is, when the pixel value of the processing target pixel is within the hatched range in FIG. 11A, the processing target pixel is classified into a class other than class 0. The above max represents the maximum value that the pixel value of the processing target pixel x can take, for example, max = 255. When max = 255, the above condition can be expressed as 8 ≦ (pix [x] / 8) ≦ 23, or expressed as 4 ≦ (pix [x] / 16) ≦ 11. You can also.

When the offset type = 6 (sao_type_idx = 6) The class classification unit 624 determines that the pixel value pix [x] of the processing target pixel x is
pix [x] ≦ (max × 1/4) or (max × 3/4) ≦ pix [x]
Is satisfied, the processing target pixel is classified into a class other than class 0. That is, when the pixel value of the processing target pixel is within the hatched range in FIG. 11B, the processing target pixel is classified into a class other than class 0. The above max represents the maximum value that the pixel value of the processing target pixel x can take, for example, max = 255. Further, when max = 255, the above condition can be expressed as (pix [x] / 8) ≦ 7 or 24 ≦ (pix [x] / 8), or (pix [x] / 16 ) ≦ 3 or 12 ≦ (pix [x] / 16).

The class classification process by the class classification unit 624 will be described in more detail as follows.

When the offset type is any one of 5 to 6, the class classification unit 624 sets the class index (class_idx) of the class to which the processing target pixel x should belong according to the depth of the processing target QAOU as follows. To derive.
When the depth of the QAOU hierarchy to be processed is smaller than the threshold: class_idx = EoTbl [sao_type_idx] [pix [x] / >> BoRefBit32]
When the depth of the QAOU hierarchy to be processed is greater than or equal to the threshold value: class_idx = EoTbl [sao_type_idx] [pix [x] / >> BoRefBit16]
Here, BoTbl [sao_type_idx] [pix [x] / >> BoRefBit32] and BoTbl [sao_type_idx] [pix [x] / >> BoRefBit16] are derived from the pixel values pix [x] and sao_type_id of the processing target pixel x. , Class_idx is a conversion table used for deriving. BoRefBit32 and BoRefBit16 are values derived by PIC_DEPTH-5 and PIC_DEPTH-4, respectively, when the image bit depth is set to PIC_DEPTH, and the pixel values are quantized to 32 or 16 values. The quantized pixel value is also described as pixquant. Shifting right by BoRefBit32 and BoRefBit16 corresponds to dividing by 1 << BoRefBit32 and 1 << BoRefBit16. This quantization width is 1 << (8-5) = 8 and 1 << (8-4) = 16, respectively, when the bit depth of the pixel value is 8 bits. Hereinafter, a case where the quantization width is 8 will be described. This quantization width is called a class width.

The class classification unit 624 performs class classification by changing the width of the class to be classified according to the depth of the QAOU hierarchy to be processed. For example, as shown in FIG. 12A, when the depth of the QAOU hierarchy to be processed is smaller than the threshold, the class classification unit 624 classifies the pixel width into 32 with the class width set to “8”. , Classify. Also, as shown in FIG. 12B, when the depth of the QAOU hierarchy to be processed is equal to or greater than the threshold, the class classification unit 624 classifies the pixel value into 16 with a class width of “16”, and class Perform classification.

Next, a specific example of the conversion table EoTbl is shown in FIG. In each of FIGS. 13A and 13B, “BO_1” indicates that “sao_type_index” = 5, and “BO_2” indicates that “sao_type_index” = 6. Also, the conversion table 1301 shown in FIG. 13A is a conversion table used when the depth of the QAOU hierarchy to be processed is smaller than the threshold, and the conversion table 1302 shown in FIG. This is a conversion table used when the depth of the QAOU hierarchy is greater than or equal to a threshold value.

In the case where the depth of the QAOU hierarchy to be processed is smaller than the threshold value and “sao_type_index” = 5, as shown in FIG. 13A, the eyelid class classification unit 624 has a pixel value pix [x] of 8 ≦ ( The processing target pixel x satisfying pix [x] / 8) ≦ 23 is classified into any one of class indices 1 to 16 according to the size of pix [x].

When “sao_type_index” = 6, the class classification unit 624 determines the pixel x to be processed that satisfies the pixel value pix [x] that satisfies pix [x] / 8) ≦ 7 or 24 ≦ (pix [x] / 8). , Pix [x] is classified into any one of class indexes 1 to 16 according to the size of pix [x].

Also, when the depth of the QAOU hierarchy to be processed is equal to or greater than the threshold value and “sao_type_index” = 5, as shown in FIG. 13B, the eyelid class classification unit 624 has a pixel value pix [x] of 4 ≦ A pixel x to be processed that satisfies (pix [x] / 16) ≦ 11 is classified into one of class indexes 1 to 8 according to the size of pix [x].

In addition, when “sao_type_index” = 6, the class classification unit 624 determines the pixel x to be processed that satisfies the pixel value pix [x] pix [x] / 16) ≦ 3 or 12 ≦ (pix [x] / 16). , Pix [x] is classified into any one of class indexes 1 to 8 according to the size of pix [x].
(Configuration of Offset Information Decoding Unit 611)
FIG. 32 shows an offset information decoding unit that changes an offset type to be used according to the type (parameter condition) of an adaptive offset parameter and / or changes the number of class classifications according to the parameter condition of the adaptive offset. FIG. 611 is a block diagram of FIG. The offset information decoding unit 611 includes an adaptive offset type decoding unit 6111, a used offset type selection unit 6112, an offset type decoding unit 6113, an adaptive offset decoding unit 6114, a used offset number selection unit 6115, and an offset decoding unit 6116. . The parameter condition is a parameter other than a value calculated from the pixel value. In addition to the hierarchy and offset type described above, the block size (QAOU size) and color components (components) described in the appendix described later are included. ), QP.

The adaptive offset type decoding unit 6111 is means for adaptively decoding an offset type according to parameter conditions from QAOU information in the encoded data, and includes a use offset type selection unit 6112 and an offset type decoding unit 6113.

The used offset type selection unit 6112 is means for selecting an offset type to be used according to the parameter condition. One of the parameter conditions is a hierarchy as described above. The number of offset types used when the hierarchy is shallow is smaller than the number of offsets used when the hierarchy is deep. The used offset type selection unit 6112 inputs the conversion table as described in FIG. 8 to the offset type decoding unit 6113 according to the parameter condition. Further, the maximum number of usable offset types is input to the offset type decoding unit 6113.

The offset type decoding unit 6113 decodes the offset type from the encoded data according to the input conversion table and the maximum number. Assuming that the maximum number of offset types is N, the range of codes that can be taken is limited to N from 0 to N−1. Therefore, the number of bits required for code encoding can be reduced. For example, ^m- bit fixed length coding can be used when the maximum number is greater than 2 ^{m−1 and} less than or equal to 2 ^m . Further, Truncated unary encoding with a maximum value of N-1 or Truncated Rice encoding can be used.

The adaptive offset decoding unit 6114 is means for adaptively decoding an offset from the QAOU information in the encoded data according to the parameter condition, and includes a used offset number selection unit 6115 and an offset decoding unit 6116. The used offset number selection unit 6115 is means for selecting the maximum value of the number of offsets to be used and the offset accuracy in accordance with the parameter conditions. One of the parameter conditions is a hierarchy, and the number of offsets used when the hierarchy is shallow is larger than the number of offsets used when the hierarchy is deep. For example, when the hierarchy is shallow, the number of offsets can be 16 as shown in FIG. 13A, and when the hierarchy is deep, the number of offsets can be 8 as shown in FIG. 13B. Further, as described above, the offset accuracy (bit depth) can be changed according to the hierarchy. The used offset number selection unit 6115 inputs the maximum offset number and the offset accuracy to the offset decoding unit 6116. The offset decoding unit 6116 decodes the offset according to the maximum number of offsets and the offset accuracy. When the number of offsets is reduced, the code amount of the offset is reduced. Similarly to the offset type, if the accuracy of each offset is determined, the range that can be taken by the code that encodes the offset is also limited, so that the number of bits required for encoding the code can be reduced.

FIG. 33A is a block diagram showing the configuration of the used offset type selection unit 6112.

The used offset type selection unit 6112 includes an offset type conversion table selection unit 6117, a first offset type conversion table storage unit 6118, and a second offset type conversion table storage unit 6119.

The offset type conversion table selection unit 6117 selects the conversion table provided in the first offset type conversion table storage unit 6118 or the conversion table provided in the second offset type conversion table storage unit 6119 according to the parameter condition. In the example described above, 801A corresponds to the conversion table included in the first offset type conversion table storage unit 6118, and 801B corresponds to the conversion table included in the second offset type conversion table storage unit 6119.

FIG. 33 (b) is a block diagram showing another configuration of the used offset type selection unit.

The used offset type selection unit 6112 ′ includes an offset type conversion table selection unit 6117 ′, an edge offset type and band offset type conversion table storage unit 6118 ′, and a horizontal edge offset type and band offset type conversion table storage unit 6119 ′.

(Configuration of class classification unit 624)
FIG. 34 is a block diagram illustrating a configuration of the class classification unit 624.

The class classification unit 624 includes an adaptive edge offset class classification unit 6241, a used edge offset class selection unit 6242, an edge offset class classification unit 6243, an adaptive band offset class classification unit 6244, a used band offset class / class width selection unit 6245, The band offset class classification unit 6246 is configured.

The class classification unit 624 classifies each pixel into a class according to the parameter condition and the offset type. If the offset type indicates an edge offset, the adaptive edge offset class classifying unit 6241 classifies the pixel. If the offset type is a band offset, the adaptive band offset class classifying unit 6244 classifies the pixel.

The adaptive edge offset class classification unit 6241 is a unit that adaptively classifies pixels into classes according to parameter conditions, and includes a used edge offset class selection unit 6242 and an edge offset class classification unit 6243. The used edge offset class selection unit 6242 selects the type of class to be used. The used edge offset class selection unit 6242 inputs the pixel value classification method to the edge offset class classification unit 6243. Specifically, a method of deriving an intermediate value EdgeType that is temporarily derived when classifying pixel values and a conversion table used to derive class_idx from EdgeType are input. An example of a method for deriving the intermediate value EdgeType has been described with reference to FIGS. 10A and 10B, and this is called a basic edge classification method. An edge classification method as will be described later with reference to FIG. 22 can also be used. One of the parameter conditions is the hierarchy as described above, and the class used when the hierarchy is shallow is selected so as to be larger than the class used when the hierarchy is deep. 1001A and 1001B are examples of conversion tables used to derive class_idx from EdgeType. A method of switching the edge derivation method according to the color component (component) is also appropriate. In this case, for luminance, a basic edge classification method using horizontal, vertical, and diagonal edge classification methods is used, and for color difference, the range of reference pixels used for edge classification to reduce line memory is horizontal when viewed from the target pixel. It is appropriate to use a horizontal edge classification method limited to the direction. The edge offset class classification unit 6243 classifies pixels based on a given classification method and a conversion table used to derive class_idx from EdgeType.

The adaptive band offset class classification unit 6244 is means for adaptively classifying pixels into classes according to parameter conditions, and includes a used band offset class / class width selection unit 6245 and a band offset class classification unit 6246. The used band offset class / class width selection unit 6245 inputs the pixel value classification method to the band offset class classification unit 6246. Specifically, a class width, which is a quantization width used when classifying pixel values into intermediate values, and a conversion table used to derive class_idx from the intermediate values are input. One of the parameter conditions is the hierarchy as described above, and the width of the class used when the hierarchy is shallow is smaller than the width of the class used when the hierarchy is deep. The band offset class classification unit 6246 classifies the pixel values into classes according to the input class width and the conversion table used to derive class_idx from the intermediate value. The class width as input may be an integer corresponding to the class width, not the class width itself. For example, when 1 << BoRefBit32 is used as the class width, the number of bits used to quantize the pixel that is the logarithm of base 2 is BoRefBit32, and a value for obtaining the number of bits used for quantizing the pixel from the bit depth of the pixel For example, if BoRefBit32 = PIC_DEPTH-5, 5 may be used instead of the class width.

In this way, the coded data is adaptively decoded according to the parameter conditions, and the pixel classification is performed.

As described above, in this embodiment, the number of class classifications is changed according to the depth of the QAOU hierarchy to be processed. More specifically, when the depth of the QAOU to be processed is large, it is classified into a smaller number of classes compared to the case where the depth is small.

Generally, the required amount of memory is memory amount = offset data length × number of classes × number of QAOUs. Therefore, the memory usage can be reduced by reducing the number of classes.

Also, in the deep hierarchy, the area of the QAOU becomes smaller and the characteristics of the pixel values in the QAOU become nearly uniform, so the effect of offset does not change much even if the number of classes is reduced.

In this embodiment, the accuracy of the offset is changed according to the depth of the QAOU hierarchy to be processed. More specifically, when the depth of the QAOU to be processed is large, the offset accuracy is lowered as compared with the case where the depth is small. This reduces the amount of code when the depth of the QAOU hierarchy to be processed is large.

In a deep hierarchy QAOU, since the number of pixels included is small, the quantization error seen in the entire QAOU is smaller than that of the upper hierarchy. Therefore, even if the offset accuracy is lowered in a deep hierarchy, the effect of the offset does not change much.

(Moving picture encoding device 2)
Next, a moving picture encoding apparatus 2 that generates encoded data # 1 by encoding an encoding target image will be described with reference to FIGS. The moving image encoding apparatus 2 includes H.264 as a part thereof. H.264 / MPEG-4. A method adopted in AVC, a method adopted in KTA software, which is a codec for joint development in VCEG (Video Coding Expert Group), and a method adopted in TMuC (Test Model under Consideration) software, which is the successor codec And the technology employed in HM (HEVC TestModel) software.

FIG. 14 is a block diagram showing a configuration of the video encoding device 2 according to the present embodiment. As illustrated in FIG. 14, the moving image encoding device 2 includes a transform / quantization unit 21, a variable-length code encoding unit 22, an inverse quantization / inverse transform unit 23, a buffer memory 24, an intra-predicted image generation unit 25, The inter prediction image generation unit 26, motion vector detection unit 27, prediction method control unit 28, motion vector redundancy deletion unit 29, adder 31, subtractor 32, deblocking filter 33, adaptive filter 70, and adaptive offset filter 80 are provided. It is the composition which includes. The moving image encoding device 2 is a device that generates encoded data # 1 by encoding moving image # 10 (encoding target image).

The transform / quantization unit 21 performs (1) DCT transform (Discrete Cosine Transform) on the prediction residual D obtained by subtracting the predicted image Pred from the encoding target image, and (2) DCT coefficients obtained by the DCT transform. (3) The quantized prediction residual QD obtained by the quantization is supplied to the variable-length code encoding unit 22 and the inverse quantization / inverse transform unit 23. The transform / quantization unit 21 selects (1) a quantization step QP used for quantization for each tree block, and (2) a quantization parameter difference indicating the size of the selected quantization step QP. Δqp is supplied to the variable length code encoding unit 22, and (3) the selected quantization step QP is supplied to the inverse quantization / inverse conversion unit 23. Here, the quantization parameter difference Δqp is the quantization parameter related to the tree block DCT transformed / quantized immediately before from the value of the quantization parameter qp (QP = 2 ^{pq / 6} ) related to the tree block to be DCT transformed / quantized. It refers to the difference value obtained by subtracting the value of qp ′.

The variable length code encoding unit 22 includes (1) a quantized prediction residual QD and Δqp supplied from the transform / quantization unit 21, and (2) a quantization parameter PP supplied from a prediction scheme control unit 28 described later, (3) The encoded data # 1 is generated by variable-length encoding the filter set number, filter coefficient group, region designation information, and on / off information supplied from the adaptive filter 70 described later. Further, the variable length code encoding unit 22 encodes the QAOU information supplied from the adaptive offset filter 80 and includes it in the encoded data # 1.

The inverse quantization / inverse transform unit 23 (1) inversely quantizes the quantized prediction residual QD, (2) performs inverse DCT (Discrete Cosine Transform) transformation on the DCT coefficient obtained by the inverse quantization, and (3) The prediction residual D obtained by the inverse DCT transform is supplied to the adder 31. When the quantization prediction residual QD is inversely quantized, the quantization step QP supplied from the transform / quantization unit 21 is used. Note that the prediction residual D output from the inverse quantization / inverse transform unit 23 is obtained by adding a quantization error to the prediction residual D input to the transform / quantization unit 21. Common names are used for

The intra predicted image generation unit 25 generates a predicted image Pred_Intra related to each partition. Specifically, (1) a prediction mode used for intra prediction is selected for each partition, and (2) a prediction image Pred_Intra is generated from the decoded image P using the selected prediction mode. The intra predicted image generation unit 25 supplies the generated intra predicted image Pred_Intra to the prediction method control unit 28.

Further, the intra predicted image generation unit 25 specifies the prediction index PI for each partition from the prediction mode selected for each partition and the size of each partition, and supplies the prediction index PI to the prediction method control unit 28. .

In addition, the intra predicted image generation unit 25 supplies the adaptive filter 70 with intra coding mode information IEM that is information indicating the size of the target partition and the prediction mode assigned to the target partition.

The motion vector detection unit 27 detects a motion vector mv related to each partition. Specifically, (1) the filtered decoded image P_FL ′ to be used as a reference image is selected, and (2) the target partition is searched by searching for the region that best approximates the target partition in the selected filtered decoded image P_FL ′. Detects a motion vector mv. Here, the filtered decoded image P_FL ′ is obtained by performing the deblocking process by the deblocking filter 33, the adaptive offset process by the adaptive offset filter 80, and the adaptive filter 70 on the decoded image that has already been decoded. It is an image obtained by performing adaptive filter processing. The motion vector detection unit 27 can read out the pixel value of each pixel constituting the filtered decoded image P_FL ′ from the buffer memory 24. The motion vector detection unit 27 supplies the detected motion vector mv to the inter prediction image generation unit 26 and the motion vector redundancy deletion unit 29 together with the reference image index RI that specifies the filtered decoded image P_FL ′ used as the reference image. To do. Note that for a partition that performs bi-directional prediction (weighted prediction), two filtered decoded images P_FL1 ′ and P_FL2 ′ are selected as reference images, and each of the two filtered decoded images P_FL1 ′ and P_FL2 ′ is selected. Corresponding motion vectors mv1 and mv2 and reference image indexes RI1 and RI2 are supplied to the inter prediction image generation unit 26 and the motion vector redundancy deletion unit 29.

The inter prediction image generation unit 26 generates a motion compensation image mc related to each inter prediction partition. Specifically, using the motion vector mv supplied from the motion vector detection unit 27, the motion compensated image mc is obtained from the filtered decoded image P_FL ′ designated by the reference image index RI supplied from the motion vector detection unit 27. Generate. Similar to the motion vector detection unit 27, the inter predicted image generation unit 26 can read out the pixel value of each pixel constituting the filtered decoded image P_FL ′ from the buffer memory 24. The inter prediction image generation unit 26 supplies the generated motion compensated image mc (inter prediction image Pred_Inter) together with the reference image index RI supplied from the motion vector detection unit 27 to the prediction method control unit 28. For the partition with bi-directional prediction (weighted prediction), (1) the motion compensated image mc1 is generated from the filtered decoded image P_FL1 ′ specified by the reference image index RI1 using the motion vector mv1, and (2 ) A motion compensated image mc2 is generated from the filtered reference image P_FL2 ′ designated by the reference image index RI2 using the motion vector mv2, and (3) an offset value is added to the weighted average of the motion compensated image mc1 and the motion compensated image mc2. Is added to generate the inter predicted image Pred_Inter.

The prediction scheme control unit 28 compares the intra predicted image Pred_Intra and the inter predicted image Pred_Inter with the encoding target image, and selects whether to perform intra prediction or inter prediction. When the intra prediction is selected, the prediction scheme control unit 28 supplies the intra prediction image Pred_Intra as the prediction image Pred to the adder 31 and the subtractor 32 and also predicts the prediction index PI supplied from the intra prediction image generation unit 25. The parameter PP is supplied to the variable length code encoder 22. On the other hand, when the inter prediction is selected, the prediction scheme control unit 28 supplies the inter prediction image Pred_Inter as the prediction image Pred to the adder 31 and the subtractor 32, and the reference image index supplied from the inter prediction image generation unit 26. RI and an estimated motion vector index PMVI and a motion vector residual MVD supplied from a motion vector redundancy deleting unit 29 (described later) are supplied to the variable length code encoding unit 22 as prediction parameters PP.

The prediction residual D is generated by the subtractor 32 by subtracting the prediction image Pred selected by the prediction method control unit 28 from the encoding target image. The prediction residual D generated by the subtractor 32 is DCT transformed / quantized by the transform / quantization unit 21 as described above. On the other hand, by adding the prediction image Pred selected by the prediction method control unit 28 to the prediction residual D generated by the inverse quantization / inverse transformation unit 23, the adder 31 generates a local decoded image P. Generated. The local decoded image P generated by the adder 31 passes through the deblocking filter 33, the adaptive offset filter 80, and the adaptive filter 70, and is then stored in the buffer memory 24 as a filtered decoded image P_FL. Used as

The motion vector redundancy deleting unit 29 deletes the redundancy in the motion vector mv detected by the motion vector detecting unit 27. Specifically, (1) an estimation method used for estimating the motion vector mv is selected, (2) an estimated motion vector pmv is derived according to the selected estimation method, and (3) the estimated motion vector pmv is subtracted from the motion vector mv. As a result, a motion vector residual MVD is generated. The motion vector redundancy deletion unit 29 supplies the generated motion vector residual MVD to the prediction method control unit 28 together with the estimated motion vector index PMVI indicating the selected estimation method.

The deblocking filter 33 determines whether the block boundary in the decoded image P or the block boundary in the decoded image P or the block boundary in the decoded image P or the block boundary in the decoded image P is smaller than a predetermined threshold value. By performing deblocking processing on the CU boundary, an image near the block boundary or the vicinity of the CU boundary is smoothed. The image subjected to the deblocking process by the deblocking filter 33 is output to the adaptive offset filter 80 as a deblocked decoded image P_DB.

The adaptive offset filter 80 generates an offset filtered decoded image P_OF by performing an adaptive offset filter process on the deblocked decoded image P_DB supplied from the deblocking filter 33. The generated offset filtered decoded image P_OF is supplied to the adaptive filter 70. Since a specific configuration of the adaptive offset filter 80 will be described later, description thereof is omitted here.

The adaptive filter 70 generates a filtered decoded image P_FL by performing an adaptive filter process on the offset filtered decoded image P_OF supplied from the adaptive offset filter 80. The filtered decoded image P_FL that has been filtered by the adaptive filter 70 is stored in the buffer memory 24. The filter coefficient used by the adaptive filter 70 is determined so that the error between the filtered decoded image P_FL and the encoding target image # 10 becomes smaller, and the filter coefficient thus determined is the filter parameter. It is encoded as FP and transmitted to the video decoding device 1.

(Adaptive offset filter 80)
Next, the adaptive offset filter 80 will be described with reference to FIG. FIG. 15 is a block diagram showing a configuration of the adaptive offset filter 80. As shown in FIG. 15, the adaptive offset filter 80 includes an adaptive offset filter information setting unit 81 and an adaptive offset filter processing unit 82.

As shown in FIG. 15, the adaptive offset filter information setting unit 81 includes an offset calculation unit 811, an offset shift unit 816, an offset clip unit 812, an offset information selection unit 813, and an offset attribute setting unit 815. .

(Offset calculation unit 811)
For all QAOUs up to a predetermined division depth included in the target processing unit (for example, LCU), the offset calculation unit 811 performs all offset types and all classes that exist according to the hierarchy of the processing target QAOU. The offset of is calculated. Here, the offset type and class are the same as those described in the description of the video decoding device 1.

Next, the flow of processing in the offset calculation unit 811 will be described with reference to FIG. FIG. 16 is a flowchart showing the flow of processing by the offset calculation unit 811.

(Step S201)
First, the offset calculation unit 811 starts a first loop using the QAOMU number of the target QAOMU to be processed as a loop variable. For example, as shown in FIG. 18, the QAOMU number is a number assigned to each QAOU so that they can be distinguished from each other. The QAOMU number is assigned from the division depth 0 to the maximum division depth. When the maximum division depth is 4, as shown in FIG. 18, values of 0 to 340 are assigned as QAOMU numbers for all QAOMUs (1 + 4 + 16 + 64 + 256 = 341) of division depths.

For example, in FIG. 18, (a) shows a QAOMU number assigned to a QAOMU having a division depth of 0, and (b) shows a QAOMU number assigned to a QAOMU having a division depth of 1. (C) shows a QAOMU number assigned to a QAOMU having a division depth of 2, and (d) shows a QAOMU number assigned to a QAOMU having a division depth of 3. In (e), a QAOMU number assigned to a QAOMU having a division depth of 4 is shown.

Therefore, in the case of the example shown in FIGS. 18A to 18E, the first loop is a loop from QAOMU number = 0 to QAOMU number = 340.

(Step S202)
Subsequently, the offset calculation unit 811 starts a second loop using the offset type that can be selected for the target QAOMU as a loop variable. The second loop is a loop from offset type 1 to offset type 6.

(Step S203)
Subsequently, the offset calculation unit 811 starts a third loop using the pixels included in the target QAOMU as a unit.

(Step S204)
Subsequently, the offset calculation unit 811 classifies the target pixel into one of a plurality of classes. More specifically, the QAOMU to be processed is classified into one of classes that can be selected according to the hierarchy. For example, when the depth of the QAOMU hierarchy to be processed is smaller than the threshold value and the offset type as the second loop variable is 1 to 4, the target pixel is classified into one of classes 1 to 4. The classification process in this step is the same as the classification process performed by the class classification unit 624 included in the adaptive offset filter 60 in the video decoding device 1.

Also, with respect to the target QAOMU, a classification count “count [part_idx] [sao_type_index] [class_idx]”, which is the number of times the pixel has been classified, is calculated for each class. Here, “part_idx” represents a QAOU index.

(Step S205)
Subsequently, the offset calculation unit 811 obtains the difference pixel value in the target pixel by taking the difference between the pixel value of the deblocked decoded image P_DB in the target pixel and the pixel value of the encoding target image # 10 in the target pixel. calculate. More specifically, P_DB (x, y) −Org (x, y) is calculated when the position of the target pixel is (x, y). Here, P_DB (x, y) represents the pixel value of the deblocked decoded image P_DB in the target pixel, and Org (x, y) represents the pixel value of the encoding target image # 10 in the target pixel. ing.

(Step S206)
This step is the end of the third loop. When this step is completed, the difference pixel values are calculated for all the pixels included in the target QAOU.

(Step S207)
Subsequently, the offset calculation unit 811 calculates an offset by dividing the sum of the difference pixel values for each pixel included in the target QAOMU for each class by the number of classifications of the class. More specifically, the offset calculation unit 811 calculates the offset “offset [part_idx] [sao_type_idx] [class_idx]” for the target QAOMU, the target offset type, and the target class using the following expression.

offset [part_idx] [sao_type_idx] [class_idx] = Σ (P_DB (x, y) -Org (x, y)) / count [part_idx] [sao_type_idx] [class_idx]
Here, the symbol Σ is the sum for the pixels classified into the target class specified by “class_idx” in the target QAOMU specified by “part_idx” and the target offset type specified by “sao_type_idx”. Is shown.

(Step S208)
This step is the end of the second loop.

(Step S209)
This step is the end of the third loop.

Through the above processing, the offset calculation unit 811 calculates offsets for all offset types and all classes for all QAOMUs up to a predetermined division depth included in the target LCU. For example, in the case of the example shown in FIGS. 18A to 18E, the offset calculation unit 811 sets the threshold value of the depth of the QAOMU layer to be processed to “3”, that is, the layers 0 to 2. If the number of classes is changed at levels 3 and 4,
((Total number of QAOMUs with division depth 0) + (total number of QAOMUs with division depth 1) + (total number of QAOMUs with division depth 2)) × (number of EO offset types) × (division depth 0 to 2 EO class number) + (BO offset type number) × (BO class number at division depth 0-2)) + (Total number of QAOMUs at division depth 3) + (Total number of QAOMUs at division depth 4) ) × ((number of EO offset types) × (number of EO classes at division depths 3 and 4) + (number of BO offset types) × (number of BO classes at division depths 3 and 4)) = (1 + 4 + 16) × ((4 × 4) + (2 × 16)) + (64 + 256) × ((4 × 2) + (2 × 8)) = 8688 (pieces)
Is calculated.

In the case where the number of classes to be classified is not changed according to the depth of the QAOMU hierarchy to be processed,
((Total number of QAOMUs with division depth 0) + ... + (total number of QAOMUs with division depth 4)) × ((number of EO offset types) × (number of EO classes) + (number of offset types of BO) × (Number of BO classes)) = (1 + 4 + 16 + 64 + 256) × ((4 × 4) + (2 × 16)) = 16368 (pieces)
Is calculated.

The offset calculation unit 811 supplies offset information including the offset, offset type, class, and QAOU structure information representing the QAOU division structure calculated by the above processing to the offset clip unit 812.

(Offset clip part 812)
The offset clip unit 812 performs clip processing on the offset supplied from the offset calculation unit 811 by any one of clip processing 1 and clip processing 2 as described below.

(Clip processing 1)
The offset clip unit 812 clips each offset supplied from the offset calculation unit 811 to a value from −8 to 7, for example, to express each offset with 4 bits. Each clipped offset is supplied to the offset information selection unit 813. The bit width to be clipped is set according to the bit depth of the image and the bit depth of the offset, as in the video decoding device 1.

Thus, by clipping each offset, the memory size of a memory (not shown) in which each offset is stored can be reduced. Moreover, since the amount of offset codes included in the encoded data # 1 can be reduced, the encoding efficiency can be improved. Moreover, since an excessive offset is suppressed, an appropriate image quality is ensured.

(Clip processing 2)
The offset clip unit 812 may be configured to set the clip range of each offset supplied from the offset calculation unit 811 to a different value depending on the offset type.

For example, when the offset type is an edge offset, the number of offset bits is 8 bits, and when the offset type is a band offset, the number of offset bits is 4 bits. More generally, N> M is satisfied when the number of offset bits when the offset type is edge offset is N bits and the number of offset bits when the offset type is band offset is M bits. The number of offset bits is determined as follows.

Thus, by making the number of offset bits different according to the offset type, it is possible to improve the encoding efficiency without requiring an excessive memory size for the memory for storing each offset.

When the threshold th for limiting the value that can be taken by the offset is greater than 2 ^{m-1 and} less than or equal to 2 ^m , m-bit fixed-length encoding is used as an encoding method for encoding the offset. Can be used. More specifically, Truncated unary encoding with the maximum value and th, or Truncated Rice encoding can be used.

Further, clip processing obtained by combining the above clip processing 1 and 2 is also included in this embodiment. Further, the adaptive offset filter 80 may be configured not to include the offset clip unit 812.

Also, the offset clip unit 812 switches the clip range according to the depth of the QAOU hierarchy to be processed in accordance with the offset accuracy. Specifically, it is assumed that the accuracy of the offset is n _A bits when the depth of the QAOU hierarchy to be processed is smaller than the threshold, and n _B bits when the depth of the QAOU hierarchy to be processed is equal to or greater than the threshold. At this time, if the depth of the QAOU layer to be processed is smaller than the threshold, the offset clip unit 812 sets the clip range to −2 ^{nA / 2} to 2 ^{nA / 2} −1. If the depth of the QAOU layer to be processed is equal to or greater than the threshold, the clip range is set to −2 ^{nA / 2} to 2 ^{nA / 2} −1, and the lower (n _A −n _B ) bits are set to “0”. " Thereby, the offset clip unit 812 can cause the offset information selection unit 813 that performs processing using the clipped offset to perform processing without considering the offset accuracy.

(Offset information selection unit 813)
The offset information selection unit 813 determines an offset type, a class, a combination of offsets, and a corresponding QAOU partition structure with a smaller RD cost (Rate-Distortion cost), and determines the determined offset type, class, offset, and QAOU information indicating the corresponding QAOM division structure is supplied to the variable-length code encoding unit 22. Also, the offset information selection unit 813 supplies the determined offset to the adaptive offset filter processing unit 82 for each QAOU.

The processing of the offset information selection unit 813 will be described in more detail with reference to FIGS. FIG. 17 is a flowchart showing the flow of processing by the offset information selection unit 813.

(Step S301)
First, the offset information selection unit 813 starts a first loop using the QAOMU number of the target QAOMU to be processed as a loop variable.

(Step S302)
Subsequently, the offset information selection unit 813 starts a second loop in which an offset type that can be selected for the target QAOMU is a loop variable. The second loop is a loop from offset type 1 to offset type 6.

(Step S303)
Subsequently, the offset information selection unit 813 calculates the square error between the offset filtered decoded image P_OF and the encoding target image # 10 in the target QAOMU for the target offset type.

(Step S304)
This step is the end of the second loop.

(Step S305)
This step is the end of the first loop. At the end of the first and second loops, square errors for all offset types are calculated for each QAOMU.

(Step S306)
Subsequently, the offset information selection unit 813 determines a QAOU partition structure in which the RD cost is smaller among the QAOU partition structures that divide a target processing unit (for example, LCU) into QAOUs.

A specific processing example of the offset information selection unit 813 in this step will be described with reference to FIGS.

FIG. 19 is a diagram showing an outline of calculating a square error for each offset type for a QAOU with a QAOU index “x”. As illustrated in FIG. 19, the offset information selection unit 813 calculates a square error for each offset type for all QAOMUs. The offset type that minimizes the calculated square error is set as the offset type of the QAOU. As a result, the offset type is determined for all QAOMUs (QAOMU numbers 0 to 340).

Next, the offset information selection unit 813 calculates the RD cost when the division depth is 0 and the RD cost when the division depth is 1 (FIG. 20A). In FIG. 20A, it is assumed that the RD cost at the division depth 1 is smaller than the RD cost at the division depth 0 (FIG. 20B).

Subsequently, the offset information selection unit 813 calculates the RD cost when the division depth is set to 2 (FIG. 20 (c)).

Subsequently, the offset information selection unit 813 compares the RD cost of the QAOMU with the division depth 1 with the RD cost of the QAOMU with the division depth 2 included in the QAOMU with the division depth 1, and determines the RD cost of the QAOMU with the division depth 2 If it is smaller, the QAOMU with the division depth 1 is updated to the QAOMU with the division depth 2 (FIG. 20 (d)). This process is repeated until the maximum division depth is reached. As a result, a QAOU partition structure with a smaller RD cost is determined.

(Adaptive offset filter processing unit 82)
The adaptive offset filter processing unit 82 adds the offset supplied from the offset information selection unit 813 to each pixel of the deblocked decoded image P_DB in the target QAOU. The adaptive offset filter processing unit 82 outputs an image obtained by performing processing on all the QAOUs included in the deblocked decoded image P_DB as an offset filtered decoded image P_OF. Note that the configuration of the adaptive offset filter processing unit 82 is the same as that of the adaptive offset filter processing unit 62, and thus the description thereof is omitted here.

Also, a configuration using an adaptive clip type may be used. That is, an adaptive clip (AC) type may be provided as one of the offset types. In this case, there are three offset types: EO, BO, and AC. In the adaptive clip type (AC), the pixel value is corrected using a clip of the lower limit value c1 and the upper limit value c2 without using an offset. Examples of the lower limit value c1 and the upper limit value c2 include c1 = 16 and c2 = 235.

∙ By using the adaptive clip type, an offset encoding process and a memory for holding a large number of offsets are not required. When the lower limit value and upper limit value of a clip are adaptively given instead of fixed values, the lower limit value and the upper limit value may be encoded. In this case, it is preferable to encode a difference from an appropriate fixed value, for example, an offset from 16 as the lower limit value and an offset from 235 as the upper limit value. In particular, since the upper limit value is a large value, the encoding efficiency is not lowered by doing so.

(Appendix 1)
Further, as described above, “according to the depth of the QAOU hierarchy” can also be referred to as “according to the size of the QAOU”. That is, the number of SAO types, the number of classes, and the offset accuracy may be varied according to the size of the QAOU.

For example, when the QAOU size is less than N × N pixels, one or more of the number of SAO types, the number of classes, and the offset accuracy are more limited than when the QAOU size is N × N pixels or more. It may be configured to. Specific examples of N include N = 64 (LCU size).

The number of SAO types is limited, for example, by limiting EO to horizontal and vertical types, using only one type of BO (conversion table 802C), using only EO and not using BO (conversion table 802A), Only one of horizontal type EO and BO can be used (conversion table 801B, conversion table 802D). The offset information decoding unit 611 uses a conversion table shown in parentheses. Note that the limitation on the number of SAO classes and the limitation on offset accuracy can be performed in the same manner as in the above-described embodiment.

Thus, by limiting the type of SAO type, the number of types, the number of classes, and the offset accuracy, it is possible to reduce the amount of memory used and the processing load. In particular, when the type type is limited to BO, or when EO is limited to horizontal and vertical types, the processing load on the class classification unit of the encoding device and the decoding device can be reduced. Further, when the number of types is limited, the cost calculation for selecting the optimum type can be omitted, so that the processing load on the encoding device can be reduced. Limiting the type of type to EO and limiting the number of classes and offset accuracy can reduce the amount of memory used to store offset. In addition, by limiting the type types to horizontal type EO and BO, temporary memory such as a line memory holding reference pixels used for class classification can be reduced, and while waiting for decoding of reference pixels The delay can be shortened. In order to reduce the amount of memory used, it is more effective to limit the number of BO classes than to limit the number of EO classes.

Such a limitation is that the operations of the offset information decoding unit 611 and the class classification unit 624 are adaptively performed according to the parameter condition when the parameter condition is set to the size of QAOUN by the configuration of the means shown in FIGS. It can be realized by changing.

(Appendix 2)
Further, the size of a QAOU (hereinafter also referred to as a luminance unit) for adding an offset to the luminance value is different from the size of a QAOU (hereinafter also referred to as a color difference unit) for adding an offset to the color difference. The configuration may be such that the type of SAO, the number of types, the number of classes, the offset accuracy, and the maximum division hierarchy are limited rather than the luminance unit.

For example, if the resolution differs between luminance and color difference, such as 4: 2: 0 format in the YUV format image format, and the color difference resolution is lower than the luminance, the fineness of the color difference division is as low as the luminance. No longer needed. Therefore, by limiting the types of SAO types, the number of types, the number of classes, the offset accuracy, and the maximum division hierarchy for the color difference unit, it is possible to reduce the amount of memory used and the processing load. Note that the restriction on the maximum hierarchy of the color difference units can be achieved, for example, by making the maximum depth of the SAO tree structure shallower than that of the luminance unit.

Such a limitation is that the operations of the offset information decoding unit 611 and the class classification unit 624 are adaptively performed according to the parameter condition when the parameter condition is set to the size of QAOUN by the configuration of the means shown in FIGS. It can be realized by changing.

In particular, when performing SAO processing for both luminance and color difference, it is necessary to provide a temporary memory such as a line memory that holds reference pixels used for class classification for each color component. Since the luminance component is less important than the color difference component, it is possible to reduce the line memory of the color difference component by limiting the type type to the horizontal type EO and BO with the parameter condition as the color difference component. In this case, the offset information decoding unit 611 uses conversion tables such as a conversion table 801B and a conversion table 802D.

It is also particularly effective to set the parameter condition as a color difference component and reduce the offset accuracy compared to the luminance component. For example, the accuracy of luminance component offset is SAO_DEPTH = MIN (PIC_DEPTH, AY)
Shift value = PIC_DEPTH-MIN (PIC_DEPTH, AY)
In the case of color difference components, the offset accuracy and shift value are
SAO_DEPTH = MIN (PIC_DEPTH, THC)
Shift value = PIC_DEPTH-MIN (PIC_DEPTH, AC)
In this case, it is appropriate to use AY> AC as the variables AY and AC for controlling accuracy. For example, AY = 10 or 9, and AC = 8. In this case, the accuracy of the luminance component offset is larger than that of the color difference component. Regarding the shift value, the shift value of the luminance component is smaller than the color difference component.

(Appendix 3)
Further, when the value of the CU quantization parameter QP (QP value) is equal to or greater than a threshold value, the type of SAO type, the number of types, the number of classes, the offset accuracy, and the maximum division hierarchy may be limited. . As the QP value, the initial QP value of the picture may be used. When the SAO processing target QAOU is along the boundary of the LCU or CU, the CU value at the position corresponding to the upper left coordinate or the center coordinate of the QAOU is used. A QP value may be used.

The higher the QP value, the lower the image quality of the predicted image, and it becomes difficult to perform detailed classification and offset correction. Therefore, the type of SAO, the number of types, the number of classes, the offset accuracy, and the maximum hierarchy of division Even if it is limited, the effect on the image quality is small. Therefore, when the QP value is equal to or greater than the threshold, if the configuration is such that the number of SAO types, the number of classes, the offset accuracy, and the maximum hierarchy of division are limited, the memory usage can be reduced without affecting the image quality, Processing load can be reduced.

Such a limitation is that the operations of the offset information decoding unit 611 and the class classification unit 624 are adaptively changed according to the parameter condition when the parameter condition is QP by the configuration of the means shown in FIGS. This can be achieved.

(Appendix 4)
In addition, for a specific type of picture, such as a B picture or a non-reference picture (IDR picture), the type of SAO type, the number of types, the number of classes, the offset accuracy, and the maximum hierarchy of division may be limited.

In the case of an I picture or a P picture, the picture quality of the picture has a great influence on the subsequent pictures, so it is necessary to keep the accuracy of the SAO high. On the other hand, since the pictures other than this do not greatly affect the subsequent pictures, the type of SAO, the number of types, the number of classes, the offset accuracy, and the like for these pictures (B picture, IDR picture) and The maximum hierarchy of division may be limited. As a result, the memory usage can be reduced and the processing load can be reduced.

Such a limitation is that the operations of the offset information decoding unit 611 and the class classification unit 624 are adaptively performed according to the parameter condition when the parameter condition is a picture type by the configuration of the means shown in FIGS. It can be realized by changing.

(Appendix 5)
Further, the configuration may be such that the number of SAO types, the number of classes, the offset accuracy, and the maximum division hierarchy are limited according to the position of the QAOU on the screen.

For example, at the edge of the screen, the number of SAO types, the number of classes, the offset accuracy, and the maximum division hierarchy may be limited.

Since the area near the center of the screen is easy for the user to pay attention to, if the image quality is lowered, the subjective image quality is lowered as it is, but even if the image quality is lowered at the periphery of the screen, the subjective image quality is not lowered as much as the area near the center of the screen.

Boundary determination processing can be reduced if the SAO type that requires sample points exceeding the screen edge is not used.

Such a limitation is that the operations of the offset information decoding unit 611 and the class classification unit 624 are adaptively performed according to the parameter condition when the parameter condition is set to the position of QAOU by the configuration of the means shown in FIGS. It can be realized by changing.

[Embodiment 2]
The following will describe another embodiment of the present invention with reference to FIG. The present embodiment is different from the above embodiment in that EO and BO are switched according to the pixel value.

The inventors of the present application have found that EO is more effective than BO in the middle gradation range, and that BO is more effective in other value ranges (low pixel value range / high pixel value range). did. Therefore, in the present embodiment, BO is used in the low pixel value region and the high pixel value region, and EO is used in the intermediate gradation region.

Specific description will be given with reference to FIG. FIG. 21 is a diagram for explaining a configuration for switching between EO and BO according to pixel values, and (a) is a diagram for explaining an outline of a configuration for switching between EO and BO according to pixel values. (b) is a diagram for explaining specific switching values, and (c) is a diagram showing the contents of the list memory stored in the offset information storage unit 621.

As shown in FIG. 21 (a), in this embodiment, when the pixel value (luminance value) is near 0 and around 255, BO is used, and EO is used in other value ranges. That is, the SAO type is classified according to the pixel value, and when the pixel value is around 0 and around 255, an offset corresponding to the pixel value is added, and in other regions, an offset corresponding to the edge type is added.

As a result of experiments by the inventors of the present application, EO and BO were found to be biased in the range of pixel values used. That is, it was found that EO is often in the intermediate gradation range. This means that in the low pixel value range and the high pixel value range, the influence on the error tends to be larger in the pixel value than in the edge type.

Therefore, in the present embodiment, EO and BO are determined according to the pixel value range, and a single SAO type uses a portion having a high error correction effect in both EO and BO. Thereby, encoding efficiency can be improved. In addition, since the number of SAO types and the total number of classes can be reduced, it is possible to reduce the amount of memory used and the processing load.

Next, this embodiment will be described in more detail. First, unlike the first embodiment, the present embodiment uses 1 to 4 for “sao_type_idx” and defines them as follows.
“Sao_type_idx” = 1: (BO_EO_0) Corresponds to EO_0 + BO of the first embodiment “sao_type_idx” = 2: (BO_EO_1) Corresponds to EO_1 + BO of the first embodiment “sao_type_idx” = 3: (BO_EO_2) Embodiment “Sao_type_idx” = 4: (BO_EO_3) Corresponding to EO_3 + BO of the first embodiment Also, EO_0 to 3 and BO_0, 1 of the first embodiment are not used.

And the class classification | category part 624 performs a class classification | category using BoTbl [BO_1] in the conversion table 1301 of FIG. 13, as shown in FIG.21 (b). If the pixel value is ¼ or less of max and ¾ or more of max, classification is performed using the conversion table 1301.

If the pixel value is between 1/4 and 3/4 of max, the edge type is determined according to EO_0 to 3 described in the first embodiment, and class classification is performed.

Thereby, the class classification unit 624 can classify the class by switching between BO and EO according to the pixel value.

Also, the offset calculation unit 811 calculates the offset by the same method as the offset calculation method described in the first embodiment. However, in the present embodiment, since the number of types of offset types is four, this point is calculated differently.

Further, as shown in FIG. 21C, the list memory 2101 stored in the offset information storage unit 621 has a QAOU index, an offset type, and a specific offset value for each class that can be selected in the offset type. Stored in association with each other.

In the case of using an offset type having both characteristics of EO and BO, first, classification is performed using EO, and a specific class (a class having a flat edge) is further classified using BO. There may be. Moreover, the structure which uses EO and BO described in Embodiment 1 together as it is may be sufficient. Further, according to the condition described in the first embodiment (whether or not the depth of the QAOU hierarchy is smaller than a threshold value), whether or not to provide an offset type having the characteristics of EO and BO is changed. It may be configured to reduce the number of classes.

[Embodiment 3]
The following will describe another embodiment of the present invention with reference to FIGS. The present embodiment is different from the above embodiment in that, in the EO classification, the pixels used for edge determination are limited to only the pixels existing in the horizontal direction of the determination target pixel.

In the above embodiment, pixels that exist above or below the target pixel are also used for edge determination. For this reason, a line buffer that holds the pixel value of the pixel before the offset processing is necessary for the pixel that exists in the upper direction of the target pixel (pixel that has undergone the offset processing).

Therefore, if the pixels used for the edge determination are limited to the pixels existing in the horizontal direction of the target pixel, there is no need to refer to the upper direction, and the amount of memory used for the line buffer can be reduced. In addition, since the upper boundary determination (such as the screen edge) is unnecessary, the processing amount can be reduced.

In addition, when only the pixels existing in the horizontal direction are used, the processing result up to the previous time can be used, so that the processing amount can be further reduced. This will be described with reference to FIG. FIG. 22A shows pixels existing in the horizontal direction, and x0 to x3 are pixel values of the respective pixels.

First, with respect to a pixel having a pixel value x1, the difference from both sides is as follows.
s1 = sign (x1-x0) -sign (x2-x1)
Next, the difference between both sides of the pixel having the pixel value x2 is calculated as follows.
s2 = sign (x2-x1) -sign (x3-x2)
Here, s1 and s2 are values used for classifying the edge at x1 and x2.

Thus, when calculating s2, the sign (x2-x1) used in s1 is reused. Therefore, the processing amount can be reduced by this amount.

Next, details of this embodiment will be described. In this embodiment, there are two types of EO. That is, only when “sao_type_idx” = 1, 2 is set as an edge offset (EO). When “sao_type_idx” = 1 (EO_0), the edge type is derived as compared with the left and right pixels of the target pixel (FIG. 22B), and when “sao_type_idx” = 2 (EO′_0) Derives the type of edge compared to the pixel located two pixels away from the left and right of the target pixel (FIG. 22C).

In addition, when “sao_type_idx” = 3, 4 is a band offset (BO). This may be read as “3, 4” in the case of “sao_type_idx” = 5, 6 in the first embodiment.

In the case of “sao_type_idx” = 2, the edge type is determined using a pixel further separated by 1 pixel from both sides of the target pixel in order to facilitate detection of an edge having a shallow angle with respect to the horizontal. is there. As shown in FIG. 22D, when the pixels adjacent to the target pixel are set as comparison targets, the difference between the pixel values may be small and may not be determined as an edge. Even in such a case, it is possible to detect an edge having a shallow angle with respect to the horizontal by taking a difference from the adjacent pixel by one pixel.

Further, instead of using both sides of the target pixel, the left side may be the adjacent pixel, the right side may be the difference of one pixel and the adjacent pixel (FIG. 23A), or vice versa (FIG. 23B). It may be used. This is particularly effective when the reference pixel hits the edge of the screen.

Furthermore, the configuration may be such that only the horizontal pixels are used. For example, only the picture in the horizontal direction is referred to only in the case of a picture that has little influence on other pictures such as a B picture and a non-reference picture, and in other cases, the configuration is the same as in the first embodiment. May be. Also, in the vicinity of the screen edge or near the slice boundary, pixels in the horizontal direction are referred to, and other configurations may be the same as in the first embodiment. In this way, the amount of memory used can be reduced and the amount of processing can also be reduced. Further, since no upward reference is performed, the boundary determination process near the screen edge or slice boundary can be reduced.

Further, it may be configured to be explicitly specified by a flag in units of pictures or blocks.
For example, when there are many horizontal edges (FIG. 23 (c)), if the process is performed by the method described in the first embodiment, it is possible to prevent the performance from being deteriorated.

Also, whether or not to use only the pixels that exist in the horizontal direction may be changed according to the condition described in the first embodiment (whether the depth of the QAOU hierarchy is smaller than a threshold). Furthermore, only in the case where BO and EO are used in combination as described in the second embodiment, EO may be configured to use only pixels that exist in the horizontal direction.

Further, the type of edge may be derived depending on whether or not the difference from the pixel existing in the horizontal direction is larger (or smaller) than the threshold value. Specifically, classes may be classified according to the following formula.

Sign (z) = + 1 (when z> th)
Sign (z) = 0 (when −th ≦ z ≦ th)
Sign (z) =-1 (when z <-th)
Here, th is a threshold value having a predetermined value.

In the case of a pixel value representing a color difference, only a pixel that exists in the horizontal direction may be used in the edge offset.

The effect of limiting to the horizontal can be obtained when there is only one horizontal edge classification method, for example, only in the case of FIG. As described above, when a plurality of different horizontal edge classification methods are combined, finer edge classification is possible than when one horizontal edge classification method, for example, FIG. 22B is used. Note that the combination in the case of using a plurality of horizontal edge classification methods is not limited to the case of using two examples by changing the distance between the target pixel and the reference pixel, as shown in FIGS. 22B and 22C. A configuration in which two threshold values used in deriving the type of the above are also possible. For example, two horizontal edge classification methods for the case where the threshold th is 0 and 1 may be provided.

The configuration for limiting the use of only the pixels in the horizontal direction is the operation of the offset information decoding unit 611 and the class classification unit 624 adaptively according to the parameter conditions by the configuration of the means shown in FIGS. 32 to 34 of the first embodiment. It can be realized by changing. For example, in the offset information decoding unit 611 in FIG. 32, the type of offset type selected by the used offset type selection unit 6112 is limited only to the horizontal edge classification method according to the parameter condition, and is limited by the offset type decoding unit 6113. Decodes the type offset type.

As shown in FIG. 33 (b), the offset type conversion table selection unit 6117 is a conversion table provided in the edge offset type and band offset type storage unit 6118 ′ or horizontal edge offset type and band offset type conversion according to the parameter conditions. A conversion table provided in the table storage unit 6119 ′ is selected. In one example, when the color component is luminance, the conversion table provided in the edge offset type and band offset type storage unit 6118 ′ is selected, and in the case of color difference, the horizontal edge offset type and band offset type conversion table storage unit 6119 is selected. A conversion table included in 'is selected.

In the class classification unit 624, the internal use offset type selection unit 6112 selects an edge classification method limited to only the horizontal edge classification method according to the parameter condition, and the edge offset class classification unit 6243 classifies the pixels. In one example, the basic edge classification method is selected when the color component is luminance, and the horizontal edge classification method is selected when the color component is color difference.

[Embodiment 4]
The following will describe another embodiment of the present invention with reference to FIGS. This embodiment is different from the above embodiment in that the offset accuracy is improved or the class is divided finely in the vicinity of the center of the color difference pixel value.

In the image, the error near the achromatic color has a characteristic that it is easily noticeable in the subjective image quality. In the pixel value of each color difference, the achromatic color is the center of the range, and if the bit depth is 8 bits, it is the color of the pixel having a pixel value of 128. Therefore, in the present embodiment, in the case of BO, the offset accuracy is increased or the class division is made finer by using a smaller class width around the pixel value 128 of the color difference. In the following, a value range near the achromatic color for the pixel value of the color difference is referred to as an achromatic color value range.

This makes it possible to increase the offset accuracy in the achromatic color value range, thereby improving the offset correction performance in the achromatic color value range. Therefore, an error from the original image in the achromatic color range is reduced, and the subjective image quality can be improved. Further, in the achromatic color value range, the class can be divided more finely than the other value ranges, so that the offset correction in the achromatic color value range can be performed more precisely. Thereby, an error from the original image can be reduced, and the subjective image quality can be improved.

In the present embodiment, the offset accuracy of the achromatic color range is improved and the class division is subdivided, but this is not limiting, and class division is performed for values in a range that is likely to affect subjective image quality. Subjective image quality can be improved by subdividing or improving the offset accuracy. In addition, with respect to the pixel value of luminance, the same effect can be obtained by a configuration in which the offset accuracy is similarly increased or the class division is subdivided in a pixel value range in which image quality degradation is conspicuous in terms of subjective image quality.

First, the case where the offset accuracy is improved will be described with reference to FIG. FIG. 24 is a diagram showing an outline of a case where the offset accuracy is improved. As shown in FIG. 24, when improving the accuracy of the offset, the accuracy of the offset in the vicinity of the pixel value 128 of the color difference is made high, and the accuracy of the offset in other value ranges is made low.

Further, the offset information storage unit 621 secures a storage area of n _B bits (for example, n _B = 8) for the offset of the high-precision range, and n _A for the offset of the low-precision range. _A storage area for bits (for example, n _A = 6) is secured. If there is a margin in the storage area, n _B bits may be unified. In this case, mounting becomes easy.

Further, the offset clip unit 812 switches the clip range between a value range where the offset is high accuracy and a value range where the offset is low accuracy. For example, for a value range with a low offset, the clip range is -2 ^{nA / 2} to 2 ^{nA / 2} -1, and for a value range with a high offset, the clip range is -2 ^{nB / 2} to 2 ^{nB. / 2} −1.

Next, the case of subdividing the class classification will be described with reference to FIG. FIG. 25 is a diagram showing an outline when class classification is subdivided, (a) shows a conversion table, and (b) to (d) are diagrams for explaining class division.

When class classification is subdivided, an offset storage area corresponding to “sao_type_idx” = 5 is secured in the color difference list memory in the offset information storage unit 621 according to the number of class classifications.

In the case of BO, the class classification unit 624 determines whether the pixel value of the target pixel is in the achromatic value range when deriving the class from the pixel value. Specifically, class classification is performed using, for example, one of the conversion tables 2501 shown in FIG.

In the conversion table 2501, class classification is performed by “class_idx = BoTbl [sao_type_idx] [pix [x] / 4]”, and a class is obtained for a pixel value that is finer than the conversion tables 1301 and 1302 in FIG. Assigned.

In “BoTbl [BO — 0] [pix / 4] (a)” of the conversion table 2501, the class is divided into 16 parts, the achromatic color range is subdivided, and the intermediate gradation range (pix / 4 = 16 to 21, 42). To 47) is shown as an example (FIG. 25B). In “BoTbl [BO — 0] [pix / 4] (b)”, the class is divided into 16 classes, the range with offset is narrowed, and the achromatic color range is subdivided while maintaining the maximum number of classes and class width. An example is shown (FIG. 25 (c)). In addition, “BoTbl [BO — 0] [pix / 4] (c)” shows an example in which the class is divided into 18 parts, the value range near the achromatic color is subdivided, and the maximum value of the class width is also maintained ( FIG. 25 (d)). As another example, the correction accuracy of the achromatic color range can be improved by making the class width of the intermediate gradation range relatively larger than the class width of the achromatic color range and reducing the total number of classes. The offset storage area can be reduced while keeping the street. This is possible because the frequency of use of offset correction in the intermediate gradation region is lower than that in the achromatic region.

Further, a configuration in which the offset accuracy is improved and the classification of the class classification in the achromatic color value range is used in combination.

Further, a configuration in which the offset accuracy differs between the pixel value indicating luminance and the pixel value indicating color difference, that is, a configuration in which the offset accuracy for the pixel value indicating the color difference is coarser than the offset accuracy for the pixel value indicating the luminance. It may be.

(Appendix 6)
Further, the offset may be added in a direction to approach an achromatic color (= color difference pixel value 128). That is, the offset addition unit 626 may handle the pixel values of 128 or more and less than 128 with the signs of the offsets to be added reversed. For example, when the offset is a, x ′ = x + a (x <128) and x ′ = x−a (x ≧ 128) may be used.

Further, when the pixel value after the offset addition exceeds 128, a configuration of clipping at 128 may be used. For example, when the pixel value is “126 (green side)” and the offset is “3”, the offset is added as it is, and not x = 126 + 3 = 129, but x = clip (126 + 3, 0, 128) = 128 may be used. Here, clip (x, y, z) indicates a process of limiting the value of x to y ≦ x ≦ z.

Further, for example, when the pixel value is “129 (red side)” and the offset is “−4”, the offset is added as it is, and x = clip is not obtained without setting x = 129 + (− 4) = 125. (129 + (− 4), 128, 255) = 128 may be set.

(Appendix 7)
In addition, in the color difference, the offset may be added to the intermediate gradation range and other value ranges. For example, as shown in FIG. 26A, class classification may be performed in consideration of achromatic pixel values. In the example shown in FIG. 26A, when “sao_type_idx” = 5 (BO_0), a class is assigned to an intermediate gradation range, and when “sao_type_idx” = 6 (BO_1), a value range other than the intermediate gradation range is assigned. A class is assigned.

Also, as shown in FIG. 26 (b), classification may be performed asymmetrically in two value ranges sandwiching achromatic pixel values. In the example shown in FIG. 26B, the class width is the same as in the example shown in FIG.

Further, as shown in FIGS. 26C and 26D, different class classifications may be used for each color difference channel (Cr and Cb). In the example shown in FIGS. 26C and 26D, the class width differs between the color difference (Cr) and the color difference (Cb). The class width can be changed by subdividing the number of classes as in the conversion table 2501.

Also, as shown in FIG. 26 (e), the color difference may be configured to classify with one BO type.

[Embodiment 5]
The following will describe another embodiment of the present invention with reference to FIGS. In the present embodiment, the difference from the above embodiment is that the offset prediction difference is encoded, not the offset value itself, that is, using the offset value and the predicted value of the offset value. The offset residual is encoded and the offset prediction value is set to “0”.

Generally, in SAO, an offset is encoded according to a class indicating an area classification. If there is no pixel classified into the class (the class is empty), “0” is encoded. Normally, the offsets of the neighborhood class and the neighborhood region have close values. Therefore, if the difference is encoded using the already-decoded offset value as a predicted value, the amount of codes can be reduced. However, if the class is empty, the code amount increases.

Therefore, in the present embodiment, the offset prediction value includes “0”, selects one prediction value from a plurality of prediction value candidates including “0”, and encodes the prediction difference to reduce the code amount of the offset. Reduced.

As a result, even if the class is empty, even if there are pixels classified into the class and an offset is encoded, an appropriate prediction value can be assigned. Can be reduced.

The configuration of the offset information decoding unit 611 ′ according to this embodiment will be described with reference to FIG. The offset information decoding unit 611 ′ is provided in place of the offset information decoding unit 611 in FIG. 1 and includes an offset residual decoding unit 651, an offset restoration unit 652, and a predicted value derivation unit 653. The predicted value derivation unit 653 includes a prediction candidate flag decoding unit 661, a fixed prediction value calculation unit 662, an encoded prediction value calculation unit 663, and a prediction value candidate selection unit 664.

The offset residual decoding unit 651 decodes the offset residual from the QAOU information included in the encoded data # 1, and supplies the decoded offset residual to the offset restoration unit 652.

The predicted value deriving unit 653 derives a predicted offset value. The prediction candidate flag decoding unit 661 decodes the prediction candidate flag from the QAOU information, and supplies the decoded prediction candidate flag to the prediction value candidate selection unit 664. Note that the prediction candidate flag decoding unit 661 may be configured not to decode the candidate selection flag when the previously decoded offset value is “0”. ”
The fixed prediction value calculation unit 662 calculates a fixed value (here, “0”) that is an offset that is encoded when there is no pixel classified into the class as a prediction value, and supplies the prediction value candidate selection unit 664 with the prediction value. To do.

The encoded predicted value calculation unit 663 reads the offset that has already been decoded from the offset information storage unit 621, calculates a predicted value, and supplies the predicted value to the predicted value candidate selection unit 664.

The prediction value candidate selection unit 664 supplies the offset restoration unit 652 with the prediction value supplied from the fixed prediction value calculation unit 662 and the prediction value supplied from the encoded prediction value calculation unit 663 according to the prediction candidate flag. Select the predicted value.

The offset restoration unit 652 calculates an offset (Offset) from the prediction value (pred) supplied from the prediction value candidate selection unit 664 and the offset residual (sao_offset) supplied from the offset residual decoding unit 651 according to the following equation. ).
“Offset” = “pred” + “sao_offset”
Here, the reason why the code amount increases when there is a class (empty class) having no pixel to be classified will be described with reference to FIG. FIG. 28 is a diagram showing an outline when there is an empty class. As shown in FIG. 28, a non-empty class is encoded with the offset of the class. Then, for each class, the difference between the predicted value and the offset is encoded using the previous offset as the predicted value. However, if there is an empty class, the predicted value becomes “0” and the difference value becomes large. As a result, the code amount increases.

Therefore, in the present embodiment, the prediction value can be selected from “0” and “previous (other than 0) offset” by the prediction candidate flag.

The syntax when the prediction candidate flag is used will be described with reference to FIG. FIG. 29 is a diagram illustrating the syntax 2901 when the prediction candidate flag is used. As shown in FIG. 29, a prediction candidate flag “sao_pred_flag” is added, and if “sao_pred_flag” = 0, the immediately decoded (non-zero) offset is set as the prediction value offsetp, and “sao_pred_flag” = 1 or the first If it is an offset, the predicted value offsetp = 0. In the syntax 2901, “sao_offset_delta” represents the difference d from the predicted value, and “sao_offset [sao_curr_depth] [ys] [xs]” = “offsetp + d”.

In addition, it is good also as a structure which does not use a prediction candidate flag according to an offset type. For example, the prediction candidate flag may not be used in the case of EO, and the prediction candidate flag may be used in the case of BO. This is because an empty class often occurs in BO depending on the distribution of pixel values.

Further, it may be configured to determine whether or not to perform predictive encoding itself according to the offset type. For example, in the case of EO, prediction encoding may be performed, and in the case of BO, prediction encoding may not be performed.

(Appendix 8)
Further, in the first to fifth embodiments, the edge offset class may be estimated, limited, or rearranged for each LCU using the intra prediction mode.

The edge direction selected as the SAO EO type is considered to be correlated with the intra prediction mode. For this reason, if the intra prediction mode of the CU at the position corresponding to the SAO QAOU is referred to, it can be used to select the EO class.

For example, EO class is estimated and determined by intra prediction mode, EO class candidates are limited, or the order (index) of EO classes is rearranged in the order of high possibility of selection. Can do.

This can be realized particularly easily if the size of the QAOU in each layer of SAO is the same size as the CU, for example if the image is divided so that the maximum size of the SAO QAOU is equal to the LCU size. It is.

(Application examples)
The moving picture decoding apparatus 1 and the moving picture encoding apparatus 2 described above can be used by being mounted on various apparatuses that perform moving picture transmission, reception, recording, and reproduction. Note that the moving image may be a natural moving image captured by a camera or the like, or an artificial moving image (including CG and GUI) generated by a computer or the like).

First, it will be described with reference to FIG. 30 that the above-described moving picture decoding apparatus 1 and moving picture encoding apparatus 2 can be used for transmission and reception of moving pictures.

FIG. 30 (a) is a block diagram showing a configuration of a transmission apparatus A in which the moving picture encoding apparatus 2 is mounted. As shown in FIG. 30 (a), the transmitting apparatus A encodes a moving image, obtains encoded data, and modulates a carrier wave with the encoded data obtained by the encoding unit A1. A modulation unit A2 that obtains a modulation signal by the transmission unit A2 and a transmission unit A3 that transmits the modulation signal obtained by the modulation unit A2. The moving image encoding device 2 described above is used as the encoding unit A1.

The transmission apparatus A has a camera A4 that captures a moving image, a recording medium A5 that records the moving image, an input terminal A6 for inputting the moving image from the outside, as a supply source of the moving image that is input to the encoding unit A1. You may further provide image processing part A7 which produces | generates or processes an image. FIG. 30A illustrates a configuration in which the transmission apparatus A includes all of these, but some of them may be omitted.

The recording medium A5 may be a recording of a non-encoded moving image, or a recording of a moving image encoded by a recording encoding scheme different from the transmission encoding scheme. It may be a thing. In the latter case, a decoding unit (not shown) for decoding the encoded data read from the recording medium A5 according to the recording encoding method may be interposed between the recording medium A5 and the encoding unit A1.

FIG. 30B is a block diagram illustrating a configuration of the receiving device B on which the moving image decoding device 1 is mounted. As illustrated in FIG. 30B, the receiving device B includes a receiving unit B1 that receives a modulated signal, a demodulating unit B2 that obtains encoded data by demodulating the modulated signal received by the receiving unit B1, and a demodulating unit. A decoding unit B3 that obtains a moving image by decoding the encoded data obtained by B2. The moving picture decoding apparatus 1 described above is used as the decoding unit B3.

The receiving device B has a display B4 for displaying a moving image, a recording medium B5 for recording the moving image, and an output terminal for outputting the moving image as a supply destination of the moving image output from the decoding unit B3. B6 may be further provided. FIG. 30B illustrates a configuration in which the receiving apparatus B includes all of these, but a part of the configuration may be omitted.

Note that the recording medium B5 may be for recording an unencoded moving image, or is encoded by a recording encoding method different from the transmission encoding method. May be. In the latter case, an encoding unit (not shown) that encodes the moving image acquired from the decoding unit B3 in accordance with the recording encoding method may be interposed between the decoding unit B3 and the recording medium B5.

Note that the transmission medium for transmitting the modulation signal may be wireless or wired. Further, the transmission mode for transmitting the modulated signal may be broadcasting (here, a transmission mode in which the transmission destination is not specified in advance) or communication (here, transmission in which the transmission destination is specified in advance). Refers to the embodiment). That is, the transmission of the modulation signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

For example, a terrestrial digital broadcast broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) is an example of a transmitting device A / receiving device B that transmits and receives a modulated signal by wireless broadcasting. A broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) for cable television broadcasting is an example of a transmitting device A / receiving device B that transmits and receives a modulated signal by cable broadcasting.

Also, a server (workstation etc.) / Client (television receiver, personal computer, smart phone etc.) such as VOD (Video On Demand) service and video sharing service using the Internet is a transmitting device for transmitting and receiving modulated signals by communication. This is an example of the A / reception device B (usually, either wireless or wired is used as a transmission medium in a LAN, and wired is used as a transmission medium in a WAN). Here, the personal computer includes a desktop PC, a laptop PC, and a tablet PC. The smartphone also includes a multi-function mobile phone terminal.

In addition to the function of decoding the encoded data downloaded from the server and displaying it on the display, the video sharing service client has a function of encoding a moving image captured by the camera and uploading it to the server. That is, the client of the video sharing service functions as both the transmission device A and the reception device B.

Next, the fact that the above-described moving picture decoding apparatus 1 and moving picture encoding apparatus 2 can be used for recording and reproduction of moving pictures will be described with reference to FIG.

FIG. 31A is a block diagram showing a configuration of a recording apparatus C equipped with the moving picture decoding apparatus 1 described above. As shown in FIG. 31 (a), the recording device C encodes a moving image to obtain encoded data, and writes the encoded data obtained by the encoding unit C1 to the recording medium M. And a writing unit C2. The moving image encoding device 2 described above is used as the encoding unit C1.

The recording medium M may be of a type built in the recording device C, such as (1) HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) SD memory. It may be of the type connected to the recording device C, such as a card or USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray Disc: registration) (Trademark) or the like may be mounted on a drive device (not shown) built in the recording apparatus C.

The recording apparatus C also serves as a moving image supply source to be input to the encoding unit C1, a camera C3 that captures moving images, an input terminal C4 for inputting moving images from the outside, and reception for receiving moving images. A unit C5 and an image processing unit C6 that generates or processes an image may be further provided. FIG. 31A illustrates a configuration in which the recording apparatus C includes all of these, but some of them may be omitted.

The receiving unit C5 may receive an unencoded moving image, or receives encoded data encoded by a transmission encoding method different from the recording encoding method. You may do. In the latter case, a transmission decoding unit (not shown) that decodes encoded data encoded by the transmission encoding method may be interposed between the reception unit C5 and the encoding unit C1.

Examples of such a recording device C include a DVD recorder, a BD recorder, and an HD (Hard Disk) recorder (in this case, the input terminal C4 or the receiving unit C5 is a main source of moving images). In addition, a camcorder (in this case, the camera C3 is a main source of moving images), a personal computer (in this case, the receiving unit C5 or the image processing unit C6 is a main source of moving images), a smartphone (this In this case, the camera C3 or the receiving unit C5 is a main source of moving images).

FIG. 31 (b) is a block diagram showing the configuration of the playback device D on which the above-described moving image decoding device 1 is mounted. As shown in FIG. 31 (b), the playback device D obtains a moving image by decoding the read data D1 read by the read unit D1 and the read data read by the read unit D1. And a decoding unit D2. The moving picture decoding apparatus 1 described above is used as the decoding unit D2.

The recording medium M may be of a type built in the playback device D such as (1) HDD or SSD, or (2) such as an SD memory card or USB flash memory. It may be of a type connected to the playback device D, or (3) may be loaded into a drive device (not shown) built in the playback device D, such as DVD or BD. Good.

Further, the playback device D has a display D3 for displaying a moving image, an output terminal D4 for outputting the moving image to the outside, and a transmitting unit for transmitting the moving image as a supply destination of the moving image output by the decoding unit D2. D5 may be further provided. FIG. 31B illustrates a configuration in which the playback apparatus D includes all of these, but a part of the configuration may be omitted.

The transmission unit D5 may transmit a non-encoded moving image, or transmits encoded data encoded by a transmission encoding method different from the recording encoding method. You may do. In the latter case, an encoding unit (not shown) that encodes a moving image with a transmission encoding method may be interposed between the decoding unit D2 and the transmission unit D5.

Examples of such a playback device D include a DVD player, a BD player, and an HDD player (in this case, an output terminal D4 to which a television receiver or the like is connected is a main moving image supply destination). . In addition, a television receiver (in this case, the display D3 is a main destination of moving images), a desktop PC (in this case, the output terminal D4 or the transmission unit D5 is a main destination of moving images), Laptop type or tablet type PC (in this case, display D3 or transmission unit D5 is the main supply destination of moving images), smartphone (in this case, display D3 or transmission unit D5 is the main supply destination of moving images) ), Digital signage (also referred to as an electronic signboard or an electronic bulletin board, and the display D3 or the transmission unit D5 is the main supply destination of moving images) is an example of such a playback device D.

(Configuration by software)
Finally, each block of the moving picture decoding apparatus 1 and the moving picture encoding apparatus 2, in particular, the variable length code decoding unit 13, the motion vector restoration unit 14, the inter prediction image generation unit 16, the intra prediction image generation unit 17, and the prediction method determination Unit 18, inverse quantization / inverse transform unit 19, deblocking filter 41, adaptive filter 50, adaptive offset filter 60, transform / quantization unit 21, variable length code encoding unit 22, inverse quantization / inverse transform unit 23, The intra prediction image generation unit 25, the inter prediction image generation unit 26, the motion vector detection unit 27, the prediction scheme control unit 28, the motion vector redundancy deletion unit 29, the deblocking filter 33, the adaptive filter 70, and the adaptive offset filter 80 are integrated. It may be realized by hardware by a logic circuit formed on a circuit (IC chip), or a CPU (central proce It may be realized by software using a ssing unit).

In the latter case, the moving picture decoding apparatus 1 and the moving picture encoding apparatus 2 include a CPU that executes instructions of a control program that realizes each function, a ROM (read only memory) that stores the program, and a RAM that expands the program. (Random access memory), a storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is to provide a computer with program codes (execution format program, intermediate code program, source program) of control programs for the video decoding device 1 and the video encoding device 2 which are software for realizing the functions described above. A readable recording medium is supplied to the moving picture decoding apparatus 1 and the moving picture encoding apparatus 2, and the computer (or CPU or MPU (micro processing unit)) is recorded on the recording medium. This can also be achieved by reading out and executing.

Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, and CD-ROMs (compact disc read-only memory) / MO discs (magneto-optical discs). ) / MD (Mini Disc) / DVD (digital versatile disc) / CD-R (CD recordable) and other optical discs, IC cards (including memory cards) / optical cards, etc., mask ROM / EPROM (Erasable programmable read-only memory) / EEPROM (electrically erasable and programmable read-only memory) / semiconductor memories such as flash ROM, or logic circuits such as PLD (Programmable logic device) and FPGA (Field Programmable Gate Array) Can be used.

Further, the moving picture decoding apparatus 1 and the moving picture encoding apparatus 2 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited as long as it can transmit the program code. For example, the Internet, an intranet, an extranet, a LAN (local area network), an ISDN (integrated services network), a VAN (value-added network), a CATV (community antenna network / cable network), a virtual private network (virtual private network) network), telephone line network, mobile communication network, satellite communication network, and the like. The transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type. For example, IEEE (institute-of-electrical-and-electronic-engineers) 1394, USB, power line carrier, cable TV line, telephone line, ADSL (asymmetric-digital-subscriber-loop) line, etc. wired such as IrDA (infrared-data association) or remote control , Bluetooth (registered trademark), IEEE802.11 wireless, HDR (high data rate), NFC (Near field communication), DLNA (Digital Living Network Alliance), mobile phone network, satellite line, terrestrial digital network, etc. Is possible. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

(Other)
The present invention can also be described as follows. An image filter device according to the present invention is an image filter device that adds an offset selected from a plurality of offsets to each pixel value of an input image composed of a plurality of unit regions, and is included in each unit region. Offset determining means for determining an offset to be added to the pixel value of each pixel for each unit area; and filter means for adding the offset determined by the offset determining means to the pixel value of the pixel included in the unit area. When the size of the unit area for which the offset is to be determined is smaller than a predetermined size, the offset determining means has an offset with a limited number of offsets that can be selected as compared with the case where the size of the unit area is equal to or larger than the predetermined size. An offset to be added to a pixel value of a pixel included in the unit area is determined from the set.

According to the above configuration, when the size of the unit area for which the offset is determined is smaller than a predetermined size, from the set of offsets that are more limited than when the size of the unit area is equal to or larger than the predetermined size, The offset to be added is determined.

When the size of the unit area is small, the number of pixels included in the unit area is small, and the pixels are likely to have approximate values. Therefore, when the size of the unit area is small, even if the number of selectable offsets is limited, the influence on the image after the offset application is small. Further, by limiting the number of offsets that can be selected, the required memory capacity can be reduced.

Therefore, according to the above configuration, it is possible to reduce the amount of memory used while reducing the effect on the image after the offset application. Also, since the number of selectable offsets is reduced, the amount of codes can be reduced, and the coding efficiency is improved.

In order to solve the above problems, an image filter device according to the present invention is an image filter device that adds an offset to each pixel value of an input image composed of a plurality of unit regions, and includes pixels included in each unit region. Offset determining means for determining the offset to be added to the pixel value for each unit region, and filter means for adding the offset determined by the offset determining means to the pixel value of the pixel included in the unit region, The filter means is characterized in that, when the size of the unit area for which the offset is determined is smaller than a predetermined size, an offset with coarser accuracy is added than when the size of the unit area is equal to or larger than the predetermined size. Yes.

According to the above configuration, when the size of the unit area for which the offset is to be determined is smaller than a predetermined size, an offset having a coarser accuracy is added than when the size of the unit area is equal to or larger than the predetermined size.

When the size of the unit area is small, the number of pixels included in the unit area is small, and the pixels are likely to have approximate values. Therefore, when the size of the unit region is small, the influence on the quantization error is small even if the offset accuracy is rough. Therefore, the influence on the image after applying the offset is small. In addition, it is possible to reduce the required memory capacity by increasing the accuracy of the offset.

Therefore, according to the above configuration, it is possible to reduce the amount of memory used while reducing the effect on the image after the offset application. Further, since the offset accuracy becomes coarse, the amount of codes can be reduced, and the coding efficiency is improved.

In order to solve the above problems, an image filter device according to the present invention is an image filter device that applies an adaptive offset (SAO: Sample ： Adaptive Offset) to each pixel value of an input image composed of a plurality of unit regions. , An offset determining unit that determines for each unit region the type of offset to be added to the pixel value of the pixel included in each unit region, and an offset according to the type of offset determined by the offset determining unit in the unit region Filter means for adding to the pixel values of the included pixels, the offset determining means for the target pixel to which the offset is to be added, for the pixels whose pixel values are near the maximum value and the minimum value, The type is determined to be band offset (BO), and the edge offset (EO) is determined for pixels in other ranges. It is characterized.

According to the above configuration, when adaptive offset is applied, a band offset is applied to pixels in the vicinity of the maximum and minimum pixel values, and an edge offset is applied to pixels in other value ranges. Is done.

In the high pixel value region and the low pixel value region, the pixel value tends to affect the error more easily than the edge. Therefore, the above configuration increases the error correction efficiency and improves the encoding efficiency. .

Also, if band offset and edge offset are used together in one type, the number of types of adaptive offset can be reduced, and the amount of memory used and the amount of processing can be reduced.

In the above configuration, the band offset refers to an offset process in which one of a plurality of offsets is added to the pixel value of the processing target pixel according to the size of the pixel value of the processing target pixel ( The same applies below). In the above configuration, the edge offset is a plurality of offsets in the pixel value of the processing target pixel according to the difference between the pixel value of the processing target pixel and the pixel values of the pixels around the processing target pixel. This is offset processing for adding any of the above (same below).

In the image filter device according to the present invention, the offset determining means performs band detection for pixels in a range of pixel values from the minimum value to a quarter of the maximum value and from a range of the maximum value from a quarter to the maximum value. The offset may be determined, and the value range other than this may be determined as the edge offset.

According to the above configuration, the pixel to which the band offset is applied and the pixel to which the edge offset is applied can be clearly separated.

In order to solve the above-described problems, an image filter device according to the present invention is an image filter device that applies an adaptive offset (SAO: Sample Adaptive Offset) to an input image, and a class for applying an edge offset (EO). Class classification means for performing edge determination to determine only the pixels existing in the horizontal direction of the target pixel, and when applying edge offset (EO), the class classification means classifies the class. And a filter means for adding a corresponding offset.

According to the above configuration, since only the pixels in the horizontal direction are referred to for edge determination, the required memory amount can be reduced as compared with the case of referring to the pixels in the upward direction. In addition, since the upper boundary determination is not necessary, the processing amount can also be reduced.

In the image filter device according to the present invention, the class classification unit may perform edge determination with reference to a pixel located at a position two pixels away from the target pixel in the horizontal direction.

According to the above configuration, since a pixel two pixels away from the target pixel is referred to, the edge can be detected even when the edge angle is shallow.

In order to solve the above problems, an image filter device according to the present invention is an image filter device that applies an adaptive offset (SAO: Sample ： Adaptive Offset) to an input image, and the pixel value indicating a color difference has a maximum value and a minimum value. Class classification means for determining a class when band offset (BO) is applied by subdividing the division width of the class around the median value, which is the median value, than other value ranges, and band offset (BO) Is applied, a filter means for adding an offset corresponding to the class classified by the class classification means is provided.

According to the above configuration, the pixel value indicating the color difference is subdivided from the division value of the class around the median value that is the median value of the maximum value and the minimum value, and the band offset (BO) is reduced. Determine the class to apply.

When the pixel value of the color difference is the median value, the pixel is achromatic. Achromatic errors are easily noticeable by humans, and the subjective image quality deteriorates. Therefore, if the class width near the median is subdivided as in the above configuration, the offset can be set finely for the pixels near the median. Thereby, subjective image quality can be improved.

In the image filter device according to the present invention, when the median value exists between the pixel value before the offset addition and the pixel value after the addition, the filter means uses the pixel value after the offset addition as the median value. You may do.

に よ According to the above configuration, the offset is not added beyond the median value. A pixel whose color difference has a median value is an achromatic color, and colors that can be perceived by human eyes change on both sides of the median value. Therefore, according to the above configuration, it is possible to prevent the color perceived by a person from being changed due to the addition of the offset.

In order to solve the above problems, an image filter device according to the present invention is an image filter device that applies an adaptive offset (SAO: Sample ： Adaptive Offset) to an input image, and the pixel value indicating a color difference has a maximum value and a minimum value. It is characterized by comprising filter means for adding an offset with higher accuracy to the pixels near the median value, which is the median value, than the pixels in other value ranges.

According to the above configuration, the pixel value indicating the color difference adds an offset with higher accuracy than the pixels in the other value ranges to the pixels in the vicinity of the median value that is the median value of the maximum value and the minimum value.

When the pixel value of the color difference is the median value, the pixel is achromatic. Achromatic errors are easily noticeable by humans, and the subjective image quality deteriorates. Therefore, as in the above configuration, if the accuracy of the offset to be added is improved in the vicinity of the median value, the offset can be finely added to the pixels in the vicinity of the median value. Thereby, subjective image quality can be improved.

In order to solve the above problem, an offset decoding apparatus according to the present invention is an offset decoding apparatus that decodes each offset referenced by an image filter that adds an offset to each pixel value of an input image, and each offset residual Offset residual decoding means for decoding the encoded data from the encoded data, prediction value deriving means for deriving the predicted value of each offset from a decoded offset or a predetermined value, and each offset for deriving the predicted value And an offset calculating means for calculating from the predicted value derived by the means and the offset residual decoded by the offset residual decoding means.

According to the above configuration, since the offset is decoded from the residual, it is possible to reduce the code amount as compared with the case where the offset is encoded as it is. In addition, since the prediction value for obtaining the residual is derived from the decoded offset or a predetermined value, by using only the decoded offset, the code amount of the difference data is made as compared with the case of encoding as it is. It can be prevented from becoming large.

For example, “0” can be given as a predetermined value.

In order to solve the above problems, an offset encoding apparatus according to the present invention is an offset encoding apparatus that encodes each offset referenced by an image filter that adds an offset to each pixel value of an input image, An offset for calculating an offset residual from each offset and the predicted value derived by the predicted value deriving means, and a predicted value deriving means for deriving the predicted value of the offset from an encoded offset or a predetermined value The apparatus is characterized by comprising: a residual calculation means; and an offset residual encoding means for encoding the offset residual calculated by the offset residual calculation means.

According to the above configuration, since the offset is decoded from the residual, it is possible to reduce the code amount as compared with the case where the offset is encoded as it is. In addition, since the prediction value for obtaining the residual is derived from the decoded offset or a predetermined value, by using only the decoded offset, the code amount of the difference data is made as compared with the case of encoding as it is. It can be prevented from becoming large.

In order to solve the above problems, the data structure of encoded data according to the present invention is encoded data data referred to by an image filter that adds an offset to each pixel value of an input image composed of a plurality of unit areas. A prediction value derivation information indicating whether the prediction value is derived from a decoded offset or a predetermined value, and the image filter includes prediction value derivation information included in the encoded data The prediction value is derived and the offset is decoded with reference to FIG.

According to the above configuration, whether the predicted value is a decoded offset or a predetermined value can be determined from the predicted value derivation information.

The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and the embodiments can be obtained by appropriately combining technical means disclosed in different embodiments. The form is also included in the technical scope of the present invention.

The present invention can be suitably used for an image filter that performs offset filtering on image data. Further, the present invention can be suitably applied to a decoding device that decodes encoded data and an encoding device that encodes encoded data.

1 Video decoding device (offset decoding device)
2 Video encoding device (offset encoding device)
60 Adaptive offset filter (image filter device)
80 Adaptive offset filter (image filter device)
623 Offset type deriving unit (offset determining means)
624 class classification unit (class classification means)
625 Offset deriving unit (offset calculating means)
626 Offset adder (filter means)
651 Offset residual decoding unit (offset residual decoding means)
653 Predicted value deriving unit (preliminary value deriving means)

Claims (12)

1. An image filter device that adds an offset selected from a plurality of offsets to each pixel value of an input image composed of a plurality of unit regions,
   Offset determining means for determining, for each unit region, an offset to be added to the pixel value of the pixel included in each unit region;
   Filter means for adding the offset determined by the offset determination means to the pixel value of the pixel included in the unit region,
   The offset determining means is a set of offsets in which the number of offsets that can be selected is limited when the size of the unit area for which the offset is determined is smaller than a predetermined size, compared to when the size of the unit area is equal to or larger than the predetermined size. An image filter device that determines an offset to be added to a pixel value of a pixel included in the unit area from among the image filter devices.
2. An image filter device that adds an offset to each pixel value of an input image composed of a plurality of unit regions,
   Offset determining means for determining, for each unit region, an offset to be added to the pixel value of the pixel included in each unit region;
   Filter means for adding the offset determined by the offset determination means to the pixel value of the pixel included in the unit region,
   When the size of the unit area for which the offset is determined is smaller than a predetermined size, the filter means adds an offset having a coarser accuracy than when the size of the unit area is equal to or larger than the predetermined size. Image filter device.
3. An image filter device that applies an adaptive offset (SAO: Sample Adaptive Offset) to each pixel value of an input image composed of a plurality of unit regions,
   Offset determining means for determining, for each unit region, the type of offset to be added to the pixel value of the pixel included in each unit region;
   Filter means for adding an offset according to the type of offset determined by the offset determination means to the pixel value of the pixel included in the unit region;
   The offset determination means determines the offset type to be a band offset (BO) for pixels whose pixel values are near the maximum value and the minimum value for the target pixel to which the offset is added, An image filter device, wherein an edge offset (EO) is determined for a pixel.
4. The offset determination means determines a band offset for pixels in the range of pixel values from the minimum value to one-fourth of the maximum value and from the third-fourth of the maximum value to the maximum value, and other value ranges The image filter device according to claim 3, wherein the edge offset is determined as an edge offset.
5. An image filter device for applying an adaptive offset (SAO: Sample Adaptive Offset) to an input image,
   Class classification means for performing edge determination to determine a class when applying an edge offset (EO) with reference to only pixels existing in the horizontal direction of the target pixel;
   An image filter device comprising: filter means for adding an offset corresponding to the class classified by the class classification means when applying edge offset (EO).
6. 6. The image filter device according to claim 5, wherein the class classification means determines an edge with reference to a pixel located at a position two pixels away from the target pixel in the horizontal direction.
7. An image filter device for applying an adaptive offset (SAO: Sample Adaptive Offset) to an input image,
   The division value of the class around the median, where the pixel value indicating the color difference is the median of the maximum value and the minimum value, is subdivided from the other range, and the class for applying the band offset (BO) is determined. Classifying means to
   An image filter device comprising: a filter unit that adds an offset corresponding to a class classified by the class classification unit when a band offset (BO) is applied.
8. 8. The filter unit according to claim 7, wherein when the median value exists between a pixel value before offset addition and a pixel value after addition, the pixel value after offset addition is used as the median value. The image filter device described in 1.
9. An image filter device for applying an adaptive offset (SAO: Sample Adaptive Offset) to an input image,
   The pixel value indicating the color difference is provided with filter means for adding an offset with higher accuracy than a pixel in the other value range to a pixel in the vicinity of the median value that is the median value of the maximum value and the minimum value. Image filter device.
10. An offset decoding device that decodes each offset referenced by an image filter that adds an offset to each pixel value of an input image,
    Offset residual decoding means for decoding each offset residual from the encoded data;
    Predicted value deriving means for deriving the predicted value of each offset from a decoded offset or a predetermined value;
    An offset decoding apparatus comprising: an offset calculating unit that calculates each offset from a predicted value derived by the predicted value deriving unit and an offset residual decoded by the offset residual decoding unit .
11. An offset encoding device that encodes each offset referenced by an image filter that adds an offset to each pixel value of an input image,
    A predicted value deriving unit for deriving a predicted value of each offset from an encoded offset or a predetermined value;
    Offset residual calculating means for calculating an offset residual from each offset and the predicted value derived by the predicted value deriving means;
    An offset encoding apparatus comprising: an offset residual encoding unit that encodes the offset residual calculated by the offset residual calculation unit.
12. A data structure of encoded data referred to by an image filter that adds an offset to each pixel value of an input image composed of a plurality of unit regions,
    Including predicted value derivation information indicating whether the predicted value is derived from a decoded offset or a predetermined value;
    A data structure of encoded data, wherein the image filter refers to predicted value derivation information included in the encoded data, derives a predicted value, and decodes an offset.

Images