WO2012081706A1

WO2012081706A1 - Image filter device, filter device, decoder, encoder, and data structure

Info

Publication number: WO2012081706A1
Application number: PCT/JP2011/079232
Authority: WO
Inventors: 知宏猪飼; 隆紀山崎
Original assignee: シャープ株式会社
Priority date: 2010-12-17
Filing date: 2011-12-16
Publication date: 2012-06-21

Abstract

An adaptive filter (17) is provided with: a region characteristic calculation unit (171) which calculates characteristic values representing the image characteristics of deblocked decoded images (P_DB) for each unit region; and a region classification unit (172) which classifies each unit area into either a filter-on region in which a filter processing unit (173) is made to operate, or a filter-off region in which the filter processing unit (173) is not made to operate, according to the characteristics values of the unit region.

Description

Image filter device, filter device, decoding device, encoding device, and data structure

The present invention relates to an image filter device and a filter device that perform image filtering. The present invention also relates to an image filter device and an encoding device and a decoding device including the filter device. The present invention also relates to a data structure of encoded data decoded by such a decoding device.

In order to efficiently transmit or record a moving image, a moving image encoding device (encoding device) that generates encoded data by encoding the moving image, and decoding by decoding the encoded data A video decoding device (decoding device) that generates an image is used. As a specific moving picture encoding method, for example, H.264 is used. H.264 / MPEG-4. AVC (Non-Patent Document 1), VCEG (Video Coding Expert Group) adopted by KTA software, which is a codec for joint development, and TMuC (Test Model Under Consideration) software, a successor codec. (Non-patent Document 2).

In such a coding system, an image (picture) that constitutes a moving image is a slice obtained by dividing the image, a maximum coding unit (LCU: Largegest Coding Unit) obtained by dividing the slice, It is managed by a coding unit (CU: Coding Unit) obtained by dividing the coding unit and a hierarchical structure consisting of blocks and partitions obtained by dividing the coding unit. In many cases, the block is minimized. Encoded as a unit.

In such an encoding method, a predicted image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and a difference image (“residual” between the predicted image and the input image is generated. (Sometimes referred to as “difference image” or “prediction residual”). As methods for generating a predicted image, methods called inter-screen prediction (inter prediction) and intra-screen prediction (intra prediction) are known.

In inter prediction, a prediction image in a prediction target frame is generated for each prediction unit by applying motion compensation using a motion vector to a reference image in a reference frame (decoded image) obtained by decoding the entire frame. Is done. On the other hand, in intra prediction, based on locally decoded images in the same frame, predicted images in the frame are sequentially generated.

In Patent Document 1, for each pixel value of a decoded image, the decoded image is classified into a plurality of regions based on the degree of activity (Activity) indicating the randomness of the image around the pixel. An adaptive loop filter (hereinafter also simply referred to as “adaptive filter”) that generates a filtered decoded image by performing filter processing while switching a filter coefficient group for each region is disclosed. Here, the filter coefficient group used for the adaptive filter is adaptively determined so as to minimize an error between the encoding target image and the filtered decoded image obtained by applying the adaptive filter to the decoded image. It has been done.

The encoding device and decoding device provided with such an adaptive filter can improve prediction accuracy and encoding efficiency by generating a prediction image with reference to the filtered decoded image.

Further, Non-Patent Document 3 divides a (local) decoded image into a plurality of regions, and according to the degree of activity (Activity) indicating local randomness of the (local) decoded image in each region, An adaptive loop filter (Adaptive Loop 行う Filter) that performs filter processing while switching filter coefficient groups for each region is disclosed. Here, the filter coefficient group for each region is determined by the encoding device so as to minimize the error between the filtered image in the region and the encoding target image.

An encoding device and a decoding device provided with such an adaptive filter generate a prediction image with reference to a filtered decoded image obtained by performing filter processing on the (local) decoded image using the adaptive filter. As a result, it is possible to improve the prediction accuracy and the encoding efficiency.

International Publication No. 2010/083438 (released July 22, 2010)

However, when the adaptive filters described in Patent Document 1 and Non-Patent Document 3 are used, there is a problem that the encoding efficiency is not improved as expected depending on the image characteristics of the encoding target image.

Regarding this problem, the inventor has obtained an experimental result that, for an area having a certain range of activity, the coding efficiency is rather lowered by applying an adaptive filter.

Also, as a method for improving the coding efficiency, a method of switching the filter coefficient group for each region based on region characteristics other than the activity in addition to the activity may be considered. However, when the filter coefficient group is switched for each region based on the two region characteristics, there is a problem that the calculation cost on the encoding device side and the decoding device side becomes high.

The present invention has been made in view of the above-described problems, and an object of the present invention is to provide an image filter device capable of improving encoding efficiency as compared with the prior art, and a decoding device including such an image filter device, and It is to realize an encoding device.

In view of the above problems, the inventor has obtained knowledge that encoding efficiency can be improved by switching on / off of filter processing according to local image characteristics in a decoded image.

In order to solve the above problem, an image filter device according to the present invention includes a filter unit that operates on an input image composed of a plurality of unit regions, and an image characteristic of the input image for each of the plurality of unit regions. A characteristic value calculating means for calculating a characteristic value representing a filter on area in which each of the unit areas is made to act on the filter means in accordance with the value of the characteristic value for the unit area calculated by the characteristic value calculating means, Alternatively, a classification unit that classifies the filter unit into any one of the filter-off regions where the filter unit does not act is provided.

According to the above configuration, the classification means determines each unit area constituting the input image according to the value of the characteristic value representing the image characteristic of the input image in the unit area, and Classify into any of the filter-off areas. The filter means acts on the input image in the unit area belonging to the filter-on area, but does not act on the input image in the unit area belonging to the filter-off area.

As described above, according to the image filter device according to the present invention configured as described above, it is possible to switch on / off of the filter process according to the local image characteristics of the input image. Therefore, by using the output image obtained by the image filter device acting on the decoded image while switching on / off of the filtering process as a reference image for generating the predicted image, the prediction accuracy and the coding efficiency are improved as compared with the conventional case. Can be improved.

When the image filter device is used in an encoding device that encodes an image and a decoding device that decodes an image from encoded data, the unit region is, for example, a unit for generating a predicted image. It may be a certain prediction unit (partition), may be a conversion unit that is a unit of frequency conversion, or may be other than those. For example, the unit area may be a unit composed of only one pixel.

Further, the data structure of the encoded data according to the present invention includes a filter unit that operates on an input image composed of a plurality of unit areas, and a characteristic value that represents an image characteristic of the input image for each of the plurality of unit areas. The characteristic value calculation means for calculating, and the filter-on region in which the filter means is activated or the filter means is not activated in accordance with the value of the characteristic value for each unit area calculated by the characteristic value calculation means. A data structure of encoded data that is referred to by an image filter device including classification means for classifying into any of the filter-off regions, and includes a filter coefficient group used by the filter means It is a feature.

The decoding apparatus that refers to the encoded data configured as described above represents filter means that operates on an input image composed of a plurality of unit regions, and represents the image characteristics of the input image for each of the plurality of unit regions. A characteristic value calculating means for calculating a characteristic value, and a filter-on area or the filter means for causing the filter means to act on each unit area according to the value of the characteristic value for the unit area calculated by the characteristic value calculating means. And classifying means for classifying it into any one of the filter-off regions that do not act on the filter. The filter means performs filter processing using a filter coefficient group included in the encoded data. Therefore, such a decoding apparatus can generate a prediction image with high prediction accuracy.

Also, since the encoded data referred to by such a decoding apparatus does not need to include a flag for switching on / off of the filtering process, the code amount of the encoded data is reduced. Therefore, the encoded data is encoded data with high encoding efficiency.

In the image filter device according to the present invention, for each of a plurality of unit regions constituting the input image, a calculation unit that calculates an average of pixel values of the input image, and each unit region is calculated by the calculation unit. According to the average of the pixel values for the unit area, the classification means for classifying into any of a plurality of types, and the pixel value of each pixel in each unit area are set for the type including the pixel And filtering means for calculating using a filter coefficient group.

According to the image filter device configured as described above, each unit area is classified into one of a plurality of types according to the average of the pixel values for the unit area calculated by the calculation unit, The pixel value of each pixel in the unit area is calculated using the filter coefficient group set for the type including the pixel. Therefore, according to the above configuration, the filter processing can be performed using the filter coefficient group set for each type, so that the encoding efficiency is improved. Further, according to the above configuration, each unit area is classified according to a characteristic value with a small calculation processing amount, ie, an average of pixel values, so that the processing amount can be reduced.

In order to solve the above-described problem, a filter device according to the present invention includes a region characteristic for calculating a region characteristic value of each unit region in a filter device including a linear filter in which filter coefficients are adaptively set. The linear filter includes an offset that depends on the region characteristic for performing a filter operation on each pixel belonging to each unit region, and the region characteristic of the unit region calculated by the region characteristic calculating unit This is performed by using an offset corresponding to the value of and a filter coefficient group that does not depend on the region characteristics.

According to the above configuration, it is possible to achieve higher encoding efficiency than a conventional adaptive filter that performs a filter operation using a certain offset.

In order to solve the above problem, a filter device according to the present invention includes a linear filter in which filter coefficient groups are adaptively set, and includes a target pixel in an output image of the linear filter. Truncation means is provided for truncating the change amount, which is the difference between the pixel value and the pixel value of the target pixel in the input image of the linear filter.

According to the above configuration, it is possible to effectively avoid a situation in which the error of the decoded image with respect to the original image becomes larger by applying the adaptive filter to the decoded image. Thereby, encoding efficiency higher than before can be realized.

In order to solve the above problem, in a filter device including a linear filter in which filter coefficients are adaptively set, at least one first filter coefficient group depending on the first region characteristic is associated. The first filter coefficient table recorded in association with the value of the first area characteristic and the second filter coefficient group depending on the second area characteristic are recorded in association with the value of the corresponding second area characteristic. Decoding means for decoding the second filter coefficient table, and area characteristic calculation means for calculating the values of the first area characteristic and the second area characteristic of each unit area, and the linear filter includes each of the unit areas belonging to each unit area. A filter operation using a pixel as a target pixel, and the unit region calculated by the region characteristic calculation unit in the first filter coefficient group recorded in the first filter coefficient table; Among the first filter coefficient group corresponding to the value of the first area characteristic and the second filter coefficient group recorded in the second filter coefficient table, the second of the unit area calculated by the area characteristic calculating means. A filter operation using the second filter coefficient group corresponding to the value of the region characteristic is executed.

According to the above configuration, each unit area can be obtained without decoding a filter coefficient table in which a plurality of filter coefficient groups depending on both the first area characteristic and the second area characteristic are recorded in association with the corresponding area characteristic pair. The filter operation according to both the first region characteristic and the second region characteristic can be performed. That is, according to the above configuration, a decoded image close to the original image can be generated with high accuracy without causing a significant increase in the code amount. As a result, higher encoding efficiency than conventional can be realized.

The decoding device and the encoding device provided with the filter device are also included in the scope of the present invention.

As described above, the image filter device according to the present invention includes a filter unit that operates on an input image composed of a plurality of unit areas, and a characteristic value that represents an image characteristic of the input image for each of the plurality of unit areas. A characteristic value calculation means for calculating the filter area, and a filter-on area that causes the filter means to act on each unit area according to the value of the characteristic value for the unit area calculated by the characteristic value calculation means, or the filter Classification means for classifying the filter into any one of the filter-off regions in which the means is not operated.

According to the image filter device, the encoding efficiency is improved as compared with the conventional case.

It is a block diagram which shows the structure of the adaptive filter with which the moving image decoding apparatus which concerns on the 1st Embodiment of this invention is provided. It is a figure which shows the data structure of the encoding data which are produced | generated by the moving image encoder which concerns on the 1st Embodiment of this invention, and are referred by the moving image decoder which concerns on this Embodiment. (A) shows the configuration of the picture layer of the encoded data, (b) shows the configuration of the slice layer included in the picture layer, and (c) shows the LCU layer included in the slice layer. (D) shows the configuration of the leaf CU included in the CU layer, (e) shows the configuration of the inter prediction information for the leaf CU, ( f) shows the configuration of the intra prediction information for the leaf CU, and (g) shows the configuration of the filter parameter included in the slice header. It is a block diagram which shows the structure of the moving image decoding apparatus which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the filter process by the adaptive filter with which the moving image decoding apparatus which concerns on the 1st Embodiment of this invention is provided. (A) shows a characteristic value calculation reference region CRR composed of pixels included in the target unit region UR and pixels adjacent to the target unit region UR, and (b) is included in the target unit region UR. A characteristic value calculation reference region CRR composed of pixels is shown. It is a graph which shows the experimental data obtained by the inventor. (A) shows the square error reduction amount with respect to the activity for each region for a plurality of sequences, and (b) shows the average (“average” for the plurality of sequences for the square error reduction amount at each activity. )) And the total number of unit areas (“count”) having each degree of activity. It is a table | surface which shows the example of a classification process by the adaptive filter which concerns on the 1st Embodiment of this invention. FIG. 6A is a diagram illustrating unit areas set by the adaptive filter according to the first embodiment of the present invention, and FIG. 6A shows the position and size of a unit area included in a target coding unit according to a CU partition flag. (B) is a diagram showing a case where a unit area included in the target unit area has a predetermined size. It is a figure for demonstrating the filter process by the adaptive filter which concerns on the 1st Embodiment of this invention, Comprising: (a) calculates the pixel value of the filter object pixel in the object unit area | region UR and the object unit area | region UR. FIG. 8 is a diagram illustrating a filter reference region R that is a set of pixels referred to for the purpose and a filter reference range RA that is defined as a union of the filter reference regions R for each filter target pixel. FIG. 4 is a diagram showing filter coefficients assigned to each pixel included in a reference region R. It is a table | surface which shows the example of a classification process by the adaptive filter which concerns on the 1st modification of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the adaptive filter which concerns on the 2nd modification of the 1st Embodiment of this invention. It is a graph which shows the experimental data obtained by the inventor. (A) shows a square error reduction amount with respect to an average pixel value for each region for a plurality of sequences, and (b) shows an average (for a plurality of sequences) for a square error reduction amount at each average pixel value. “Average”) and the total number of unit areas (“count”) having each average pixel value. It is a table | surface which shows the example of a classification process by the adaptive filter which concerns on the 2nd modification of the 1st Embodiment of this invention. It is a table | surface which shows the example of a classification process by the adaptive filter which concerns on the 3rd modification of the 1st Embodiment of this invention. It is a figure which shows a part of syntax contained in the filter parameter which the adaptive filter which concerns on the 4th modification of the 1st Embodiment of this invention refers. It is a figure for demonstrating the filter process by the adaptive filter which concerns on the 4th modification of the 1st Embodiment of this invention. (A) to (d) show each coding unit to be subjected to on / off control in the maximum coding unit when the value of the syntax alf_cu_control_max_depth is 0 to 3, together with a branch diagram representing a hierarchical structure. ing. It is a figure for demonstrating the classification process by the adaptive filter which concerns on the 4th modification of the 1st Embodiment of this invention. (A) shows an example of classification processing by the adaptive filter, and (b) shows the position and size of the unit area included in the maximum coding unit target coding unit hierarchically specified by the CU partitioning flag. (C) shows a case where the unit area included in the target unit area has a predetermined size. It is a block diagram which shows the structure of the adaptive filter which concerns on the 5th modification of the 1st Embodiment of this invention. It is a table | surface which shows the example of a classification process by the adaptive filter which concerns on the 5th modification of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the adaptive filter which concerns on the 6th modification of the 1st Embodiment of this invention. It is a table | surface which shows the classification processing example by the adaptive filter which concerns on the 6th modification of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the moving image encoder which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the adaptive filter with which the moving image encoder which concerns on the 1st Embodiment of this invention is provided. It is a block diagram which shows the structure of the adaptive filter which concerns on the 2nd modification of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the adaptive filter which concerns on the 5th modification of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the adaptive filter which concerns on the 6th modification of the 1st Embodiment of this invention. It is a block diagram which shows the 1st structural example of the adaptive filter which concerns on the 2nd Embodiment of this invention. FIG. 27 is a block diagram illustrating a configuration example of an adaptive filter suitable for mounting on an encoding device when the adaptive filter of FIG. 26 is mounted on a decoding device. It is a block diagram which shows the 2nd structural example of the adaptive filter which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the adaptive filter suitable for mounting in an encoding apparatus, when mounting the adaptive filter of FIG. 28 in a decoding apparatus. It is a block diagram which shows the 3rd structural example of the adaptive filter which concerns on the 2nd Embodiment of this invention. [Fig. 31] Fig. 31 is a block diagram illustrating a configuration example of an adaptive filter suitable for mounting on an encoding device when the adaptive filter of Fig. 30 is mounted on a decoding device. It is a block diagram which shows the 4th structural example of the adaptive filter which concerns on the 2nd Embodiment of this invention. [Fig. 30] Fig. 30 is a block diagram illustrating a configuration example of an adaptive filter suitable for mounting on an encoding device when the adaptive filter of Fig. 29 is mounted on a decoding device. It is a block diagram which shows the 1st structural example of the adaptive filter which concerns on the 3rd Embodiment of this invention. It is a figure which shows the reference area of the 1st spatial filter with which the adaptive filter of FIG. 34 is provided. It is a figure which shows the reference area of the 2nd spatial filter with which the adaptive filter of FIG. 34 is provided. It is a figure which shows another reference area | region of the 1st spatial filter with which the adaptive filter of FIG. 34 is provided. It is a figure which shows another reference area | region of the 2nd spatial filter with which the adaptive filter of FIG. 34 is provided. FIG. 43 is a block diagram illustrating a configuration example of an adaptive filter suitable for mounting on an encoding device when the adaptive filter of FIGS. 34 and 42 is mounted on a decoding device. It is a block diagram which shows the 2nd structural example of the adaptive filter which concerns on the 3rd Embodiment of this invention. [Fig. 41] Fig. 41 is a block diagram illustrating a configuration example of an adaptive filter suitable for mounting on an encoding device when the adaptive filter of Fig. 40 is mounted on a decoding device. It is a block diagram which shows the 3rd structural example of the adaptive filter which concerns on the 3rd Embodiment of this invention. FIG. 43 is a diagram illustrating a reference region of an enlarged linear filter included in the adaptive filter of FIG. 42. FIG. 43 is a diagram illustrating another reference region of the enlarged linear filter included in the adaptive filter of FIG. 42. It is a block diagram which shows the structural example of the adaptive filter which concerns on the 4th Embodiment of this invention. FIG. 46 is a block diagram illustrating a configuration example of an adaptive filter suitable for mounting on an encoding device when the adaptive filter of FIG. 45 is mounted on a decoding device. It is a figure which shows the data structure of the coding data produced | generated by the moving image encoder shown in FIG. 49, and decoded by the moving image decoder shown in FIG. It is the block diagram which showed the structure for the moving image decoding apparatus suitable for mounting of the adaptive filter which concerns on each embodiment. It is the block diagram which showed the structure of the moving image encoder suitable for mounting of the adaptive filter which concerns on each embodiment. It is a figure which shows the modification of the coding data shown in FIG. It is the figure shown about the structure of the transmitter which mounts the said moving image encoder, and the receiver which mounts the said moving image decoder. (A) shows a transmitting apparatus equipped with a moving picture coding apparatus, and (b) shows a receiving apparatus equipped with a moving picture decoding apparatus. It is the figure shown about the structure of the recording device which mounts the said moving image encoder, and the reproducing | regenerating apparatus which mounts the said moving image decoder. (A) shows a recording apparatus equipped with a moving picture coding apparatus, and (b) shows a reproduction apparatus equipped with a moving picture decoding apparatus.

<< Embodiment 1 >>
A first embodiment of a decoding device and an encoding device according to the present invention will be described below with reference to the drawings. Note that the decoding apparatus according to the present embodiment decodes a moving image from encoded data. Therefore, hereinafter, this is referred to as “moving image decoding apparatus”. In addition, the encoding device according to the present embodiment generates encoded data by encoding a moving image. Therefore, in the following, this is referred to as a “video encoding device”.

However, the scope of application of the present invention is not limited to this. That is, as will be apparent from the following description, the features of the present invention can be realized without assuming a plurality of frames. That is, the present invention can be applied to a general decoding apparatus and a general encoding apparatus regardless of whether the target is a moving image or a still image.

(Configuration of encoded data # 1)
Prior to the description of the moving picture decoding apparatus 1 according to the present embodiment, the configuration of the encoded data # 1 generated by the moving picture encoding apparatus 2 according to the present embodiment and decoded by the moving picture decoding apparatus 1 will be described with reference to FIG. Will be described with reference to FIG. The encoded data # 1 has a hierarchical structure including a sequence layer, a GOP (Group Of Pictures) layer, a picture layer, a slice layer, and a maximum coding unit (LCU: Large Coding Unit) layer.

FIG. 2 shows the hierarchical structure below the picture layer in the encoded data # 1. FIGS. 2A to 2F are a picture layer P, a slice layer S, an LCU layer LCU, a leaf CU included in the LCU (denoted as CUL in FIG. 2D), and inter prediction (inter-screen prediction), respectively. It is a figure which shows the structure of inter prediction information PI_Inter which is the prediction information PI about a partition, and intra prediction information PI_Intra which is the prediction information PI about an intra prediction (prediction in a screen) partition.

(Picture layer)
The picture layer P is a set of data that is referenced by the video decoding device 1 in order to decode a target picture that is a processing target picture. As shown in FIG. 2A, the picture layer P includes a picture header PH and slice layers S1 to SNs (Ns is the total number of slice layers included in the picture layer P).

The picture header PH includes a coding parameter group referred to by the video decoding device 1 in order to determine a decoding method of the target picture. For example, the encoding mode information (entropy_coding_mode_flag) indicating the variable length encoding mode used in encoding by the moving image encoding device 2 is an example of an encoding parameter included in the picture header PH.

(Slice layer)
Each slice layer S included in the picture layer P is a set of data referred to by the video decoding device 1 in order to decode a target slice that is a slice to be processed. As shown in FIG. 2B, the slice layer S includes a slice header SH and LCU layers LCU1 to LCUn (Nc is the total number of LCUs included in the slice S).

The slice header SH includes a coding parameter group that the moving image decoding apparatus 1 refers to in order to determine a decoding method of the target slice. Slice type designation information (slice_type) for designating a slice type is an example of an encoding parameter included in the slice header SH.

As slice types that can be specified by the slice type specification information, (1) I slice that uses only intra prediction at the time of encoding, (2) P slice that uses unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding may be used.

Also, the slice header SH includes a filter parameter FP that is referred to by an adaptive filter included in the video decoding device 1. The configuration of the filter parameter FP will be described later and will not be described here.

(LCU layer)
Each LCU layer LCU included in the slice layer S is a set of data that the video decoding device 1 refers to in order to decode the target LCU that is the processing target LCU.

The LCU layer LCU is composed of a plurality of coding units (CU: Coding Units) obtained by hierarchically dividing the LCU into a quadtree. In other words, the LCU layer LCU is a coding unit corresponding to the highest level in a hierarchical structure that recursively includes a plurality of CUs. As shown in FIG. 2C, each CU included in the LCU layer LCU has a hierarchical structure that recursively includes a CU header CUH and a plurality of CUs obtained by dividing the CU into quadtrees. is doing.

The size of each CU excluding the LCU is half the size of the CU to which the CU directly belongs (that is, the CU one layer higher than the CU), and the size that each CU can take is encoded data # 1. Dependent on the size and hierarchical depth of the LCU included in the sequence parameter set SPS. For example, when the size of the LCU is 128 × 128 pixels and the maximum hierarchical depth is 5, the CUs in the hierarchical level below the LCU have five sizes, that is, 128 × 128 pixels and 64 × 64 pixels. , 32 × 32 pixels, 16 × 16 pixels, and 8 × 8 pixels. A CU that is not further divided is called a leaf CU.

(CU header)
The CU header CUH includes a coding parameter referred to by the video decoding device 1 in order to determine a decoding method of the target CU. Specifically, as shown in FIG. 2C, a CU division flag SP_CU that specifies whether or not the target CU is further divided into four subordinate CUs is included. When the CU division flag SP_CU is 0, that is, when the CU is not further divided, the CU is a leaf CU.

(Leaf CU)
A CU (CU leaf) that is not further divided is handled as a prediction unit (PU: Prediction Unit) and a transform unit (TU: Transform Unit).

As shown in FIG. 2 (d), the leaf CU (denoted as CUL in FIG. 2 (d)) includes (1) PU information PUI that is referred to when the moving image decoding apparatus 1 generates a predicted image, and (2) The TU information TUI that is referred to when the residual data is decoded by the moving picture decoding apparatus 1 is included.

The skip flag SKIP is a flag indicating whether or not the skip mode is applied to the target PU. When the value of the skip flag SKIP is 1, that is, when the skip mode is applied to the target leaf, PU information PUI and TU information TUI in the leaf CU are omitted. Note that the skip flag SKIP is omitted for the I slice.

The PU information PUI includes a skip flag SKIP, prediction type information PT, and prediction information PI as shown in FIG. The prediction type information PT is information that specifies whether intra prediction or inter prediction is used as a predicted image generation method for the target leaf CU (target PU). The prediction information PI includes intra prediction information PI_Intra or inter prediction information PI_Inter depending on which prediction method is specified by the prediction type information PT. Hereinafter, a PU to which intra prediction is applied is also referred to as an intra PU, and a PU to which inter prediction is applied is also referred to as an inter PU.

The PU information PUI includes information specifying the shape and size of each partition included in the target PU and the position in the target PU. Here, the partition is one or a plurality of non-overlapping areas constituting the target leaf CU, and the generation of the predicted image is performed in units of partitions.

As shown in FIG. 2D, the TU information TUI specifies a quantization parameter difference Δqp (tu_qp_delta) that specifies the magnitude of the quantization step, and a division pattern for each block of the target leaf CU (target TU). TU partition information SP_TU and quantized prediction residuals QD1 to QDNT (NT is the total number of blocks included in the target TU) are included.

The quantization parameter difference Δqp is a difference qp−qp ′ between the quantization parameter qp in the target TU and the quantization parameter qp ′ in the TU encoded immediately before the TU.

TU partition information SP_TU is information that specifies the shape and size of each block included in the target TU and the position in the target TU. Each TU can be, for example, a size from 64 × 64 pixels to 2 × 2 pixels. Here, the block is one or a plurality of non-overlapping areas constituting the target leaf CU, and encoding / decoding of the prediction residual is performed in units of blocks.

Each quantized prediction residual QD is encoded data generated by the moving image encoding apparatus 2 performing the following processes 1 to 3 on a target block that is a processing target block. Process 1: DCT transform (Discrete Cosine Transform) is performed on the prediction residual obtained by subtracting the prediction image from the encoding target image. Process 2: The DCT coefficient obtained in Process 1 is quantized. Process 3: The DCT coefficient quantized in Process 2 is variable length encoded. The quantization parameter qp described above represents the magnitude of the quantization step QP used when the moving picture coding apparatus 2 quantizes the DCT coefficient (QP = 2 ^{qp / 6} ).

(Inter prediction information PI_Inter)
The inter prediction information PI_Inter includes a coding parameter that is referred to when the video decoding device 1 generates an inter prediction image by inter prediction. As shown in FIG. 2 (e), the inter prediction information PI_Inter includes inter PU partition information SP_Inter that specifies a partition pattern for each partition of the target PU, and inter prediction parameters PP_Inter1 to PP_InterNe (Ne for each partition). The total number of inter prediction partitions included in the target PU).

Specifically, the inter-PU partition information SP_Inter is information for designating the shape and size of each inter prediction partition included in the target PU (inter PU) and the position in the target PU.

The inter PU is composed of four symmetric splittings of 2N × 2N pixels, 2N × N pixels, N × 2N pixels, and N × N pixels, and 2N × nU pixels, 2N × nD pixels, and nL × 2N. It is possible to divide into 8 types of partitions in total by four asymmetric splits of pixels and nR × 2N pixels. Here, the specific value of N is defined by the size of the CU to which the PU belongs, and the specific values of nU, nD, nL, and nR are determined according to the value of N. For example, an inter PU of 128 × 128 pixels is 128 × 128 pixels, 128 × 64 pixels, 64 × 128 pixels, 64 × 64 pixels, 128 × 32 pixels, 128 × 96 pixels, 32 × 128 pixels, and 96 × It is possible to divide into 128-pixel inter prediction partitions.

(Inter prediction parameter PP_Inter)
As illustrated in FIG. 2E, the inter prediction parameter PP_Inter includes a reference image index RI, an estimated motion vector index PMVI, and a motion vector residual MVD.

The motion vector residual MVD is encoded data generated by the moving image encoding device 2 executing the following processes 4 to 6. Process 4: Select an encoded / decoded locally decoded image (more precisely, an image obtained by performing deblocking processing and adaptive filtering on the encoded / decoded local decoded image) The motion vector mv for the target partition is derived with reference to the selected encoded / decoded local decoded image (hereinafter also referred to as “reference image”). Process 5: An estimation method is selected, and an estimated value (hereinafter also referred to as “estimated motion vector”) pmv of the motion vector mv assigned to the target partition is derived using the selected estimation method. Process 6: The motion vector residual MVD obtained by subtracting the estimated motion vector pmv derived in Process 5 from the motion vector mv derived in Process 4 is encoded.

The reference image index RI designates the locally decoded image (reference image) that has been encoded / decoded selected in the process 4. The estimated motion vector index PMVI described above is the estimation method selected in the process 5. Is specified. The estimation methods that can be selected in the processing 5 include: (1) a locally decoded image being encoded / decoded (more precisely, a region that has already been decoded in a locally decoded image being encoded / decoded). In an image obtained by performing block processing and adaptive filter processing), a median of a motion vector allocated to a partition adjacent to the target partition (hereinafter also referred to as “adjacent partition”) is used as an estimated motion vector pmv. (2) In a locally decoded image that has been encoded / decoded, a motion vector assigned to a partition (often referred to as a “collocated partition”) occupying the same position as the target partition is used as an estimated motion vector pmv, etc. Is mentioned.

Note that, as shown in FIG. 2E, the prediction parameter PP related to the partition for which unidirectional prediction is performed includes one reference image index RI, one estimated motion vector index PMVI, and one motion vector residual MVD. However, the prediction parameters PP for a partition that performs bi-directional prediction (weighted prediction) include two reference image indexes RI1 and RI2, two estimated motion vector indexes PMVI1 and PMVI2, and two motion vector residuals MVD1. And MVD2.

(Intra prediction information PI_Intra)
The intra prediction information PI_Intra includes an encoding parameter that is referred to when the video decoding device 1 generates an intra predicted image by intra prediction. As shown in FIG. 2 (f), the intra prediction information PI_Intra includes intra PU partition information SP_Intra that specifies a partition pattern of the target PU (intra PU) into each partition, and intra prediction parameters PP_Intra1 to PP_IntraNa for each partition. (Na is the total number of intra prediction partitions included in the target PU).

Specifically, the intra-PU partition information SP_Intra is information that specifies the shape and size of each intra-predicted partition included in the target PU, and the position in the target PU. The intra PU split information SP_Intra includes an intra split flag (intra_split_flag) that specifies whether or not the target PU is split into partitions. If the intra partition flag is 1, the target PU is divided symmetrically into four partitions. If the intra partition flag is 0, the target PU is not divided and the target PU itself is one partition. Are treated as Therefore, if the size of the target PU is 2N × 2N pixels, the intra prediction partition can take any of 2N × 2N pixels (no division) and N × N pixels (four divisions) (where, N = 2 ⁿ , n is an arbitrary integer of 1 or more). For example, a 128 × 128 pixel intra PU can be divided into 128 × 128 pixel and 64 × 64 pixel intra prediction partitions.

(Intra prediction parameter PP_Intra)
As shown in FIG. 2F, the intra prediction parameter PP_Intra includes an estimation flag MPM and a residual prediction mode index RIPM. The intra prediction parameter PP_Intra is a parameter for designating an intra prediction method (prediction mode) for each partition.

The estimation flag MPM is a flag indicating whether or not the prediction mode estimated based on the prediction mode allocated to the peripheral partition of the target partition that is the processing target is the same as the prediction mode for the target partition. . Here, examples of partitions around the target partition include a partition adjacent to the upper side of the target partition and a partition adjacent to the left side of the target partition.

The residual prediction mode index RIPM is an index included in the intra prediction parameter PP_Intra when the estimated prediction mode and the prediction mode for the target partition are different, and is an index for designating a prediction mode assigned to the target partition. It is.

(Filter parameter FP)
As described above, the slice header SH includes the filter parameter FP that is referred to by the adaptive filter included in the video decoding device 1. Here, as will be described later, the adaptive filter included in the video decoding device 1 classifies each of one or a plurality of unit areas included in the target slice or the target LCU into one of a plurality of types, and the unit area Is switched according to the type to which the unit area belongs. Further, the adaptive filter process is performed on the unit area for which the adaptive filter process is set to ON using the filter coefficient group selected according to the type to which the unit area belongs.

Here, the unit region is any of a slice, an LCU, a CU (other than a leaf CU), a leaf CU (a prediction unit PU, a transform unit TU), a partition that configures a prediction unit PU, and a block that configures a transform unit TU. It may be a block of 4 × 4 pixels, 8 × 8 pixels, etc. different from the CU, etc., or may be an area composed of only one pixel, or other than those. May be.

Fig. 2 (g) shows the data structure of the filter parameter FP. As shown in FIG. 2G, the filter parameter FP includes a filter coefficient group n (n is a filter number for identifying the filter coefficient groups from each other, and n = 1,..., Nf. Nf represents the total number of filter coefficient groups included in the filter parameter FP).

In addition, the above-described unit area is any one of a slice, an LCU, a CU (other than a leaf CU), a leaf CU (a prediction unit PU, a transform unit TU), a partition constituting the prediction unit PU, and a block constituting the transform unit TU. Otherwise, the filter parameter FP can be configured to include area designation information for designating the position and size of each unit area in the target slice or the target LCU. The adaptive filter included in the moving image decoding apparatus 1 can specify the position and size of each unit region by referring to the region designation information. However, the filter parameter FP may be configured not to include such region designation information, and the moving image encoding device 2 and the moving image decoding device 1 may use a unit region having a predetermined size.

On the other hand, the unit area described above includes a slice, an LCU, a CU (other than a leaf CU), a leaf CU (a prediction unit PU, a transform unit TU), a partition constituting the prediction unit PU, and a block constituting the transform unit TU. In any case, the adaptive filter included in the video decoding device 1 refers to the CU partition flag SP_CU, inter-PU partition information SP_Inter, intra-PU partition information SP_Intra, TU partition information SP_TU, etc. The position and size of the unit area can be specified hierarchically.

The filter coefficient group I (I = 1 to Nf) includes (1) tap number designation information for designating the number of filter taps (or the total number of filter coefficients for each filter coefficient group), and (2) filter coefficients a_I ₀ to a_I. _NT-1 (NT is the total number of filter coefficients included in the filter coefficient group I) and (3) an offset oI. Note that filter coefficient groups with different filter numbers are used for the filter processing of unit areas classified into different types. For example, the filter coefficient group 0 and the filter coefficient group 1 are used for the filter processing of the unit area classified into type 1 and the unit area classified into type 2 different from type 1, respectively.

Hereinafter, the video decoding device 1 according to the present embodiment will be described with reference to FIGS. The moving picture decoding apparatus 1 includes H.264 as a part thereof. H.264 / MPEG-4. Decoding device including technology adopted in KTA software which is a codec for joint development in AVC and VCEG (Video Coding Expert Group) and technology adopted in TMuC (Test Model Under Consistration) software which is the successor codec It is.

FIG. 3 is a block diagram showing a configuration of the moving picture decoding apparatus 1. As shown in FIG. 3, the moving picture decoding apparatus 1 includes a variable length code decoding unit 11, a predicted image generation unit 12, an inverse quantization / inverse conversion unit 13, an adder 14, a frame memory 15, a deblocking filter 16, and The adaptive filter 17 is provided. As shown in FIG. 3, the predicted image generation unit 12 includes a motion vector restoration unit 12a, an inter predicted image generation unit 12b, an intra predicted image generation unit 12c, and a prediction method determination unit 12d. The moving picture decoding apparatus 1 is an apparatus for generating moving picture # 2 by decoding encoded data # 1.

(Variable-length code decoding unit 11)
The variable length code decoding unit 11 decodes the prediction parameter PP related to each partition from the encoded data # 1, and supplies the decoded prediction parameter PP to the predicted image generation unit 12. Specifically, for the inter prediction partition, the variable length code decoding unit 11 receives the inter prediction parameter PP_Inter including the reference image index RI, the estimated motion vector index PMVI, and the motion vector residual MVD from the encoded data # 1. These are decoded and supplied to the motion vector restoration unit 12a. On the other hand, for the intra prediction partition, the intra prediction parameter PP_Intra including the estimation flag MPM, the residual index RIPM, and the additional index AI is decoded from the encoded data # 1, and these are supplied to the intra prediction image generation unit 12c.

Also, the variable length code decoding unit 11 decodes the prediction type information PT for each partition from the encoded data # 1, and supplies this to the prediction method determination unit 12d. Further, the variable length code decoding unit 11 decodes the quantization prediction residual QD for each block and the quantization parameter difference Δqp for the TU including the block from the encoded data # 1, and performs inverse quantization / inverse This is supplied to the conversion unit 13. Further, the variable length code decoding unit 11 decodes the filter parameter FP from the encoded data # 1 and supplies this to the adaptive filter 17.

(Predicted image generation unit 12)
The predicted image generation unit 12 identifies whether each partition is an inter prediction partition for performing inter prediction or an intra prediction partition for performing intra prediction based on the prediction type information PT for each partition. In the former case, the inter prediction image Pred_Inter is generated, and the generated inter prediction image Pred_Inter is supplied to the adder 14 as the prediction image Pred. In the latter case, the intra prediction image Pred_Intra is generated, The generated intra predicted image Pred_Intra is supplied to the adder 14. Note that, when the skip mode is applied to the processing target PU, the predicted image generation unit 12 omits decoding of other parameters belonging to the PU.

(Motion vector restoration unit 12a)
The motion vector restoration unit 12a restores the motion vector mv related to each inter prediction partition from the motion vector residual MVD related to that partition and the restored motion vector mv ′ related to another partition. Specifically, (1) the estimated motion vector pmv is derived from the restored motion vector mv ′ according to the estimation method specified by the estimated motion vector index PMVI, and (2) the derived estimated motion vector pmv and the motion vector remaining are derived. The motion vector mv is obtained by adding the difference MVD. It should be noted that the restored motion vector mv ′ relating to other partitions can be read from the frame memory 15. The motion vector restoration unit 12a supplies the restored motion vector mv to the inter predicted image generation unit 12b together with the corresponding reference image index RI.

(Inter prediction image generation unit 12b)
The inter prediction image generation unit 12b generates a motion compensated image mc related to each inter prediction partition by inter-screen prediction. Specifically, using the motion vector mv supplied from the motion vector restoration unit 12a, the motion compensation image mc from the filtered decoded image P_ALF ′ designated by the reference image index RI supplied from the motion vector restoration unit 12a. Is generated. Here, the filtered decoded image P_ALF ′ is an image obtained by performing deblocking processing by the deblocking filter 16 and filtering processing by the adaptive filter 17 on the decoded image that has already been decoded for the entire frame. The inter-predicted image generation unit 12b can read out the pixel value of each pixel constituting the filtered decoded image P_ALF ′ from the frame memory 15. The motion compensated image mc generated by the inter predicted image generation unit 12b is supplied to the prediction method determination unit 12d as an inter predicted image Pred_Inter.

(Intra predicted image generation unit 12c)
The intra predicted image generation unit 12c generates a predicted image Pred_Intra related to each intra prediction partition. Specifically, first, a prediction mode is specified based on the intra prediction parameter PP_Intra supplied from the variable length code decoding unit 11, and the specified prediction mode is assigned to the target partition in, for example, raster scan order.

Here, specification of the prediction mode based on the intra prediction parameter PP_Intra can be performed as follows. (1) The estimation flag MPM is decoded, and the estimation flag MPM indicates that the prediction mode for the target partition to be processed is the same as the prediction mode assigned to the peripheral partition of the target partition. If it is, the prediction mode assigned to the partition around the target partition is assigned to the target partition. (2) On the other hand, if the estimation flag MPM indicates that the prediction mode for the target partition to be processed is not the same as the prediction mode assigned to a partition around the target partition, the remaining The prediction mode index RIPM is decoded, and the prediction mode indicated by the residual prediction mode index RIPM is assigned to the target partition.

The intra predicted image generation unit 12c generates a predicted image Pred_Intra from the (local) decoded image P by intra prediction according to the prediction method indicated by the prediction mode assigned to the target partition. The intra predicted image Pred_Intra generated by the intra predicted image generation unit 12c is supplied to the prediction method determination unit 12d. Note that the intra predicted image generation unit 12c may be configured to generate the predicted image Pred_Intra from the filtered decoded image P_ALF by intra prediction.

(Prediction method determination unit 12d)
The prediction method determination unit 12d determines whether each partition is an inter prediction partition that should perform inter prediction or an intra prediction partition that should perform intra prediction based on the prediction type information PT about the PU to which each partition belongs. To do. In the former case, the inter prediction image Pred_Inter generated by the inter prediction image generation unit 12b is supplied to the adder 14 as the prediction image Pred. In the latter case, the inter prediction image generation unit 12c generates the inter prediction image Pred_Inter. The intra predicted image Pred_Intra that has been processed is supplied to the adder 14 as the predicted image Pred.

(Inverse quantization / inverse transform unit 13)
The inverse quantization / inverse transform unit 13 (1) inversely quantizes the quantized prediction residual QD, (2) performs inverse DCT (Discrete Cosine Transform) transform on the DCT coefficient obtained by the inverse quantization, and (3) The prediction residual D obtained by the inverse DCT transform is supplied to the adder 14. When the quantization prediction residual QD is inversely quantized, the inverse quantization / inverse transform unit 13 derives the quantization step QP from the quantization parameter difference Δqp supplied from the variable length code decoding unit 11. The quantization parameter qp can be derived by adding the quantization parameter difference Δqp to the quantization parameter qp ′ relating to the TU that has been inversely quantized / inversely DCT transformed immediately before, and the quantization step QP is derived from the quantization step qp, for example, QP = 2 ^{pq / 6} . The generation of the prediction residual D by the inverse quantization / inverse transform unit 13 is performed in units of blocks obtained by dividing TUs or TUs.

(Adder 14)
The adder 14 generates the decoded image P by adding the prediction image Pred supplied from the prediction image generation unit 12 and the prediction residual D supplied from the inverse quantization / inverse conversion unit 13. The generated decoded image P is stored in the frame memory 15.

(Deblocking filter 16)
The deblocking filter 16 determines the block boundary in the decoded image P or the block boundary in the decoded image P when the difference between the pixel values of pixels adjacent to each other via the block boundary or the partition boundary in the decoded image P is smaller than a predetermined threshold value. By performing a deblocking process on the partition boundary, the block boundary or an image near the partition boundary is smoothed. The image subjected to the deblocking process by the deblocking filter 16 is stored in the frame memory 15 as a deblocked decoded image P_DB.

(Adaptive filter 17)
The adaptive filter 17 (1) sets one or a plurality of unit areas constituting the deblocked decoded image P_DB to any of a plurality of types according to the image characteristics of the deblocked decoded image P_DB in the unit area. (2) ON / OFF of adaptive filter processing for the unit area is switched according to the type to which the unit area belongs. Further, (3) adaptive filter processing is performed on the deblocked decoded image P_DB in the unit region for which the adaptive filter processing is turned on using a filter coefficient group selected according to the type to which the unit region belongs. .

FIG. 1 is a block diagram showing the configuration of the adaptive filter 17. As shown in FIG. 1, the adaptive filter 17 includes a region characteristic value calculation unit 171, a region classification unit 172, and a filter processing unit 173.

(Regional characteristic value calculation unit 171)
The region characteristic value calculation unit 171 reads each pixel value of the deblocked decoded image P_DB in the characteristic value calculation reference region including the target unit region to be processed from the frame memory 15, and based on these pixel values, the target unit A characteristic value (also referred to as an area characteristic value) representing the image characteristic of the deblocked decoded image P_DB in the area is calculated. Here, the characteristic value calculation reference region refers to a region formed of pixels referred to for calculating the characteristic value of the deblocked decoded image P_DB in the target unit region. The characteristic value calculation reference area is composed of only the target unit area, or is composed of the target unit area and an area around the target unit area. Note that the region characteristic value calculation unit 171 is a configuration for calculating a characteristic value related to the target unit region, and can also calculate the characteristic value using another image having a correlation with the target unit region. Therefore, the region characteristic value calculation unit 171 is not limited to the deblocked decoded image P_DB, and may calculate the characteristic value with reference to another image, for example, the decoded image P. Also in this case, the pixel value located in the coordinate area corresponding to the target unit area is used. In the present embodiment, the adaptive filter 17 having the deblocked decoded image P_DB as an input is described. However, the adaptive filter 17 may be configured to have a plurality of inputs. In this way, when the adaptive filter 17 has a plurality of different input images, the characteristic value may be calculated using one of these input images.

As shown in FIG. 1, the region characteristic value calculation unit 171 includes an activity calculation unit 171a. The activity calculation unit 171a calculates an activity ACT as the characteristic value of the deblocked decoded image P_DB in the target unit region. Here, generally speaking, the activity is an index indicating the randomness of the image, and whether there is a large high-frequency component in the conversion coefficient obtained when the DCT conversion is performed on the region including the divided region. However, here, the calculation is based on the pixel values of adjacent pixels. The greater the difference in pixel value between adjacent pixels, the greater the activity. The calculated activity ACT for each divided region is supplied to the region classification unit 172.

When the target unit area has a rectangular shape of Bx pixels × By pixels, the coordinates of each pixel in the target unit area are (i, j) (i and j are 0 ≦ i ≦ Bx−1 and 0 ≦ j ≦ By−, respectively). If the pixel value of the deblocked decoded image P_DB at the coordinates is represented by S (i, j), the activity calculation unit 171a uses, for example, the following formula (1a): Thus, the activity ACT can be calculated.

Formula (1a) corresponds to evaluating the activity using the sum of absolute values of adjacent pixel values (Sum of Absolute Difference, SAD) as an index.

Further, the activity calculation unit 171a may be configured to calculate the activity ACT by using the following mathematical formula (1b).

Formula (1b) corresponds to the evaluation of activity using the sum of modified Laplacians (Sum-ModifiedModLaplacian, SML) of the pixel values of adjacent pixels as an index.

When calculating the activity for the target unit region using the formula (1a) or (1b), the activity calculation unit 171a applies not only to the pixel S (i, j) in the target unit region, but also to the target unit region. Reference is also made to adjacent pixels S (−1, j), S (Bx, j), S (i, −1), and S (i, By). That is, the characteristic value calculation reference region when using the formula (1a) or (1b) is the pixel S (i, j) in the target unit region and the pixel S (−1, j) adjacent to the target unit region. , S (Bx, j), S (i, −1), and S (i, By). FIG. 4A shows the characteristic value calculation reference region CRR when the activity calculation unit 171a calculates the activity ACT for the target unit region UR of 4 × 4 pixels using the formula (1a) or (1b). Shown in

When the target unit area is in contact with the slice boundary or the LCU boundary, the pixel value of the deblocked decoded image P_DB may not be referred to for some pixels in the characteristic value calculation reference area. For example, when the target unit area is adjacent to an undecoded LCU, the pixel value of the undecoded LCU cannot be referred to. In such a case, as an exception process, the activity calculation unit 171a calculates the activity ACT without referring to those pixel values that cannot be referred to. For example, when the right side and the lower side of the target unit region are adjacent to an undecoded LCU, the activity calculation unit 171a obtains the following obtained by partially limiting the range of the sum of i and j in Equation (1a) What is necessary is just to calculate activity ACT using Numerical formula (1a ').

The same applies to Equation (1b).

On the other hand, the activity calculation unit 171a refers to only the pixel value in the target unit region for the activity of the target unit region regardless of whether the target unit region is in contact with the slice boundary or the LCU boundary. It can also be configured to calculate. For example, the activity calculation unit 171a can also calculate the activity ACT by using the following formula (1c).

Equation (1c) corresponds to evaluating the activity of the pixel value of the adjacent pixel using the SAD of the pixel value of the adjacent pixel as an index, as in Equation (1a). Differently, the degree of activity is calculated by referring to only the pixels included in the target unit area.

Further, the activity calculation unit 171a may be configured to calculate the activity ACT by using the following formula (1d).

Similar to Equation (1b), Equation (1d) corresponds to the evaluation of the activity using the SML of the pixel value of the adjacent pixel as an index, but unlike Equation (1d), In other words, the degree of activity is calculated by referring only to the included pixels.

As described above, when the activity calculation unit 171a calculates the activity for the target unit region by referring only to the pixel value in the target unit region, as shown in FIG. 4B, the characteristic value calculation reference is made. The region CRR is the same region as the target unit region UR. A variance VP, which will be described later, is also an example of the degree of activity that can be calculated with reference to only pixel values in the target unit region.

Therefore, when the activity level calculation unit 171a calculates the activity level for the target unit area by referring only to the pixel values in the target unit area, the activity level calculation unit 171a compares the pixel value other than the target unit area with reference to the pixel value. Thus, the number of pixels read from the frame memory 15 is reduced, and the data transmission amount from the frame memory 15 to the adaptive filter 16 is reduced. Further, since the number of pixels referred to calculate the activity level is small and exception processing is unnecessary, the processing amount for calculating the activity level is reduced.

In addition, the activity calculation part 171a which calculates activity ACT by using Formula (1c) can also be expressed as follows. That is, the activity calculation unit 171a determines the activity ACT (i, j) for the pixel (i, j) in the vicinity of the coordinates (i, j) {(i, j), (i + 1, j), (i , J + 1)} with reference to the pixel value of each pixel, and ACT (i in the inner region {(i, j) | 0 ≦ i ≦ Bx−2, 0 ≦ j ≦ By−2} of the target unit region , J) is used to calculate the ACT for the target unit region.

Also, the activity calculation unit 171a that calculates the activity ACT by using the mathematical formula (1d) can be expressed as follows. That is, the activity calculation unit 171a determines the activity ACT (i, j) for the pixel (i, j) in the vicinity of the coordinates (i, j) {(i, j), (i + 1, j), (i , J + 1), (i−1, j), (i, j−1)} with reference to the pixel value of each pixel, the inner region {(i, j) | 1 ≦ i ≦ The ACT for the target unit region is calculated using the sum of ACT (i, j) in Bx-2, 1 ≦ j ≦ By-2}.

As described above, the activity calculation unit 171a that calculates the activity with reference to only the pixels included in the target unit region has the entire vicinity of the target unit region as an internal region including pixels included in the target unit region. The degree of activity of each target pixel region is calculated from the degree of activity of each pixel belonging to the internal area after deriving the degree of activity of each pixel belonging with reference to the pixel values of the pixels included in the vicinity of the pixel. It can also be expressed as a thing.

Note that the configuration for calculating the degree of activity with reference to only the pixels included in the target unit region is not limited to the above example, and the target unit region is a pixel whose entire neighborhood is included in the target unit region. Is divided into a peripheral region consisting of pixels that are not included in the target unit region, and the activity of each pixel belonging to the internal region is referred to the pixel value of the pixel in the vicinity of the pixel. And the activity of each pixel belonging to the peripheral region is derived by referring to the pixel value of the pixel in the vicinity of the pixel and in the unit region, and then belonging to the unit region A configuration may be employed in which the degree of activity for the unit region is calculated from the degree of activity of each pixel.

(Region classification unit 172)
The region classification unit 172 classifies the target unit region into one of a plurality of types based on the activity ACT calculated by the region characteristic value calculation unit 171 as the characteristic value for the target unit region. Subsequently, according to the type, the target region is classified into either a region where the filter process is off (filter off region) or a region where the filter process is on (filter on region). As will be described later, the relationship between the type and the region may be fixed in advance, or a flag indicating which type is classified into which region may be encoded. Hereinafter, a type classified into the filter-on region is referred to as a filter-on type, and a type classified into the filter-off region is referred to as a filter-off type. Further, the target unit area may be classified into the filter-on area and the filter-off area from the direct characteristic value (activity ACT) without classifying the target unit area into a type.

When the target unit region is classified as a filter-on region, the region classification unit 172 is a filter coefficient group included in the filter parameter FP supplied from the variable length code decoding unit 11, and includes a type to which the target unit region belongs. Are assigned to the target unit area. On the other hand, when the target unit region is classified as the filter-off region, the region classification unit 172 does not assign a filter coefficient group to the target unit region.

The area classification unit 172 includes (1) on / off information U indicating whether the target unit area is classified as a filter on area or a filter off area, and (2) the target unit area is a filter on area. In the case of the classification, the filter coefficient group allocated to the target unit area is supplied to the filter processing unit 173. Note that the on / off information U indicates on / off of the filter processing for the target unit region, and can therefore be expressed by 1 bit. Since the classification of the target unit area into the filter-on area / filter-off area is equivalent to the classification of the target unit area into the filter-on type / filter-off type, in the following description, the filter-on area / filter-off area is the same value. Handles the OFF area and the filter ON type / filter OFF type without distinction.

Hereinafter, prior to description of a specific example of classification processing by the region classification unit 172, experimental data obtained by the inventor will be described with reference to FIG.

FIG. 5A is a graph showing experimental data obtained by the inventor, and a plurality of sequences (“BasketballDrill_q32”, “PartyScene_q32”, “RaceHorses_q32”, “BQSquare_q32”) whose quantization parameter value is 32. ) For each unit region, and a square error reduction amount obtained when adaptive filter processing is performed for each unit region. Here, the square error reduction amount has a positive correlation with the coding efficiency, and indicates that the larger the square error reduction amount, the better the coding efficiency. Further, it is indicated that the unit area in which the square error reduction amount is negative has a negative contribution to the coding efficiency, that is, a contribution to reduce the coding efficiency. In addition, since the filter coefficients need to be encoded in order to perform the adaptive filter processing, there is a possibility that the encoding efficiency may be reduced not only when the square error reduction amount is negative but also when the square error is small. The activity in the experimental data of FIG. 5 is calculated using the above-described mathematical formula (1a).

As shown in FIG. 5A, there is an activity range in which the square error reduction amount is negative for any sequence. In particular,
・ For the sequence “BasketballDrill_q32”, the range of activity is about 5 or less ・ For the sequence “PartyScene_q32”, the range of activity is about 8 or less ・ For the sequence “RaceHorses_q32”, the range of activity is about 3 or less ・ Sequence Regarding “BQSquare_q32”, the square error reduction amount is negative in the range where the activity is approximately 5 or more and approximately 20 or less and in each range where the activity is approximately 3 or less.

FIG. 5B is a graph showing the average (“average”) for a plurality of sequences and the total number of unit areas (“count”) having each activity, with respect to the square error reduction amount at each activity. .

As shown in FIG. 5B, the value of the graph indicating the average of a plurality of sequences is negative in the range where the activity is approximately 8 or less.

Thus, for each sequence, there is an activity range in which the square error reduction amount is negative. Based on the above experimental data, the inventor obtained knowledge that coding efficiency is improved by adopting a configuration in which adaptive filter processing is not performed for unit regions having an activity with a negative square error reduction amount. It was.

Hereinafter, a specific example of classification processing by the region classification unit 172 will be described with reference to FIG. The area classification unit 172 can be configured to perform any of the classification processes among the following (classification process example 1-1) to (classification process example 1-7). Further, among the following (Classification Processing Example 1-1) to (Classification Processing Example 1-7), classification processing designating information for designating which classification processing is performed is included in the encoded data # 1, and the region The classification unit 172 may be configured to selectively perform the classification process designated by the classification process designation information among (Classification process example 1-1) to (Classification process example 1-7).

In the following description, it is assumed that type 0 is a filter-off type, and types 1 to Non (Non is the total number of filter-on types) are filter-on types. In the following, it is assumed that the region classification unit 172 assigns the filter coefficient group I to the target unit region classified as type I (I = 1 to Non).

(Classification example 1-1)
In this processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

・ Type 0: ACT ≦ Th1
・ Type 1: Th1 <ACT
Here, Th1 represents a predetermined threshold value. As a specific value of the threshold Th1, for example, a value predetermined for each sequence may be used in consideration of the experimental data shown in FIG. In particular,
・ Th1 = 5 for the sequence “BasketballDrill_q32”
-For the sequence "PartyScene_q32", Th1 = 8
・ Th1 = 3 for the sequence “RaceHorses_q32”
-For the sequence "BQSquare_q32", Th1 = 20
May be used. When the threshold value Th1 is determined without depending on the sequence, Th1 = 8 may be used.

As described above, by setting a unit region whose activity is equal to or less than a predetermined threshold as a type region 0, that is, a unit region that is not subjected to adaptive filter processing, the negative contribution to the coding efficiency can be reduced. Therefore, the encoding efficiency is improved.

Further, the value of the threshold Th1 is preferably set according to a specific calculation formula for calculating the activity. For example, the value of the threshold Th1 that is preferably used in the configuration that calculates the activity ACT using the formula (1b) may be different from the value of the threshold Th1 that is used in the configuration that uses the formula (1a).

(Classification example 1-2)
In this processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

・ Type 0: ACT ≦ Th1
・ Type 1: Th1 <ACT ≦ Th2
・ Type 2: Th2 <ACT
Here, Th2 indicates a predetermined threshold value, and it is assumed that Th1 <Th2. A specific value of the threshold Th2 may be determined in advance so that the encoding efficiency is improved.

Also in this processing example, a unit area whose activity is equal to or less than a predetermined threshold is set to type 0, that is, a unit area for which adaptive filter processing is not performed. Therefore, the negative contribution to the coding efficiency can be reduced, so that the coding efficiency is improved. In addition, since any one of a plurality of filter coefficient groups is applied to the unit area for which the filter processing is turned on according to the value of the degree of activity, the encoding efficiency can be further improved. In addition, it is preferable to use a predetermined value as the specific value of the threshold Th2 in this processing example so that the coding efficiency is improved.

(Classification example 1-3)
In the present processing example, the area classification unit 172 classifies the target unit area as follows as shown in FIG.

・ Type 1: ACT ≦ Th2
・ Type 0: Th2 <ACT
Here, the value of the threshold Th2 may be determined in advance by experiments or the like so that the processing amount is further reduced while maintaining high coding efficiency. For example, the experimental data shown in FIG. 5 indicates that when the activity value is approximately 90 or more, even if adaptive filter processing is performed, the coding efficiency is hardly improved. Therefore, if the threshold value Th2 is set to about 90, the processing amount can be reduced while maintaining high coding efficiency.

(Classification process example 1-4)
In this processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

・ Type 1: ACT ≦ Th1
・ Type 2: Th1 <ACT ≦ Th2
・ Type 0: Th2 <ACT
Here, the specific value of the threshold Th2 in this processing example is set to a value of about 90 so that the processing amount is further reduced while maintaining high coding efficiency, as in the classification processing example 1-3. You just have to. In addition, it is preferable to use a predetermined value for the threshold value Th1 in this processing example so that the encoding efficiency is improved.

The encoding efficiency can be effectively improved by using such threshold values Th1 and Th2.

(Classification example 1-5)
In this processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

Type 0: ACT ≦ Th1 or Th2 <ACT
・ Type 2: Th1 <ACT ≦ Th2
Here, the specific value of the threshold value Th1 in the present processing example may be determined in the same manner as in the classification processing example 1-1. The specific value of the threshold value Th2 in this processing example may be determined in the same manner as in the classification processing example 1-3.

(Classification example 1-6)
In this processing example, the region classification unit 172 classifies the target unit region as follows, as illustrated in FIG.

・ Type 1: ACT ≦ Th1
・ Type 2: Th1 <ACT ≦ Th2
・ Type 3: Th2 <ACT
Here, the specific values of the threshold values Th1 and Th2 in this processing example may be determined in advance so as to improve the encoding efficiency.

In the processing example, it is preferable that the region characteristic value calculation unit 171 refers to only the pixel value in the target unit region when calculating the activity ACT for the target unit region. As a result, the number of pixels read from the frame memory 15 is reduced and the amount of data transmission from the frame memory 15 to the adaptive filter 16 is reduced compared to a configuration that also refers to pixel values other than the target unit region. Further, since the number of pixels referred to calculate the activity level is small and exception processing is unnecessary, the processing amount for calculating the activity level is reduced.

Therefore, according to the present processing example, it is possible to improve the encoding efficiency while reducing the processing amount.

(Classification example 1-7)
As described above, the processing example in which the region classification unit 172 determines the on / off of the filter processing for the target unit region by comparing the activity ACT with a predetermined threshold has been described. However, the present embodiment is not limited to this. It is not something that can be done.

For example, the filter parameter FP includes on / off designation information V for designating the on / off of the filter processing for each type, and the area classification unit 172 refers to the on / off designation information V, thereby performing the filter processing for each type. On / off may be determined. Hereinafter, an example of such classification processing will be described with reference to FIG. Unlike the above-described on / off information U, the on / off designation information V is information for designating the on / off of the filtering process for each type, and therefore the bit length depends on the total number of types.

In the following description, a case is described in which the total number of types is 4, and the filter parameter FP includes 4-bit on / off designation information V as information for designating on / off of the filter processing for each type. However, the present processing example is not limited to this, and is generally applicable when the total number of types is Ntp and the filter parameter FP includes Ntp bit on / off designation information V.

FIG. 6G is a table showing the classification processing in this processing example. As shown in FIG. 6G, the region classification unit 172 classifies the target unit region as follows.
・ Type A: ACT ≦ Th
・ Type B: Th <ACT ≦ 2 × Th
Type C: 2 × Th <ACT ≦ 3 × Th
・ Type D: 3 × Th <ACT
Further, the region classification unit 172 refers to the on / off designation information V included in the filter parameter FP, and determines the on / off of the filter processing for each type. For example, when the on / off designation information V is “0101” in binary expression, type A and type C are set to the filter off type, and type B and type D are set to the filter on type. It should be noted that a specific value of the threshold Th is preferably a predetermined value so that the encoding efficiency is improved.

The region classification unit 172 assigns a filter coefficient group included in the filter parameter FP to the target unit region when the target unit region is classified into the filter on type. Here, when a plurality of filter coefficient groups are included in the filter parameter FP, the same selection rule as the selection rule used in the moving image encoding device 2 that generates the encoded data # 1 is used. Any one of the plurality of filter coefficient groups may be selected and assigned to the target unit area.

On the other hand, when the target unit region is classified into the filter-off type, the region classification unit 172 does not assign a filter coefficient group to the target unit region.

The area classification unit 172 includes (1) on / off information U indicating whether the target unit area is classified as a filter on type or a filter off type, and (2) the target unit area is classified as a filter on type. If it is, the filter coefficient group assigned to the target unit area is supplied to the filter processing unit 173.

Also in this processing example, since the filter processing can be switched on and off for each activity range, the encoding efficiency is improved.

(How to set each unit area by the area classification unit 172)
In the above description, the region classification unit 172 sets each unit region as a CU partition flag SP_CU, inter-PU partition information SP_Inter, intra-PU partition information SP_Intra, TU partition information SP_TU, and the like included in the encoded data # 1. May be set so as to have a size and position specified hierarchically, or may be set to have a size and position specified by the area specifying information included in the encoded data # 1, It may be set to have a predetermined size (the same applies hereinafter).

FIG. 7A shows an example of a unit area that is hierarchically specified by the CU division flag SP_CU included in the encoded data # 1. In the example shown in FIG. 7A, the maximum hierarchical depth for the target LCU is 2, and the positions and sizes of the unit areas UR0 to UR6 included in the target LCU are hierarchically specified by the CU division flag SP_CU. .

On the other hand, an example of a unit area having a predetermined size is shown in FIG. In FIG. 7B, the target LCU (for example, 16 × 16 pixels) is divided by unit areas UR0 to UR15 having a predetermined size (for example, 4 × 4 pixels). Each unit area can be identified from each other by an index assigned in the raster scan order as shown in FIG. 7B, for example.

In the configuration in which each unit area is set to have a predetermined size, the CU partition flag SP_CU, inter-PU partition information SP_Inter, intra-PU partition information SP_Intra, and TU partition information SP_TU included in the encoded data # 1 The processing amount can be reduced as compared with the case of referring to the above.

(Filter processing unit 173)
When the on / off information U supplied from the region classification unit 172 indicates the filter on type, the filter processing unit 173 (1) each pixel value of the deblocked decoded image P_DB in the filter reference range including the target unit region (2) A filtered decoded image P_ALF in the target unit region is generated by performing filter processing using the filter coefficient group supplied from the region classification unit 172 on these pixels. To do. The generated filtered decoded image P_ALF is stored in the frame memory 15. Here, the filter reference range refers to a range that is referred to in order to generate the filtered decoded image P_ALF in the target unit region, and includes a target unit region and a region around the target unit region.

On the other hand, when the on / off information U supplied from the region classification unit 172 indicates the filter off type, the filter processing unit 173 does not perform the filtering process on the target unit region.

In the following description, the filter coefficient group supplied from the region classification unit 172 is the filter coefficient group I (I is any integer satisfying 1 ≦ I ≦ Nf, and Nf is the total number of filter coefficient groups included in the filter parameter FP) ). Further, the pixel value of the filter target pixel in the filtered decoded image P_ALF (also referred to as “filtered image”) in the target unit region is represented as SF (x ′, y ′), and the deblocked decoded image in the filter reference range. The pixel value of P_DB (also referred to as “pre-filter image”) is represented as S (x, y).

The filter processing unit 173 calculates the pixel value SF (x ′, y ′) by the following formula (2).

Here, the coordinates (x, y) may be the same coordinates as the coordinates (x ′, y ′), or may be different coordinates as long as they have a one-to-one correspondence. AI (i, j) represents a filter coefficient to be multiplied by the pixel value S (x + i, y + j) of the pre-filter image among the filter coefficients included in the filter coefficient group I, and oI represents the filter coefficient group. I represents an offset included in I.

R represents a region (also referred to as “filter reference region R”) that is referred to in order to calculate the pixel value of the filter target pixel, and is set according to the position of the filter target pixel. On the other hand, the above-described filter reference range (also referred to as “filter reference range RA”) is defined as the union of filter reference regions R for each filter target pixel. The filter reference range RA can also be expressed as a set of pixels required for calculating all pixel values of the filtered image in the target unit region.

FIG. 8A shows a filter reference region R and a filter reference range RA when the target unit region UR is 8 × 8 pixels and the filter reference region is 5 × 5 taps. In FIG. 8A, the hatched pixels indicate the filter target pixels S (x ′, y ′).

FIG. 8B shows an example of how to assign filter coefficients to each pixel included in the filter reference region R. Also in FIG. 8B, the hatched pixels indicate the filter target pixels S (x ′, y ′). As shown in FIG. 8B, each filter coefficient can be assigned to each pixel included in the filter reference region R so as to have a rotational symmetry of 180 degrees. That is, aI (i, j) in Equation (2) is obtained by using the filter coefficients a0 to a12 included in the filter parameter FP, so that aI (−3, −3) = aI (3,3) = a0, aI ( −2, −3) = aI (2,3) = a1,..., And can be set to satisfy aI (i, j) = aI (−i, −j). .

However, the present embodiment is not limited to this, and the assignment of each filter coefficient to each pixel value may not have rotational symmetry. In addition, the filter reference region R may be a rhombus region composed of pixels having a city area distance from the filter target pixel of Ncb or less in units of pixels, or regions having other shapes. Good.

The method of assigning the filter coefficient to each pixel included in the filter reference region R and the shape of the filter reference region R may be appropriately set according to the configuration of the moving picture encoding device that generates the encoded data # 1. Good.

<Modification 1>
The activity calculation unit 171a included in the region characteristic value calculation unit 171 uses the vertical activity ACT_V and the horizontal activity as the characteristic value of the deblocked decoded image P_DB in the target unit region instead of the activity ACT described above. The degree ACT_H may be calculated, and the region classification unit 172 may classify the target unit region into one of a plurality of types based on the horizontal direction activity ACT_H and the vertical direction activity ACT_V.

Here, the horizontal direction activity ACT_H and the vertical direction activity ACT_V can be calculated by, for example, the first term and the second term of the mathematical formula (1a) described above. In addition, the horizontal direction activity ACT_H and the vertical direction activity ACT_V may be calculated by the first term and the second term of any one of the above formulas (1b) to (1d), respectively. it can.

The region classification unit 172 classifies the target unit region into one of a plurality of types based on the horizontal direction activity ACT_H and the vertical direction activity ACT_V calculated by the activity calculation unit 171a. Below, with reference to FIG. 9, the classification | category process in this modification is demonstrated. The region classification unit 172 may be configured to perform any of the classification processes from the following (Classification process example 2-1) to (Classification process example 2-5). Also, among the following (Classification Processing Example 2-1) to (Classification Processing Example 2-5), classification processing designation information for designating which classification processing is performed is included in the encoded data # 1, The classification unit 172 may be configured to selectively perform the classification process designated by the classification process designation information among (Classification process example 2-1) to (Classification process example 2-5).

(Classification example 2-1)
In the present processing example, the area classification unit 172 classifies the target unit area as follows as shown in FIG.

Type 0: ACT_H ≦ ThHor1 and ACT_V ≦ ThVer1
Type 1: ThHor1 <ACT_H and ACT_V ≦ ThVer1
Type 2: ACT_H ≦ ThHor1 and ThVer1 <ACT_V
Type 3: ThHor1 <ACT_H and ThVer1 <ACT_V
Here, as the values of the thresholds ThHor1 and ThVer1, values that are set in advance so as to improve the encoding efficiency may be used.

In the present processing example, the target unit area is classified into one of a plurality of types including type 0 according to each value of the horizontal direction activity ACT_H and the vertical direction activity ACT_V, so that the deblocked decoded image P_DB The classification process can be performed more appropriately according to the image characteristics along the horizontal and vertical directions of the image. Therefore, according to this processing example, encoding efficiency can be improved more effectively.

(Classification example 2-2)
In the present processing example, the region classification unit 172 classifies the target unit region as follows as shown in FIG. 9B.

Type 1: ACT_H ≦ ThHor1 and ACT_V ≦ ThVer1
Type 2: ACT_H ≦ ThHor1 and ThVer1 <ACT_V
Type 3: ThHor1 <ACT_H and ACT_V ≦ ThVer1
Type 4: ThHor1 <ACT_H and ThVer1 <ACT_V
Here, as the values of the thresholds ThHor1 and ThVer1, values that are set in advance so as to improve the coding efficiency may be used, as in the classification processing example 2-2.

As described above, in this processing example, the target unit area is classified into one of a plurality of types not including type 0 according to each value of the horizontal activity ACT_H and the vertical activity ACT_V.

In the processing example, it is preferable that the region characteristic value calculation unit 171 refers to only the pixel value in the target unit region when calculating the activity ACT for the target unit region. As a result, the number of pixels read from the frame memory 15 is reduced and the amount of data transmission from the frame memory 15 to the adaptive filter 16 is reduced compared to a configuration that also refers to pixel values other than the target unit region. In addition, since the number of pixels referred to calculate the vertical activity and the horizontal activity is small and exception processing is unnecessary, the processing amount for calculating the activity is reduced.

(Classification process example 2-3)
In the present processing example, the area classification unit 172 classifies the target unit area as follows as illustrated in FIG.

Type 0: (ACT_H ≦ ThHor1 and ACT_V ≦ ThVer1), or ThHor2 <ACT_H, or ThVer2 <ACT_V
Type 1: ThHor1 <ACT_H ≦ ThHor2 and ACT_V ≦ ThVer1
Type 2: ACT_H ≦ ThHor1 and ThVer1 <ACT_V ≦ ThVer2
Type 3: ThHor1 <ACT_H ≦ ThHor2 and ThVer1 <ACT_V ≦ ThVer2
Here, as the values of the thresholds ThHor1, ThVer1, ThHor2, and ThVer2, values that are set in advance so as to improve the encoding efficiency may be used.

In the present processing example, not only the threshold values ThHor1 and ThVer1 but also the threshold values ThHor2 and ThVer2 are used to perform a finer classification process, thereby further improving the encoding efficiency.

As in the above configuration, one configuration of the present invention is a combination of a plurality of characteristic values (here, the horizontal activity ACT_H and the vertical activity) as the characteristic values used for the classification of the filter-on region and the filter-off region. This includes the case of using ACT_V). Similarly, one configuration of the present invention includes a case where a combination of a plurality of characteristic values is used as a characteristic value used for assigning a filter coefficient group to the filter-on region.

(Classification process example 2-4)
In this processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

Type 0: ACT ≦ Th1 or Th2 <ACT
Type 1: Th1 <ACT ≦ Th2 and ACT_H ≦ ACT_V
Type 2: Th1 <ACT ≦ Th2 and ACT_V <ACT_H
Here, ACT indicates the above-described activity ACT, and the activity calculation unit 171a is configured to individually calculate the horizontal activity ACT_H and the vertical activity ACT_V as in the present modification. In
ACT = ACT_H + ACT_V
Can be obtained.

In the present processing example, when the activity level ACT satisfies Th1 <ACT ≦ Th2, the filter coefficient group is switched according to the magnitude of the horizontal direction activity level ACT_H and the vertical direction activity level ACT_V. Can be improved. When ACT_H ≦ ACT_V, that is, when the vertical activity ACT_V is equal to or higher than the horizontal activity ACT_H, there is a high possibility that a vertical component change such as a horizontal edge exists, and when ACT_H> ACT_V, the vertical edge, etc. There is a high probability that there will be horizontal changes. In this processing example, the directionality of the image in the target unit region is calculated according to the magnitudes of the characteristic values ACT_H and ACT_V, and a filter is selected according to the directionality. In this processing example, ACT_H ≦ ACT_V, that is, the directionality when the change in the vertical direction is larger is expressed as directionality 0, and ACT_V <ACT_H, that is, the directionality when the change in the horizontal direction is larger. Is expressed as directionality 1. Here, 0 and 1 are indexes (also referred to as directionality indexes) for distinguishing the directionality of images from each other. The region classification unit 172 selects a filter type according to a direction index that is a value indicating the direction in the target unit region.

Note that, as in the above configuration, the characteristic value used for the classification of the filter-on region and the filter-off region (here, the activity ACT), and the characteristic value used for assigning the filter coefficient group to the filter-on region (here, the horizontal direction activity) ACT_H and vertical activity ACT_V) may be different. That is, the present invention has, as one configuration, a means for classifying the filter on area and the filter off area based on a certain characteristic value, and a means for assigning a filter coefficient group to the filter on area based on a characteristic value different from the certain characteristic value. The structure provided with.

(Classification example 2-5)
In this processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

Type 0: ACT ≦ Th1, or (Th2 <ACT and 2 × ACT_H <ACH_H), or (Th1 <ACT ≦ Th2 and 2 × ACT_H ≧ ACT_V and 2 × ACT_V ≧ ACT_H)
Type 1: Th1 <ACT ≦ Th2 and 2 × ACT_H <ACT_V
Type 2: Th1 <ACT ≦ Th2 and 2 × ACT_V <ACT_H
Type 3: (Th2 <ACT and 2 × ACT_H <ACT_V), or (Th2 <ACT and 2 × ACT_H ≧ ACT_V and 2 × ACT_V ≧ ACT_H)
Here, ACT represents the above-described activity ACT.

Note that when 2 × ACT_H <ACT_V, that is, when the vertical activity ACT_V is greater than the horizontal activity ACT_H by a predetermined degree or more (here, twice or more), there is a horizontal edge. In contrast, when 2 × ACT_V <ACT_H, there is a high possibility that a vertical edge exists. Further, when 2 × ACT_H ≧ ACT_V and 2 × ACT_V ≧ ACT_H, there is a high possibility that an oblique direction edge exists. In this processing example, the directionality of the image in the target unit region is calculated according to the magnitudes of the characteristic values ACT_H and ACT_V, and a filter is selected according to the directionality. In this processing example, the directionality in the case of 2 × ACT_H <ACT_V is expressed as directionality 0 using the directionality indexes 0 to 2, and the directionality in the case of 2 × ACT_V <ACT_H is set to the direction. The directionality in the case of 2 × ACT_H ≧ ACT_V and 2 × act_v ≧ act_h is expressed as directionality 2.

Directionality

0, 1, and 2 correspond to the cases where there are horizontal edges, vertical edges, and diagonal edges, respectively.

The region classifying unit 172 selects the filter type for the target unit region in accordance with the directionality index that is a value indicating the directionality of the image in the target unit region.

In this processing example, since the type 0 is assigned according to not only the activity but also the activity and directionality and the filter is turned off, the coding efficiency can be improved efficiently. Here, the filter is turned off when the activity is low. When the degree of activity is medium, the diagonal filter is turned off, and the

filters

1 and 2 are assigned to the vertical and horizontal directions. When the degree of activity is high, the horizontal filter is turned off and the filter 3 is assigned in a direction other than the horizontal direction. Here, the filter 0 is set when the activity is medium and the directionality is oblique. In the sequence to which this case is applied, there are few oblique lines in the region of activity, and encoding is performed by assigning filters in the oblique direction. This is because the efficiency is not improved. Note that this example is an example, and appropriate allocation differs depending on sequence dependency.

<Modification 2>
In the above description, the adaptive filter 17 causes the region characteristic value calculation unit 171 to use the activity ACT (or the horizontal activity ACT_H and the vertical activity ACT_V as the characteristic value of the deblocked decoded image P_DB in the target unit region. ) And the region classification unit 172 classifies the target unit region into one of a plurality of types based on the activity ACT (or horizontal activity ACT_H and vertical activity ACT_V). Although mentioned, this embodiment is not limited to this. Hereinafter, an average of pixel values (also referred to as an average pixel value) is calculated as the characteristic value of the deblocked decoded image P_DB in the target unit region, and the target unit region is selected from a plurality of types based on the calculated average pixel value. A configuration to be classified will be described.

FIG. 10 is a block diagram showing the configuration of the adaptive filter 17 according to this modification. As shown in FIG. 10, the region characteristic value calculation unit 171 in the adaptive filter 17 according to this modification includes a pixel value reference unit 171 b.

The pixel value reference unit 171b calculates the average pixel value MP by reading each pixel value of the deblocked decoded image P_DB in the target unit region to be processed from the frame memory 15 and taking the average of the pixel values.

When the target unit area has a rectangular shape of Bx pixels × By pixels, the coordinates of each pixel in the target unit area are (i, j) (i and j are 0 ≦ i ≦ Bx−1 and 0 ≦ j ≦ By−, respectively). 1), and the pixel value of the deblocked decoded image P_DB at that coordinate is represented as S (i, j), the pixel value reference unit 171b uses, for example, the following formula (3): Thus, the average pixel value MP can be calculated by taking a simple average of the pixel values.

The pixel value reference unit 171b may be configured to calculate the average pixel value MP by taking a weighted average using a predetermined weight coefficient for each pixel value instead of taking a simple average of each pixel value. .

The region classification unit 172 according to this modification classifies the target unit region into one of a plurality of types based on the average pixel value MP calculated by the region characteristic value calculation unit 171 as the characteristic value for the target unit region. The target unit area is further classified into a filter-on area or a filter-off area depending on the type. When the target unit region is classified as a filter-on region, the region classification unit 172 is a filter coefficient group included in the filter parameter FP supplied from the variable length code decoding unit 11, and includes a type to which the target unit region belongs. Are assigned to the target unit area. On the other hand, when the target unit region is classified as the filter-off region, the region classification unit 172 does not assign a filter coefficient group to the target unit region.

The area classification unit 172 includes (1) on / off information U indicating whether the target unit area is classified as a filter on type or a filter off type, and (2) the target unit area is a filter on type. In the case of the classification, the filter coefficient group allocated to the target unit area is supplied to the filter processing unit 173.

Hereinafter, the experimental data obtained by the inventor will be described with reference to FIG. 11 prior to description of a specific example of the classification processing by the region classification unit 172 according to the present modification.

FIG. 11A is a graph showing experimental data obtained by the inventor, and is for a plurality of sequences (“PartyScene_q32”, “RaceHorses_q32”, “BQSquare_q32”) having a quantization parameter value of 32. It is a graph which shows the average pixel value for every unit area | region, and the square error reduction amount obtained when an adaptive filter process is performed about each unit area | region. Here, the square error reduction amount has a positive correlation with the coding efficiency, and indicates that the larger the square error reduction amount, the better the coding efficiency. Further, it is indicated that the unit area in which the square error reduction amount is negative has a negative contribution to the coding efficiency, that is, a contribution to reduce the coding efficiency.

As shown in FIG. 11A, there is an activity range in which the square error reduction amount is negative for any sequence. In particular,
For the sequence “PartyScene_q32”, a range in which the average pixel value is approximately 30 or less, and a range in which the average pixel value is 230 or more. For the sequence “RaceHorses_q32”, a range in which the average pixel value is approximately 40 to 50, and For the range / sequence “BQSquare_q32” where the average pixel value is 225 or more, the square error reduction amount is negative in the range where the average pixel value is about 30 or less and the range where the average pixel value is about 235 or more. ing.

FIG. 11B is a graph showing the average (“average”) for a plurality of sequences and the total number of unit areas (“count”) having each average pixel value for the square error reduction amount at each average pixel value. It is.

As shown in FIG. 11B, the value of the graph indicating the average of a plurality of sequences is negative in the range where the average pixel value is about 30 or less and in the range where the average pixel value is about 230 or more.

Thus, for each sequence, there is a range of average pixel values where the square error reduction amount is negative. Based on the above experimental data, the inventor has found that the coding efficiency is improved by adopting a configuration in which the adaptive filter processing is not performed for the unit region having the average pixel value in which the square error reduction amount is negative. Obtained.

Hereinafter, a specific example of the classification processing by the region classification unit 172 according to the present modification will be described with reference to FIG. The region classification unit 172 can be configured to perform any of the classification processes among the following (classification process examples 3-1) to (classification process examples 3-7). In addition, among the following (Classification Processing Example 3-1) to (Classification Processing Example 3-7), classification processing designation information for designating which classification processing is performed is included in the encoded data # 1, and the region The classification unit 172 may be configured to selectively perform a classification process designated by the classification process designation information among (Classification process examples 3-1) to (Classification process examples 3-7).

In the following description, it is assumed that type 0 is a filter-off type, and types 1 to Non (Non is the total number of filter-on types) are filter-on types. The region classification unit 172 assigns the filter coefficient group I to the target unit region classified as type I (I = 1 to Non).

(Classification example 3-1)
In this processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

・ Type 0: MP ≦ ThM1
・ Type 1: ThM1 <MP
Here, ThM1 indicates a predetermined threshold value. The specific value of the threshold ThM1 may be, for example, a value determined in advance for each sequence in view of the experimental data shown in FIG. In particular,
-For the sequence "PartyScene_q32", Th1M = 30
-For the sequence “RaceHorses_q32”, Th1M = 50
-For the sequence “BQSquare_q32”, Th1M = 30
May be used. Further, when the threshold Th1M is determined without depending on the sequence, ThM1 = 30 may be used.

In this way, by setting a unit region whose average pixel value is equal to or less than a predetermined threshold as a type region 0, that is, a unit region that is not subjected to adaptive filter processing, negative contribution to coding efficiency can be reduced. As a result, encoding efficiency is improved.

(Classification example 3-2)
In this processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

・ Type 0: MP ≦ ThM1
Type 1: ThM1 <MP ≦ ThM2
・ Type 2: ThM2 <MP
Here, ThM2 indicates a predetermined threshold value, and it is assumed that ThM1 <ThM2. A specific value of the threshold ThM2 may be determined in advance so that encoding efficiency is improved.

Also in this processing example, a unit area whose average pixel value is equal to or less than a predetermined threshold is set to type 0, that is, a unit area for which no adaptive filter processing is performed. Therefore, the negative contribution to the coding efficiency can be reduced, so that the coding efficiency is improved. In addition, since any one of a plurality of filter coefficient groups is applied to the unit area where the filter processing is turned on according to the value of the average pixel value, the encoding efficiency can be further improved. It should be noted that a specific value of the threshold value ThM2 in the present processing example is preferably a value determined in advance so that the encoding efficiency is improved.

(Classification example 3-3)
In the present processing example, the area classification unit 172 classifies the target unit area as follows as shown in FIG.

・ Type 1: MP ≦ ThM2
・ Type 0: ThM2 <MP
Here, ThM2 indicates a predetermined threshold value. As a specific value of the threshold ThM2, for example, a value predetermined for each sequence may be used in view of the experimental data shown in FIG. In particular,
For the sequence “PartyScene_q32”, Th2M = 230
-For the sequence “RaceHorses_q32”, Th2M = 225
-For the sequence “BQSquare_q32”, Th2M = 235
May be used. Further, when the threshold Th2M is determined without depending on the sequence, ThM2 = 230 may be used.

In this way, by setting a unit region whose average pixel value is equal to or greater than a predetermined threshold as a type region 0, that is, a unit region that is not subjected to adaptive filter processing, negative contribution to coding efficiency can be reduced. As a result, encoding efficiency is improved.

(Classification example 3-4)
In this processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

・ Type 1: MP ≦ ThM1
・ Type 2: ThM1 <MP ≦ ThM2
・ Type 0: ThM2 <MP
Here, a specific value of the threshold ThM2 may be determined in the same manner as in the classification processing example 3-3. In addition, a specific value of the threshold ThM1 may be determined in advance so that encoding efficiency is improved. By using such threshold values ThM1 and ThM2, the encoding efficiency can be effectively improved.

(Classification example 3-5)
In the present processing example, the area classification unit 172 classifies the target unit area as follows as shown in FIG.

Type 0: MP ≦ ThM1 or ThM2 <MP
・ Type 2: ThM1 <MP ≦ ThM2
Here, the specific value of the threshold ThM1 in this processing example may be determined in the same manner as in the classification processing example 3-1. The specific value of the threshold ThM2 in this processing example may be determined in the same manner as in the classification processing example 3-3.

The encoding efficiency can be effectively improved by using such threshold values ThM1 and ThM2.

(Classification example 3-6)
In this processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

・ Type 1: ACT ≦ ThM1
・ Type 2: ThM1 <ACT ≦ ThM2
・ Type 3: ThM2 <ACT
Here, the specific values of the threshold values ThM1 and ThM2 in this processing example may be determined in advance so as to improve the encoding efficiency.

In the present processing example, the filter coefficient group can be switched according to the value of the average pixel value in the target unit region, so that the encoding efficiency is improved.

(Classification example 3-7)
As described above, the processing example in which the region classification unit 172 determines the on / off of the filtering process for the target unit region by comparing the average pixel value MP with a predetermined threshold value has been described. It is not limited.

For example, the filter parameter FP includes on / off designation information V for designating the on / off of the filter processing for each type, and the area classification unit 172 refers to the on / off designation information V, thereby performing the filter processing for each type. On / off may be determined. Hereinafter, an example of such a classification process will be described with reference to FIG. Unlike the above-described on / off information U, the on / off designation information V is information for designating the on / off of the filtering process for each type, and therefore the bit length depends on the total number of types.

FIG. 12G is a diagram illustrating a classification process in this process example. As shown in FIG. 12G, the region classification unit 172 classifies the target unit region as follows.
・ Type A: MP ≦ ThM
・ Type B: ThM <MP ≦ 2 × ThM
・ Type C: 2 × ThM <MP ≦ 3 × ThM
・ Type D: 3 × ThM <MP
Further, the region classification unit 172 refers to the on / off designation information V included in the filter parameter FP, and determines the on / off of the filter processing for each type. For example, when the on / off designation information V is “0101” in binary expression, type A and type C are set to the filter off type, and type B and type D are set to the filter on type.

When the target unit region belongs to the filter on type, the region classification unit 172 assigns a filter coefficient group included in the filter parameter FP to the target unit region. Here, when a plurality of filter coefficient groups are included in the filter parameter FP, the same selection rule as the selection rule used in the moving image encoding device 2 that generates the encoded data # 1 is used. Any one of the plurality of filter coefficient groups may be selected and assigned to the target unit area.

On the other hand, when the target unit region belongs to the filter-off type, the region classification unit 172 does not assign a filter coefficient group to the target unit region.

Also in this processing example, since the filter processing can be switched on and off for each range of the average pixel value, the encoding efficiency is improved.

<Modification 3>
In Modification 2 described above, the pixel value reference unit 171b included in the region characteristic value calculation unit 171 takes the average of the pixel values as the characteristic value of the deblocked decoded image P_DB in the target unit region to be processed. In the above description, the average pixel value MP is calculated, and the region classification unit 172 classifies the target unit region into one of a plurality of types based on the average pixel value MP. It is not limited to this.

In this modification, the pixel value reference unit 171b included in the region characteristic value calculation unit 171 calculates the average pixel value MP and the variance VP of each pixel value as the characteristic value of the deblocked decoded image P_DB in the target unit region. The area classification unit 172 classifies the target unit area into one of a plurality of types based on the average pixel value MP and the variance VP.

Here, if the target unit area is a rectangle of Bx pixels × By pixels, the coordinates of each pixel in the target unit area are (i, j) (i and j are 0 ≦ i ≦ Bx−1 and 0 ≦ j, respectively). ≦ By−1), and the pixel value of the deblocked decoded image P_DB at the coordinates is represented by S (i, j), the pixel value reference unit 171b may, for example, ) Can be used to calculate the variance VP.

Note that the variance VP calculated by the pixel value reference unit 171b functions as an index indicating the randomness of the deblocked decoded image P_DB in the target unit region, similarly to the activity ACT described above.

In addition, the pixel value reference unit 171b according to this modification example calculates the average pixel value MP using, for example, Equation (3).

The area classification unit 172 according to the present modification example sets the target unit area to one of a plurality of types based on the average pixel value MP and the variance VP calculated by the area characteristic value calculation unit 171 as the characteristic values for the target unit area. Classify. Here, the plurality of types include both a type in which filter processing is off (filter off type) and a type in which filter processing is on (filter on type). When the target unit region belongs to the filter-on type, the region classification unit 172 is a filter coefficient group included in the filter parameter FP supplied from the variable length code decoding unit 11, and corresponds to the type to which the target unit region belongs. The assigned filter coefficient group is assigned to the target unit area. On the other hand, when the target unit region belongs to the filter-off type, the region classification unit 172 does not assign a filter coefficient group to the target unit region.

Hereinafter, a specific example of the classification processing by the region classification unit 172 according to this modification will be described with reference to FIG. The region classification unit 172 may be configured to perform any of the classification processes from the following (Classification process example 4-1) to (Classification process example 4-2). Further, among the following (Classification Processing Example 4-1) to (Classification Processing Example 4-2), classification processing designating information for designating which classification processing is performed is included in the encoded data # 1, and the region The classification unit 172 may be configured to selectively perform the classification process designated by the classification process designation information among (Classification process example 4-1) to (Classification process example 4-2).

(Classification example 4-1)
In the present processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

・ Type 0: VP ≦ ThV1
Type 1: ThV1 <VP and MP ≦ ThM1
Type 2: ThV1 <VP and ThM1 <MP
Here, the threshold values ThM1 and ThV1 may be determined so that the coding efficiency is improved.

In the present processing example, the distribution unit VP, that is, the target unit region in which the randomness of the image is equal to or less than the threshold value ThV1 is set as a unit region that is not subjected to the filtering process. Therefore, the encoding efficiency is improved as in the configuration in which the unit area whose activity ACT is equal to or less than the predetermined threshold is set as a unit area that is not subjected to the filtering process.

Further, when the value of the variance VP is larger than the threshold value ThM1, the filter coefficient group is switched according to the magnitude of the average pixel value, so that the coding efficiency can be improved more effectively.

(Classification example 4-2)
In the present processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

Type 0: VP ≦ ThV1, or Th2 <VP, or ThM2 <MP
Type 1: ThV1 <VP ≦ ThV2 and MP ≦ ThM1
Type 2: ThV1 <VP ≦ ThV2 and ThM1 <MP ≦ ThM1
Here, the threshold values ThM1, ThV1, and ThM2 may be determined so that the coding efficiency is improved. Further, the value of the threshold ThV2 may be determined so as to reduce the processing amount while maintaining high coding efficiency.

In the present processing example, the unit region where the variance VP is equal to or less than the threshold value ThV1 and the unit region where the variance VP is greater than the threshold value ThV2 are set as the unit region where the filter process is not performed. Further, since the filter coefficient group is switched according to the average pixel value for the unit area to be subjected to the filter process, the encoding efficiency can be effectively improved.

<Modification 4>
As described above, the characteristic value of the deblocked decoded image P_DB in the target unit area is calculated, and the adaptive filter that switches on / off the filter process for the target unit area based on the calculated characteristic value has been described. The form is not limited to this.

In this modification, an on / off flag for designating on / off of the filter processing is included in the encoded data # 1 for each of the encoding units up to a predetermined division depth, and the adaptive filter and the on / off flag are deblocked. A configuration for switching on / off of the filtering process with reference to the characteristic value of the decoded image P_DB will be described with reference to FIGS. 14 and 15. FIG. 14 is a diagram illustrating a part of syntax included in the filter parameter FP (denoted as alf_param in FIG. 14) of the encoded data # 1 according to the present modification.

The configuration of the encoded data # 1 according to this modification is substantially the same as the configuration of the encoded data # 1 described above, but differs in the following points. That is, as shown in FIG. 14, in the filter parameter FP of the encoded data # 1 according to the present modification, (1) syntax adaptive_loop_filter_flag that specifies whether or not to perform adaptive filter processing on the target slice, (2) Syntax alf_cu_control_flag that specifies whether or not to turn on / off the adaptive filter processing for each coding unit, (3) Maximum division depth (maximum split) from the maximum coding unit for the coding unit that is the target of on / off control (depth) specifying syntax (hierarchy specifying information) alf_cu_control_max_depth, (4) syntax alf_length_cu_control_info specifying the number of coding units to be on / off controlled, and (5) coding unit to be on / off controlled Syntax (on / off flag) alf_cu_flag [i] that specifies on / off of adaptive filter processing for each It is included. FIGS. 15 (a) to 15 (d) show respective coding units to be subjected to on / off control in the maximum coding unit LCU when the value of alf_cu_control_max_depth is 0 to 3, together with a branch diagram showing a hierarchical structure. Show.

As shown in FIG. 15A, when alf_cu_control_max_depth = 0, the filter parameter FP includes an on / off flag alf_cu_flag [0] for the entire maximum coding unit LCU.

As shown in FIG. 15B, when alf_cu_control_max_depth = 1, the filter parameter FP includes on / off flags alf_cu_flag [0] to alf_cu_flag for each of the encoding units 0 to 3 one layer lower than the maximum encoding unit LCU. [3] is included.

As shown in FIG. 15C, when alf_cu_control_max_depth = 2, the filter parameter FP includes encoding unit 0 and encoding unit 7 that are one layer lower than the maximum encoding unit LCU, and maximum encoding unit LCU. On / off flags alf_cu_flag [0] to alf_cu_flag [7] for each of the encoding units 1 to 6 in the lower hierarchy are included.

As shown in FIG. 15 (d), when alf_cu_control_max_depth = 3, the filter parameter FP includes coding unit 0 and coding unit 10 that are one layer lower than the maximum coding unit LCU, and two layers from the maximum coding unit LCU. On-off flags alf_cu_flag [0] to alf_cu_flag [10] are included for each of the lower encoding units 1 to 4, the encoding unit 9, and the encoding units 5 to 8 that are three layers lower than the maximum encoding unit LCU. . FIG. 16A shows an activity ACT as a characteristic value for the target unit region and an on / off flag decoded from the encoded data # 1 by the region classification unit 172 included in the adaptive filter 17 according to this modification. An example of processing for classifying the target unit region into either type 0 or type 1 with reference to the on / off flag alf_cu_flag_ [i] for the coding unit including the target unit region is shown.

As shown in FIG. 16A, when the on / off flag alf_cu_flag_ [i] for the coding unit including the target unit region is 0, that is, the on / off flag alf_cu_flag [i] performs the filtering process on the coding unit. When it is specified that there is not, the area classification unit 172 classifies the target unit area as type 0 regardless of the magnitude of the characteristic value. Therefore, the filter processing unit 173 included in the adaptive filter 17 does not perform filter processing on such a target unit region.

On the other hand, when the on / off flag alf_cu_flag [i] for the coding unit including the target unit region is 1, that is, when the on / off flag alf_cu_flag [i] specifies that the filtering process is performed for the coding unit. The area classification unit 172 classifies the target unit area into any type according to the magnitude of the characteristic value. Therefore, the filter processing unit 173 included in the adaptive filter 17 performs the filter process only on the target unit region whose on / off flag alf_cu_flag_ [i] is 1 and classified into a type other than type 0.

In FIG. 16A, the activity is exemplified as the characteristic value. However, the present modification is not limited to this, and an average pixel value or directionality may be used as the characteristic value. . In addition, the present modification is classified based on the characteristic values described in the classification processing examples 1-1 to 1-7, 2-1 to 2-4, 3-1 to 3-7, and 4-1 to 4-2. Can be widely applied to a classification method obtained by combining any of the above and classification based on the on / off flag alf_cu_flag [i].

In the following, the case where alf_cu_control_max_depth = 0 and alf_cu_control_max_depth = 1 will be described as an example, but the same processing can be performed for alf_cu_control_max_depth = 2 and alf_cu_control_max_depth = 3.

FIG. 16B shows a case where alf_cu_control_max_depth = 0, and when the on / off flag alf_cu_flag [0] for the target LCU specifies ON, unit areas UR0 to UR0 to be obtained by dividing the target LCU. UR6 is shown. In FIG. 16B, unit areas classified into types other than type 0, that is, unit areas that are actually subjected to filter processing by the filter processing unit 173 are hatched. FIG. 16B illustrates a case where each unit area has a size and a position that are hierarchically specified by the CU division flag SP_CU included in the encoded data # 1. However, the present modification is not limited to this, and each unit area may have a predetermined size.

In addition, it is preferable that the unit area for calculating characteristics is equal to or less than the minimum CU (minimum coding unit). By setting the unit target area to be equal to or smaller than the minimum coding unit, it is possible to switch the filter processing in units that are the same as or smaller than the control unit performed in the coding unit.

Further, the unit area for calculating the characteristics is preferably a unit area composed of four or more pixels (2 × 2 pixels, 1 × 4 pixels, 4 × 1 pixels). As a result, on / off of filter processing and filter coefficient assignment are not switched for an excessively fine unit, the processing amount can be reduced. When the unit area for calculating the characteristic is small, the memory size for temporarily storing the calculated characteristic value until the filtering process is increased. However, the memory size can be reduced by using four or more pixels. Can be reduced.

FIG. 16C shows a case where alf_cu_control_max_depth = 1, and alf_cu_flag [0], alf_cu_flag [2] among the on / off flags alf_cu_flag [i] (i = 0 to 3) for the respective coding units CU0 to CU3. ] And alf_cu_flag [3] indicate ON, and alf_cu_flag [1] indicates OFF, the unit regions UR00 to UR33 obtained by dividing each CU are illustrated. In FIG. 16C as well, the unit areas classified into types other than type 0 are shaded. FIG. 16C illustrates a case where each unit area has a predetermined size. However, the present modification is not limited to this, and even when alf_cu_control_max_depth is not 0, each unit area includes the CU partitioning flag SP_CU included in the encoded data # 1 as in the example illustrated in FIG. It is also possible to have a size and position specified hierarchically by.

As is apparent from FIGS. 16B to 16C, the adaptive filter 17 according to the present modification switches on / off of the filter process in units smaller than the encoding unit corresponding to the maximum division depth specified by alf_cu_control_max_depth. be able to. That is, by setting the unit area for calculating the characteristics to a small unit that does not depend on the hierarchy designation information, even when the control area designated by the hierarchy designation information is large, the on / off control can be performed in a smaller unit. Therefore, since it is possible to perform fine on / off control while suppressing the code amount of the on / off flag, encoding efficiency is improved.

In addition, when the target unit area for performing characteristic calculation is determined according to the control unit determined by the hierarchy designation information, it is necessary to perform characteristic calculation processing according to the structure of the control unit. In this case, the processing becomes complicated with many branches, and the amount of calculation increases, and the scale of software and circuits increases. As described above, when the target unit area for performing characteristic calculation is determined without depending on the hierarchy designation information, the complexity of characteristic value calculation is reduced.

<Modification 5>
In (Classification Processing Example 2-4) and (Classification Processing Example 2-5), the region classification unit 172 explained the configuration for identifying the directionality according to the activity in each direction. The configuration for doing this is not limited to these examples.

As shown in FIG. 17, the area characteristic value calculation unit 171 in the present modification includes a directionality calculation unit 171c in addition to the activity calculation unit 171a. Here, the directionality calculation unit 171c calculates the edge angle Theta as the directionality of the deblocked decoded image P_DB in the target unit region. In addition, the directionality calculation unit 171c supplies the calculated edge angle Theta to the region classification unit 172. The region classification unit 172 selects a filter type according to the activity ACT and the edge angle Theta.

(Direction calculation unit 171c)
The directionality calculation unit 171c calculates the edge angle Theta of the deblocked decoded image P_DB in the target unit region. Hereinafter, the directionality calculation processing by the directionality calculation unit 171c will be described for each of the case where the target unit region includes only one pixel and the case where the target unit region includes a plurality of pixels.

(Direction calculation processing when the target unit area includes only one pixel)
When the target unit region includes only one pixel (target pixel), the directionality calculation unit 171c sets a 3 × 3 pixel reference region centered on the target pixel, and for each pixel in the reference region, The pixel values Sobel_x and Sobel_y after the Sobel filter are calculated by applying the Sobel filter matrices SFM_x and SFM_y defined by the following equations.

Subsequently, the directionality calculating unit 171c calculates the edge angle Theta by the following mathematical formula.

Theta = arctan (Sobel_x / Sobel_y)
Here, arctan represents an inverse function of the trigonometric function tan. The angle Theta increases clockwise and is expressed in a range from 0 degrees to 180 degrees (0 ° ≦ Theta <180 °).

(Theta calculation process when the target unit region includes a plurality of pixels)
When the target unit region includes a plurality of pixels, the directionality calculation unit 171c first sets a reference region of 3 × 3 pixels for each pixel included in the target unit region, and performs a source search for each pixel by the method described above. Pixel values Sobel_x and Sobel_y after the bell filter are calculated.

Subsequently, the directionality calculation unit 171c calculates the edge strength ES for each pixel by the following mathematical formula, and uses the pixel values Sobel_x and Sobel_y after the Sobel filter for the pixel having the largest edge strength ES. The obtained edge angle Theta is supplied to the region classification unit 172.

ES = (Sobel_x) ² + (Sobel_y) ²
The directionality calculation unit 171c may be configured to output a number (direction index) corresponding to the edge direction without calculating the angle of the edge.

For example, instead of using the inverse function of the trigonometric function tan, the direction index can be generated as follows by dividing the case according to the magnitude relation between Sobel_y and Sobel_x.

Hereinafter, a specific example of the classification processing by the region classification unit 172 according to the present modification will be described with reference to FIG. The area classification unit 172 may be configured to perform any of the classification processes among the following (classification process example 5-1) to (classification process example 5-2). Further, among the following (Classification Processing Example 5-1) to (Classification Processing Example 5-2), classification processing designating information for designating which classification processing is performed is included in the encoded data # 1, and the region The classification unit 172 may be configured to selectively perform the classification process designated by the classification process designation information among (Classification Process Example 5-1) to (Classification Process Example 5-2).

(Classification example 5-1)
In this processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

・ Type 0: ACT ≦ Th
Type 1: ACT> Th and (0 ° ≦ Theta <45 ° or 135 ° ≦ Theta <180 °)
Type 2: ACT> Th and 45 ° ≦ Theta <135 °
In the present processing example, a target unit region whose activity ACT is equal to or less than a predetermined threshold Th is set as a unit region that is not subjected to filter processing. Further, when the activity ACT is larger than the threshold Th, the target unit region is classified into one of two types according to the edge angle.

(Classification example 5-2)
In this processing example, the region classification unit 172 classifies the target unit region as follows, as shown in FIG.

・ Type 0: ACT ≦ Th1
Type 1: Th1 <ACT ≦ Th2 and (0 ° ≦ Theta <22.5 ° or 157.5 ° ≦ Theta <180 °)
Type 2: Th1 <ACT ≦ Th2 and 22.5 ° ≦ Theta <67.5 °
Type 3: Th1 <ACT ≦ Th2 and 67.5 ° ≦ Theta <112.5 °
Type 4: Th1 <ACT ≦ Th2 and 112.5 ° ≦ Theta <157.5 °
・ Type 5: ACT> Th2
In the present processing example, the target unit area whose activity ACT is equal to or less than the predetermined threshold Th1 is set as a unit area that is not subjected to the filter process. When the activity ACT is larger than the threshold value Th1 and equal to or smaller than the threshold value Th2, the target unit region is classified into one of four types (type 1 to type 4) according to the edge angle. When the activity ACT is larger than the threshold Th2, the target unit area is classified as type 5.

Note that the classification process in the present modification is not limited to the above example. For example, when the activity ACT is larger than the threshold Th2, the target unit region is selected from any of a plurality of types according to the edge angle. It is good also as a structure classified into these.

<Modification 6>
As shown in FIG. 19, the region characteristic value calculation unit 171 in this modification includes both an activity calculation unit 171a and a pixel value reference unit 171b. The region classification unit 172 selects a filter type according to the activity ACT and the average pixel value MP. Since the activity calculation processing by the activity calculation unit 171a and the average pixel value calculation processing by the pixel value reference unit 171b have already been described, description thereof will be omitted here.

Hereinafter, a specific example of the classification processing by the region classification unit 172 according to the present modification will be described with reference to FIG. The region classification unit 172 may be configured to perform any of the classification processes from the following (Classification process example 6-1) to (Classification process example 6-2). Further, among the following (Classification Processing Example 6-1) to (Classification Processing Example 6-2), classification processing designating information for designating which classification processing is performed is included in the encoded data # 1, and the region The classification unit 172 may be configured to selectively perform the classification process designated by the classification process designation information among (Classification process example 6-1) to (Classification process example 6-2).

(Classification process example 6-1)
In this processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

Type 0: ACT ≦ Th1, or ACT> Th2, or (Th1 <ACT ≦ Th2 and MP ≦ ThM1)
Type 1: Th1 <ACT ≦ Th2 and ThM1 <MP ≦ ThM2
Type 2: Th1 <ACT ≦ Th2 and ThM2 <MP
In this processing example, the target unit area whose activity ACT is equal to or smaller than the predetermined threshold Th1 and the target unit area whose activity ACT is larger than the predetermined threshold Th2 are set as unit areas not subjected to the filter process. Further, the target unit area whose activity ACT is larger than the threshold Th1 and equal to or smaller than the threshold Th2 and whose average pixel value MP is equal to or smaller than the predetermined threshold ThM1 is also set as a unit area not subjected to the filtering process. Further, when the activity ACT is larger than the threshold Th1 and equal to or smaller than the threshold Th2, and the average pixel value MP is larger than the threshold ThM1, it is classified into either type 1 or type 2 according to the average pixel value MP. To do.

(Classification example 5-2)
In this processing example, the area classification unit 172 classifies the target unit area as follows, as shown in FIG.

Type 0: ACT ≦ Th1, or (Th1 <ACT ≦ Th2 and MP ≦ ThM1), or (Th2 <ACT and MP ≦ ThM1)
Type 1: Th1 <ACT ≦ Th2 and ThM1 <MP ≦ ThM2
Type 2: Th1 <ACT ≦ Th2 and ThM2 <MP
Type 3: Th2 <ACT and ThM1 <MP
In the present processing example, the target unit area whose activity ACT is equal to or less than the predetermined threshold Th1 is set as a unit area that is not subjected to the filter process. Further, the target unit region whose average pixel value MP is equal to or less than the predetermined threshold ThM1 is also set as a unit region that is not subjected to the filtering process. Further, when the activity ACT is larger than the threshold Th1 and equal to or smaller than the threshold Th2, and the average pixel value MP is larger than the threshold ThM1, it is classified into either type 1 or type 2 according to the average pixel value MP. To do. Further, when the activity ACT is larger than the threshold value Th2 and the average pixel value MP is larger than the threshold value ThM1, it is classified as type 3.

(Moving picture encoding device 2)
The configuration of the video encoding device 2 according to the present embodiment will be described with reference to FIGS. The moving image encoding apparatus 2 includes H.264 as a part thereof. H.264 / MPEG-4. Coding including technology adopted in KTA software, which is a codec for joint development in AVC and VCEG (Video Coding Expert Group), and technology adopted in TMuC (Test Model under Consideration) software, which is the successor codec Device.

FIG. 21 is a block diagram showing a configuration of the moving picture encoding apparatus 2. As illustrated in FIG. 21, the moving image encoding device 2 includes a predicted image generation unit 21, a transform / quantization unit 22, an inverse quantization / inverse transform unit 23, an adder 24, a frame memory 25, a deblocking filter 26, An adaptive filter 27, a variable length code encoding unit 28, and a subtractor 29 are provided. Further, as shown in FIG. 21, the predicted image generation unit 21 includes an intra predicted image generation unit 21a, a motion vector detection unit 21b, an inter predicted image generation unit 21c, a prediction method control unit 21d, and a motion vector redundancy deletion unit. 21e. The moving image encoding device 2 is a device that generates encoded data # 1 by encoding moving image # 10 (encoding target image).

(Predicted image generation unit 21)
The predicted image generation unit 21 recursively divides the processing target LCU into one or a plurality of lower-order CUs, further divides each leaf CU into one or a plurality of partitions, and uses an inter-screen prediction for each partition. A predicted image Pred_Inter or an intra predicted image Pred_Intra using intra prediction is generated. The generated inter prediction image Pred_Inter and intra prediction image Pred_Intra are supplied to the adder 24 and the subtractor 29 as the prediction image Pred.

Note that the prediction image generation unit 21 omits encoding of other parameters belonging to the PU for the PU to which the skip mode is applied. Also, (1) the mode of division into lower CUs and partitions in the target LCU, (2) whether to apply the skip mode, and (3) which of the inter predicted image Pred_Inter and the intra predicted image Pred_Intra for each partition Whether to generate is determined so as to optimize the encoding efficiency.

(Intra predicted image generation unit 21a)
The intra predicted image generation unit 21a generates a predicted image Pred_Intra for each partition by intra prediction. Specifically, (1) a prediction mode used for intra prediction is selected for each partition, and (2) a prediction image Pred_Intra is generated from the decoded image P using the selected prediction mode. The intra predicted image generation unit 21a supplies the generated intra predicted image Pred_Intra to the prediction method control unit 21d.

In addition, the intra predicted image generation unit 21a determines an estimated prediction mode for the target partition from the prediction modes assigned to the peripheral partitions of the target partition, and the estimated prediction mode and the prediction mode actually selected for the target partition Are supplied as a part of the intra prediction parameter PP_Intra to the variable length code encoding unit 27 via the prediction scheme control unit 21d, and the variable length code encoding unit 27 The flag MPM is included in the encoded data # 1.

In addition, when the estimated prediction mode for the target partition is different from the prediction mode actually selected for the target partition, the intra predicted image generation unit 21a sets the residual prediction mode index RIPM indicating the prediction mode for the target partition. Then, as a part of the intra prediction parameter PP_Intra, the variable length code encoding unit 27 supplies the residual prediction mode index RIPM to the encoded data # 1 via the prediction scheme control unit 21d. It is assumed to be included

Note that the intra predicted image generation unit 21a may be configured to generate a predicted image Pred_Intra from the filtered decoded image P_ALF by intra-screen prediction.

(Motion vector detection unit 21b)
The motion vector detection unit 21b detects a motion vector mv regarding each partition. Specifically, (1) the filtered decoded image P_ALF ′ used as the reference image is selected, and (2) the target partition is searched by searching for the region that best approximates the target partition in the selected filtered decoded image P_ALF ′. Detects a motion vector mv. Here, the filtered decoded image P_ALF ′ is an image obtained by performing deblocking processing by the deblocking filter 26 and filtering processing by the adaptive filter 27 on the decoded image that has already been decoded for the entire frame. The motion vector detection unit 21b can read out the pixel value of each pixel constituting the filtered decoded image P_ALF ′ from the frame memory 25. The motion vector detection unit 21b supplies the detected motion vector mv to the inter predicted image generation unit 21c and the motion vector redundancy deletion unit 21e together with the reference image index RI that specifies the filtered decoded image P_ALF ′ used as the reference image. To do.

(Inter prediction image generation unit 21c)
The inter prediction image generation unit 21c generates a motion compensated image mc related to each inter prediction partition by inter-screen prediction. Specifically, the motion compensation image mc is obtained from the filtered decoded image P_ALF ′ designated by the reference image index RI supplied from the motion vector detection unit 21b, using the motion vector mv supplied from the motion vector detection unit 21b. Generate. Similar to the motion vector detection unit 21b, the inter prediction image generation unit 21c can read out the pixel value of each pixel constituting the filtered decoded image P_ALF ′ from the frame memory 25. The inter prediction image generation unit 21c supplies the generated motion compensated image mc (inter prediction image Pred_Inter) together with the reference image index RI supplied from the motion vector detection unit 21b to the prediction method control unit 21d.

(Prediction method controller 21d)
The prediction scheme control unit 21d compares the intra predicted image Pred_Intra and the inter predicted image Pred_Inter with the encoding target image, and selects whether to perform intra prediction or inter prediction. When intra prediction is selected, the prediction scheme control unit 21d supplies the intra prediction image Pred_Intra as the prediction image Pred to the adder 24 and the subtractor 29, and also uses the intra prediction parameter PP_Intra supplied from the intra prediction image generation unit 21a. This is supplied to the variable length code encoder 28. On the other hand, when the inter prediction is selected, the prediction scheme control unit 21d supplies the inter prediction image Pred_Inter as the prediction image Pred to the adder 24 and the subtractor 29, and the reference image index RI and motion vector redundancy described later. The estimated motion vector index PMVI and the motion vector residual MVD supplied from the deletion unit 21e are supplied to the variable length code encoding unit 27 as an inter prediction parameter PP_Inter.

(Motion vector redundancy deleting unit 21e)
The motion vector redundancy deletion unit 21e deletes redundancy in the motion vector mv detected by the motion vector detection unit 21b. Specifically, (1) an estimation method used for estimating the motion vector mv is selected, (2) an estimated motion vector pmv is derived according to the selected estimation method, and (3) the estimated motion vector pmv is subtracted from the motion vector mv. As a result, a motion vector residual MVD is generated. The motion vector redundancy deleting unit 21e supplies the generated motion vector residual MVD to the prediction method control unit 21d together with the estimated motion vector index PMVI indicating the selected estimation method.

(Transformation / quantization unit 22)
The transform / quantization unit 22 performs (1) DCT transform (Discrete Cosine Transform) for each block (transform unit) on the prediction residual D obtained by subtracting the predicted image Pred from the encoding target image, and (2) obtains the DCT transform. The obtained DCT coefficients are quantized, and (3) the quantized prediction residual QD obtained by the quantization is supplied to the variable length code encoding unit 27 and the inverse quantization / inverse transform unit 23. The transform / quantization unit 22 (1) selects a quantization step QP to be used for quantization for each TU, and (2) sets a quantization parameter difference Δqp indicating the size of the selected quantization step QP. This is supplied to the variable length code encoding unit 29, and (3) the selected quantization step QP is supplied to the inverse quantization / inverse transform unit 23. Here, the quantization parameter difference Δqp is the quantization parameter related to the TU that has been DCT transformed / quantized immediately before from the value of the quantization parameter qp (for example, QP = 2 ^{pq / 6} ) relating to the macroblock to be DCT transformed / quantized. It refers to the difference value obtained by subtracting the value of qp ′.

(Inverse quantization / inverse transform unit 23)
The inverse quantization / inverse transform unit 23 (1) inversely quantizes the quantized prediction residual QD, (2) performs inverse DCT (Discrete Cosine Transform) transformation on the DCT coefficient obtained by the inverse quantization, and (3) The prediction residual D obtained by the inverse DCT transform is supplied to the adder 24. When the quantization prediction residual QD is inversely quantized, the quantization step QP supplied from the transform / quantization unit 22 is used. Note that the prediction residual D output from the inverse quantization / inverse transform unit 23 is obtained by adding a quantization error to the prediction residual D input to the transform / quantization unit 22. Common names are used for this purpose.

(Adder 24)
The adder 24 adds the predicted image Pred selected by the prediction scheme control unit 21d to the prediction residual D generated by the inverse quantization / inverse transform unit 23, thereby obtaining the (local) decoded image P. Generate. The (local) decoded image P generated by the adder 24 is supplied to the deblocking filter 27 and stored in the frame memory 25, and is used as a reference image in intra prediction.

(Variable-length code encoding unit 28)
The variable-length code encoding unit 28 (1) the quantization prediction residual QD and Δqp supplied from the transform / quantization unit 22, and (2) the quantization parameter PP (inter prediction) supplied from the prediction scheme control unit 21d. The parameter PP_Inter and the intra prediction parameter PP_Intra) and (3) the filter parameter FP supplied from the adaptive filter 27 is variable-length-encoded to generate encoded data # 1.

As a specific encoding method by the variable-length code encoding unit 28, CABAC (Context-based Adaptive Binary Arithmetic Coding), which is one arithmetic coding / decoding method, or non-arithmetic encoding / decoding one method. CAVLC (Context-based Adaptive VLC) which is a method is used.

(Subtractor 29)
The subtracter 29 generates the prediction residual D by subtracting the prediction image Pred selected by the prediction method control unit 21d from the encoding target image. The prediction residual D generated by the subtractor 29 is DCT transformed / quantized by the transform / quantization unit 22.

(Deblocking filter 26)
When the difference between the pixel values of pixels adjacent to each other via a block boundary or partition boundary in the decoded image P is smaller than a predetermined threshold, the deblocking filter 26 By performing a deblocking process on the partition boundary, the block boundary or an image near the partition boundary is smoothed. The image subjected to the deblocking process by the deblocking filter 26 is stored in the frame memory 25 as a deblocked decoded image P_DB.

(Adaptive filter 27)
The adaptive filter 27 (1) sets one or a plurality of unit areas constituting the deblocked decoded image P_DB to any of a plurality of types according to the image characteristics of the deblocked decoded image P_DB in the unit area. (2) ON / OFF of adaptive filter processing for the unit area is switched according to the type to which the unit area belongs. Also, (3) filter coefficients for each type so that the error between the deblocked decoded image P_DB and the encoding target image (moving image # 10) in the unit area where the adaptive filter processing is set to ON is minimized. (4) Using the determined filter coefficient group, adaptive filter processing is performed on the deblocked decoded image P_DB.

FIG. 22 is a block diagram showing the configuration of the adaptive filter 27. As illustrated in FIG. 22, the adaptive filter 27 includes a region characteristic value calculation unit 271, a region classification unit 272, and a filter processing unit 273.

(Region characteristic value calculation unit 271)
The region characteristic value calculation unit 271 reads each pixel value of the deblocked decoded image P_DB in the characteristic value calculation reference region including the target unit region to be processed from the frame memory 15, and based on the pixel value, the target unit A characteristic value (also referred to as an area characteristic value) representing the image characteristic of the deblocked decoded image P_DB in the area is calculated. Here, the characteristic value calculation reference region refers to a region formed of pixels referred to for calculating the characteristic value of the deblocked decoded image P_DB in the target unit region. The characteristic value calculation reference area is composed of only the target unit area, or is composed of the target unit area and an area around the target unit area.

As shown in FIG. 22, the region characteristic value calculation unit 271 includes an activity calculation unit 271a. The activity calculation unit 271a calculates an activity ACT as the characteristic value of the deblocked decoded image P_DB in the target unit region. Further, the calculated activity ACT for each divided region is supplied to the region classification unit 272.

The specific method for calculating the activity level by the activity level calculation unit 271a is the same as the method for calculating the activity level by the activity level calculation unit 171a included in the moving image decoding apparatus 1, and thus the description thereof is omitted here.

(Area classification unit 272)
The region classification unit 272 classifies the target unit region into one of a plurality of types based on the activity ACT calculated by the region characteristic value calculation unit 271 as the characteristic value for the target unit region. Here, the plurality of types include both a type in which filter processing is off (filter off type) and a type in which filter processing is on (filter on type). The region classification unit 272 supplies type information indicating to which type the target unit region has been classified to the filter processing unit 273.

Specifically, the area classification unit 272 may be configured to perform any one of the following classification processing examples 1-1 ′ to (classification processing examples 1-6 ′). The region classification unit 272 is configured to selectively perform any one of the following (classification processing example 1-1 ′) to (classification processing example 1-6 ′), It is good also as a structure which includes the classification process designation | designated information which shows whether it performed to encoded data # 1. Note that type 0 is a filter-off type, and types 1 to Non (Non is the total number of filter-on types) are filter-on types.

(Classification Processing Example 1-1 ′) to (Classification Processing Example 1-6 ′)
The same processing as (classification processing example 1-1) to (classification processing example 1-6) by the region classification unit 172 is performed. Since the specific classification method has already been described, the description is omitted here.

(Classification example 1-7 ′)
Substantially the same processing as (classification processing example 1-7) by the region classification unit 172 is performed. However, when the region classification unit 172 performs the main classification process, the filter processing unit 273 has higher encoding efficiency for on / off of the filter processing for each type (for example, types A to D in FIG. 6G). The on / off designation information V indicating the determined on / off is supplied to the region classification unit 172.

(Filter processing unit 273)
The filter processing unit 273 classifies each unit region included in the target slice or the target LCU based on the type information supplied from the region classifying unit 272, and outputs the deblocked decoded image P_DB and the encoding target image (moving image). A filter coefficient group is determined for each type so that the error from # 10) is minimized.

More specifically, an image made up of unit regions classified as type I in the deblocked decoded image P_DB in the target slice or the target LCU is represented as an image I, and coordinates of pixels included in the image I are (x, The pixel value of the pixel that is y) is expressed as SI (x, y), and the pixel value of the pixel whose coordinates are (x, y) among the pixels included in the encoding target image in the target slice or the target LCU is ST ( x, y), the filter processing unit 273 derives, for example, a filter coefficient aI (i, j) and an offset oI that minimize the square error E given by the following equation (5). .

The filter processing unit 273 derives a filter coefficient group I including the filter coefficient aI (i, j) and the offset oI for all unit areas classified into types other than type 0 by the area classifying unit 272. For example, when the target slice or the target LCU is configured from unit areas classified into type 0, type 1, and type 2, the filter processing unit 273 performs processing for the unit areas classified into type 1. A filter coefficient group 1 and a filter coefficient group 2 for a unit region classified as type 2 are derived.

Further, the filter processing unit 273 performs filtering on the deblocked decoded image P_DB in each unit region included in the target slice or the target LCU by performing filter processing using the filter coefficient group derived for each type. A decoded image P_ALF is generated. For example, when the target slice or the target LCU is composed of unit areas classified into type 0, type 1, and type 2, the filter processing unit 273 performs the deblocked decoding in the target slice or the target LCU. A filter process using the filter coefficient group 1 is performed on a unit area classified as type 1 in the image P_DB, and a filter process using the filter coefficient group 2 is performed on a unit area classified as type 2.

Since the specific filter processing by the filter processing unit 273 is the same as the filter processing by the filter processing unit 173 included in the video decoding device 1, description thereof is omitted here.

The filter coefficient group for each type derived by the filter processing unit 273 is supplied to the variable length code encoding unit 28 as a part of the filter parameter FP.

<Modification 1 '>
The activity calculation unit 271a included in the region characteristic value calculation unit 271 uses, as the characteristic value of the deblocked decoded image P_DB in the target unit region, the vertical activity ACT_V and the horizontal activity instead of the activity ACT described above. The degree ACT_H may be calculated, and the region classification unit 272 may classify the target unit region into one of a plurality of types based on the horizontal direction activity ACT_H and the vertical direction activity ACT_V.

The region classification unit 272 classifies the target unit region into one of a plurality of types based on the horizontal direction activity ACT_H and the vertical direction activity ACT_V calculated by the activity calculation unit 271a. The region classification unit 272 supplies type information indicating to which type the target unit region has been classified to the filter processing unit 273.

The classification processing by the region classification unit 272 according to the present modification is the same as the classification processing by the region classification unit 172 described in <Modification 1>, and thus description thereof is omitted here.

<Modification 2 '>
The adaptive filter 27 calculates an average of pixel values (also referred to as an average pixel value) as a characteristic value of the deblocked decoded image P_DB in the target unit region, and sets the target unit region to a plurality of types based on the calculated average pixel value. It is good also as a structure classified into either.

FIG. 23 is a block diagram showing a configuration of the adaptive filter 27 according to this modification. As shown in FIG. 23, the region characteristic value calculation unit 271 in the adaptive filter 27 according to this modification includes a pixel value reference unit 271b.

The pixel value reference unit 271b calculates the average pixel value MP by reading each pixel value of the deblocked decoded image P_DB in the target unit area to be processed from the frame memory 25 and taking the average of the pixel values.

Since the calculation method of the average pixel value MP by the pixel value reference unit 271b is the same as the calculation method of the average pixel value MP by the pixel value reference unit 171b included in the moving image decoding device 1, the description thereof is omitted here.

In addition, the region classification unit 272 according to the present modification classifies the target unit region into one of a plurality of types based on the average pixel value MP calculated by the region characteristic value calculation unit 271 as the characteristic value for the target unit region. To do. Here, the plurality of types include both a type in which filter processing is off (filter off type) and a type in which filter processing is on (filter on type). The region classification unit 272 supplies type information indicating to which type the target unit region has been classified to the filter processing unit 273.

Specifically, the area classification unit 272 can be configured to perform any of the following classification processing examples (classification processing example 3-1 ′) to (classification processing example 3-7 ′). The region classification unit 272 is configured to selectively perform any one of the following (classification processing examples 3-1 ′) to (classification processing examples 3-7 ′). It is good also as a structure which includes the classification process designation | designated information which shows whether it performed to encoded data # 1. Note that type 0 is a filter-off type, and types 1 to Non (Non is the total number of filter-on types) are filter-on types.

(Classification Processing Example 3-1 ′) to (Classification Processing Example 3-6 ′)
Processing similar to (classification processing example 3-1) to (classification processing example 3-6) by the area classification unit 272 is performed. Since the specific classification method has already been described, the description is omitted here.

(Classification example 3-7 ')
Substantially the same processing as (Classification processing example 3-7) by the region classification unit 272 is performed. However, when the region classification unit 272 performs the main classification process, the filter processing unit 273 has higher encoding efficiency for on / off of the filter processing for each type (for example, types A to D in FIG. 12G). The on / off designation information V indicating the determined on / off is supplied to the region classification unit 272.

<Modification 3 '>
The pixel value reference unit 271b included in the region characteristic value calculation unit 271 calculates the average pixel value MP and the variance VP of each pixel value as the characteristic value of the deblocked decoded image P_DB in the target unit region, and the region classification unit 272 The target unit area may be classified into one of a plurality of types based on the average pixel value MP and the variance VP.

The calculation process of the variance VP by the pixel value reference unit 271b according to this modification is the same as the calculation process of the variance VP by the pixel value reference unit 171b described in <Modification 3>, and thus the description thereof is omitted here.

The region classification unit 272 according to the present modification example sets the target unit region to one of a plurality of types based on the average pixel value MP and the variance VP calculated by the region characteristic value calculation unit 271 as the characteristic value for the target unit region. Classify. Here, the plurality of types include both a type in which filter processing is off (filter off type) and a type in which filter processing is on (filter on type). The region classification unit 272 supplies type information indicating to which type the target unit region has been classified to the filter processing unit 273.

The classification processing by the region classification unit 272 according to the present modification is the same as the classification processing by the region classification unit 172 described in <Modification 3>, and thus description thereof is omitted here.

<Modification 4 '>
The adaptive filter 27 according to this modification example (1) determines whether or not to perform adaptive filter processing on the target slice, and supplies a syntax adaptive_loop_filter_flag indicating the determined result to the variable-length code encoding unit 28; (2) a function for determining whether to control on / off of the adaptive filter processing for each coding unit, and supplying a syntax alf_cu_control_flag indicating the determined result to the variable-length code encoding unit 28; and (3) on / off control. The maximum division depth (maximum split depth) from the maximum coding unit is determined for the coding unit to be processed, and the syntax (hierarchy designation information) alf_cu_control_max_depth indicating the determined result is input to the variable-length code encoding unit 28 The function to be supplied and (4) the number of coding units to be subjected to on / off control is determined, and the syntax alf_length_cu_control_info indicating the determined result is variable (5) A syntax (on / off flag) alf_cu_flag [i] indicating the result of determining the on / off of the adaptive filter processing for each coding unit to be subjected to on / off control ] To the variable-length code encoding unit 28.

Further, the adaptive filter 27 according to this modification can be configured to perform the determinations (1) to (5) so that the encoding efficiency is further improved.

In a video encoding device, it is desirable to test encoding in a plurality of hierarchy designation information, and to perform optimization such as selecting and coding the optimum hierarchy. However, characteristic calculation is performed depending on the hierarchy designation information. When the position and size of the unit area to be changed are changed, it is necessary to redo the characteristic calculation every time encoding in different hierarchy designation information is tested. Such recalculation is not appropriate because it increases the amount of calculation.

The region classification unit 272 according to the present modification refers to the characteristic value for the target unit region and the on / off flag alf_cu_flag [i] for the encoding unit including the target unit region, and sets the target unit region to a plurality of types. It classifies in either.

The classification processing by the region classification unit 272 according to the present modification is the same as the classification processing by the region classification unit 172 described in <Modification 4>, and thus description thereof is omitted here.

<Modification 5 '>
As shown in FIG. 24, the region characteristic value calculation unit 271 according to this modification includes a directionality calculation unit 271c in addition to the activity calculation unit 271a. The directionality calculation unit 271c calculates the edge direction Theta in the same manner as the directionality calculation unit 171c described in <Modification 5>.

The classification process by the area classification unit 272 according to this modification is the same as the classification process by the area classification unit 172 described in <Modification 5>.

<Modification 6 '>
As shown in FIG. 25, the region characteristic value calculation unit 271 according to the present modification includes a pixel value reference unit 271b in addition to the activity calculation unit 271a. The classification process by the area classification unit 272 according to the present modification is the same as the classification process by the area classification unit 172 described in <Modification 6>.

(Appendix 1)
As described above, the image filter device according to the present invention includes a filter unit that operates on an input image composed of a plurality of unit areas, and a characteristic value that represents an image characteristic of the input image for each of the plurality of unit areas. A characteristic value calculation means for calculating the filter area, and a filter-on area that causes the filter means to act on each unit area according to the value of the characteristic value for the unit area calculated by the characteristic value calculation means, or the filter And classifying means for classifying the filter into any one of the filter-off regions in which the means is not operated.

The classifying unit further classifies each unit region belonging to the filter-on region into one of a plurality of types according to the value of the characteristic value for the unit region calculated by the characteristic value calculating unit. The filter means preferably calculates the pixel value of each pixel in each unit region belonging to the filter-on region using a filter coefficient group set for a type including the pixel.

According to the above configuration, the classification unit assigns each of the unit areas belonging to the filter-on area to any one of a plurality of types according to the value of the characteristic value for the unit area calculated by the characteristic value calculation unit. Further, the filter means calculates the pixel value of each pixel in each unit region belonging to the filter-on region using a filter coefficient group set for the type including the pixel. Therefore, according to the above configuration, even when the image characteristics of the input image are different for each unit region, it is possible to perform the filtering process using the filter coefficient group set for each type. That is, even when the image characteristics of the input image are different for each unit region, an output image that is more suitable for generating a predicted image can be generated. Therefore, prediction accuracy and encoding efficiency can be improved by using an output image generated by the image filter device according to the present invention configured as described above as a reference image for generating a predicted image.

Further, it is preferable that the characteristic value calculation means calculates the activity of the input image in the unit area as a characteristic value for each unit area.

According to the above configuration, the characteristic value calculating means calculates the activity as a characteristic value representing the image characteristic of the input image in each unit area. Therefore, according to the above configuration, an output image that is more suitable for generating a predicted image can be generated even when the input image has a different degree of activity for each unit region. The activity level can also be expressed as an index indicating the randomness of the image.

In addition, the characteristic value calculation means calculates the activity of each pixel belonging to the internal region, the entire neighborhood of which is composed of pixels included in the unit region, as the pixel value of the pixel included in the vicinity of the pixel. It is preferable to calculate the activity of the unit region from the activity of each pixel belonging to the inner region after being derived with reference to FIG.

According to the above configuration, in each unit region, the activity of each pixel belonging to the internal region whose entire neighborhood is composed of pixels included in the unit region is referred to the pixel value of the pixel included in the vicinity of the pixel. Then, the activity for the unit region is calculated from the activity of each pixel belonging to the internal region. Therefore, according to the above configuration, the degree of activity for each unit region can be calculated by referring to only the pixels in the unit region. In other words, the degree of activity for a certain unit area can be calculated without referring to pixels that are not included in the unit area.

Therefore, according to the above configuration, it is possible to reduce the data transfer amount and the processing amount for calculating the activity while maintaining high encoding efficiency.

Further, it is preferable that the characteristic value calculation means calculates an average of pixel values of the input image in the unit area as a characteristic value for each unit area.

According to the above configuration, the characteristic value calculating means calculates an average of the pixel values in the unit area as a characteristic value representing the image characteristic of the input image in each unit area. Therefore, according to the above configuration, even when the input image has an average of different pixel values for each unit region, an output image that is more suitable for generating a predicted image can be generated.

Further, it is preferable that the characteristic value calculation means calculates a plurality of characteristic values having different calculation methods for each unit region.

According to the above configuration, the characteristic value calculation means calculates a plurality of characteristic values having different calculation methods for each unit region. Therefore, according to the above configuration, each unit region can be classified more appropriately, so that the encoding efficiency can be further improved.

Further, the characteristic value calculation means calculates an average of pixel values of the input image in the unit area and a variance of pixel values of the input image in the unit area as characteristic values for each unit area. Is preferred.

According to the above configuration, the characteristic value calculation means includes, as characteristic values for each unit region, an average of the pixel values of the input image in the unit region, and a variance of the pixel values of the input image in the unit region. Is calculated. Therefore, according to the above configuration, each unit region can be classified more appropriately, so that the encoding efficiency can be further improved.

Note that the dispersion of pixel values can also be regarded as an index indicating the randomness of the image. Further, the variance of the pixel values is obtained by taking the difference between the square average of the pixel values and the square of the average of the pixel values. Here, as the average of the pixel values used for calculating the variance of the pixel values, the average value of the pixel values calculated as one of the characteristic values can be used as it is, while suppressing an increase in the processing amount. Thus, each unit area can be classified more appropriately.

Further, the characteristic value calculation means calculates the vertical activity of the input image in the unit region and the horizontal activity of the input image in the unit region as the characteristic value for each unit region. Is preferable.

According to the above configuration, the characteristic value calculation means uses the vertical activity of the input image in the unit region and the horizontal activity of the input image in the unit region as the characteristic value for each unit region. Calculate the degree. Therefore, according to said structure, according to the directionality of the activity of an input image, each unit area | region can be classified appropriately. Further, since the unit areas are classified using the activity in each direction, the processing amount can be reduced as compared with the configuration in which the unit areas are classified using the two-dimensional activity.

In addition, the classification unit obtains an on / off flag for designating on / off of the filtering process for each predetermined unit including one or more unit areas from outside, and the unit area included in the unit for which the on / off flag designates on Is preferably classified into either the filter-on region or the filter-off region.

According to the above configuration, the unit area included in the unit in which the on / off flag designates on is classified into either the filter on area or the filter off area. Therefore, according to the above configuration, the code amount of data to be referred to is reduced as compared with the configuration in which the on / off of the filtering process is switched only by the on / off flag.

When the image filter device is used in an encoding device that encodes an image and a decoding device that decodes an image from encoded data, the predetermined unit is, for example, an encoding / decoding unit. A coding unit (CU: Coding Unit) can be used.

Further, the predetermined unit related to the on / off flag is hierarchically specified by hierarchical specification information acquired from the outside, and the unit region is composed of pixels that are equal to or smaller than the minimum encoding unit and equal to or larger than four pixels. It is preferable that the area is determined regardless of the hierarchy designation information.

According to the above configuration, the predetermined unit is hierarchically specified by hierarchical designation information acquired from the outside, and the unit area is determined without depending on the hierarchical designation information. Therefore, according to the above configuration, it is possible to switch the filtering process on and off for each finer unit while reducing the code amount of the hierarchy designation information. In addition, the complexity of the characteristic value calculation is reduced as compared with the case of changing with the hierarchy designation information. In addition, in the encoding device, it is not necessary to recalculate the characteristic value when determining the optimum hierarchy designation information, so that the amount of calculation is reduced. Moreover, according to said structure, the said unit area | region is comprised from 4 or more pixels. Therefore, according to the above configuration, since the filter processing is not switched on and off for an excessively small unit, the processing amount can be reduced.

The decoding device according to the present invention is a decoding device that decodes encoded data and generates a decoded image after filtering, wherein the image filter device and the predicted image in each unit region are represented by the image filter device. A prediction image generation unit that generates a reference image by referring to the generated output image, and the image filter device includes a prediction image generated by the prediction image generation unit, a residual image decoded from the encoded data, and It is characterized in that the decoded image obtained by adding is used as an input image, and the decoded image after filtering is generated as an output image.

According to the decoding device according to the present invention configured as described above, the image filter device included in the decoding device includes the prediction image generated by the prediction image generation unit and the residual decoded from the encoded data. Since the decoded image obtained by adding the difference image is used as the input image, and the filtering process is switched on / off for each unit area according to the characteristic value representing the image characteristic of the input image (decoded image), the input image Even when the image characteristics are different for each unit region, an output image suitable as an image to be referred to for generating a predicted image can be generated. Therefore, according to the above configuration, the prediction accuracy of the predicted image is improved.

According to the encoding device having the configuration corresponding to the above configuration, it is possible to improve the prediction accuracy of the predicted image even when the image characteristics of the encoding target image in each unit region are different for each unit region. Therefore, encoded data with high encoding efficiency can be generated. Moreover, according to the decoding apparatus having the above configuration, it is possible to appropriately decode such encoded data with high encoding efficiency.

An encoding apparatus according to the present invention is an encoding apparatus that generates encoded data by encoding a residual image between an encoding target image and a prediction image, the image filter device, and each unit Prediction image generation means for generating the prediction image in the region with reference to the output image generated by the image filter device, and the image filter device includes the prediction image generated by the prediction image generation means and the prediction image generation device. A decoded image obtained by adding the residual image is used as an input image, and an output image is generated.

According to the encoding apparatus according to the present invention having the above-described configuration, the image filter apparatus included in the encoding apparatus includes a prediction image generated by the prediction image generation unit, an encoding target image, and a prediction image. Since the decoded image obtained by adding the residual images is used as the input image, and the filtering process is switched on and off for each unit area according to the characteristic value representing the image characteristics of the input image (decoded image), the image of the input image Even when the characteristics are different for each unit region, it is possible to generate an output image suitable as an image to be referred to for generating a predicted image. Therefore, according to the above configuration, the prediction accuracy of the predicted image is improved. Since the prediction accuracy is improved, the encoding device can generate encoded data with high encoding efficiency.

In the encoding device according to the present invention, the filter coefficient group used by the filter means minimizes the difference between the encoding target image and the output image in each unit region included in the filter-on region. It is preferable that it is determined to be.

According to the above configuration, the filter coefficient group used by the filter means minimizes the difference between the encoding target image and the output image in each unit area included in the filter-on area for the filter-on area. Therefore, the encoding device including the filter unit can generate a prediction image with high prediction accuracy. Thereby, encoded data with high encoding efficiency can be generated. When each unit region belonging to the filter-on region is further classified into any of the plurality of types, the filter coefficient group used by the filter unit is an encoding target image and the output image. It is determined to minimize the difference for each type.

(Appendix 2)
Each block of the moving picture decoding apparatus 1 and the moving picture encoding apparatus 2 described above may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or may be a CPU (Central Processing Unit). You may implement | achieve by software using.

In the latter case, each device includes a CPU that executes instructions of a program that realizes each function, a ROM (Read （Memory) that stores the program, a RAM (Random Memory) that expands the program, the program, and various types A storage device (recording medium) such as a memory for storing data is provided. An object of the present invention is to provide a recording medium in which a program code (execution format program, intermediate code program, source program) of a control program of each of the above devices, which is software that realizes the above-described functions, is recorded so as to be readable by a computer. This can also be achieved by supplying to each of the above devices and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, and disks including optical disks such as CD-ROM / MO / MD / DVD / CD-R. IC cards (including memory cards) / optical cards, semiconductor memories such as mask ROM / EPROM / EEPROM / flash ROM, or PLD (Programmable logic device) or FPGA (Field Programmable Gate Array) Logic circuits can be used.

Further, each of the above devices may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited as long as it can transmit the program code. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication network, and the like can be used. The transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type. For example, even with wired lines such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared rays such as IrDA and remote control, Bluetooth (registered trademark), IEEE 802.11 wireless, HDR ( It can also be used by radio such as High Data Rate (NFC), Near Field Communication (NFC), Digital Living Network Alliance (DLNA), mobile phone network, satellite line, and digital terrestrial network.
(Technical background regarding Embodiments 2 to 4)
In order to efficiently transmit or record a moving image, a moving image encoding device (encoding device) that generates encoded data by encoding the moving image, and decoding by decoding the encoded data A video decoding device (decoding device) that generates an image is used. As a specific moving picture encoding method, for example, H.264 is used. H.264 / MPEG-4. Examples thereof include a method adopted in KTA software which is a codec for joint development in AVC and VCEG (Video Coding Expert Group).

In such an encoding method, an image (picture) constituting a moving image is obtained by dividing a slice obtained by dividing an image, a macroblock obtained by dividing the slice, and a macroblock. It is managed by a hierarchical structure consisting of blocks to be encoded, and is usually encoded for each block.

In such an encoding method, a predicted image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and difference data between the predicted image and the input image is encoded. It becomes. As methods for generating a predicted image, methods called inter-screen prediction (inter prediction) and intra-screen prediction (intra prediction) are known.

In intra prediction, predicted images in a corresponding frame are sequentially generated based on a locally decoded image in the same frame. Specifically, in intra prediction, usually, one prediction direction is selected from prediction directions included in a predetermined prediction direction (prediction mode) group for each prediction unit (for example, block), and A prediction pixel value on the prediction target region is generated by extrapolating the pixel value of the reference pixel in the locally decoded image in the selected prediction direction. Also, in inter prediction, by applying motion compensation using a motion vector to a reference image in a reference frame (decoded image) in which the entire frame is decoded, a predicted image in a prediction target frame is converted into a prediction unit ( For example, it is generated for each block).

Non-Patent Document 3 divides a (local) decoded image into a plurality of regions, and determines each region according to the level of activity indicating the local randomness of the (local) decoded image in each region. Discloses an adaptive loop filter (hereinafter simply referred to as “adaptive filter”) that performs filter processing while switching the filter coefficient group. Here, the filter coefficient group for each region is determined by the encoding device so as to minimize the error between the filtered image in the region and the encoding target image.

An encoding device and a decoding device provided with such an adaptive filter generate a prediction image with reference to a filtered decoded image obtained by performing filter processing on the (local) decoded image using the adaptive filter. As a result, it is possible to improve the prediction accuracy and the encoding efficiency.
(Task)
However, the conventional adaptive filter still has room for improving the encoding efficiency.

As a method for improving the coding efficiency, a method of switching the filter coefficient group for each region based on region characteristics other than the activity in addition to the activity may be considered. However, when the filter coefficient group is switched for each region based on the two region characteristics, there is a problem that the calculation cost on the encoding device side and the decoding device side becomes high.

The present invention has been made in view of the above-described problems, and an object of the present invention is to realize an adaptive filter with higher encoding efficiency than before without causing a significant increase in calculation cost.
(Configuration for solving the above problems)
In order to solve the above-described problems, a filter device according to the present invention includes a region characteristic calculation unit that calculates a value of a region characteristic of each unit region in a filter device including a linear filter in which filter coefficients are adaptively set. The linear filter is an offset that depends on the region characteristic for performing a filter operation on each pixel belonging to each unit region, and the value of the region characteristic of the unit region calculated by the region characteristic calculating unit This is performed using an offset according to the above and a filter coefficient group that does not depend on the region characteristics.

In the filter device according to the present invention, the region characteristic has a value corresponding to a color difference component type, and the linear filter performs the filter operation using an offset corresponding to the color difference component type. preferable.

When the color difference component types are different, the optimum offset value (maximizing encoding efficiency) is different. Therefore, according to the above configuration, higher encoding efficiency can be achieved than when the filter operation is performed using an offset that does not depend on the type of the color difference signal.

In order to solve the above problem, a filter device according to the present invention is a filter device including a linear filter in which filter coefficient groups are adaptively set, and a pixel value of a target pixel in an output image of the linear filter And truncating means for truncating the change amount, which is the difference between the pixel value of the target pixel in the input image of the linear filter.

The filter device according to the present invention includes a flag decoding unit that decodes a flag indicating whether or not to truncate the change amount, and outputs an output image of the linear filter by truncating the change amount or outputting the change amount. It is desirable to further include a switch for switching whether to output without truncation according to the value of the flag decoded by the flag decoding means.

According to the above configuration, it is possible to avoid a situation in which the decoding accuracy / encoding efficiency is lowered due to the additional amount of truncation. In addition, since it is determined on the encoding device side whether or not the change amount truncation is valid, a decrease in decoding accuracy / encoding efficiency can be avoided without imposing an excessive load on the decoding device. .

The filter device according to the present invention is used to truncate the change amount when executing a region characteristic calculation unit that calculates a region characteristic of each unit region and a filter process that uses a pixel belonging to each unit region as an output pixel. It is desirable that the apparatus further comprises threshold value determining means for determining the threshold value in the unit area according to the area characteristic calculated by the area characteristic calculating means.

At this time, the threshold value determination means may determine the threshold value based on a correspondence relationship between the predetermined region characteristic and the threshold value, or may determine the region characteristic determined on the encoding device side. The threshold value may be determined based on a correspondence relationship with the region characteristic. In the latter case, for example, a configuration further including a threshold table decoding unit that decodes a threshold table recorded by associating the region characteristics with the threshold is employed.

According to the above configuration, whether or not to truncate the change amount can be changed according to the region characteristics, so that the decoding accuracy / encoding efficiency can be further improved.

In order to solve the above-described problem, in a filter device including a linear filter in which filter coefficients are adaptively set, at least one first filter coefficient group depending on the first region characteristic corresponds to the first filter coefficient group. The first filter coefficient table recorded in association with the value of the region characteristic and the second filter coefficient group depending on the second region characteristic are recorded in association with the corresponding second region characteristic value. A decoding means for decoding the two-filter coefficient table; and an area characteristic calculating means for calculating the values of the first area characteristic and the second area characteristic of each unit area, wherein the linear filter calculates each pixel belonging to each unit area. A filter operation for a target pixel, and the first of the unit areas calculated by the area characteristic calculation means in the first filter coefficient group recorded in the first filter coefficient table; Of the first filter coefficient group corresponding to the area characteristic value and the second filter coefficient group recorded in the second filter coefficient table, the second area characteristic of the unit area calculated by the area characteristic calculating means A filter operation using a second filter coefficient group corresponding to the value is executed.

The filter device according to the present invention includes: a flag decoding unit that decodes a flag indicating whether or not to use the second filter coefficient group for a filter operation in the linear filter; the first filter coefficient group in the linear filter; Whether to output an output image obtained by a filter operation using the second filter coefficient group to the outside or to output an output image obtained by a filter operation using only the first filter coefficient group in the linear filter to the outside It is desirable to further include a switch that switches according to the value of the flag decoded by the flag decoding means.

According to the above configuration, it is possible to avoid a situation in which decoding accuracy / encoding efficiency is lowered by using the second filter coefficient group for the filter operation in the linear filter. In addition, since the determination as to whether or not the use of the second filter coefficient group is effective is performed on the encoding device side, the decoding accuracy / encoding efficiency is reduced without imposing an excessive load on the decoding device. Can be avoided.

A decoding device and an encoding device including the filter device are also included in the scope of the present invention.
(Effect of the above configuration)
According to the present invention, it is possible to realize an adaptive filter having higher encoding efficiency than the conventional one without causing a significant increase in calculation cost.
<< Embodiment 2 >>
A second embodiment of the present invention will be described with reference to FIGS.

(Configuration example 1)
A first configuration example of the adaptive filter according to the present embodiment will be described with reference to FIG.

The adaptive filter 100 shown in FIG. 26 is an adaptive filter suitable for mounting in a decoding device, and includes a filter setting unit 110 and a filter calculation unit 120.

The filter setting unit 110 is a means for setting filter characteristics, and includes a filter coefficient decoding unit 111.

The filter coefficient decoding unit 111 decodes a filter coefficient group w = (w [1], w [2],..., W [N], c) from the encoded data. The filter coefficient group w decoded by the filter coefficient decoding unit 111 is supplied to the filter calculation unit 120.

The filter calculation unit 120 is a means for executing a filter calculation, and includes a linear filter 121, a change amount calculation unit 122, a change amount restriction unit 123, a pixel value change unit 124 (truncating means), and a threshold value determination unit 125. ing.

The linear filter 121 has pixel values s [1], s [2],..., S [N] of reference pixels in the input image (decoded image or decoded image after deblocking) according to the equation (A1). From this, the pixel value s ′ of the target pixel in the output image is calculated.

s ′ = w [1] s [1] + w [2] s [2] +... + w [N] s [N] + c
... (A1)
Here, w [1], w [2],..., W [N], c are elements of the filter coefficient group w decoded by the filter coefficient decoding unit 121. The calculated pixel value s ′ is supplied to the subsequent change amount limiting unit 123.

The change amount calculation unit 122 subtracts the pixel value s ′ of the target pixel in the output image of the linear filter 121 from the pixel value s of the same pixel (a pixel occupying the same position as the target pixel) in the input image of the linear filter 121. The calculated difference Δs = s−s ′ is supplied to the change amount limiting unit 123 at the subsequent stage. In the present specification, this difference Δs is described as “amount of change”.

The change amount limiting unit 123 truncates the change amount Δs according to the equation (A2). The truncated change amount Δs ′ is supplied to the subsequent pixel value changing unit 124.

Δs ′ = th (Δs> th)
= Δs (−th ≦ Δs ≦ th) (A2)
= -Th (Δs <-th)
Note that the method of truncating the change amount Δs is not limited to this. That is, for example, the change amount limiting unit 123 may truncate the change amount Δs according to the equation (A2 ′).

Δs ′ = 0 (Δs> th)
= Δs (−th ≦ Δs ≦ th) (A2 ′)
= 0 (Δs <-th)
The pixel value changing unit 124 subtracts the truncated change amount Δs ′ from the pixel value s in the input image of the linear filter 121. The calculated difference s ″ = s−Δs ′ is output to the outside of the filter calculation unit 120.

The threshold determination unit 125 determines the threshold th used for truncating the change amount Δs. In this configuration example, a predetermined threshold th0 is used as the threshold th used to truncate the change amount Δs. In this case, the threshold value determination unit 125 reads the threshold value th0 from a memory (not shown) and notifies the change amount restriction unit 123 described above.

In addition, the filter calculation part 120 should just obtain the same calculation result as the calculation result mentioned above, and does not need to follow the calculation procedure same as the calculation procedure mentioned above. For example, instead of the configuration in which Δs = s−s ′ is calculated by the change amount calculation unit 122 and the difference s ″ = s−Δs ′ is calculated by the pixel value change unit 124, the difference is calculated by the change amount calculation unit 122. A configuration may be adopted in which Δs = s′−s is calculated and the sum s ″ = s + Δs ′ is calculated by the pixel value changing unit 124. It will be apparent that the same result can be obtained with the latter configuration as with the former configuration.

Also, the elements of the filter coefficient group w decoded by the filter coefficient decoding unit 121 may be quantized filter coefficients. In this case, the linear filter 121 may be configured such that (a) the quantized filter coefficient is dequantized and then the filter operation using the dequantized filter coefficient is performed, and (b) A configuration in which the filter operation using the filter coefficient before inverse quantization is performed and then the result of the filter operation is shifted to the right may be employed. For example, when the quantization accuracy of the filter coefficient group w is 8 bits, the filter coefficient after quantization is 256 times the filter coefficient before quantization, so by shifting the filter operation result to the right by 8 bits The same calculation result can be obtained except for calculation errors such as roundoff error and the case of performing filter calculation using filter coefficients before dequantization.

When the decoding apparatus is equipped with the adaptive filter 100 shown in FIG. 26, the coding apparatus may be equipped with the adaptive filter 100 'shown in FIG. As shown in FIG. 27, in the adaptive filter 100 ′ on the encoding device side, the filter coefficient decoding unit 111 in the adaptive filter 100 on the decoding device side is replaced with a filter coefficient determination unit 111a and a filter coefficient encoding unit 111b. Configuration is taken.

In the adaptive filter 100 ′ on the encoding device side, (1) the filter coefficient determination unit 111a minimizes the square error with respect to the original image (encoding target image) of the output image of the linear filter 121 using the least square method. And (2) the filter coefficient encoding unit 111b encodes the filter coefficient group w determined by the filter coefficient determination unit 111a. The other configuration of the adaptive filter 100 ′ on the encoding device side is the same as that of the adaptive filter 100 on the decoding device side, and the description thereof is omitted here.

(Configuration example 2)
Next, a second configuration example of the adaptive filter will be described with reference to FIG.

The adaptive filter 100 shown in FIG. 28 is an improvement of the adaptive filter 100 according to the first configuration example, and is suitable for mounting in a decoding device, like the adaptive filter 100 according to the first configuration example. . The differences between the adaptive filter 100 according to this configuration example and the adaptive filter 100 according to the first configuration example are the following two points.

In the present configuration example, a restriction flag decoding unit 112 is added to the filter setting unit 110. The restriction flag decoding unit 112 is means for decoding the restriction flag from the encoded data. Here, the restriction flag refers to a flag that specifies whether to output the pre-truncated pixel value s ″ or the post-truncated pixel value s ′ as the pixel value of the target pixel in the output image.

In the present configuration example, a switch 126 is added to the filter calculation unit 120. The switch 126 determines whether to output the post-truncated pixel value s ″ or the pre-truncated pixel value s ′ as the pixel value of the target pixel in the output image. It is means for switching according to the value of the flag.

When the adaptive filter 100 shown in FIG. 28 is installed in the decoding apparatus, the encoding apparatus may be equipped with the adaptive filter 100 ′ shown in FIG. 29. As shown in FIG. 29, in the adaptive filter 100 ′ on the encoding device side, instead of the restriction flag decoding unit 112 of the adaptive filter 100 on the decoding device side, a restriction flag determining unit 112a and a restriction flag encoding unit 112b are provided. It is added to the filter setting unit 110.

In the adaptive filter 100 ′ on the encoding device side, (1) the restriction flag determination unit 112a determines whether or not the change amount truncation is valid for each unit area, and then the value of the restriction flag according to the determination result. (2) The restriction flag encoding unit 112b encodes the value of the restriction flag determined by the restriction flag determination unit 112a. For example, in the unit area in which the truncation of the change amount is determined to be valid, the limit flag determination unit 112a sets the first value (eg, “1”) as the limit flag and determines that the truncation of the change amount is not valid. In the unit area, the restriction flag is set to a second value (for example, “0”) different from the first value. Other configurations of the adaptive filter 100 ′ on the encoding device side are the same as those shown in the first configuration example.

It should be noted that an arbitrary area obtained by dividing a frame can be used as a determination unit for determining whether or not the change amount truncation is effective. For example, slice, macroblock, block, LCU (Largest Coding Unit), CT (Coding Tree), CU (Coding Unit), PT (Prediction Tree), PU (Prediction Unit), TT (Transformation Tree), TU (Transformation Unit) ) Or M × N block (M represents the width of the block, for example, takes a value such as 1, 2, 4, or 8. N represents the height of the block, for example 1, 2, 4 or the like. Or a value such as 8, etc. The same applies to the following) can be used as a determination unit for determining whether or not the truncation of the change amount is effective. For example, when the determination unit of whether or not the change amount truncation is valid is a slice, the restriction flag encoding unit 112b encodes a restriction flag for each slice and embeds the code in each slice header. In this case, in the adaptive filter 100 on the decoding device side, the switch 126 is switched for each slice.

As a method for determining whether or not the change amount truncation is valid, for example, (1) an error when truncating the change amount (error with respect to the original image of the output image of the pixel value changing unit 124), and the change amount And (2-1) error when truncating the change amount, error when not truncating the change amount, and calculating the error when the change amount is not truncated (error relative to the original image of the output image of the linear filter 121) If the change amount is smaller than that, it is determined that the change amount truncation is effective. (2-2) If not, the change amount truncation is determined not to be effective. As another method, (1) the encoding efficiency when the change amount is truncated and the encoding efficiency when the change amount is not truncated are calculated, and (2-1) the change amount is truncated. When the encoding efficiency of the above becomes higher than the encoding efficiency when the change amount is not truncated, it is determined that the change amount truncation is effective. (2-2) Otherwise, There is a method of determining that the truncation of the change amount is not effective.

In this way, by performing the truncation of the change amount only in the unit for which the truncation of the change amount is determined to be valid on the encoding device side, the decoding accuracy / encoding efficiency is reduced by the truncation of the change amount. The situation can be avoided. In addition, since it is determined on the encoding device side whether or not the change amount truncation is valid, a decrease in decoding accuracy / encoding efficiency can be avoided without imposing an excessive load on the decoding device. .

(Configuration example 3)
Next, a third configuration example of the adaptive filter 100 will be described with reference to FIG.

An adaptive filter 100 shown in FIG. 30 is an improvement of the adaptive filter 100 according to the second configuration example, and is suitable for mounting in a decoding device, like the adaptive filter 100 according to the second configuration example. . The differences between the adaptive filter 100 according to this configuration example and the adaptive filter 100 according to the second configuration example are the following two points.

In the present configuration example, a region characteristic calculation unit 127 is added to the filter calculation unit 120. The area characteristic calculation unit 127 calculates the area characteristic k of each unit area based on the input image (decoded image or decoded image_DB after deblocking).

In the present configuration example, the threshold value determination unit 125 determines the value of the threshold value th used for the filter calculation for pixels belonging to each unit area. More specifically, the threshold value th (k) associated with the region characteristic k calculated by the region characteristic calculation unit 127 is specified in the threshold value table TH described later.

Here, the calculation unit of the region characteristic k may be any of a slice, macroblock, block, LCU, CT, CU, PT, PU, TT, TU, or M × N block. For example, when the calculation unit of the region characteristic k is a macro block, the threshold value determination unit 125 sets the region of the macro block as the threshold value th used for the filter calculation for pixels belonging to the macro block for each macro block. The value th (k) corresponding to the characteristic k is specified.

It should be noted that the region characteristic k that can be used to determine the threshold th includes, for example, activity. The activity is a publicly known index indicating the local randomness of the decoded image. When the activity is used to determine the threshold th, the region characteristic calculation unit 127 calculates, for example, the activity k of each unit region according to the equation (A3).

k = Σ | s (i, j) −s (i + 1, j) |
+ Σ | s (i, j) −s (i, j + 1) | (A3)
Here, (i, j) represents coordinates with the upper left corner of the target unit region as the origin, and S (i, j) represents a pixel (i in the input image (decoded image or decoded image_DB after deblocking)). , J), and Σ represents the sum over all the pixels constituting the target unit region. Expression (A3) corresponds to evaluating the degree of activity using the sum of absolute values of differences from adjacent pixel values (Sum of Absolute Difference, SAD) as an index.

Note that the activity is not limited to the one defined by the equation (A3) as long as it is an index of local randomness of the decoded image. For example, when the size (number of pixels) of the unit region is not constant, the normalized activity k ′ obtained by dividing the activity k defined by the equation (A3) by the size (number of pixels) M of the unit region. = K / M may be used instead of the activity k.

Further, as a method for determining the threshold th according to the activity k, for example, a method of referring to the threshold table TH can be mentioned. As the threshold table TH, for example, a table in which the activity k and the threshold th (k) corresponding to the activity k are recorded in association with each other can be used. In this case, the threshold determination unit 125 specifies the threshold th (k) associated with the activity k calculated by the activity calculation unit 127 in the threshold table TH. An example of the threshold table TH is shown in Table 1.

Note that in Table 1, the value of the threshold th is decreased as the activity decreases. This is to prevent linking that may occur when a filter process with a large change amount Δs is applied to a flat portion with low activity.

The threshold table TH referred to for determining the threshold th may be (1) created in advance or (2) created on the encoding device side based on the original image. Alternatively, (3) it may be created on the decoding device side based on the decoded image, but in this configuration example, a threshold table TH created in advance is referred to. That is, the threshold value determination unit 125 reads out and uses a predetermined threshold value table TH from the memory.

As described above, by setting the threshold th used for the filter calculation with the pixel belonging to each unit region as the target pixel to a value corresponding to the characteristic amount indicating the region characteristic of the unit region, noise (edge) due to the application of the filter processing is set. Occurrence of blurring around the periphery and linking occurring on a flat portion can be avoided.

When the adaptive filter 100 shown in FIG. 30 is installed in the decoding apparatus, the adaptive filter 100 ′ shown in FIG. 31 may be installed in the encoding apparatus. In the adaptive filter 100 ′ on the encoding device side, a region characteristic calculation unit 127 is added to the filter calculation unit 120 as in the adaptive filter 100 on the decoding device side. Similar to the region characteristic calculation unit 127 in the adaptive filter 100 on the decoding device side, the region characteristic calculation unit 127 in the adaptive filter 100 ′ on the encoding device side applies an input image (local decoded image or local decoded image_DB after deblocking) to the input image. Based on this, the region characteristic k of each unit region is calculated. Other configurations of the adaptive filter 100 ′ on the encoding device side are the same as those shown in the second configuration example.

In addition, in this structure, although the threshold value th was demonstrated based on the area | region characteristic k (for example, activity) of each unit area, it was not limited to this. That is, a configuration may be adopted in which the threshold th is determined based on an encoding parameter (for example, a quantization parameter) regarding each unit region. For example, when adopting a configuration in which the threshold th is determined based on the quantization parameter, the threshold th is relatively increased in the region where the quantization parameter value is set to be large, and the quantization parameter value is set to be small. The threshold th may be set relatively small in the region. This is because, in the region where the quantization parameter is small, the coding noise is also small, and therefore the filter operation with a large change amount may reduce the decoding accuracy.

(Configuration example 4)
Next, a fourth configuration example of the adaptive filter will be described with reference to FIG.

The adaptive filter 100 shown in FIG. 29A is an improvement of the adaptive filter 100 according to the third configuration example, and is suitable for mounting in a decoding device, like the adaptive filter 100 according to the third configuration example. Is. The differences between the adaptive filter 100 according to this configuration example and the adaptive filter 100 according to the third configuration example are the following four points.

In this configuration example, a threshold table decoding unit 113 is added to the filter setting unit 110. The threshold table decoding unit 113 decodes the threshold table TH from the encoded data. Here, as described above, the threshold value table TH refers to a table in which the area characteristic k and the threshold value th (k) corresponding to the area characteristic k are recorded in association with each other.

In the present configuration example, the threshold value determination unit 125 determines the threshold value th (k) used in the filter calculation for the pixels belonging to each unit area. More specifically, the threshold value th (k) associated with the region characteristic k calculated by the region characteristic calculation unit 127 is specified in the threshold table TH decoded by the threshold table decoding unit 113.

In this configuration example, the filter coefficient decoding unit 111 in the adaptive filter 100 according to the third configuration example is replaced with a filter coefficient table decoding unit 114. The filter coefficient table decoding unit 114 decodes the filter coefficient table W from the encoded data. Here, the filter coefficient table W refers to a table in which the region characteristic k and the filter coefficient group w (k) corresponding to the region characteristic k are recorded in association with each other.

In the present configuration example, a filter coefficient determination unit 128 is added to the filter calculation unit 120. The filter coefficient determination unit 128 determines a filter coefficient group w (k) used in the filter calculation for pixels belonging to each unit region. More specifically, in the filter coefficient table W decoded by the filter coefficient table decoding unit 114, the filter coefficient group w (k) associated with the region characteristic k calculated by the region characteristic calculation unit 127 is specified.

Here, the unit for calculating the region characteristic k may be any of a slice, macroblock, block, LCU, CT, CU, PT, PU, TT, TU, or M × N block. For example, when the unit for calculating the region characteristic k is a macro block, the threshold value determination unit 125 determines a threshold value th (k) corresponding to the region characteristic k of the macro block for each macro block, and the filter coefficient determination unit 128. For each macroblock, a filter coefficient group w (k) corresponding to the region characteristic k of the macroblock is determined.

When the decoding apparatus is equipped with the adaptive filter 100 shown in FIG. 32, the coding apparatus may be equipped with the adaptive filter 100 'shown in FIG. In the adaptive filter 100 ′ on the encoding device side, (1) instead of the threshold table decoding unit 113 in the adaptive filter 100 on the decoding device side, a threshold table creation unit 113 a and a threshold table encoding unit 113 b include a filter setting unit 110. (2) Instead of the filter coefficient table decoding unit 114 in the adaptive filter 100 on the decoding device side, a filter coefficient table creation unit 114a and a filter coefficient table encoding unit 114b are added to the filter setting unit 110. .

In the adaptive filter 100 ′ on the encoding device side, (1) the threshold value table creating unit 113a creates the threshold value table TH, and (2) the threshold value table coding unit 113b is created by the threshold value table creating unit 113a. The table TH is encoded. Here, the threshold table creation unit 113a determines each element of the threshold table TH by the following processing.

That is, the threshold value table creating unit 113a uses the filter coefficient table W created by the filter coefficient table creating unit 114a and threshold values TH1 to THNt (Nt is the total number of threshold values) prepared in advance, for each area characteristic k. Filter processing is performed when a threshold value of a type is used. A square error between the image after filtering and the encoding target image is calculated. Subsequently, a value that minimizes the square error among Nt square errors is set as the threshold th (k) for each region characteristic k.

In the adaptive filter 100 ′ on the encoding device side, (1) the filter coefficient table creation unit 114a creates the filter coefficient table W, and (2) the filter coefficient table coding unit 114b performs the filter coefficient table creation unit. The filter coefficient table W created by 114a is encoded. Here, the filter coefficient table creation unit 114a determines a filter coefficient table W that minimizes a square error with respect to the original image (encoding target image) of the output image of the linear filter 121 using the least square method.

In this configuration, the configuration in which the threshold table TH itself and the filter coefficient table W itself are encoded is shown, but the configuration of the adaptive filter 100 on the encoding device side is not limited to this. In particular, when the threshold table TH / filter coefficient table W is selected from a plurality of predetermined threshold tables TH / filter coefficient tables W, the index of the selected threshold table TH / filter coefficient table W is encoded. You may employ | adopt the structure which makes it. Usually, the code obtained by encoding the index of the threshold table TH / filter coefficient table W is shorter than the code obtained by encoding the threshold table TH / filter coefficient table W itself. Therefore, if such a configuration is adopted, encoding efficiency can be improved.
<< Embodiment 3 >>
A third embodiment of the present invention will be described with reference to FIGS.

[Configuration example 1]
A first configuration example of the adaptive filter according to the present embodiment will be described with reference to FIG.

The adaptive filter 200 shown in FIG. 34 is an adaptive filter suitable for mounting in a decoding device, and includes a filter setting unit 210 and a filter calculation unit 220.

The filter setting unit 210 is a means for setting filter characteristics, and includes a filter coefficient decoding unit 211.

The filter coefficient decoding unit 211 decodes the first filter coefficient table W1, the offset c, and the second filter coefficient table W2 from the encoded data. Here, the first filter coefficient table W1 is a table in which a first region characteristic k1 described later and a first filter coefficient group w1 (k1) are recorded in association with each other, and the second filter coefficient table W2 is a first filter coefficient table W2 described later. It is the table which recorded 2 area | region characteristic k2 and 2nd filter coefficient group w2 (k2) mutually linked | related.

The filter calculation unit 220 is a means for executing a filter calculation, and includes a first spatial filter 221, an offset addition unit 222, a second linear filter 223, an addition unit 224, and a region characteristic calculation unit 225.

The first spatial filter 221 uses the pixel values s1 [1], s1 [2],..., S1 [] of the reference pixels in the input image (decoded image or decoded image after deblocking) according to the equation (B1). N1], the pixel value s1 ′ of the target pixel in the output image is calculated.

s1 ′ = w1 (k1) [1] s1 [1] + w1 (k1) [2] s1 [2] +.
+ W1 (k1) [N1] s1 [N1] (B1)
Here, w1 (k1) [1], w1 (k1) [2],..., W1 (k1) [N1] are unit regions including the target pixel calculated by the region characteristic calculation unit 225 described later. Are the elements of the first filter coefficient group w1 (k1) corresponding to the first region characteristic k1. More specifically, in the first filter coefficient table W1 decoded by the filter coefficient decoding unit 221, the first filter coefficient group w1 (k1) associated with the first region characteristic k1 calculated by the region characteristic calculation unit 225. Elements. The pixel value s1 ′ calculated by the first spatial filter 221 is supplied to the subsequent offset addition unit 222.

Note that the reference region (collection of reference pixels) in the first spatial filter 221 will be described later with reference to another drawing.

The offset addition unit 222 is a means for adding the offset c to s1 ′ calculated by the first spatial filter 221. Here, c is an offset decoded by the filter coefficient decoding unit 221. The sum s <b> 1 ′ + c calculated by the offset addition unit 222 is supplied to the subsequent addition unit 224.

The second spatial filter 223 uses the pixel values s2 [1], s2 [2],..., S2 [] of the reference pixel in the input image (decoded image or decoded image after deblocking) according to the equation (B2). N2], the pixel value s2 ′ of the target pixel in the output image is calculated.

s2 ′ = w2 (k2) [1] s2 [1] + w2 (k2) [2] s2 [2] +.
+ W2 (k2) [N2] s2 [N2] (B2)
Here, w2 (k2) [1], w2 (k2) [2],..., W2 (k2) [N2] are unit regions including the target pixel calculated by the region characteristic calculation unit 225 described later. This is an element of the second filter coefficient group w2 (k2) corresponding to the second region characteristic k2. More specifically, in the second filter coefficient table W2 decoded by the filter coefficient decoding unit 221, the second filter coefficient group w2 (k2) associated with the second region characteristic k2 calculated by the region characteristic calculation unit 225. Elements. The pixel value s2 ′ calculated by the second spatial filter 223 is supplied to the subsequent addition unit 224.

Note that the reference region (collection of reference pixels) in the second spatial filter 223 will be described later with reference to another drawing.

The addition unit 224 adds s2 ′ calculated by the second spatial filter 223 to the sum s1 ′ + c calculated by the offset addition unit 222. The sum s 1 ′ + c + s 2 ′ calculated by the adder 224 is output to the outside of the adaptive filter 200.

The region characteristic calculation unit 225 calculates a first region characteristic k1 and a second region characteristic k2 for each unit region. The first region characteristic k1 and the second region characteristic k2 are different region characteristics, for example, activity and edge direction. Arbitrary areas obtained by dividing a frame, for example, slices, macroblocks, blocks, LCUs, CTs, CUs, PTs, PUs, TTs, TUs, or M × N blocks, a first area characteristic k1 and a second area The calculation unit of the characteristic k2 can be used. For example, when the calculation unit of the first region characteristic k1 and the second region characteristic k2 is a macro block, the first filter coefficient group w1 (k1) and the second filter coefficient group w2 (k2) are updated for each macro book. Will be.

Next, reference regions (collection of reference pixels) in the first spatial filter 221 and the second spatial filter 223 will be described with reference to FIGS.

FIG. 35 is a diagram illustrating a setting example of the reference area of the first spatial filter 221. In the rectangular area composed of 5 × 5 pixels shown in the figure, the center pixel corresponds to the target pixel, and the black pixel corresponds to the reference pixel. In addition, white characters written on black pixels indicate filter coefficients to be multiplied by the pixel values of the pixels. Although these filter coefficients are determined depending on the first region characteristic k1, the notation of “(k1)” is omitted in FIG.

FIG. 36 is a diagram illustrating a setting example of the reference area of the second spatial filter 223 used together with the first spatial filter 221 in which the reference area is set as illustrated in FIG. (A) shows a reference area when k2 = 0, (b) shows a reference area when k2 = 1, (c) shows a reference area when k2 = 2, (d ) Indicates a reference area when k2 = 3. In the rectangular area composed of 5 × 5 pixels shown in each figure, the center pixel corresponds to the target pixel, and the black pixel corresponds to the reference pixel. In addition, white characters written on black pixels indicate filter coefficients to be multiplied by the pixel values of the pixels. Although these filter coefficients are determined depending on the second region characteristic k2, the notation of “(k2)” is omitted in FIG.

When the reference region of the first spatial filter 221 and the reference region of the second spatial filter 223 match, the first filter coefficient group w1 (k1) and the second filter coefficient group w2 (k2) are linearly dependent. Therefore, the encoding device cannot calculate the first filter coefficient group w1 (k1) and the second filter coefficient group w2 (k2). However, as shown in FIGS. 35 and 36, if the reference region of the first spatial filter 221 is different from the reference region of the second spatial filter 223, the first filter coefficient group w1 (k1 ) And the second filter coefficient group w2 (k2) cannot be calculated.

FIG. 37 is a diagram illustrating another setting example of the reference region of the first spatial filter 221. In the rectangular area composed of 7 × 5 pixels shown in the figure, the center pixel corresponds to the target pixel, and the black pixel corresponds to the reference pixel. In addition, white characters written on black pixels indicate filter coefficients to be multiplied by the pixel values of the pixels. Although these filter coefficients are determined depending on the first region characteristic k1, the notation of “(k1)” is omitted in FIG.

FIG. 38 is a diagram illustrating a setting example of the reference region of the second spatial filter 223 used together with the first spatial filter 221 in which the reference region is set as illustrated in FIG. (A) shows a reference area when k2 = 0, (b) shows a reference area when k2 = 1, (c) shows a reference area when k2 = 2, (d ) Indicates a reference area when k2 = 3. In the rectangular area composed of 5 × 5 pixels shown in each figure, the center pixel corresponds to the target pixel, and the black pixel corresponds to the reference pixel. In addition, white characters written on black pixels indicate filter coefficients to be multiplied by the pixel values of the pixels. Although these filter coefficients are determined depending on the second region characteristic k2, the notation of “(k2)” is omitted in FIG.

37 and 38, the reference region of the first spatial filter 221 is different from the reference region of the second spatial filter 223. Therefore, there is no possibility that the first filter coefficient group w1 (k1) and the second filter coefficient group w2 (k2) cannot be calculated in the encoding device.

As described above, the adaptive filter 200 according to this configuration example includes (1) the first filter coefficient table W1 in which a plurality of first filter coefficient groups w1 (k1) are recorded in association with the corresponding first region characteristics k1, And a filter coefficient decoding unit 211 for decoding the second filter coefficient table W2 in which a plurality of second filter coefficient groups w2 (k2) are recorded in association with the corresponding second area characteristic k2, and (2) each unit area An area characteristic calculation unit 225 for calculating the first area characteristic k1 and the second area characteristic k2, and (3) a filter operation using each pixel belonging to each unit area as a target pixel, which is recorded in the first filter coefficient table W1. Among the plurality of first filter coefficient groups w (k1) that have been calculated, the filter coefficient group w1 (k1) corresponding to the first area characteristic k1 of the unit area calculated by the area characteristic calculation unit 225, and Among the plurality of second filter coefficient groups w (k2) recorded in the second filter coefficient table W2, the filter coefficient group w2 (k2) corresponding to the second area characteristic k2 of the unit area calculated by the area characteristic calculation unit 225 And a filter calculation unit 220 that executes a filter calculation using the above.

Therefore, when the adaptive filter 200 according to this configuration example is used, a plurality of filter coefficient groups w (k1, k2) depending on both the first region characteristic k1 and the second region characteristic k2 are converted into corresponding region characteristic pairs (k1 , K2), the filter operation corresponding to both the first region characteristic k1 and the second region characteristic k2 of each unit region can be performed without decoding the filter coefficient table W recorded in association with the unit region. That is, if the adaptive filter according to this configuration example is used, a decoded image close to the original image can be accurately reproduced without causing a significant increase in the code amount.

Note that the elements of the first filter coefficient group w1 (k1) and the second filter coefficient group w2 (k2) decoded by the filter coefficient decoding unit 221 may be quantized filter coefficients. Further, the offset c decoded by the filter coefficient decoding unit 221 may also be a quantized offset. In this case, the linear filter comprising the first spatial filter and the offset adding unit 222 performs (a) a filter operation using the filter coefficient after the inverse quantization after dequantizing the quantized filter coefficient and the offset. Or (b) a configuration in which a filter operation using a filter coefficient and an offset before inverse quantization is performed, and then a result of the filter operation is shifted to the right. Similarly, the linear filter including the second spatial filter may be configured to perform a filter operation using the filter coefficient after inverse quantization after dequantizing the quantized filter coefficient, b) A configuration in which the filter operation using the pre-quantization filter coefficient is performed and then the result of the filter operation is shifted to the right may be employed. For example, when the quantization accuracy of the filter coefficient group w is 8 bits, the filter coefficient after quantization is 256 times the filter coefficient before quantization, so by shifting the filter operation result to the right by 8 bits The same calculation result as that of the configuration (a) is obtained.

When the adaptive filter 200 (FIG. 34) according to this configuration example is installed in the decoding apparatus, the encoding apparatus may be equipped with the adaptive filter 200 'shown in FIG. As shown in FIG. 39, in the adaptive filter 200 ′ on the encoding device side, the filter coefficient decoding unit 211 in the adaptive filter 200 on the decoding device side is replaced with a filter coefficient table creation unit 211a and a filter coefficient table encoding unit 211b. The replacement configuration is taken.

In the adaptive filter 200 ′ on the encoding device side, (1) the filter coefficient table creation unit 211a creates the first filter coefficient table W1 and the second filter coefficient table W2, and (2) the filter coefficient table coding unit 211b. Encodes the first filter coefficient table W1 and the second filter coefficient table W2 created by the filter coefficient table creation unit 211aa. Here, the filter coefficient table creation unit 211a uses the least square method to minimize the square error with respect to the original image of the filter output image (the output image of the addition unit 224) and the second filter coefficient table W1. The table W2 is determined.

[Configuration example 2]
A second configuration example of the adaptive filter according to the present embodiment will be described with reference to FIG.

40 is an adaptive filter suitable for mounting in a decoding device, and includes a filter setting unit 210 and a filter calculation unit 220.

The filter setting unit 210 is a means for setting filter characteristics, and includes a filter coefficient decoding unit 211 and a second region characteristic use flag decoding unit 212.

The filter coefficient decoding unit 211 decodes the first filter coefficient table W1, the offset c, and the second filter coefficient table W2 from the encoded data, as in the first configuration example.

The second region characteristic use flag decoding unit 212 decodes the second region characteristic use flag enable_dir from the encoded data. The second region characteristic use flag enable_dir is a 1-bit specifying whether or not to perform a filter operation using the filter coefficient group w2 (k2) determined according to the second region characteristic k2 in the filter operation for each unit region. The flag is “1” when the filter operation using the filter coefficient group w2 (k2) is to be performed, and the value “0” when the filter operation using the filter coefficient group w2 (k2) is not to be performed. Take. Note that the setting unit of the second region characteristic use flag enable_dir is arbitrary, and may or may not be the same as the calculation unit of the first region characteristic k1 and the second region characteristic k2.

The filter calculation unit 220 is a means for performing a filter calculation, and includes a first spatial filter 221, an offset addition unit 222, a second linear filter 223, an addition unit 224, a region characteristic calculation unit 225, and a switch 226. ing.

Similar to the first configuration example, the first spatial filter 221 has pixel values s1 [1] and s1 of reference pixels in the input image (decoded image or decoded image after deblocking) according to the above-described equation (B1). From [2],..., S1 [N1], the pixel value s1 ′ of the target pixel in the output image is calculated.

The offset addition unit 222 adds the offset c to s1 ′ calculated by the first spatial filter 221 as in the first configuration example. However, in this configuration example, the sum s1 ′ + c calculated by the offset adding unit 222 is supplied to the subsequent adding unit 224 and the switch 226. Note that the output image of the offset adding unit 222 is hereinafter also referred to as “one filter output image”.

Similarly to the first configuration example, the second spatial filter 223 has pixel values s2 [1] and s2 of reference pixels in the input image (decoded image or decoded image after deblocking) according to the above-described equation (B2). From [2],..., S2 [N2], the pixel value s2 ′ of the target pixel in the output image is calculated. The pixel value s <b> 2 ′ calculated by the second spatial filter 223 is supplied to the subsequent adding unit 224. However, when the value of the second region characteristic use flag relating to the unit region including the target pixel (the setting unit of the second region characteristic use flag) is “0”, the filter calculation shown in the above-described equation (B2) is executed. do not do.

As in the first configuration example, the adding unit 224 adds s2 ′ calculated by the second spatial filter 223 to the sum s1 ′ + c calculated by the offset adding unit 222. However, in this configuration example, the sum s 1 ′ + c + s 2 ′ calculated by the adder 224 is supplied to the switch 226 at the subsequent stage. Note that the output image of the adding unit 224 is hereinafter also referred to as “two-filter output image”.

Similar to the first configuration example, the region characteristic calculation unit 225 calculates the first region characteristic k1 and the second region characteristic k2 for each unit region (region characteristic calculation unit). For example, when the unit for calculating the first region characteristic k1 and the second region characteristic k2 is a macro block, the first filter coefficient group w1 (k1) and the second filter coefficient group w2 (k2) are provided for each macro book. Will be updated.

The switch 226 determines whether to output the one-filter output image or the two-filter output image to the value of the second region characteristic use flag enable_dir decoded by the second region characteristic use flag decoding unit 212. Switch accordingly. More precisely, the sum s1 ′ + c calculated by the offset adder 222 or the sum s1 ′ + c + s2 ′ calculated by the adder 225 is output as the pixel value of the target pixel in the output image. Is switched according to the value of the second region characteristic use flag enable_dir decoded by the second region characteristic use flag decoding unit 212. More specifically, when the value of the second region characteristic use flag enable_dir is “0”, the sum s1 ′ + c calculated by the offset adding unit 222 is output, and the second region characteristic use flag enable_dir is set. When the value is “1”, the sum s1 ′ + c + s2 ′ calculated by the adder 225 is output.

As described above, the adaptive filter 200 according to the present configuration example (1) the second region characteristic indicating whether or not to perform the filter operation using the second filter coefficient group w2 (k2) corresponding to the second region characteristic k2. A second region characteristic use flag decoding unit 212 that decodes the use flag is further provided. (2) The filter calculation unit 120 corresponds to the first region characteristic k1 according to the value of the second region characteristic use flag. The filter operation using only one filter coefficient group w1 (k1) is executed, or the first filter coefficient group w1 (k1) corresponding to the first area characteristic k1 and the second filter corresponding to the second area characteristic k2 It is configured to switch whether to perform a filter operation using both of the coefficient group w2 (k2).

Therefore, when the adaptive filter 200 according to the present configuration example is used, only the first filter coefficient group w1 (k1) is applied to a unit region that does not need to be subjected to filter operation using the second filter coefficient group w2 (k2). The filter operation using is executed. Therefore, if the adaptive filter 200 according to this configuration example is used, it is possible to further reduce the calculation cost required for the filter operation on the unit region that does not need to be subjected to the filter operation using the second filter coefficient group w2 (k2).

Note that when the adaptive filter 200 (FIG. 40) according to the present configuration example is installed in the decoding apparatus, the encoding apparatus may be equipped with the adaptive filter 200 'shown in FIG. As shown in FIG. 41, in the adaptive filter 200 ′ on the encoding device side, instead of the second region characteristic use flag decoding unit 212 in the adaptive filter 200 on the decoding device side, a second region characteristic use flag determining unit 212a and A second region characteristic use flag encoding unit 212b is added.

In the adaptive filter 200 ′ on the encoding device side, (1) the filter coefficient table creation unit 211a creates the first filter coefficient table W1 and the second filter coefficient table W2, and (2) the filter coefficient table coding unit 211b. Encodes the first filter coefficient table W1 and the second filter coefficient table W2 created by the filter coefficient table creation unit 211aa. Here, the filter coefficient table creation unit 211a uses the least square method to minimize the square error with respect to the original image (encoding target image) of (a) 1 filter output image (output image of the offset addition unit 222). A filter coefficient table pair (W1, W2) comprising a first filter coefficient table W1 and (b) a second filter coefficient table W2 that minimizes a square error with respect to the original image of the two-filter output image (the output image of the adder 224) ) And create. The filter coefficient table encoding unit 211b generates the first filter coefficient generated by the filter coefficient table generating unit 211a when the least square error with respect to the original image is smaller in the 1 filter output image than in the 2 filter output image. When only the table W1 is encoded and the least square error with respect to the original image is smaller in the two-filter output image than in the one-filter output image, the first filter coefficient table W1 and the first filter coefficient table W1 created by the filter coefficient table creation unit 211a The 2 filter coefficient table W2 is encoded.

Further, in the adaptive filter 200 ′ on the encoding device side, (1) the second region characteristic use flag determination unit 212a performs the minimum square error with respect to the original image of the one filter output image and the minimum with respect to the original image of the two filter output image. After comparing with the square error, the value of the second region characteristic use flag is determined according to the comparison result. (2) The second region characteristic use flag encoding unit 212b is a second region characteristic use flag determining unit. The value of the second region characteristic use flag determined by 212a is encoded. More specifically, when the least square error with respect to the original image is smaller in the 1-filter output image than in the 2-filter output image, “0” is encoded as the second region characteristic use flag enable_dir, and the 2-filter output When the least square error with respect to the original image is smaller in the image than in the one-filter output image, “1” is encoded as the second region characteristic use flag enable_dir.

For example, when six sets of filter coefficient groups w1 (k1) are included in the first filter coefficient table W1 and four sets of second filter coefficient groups w2 (k2) are included in the second filter coefficient table W2, The necessary number of executions of the least squares method is six times (when the second filter coefficient group w2 (k2) is determined depending on the first filter coefficient group w1 (k1)) or four times (first filter coefficient group). w1 (k1) is determined depending on the second filter coefficient group w2 (k2)). That is, according to this configuration example, when the filter coefficient group w (k1, k2) determined depending on both the first region characteristic k1 and the second region characteristic k2 is used as in the conventional case, the minimum two times of 24 times are used. Where multiplication is required, the number of times can be significantly reduced.

[Configuration example 3]
A third configuration example of the adaptive filter according to the present embodiment will be described with reference to FIG.

42 is an adaptive filter suitable for mounting in a decoding device, and includes a filter setting unit 210 and a filter calculation unit 220.

The filter setting unit 210 is a means for setting filter characteristics, and includes a filter coefficient decoding unit 211 and an expanded filter coefficient setting unit 213.

The filter coefficient decoding unit 211 decodes the first filter coefficient table W1, the offset c, and the second filter coefficient table W2 from the encoded data, as in the second configuration example. The first filter coefficient table W1, the offset c, and the second filter coefficient table W2 decoded by the filter coefficient decoding unit 211 are supplied to the enlarged filter coefficient calculation unit 213.

The enlarged filter coefficient setting unit 213 creates an enlarged filter coefficient table W based on the first filter coefficient table W1, the offset c, and the second filter coefficient table W2. Here, the enlarged filter coefficient table W includes a region characteristic pair (k1, k2) and a filter coefficient group w (k1, k2) = (w (k1, k2) [1], w (k1, k2) [2], ..., w (k1, k2) [N], c) are recorded in association with each other. The region characteristic pair (k1, k2) is a pair of the first region characteristic k1 and the second region characteristic k2, and the filter coefficient group w (k1, k2) is a filter coefficient of the expanded linear filter 227 described later. It is used as a group. The expanded filter coefficient setting unit 213 includes each filter coefficient group w (k1, k2) included in the expanded filter coefficient table W so that the expanded linear filter 227 becomes a filter equivalent to the filter operation unit 220 according to the first configuration example. ) Is set.

The filter calculation unit 220 includes an enlarged linear filter 227 and a region characteristic calculation unit 225.

The magnifying linear filter 227 performs pixel values s [1], s [2],..., S [N] of reference pixels in the input image (decoded image or decoded image after deblocking) according to the equation (B3). ], The pixel value s ′ of the target pixel in the output image is calculated.

s ′ = w (k1, k2) [1] s [1] + w (k1, k2) [2] s [2] +.
+ W (k1, k2) [N] s [N] + c (B3)
Here, w (k1, k2) [1], w (k1, k2) [2],..., W (k1, k2) [N] are target pixels calculated by the region characteristic calculation unit 225. Is an element of the enlarged filter coefficient group w (k1, k2) corresponding to the first region characteristic k1 and the second region characteristic k2 of the unit region including. More specifically, in the enlarged filter coefficient table W created by the enlarged filter coefficient setting unit 213, an area characteristic pair (k1) composed of the first area characteristic k1 and the second area characteristic k2 calculated by the area characteristic calculation unit 225. , K2) are elements of the enlarged filter coefficient group w (k1, k2).

The area characteristic calculation unit 225 calculates the first area characteristic k1 and the second area characteristic k2 for each unit area, as in the first configuration example. For example, when the unit for calculating the first region characteristic k1 and the second region characteristic k2 is a macro block, the enlarged filter coefficient group w (k1, k2) is set for each macro book.

Next, the reference region in the enlarged linear filter 227 will be described with reference to FIGS.

FIG. 43 is an enlarged view equivalent to the filter operation unit 220 including the first spatial filter 221 in which the reference area is set as shown in FIG. 35 and the second spatial filter 223 in which the reference area is set as shown in FIG. FIG. 10 is a diagram illustrating a setting example of a reference region of the linear filter 227. (A) shows a reference area when k2 = 0, (b) shows a reference area when k2 = 1, (c) shows a reference area when k2 = 2, (d ) Indicates a reference area when k2 = 3. In the rectangular area composed of 5 × 5 pixels shown in the figure, the center pixel corresponds to the target pixel, and the black pixel corresponds to the reference pixel. In addition, white characters written on black pixels indicate filter coefficients to be multiplied by the pixel values of the pixels. Although these filter coefficients are determined depending on the first region characteristic k1 and the second region characteristic k2, the notation of “(k1, k2)” is omitted in FIG.

44 is an enlargement equivalent to the filter operation unit 220 including the first spatial filter 221 in which the reference region is set as shown in FIG. 37 and the second spatial filter 223 in which the reference region is set as shown in FIG. FIG. 10 is a diagram illustrating a setting example of a reference region of the linear filter 227. (A) shows a reference area when k2 = 0, (b) shows a reference area when k2 = 1, (c) shows a reference area when k2 = 2, (d ) Indicates a reference area when k2 = 3. In the rectangular area composed of 7 × 5 pixels shown in the figure, the center pixel corresponds to the target pixel, and the black pixel corresponds to the reference pixel. In addition, white characters written on black pixels indicate filter coefficients to be multiplied by the pixel values of the pixels. Although these filter coefficients are determined depending on the first region characteristic k1 and the second region characteristic k2, the notation of “(k1, k2)” is omitted in FIG.

As described above, similarly to the adaptive filter 200 according to the first configuration example, the adaptive filter 200 according to the configuration example also includes (1) the first filter coefficient group w1 (k1) that depends on the first region characteristic k1, At least one first filter coefficient table W1 recorded in association with the corresponding first area characteristic k1 and at least one second filter coefficient group w2 (k2) depending on the second area characteristic k2 correspond to each other. A filter coefficient decoding unit 211 that decodes the second filter coefficient table W2 recorded in association with the area characteristic k2, and (2) an area characteristic calculation unit 225 that calculates the first area characteristic k1 and the second area characteristic k2 of each unit area. And (3) a filter operation using each pixel belonging to each unit area as a target pixel, and a plurality of first filter coefficient groups w (k) recorded in the first filter coefficient table W1. ), The filter coefficient group w1 (k1) corresponding to the first area characteristic k1 of the unit area calculated by the area characteristic calculation unit 225, and a plurality of second filter coefficients recorded in the second filter coefficient table W2. An extended linear filter 227 that executes a filter operation using the filter coefficient group w2 (k2) corresponding to the second region characteristic k2 of the unit region calculated by the region characteristic calculation unit 225 in the group w (k2). I have.

Note that when the adaptive filter 200 according to this configuration example is mounted on the decoding device, the encoding device may be mounted with an adaptive filter 200 ′ shown in FIG.
<< Embodiment 4 >>
A fourth embodiment of the present invention will be described with reference to FIGS. 45 to 46. FIG.

A configuration example of the adaptive filter according to the present embodiment will be described with reference to FIG.

45 is an adaptive filter suitable for mounting on a decoding device, and includes a filter setting unit 310 and a filter calculation unit 320.

The filter setting unit 310 is a means for setting filter characteristics, and includes a spatial filter coefficient decoding unit 311 and an offset table decoding unit 312.

The spatial filter coefficient decoding unit 311 decodes the spatial filter coefficient group w = (w [1], w [2],..., W [N]) from the encoded data. The spatial filter coefficient group w decoded by the spatial filter coefficient decoding unit 311 is supplied to a spatial filter 321 described later.

The offset table decoding unit 312 decodes the offset table C from the encoded data. Here, the offset table C refers to a table in which at least one offset c (k ′) depending on the region characteristic k ′ is recorded in association with the corresponding region characteristic k ′. The offset table C decoded by the offset table decoding unit 312 is supplied to an offset addition unit 324 described later.

The filter calculation unit 320 is a means for executing a filter calculation, and includes a spatial filter 321, a region characteristic calculation unit 322, an offset determination unit 323, and an offset addition unit 324.

The spatial filter 321 has pixel values s [1], s [2],..., S [N] of reference pixels in the input image (decoded image or decoded image after deblocking) according to the equation (C1). From this, the pixel value s ′ of the target pixel in the output image is calculated.

s ′ = w [1] s [1] + w [2] s [2] +... + w [N] s [N]
... (C1)
Here, w [1], w [2],..., W [N] are elements of the spatial filter coefficient group w decoded by the spatial filter coefficient decoding unit 321. The calculated pixel value s ′ is supplied to the subsequent offset addition unit 324.

The region characteristic calculation unit 322 calculates a region characteristic k ′ of each unit region in the input image (decoded image or decoded image after deblocking). As the region characteristic k ′ in the present configuration example, for example, a luminance magnitude (band offset), an edge type (edge offset), or the like can be used. When the magnitude of luminance is used as the region characteristic k ′, for example, k ′ = Y / 16 (a value obtained by quantizing the luminance value Y of the target pixel) may be set. The region characteristic k ′ calculated by the region characteristic calculation unit 322 is supplied to the offset determination unit 323.

The offset determination unit 323 determines the value of the offset c used for the filter calculation for the pixels belonging to each unit area. More specifically, in the offset table C decoded by the offset table decoding unit 312, the offset c (k ′) associated with the region characteristic k ′ of the unit region calculated by the region characteristic calculation unit 322 is specified. . The offset c (k ′) determined by the offset determination unit 323 is supplied to the offset addition unit 324.

The offset addition unit 324 adds the offset c (k ′) determined by the offset determination unit 323 to the pixel value s ′ calculated by the spatial filter 321. The sum s ′ + c (k ′) calculated by the offset adding unit 324 is output to the outside of the adaptive filter 300.

An arbitrary area obtained by dividing the frame can be used as a unit for calculating the area characteristic k ′. For example, a slice, a macro block, a block, an LCU, a CT, a CU, a PT, a PU, a TT, a TU, or an M × N block can be used as a unit for calculating the region characteristic k ′. For example, when the calculation unit of the region characteristic k ′ is a macroblock, an offset c (k ′) corresponding to the region characteristic k ′ of the macroblock is used for the filter calculation for pixels belonging to each macroblock. be able to.

As described above, the adaptive filter 300 according to the present configuration example performs the filter operation on each pixel belonging to each unit region by using the offset c (k ′) depending on the region characteristic k ′ of the unit region and the unit. A filter operation unit 320 is provided that executes using a filter coefficient group w that does not depend on the region characteristic k ′ of the region. Therefore, it is possible to achieve higher encoding efficiency than a conventional adaptive filter that performs a filter operation using a certain offset c.

Further, in the case where the filter operation is performed using the constant spatial filter coefficient group w as in the adaptive filter 300 according to this configuration example, the spatial filter coefficient w (k ′) corresponding to the region characteristic k ′ is set. The calculation cost on both the encoding device side and the decoding device side can be reduced and the code amount of the encoded data can be reduced as compared with the case where the filter operation is executed by using. That is, higher encoding efficiency can be realized at a lower calculation cost than the conventional adaptive filter.

In this configuration, the filter operation is performed using a certain spatial filter coefficient group w. However, the present invention is not limited to this. That is, the filter coefficient group w only needs to be independent of the area characteristic k ′, and the filter coefficient group w (k ″) corresponding to the area characteristic k ″ different from the area characteristic k ′ is used for filtering. You may employ | adopt the structure which performs a calculation. That is, you may perform the filter calculation according to (C2) Formula.

s ′ = w (k ″) [1] s [1] + w (k ″) [2] s [2] +.
+ W (k ″) [N] s [N] + c (k ′) (C2)
Further, a configuration may be adopted in which a filter operation is performed using an offset c (k ′, k ″) corresponding to these two region characteristics k ′, k ″. That is, you may perform the filter calculation according to (C3) Formula.

s ′ = w (k ″) [1] s [1] + w (k ″) [2] s [2] +.
+ W (k ″) [N] s [N] + c (k ′, k ″) (C3)
Further, a configuration may be adopted in which a filter operation is performed using a spatial filter coefficient group w (k ′, k ″) corresponding to these two region characteristics k ′, k ″. That is, you may perform the filter calculation according to (C4) Formula.

s ′ = w (k ′, k ″) [1] s [1] + w (k ′, k ″) [2] s [2] +.
+ W (k ′, k ″) [N] s [N] + c (k ′) (C4)
In these modifications, when the magnitude of luminance is used as the first region characteristic k ′, for example, the activity can be used as the second region characteristic k ″. When used as the first region characteristic k ′, the magnitude of the luminance can be used as the second region characteristic k ″.

In this configuration example, the amount calculated from the pixel values of the pixels existing in and / or around the unit region is used as the region characteristic k ′ of each unit region. However, the present invention is not limited to this. is not. In other words, any amount can be used as the region characteristic k ′ of the unit region as long as it is an amount indicating the characteristic of the unit region (in other words, an amount that can be used for classification of the unit region). For example, an encoding parameter relating to a unit region or an encoding parameter relating to a unit including the unit region (for example, a macro block including a block) can be used as the region characteristic k ′ of the unit region. The intra prediction mode and the inter prediction mode are examples of encoding parameters that can be used as the region characteristic k ′. Further, as will be described later, the type of image to which the unit area belongs may be used as the area characteristic k ′ of the unit area. The same can be said for the region characteristic k ″ described above.

Further, a plurality of different offsets c (k) corresponding to each of a plurality of different values k ′ = k ′ (0), k ′ (1),..., K ′ (M) of the region characteristic k ′. '(0)), c (k' (0)),..., C (k '(M)), when encoding each offset c (k' (t)) (or its Instead of encoding the quantized value as it is, a difference value (or a quantized value thereof) from the encoded offset c (k ′ (t−1)) may be encoded.

For example, when encoding the difference value Δ (k ′ (t)) = c (k ′ (t−1)) − c (k ′ (t)) on the encoding device side, the offset table decoding unit 312 After decoding the difference value Δ (k ′ (t)), the offset value c (k ′ (t)) is obtained according to (C5) using the decoded offset value c (k ′ (t−1)).

c (k ′ (t)) = c (k ′ (t−1)) − Δ (k ′ (t)) (C5)
Also, when encoding the difference value Δ ′ (k ′ (t)) = c (k ′ (t)) − c (k ′ (t−1)) on the encoding device side, the offset table decoding unit 312 , After decoding the difference value Δ ′ (k ′ (t)), the offset value c (k ′ (t)) is calculated according to (C6) using the decoded offset value c (k ′ (t−1)). obtain.

c (k ′ (t)) = c (k ′ (t−1)) + Δ (k ′ (t)) (C6)
Of the plurality of offsets c (k ′ (0)), c (k ′ (0)),..., C (k ′ (M)), one offset serving as a starting point of encoding / decoding, For example, c (k ′ (0)) is encoded / decoded regardless of other offsets.

Further, in this configuration example, the configuration for executing the filter operation using the offset c (k ′) corresponding to the region characteristic k ′ has been described, but the configuration using the offset c corresponding to the target to be subjected to the filter processing may also be used. Conceivable. Specifically, an offset c having different values is used when a filter operation is performed on the color difference signal Cb and when a filter operation is performed on the color difference signal Cr.

The adaptive filter is based on the principle that the spatial correlation between pixels existing in the original picture is reproduced by filtering. Since the color difference component Cb and the color difference component Cr have edges in the same direction at substantially the same location, the inter-pixel correlation between a certain pixel and its surrounding pixels is often similar to each other. The common use of the spatial filter w (k ″) determined in relation to the feature of the inter-pixel correlation between the color difference component Cb and the color difference component Cr particularly increases the code amount for the filter coefficient group of the adaptive filter. This is particularly appropriate at a relatively low bit rate, which is difficult to apply, however, because the offset between the decoded image and the original image cannot be said to be close between the color difference component Cb and the color difference component Cr. It is appropriate to use the offset c having different values for the color difference component Cb and the color difference component Cr.

Furthermore, a spatial filter coefficient group w (k ″) corresponding to various region characteristics k ″ may be used. Similarly to the offset c, when performing a filter operation on the color difference signal Cb, A spatial filter coefficient group w having different values may be used when the filter operation is executed. The chrominance component Cb and the chrominance component Cr have edges in the same direction at substantially the same place, but their strengths are not necessarily close. Therefore, when encoding at a relatively high bit rate, the chrominance component Cb and the chrominance component It is appropriate to use a spatial filter coefficient group w different from Cr.

When the adaptive filter 300 shown in FIG. 45 is installed in the decoding device, the encoding device may be installed with the adaptive filter 300 ′ shown in FIG. 46. As shown in FIG. 46, in the adaptive filter 300 ′ on the encoding device side, the spatial filter coefficient decoding unit 311 in the adaptive filter 300 on the decoding device side is replaced with an adaptive filter coefficient determination unit 3111 and a spatial filter coefficient encoding unit 311b. The offset table decoding unit 312 in the adaptive filter 300 on the decoding device side is replaced with an offset table encoding unit 312a.

In the adaptive filter 300 ′ on the encoding device side, (1) the adaptive filter coefficient determination unit 3111 uses the least square method to calculate a square error with respect to the original image (encoding target image) of the output image of the filter operation unit 320. The spatial filter coefficient group w and the offset table C to be minimized are determined. The objective function to be minimized by the least square method is a function having the order N of the spatial filter coefficient group and the order of the offset table C (for example, N2) as variables, and becomes 0 by partially differentiating the objective function with respect to the N + N2 variable. It can be calculated by solving simultaneous equations. Here, when a plurality of spatial filter coefficient groups w are used according to various area characteristics, the square of the output image with respect to the original image (encoding target image) for each of the various area characteristics for switching the spatial filter coefficient group w. The spatial filter coefficient group w and the offset table C that minimize the error are determined. That is, when the spatial filter coefficient group w (k ′, k ″) is determined according to the two region characteristics k ′ and k ″, the least square method is calculated for each region characteristic pair (k ′, k ″). If the spatial filter coefficient group w (k ″) is determined according to one characteristic k ″, the least square method is calculated for each region characteristic k ″. When the filter coefficient group w is determined to be one, one least square method is calculated. Also, (2) the spatial filter coefficient encoding unit 311b encodes the spatial filter coefficient group w determined by the adaptive filter coefficient determination unit 311a, and (3) the offset table encoding unit 312a performs the adaptive filter coefficient determination unit 311a. The offset table C created by is encoded.

[Encoded data / decoding device / encoding device]
A video encoding device (encoding device) and a video decoding device (decoding device) on which the adaptive filters according to Embodiments 2 to 4 described above are mounted will be described. A structure of encoded data generated by the moving image encoding device and decoded by the moving image decoding device will also be described.

<Encoded data>
The structure of encoded data # 1 generated by the moving image encoding device 2 and decoded by the moving image decoding device 1 will be described with reference to FIG. 47 and FIG.

The encoded data # 1 generated by the moving image encoding device 2 and decoded by the moving image decoding device 1 includes a sequence layer, a GOP (Group Of Pictures) layer, a picture layer, a slice layer, and a macroblock layer. Has a hierarchical structure.

FIG. 47 shows the structure of encoded data for the layers below the picture layer. 47A to 47F show the structures of the picture layer P, slice layer S, macroblock layer MB, macroblock header MBH, inter prediction partition prediction parameter PP, and intra prediction partition prediction parameter PP, respectively. FIG.

The picture layer P is a set of data referred to by the video decoding device 1 in order to decode the target picture. As shown in FIG. 47A, the picture layer P includes a picture header PH and slice layers S ₁ to S _Ns (Ns is the total number of slice layers included in the picture layer P).

The picture header PH includes a coding parameter group referred to by the video decoding device 1 in order to determine a decoding method of the target picture. For example, the encoding mode information (entoropy_coding_mode_flag) indicating the variable length encoding mode used in encoding by the moving image encoding device 2 is an example of an encoding parameter included in the picture header PH.

Each slice layer S included in the picture layer P is a set of data referred to by the video decoding device 1 in order to decode the target slice. As shown in FIG. 47 (b), the slice layer S includes a slice header SH and macroblock layers MB ₁ to MB _Nm (Nm is the total number of macroblocks included in the slice S).

The slice header SH includes a coding parameter group that the moving image decoding apparatus 1 refers to in order to determine a decoding method of the target slice. (1) Slice type designation information (slice_type) for designating a slice type, (2) POC designation information (pic_order_cnt_lbs, delta_pic_order_cnt, etc.) for designating the display order (POC: Picture Order Count) of a picture including the target slice, and ( 3) The weighting factor designation information (pred_weight_table) that designates the weighting factor used when the moving image coding apparatus 2 performs the coding is an example of a coding parameter included in the slice header SH.

As slice types that can be specified by the slice type specification information, (1) I slice using only intra prediction at the time of encoding, and (2) P using unidirectional prediction or intra prediction at the time of encoding. Slice, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding.

Also, the slice header SH includes a filter parameter FP that is referred to by the adaptive filter included in the video decoding device 1.

Each macroblock layer MB included in the slice layer S is a set of data referred to by the video decoding device 1 in order to decode the target macroblock. As shown in FIG. 47 (c), the macroblock layer MB includes a skip flag SKIP, a macroblock header MBH, prediction parameters PP ₁ to PP _NP , and quantized prediction errors QD ₁ to QD _Nb . Here, Np represents the number of partitions (prediction units) included in the target macroblock, and Nb represents the number of blocks (conversion units) included in the target macroblock. When the value of the skip flag SKIP is 1, that is, when the target macroblock is a skip block, the macroblock header MBH, the prediction parameters PP ₁ to PP _NP , and the quantized prediction errors QD ₁ to QD in that macroblock layer _Nb is omitted.

The macroblock header MBH includes an encoding parameter referred to by the video decoding device 1 in order to determine a decoding method of the target macroblock. Specifically, as shown in FIG. 47 (d), macroblock type designation information MBT (mb_type) for designating the macroblock type of the target macroblock, CBP (coded_block_pattern) for designating the encoded block pattern, and quantization step Quantization parameter difference Δqp (mb_qp_delta) that specifies the size of. The macroblock type designation information MBT includes prediction unit designation information PT and conversion unit designation information TT.

The prediction unit designation information PT includes a partition pattern (that is, the size of each partition included in the target macroblock and a position in the target macroblock) into the partition (prediction unit) of the target macroblock, and video coding. The prediction method (L0 unidirectional prediction, L1 unidirectional prediction, bidirectional prediction, intra prediction, etc.) used when the apparatus 2 generates a predicted image in each partition is designated. The partition size is, for example, 16 × 16 pixels, 8 × 8 pixels, 4 × 4 pixels, 16 × 8 pixels, 8 × 16 pixels, 8 × 4 pixels, 4 × 8 pixels, 8 × 1 pixels, and 1 X 8 pixels can be selected. More generally, each partition has M × N pixels (M is an integer equal to or smaller than the number of pixels in the horizontal direction of the target macroblock, and N is an integer equal to or smaller than the number of pixels in the vertical direction of the target macroblock). Can be an area.

On the other hand, the conversion unit designation information TT designates the division pattern (that is, the size of each block included in the target macroblock and the position in the target macroblock) into the block (conversion unit) of the target macroblock. Each block is 16 × 16 pixels, 8 × 8 pixels, 4 × 4 pixels, 16 × 8 pixels, 8 × 16 pixels, 8 × 4 pixels, 4 × 8 pixels, 8 × 1 pixels, and 1 × 8 pixels. Can be selected from. More generally, each block has M ′ × N ′ pixels (M ′ is an integer less than or equal to the number of pixels on one side of the target macroblock, and N ′ is less than or equal to the number of pixels on the other side of the target macroblock. (Which is an integer).

In the above description, the partition when the macro block is a square area of 16 × 16 pixels is exemplified, but the present invention is not limited to this. For example, for a 64 × 64 pixel macroblock, a 64 × 64 pixel or 32 × 32 pixel square area, or 64 × 32 pixel, 32 × 64 pixel, 32 × 16 pixel, or 16 × 32 pixel The rectangular area is also allowed as a partition or block.

The quantization parameter difference Δqp is a difference qp−qp ′ between the quantization parameter qp in the target macroblock and the quantization parameter qp ′ in the macroblock encoded immediately before the macroblock.

Each quantized prediction residual QD _n included in the macroblock layer MB is encoded data generated by the moving image encoding apparatus 2 performing the following processes 1 to 3 on the target block. Process 1: DCT transform (Discrete Cosine Transform) is performed on the prediction residual obtained by subtracting the prediction image from the encoding target image. Process 2: The DCT coefficient obtained in Process 1 is quantized. Process 3: The DCT coefficient quantized in Process 2 is variable length encoded. The quantization parameter qp described above represents the size of the quantization step QP used when the moving image encoding apparatus 2 quantizes the DCT coefficient (QP = 2 ^{pq / 6} ).

Among the prediction parameters PP included in the macroblock layer MB, the prediction parameter PP related to the inter prediction partition in which the prediction image is generated by the inter prediction includes a reference image index RI, an estimated motion vector, as shown in FIG. The index PMVI and the motion vector residual MVD are included.

The reference image index RI described above specifies the locally decoded image (reference image) that has been encoded / decoded selected in the process 4, and the estimated motion vector index PMVI described above is selected in the process 5. Specifies the estimation method. The estimation methods that can be selected in the processing 5 include: (1) a locally decoded image being encoded / decoded (more precisely, a region that has already been decoded in a locally decoded image being encoded / decoded). In an image obtained by performing block processing and adaptive filter processing), a median of a motion vector allocated to a partition adjacent to the target partition (hereinafter also referred to as “adjacent partition”) is used as an estimated motion vector pmv. (2) In a locally decoded image that has been encoded / decoded, a motion vector assigned to a partition (often referred to as a “collocated partition”) occupying the same position as the target partition is used as an estimated motion vector pmv, etc. Is mentioned.

Note that, as shown in FIG. 47 (e), the prediction parameter PP related to the partition for which unidirectional prediction is performed includes one each of the reference image index RI, the estimated motion vector index PMVI, and the motion vector residual MVD. However, the prediction parameters PP for a partition that performs bi-directional prediction (weighted prediction) include two reference image indexes RI1 and RI2, two estimated motion vector indexes PMVI1 and PMVI2, and two motion vector residuals MVD1. And MVD2.

Among the prediction parameters PP included in the macroblock layer MB, the prediction parameter PP related to the intra prediction partition for which a prediction image is generated by intra prediction includes a prediction index PI as shown in FIG. 47 (f). Here, the prediction index PI is an index for designating an intra prediction method (prediction mode) for the target partition. Note that since the types of prediction modes that can be selected for the target partition differ depending on the size of the target partition, the range of values that the prediction index can take also differs depending on the size of the target partition.

Note that the data structure of the encoded data # 1 is not limited to that shown in FIG. For example, a data structure as shown in FIG. 50 obtained by changing a part of a layer structure below a picture in a moving image may be adopted.

FIG. 50 is a diagram showing a modification of the data structure of the encoded data # 1. In FIG. 50, (a) is a picture layer PCT corresponding to a picture, (b) is a slice layer S corresponding to a slice obtained by dividing the picture, and (c) is a tree obtained by dividing the slice. Tree block layers TBLK and (d) corresponding to blocks (Tree blocks) indicate CU layer CUs corresponding to CUs (Coding Units) obtained by dividing the tree blocks.

The picture layer PICT includes various types of information that the video decoding device 1 refers to in order to decode a picture (also referred to as a target picture). Specifically, as shown in FIG. 50A, a picture header PH and slice layers S ₁ to S _NS (NS is the total number of slices included in the target picture) are included.

In the following description, when it is not necessary to distinguish each of the slice layers S ₁ to S _NS , the reference numerals are omitted and the slice layers S are described. The same applies to codes of other layers included in the encoded data # 1.

When entropy_coding_mode_flag is 0, the target picture is encoded by CAVLC (Context-based “Adaptive” Variable “Length” Coding). When entropy_coding_mode_flag is 1, the target picture is encoded by CABAC (Context-based Adaptive Binary Arithmetic Coding).

Note that the picture header PH is also referred to as a picture parameter set (PPS).

The slice layer S includes various types of information that the video decoding device 1 refers to in order to decode a slice (also referred to as a target slice). Specifically, as shown in FIG. 50B, a slice header SH and tree block layers TBLK ₁ to TBLK _NC (NC is the total number of tree blocks included in the target slice) are included.

The slice header SH includes a coding parameter group that the moving image decoding apparatus 1 refers to in order to determine a decoding method of the target slice. The slice type designation information (slice_type) that designates the slice type of the target slice is an example of an encoding parameter included in the slice header SH.

Further, when the filter coefficient group of the adaptive filter is switched for each slice, the filter parameter FP that defines the filter coefficient group of the adaptive filter that acts on the target slice is also included in the slice header SH.

The tree block layer TBLK includes various types of information that the video decoding device 1 refers to in order to decode a tree block (also referred to as a target tree block). Specifically, as shown in FIG. 50C, a tree block header TBLKH and CU information CUI ₁ to CUI _NL (NL is the total number of CUs included in the target tree block) are included.

Here, the CU is obtained by recursively dividing the tree block into quadtrees. Recursive quadtree partitioning refers to a process in which a tree block is divided into four equal parts and then a step of dividing the partition obtained by the previous partitioning into four equal parts is repeated. At this time, only a part of the partition obtained by the previous division is allowed to be divided into four equal parts. That is, CUs having different sizes can coexist in the tree block.

The tree structure obtained by this recursive quadtree partitioning is hereinafter referred to as a coding tree. The tree block corresponds to the root of the coding tree, and the CU corresponds to the leaf of the coding tree. Note that a leaf of the coding tree is a node corresponding to the CU, and thus may be referred to as a coding node (coding node).

Also, the size that each CU can take depends on the size of the tree block and the maximum hierarchical depth (maximummaximhierarchical depth) included in the sequence parameter set SPS of the encoded data # 1. For example, when the size of the tree block is 64 × 64 pixels and the maximum hierarchical depth is 3, there are three types of sizes, that is, CUs of 64 × 64 pixels, 32 × 32 pixels, and 16 × 16 pixels. Can exist.

The tree block header TBLKH includes an encoding parameter referred to by the video decoding device 1 in order to determine a decoding method of the target tree block. The tree block division information SP_TBLK that specifies the division pattern of the target tree block into each CU, the quantization parameter difference Δqp (qp_delta) that specifies the size of the quantization step, and the like are encoded parameters included in the tree block header TBLKH. It is an example.

The tree block division information SP_TBLK is information indicating a division pattern when the target tree block is divided into CUs, in other words, information representing a coding tree. The tree block division information SP_TBLK can be realized, for example, by a set of flags (split_coding_unit_flag) indicating whether or not each node is divided into four in the coding tree.

Further, the quantization parameter difference Δqp is a difference qp−qp ′ between the quantization parameter qp in the target tree block and the quantization parameter qp ′ in the tree block encoded immediately before the target tree block.

Before describing the CU information CUI, the structure of the CU will be described.

• At least one prediction block is obtained by dividing the CU. In other words, the CU is covered by at least one prediction block that does not overlap each other. A tree having a node corresponding to a CU as a root and a node corresponding to a prediction block as a leaf is called a prediction tree (PT). A prediction block is an execution unit of prediction processing, and is also referred to as PU (prediction unit).

There are roughly two types of division in the prediction tree: intra prediction and inter prediction. In the case of intra prediction, there are 2N × 2N (the same size as the encoding node) and N × N division methods. In the case of inter prediction, there are 2N × 2N (the same size as the encoding node), 2N × N, N × 2N, N × N, and the like.

Moreover, at least one conversion block is obtained by dividing the CU. In other words, the CU is covered by at least one transform block that does not overlap each other. A tree having a node corresponding to a CU as a root and a node corresponding to a prediction block as a leaf is referred to as a transform tree (TT). The transformation block is another meaning of execution of DCT transformation, and is also called TU (transform unit).

The division in the transformation tree includes the one in which an area having the same size as the encoding node is assigned as the transformation block, and the one in the recursive quadtree division as in the above-described division of the tree block.

The CU information CUI includes various types of information that the video decoding device 1 refers to in order to decode a CU (hereinafter also referred to as a target CU). Specifically, as shown in FIG. 50 (d), a skip flag SKIP, PT information PTI, and TT information TTI are included.

The skip flag SKIP is a flag indicating whether or not the skip mode is applied to the target CU. When the value of the skip flag SKIP is 1, that is, when the skip mode is applied to the target CU, the CU The PT information PTI and the TT information TTI in the information CUI are omitted. Note that the skip flag SKIP is omitted for the I slice.

PT information PTI is information regarding the prediction tree PT having the target CU as a root. Specifically, as shown in FIG. 50 (d), the prediction type information PType and the prediction information PInfo are included.

Prediction type information PType is information that specifies whether intra prediction or inter prediction is used to generate a predicted image in the target CU. In the prediction information PInfo, PT division information (not shown) for specifying a division pattern for dividing the target CU into PUs, and a prediction parameter referred to by the video decoding device 1 to generate a prediction image of each PU (Not shown) is included.

The prediction parameters of a PU for which a predicted image is generated by inter prediction include a reference image index, an estimated motion vector index, a motion vector residual, and the like. On the other hand, the prediction parameters related to the intra PU for which the prediction image is generated by the intra prediction include a prediction index for specifying the prediction mode.

The TT information TTI is information related to a conversion tree having the target CU as a root. Specifically, as shown in FIG. 50 (d), TT division information SP_TT and quantization prediction residuals QD ₁ to QD _NT (NT is the total number of blocks included in the target CU) are included.

TT division information SP_TT is information expressing a division pattern when dividing the target CU into TUs, in other words, information expressing a tree structure of a conversion tree having the target CU as a root. The TT division information SP_TT can be configured with, for example, a flag (split_transform_unit_flag) indicating whether or not to perform division of the target CU and information (trafoDepth) indicating the division depth. For example, when the size of the CU is 64 × 64, each TU obtained by the division can take a size from 32 × 32 pixels to 2 × 2 pixels.

The quantized prediction residuals QD ₁ to QD _NT are codes obtained by performing the following processing on each TU. (Process 1) DCT transform (Discrete Cosine Transform) is performed on the prediction residual obtained by subtracting the prediction image from the encoding target image. (Process 2) The transform coefficient obtained in Process 1 is quantized. (Process 3) The transform coefficient quantized in Process 2 is variable length encoded. The quantization parameter qp described above represents the size of the quantization step QP used when quantizing the transform coefficient (QP = 2qp / 6).

<Video decoding device>
With reference to FIG. 48, a description will be given of a moving picture decoding apparatus in which the

adaptive filter

100, 200, 300 according to Embodiments 2 to 4 described above is mounted. The moving image decoding apparatus 1001 includes, as a part thereof, H.264. H.264 / MPEG-4. It is a decoding device including AVC and its successor technology.

FIG. 48 is a block diagram showing a configuration of the moving picture decoding apparatus 1001. The video decoding device 1001 is a device for generating video # 1002 by decoding the encoded data # 1001, and as shown in FIG. 48, a variable length code decoding unit 1013, a motion vector restoration unit 1014, A buffer memory 1015, an inter prediction image generation unit 1016, an intra prediction image generation unit 1017, a prediction method determination unit 1018, an inverse quantization / inverse transformation unit 1019, an adder 1020, a deblocking filter 1041, and an adaptive filter 1042 are provided. Yes. The

adaptive filter

100, 200, 300 according to Embodiments 2 to 4 described above is used as the adaptive filter 1042.

The variable length code decoding unit 1013 decodes the prediction parameter PP related to each partition from the encoded data # 1001. That is, for the inter prediction partition, the reference image index RI, the estimated motion vector index PMVI, and the motion vector residual MVD are decoded from the encoded data # 1001 and supplied to the motion vector restoration unit 1014. On the other hand, with respect to the intra prediction partition, (1) size designation information for designating the size of the partition and (2) prediction index designation information for designating the prediction index are decoded from the encoded data # 1001, and this is decoded into the intra prediction image. It supplies to the production | generation part 1017. Further, the variable length code decoding unit 1013 decodes the macroblock type MBT from the encoded data, and supplies this to the prediction method determination unit 1018 (not shown). Furthermore, the variable length code decoding unit 1013 decodes the quantization prediction residual QD for each block and the quantization parameter difference Δqp for the macroblock including the block from the encoded data # 1001, and dequantizes and decodes them. This is supplied to the inverse transform unit 1019. Also, the variable length code decoding unit 1013 supplies the filter parameter FP included in the encoded data # 1001 to the adaptive filter 1042.

The motion vector restoration unit 1014 restores the motion vector mv related to each inter prediction partition from the motion vector residual MVD related to that partition and the restored motion vector mv ′ related to other partitions. Specifically, (1) the estimated motion vector pmv is derived from the restored motion vector mv ′ according to the estimation method specified by the estimated motion vector index PMVI, and (2) the derived estimated motion vector pmv and the motion vector remaining are derived. The motion vector mv is obtained by adding the difference MVD. It should be noted that the restored motion vector mv ′ related to other partitions can be read from the buffer memory 1015. The motion vector restoration unit 1014 supplies the restored motion vector mv to the inter predicted image generation unit 1017 together with the corresponding reference image index RI. For the inter prediction partition that performs bi-directional prediction (weighted prediction), the restored two motion vectors mv1 and mv2 are supplied to the inter prediction image generation unit 1017 together with the corresponding reference image indexes RI1 and RI2.

The inter prediction image generation unit 1016 generates a motion compensation image mc related to each inter prediction partition. Specifically, using the motion vector mv supplied from the motion vector restoration unit 1014, the motion compensated image mc from the filtered decoded image P_FL ′ designated by the reference image index RI also supplied from the motion vector restoration unit 1014. Is generated. Here, the filtered decoded image P_FL ′ is obtained by performing deblocking processing by the deblocking filter 1041 and adaptive filtering processing by the adaptive filter 1042 on the decoded image that has already been decoded. The inter prediction image generation unit 1016 is an image, and can read out the pixel value of each pixel constituting the filtered decoded image P_FL ′ from the buffer memory 1015. The motion compensated image mc generated by the inter predicted image generating unit 1016 is supplied to the prediction method determining unit 1018 as an inter predicted image Pred_Inter. For the inter prediction partition that performs bi-directional prediction (weighted prediction), (1) a motion compensated image mc1 is generated from the filtered decoded image P_FL1 ′ specified by the reference image index RI1 using the motion vector mv1. (2) A motion compensated image mc2 is generated from the filtered decoded image P_FL2 ′ specified by the reference image index RI2 using the motion vector mv2, and (3) weighting between the motion compensated image mc1 and the motion compensated image mc2 An inter predicted image Pred_Inter is generated by adding an offset value to the average.

The intra predicted image generation unit 1017 generates a predicted image Pred_Intra related to each intra prediction partition. Specifically, first, referring to the correspondence table recorded in its own memory, the prediction mode is specified based on the size designation information and the prediction index designation information supplied from the variable length code decoding unit 1013, The identified prediction mode is assigned to the target partition in, for example, raster scan order. Subsequently, a predicted image Pred_Intra is generated from the decoded image P according to the prediction method indicated by the prediction mode. The intra predicted image Pred_Intra generated by the intra predicted image generation unit 1017 is supplied to the prediction method determination unit 1018. In addition, since the specific production | generation process of the intra estimated image Pred_Intra by the intra estimated image generation part 1017 is mentioned later, description is abbreviate | omitted here.

The prediction method determination unit 1018 determines whether each partition is an inter prediction partition that should perform inter prediction or an intra prediction partition that should perform intra prediction, based on the macroblock type MBT. In the former case, the inter predicted image Pred_Inter generated by the inter predicted image generation unit 1016 is supplied to the adder 1020 as the predicted image Pred. In the latter case, the intra predicted image generation unit 1017 generates the inter predicted image Pred_Inter. The intra predicted image Pred_Intra that has been processed is supplied to the adder 1020 as the predicted image Pred.

The inverse quantization / inverse transform unit 1019 (1) inversely quantizes the quantized prediction residual QD, (2) performs inverse DCT (Discrete Cosine Transform) transformation on the DCT coefficient obtained by the inverse quantization, and (3) The prediction residual D obtained by the inverse DCT transform is supplied to the adder 1020. When the quantization prediction residual QD is inversely quantized, the inverse quantization / inverse transform unit 1019 derives a quantization step QP from the quantization parameter difference Δqp supplied from the variable length code decoding unit 1013. The quantization parameter qp can be derived by adding the quantization parameter difference Δqp to the quantization parameter qp ′ relating to the macroblock that has been inversely quantized / inversely DCT transformed immediately before, and the quantization step QP is performed from the quantization step qp to QP. = 2pq / 6. In addition, the generation of the prediction residual D by the inverse quantization / inverse transform unit 1019 is performed in units of blocks (transform units).

The adder 1020 generates a decoded image P by adding the prediction image Pred supplied from the prediction method determination unit 1018 and the prediction residual D supplied from the inverse quantization / inverse transformation unit 1019.

The deblocking filter 1041 determines the block boundary in the decoded image P when the difference between the pixel values of the pixels adjacent to each other via the block boundary in the decoded image P or the macroblock boundary is smaller than a predetermined threshold, or By performing deblocking processing on the macroblock boundary, an image near the block boundary or the macroblock boundary is smoothed. The image subjected to the deblocking process by the deblocking filter 1041 is output to the adaptive filter 42 as a deblocked decoded image P_DB.

The adaptive filter 1042 subjects the deblocked decoded image P_DB supplied from the deblocking filter 1041 to a filtering process using a filter coefficient obtained by decoding the filter parameter FP, so that the filtered decoded image P_FL Is generated. The image subjected to the filter processing by the adaptive filter 1042 is output to the outside as the filtered decoded image P_FL, and is associated with the POC designation information decoded from the encoded data by the variable length code decoding unit 1013 in the buffer memory 1015. Stored.

In addition, although the structure of the moving image decoding apparatus 1001 corresponding to the case where encoded data # 1001 has the data structure shown in FIG. 47 was demonstrated here, the structure of the moving image decoding apparatus 1001 is not limited to this. That is, the video decoding apparatus 1 can be made to correspond to the encoded data # 1001 having the data structure shown in FIG.

<Moving picture encoding apparatus>
With reference to FIG. 49, a description will be given of a moving picture encoding apparatus in which the adaptive folders 100 ′, 200 ′, and 300 ′ according to Embodiments 2 to 4 described above are mounted. The moving image encoding apparatus 1002 includes, as a part thereof, an H.264 format. H.264 / MPEG-4. It is a decoding device including AVC and its successor technology.

FIG. 49 is a block diagram showing a configuration of the moving picture encoding apparatus 1002. The moving image encoding apparatus 1002 is an apparatus that generates encoded data # 1001 by encoding moving image # 1010 (encoding target image). As illustrated in FIG. 49, as illustrated in FIG. Variable length code encoding unit 1022, inverse quantization / inverse transform unit 1023, buffer memory 1024, intra prediction image generation unit 1025, inter prediction image generation unit 1026, motion vector detection unit 1027, prediction scheme control unit 1028, motion vector redundancy A deletion unit 1029, an adder 1031, a subtractor 1032, a deblocking filter 1033, and an adaptive filter 1034. The adaptive folders 100 ′, 200 ′, and 300 ′ according to Embodiments 2 to 4 described above are used as the adaptive folder 1034.

The transform / quantization unit 1021 performs (1) DCT transform (Discrete Cosine Transform) for each block on the prediction residual D obtained by subtracting the predicted image Pred from the encoding target image, and (2) DCT coefficients obtained by the DCT transform. (3) The quantized prediction residual QD obtained by the quantization is supplied to the variable-length code encoding unit 22 and the inverse quantization / inverse transform unit 1023. The transform / quantization unit 1021 selects (1) a quantization step QP to be used for quantization for each macroblock, and (2) a quantization parameter difference Δqp indicating the size of the selected quantization step QP. Is supplied to the variable-length code encoding unit 1022, and (3) the selected quantization step QP is supplied to the inverse quantization / inverse transform unit 1023. Here, the quantization parameter difference Δqp is the quantization parameter qp related to the macroblock DCT transformed / quantized immediately before from the value of the quantization parameter qp (QP = 2pq / 6) related to the macroblock to be DCT transformed / quantized. The difference value obtained by subtracting the value of '.

The variable length code encoding unit 1022 includes (1) a quantization prediction residual QD and Δqp supplied from the transform / quantization unit 1021, and (2) a quantization parameter PP supplied from a prediction scheme control unit 1028 described later. (3) The encoded data # 1001 is generated by variable-length encoding the filter parameter FP supplied from the adaptive filter 1034 described later.

The inverse quantization / inverse transform unit 1023 (1) inversely quantizes the quantized prediction residual QD, (2) performs inverse DCT (Discrete Cosine Transform) transformation on the DCT coefficient obtained by the inverse quantization, and (3) The prediction residual D obtained by the inverse DCT transform is supplied to the adder 1031. When the quantization prediction residual QD is inversely quantized, the quantization step QP supplied from the transform / quantization unit 1021 is used. Note that the prediction residual D output from the inverse quantization / inverse transform unit 1023 is obtained by adding a quantization error to the prediction residual D input to the transform / quantization unit 1021. Common names are used for this purpose.

The intra predicted image generation unit 1025 generates a predicted image Pred_Intra related to each partition. Specifically, (1) a prediction mode used for intra prediction is selected for each partition, and (2) a prediction image Pred_Intra is generated from the decoded image P using the selected prediction mode. The intra predicted image generation unit 1025 supplies the generated intra predicted image Pred_Intra to the prediction method control unit 1028.

Also, the intra predicted image generation unit 1025 refers to the correspondence table stored in its own memory, and specifies the prediction index PI for each partition from the prediction mode selected for each partition and the size of each partition. The prediction index designation information indicating the prediction index PI for each partition is supplied to the prediction method control unit 28.

In addition, the production | generation of the intra estimated image by the intra estimated image production | generation part 1025 can be performed by the process similar to the process demonstrated in (the production | generation process of the intra estimated image by the intra estimated image generation part 1017).

The motion vector detection unit 1027 detects a motion vector mv related to each partition. Specifically, (1) the filtered decoded image P_FL ′ to be used as a reference image is selected, and (2) the target partition is searched by searching for the region that best approximates the target partition in the selected filtered decoded image P_FL ′. Detects a motion vector mv. Here, the filtered decoded image P_FL ′ is obtained by performing deblocking processing by the deblocking filter 1033 and adaptive filtering processing by the adaptive filter 1034 on the decoded image that has already been decoded. It is an image, and the motion vector detection unit 1027 can read out the pixel value of each pixel constituting the filtered decoded image P_FL ′ from the buffer memory 1024. The motion vector detection unit 1027 supplies the detected motion vector mv to the inter prediction image generation unit 1026 and the motion vector redundancy deletion unit 1029 together with the reference image index RI that specifies the filtered decoded image P_FL ′ used as the reference image. To do. Note that for a partition that performs bi-directional prediction (weighted prediction), two filtered decoded images P_FL1 ′ and P_FL2 ′ are selected as reference images, and each of the two filtered decoded images P_FL1 ′ and P_FL2 ′ is selected. Corresponding motion vectors mv1 and mv2 and reference image indexes RI1 and RI2 are supplied to the inter predicted image generation unit 1026 and the motion vector redundancy deletion unit 1029.

The inter prediction image generation unit 1026 generates a motion compensation image mc related to each inter prediction partition. Specifically, using the motion vector mv supplied from the motion vector detection unit 1027, the motion compensation image mc is obtained from the filtered decoded image P_FL ′ specified by the reference image index RI supplied from the motion vector detection unit 1027. Generate. Similar to the motion vector detection unit 1027, the inter predicted image generation unit 1026 can read out the pixel value of each pixel constituting the filtered decoded image P_FL ′ from the buffer memory 1024. The inter prediction image generation unit 1026 supplies the generated motion compensated image mc (inter prediction image Pred_Inter) to the prediction scheme control unit 1028 together with the reference image index RI supplied from the motion vector detection unit 1027. For the partition with bi-directional prediction (weighted prediction), (1) the motion compensated image mc1 is generated from the filtered decoded image P_FL1 ′ specified by the reference image index RI1 using the motion vector mv1, and (2 ) A motion compensated image mc2 is generated from the filtered reference image P_FL2 ′ specified by the reference image index RI2 using the motion vector mv2, and (3) an offset value is added to the weighted average of the motion compensated image mc1 and the motion compensated image mc2. Is added to generate the inter predicted image Pred_Inter.

The prediction scheme control unit 1028 compares the intra predicted image Pred_Intra and the inter predicted image Pred_Inter with the encoding target image, and selects whether to perform intra prediction or inter prediction. When intra prediction is selected, the prediction scheme control unit 1028 supplies the intra prediction image Pred_Intra as the prediction image Pred to the adder 1031 and the subtractor 1032 and also predicts the prediction index PI supplied from the intra prediction image generation unit 1025. The parameter PP is supplied to the variable length code encoding unit 1022. On the other hand, when the inter prediction is selected, the prediction scheme control unit 1028 supplies the inter prediction image Pred_Inter as the prediction image Pred to the adder 1031 and the subtracter 1032 and the reference image index supplied from the inter prediction image generation unit 1026. RI and an estimated motion vector index PMVI and a motion vector residual MVD supplied from a motion vector redundancy deletion unit 1029 (described later) are supplied as prediction parameters PP to a variable length code encoding unit.

The prediction residual D is generated by the subtracter 1032 by subtracting the prediction image Pred selected by the prediction method control unit 1028 from the encoding target image. The prediction residual D generated by the subtracter 1032 is DCT transformed / quantized by the transform / quantization unit 1021 as described above. On the other hand, by adding the prediction image Pred selected by the prediction method control unit 1028 to the prediction residual D generated by the inverse quantization / inverse transformation unit 1023, the adder 1031 generates a local decoded image P. Generated. The local decoded image P generated by the adder 1031 passes through the deblocking filter 1033 and the adaptive filter 1034, and is then stored in the buffer memory 1024 as a filtered decoded image P_FL and used as a reference image in inter prediction.

The motion vector redundancy deletion unit 1029 deletes the redundancy in the motion vector mv detected by the motion vector detection unit 1027. Specifically, (1) an estimation method used for estimating the motion vector mv is selected, (2) an estimated motion vector pmv is derived according to the selected estimation method, and (3) the estimated motion vector pmv is subtracted from the motion vector mv. As a result, a motion vector residual MVD is generated. The motion vector redundancy deletion unit 1029 supplies the generated motion vector residual MVD to the prediction method control unit 1028 together with the estimated motion vector index PMVI indicating the selected estimation method.

The deblocking filter 1033 determines the block boundary in the decoded image P or the block boundary in the decoded image P when the difference between the pixel values of pixels adjacent to each other via the block boundary or the macroblock boundary in the decoded image P is smaller than a predetermined threshold value. By performing deblocking processing on the macroblock boundary, an image near the block boundary or the macroblock boundary is smoothed. The image that has been deblocked by the deblocking filter 1033 is output to the adaptive filter 1034 as a deblocked decoded image P_DB.

The adaptive filter 1034 generates a filtered decoded image P_FL by performing an adaptive filter process on the deblocked decoded image P_DB supplied from the deblocking filter 1033. The filtered decoded image P_FL that has been filtered by the adaptive filter 1034 is stored in the buffer memory 1024.

Here, the configuration of moving image encoding device 1002 corresponding to the case where encoded data # 1001 has the data structure shown in FIG. 47 has been described, but the configuration of moving image encoding device 1002 is not limited to this. That is, by appropriately changing the processing unit of various processes, the moving picture encoding apparatus 1002 can be made to correspond to the encoded data # 1001 having the data structure shown in FIG.
<< Application example >>
The above-described moving

image encoding devices

2 and 1002 and moving

image decoding devices

1 and 1001 can be used by being mounted on various devices that transmit, receive, record, and reproduce moving images. The moving image may be a natural moving image captured by a camera or the like, or may be an artificial moving image (including CG and GUI) generated by a computer or the like. In the following, the moving image encoding device 2 and the moving image decoding device 1 will be described as an example, but the same applies to the moving image encoding device 1002 and the moving image decoding device 1001.

First, it will be described with reference to FIG. 51 that the above-described moving picture encoding apparatus 2 and moving picture decoding apparatus 1 can be used for transmission and reception of moving pictures.

FIG. 51A is a block diagram illustrating a configuration of a transmission device PROD_A in which the moving image encoding device 2 is mounted. As illustrated in (a) of FIG. 51, the transmission device PROD_A modulates a carrier wave with an encoding unit PROD_A1 that obtains encoded data by encoding a moving image and the encoded data obtained by the encoding unit PROD_A1. Thus, a modulation unit PROD_A2 that obtains a modulation signal and a transmission unit PROD_A3 that transmits the modulation signal obtained by the modulation unit PROD_A2 are provided. The moving image encoding apparatus 2 described above is used as the encoding unit PROD_A1.

The transmission device PROD_A is a camera PROD_A4 that captures a moving image, a recording medium PROD_A5 that records the moving image, an input terminal PROD_A6 that inputs the moving image from the outside, as a supply source of the moving image input to the encoding unit PROD_A1. An image processing unit A7 that generates or processes an image may be further provided. In FIG. 51A, a configuration in which all of these are provided in the transmission device PROD_A is illustrated, but a part may be omitted.

The recording medium PROD_A5 may be a recording of a non-encoded moving image, or a recording of a moving image encoded by a recording encoding scheme different from the transmission encoding scheme. It may be a thing. In the latter case, a decoding unit (not shown) for decoding the encoded data read from the recording medium PROD_A5 according to the recording encoding method may be interposed between the recording medium PROD_A5 and the encoding unit PROD_A1.

FIG. 51B is a block diagram illustrating a configuration of a receiving device PROD_B in which the moving image decoding device 1 is mounted. As illustrated in FIG. 51 (b), the receiving device PROD_B includes a receiving unit PROD_B1 that receives a modulated signal, a demodulating unit PROD_B2 that obtains encoded data by demodulating the modulated signal received by the receiving unit PROD_B1, and a demodulator. A decoding unit PROD_B3 that obtains a moving image by decoding the encoded data obtained by the unit PROD_B2. The moving picture decoding apparatus 1 described above is used as the decoding unit PROD_B3.

The receiving device PROD_B has a display PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal for outputting the moving image to the outside as a supply destination of the moving image output by the decoding unit PROD_B3. PROD_B6 may be further provided. FIG. 51B illustrates a configuration in which all of these are provided in the receiving device PROD_B, but some of them may be omitted.

The recording medium PROD_B5 may be used for recording a non-encoded moving image, or may be encoded using a recording encoding method different from the transmission encoding method. May be. In the latter case, an encoding unit (not shown) for encoding the moving image acquired from the decoding unit PROD_B3 according to the recording encoding method may be interposed between the decoding unit PROD_B3 and the recording medium PROD_B5.

Note that the transmission medium for transmitting the modulation signal may be wireless or wired. Further, the transmission mode for transmitting the modulated signal may be broadcasting (here, a transmission mode in which the transmission destination is not specified in advance) or communication (here, transmission in which the transmission destination is specified in advance). Refers to the embodiment). That is, the transmission of the modulation signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

For example, a terrestrial digital broadcast broadcasting station (broadcasting equipment or the like) / receiving station (such as a television receiver) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by wireless broadcasting. Further, a broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by cable broadcasting.

Also, a server (workstation etc.) / Client (television receiver, personal computer, smart phone etc.) such as VOD (Video On Demand) service and video sharing service using the Internet is a transmitting device for transmitting and receiving modulated signals by communication. This is an example of PROD_A / reception device PROD_B (usually, either a wireless or wired transmission medium is used in a LAN, and a wired transmission medium is used in a WAN). Here, the personal computer includes a desktop PC, a laptop PC, and a tablet PC. The smartphone also includes a multi-function mobile phone terminal.

In addition to the function of decoding the encoded data downloaded from the server and displaying it on the display, the video sharing service client has a function of encoding a moving image captured by the camera and uploading it to the server. That is, the client of the video sharing service functions as both the transmission device PROD_A and the reception device PROD_B.

Next, it will be described with reference to FIG. 52 that the moving picture encoding apparatus 2 and the moving picture decoding apparatus 1 described above can be used for recording and reproduction of moving pictures.

FIG. 52 (a) is a block diagram showing a configuration of a recording apparatus PROD_C in which the above-described moving picture encoding apparatus 2 is mounted. As shown in (a) of FIG. 52, the recording device PROD_C includes an encoding unit PROD_C1 that obtains encoded data by encoding a moving image, and the encoded data obtained by the encoding unit PROD_C1 on the recording medium PROD_M. A writing unit PROD_C2 for writing. The moving image encoding apparatus 2 described above is used as the encoding unit PROD_C1.

The recording medium PROD_M may be of a type built in the recording device PROD_C, such as (1) HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) SD memory. It may be of the type connected to the recording device PROD_C, such as a card or USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray Disc: registration) Or a drive device (not shown) built in the recording device PROD_C.

The recording device PROD_C is a camera PROD_C3 that captures moving images as a supply source of moving images to be input to the encoding unit PROD_C1, an input terminal PROD_C4 for inputting moving images from the outside, and reception for receiving moving images. The unit PROD_C5 and an image processing unit C6 that generates or processes an image may be further provided. FIG. 52A illustrates a configuration in which the recording apparatus PROD_C includes all of these, but a part of the configuration may be omitted.

The receiving unit PROD_C5 may receive a non-encoded moving image, or may receive encoded data encoded by a transmission encoding scheme different from the recording encoding scheme. You may do. In the latter case, a transmission decoding unit (not shown) that decodes encoded data encoded by the transmission encoding method may be interposed between the reception unit PROD_C5 and the encoding unit PROD_C1.

Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, and an HDD (Hard Disk Drive) recorder (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is a main supply source of moving images). . In addition, a camcorder (in this case, the camera PROD_C3 is a main source of moving images), a personal computer (in this case, the receiving unit PROD_C5 or the image processing unit C6 is a main source of moving images), a smartphone (in this case In this case, the camera PROD_C3 or the receiving unit PROD_C5 is a main supply source of moving images) is also an example of such a recording device PROD_C.

FIG. 52 (b) is a block diagram showing a configuration of a playback device PROD_D in which the above-described video decoding device 1 is mounted. As shown in (b) of FIG. 52, the playback device PROD_D reads the moving image by decoding the read data PROD_D1 for reading the encoded data written on the recording medium PROD_M and the encoded data read by the read unit PROD_D1. And a decoding unit PROD_D2 to be obtained. The moving picture decoding apparatus 1 described above is used as the decoding unit PROD_D2.

Note that the recording medium PROD_M may be of the type built into the playback device PROD_D, such as (1) HDD or SSD, or (2) such as an SD memory card or USB flash memory, It may be of a type connected to the playback device PROD_D, or (3) may be loaded into a drive device (not shown) built in the playback device PROD_D, such as DVD or BD. Good.

In addition, the playback device PROD_D has a display PROD_D3 that displays a moving image, an output terminal PROD_D4 that outputs the moving image to the outside, and a transmission unit that transmits the moving image as a supply destination of the moving image output by the decoding unit PROD_D2. PROD_D5 may be further provided. FIG. 52B illustrates a configuration in which the playback apparatus PROD_D includes all of these, but some of the configurations may be omitted.

The transmission unit PROD_D5 may transmit an unencoded moving image, or transmits encoded data encoded by a transmission encoding method different from the recording encoding method. You may do. In the latter case, it is preferable to interpose an encoding unit (not shown) that encodes a moving image with an encoding method for transmission between the decoding unit PROD_D2 and the transmission unit PROD_D5.

Examples of such a playback device PROD_D include a DVD player, a BD player, and an HDD player (in this case, an output terminal PROD_D4 to which a television receiver or the like is connected is a main supply destination of moving images). . In addition, a television receiver (in this case, the display PROD_D3 is a main supply destination of moving images), a digital signage (also referred to as an electronic signboard or an electronic bulletin board), and the display PROD_D3 or the transmission unit PROD_D5 is the main supply of moving images. Desktop PC (in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is the main video image supply destination), laptop or tablet PC (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a moving image) A smartphone (which is a main image supply destination), a smartphone (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a main moving image supply destination), and the like are also examples of such a playback device PROD_D.

(Summary of Embodiments 2 to 4)
As described above, in the adaptive filter 100 including the linear filter 121 in which filter coefficients are adaptively set, the pixel value of the target pixel in the output image of the linear filter 121 and the pixel value of the target pixel in the input image of the linear filter 121 Truncating means for truncating the change amount Δs, which is the difference between the two, is provided.

(About correspondence with HEVC)
In the above embodiment, LCU (Largest Coding Unit) is H.264. It corresponds to the root of a coding tree (Coding Tree) of HEVC (High Efficiency Video Coding) proposed as a successor to H.264 / MPEG-4 AVC, and a leaf CU is a CU (Coding Unit, coding) It is also called the leaf of the tree). Moreover, PU and TU in the said embodiment are respectively equivalent to the prediction tree (Prediction Tree) and transformation tree (transform tree) in HEVC. Moreover, the partition of PU in the said embodiment is corresponded to PU (Prediction Unit) in HEVC. In the above embodiment, a block obtained by dividing a TU corresponds to a TU (Transformation Unit) in HEVC.

The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

The present invention can be suitably used for an image filter device that filters an image. Further, the present invention can be suitably used for a decoding device that decodes encoded data and an encoding device that generates encoded data. Further, it can be suitably used for an image encoding device such as a moving image encoding device and an image decoding device such as a moving image decoding device.

1 Video decoding device (decoding device)
12 Predictive image generation unit (predictive image generation means)
17 Adaptive filter (image filter device)
171 Area characteristic value calculation unit (characteristic value calculation means)
171a Activity calculation unit 171b Pixel value reference unit 172 Region classification unit (classification means)
173 Filter processing unit (filter means)
2 Video encoding device (encoding device)
21 Predicted image generating unit (predicted image generating means)
27 Adaptive filter (image filter device)
271 Area characteristic value calculation unit (characteristic value calculation means)
271a Activity calculation unit 271b Pixel value reference unit 272 Region classification unit (classification means)
273 Filter processing unit (filter means)
100, 200, 300, 42 Adaptive filter (filter device on the decoding device side)
110, 210, 310

Filter setting unit

120, 220, 320 Filter operation unit 100 ', 200', 300 ', 34 Adaptive filter (filter device on the encoding device side)
110 ′, 210 ′, 310 ′ Filter setting unit 120 ′, 220 ′, 320 ′ Filter operation unit 311 Spatial filter coefficient decoding unit 312 Offset table decoding unit 311a Adaptive filter coefficient determination unit 311b Spatial filter coefficient encoding unit 312b Offset table code 1001 moving picture decoding apparatus (decoding apparatus)
1002 Moving picture encoding apparatus

Claims

Filter means for acting on an input image composed of a plurality of unit regions;
For each of the plurality of unit areas, a characteristic value calculating means for calculating a characteristic value representing an image characteristic of the input image;
Depending on the value of the characteristic value for the unit area calculated by the characteristic value calculation means, each unit area is either a filter-on area in which the filter means is applied or a filter-off area in which the filter means is not applied. Classification means for classifying
An image filter device.
The classification means further classifies each unit area belonging to the filter-on area into any of a plurality of types according to the value of the characteristic value for the unit area calculated by the characteristic value calculation means,
The filter means calculates a pixel value of each pixel in each unit region belonging to the filter-on region using a filter coefficient group set for a type including the pixel;
The image filter device according to claim 1.
The characteristic value calculation means calculates the activity of the input image in the unit area as a characteristic value for each unit area.
The image filter device according to claim 1, wherein the image filter device is an image filter device.
The characteristic value calculation means refers to the degree of activity of each pixel belonging to the internal region, the entire neighborhood of which is composed of pixels included in the unit region, and the pixel value of the pixel included in the vicinity of the pixel. And then calculating the activity of the unit region from the activity of each pixel belonging to the internal region.
The image filter device according to claim 3.
The characteristic value calculation means calculates an average of pixel values of the input image in the unit area as a characteristic value for each unit area.
The image filter device according to claim 1, wherein the image filter device is an image filter device.
The characteristic value calculation means calculates a plurality of characteristic values with different calculation methods for each unit region.
The image filter device according to claim 1, wherein the image filter device is an image filter device.
The characteristic value calculating means calculates an average of pixel values of the input image in the unit area and a variance of pixel values of the input image in the unit area as characteristic values for each unit area.
The image filter device according to claim 6.
The characteristic value calculation means calculates the vertical activity of the input image in the unit area and the horizontal activity of the input image in the unit area as characteristic values for each unit area.
The image filter device according to claim 6.
The classification means obtains an on / off flag for designating on / off of the filtering process for each predetermined unit including one or more unit areas from outside, and unit areas included in the unit for which the on / off flag designates on, Classify into either the filter-on region or the filter-off region;
The image filter device according to claim 1, wherein the image filter device is an image filter device.
The predetermined unit related to the on / off flag is hierarchically specified by hierarchical specification information acquired from the outside, and the unit area is composed of pixels of the minimum encoding unit and 4 pixels or more. An area that is determined without depending on the hierarchy designation information.
The image filter device according to claim 9.
A decoding device that decodes encoded data and generates a decoded image after filtering,
The image filter device according to any one of claims 1 to 10,
Predicted image generation means for generating a predicted image in each unit region with reference to the output image generated by the image filter device;
With
The image filter device uses, as an input image, a decoded image obtained by adding the predicted image generated by the predicted image generation unit and a residual image decoded from the encoded data as an output image. Generate a decoded image after filtering.
A decoding device characterized by the above.
An encoding device that generates encoded data by encoding a residual image between an encoding target image and a predicted image,
The image filter device according to any one of claims 1 to 10,
Predicted image generation means for generating the predicted image in each unit region with reference to the output image generated by the image filter device;
With
The image filter device generates an output image using a decoded image obtained by adding the prediction image generated by the prediction image generation unit and the residual image as an input image,
An encoding apparatus characterized by that.
The filter coefficient group used by the filter means is determined so as to minimize the difference between the encoding target image and the output image in each unit region included in the filter-on region.
The encoding apparatus according to claim 12.
Filter means acting on an input image composed of a plurality of unit areas; characteristic value calculation means for calculating a characteristic value representing the image characteristics of the input image for each of the plurality of unit areas; and Classifying means for classifying into either a filter-on region in which the filter means is applied or a filter-off region in which the filter means is not operated according to the value of the characteristic value for the unit area calculated by the value calculation means, A data structure of encoded data referred to by an image filter device comprising:
Including filter coefficient groups used by the filter means,
A data structure of encoded data characterized by the above.
For each of a plurality of unit areas constituting the input image, calculation means for calculating the average of the pixel values of the input image;
Classifying means for classifying each unit area into one of a plurality of types according to the average of the pixel values for the unit area calculated by the calculating means,
Filter means for calculating a pixel value of each pixel in each unit area using a filter coefficient group set for a type including the pixel;
An image filter device comprising:
In a filter device comprising a linear filter in which filter coefficients are adaptively set,
A region characteristic calculation means for calculating the value of the region characteristic of each unit region is provided,
The linear filter is an offset that depends on the region characteristic for performing a filter operation on each pixel belonging to each unit region, and is in accordance with the region characteristic value of the unit region calculated by the region characteristic calculating unit. Execute using offset and filter coefficient group that does not depend on the region characteristics,
And a filter device.
The region characteristic has a value corresponding to the type of the color difference component,
The linear filter performs the filter operation using an offset according to the type of color difference component.
The filter device according to claim 16.
In a filter device comprising a linear filter in which filter coefficients are adaptively set,
Truncating means for truncating the amount of change that is the difference between the pixel value of the target pixel in the output image of the linear filter and the pixel value of the target pixel in the input image of the linear filter;
And a filter device.
Flag decoding means for decoding a flag indicating whether to truncate the change amount;
A switch that switches whether to output the output image of the linear filter by truncating the change amount or outputting the change amount without truncation according to the value of the flag decoded by the flag decoding means; , Further comprising
The filter device according to claim 18.
Area characteristic calculating means for calculating area characteristics of each unit area;
A threshold value used to truncate the change amount when executing a filtering process using a pixel belonging to each unit area as an output pixel is determined according to the area characteristic of the unit area calculated by the area characteristic calculation unit. Threshold value determining means for
The filter device according to claim 18 or 19, wherein
Threshold table decoding means for decoding the threshold table recorded in association with the region characteristics and the threshold,
The threshold value determination means determines the threshold value with reference to the threshold value table;
The filter device according to claim 20.
In a filter device comprising a linear filter in which filter coefficients are adaptively set,
A first filter coefficient table in which at least one first filter coefficient group depending on the first area characteristic is recorded in association with a value of the corresponding first area characteristic, and a second filter coefficient depending on the second area characteristic Decoding means for decoding a second filter coefficient table in which at least one group is recorded in association with a corresponding second region characteristic value;
Area characteristic calculating means for calculating values of the first area characteristic and the second area characteristic of each unit area,
The linear filter is a filter operation using each pixel belonging to each unit region as a target pixel, and the region characteristic calculation unit calculates the first filter coefficient group recorded in the first filter coefficient table. Of the first filter coefficient group corresponding to the value of the first area characteristic of the unit area and the second filter coefficient group recorded in the second filter coefficient table, the unit area calculated by the area characteristic calculating means Performing a filter operation using the second filter coefficient group corresponding to the value of the second region characteristic;
And a filter device.
Flag decoding means for decoding a flag indicating whether or not to use the second filter coefficient group for filter operation in the linear filter;
An output image obtained by a filter operation using the first filter coefficient group and the second filter coefficient group in the linear filter is output to the outside, or a filter operation using only the first filter coefficient group in the linear filter A switch for switching whether to output the output image obtained in accordance with the value of the flag decoded by the flag decoding means,
The filter device according to claim 22.
A decoding device for decoding an image, comprising the filter device according to any one of claims 16 to 23 as an adaptive filter that acts on the decoded image.
An encoding apparatus that encodes an image, the encoding apparatus including the filter device according to any one of claims 16 to 23 as a filter that acts on a locally decoded image.