JP7485838B2 - Image decoding device, image decoding method, and program - Google Patents

Description

The present invention relates to an image decoding device, an image decoding method, and a program.

Conventionally, a technique is known that realizes nonlinear filtering by clipping the input signal to an "ALF (Adaptive Loop Filter)" at a threshold value (see, for example, Non-Patent Document 1).

The thresholds are defined by mathematical expressions, and the final values are derived according to the internal bit depth settings, etc. The thresholds are defined separately for the luminance signal and the color difference signal.

Versatile Video Coding (Draft 5), JVET-N1001

However, in the above-mentioned conventional technology, the magnitude relationship between the luminance signal threshold and the color difference signal threshold is inverted depending on the internal bit depth. This causes a problem in that even for the same input signal, the characteristics of the decoded image change depending on the internal bit depth setting, which may affect the subjective image quality.

The present invention has been made in consideration of the above-mentioned problems, and aims to provide an image decoding device, an image decoding method, and a program that can prevent a situation in which the characteristics of a decoded image change and affect the subjective image quality.

The first feature of the present invention is an image decoding device comprising a filter unit configured to receive an unfiltered decoded signal as input and output a filtered decoded signal, the filter unit configured to perform clipping on the unfiltered decoded signal so that the absolute value of the difference between a reference pixel value and a pixel value of the unfiltered decoded signal is equal to or less than a predefined threshold, and to generate the filtered decoded signal by a linear weighted sum of the value after the clipping and the pixel value of the unfiltered decoded signal, the gist being that the magnitude relationship between the threshold for the luminance signal and the threshold for the chrominance signal is preserved even if the internal bit depth changes.

The second feature of the present invention is an image decoding method having a step of receiving an unfiltered decoded signal as an input and outputting a filtered decoded signal, in which, in the step, clipping is performed on the unfiltered decoded signal so that the absolute value of the difference between a reference pixel value and a pixel value of the unfiltered decoded signal is equal to or less than a predefined threshold, and the filtered decoded signal is generated by linearly weighting the value after the clipping and the pixel value of the unfiltered decoded signal, and the magnitude relationship between the threshold for the luminance signal and the threshold for the chrominance signal is defined so as to be preserved even if the internal bit depth changes.

The third feature of the present invention is a program for use in an image decoding device, which causes a computer to execute a process of receiving an unfiltered decoded signal as input and outputting a filtered decoded signal, in which clipping is performed on the unfiltered decoded signal so that the absolute value of the difference between a reference pixel value and a pixel value of the unfiltered decoded signal is equal to or less than a predefined threshold, and the filtered decoded signal is generated by linearly weighting the value after the clipping and the pixel value of the unfiltered decoded signal, and the magnitude relationship between the threshold value for the luminance signal and the threshold value for the chrominance signal is defined so as to be preserved even if the internal bit depth changes.

The present invention provides an image decoding device, an image decoding method, and a program that can prevent changes in the characteristics of a decoded image, which can affect the subjective image quality.

1 is a diagram illustrating an example of a configuration of an image processing system 10 according to an embodiment. FIG. 2 is a diagram illustrating an example of functional blocks of an image encoding device 100 according to an embodiment. 1 is a diagram illustrating an example of functional blocks of an in-loop filter processing unit 150 of an image encoding device 100 according to an embodiment. 11 is a diagram for explaining an example of the function of a filter section 150B of an in-loop filter processing section 150 of an image encoding device 100 according to an embodiment. FIG. 11 is a diagram for explaining an example of the function of a filter section 150B of an in-loop filter processing section 150 of an image encoding device 100 according to an embodiment. FIG. 11 is a diagram for explaining an example of the function of a filter section 150B of an in-loop filter processing section 150 of an image encoding device 100 according to an embodiment. FIG. 11 is a diagram for explaining an example of functions of a filter section 150B of an in-loop filter processing section 150 of an image encoding device 100 according to an embodiment. FIG. FIG. 2 is a diagram illustrating an example of functional blocks of an image decoding device 200 according to an embodiment. 2 is a diagram illustrating an example of functional blocks of an in-loop filter processing unit 250 of an image decoding device 200 according to an embodiment. FIG. 13 is a flowchart showing an example of a processing procedure of an in-loop filtering unit 250 of the image decoding device 200 according to an embodiment.

The following describes embodiments of the present invention with reference to the drawings. Note that the components in the following embodiments can be replaced with existing components as appropriate, and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

First Embodiment
An image processing system 10 according to a first embodiment of the present invention will be described below with reference to Figures 1 to 10. Figure 1 is a diagram showing an image processing system 10 according to the present embodiment.

As shown in FIG. 1, the image processing system 10 has an image encoding device 100 and an image decoding device 200.

The image encoding device 100 is configured to generate encoded data by encoding an input image signal. The image decoding device 200 is configured to generate an output image signal by decoding the encoded data.

Here, the encoded data may be transmitted from the image encoding device 100 to the image decoding device 200 via a transmission path. Also, the encoded data may be stored in a storage medium and then provided from the image encoding device 100 to the image decoding device 200.

(Image encoding device 100)
The image encoding device 100 according to this embodiment will be described below with reference to Fig. 2. Fig. 2 is a diagram showing an example of functional blocks of the image encoding device 100 according to this embodiment.

As shown in FIG. 2, the image encoding device 100 includes an inter prediction unit 111, an intra prediction unit 112, a subtractor 121, an adder 122, a transform/quantization unit 131, an inverse transform/inverse quantization unit 132, an encoding unit 140, an in-loop filter processing unit 150, and a frame buffer 160.

The inter prediction unit 111 is configured to generate a prediction signal by inter prediction (inter-frame prediction).

Specifically, the inter prediction unit 111 is configured to identify a reference block included in a reference frame by comparing a frame to be coded (hereinafter, the target frame) with a reference frame stored in the frame buffer 160, and to determine a motion vector for the identified reference block.

Furthermore, the inter prediction unit 111 is configured to generate a prediction signal included in the prediction block for each prediction block based on the reference block and the motion vector. The inter prediction unit 111 is configured to output the prediction signal to the subtractor 121 and the adder 122. Here, the reference frame is a frame different from the target frame.

The intra prediction unit 112 is configured to generate a prediction signal by intra prediction (intra-frame prediction).

Specifically, the intra prediction unit 112 is configured to identify a reference block included in the target frame and generate a prediction signal for each prediction block based on the identified reference block. The intra prediction unit 112 is also configured to output the prediction signal to the subtractor 121 and the adder 122.

Here, the reference block is a block that is referenced for the block to be predicted (hereinafter, the target block). For example, the reference block is a block adjacent to the target block.

The subtractor 121 is configured to subtract the prediction signal from the input image signal and output the prediction residual signal to the transform/quantization unit 131. Here, the subtractor 121 is configured to generate a prediction residual signal that is the difference between the prediction signal generated by intra prediction or inter prediction and the input image signal.

The adder 122 is configured to add the prediction signal to the prediction residual signal output from the inverse transform/inverse quantization unit 132 to generate a pre-filter decoded signal, and output the pre-filter decoded signal to the intra prediction unit 112 and the in-loop filter processing unit 150.

Here, the unfiltered decoded signal constitutes a reference block used by the intra prediction unit 112.

The transform/quantization unit 131 is configured to perform a transform process on the prediction residual signal and to obtain coefficient level values. Furthermore, the transform/quantization unit 131 may be configured to quantize the coefficient level values.

The conversion process is a process of converting a prediction residual signal into a frequency component signal. In such a conversion process, a basis pattern (transformation matrix) corresponding to a discrete cosine transform (DCT) may be used, or a basis pattern (transformation matrix) corresponding to a discrete sine transform (DST) may be used.

The inverse transform and inverse quantization unit 132 is configured to perform inverse transform processing of the coefficient level values output from the transform and quantization unit 131. Here, the inverse transform and inverse quantization unit 132 may be configured to perform inverse quantization of the coefficient level values prior to the inverse transform processing.

Here, the inverse transformation process and inverse quantization are performed in the reverse order to the transformation process and quantization performed by the transformation/quantization unit 131.

The encoding unit 140 is configured to encode the coefficient level values output from the transform/quantization unit 131 and output the encoded data.

Here, for example, the coding is entropy coding, which assigns codes of different lengths based on the probability of occurrence of coefficient level values.

The encoding unit 140 is also configured to encode control data used in the decoding process in addition to the coefficient level values.

Here, the control data may include size data such as coding block (CU: Coding Unit) size, prediction block (PU: Prediction Unit) size, and transform block (TU: Transform Unit) size.

The in-loop filter processing unit 150 is configured to perform filtering on the unfiltered decoded signal output from the adder 122, and to output the filtered decoded signal to the frame buffer 160.

The in-loop filter processing unit 150 may be configured to determine parameters related to the filter processing using the input image signal and the unfiltered decoded signal as input, and output such parameters to the encoding unit 140. The encoding unit 140 may be configured to encode such parameters and transmit them to the image decoding device 200 as additional information.

Here, for example, the filtering process is an adaptive loop filtering process that reduces the coding distortion of the decoded image.

The frame buffer 160 is configured to store reference frames used by the inter prediction unit 111.

Here, the filtered decoded signal constitutes a reference frame used by the inter prediction unit 111.

(In-loop filter processing unit 150)
The in-loop filtering unit 150 according to this embodiment will be described below. Fig. 3 is a diagram showing the in-loop filtering unit 150 according to this embodiment.

As shown in FIG. 3, the in-loop filter processing unit 150 has a class determination unit 150A, a filter unit 150B, and a parameter determination unit 150C.

The in-loop filter processing unit 150 can select whether to apply adaptive loop filter processing or not for each coding tree block (CTU). In addition, when adaptive loop filter processing is applied, the in-loop filter processing unit 150 can select which filter set to use from among multiple filter sets.

Each filter set contains up to 25 types (classes) of filters for luminance signals and one class of filter for color difference signals.

Parameters including information on whether or not to apply such adaptive loop filter processing and information on the filter set to be used in the adaptive loop filter processing are coded for each CTU as additional information, as described below, and transmitted to the image decoding device 200. Note that such parameters are determined by the parameter determination unit 150C, as described below.

Here, for example, if the input signal is in "YCbCr 4:2:0 format," the CTU may define each block by dividing the luminance (Y) signal into blocks of 128 x 128 pixels, and the color difference signals (Cb, Cr) into blocks of 64 x 64 pixels.

For pixels of the luminance signal and color difference signal that belong to a CTU for which it is determined that adaptive loop filter processing is not to be applied, the determination process by the class determination unit 150A and the filtering process by the filter unit 150B, which will be described later, can be omitted.

The class determination unit 150A is configured to receive the unfiltered decoded signal as input and output class determination information indicating which type (class) of filter to use from among multiple predetermined types of filters (adaptive loop filters).

Here, the class determination unit 150A is configured to divide the unfiltered decoded signal into small blocks and determine the class to be used for each block. For example, the class determination unit 150A can determine which class of filter out of 25 types should be used for each 4x4 pixel block.

Here, any method can be used to determine the class, as long as it is a method that can make a determination using only information obtained from both the image encoding device 100 and the image decoding device 200 as input.

For example, in Non-Patent Document 1, the above-mentioned class determination is performed using the gradient of pixel values of the unfiltered decoded signal. By using only information obtained on both the image encoding device 100 side and the image decoding device 200 side, the image decoding device 200 side can determine the class in the same way as the image encoding device 100 side, so there is no need to transmit class determination information from the image encoding device 100 side to the image decoding device 200 side.

The filter unit 150B is configured to receive the unfiltered decoded signal as input and to output the filtered decoded signal.

Specifically, the filter unit 150B is configured to perform filtering using the unfiltered decoded signal, the class determination information output from the class determination unit 150A, and the parameters (a group of parameters for loop filter processing) input from the parameter determination unit 150C as inputs, and to output the filtered decoded signal.

The value of each pixel in the filtered decoded signal can be calculated, for example, using the following formula:

ここで、Ｉ（ｘ,ｙ）は、（ｘ,ｙ）の座標にあるフィルタ処理前復号信号の画素値であり、Ｏ（ｘ,ｙ）は、（ｘ,ｙ）の座標にあるフィルタ処理後復号信号の画素値であり、（ｉ,ｊ）は、（ｘ,ｙ）の座標にある画素に対する参照画素の相対位置を示す座標（参照画素位置）であり、Ｃ（ｉ,ｊ）は、参照画素位置（ｉ,ｊ）に対応するフィルタ係数であり、Ｋ（）は、以下に示すクリッピング処理であり、ｋ（ｉ,ｊ）は、クリッピング処理で用いられる閾値である。

Here, I(x,y) is the pixel value of the unfiltered decoded signal at the coordinates (x,y), O(x,y) is the pixel value of the filtered decoded signal at the coordinates (x,y), (i,j) is the coordinates (reference pixel position) indicating the relative position of the reference pixel with respect to the pixel at the coordinates (x,y), C(i,j) is the filter coefficient corresponding to the reference pixel position (i,j), K() is the clipping process shown below, and k(i,j) is the threshold used in the clipping process.

K(I,k)=min(k,max(-k,I))
Here, min() is a function that returns the minimum value among the arguments, and max() is a function that returns the maximum value among the arguments. Therefore, K() is a process that returns -k if the input value I is smaller than -k, returns k if the input value I is larger than k, and otherwise returns the input value I as is.

In other words, the filter unit 150B is configured to perform clipping processing on the unfiltered decoded signal so that the absolute value of the difference between the reference pixel value I(x+i,y+j) and the pixel value I(x,y) of the unfiltered decoded signal is equal to or less than a predefined threshold value k(i,j).

Furthermore, the filter unit 150B is configured to generate a filtered decoded signal by performing a linear weighted sum of the value after clipping and the pixel value I(x, y) of the unfiltered decoded signal.

The filter coefficients C are determined by the parameter determination unit 150C and transmitted to the image decoding device 200. In order to reduce the amount of coding for information related to the filter coefficients C, the number of filter coefficients C can be reduced by applying the same filter coefficient C to multiple pixels. A specific example is shown in FIG. 4.

Figure 4(a) shows an example of the arrangement of filter coefficients C in filtering a luminance signal, and Figure 4(b) shows an example of the arrangement of filter coefficients C in filtering a color difference signal.

As shown in FIG. 4(a), for example, 12 types of filter coefficients C0 to C11 are used for the luminance signal. In FIG. 4(a), the pixel marked X is the pixel position (x, y) whose pixel value is corrected by this filtering process. Pixels to which the same filter coefficient C is applied are arranged point-symmetrically around the pixel position (x, y).

As shown in FIG. 4(b), for the color difference signal, the other filter coefficients are similarly arranged point-symmetrically around the pixel marked X.

For example, when the reference pixel is a pixel outside the picture, filtering can be achieved by copying (called padding) the pixel value of the unfiltered decoded signal located at the picture boundary.

Figure 5 shows an example of filtering at the bottom edge of a picture (picture boundary).

In FIG. 5(a), the grayed-out filter coefficients C0 to C3 refer to pixels outside the picture. In this case, the input to the grayed-out filter coefficients C2 and C0 uses the same pixel value as the pixel value corresponding to the filter coefficient C6 located immediately above the filter coefficient C2 in FIG. 5(a).

Similarly, the input to the grayed out filter coefficient C3 is the same pixel value as the pixel value corresponding to the filter coefficient C7 located immediately above the filter coefficient C3 in FIG. 5(a), and the input to the grayed out filter coefficient C1 is the same pixel value as the pixel value corresponding to the filter coefficient C5 located immediately above the filter coefficient C1 in FIG. 5(a).

In addition, taking into account the point symmetry of the filter coefficient C described above, when padded values are used for some of the reference pixels, the padded values can also be used as input to the filter coefficient C at the positions corresponding to those reference pixels.

In the example of FIG. 5(b), in addition to the filtering process at the bottom edge of the picture described in the example of FIG. 5(a), padded values may also be input for the grayed-out filter coefficients C0 to C3.

For example, the pixel value corresponding to the filter coefficient C6 immediately below the filter coefficient C2 may be padded and used as the input to the filter coefficients C0 and C2 in FIG. 5(b).

Similarly, for the input of filter coefficients C1 and C3, the pixel values corresponding to the filter coefficients C5 and C7 immediately below filter coefficients C1 and C3, respectively, may be used by padding.

In the above example, filtering was described for the bottom edge of the picture, but similar filtering can be applied to the top, left, and right edges of the picture.

In addition to picture boundaries, situations in which pixel values beyond the boundary cannot be referenced, such as boundaries between parallel encoding processing units called slices or tiles, or boundaries in pipeline processing called virtual boundaries, can be dealt with in a similar manner by applying the above-mentioned filtering method.

Note that by making the filter processing at the above-mentioned picture boundaries, tile boundaries, slice boundaries, virtual boundaries, etc. the same process, implementation can be simplified compared to when different processes are performed for each boundary.

The thresholds used in the clipping process are set separately for each filter class and each filter coefficient C.

For example, if there are 25 classes of filters for luminance signals, and as described above, there are 12 types of filter coefficients C for each class, it is necessary to set thresholds for a maximum of 25 x 12 = 300 filter coefficients. Such thresholds are determined by the parameter determination unit 150C, which will be described later, and are transmitted to the image decoding device 200 as additional information.

Here, in order to reduce the amount of code related to the information related to such thresholds, instead of transmitting the thresholds themselves as the information related to such thresholds, it is also possible to code only index information indicating which of several predetermined thresholds is to be used for each filter coefficient.

In addition, if four types of thresholds for the luminance signal and four types of thresholds for the color difference signal are prepared in advance, the thresholds for the luminance signal and the color difference signal can be defined, for example, as shown in Figure 6 (a).

Since the variance of the luminance signal tends to be greater than the variance of the color difference signal, by setting the threshold for the luminance signal to be equal to or greater than the threshold for the color difference signal, as shown in Figure 6(a), clipping processing that takes into account the characteristics of each signal is possible.

That is, as shown in Figure 6(a), even if the internal bit depth ("bitdepth" in Figure 6) changes, the magnitude relationship between the threshold value for the luminance signal and the threshold value for the color difference signal (for example, the relationship that the threshold value for the luminance signal is equal to or greater than the threshold value for the color difference signal) is defined to be preserved.

Also, as shown in FIG. 6(a), the threshold for the luminance signal and the threshold for the color difference signal may be defined so that they are calculated by multiplying them by the same factor depending on the change in the internal bit depth. With this configuration, even if the internal bit depth changes, the magnitude relationship between the threshold for the luminance signal and the threshold for the color difference signal is preserved, and the same clipping process can be performed regardless of the internal bit depth.

Here, the internal bit depth refers to the bit precision when calculating pixel values of the luminance signal and the color difference signal during the encoding process and the decoding process. Such internal bit depth takes an integer value of, for example, 8 to 16 bits, and is transmitted from the image encoding device 100 to the image decoding device 200 as additional information.

Furthermore, by defining the magnification factor related to the internal bit depth as a power of 2 as shown in Figure 6(a), such clipping processing can be achieved by bit shifting alone as shown in Figure 6(b), thereby reducing the processing load on the hardware and software.

In other words, the threshold value for the luminance signal and the threshold value for the color difference signal may be defined so that they are calculated by bit shifting according to changes in the internal bit depth.

Note that the threshold value corresponding to index 0 in Figures 6(a) and 6(b) is the same as the maximum value that can be expressed with the corresponding internal bit depth. Therefore, by selecting such a threshold value, it is essentially equivalent to not performing clipping processing on the filter coefficient C corresponding to that threshold value.

To achieve the same effect as the above example, it is not necessary to set the multiplication factor for all thresholds to the same value, as mentioned above. As long as it is guaranteed that in a predetermined internal bit depth domain (e.g., 8 to 16 bits), the magnitude relationship of the thresholds corresponding to each index is not reversed in the luminance signal and in the chrominance signal, and that the magnitude relationship of the thresholds corresponding to the same index is not reversed between the luminance signal and the chrominance signal, there is no problem with setting the multiplication factor for each threshold to a different value.

When N types of thresholds for luminance signals and thresholds for color difference signals are provided, each of which is assigned an index from 0 to N-1 (N is a natural number equal to or greater than 1), the thresholds for luminance signals and thresholds for color difference signals having the same index may be defined to be calculated by multiplying by the same magnification factor according to the internal bit depth.

In the above example, a magnification factor is applied to the threshold value depending on the internal bit depth, but an offset value may be added instead of a magnification factor.

Even if the internal bit depth of the luminance signal and the internal bit depth of the color difference signal are different, by defining a threshold value for the luminance signal and a threshold value for the color difference signal, as shown in Figure 6, it is possible to define the relationship between the threshold values used in the clipping process between the luminance signal and the color difference signal so that they always remain constant when the dynamic ranges of the luminance signal and the color difference signal are converted to be the same.

In the above example, a case where a threshold is set for each class and each filter coefficient C has been described, but it is also possible to set the same threshold for all filter coefficients C for each class, that is, to set one threshold for each class. In this case, the number of thresholds to be transmitted is reduced, and therefore the amount of code related to the threshold index can be reduced.

In addition, as described below, a flag indicating whether or not clipping is performed for each class is transmitted prior to the threshold index. Therefore, if there is only one type of threshold for each class, there is no need to define a threshold corresponding to the above-mentioned index 0 (equivalent to no clipping). Therefore, in the above example, the possible threshold patterns can be reduced from four to three, which is expected to further reduce the amount of code related to the threshold index.

In the above example, different values are set for the threshold for the luminance signal and the threshold for the color difference signal, but it is also possible to set both to the same value. This allows, for example, the hardware to share circuits related to clipping processing for the luminance signal and the color difference signal, thereby reducing the circuit size.

The threshold value for the luminance signal and the threshold value for the color difference signal may be defined so that they are calculated by performing calculations according to the current internal bit depth, based on a reference internal bit depth of 10 bits.

For example, Figure 6(a) shows an example in which the thresholds for a 10-bit luminance signal are 1024, 181, 32, and 6, and the thresholds for a 10-bit chrominance signal are 1024, 161, 25, and 4, and the final thresholds are obtained by multiplying the current internal bit depth (bitdepth in Figure 6) by a power of 2.

In FIG. 6, the threshold value for 10 bits is expressed as a specific numerical value (1024, 181, etc.), but it may also be defined in the form of a mathematical formula.

For example, as shown in Figure 7, the threshold for a 10-bit luminance signal may be defined as "2^(10(10 x (4 - index)/4) (where index = 0, 1, 2, 3))" and multiplied by "2^(bitdepth - 10)" to obtain the final threshold according to the current internal bit depth.

Similarly, the definition of the color difference signal shown in Figure 7 is that "2^(8 x (3-index)/4) x 2^(2)" is the threshold for 10 bits, and by multiplying this by "2^(bitdepth-10)", the threshold corresponding to the current internal bit depth is calculated.

The parameter determination unit 150C is configured to receive the input image signal and the unfiltered decoded signal, determine parameters related to the adaptive loop filter, and output the parameters to the encoding unit 140 as additional information, and also output the determined parameters to the filter unit 150B.

The parameters determined by the parameter determination unit 150C include, for example, the following:

First, such parameters include a filter set on a frame-by-frame basis. Here, one filter set includes filters for luminance signals of up to 25 classes and a filter for chrominance signals of one class. For each class of filter, the values of the filter coefficients, a flag indicating whether or not clipping processing is to be performed, and a threshold index used in clipping processing for each filter coefficient when clipping processing is to be performed for that class are determined.

The parameter determination unit 150C has multiple filter sets for a given frame and can select which filter set to use for each region, as described below.

Secondly, such a parameter may be, for example, a flag indicating whether or not to apply an adaptive loop filter for each CTU. When an adaptive loop filter is applied, the parameter determination unit 150C may set index information indicating which of multiple filter sets is to be used.

The method for determining these parameters can be any known method, so details will be omitted.

In addition, although not shown in FIG. 3, the parameter determination unit 150C may use the results of the determination by the class determination unit 150A or the results of the filter unit 150B implemented with provisional parameter settings to determine the parameters.

(Image Decoding Device 200)
The image decoding device 200 according to this embodiment will be described below with reference to Fig. 7. Fig. 7 is a diagram showing an example of functional blocks of the image decoding device 200 according to this embodiment.

As shown in FIG. 7, the image decoding device 200 includes a decoding unit 210, an inverse transform/inverse quantization unit 220, an adder 230, an inter prediction unit 241, an intra prediction unit 242, an in-loop filter processing unit 250, and a frame buffer 260.

The decoding unit 210 is configured to decode the encoded data generated by the image encoding device 100 and to decode the coefficient level values.

Here, for example, the decoding is entropy decoding, which is the reverse procedure of the entropy encoding performed by the encoding unit 140.

The decoding unit 210 may also be configured to obtain control data by decoding the encoded data.

As mentioned above, the control data may include size data such as the coding block size, the prediction block size, and the transform block size.

The inverse transform/inverse quantization unit 220 is configured to perform inverse transform processing of the coefficient level values output from the decoding unit 210. Here, the inverse transform/inverse quantization unit 220 may be configured to perform inverse quantization of the coefficient level values prior to the inverse transform processing.

Here, the inverse transformation process and inverse quantization are performed in the reverse order to the transformation process and quantization performed by the transformation/quantization unit 131.

The adder 230 is configured to add the prediction signal to the prediction residual signal output from the inverse transform/inverse quantization unit 220 to generate a pre-filter decoded signal, and output the pre-filter decoded signal to the intra prediction unit 242 and the in-loop filter processing unit 250.

Here, the unfiltered decoded signal constitutes a reference block used by the intra prediction unit 242.

Like the inter prediction unit 111, the inter prediction unit 241 is configured to generate a prediction signal by inter prediction (inter-frame prediction).

Specifically, the inter prediction unit 241 is configured to generate a prediction signal for each prediction block based on a motion vector decoded from the encoded data and a reference signal included in the reference frame. The inter prediction unit 241 is configured to output the prediction signal to the adder 230.

The intra prediction unit 242, like the intra prediction unit 112, is configured to generate a prediction signal by intra prediction (intra-frame prediction).

Specifically, the intra prediction unit 242 is configured to identify a reference block included in the target frame and generate a prediction signal for each prediction block based on the identified reference block. The intra prediction unit 242 is configured to output the prediction signal to the adder 230.

Similar to the in-loop filter processing unit 150, the in-loop filter processing unit 250 is configured to perform filtering on the unfiltered decoded signal output from the adder 230, and to output the filtered decoded signal to the frame buffer 260.

Here, for example, the filtering process is an adaptive loop filtering process that reduces the coding distortion of the decoded image.

Like frame buffer 160, frame buffer 260 is configured to store reference frames used by inter prediction unit 241.

Here, the filtered decoded signal constitutes a reference frame used by the inter prediction unit 241.

(In-loop filter processing unit 250)
The in-loop filter processor 250 according to this embodiment will be described below. Fig. 9 is a diagram showing the in-loop filter processor 250 according to this embodiment.

As shown in FIG. 9, the in-loop filter processing unit 250 has a class determination unit 250A and a filter unit 250B.

Like class determination unit 150A, class determination unit 250A is configured to receive the unfiltered decoded signal as input and output class determination information indicating which class of filter to use from among multiple predefined types of in-loop filters.

Similar to filter unit 150B, filter unit 250B is configured to perform filtering based on the unfiltered decoded signal, the class determination information determined by class determination unit 250A, and parameters transmitted as additional information from the image encoding device 100 side, and to output the filtered decoded signal.

Figure 10 is a flowchart showing an example of the processing procedure of the in-loop filter processing unit 250 of the image decoding device 200 according to this embodiment.

As shown in FIG. 10, in step S101, the in-loop filter processing unit 250 receives the unfiltered decoded signal as input, and outputs class determination information indicating which class of filter to use from among multiple types of predefined in-loop filters.

In step S102, the in-loop filter processing unit 250 performs filtering based on the unfiltered decoded signal, the class determination information determined by the class determination unit 250A, and parameters transmitted as additional information from the image encoding device 100, and outputs a filtered decoded signal.

According to the image encoding device 100 and image decoding device 200 of this embodiment, when the dynamic ranges of the luminance signal and the color difference signal are converted to be the same in the threshold processing of the adaptive interpolation filter, the magnitude relationship between the threshold value for the luminance signal and the threshold value for the color difference signal remains unchanged regardless of the internal bit depth setting, so that it is possible to prevent unintended changes in the characteristics of the subjective image quality.

Note that the image encoding device 100 and image decoding device 200 described above may be realized as a program that causes a computer to execute each function (each process).

In the above embodiment, the present invention has been described by taking as an example the application to the image encoding device 100 and the image decoding device 200, but the present invention is not limited to such an example and can be similarly applied to an image encoding/decoding system having the functions of the image encoding device 100 and the image decoding device 200.

10... Image processing system 100... Image encoding device 111, 241... Inter prediction unit 112, 242... Intra prediction unit 121... Subtractor 122, 230... Adder 131... Transformation and quantization unit 132, 220... Inverse transformation and inverse quantization unit 140... Encoding unit 150, 250... In-loop filter processing unit 150A, 250A... Class determination unit 150B, 250B... Filter unit 150C... Parameter determination unit 160, 260... Frame buffer 200... Image decoding device 210... Decoding unit

Claims (3)

An image decoding device,
A class determination unit configured to determine which class of filter to use using an unfiltered decoded signal as input;
a parameter determination unit configured to determine a filter coefficient for each of the classes;
a filter unit configured to receive the unfiltered decoded signal as an input and to output a filtered decoded signal;
the filter unit is configured to perform clipping processing on a difference value between a reference pixel value and a pixel value of the unfiltered decoded signal such that an absolute value of the difference value between the reference pixel value and a pixel value of the unfiltered decoded signal is equal to or less than a predefined threshold, and to generate the filtered decoded signal by a linear weighted sum of the value after the clipping processing, the filter coefficient, and the pixel value of the unfiltered decoded signal,
the threshold is set for each of the classes and for each of the filter coefficients;
The threshold value for the luminance signal and the threshold value for the color difference signal are defined so as to be the same value even if the internal bit depth changes,
An image decoding device, characterized in that the threshold value is defined to double when the internal bit depth increases by 1. 1. An image decoding method, comprising:
A step A of determining which class of filter to use using an unfiltered decoded signal as an input;
A step B of determining filter coefficients for each of the classes;
A step C of receiving the unfiltered decoded signal and outputting a filtered decoded signal,
in the step C, a clipping process is performed on a difference value between a reference pixel value and a pixel value of the unfiltered decoded signal so that an absolute value of the difference value between the reference pixel value and a pixel value of the unfiltered decoded signal is equal to or less than a predefined threshold, and the filtered decoded signal is generated by a linear weighted sum of the value after the clipping process, the filter coefficient, and the pixel value of the unfiltered decoded signal;
the threshold is set for each of the classes and for each of the filter coefficients;
The threshold value for the luminance signal and the threshold value for the color difference signal are defined so as to be the same value even if the internal bit depth changes,
An image decoding method, characterized in that the threshold is defined to double when the internal bit depth increases by 1. A program for causing a computer to function as an image decoding device,
A class determination unit configured to determine which class of filter to use using an unfiltered decoded signal as input;
a parameter determination unit configured to determine a filter coefficient for each of the classes;
a filter unit configured to receive the unfiltered decoded signal as an input and to output a filtered decoded signal;
the filter unit is configured to perform clipping processing on a difference value between a reference pixel value and a pixel value of the unfiltered decoded signal such that an absolute value of the difference value between the reference pixel value and a pixel value of the unfiltered decoded signal is equal to or less than a predefined threshold, and to generate the filtered decoded signal by a linear weighted sum of the value after the clipping processing, the filter coefficient, and the pixel value of the unfiltered decoded signal,
the threshold is set for each of the classes and for each of the filter coefficients;
The threshold value for the luminance signal and the threshold value for the color difference signal are defined so as to be the same value even if the internal bit depth changes,
The program is characterized in that the threshold value is defined to double when the internal bit depth increases by 1.

Images