JP6957873B2 - Coding device, decoding device, coding method, decoding method, coding program and decoding program - Google Patents

Description

The present invention relates to a coding device, a decoding device, a coding method, a decoding method, a coding program and a decoding program, and can be applied to, for example, a coding device for compressing and coding video information and a decoding device for decoding. Is.

H. 264 / MPEG-4 AVC (hereinafter referred to as AVC) and H.A. The compression coding process of video information by the video coding method represented by 265 / MPEG-H HEVC (hereinafter referred to as HEVC) is performed by intra-prediction, motion compensation prediction, etc. for each processing unit in which the input target image is divided. The conversion coefficient obtained by performing spatial transformation such as discrete cosine transform on the predicted residual signal, which is the difference between the predicted image obtained by inter-prediction and the input target image, is quantized and entropy-encoded. Achieves high-efficiency video compression.

FIG. 3 is a configuration diagram showing a configuration of a conventional video coding device. When a coding technique such as HEVC is used, the input coded image (input video) is given to the difference processing unit 301. The difference processing unit 301 predicts the input coded image from the inter prediction unit 310 with motion compensation or the coded pixels in the screen for each processing unit area such as the coding unit. The difference from the predicted image according to 311 is taken, and the predicted residual signal which is the difference is obtained. The conversion unit 302 converts the predicted residual signal by DCT (discrete cosine transform) or DST (discrete sine transform). The quantization unit 303 quantizes the obtained conversion coefficient, and the entropy coding unit 304 outputs the coded stream by entropy-coding the quantized conversion coefficient such as a variable length code or an arithmetic code. The quantized conversion coefficient is inversely quantized by the inverse quantization unit 305 and inversely transformed by the inverse conversion unit 306. The addition unit 307 obtains a reconstructed image by adding the predicted image to the inversely transformed conversion coefficient. Further, the reconstructed image is filtered by the in-loop filter unit 308 such as a deblocking filter that reduces block distortion, and the reconstructed image after the filter is used to compensate for the motion of the inter-prediction at the time of encoding the subsequent image. It is held in the reference image buffer 309 as the reference image of.

FIG. 4 is a configuration diagram showing a configuration of a conventional video decoding device. When the coded stream generated by the coding device is input, the coded stream is entropy-decoded by the entropy decoding unit 404 to obtain a conversion coefficient such as DCT, coding mode information, and motion vector information. The conversion coefficient is inversely quantized by the inverse quantization unit 405 and inversely transformed by the inverse conversion unit 406. The addition unit 407 generates the same reconstructed image as the encoding device by adding the prediction image generated by the inter prediction unit 410 or the intra prediction unit 411 to the inversely converted conversion coefficient. The reconstructed image is output as a decoded image by applying a filter by the in-loop filter unit 408 such as a deblocking filter, and is held in the reference image buffer 409 as a reference image for subsequent inter-prediction.

In HEVC, in addition to the deblocking filter, an in-loop filter called a sample adaptive offset (hereinafter, also referred to as “SAO”) is used as the in-loop filter. This is used to reduce image quality deterioration such as ringing distortion that occurs in the block. In SAO, the pixels in the area to be processed are classified into several classes, and the filter processing is performed using the offset value for correcting the pixel value for each classification.

FIG. 5 is a configuration diagram showing sample adaptive offset processing in conventional HEVC. The classification number calculation unit 31 calculates the classification number for each pixel in the area to be processed by referring to the target pixel and the pixels around it. The offset addition unit 33 obtains the pixel value of the processing result by adding the offset value obtained from the offset table storage unit 32 that holds the offset value for each classification number to the value of the target pixel.

In SAO in HEVC, two types of filter methods are defined, which are called band offset (hereinafter referred to as "BO") and edge offset (hereinafter referred to as "EO"), respectively. It has been. The edge offset is further defined as an edge type in four directions.

FIG. 6 is a diagram showing an outline of band offset of conventional HEVC. The pixels to be processed are classified into a range of 32 steps of the pixel values, and an offset value is added to the pixels belonging to each classification for four consecutive classifications. That is, the same offset value is added to the pixels belonging to the same classification. In the case of BO, the classification number depends only on the target pixel and does not refer to the peripheral pixels.

FIG. 7 is a diagram showing an outline of the edge offset of the conventional HEVC. In the edge offset, as shown in FIG. 7, classification is performed using the pixel c to be processed and the two pixels a and b adjacent to the pixel c. As the reference patterns of the pixels a and b, four patterns as shown in FIG. 6 are defined according to the type of edge offset. For example, when applying the vertical pattern of the edge offset to the processing target area, the pixels adjacent to the upper and lower sides of each pixel c in the processing area are referred to as pixels a and b, respectively, and the pixels a, b, It is classified into five classes according to the magnitude relationship of the pixel values of c. Equation (1) is an equation for obtaining the classification number edgeIdx.
edgeIdx = 2 + Sign (ca) + Sign (c-b) ... (1)
However, Sign () is a function that gives "1" or "-1" depending on the sign of positive or negative, or "0" in the case of 0.

In HEVC, filtering processing is performed using offset values for four classifications (0, 1, 3, 4) in which the classification number edgeIdx is other than 2. That is, the same offset value is added to the pixels having the same classification number.

As a method for further enhancing such in-loop filter processing, for example, a method for extending the SAO processing method as in Non-Patent Document 1 has been proposed.

In the method of Non-Patent Document 1, as shown in FIG. 8, depending on the magnitude relationship of the pixel values between the pixel C to be processed and the four pixels (C0, C1, C2, C3) around it (up, down, left, and right). In the case where the number of pixels having a pixel value larger (smaller) than C is 3 and the number of pixels is 4, a process of adding an offset value according to the average value of those pixels is performed. A method has been proposed in which such a processing method is added to a processing method such as BO or EO so that it can be selected.

Marta Karczewicz, et al. “Peek Sample Adaptive Offset,” JVET-D0133 (http://phenix.it-sudaparis.eu/jvet/doc_end_user/documents/4_Chengdu/wg11/JVET-

However, in the conventional sample adaptive offset processing, the filter processing that is not necessarily optimal for the target image is performed by the non-linear filter processing that depends on the magnitude relationship of each pixel, and the effect of improving the image quality deterioration is larger. A method that can be filtered is desired.

Therefore, the present invention provides a coding apparatus capable of performing more optimum filtering according to the properties of the image to be coded, reducing image quality deterioration, and generating a coded stream having higher coding efficiency. An object of the present invention is to provide a decoding device, a coding method, a decoding method, a coding program, and a decoding program.

In order to solve the above problems, the first coding apparatus according to the present invention includes a pixel value correction means for correcting the pixel value of the target pixel in the processing unit region of the reconstructed image, and is obtained by the pixel value correction means. In a coding device that encodes using the obtained image and the input image, the pixel value correction means refers to (1) a linear filter coefficient storage unit that stores a linear filter coefficient and (2) a linear filter coefficient. Te, based on the pixel value of the target pixel and the pixel values of the peripheral pixels, have a linear correction processing unit that performs linear correction process, the linear correction unit, the pixel value c of the target pixel, a plurality A coding apparatus characterized by performing linear correction processing for obtaining a corrected pixel value c'according to the equation (A) with reference to the pixel value ni of the peripheral pixels of the above.
c'= Clip1 (c + ((Σiai (ni-c) + b + r) >> s)) ... (A)
Here, ai and b are linear filter coefficients, s and r are shift amounts s and rounding r = 1 << (s-1) for performing fixed minority operations in integer operations, and Clip1 is a pixel value value. Is a function that clips to a range of valid pixel values.
The second coding apparatus according to the present invention includes a pixel value correction means for correcting the pixel value of the target pixel in the processing unit region of the reconstructed image, and the image obtained by the pixel value correction means and the input image are combined. In a coding device that encodes using the pixel value correction means, the pixel value correction means refers to the linear filter coefficient storage unit that stores the linear filter coefficient and the linear filter coefficient, and the pixel value of the target pixel and the pixel value of its peripheral pixels. It has a linear correction processing unit that performs linear correction processing based on It is characterized in that linear correction processing is performed based on.

A third decoding device according to the present invention is a decoding device including a pixel value correction means that decodes an input coded stream and corrects a pixel value in a processing unit region of a reconstructed image. , (1) with reference to the linear filter coefficient included in the coded stream, on the basis of the pixel value of the target pixel and the pixel values of the peripheral pixels, have a linear correction processing unit that performs linear correction process, linear correction A feature of the processing unit is that the pixel value c of the target pixel is subjected to linear correction processing for obtaining the corrected pixel value c'according to the equation (A) with reference to the pixel values ni of a plurality of peripheral pixels. Decryptor.
c'= Clip1 (c + ((Σiai (ni-c) + b + r) >> s)) ... (A)
Here, ai and b are linear filter coefficients, s and r are shift amounts s and rounding r = 1 << (s-1) for performing fixed minority operations in integer operations, and Clip1 is a pixel value value. Is a function that clips to a range of valid pixel values.
The fourth decoding device according to the present invention is a decoding device including a pixel value correction means for decoding an input coded stream and correcting the pixel value of the target pixel in the processing unit area of the reconstructed image. The correction means has a linear correction processing unit that performs linear correction processing based on the pixel value of the target pixel and the pixel value of the peripheral pixels with reference to the linear filter coefficient included in the coded stream, and linear correction. The processing unit is characterized in that it performs linear correction processing based on a linear combination of values obtained from the difference between the pixel values of peripheral pixels and the pixel values of target pixels by a linear filter coefficient.

The fifth coding method according to the present invention includes a pixel value correction means for correcting the pixel value of the target pixel in the processing unit region of the reconstructed image, and the image obtained by the pixel value correction means and the input image are combined. In the coding method of encoding using the pixel value correction means, (1) the linear filter coefficient is stored, and (2) the linear filter coefficient is referred to, and the pixel value of the target pixel and the pixel value of the peripheral pixels thereof are referred to. based on the bets, linear correction process rows stomach, the pixel value correction unit, the pixel value c of the target pixel with reference to pixel values ni of a plurality of surrounding pixels, according to equation (a), the corrected A coding method characterized by performing a linear correction process for obtaining the pixel value c'of.
c'= Clip1 (c + ((Σiai (ni-c) + b + r) >> s)) ... (A)
Here, ai and b are linear filter coefficients, s and r are shift amounts s and rounding r = 1 << (s-1) for performing fixed minority operations in integer operations, and Clip1 is a pixel value value. Is a function that clips to a range of valid pixel values.

The sixth decoding method according to the present invention is a decoding method including a pixel value correction means for decoding an input coded stream and correcting a pixel value in a processing unit area of a reconstructed image. refers to the linear filter coefficient included in the coded stream, on the basis of the pixel value of the target pixel and the pixel values of the peripheral pixels, have row linear correction processing, the pixel value correcting means, the target pixel pixel A decoding method characterized in that a linear correction process for obtaining a corrected pixel value c'according to the equation (A) is performed with reference to the pixel values ni of a plurality of peripheral pixels with respect to the value c.
c'= Clip1 (c + ((Σiai (ni-c) + b + r) >> s)) ... (A)
Here, ai and b are linear filter coefficients, s and r are shift amounts s and rounding r = 1 << (s-1) for performing fixed minority operations in integer operations, and Clip1 is a pixel value value. Is a function that clips to a range of valid pixel values.

According to the present invention, in the sample adaptive offset processing of the video coding technology, in addition to the non-linear filter processing that depends on the magnitude relationship with the peripheral pixels as in the prior art, the linear filter processing using the peripheral pixel values. Is also selectable, and more optimal filter processing can be performed according to the properties of the image to be encoded, so that deterioration of image quality can be reduced and a coded stream with higher coding efficiency can be generated. ..

It is a block diagram which shows the structure of the image coding apparatus which concerns on 1st Embodiment. It is a block diagram which shows the structure of the image decoding apparatus which concerns on 1st Embodiment. It is a block diagram which shows the structure of the video coding apparatus of the prior art. It is a block diagram which shows the structure of the video decoding apparatus of the prior art. It is a block diagram which shows the sample adaptive offset process in the conventional HEVC. It is a figure which shows the outline of the band offset of the conventional HEVC. It is a figure which shows the outline of the edge offset of the conventional HEVC. It is explanatory drawing which shows the method which extended the processing method of the conventional sample adaptive offset. It is a block diagram which shows the structure of the sample adaptation offset processing part which concerns on 1st Embodiment. It is a figure which shows the reference pattern of the peripheral pixel of the linear filter which concerns on 1st Embodiment. It is a block diagram which shows the structure of the sample adaptation offset processing part 20 which concerns on 2nd Embodiment.

(A) First Embodiment In the following, the first embodiment of the coding device, the decoding device, the coding method, the decoding method, the coding program and the decoding program according to the present invention will be described in detail with reference to the drawings. do.

(A-1) Configuration of First Embodiment (A-1-1) Detailed Configuration of Video Coding Device FIG. 1 is a configuration diagram showing a configuration of a video coding device according to the first embodiment. ..

In FIG. 1, the video coding apparatus 1 according to the first embodiment includes a difference processing unit 101, a conversion unit 102, a quantization unit 103, an entropy coding unit 104, an inverse quantization unit 105, an inverse conversion unit 106, and an addition. It includes a unit 107, an in-loop filter unit 108, a reference image buffer 109, an inter-prediction unit 110, an intra-prediction unit 111, a switching unit 112, and a filter coefficient determination unit 120.

The video coding device 1 according to the first embodiment may be configured as hardware such as a dedicated IC chip on which each component shown in FIG. 1 is mounted, or the CPU and a program executed by the CPU. Although it may be configured by software centering on, functionally, it can be represented by FIG.

In the first embodiment, the coding method is H.I. An example shows a case where the coding method standardized by 265 / MPEG-H HEVC is used as the keynote. However, the coding method is H. It is not limited to 265 / MPEG-H HEVC, and various coding methods can be applied as long as the reconstructed image can be encoded with a process of correcting the pixel value. The coding method is H. Not limited to 265 / MPEG-H HEVC, for example, H.M. 264 / MPEG-4 AVC standardization technology and H.A. It can also be applied to an extended coding method based on the standardization technology of 265 / MPEG-H HEVC.

The video coding device 1 encodes the input coded image (input video) for each predetermined unit area such as a coding unit, and outputs a coded stream.

The coded image (input video) input to the video coding apparatus 1 is divided into predetermined processing unit areas such as a coding unit by, for example, a screen dividing unit (not shown), and is divided into a difference processing unit 101. Given.

The difference processing unit 101 includes an input image divided into a predetermined processing unit area in order to obtain a prediction residual signal, and a prediction image from the inter prediction unit 110 or the intra prediction unit 111 corresponding to the processing unit area. The difference is obtained, and the difference is given to the conversion unit 102 as a predicted residual signal.

The conversion unit 102 converts the input predicted residual signal into a conversion coefficient. For example, DCT (discrete cosine transform), DST (discrete cosine transform), and the like can be applied to the conversion unit 102.

The quantization unit 103 quantizes the conversion coefficient converted by the conversion unit 102. The quantization unit 103 outputs the obtained quantized conversion coefficient to the entropy coding unit 104 and the inverse quantization unit 105.

The entropy coding unit 104 entropy-codes the quantized conversion coefficient or the like from the quantized unit 103 in order to compress the bias of the appearance probability of the code, and outputs a coded video stream. Further, the entropy coding unit 104 multiplexes and outputs the filter coefficient used in the sample adaptive offset processing unit 10 determined by the filter coefficient determining unit 120 described later in the coded video stream.

The inverse quantization unit 105 dequantizes the quantized conversion coefficient from the quantization unit 103 in order to restore the residual signal (residual image) from the coded signal.

The inverse transformation unit 106 reverse-converts the signal inversely quantized by the inverse quantization unit 105 to restore the residual signal (residual image) and gives it to the addition unit 107.

The addition unit 107 adds the predicted image from the inter prediction unit 110 or the intra prediction unit 111 to the restored residual signal from the inverse conversion unit 106 via the switching unit 112, and is decoded again on the decoding side. The constituent image is obtained. The addition unit 107 gives the reconstructed image to the in-loop filter unit 108.

The in-loop filter unit 108 filters the reconstructed image from the addition unit 107 in order to reduce the coding distortion (for example, block distortion, ringing distortion, etc.) caused by the quantization process in the coding loop.

Further, the in-loop filter unit 108 has a sample adaptive offset processing unit 10. The detailed configuration of the sample adaptive offset processing unit 10 will be described later. In addition to the sample adaptive offset processing unit 10, the in-loop filter unit 108 may use a filter such as the deblocking filter used in HEVC in combination.

The reference image buffer 109 holds the image output from the in-loop filter unit 108 as a reference image. The output image from the in-loop filter unit 108 serves as a reference image for motion compensation of the inter-prediction at the time of coding the subsequent input image.

The inter-prediction unit 110 acquires an image held in the reference image buffer 109 as a reference image and performs motion compensation prediction.

The intra prediction unit 111 performs intra prediction using the reconstructed image output from the addition unit 107.

The switching unit 112 switches the output of the inter prediction unit 110 or the intra prediction unit 111 according to the coding mode (intra mode or inter mode).

The filter coefficient determining unit 120 determines the filter coefficient used in the sample adaptive offset processing unit 10 of the in-loop filter unit 108. Further, the filter coefficient determining unit 120 gives the determined filter coefficient to the entropy coding unit 104 in order to notify the video decoding apparatus side.

(A-1-2) Detailed Configuration of Video Decoding Device FIG. 2 is a configuration diagram showing a configuration of the video decoding device 2 according to the first embodiment.

In FIG. 2, the video decoding apparatus 2 according to the first embodiment has an entropy decoding unit 204, an inverse quantization unit 205, an inverse conversion unit 206, an addition unit 207, an in-loop filter unit 208, a reference image buffer 209, and an inter-prediction. It has a unit 210, an intra prediction unit 211, and a switching unit 212.

The video decoding device 2 according to the first embodiment may be configured as hardware such as a dedicated IC chip on which each component shown in FIG. 2 is mounted, or a CPU and a program executed by the CPU may be configured. It may be configured as software as the center, but functionally, it can be represented by FIG.

The video decoding device 2 decodes the input encoded stream and outputs the decoded video.

The entropy decoding unit 204 entropy-decodes the input coded stream to acquire conversion coefficients, coding mode information, motion vector information, and the like. Further, the entropy decoding unit 204 extracts the filter coefficient multiplexed on the coded stream and gives it to the in-loop filter unit 208.

The dequantization unit 205 dequantizes the conversion coefficient from the entropy decoding unit 204.

The inverse conversion unit 206 reverse-converts the signal inversely quantized by the inverse quantization unit 205, restores the residual signal (residual image), and gives it to the addition unit 207.

The inter-prediction unit 210 acquires an image held in the reference image buffer 209 as a reference image and performs motion compensation prediction.

The intra prediction unit 211 uses the reconstructed image output from the addition unit 207 to perform intra prediction from the reconstructed pixels and the like in the screen.

The switching unit 212 switches the output of the inter prediction unit 210 or the intra prediction unit 211 according to the coding mode (intra mode or inter mode) in order to switch the prediction image.

The addition unit 207 adds a prediction image from the inter prediction unit 210 or the intra prediction unit 211 to the restored residual signal from the inverse conversion unit 206 via the switching unit 212 to generate a reconstructed image. Is. The addition unit 127 gives the reconstructed image to the in-loop filter unit 208.

The in-loop filter unit 208 filters the reconstructed image from the addition unit 207 in order to reduce the coding distortion (for example, block distortion, ringing distortion, etc.) caused by the quantization processing in the coding loop.

The decoded image obtained by the in-loop filter unit 208 is output as a decoding process result, and is held in the reference image buffer 209 as a reference image for subsequent inter-prediction.

Further, the in-loop filter unit 208 has a sample adaptive offset processing unit 10.

The difference from the conventional video decoding apparatus is that in the in-loop filter unit 208, the sample adaptive offset processing unit 10 has the sample adaptive offset processing unit 10, and the information of the filter coefficient used in the sample adaptive offset processing unit 10 is included in the coded stream to be input. It has been. The sample adaptive offset processing unit 10 performs sample adaptive offset processing using the filter coefficient obtained by entropy decoding.

Similar to the video coding device 1, the in-loop filter unit 208 may use a filter such as the deblocking filter used in HEVC in combination with the sample adaptive offset processing unit 10.

(A-1-3) Detailed Configuration of Sample Adaptive Offset Processing Unit 10 FIG. 9 is a configuration diagram showing a configuration of the sample adaptive offset processing unit 10 according to the first embodiment.

In the sample adaptation offset processing unit 10 according to the first embodiment, in addition to the non-linear processing similar to the sample adaptation offset processing of the prior art as shown in FIG. 5, the sample adaptation of the linear filter processing type as shown in FIG. 9 Offset processing (hereinafter, also referred to as “LO”) is provided.

As for the selection between the conventional non-linear processing and the linear processing of the first embodiment, as will be described later, for example, the filter result depending on the type of SAO is evaluated for an error (that is, distortion) with respect to the input image, and the image quality is improved. A method such as selecting a SAO type with less deterioration can be applied. In the following, linear processing will be mainly described with reference to FIG.

In FIG. 9, the sample adaptive offset processing unit 10 includes a linear filter processing unit 11 and a linear filter coefficient storage unit 12. As described above, the sample adaptive offset processing unit 10 shown in FIG. 9 is included in the in-loop filter unit 108 of the video coding device 1 and the in-loop filter unit 208 of the video decoding device 2.

The linear filter coefficient storage unit 12 stores the linear filter coefficient. On the video coding side, the linear filter coefficient storage unit 12 stores the filter coefficient determined by the filter coefficient determination unit 120 in FIG. 1. On the video decoding side, the linear filter coefficient storage unit 12 stores the filter coefficient obtained by the entropy decoding unit 204 of FIG.

The linear filter processing unit 11 inputs the processing target pixel and the pixels around it for each pixel in the area to be processed, refers to the filter coefficient of the filter coefficient storage unit, and performs the linear filter processing described later. Then, the pixel value of the filter result for the target pixel is output.

(A-2) Operation of First Embodiment Next, the operation of the sample adaptive offset processing according to the first embodiment will be described in detail with reference to the drawings.

As shown in FIG. 9, the sample adaptive offset processing of the first embodiment refers to peripheral pixel values, not the offset value addition processing obtained by table lookup as in the conventional sample adaptive processing (SAO). Filter processing is performed using the linear processing. Instead of the offset table of the prior art, the filter coefficient used in the linear filtering process can be freely set.

FIG. 10 is a diagram showing a reference pattern of peripheral pixels of the linear filter according to the embodiment.

For example, as shown in FIG. 10A, the pixels of the filter processing result are obtained by referring to the _{four pixels n 0} , n ₁ , n ₂ , and n ₃ adjacent to the pixel c to be processed on the top, bottom, left, and right. Find the value c'as in the following equation.
c '= Clip1 (c + ( (Σ i a i (n i -c) + b + r) >> s)) ... (3)

Here, _ai and b are filter coefficients, which are supplied from the linear filter coefficient storage unit 12. The subscript i is the position i = 0 to 3 of the peripheral pixel. s and r are a shift amount s and rounding r = 1 << (s-1) for performing a fixed minority operation in an integer operation.

Clip1 is a function that clips the pixel value to a valid pixel value range. For example, when an 8-bit image is a processing target, it is in the range of 0 to 255, and in the case of a 10-bit image, it is 0 to 1023. Clip to the range of. Note that clipping processing may be omitted depending on the configuration such as further filtering the processing result of the sample adaptive offset.

In the equation (3), the constant term b is provided, but the constant term b may not be used (that is, b = 0) and _{only the filter coefficient ai} may be used. Further, the presence / absence of the constant term b may be set so that signaling can be performed by, for example, higher-level syntax, and the coding of the constant term b may be omitted when there is no constant term b. Further, the shift amount s representing the fixed minority accuracy of the filter coefficient may also be a predetermined fixed value, and the value of the shift amount s is set so that the value of the shift amount s can be signaled by the upper syntax or the like. You may be able to do it.

Incidentally, and when the peripheral pixel n _i comprises pixels outside the screen, the operation when the peripheral pixel n _i comprises a screen dividing border pixels outside like slices or tiles, such as the other edge offset (EO) Similar to the SAO of the type that refers to the peripheral pixels, the target pixel is used as the filter result pixel as it is without performing the filter processing according to the flag of the upper syntax (that is, the sample adaptive offset processing is performed on the target pixel). (Not applicable) It may be configured. Further, in this case, the values of the pixels that cannot be referred to (not referenced) _{are regarded as ni} = c and processed (or _{the term of ai} (ni _i −c) is ignored), and the pixels around the boundary are also sampled. It may be configured to be the target of adaptive offset processing. That is, when referring to the outside of the screen or the outside of the screen division area where the reference is prohibited depending on the setting among the plurality of peripheral pixels, the target pixel may be excluded from the linear correction processing. Further, when referring to the outside of the screen or the outside of the screen division area where the reference is prohibited depending on the setting among the plurality of peripheral pixels, the linear correction process in which the correction term for the peripheral pixels not referred to is omitted may be executed. In this way, both operation modes may be provided by signaling whether or not the peripheral pixels are to be processed, or either one may be selected.

In the video coding apparatus 1 (see FIG. 1) using the sample adaptive offset processing unit 10 of the first embodiment, the filter coefficient determining unit 120 designs the optimum filter coefficient and determines the optimum filter coefficient.

Here, the design process of the filter coefficient in the filter coefficient determination unit 120 will be described.

And each target pixel c _j in the area using the same filter coefficients, the pixel value of the input image at the position of the result c _{j 'target} pixel by a linear filter of Formula (3) with respect to c _j and C _j.

The filter coefficient determining unit 120 designs the _{filter coefficients a i} and b so that the sum of the errors with respect to the pixel value C _{j of the input image of the pixel value c j'after the filter processing is minimized. That is, the filter coefficients ai} and b that minimize the sum of squares E of the error as in the following equation (4) are obtained by using the least squares method. The sum of squares E of the error is performed by collecting the pixels in the region where the same filter coefficient will be used.
E = Σ (c _j'- C _j ) ² ... (4)

_{When finding the coefficients ai} and b that minimize the sum of squares E of the error as in equation (4), _{the equations ai} and b such that the equation obtained by partially differentiating E with respect to _ai and b becomes 0 are the following equations. It is obtained by solving simultaneous equations such as (5).
∂E / ∂a _i = 0
∂E / ∂b = 0… (5)

Various methods can be applied to select whether to use SAO using the linear filter optimally designed in this way or another SAO type such as band offset (BO) or edge offset (EO).

For example, the error (distortion) of the filter result by each SAO type with respect to the input image may be evaluated, and the SAO type with the least deterioration in image quality from the input image may be selected.

Further, for example, it may be determined by using a rate distortion optimization process. That is, when each SAO type filter is used, the distortion D of the filter processing result for the input image and the SAO type are required to be used (necessary for coding the SAO type information and the offset or filter coefficient). The trade-off of the code amount R is evaluated as the rate distortion cost J = D + λR using the Lagrange multiplier λ, and the SAO type filter that minimizes the rate distortion cost J is selected for the target region. You may try to do it.

Using the filter coefficient optimally designed as described above, sample adaptive offset processing is performed to perform coding processing.

The filter coefficient obtained by this is given to the in-loop filter unit 108, the filter coefficient is stored in the linear filter coefficient storage unit 12 of the sample adaptive offset processing unit 10, and the filter coefficient is given to the entropy coding unit 104. , According to the coding method, the filter coefficients are multiplexed into the coded stream.

Here, as a method of multiplexing the filter coefficients, for example, the processing type of SAO (for example, an area of up to 64 × 64 pixels in HEVC) for each processing unit area like SAO of HEVC. , Band offset (BO), edge offset (EO), linear filter type offset (LO) according to this embodiment), and when the linear filter type SAO is selected as the processing type, the target processing unit area. The filter coefficients _ai and b used in the above may be signaled, or may be encoded by a higher-level syntax such as a parameter set such as a slice header or a picture parameter set. Further, the four coefficient filter coefficients _ai may be configured by diverting the same entropy coding method as for coding the four offset values of BO and EO.

In the video decoding apparatus 2 (see FIG. 2) using the sample adaptive offset processing unit 10 of the first embodiment, the entropy decoding unit 204 uses the filter coefficient used in the sample adaptive offset of the linear filter type multiplexed on the coded stream. To decrypt.

The filter coefficient obtained by decoding is given to the in-loop filter unit 208, and the filter coefficient is stored in the linear filter coefficient storage unit 12 of the sample adaptive offset processing unit 10.

The linear filter processing unit 11 performs decoding processing by performing sample adaptive offset processing using the filter coefficients stored in the linear filter coefficient storage unit 12.

(A-3) Effect of First Embodiment As described above, according to the first embodiment, the following effects can be obtained.

In the sample adaptive offset processing of the video coding technology, in addition to the non-linear filter processing that depends on the magnitude relationship with the peripheral pixels as in the conventional technology, it is possible to select a linear filter processing that uses the peripheral pixel values. By making it possible to perform more optimal filter processing according to the properties of the image to be converted, deterioration in image quality can be reduced and a coded stream with higher coding efficiency can be generated.

(B) Second Embodiment Next, a second embodiment of the coding device, the decoding device, the coding method, the decoding method, the coding program, and the decoding program according to the present invention will be described in detail with reference to the drawings. do.

(B-1) Configuration of Second Embodiment In the second embodiment, in addition to the processing described in the first embodiment, a plurality of reference patterns of peripheral pixels referenced by a linear filter are prepared and selected. Sample adaptive offset processing is performed by linear filter processing using the peripheral pixels of the reference pattern.

Since the basic configurations of the video coding device and the video decoding device according to the second embodiment are the same as or correspond to the configurations of the first embodiment, FIGS. 1 and 2 are also referred to here.

In the second embodiment, the configuration of the sample adaptive offset processing unit included in each of the in-loop filter unit 108 in FIG. 1 and the in-loop filter unit 208 in FIG. 2 is different. Therefore, the sample adaptive offset processing unit of the second embodiment is referred to as the sample adaptive offset processing unit 20, and this will be described.

FIG. 11 is a configuration diagram showing the configuration of the sample adaptive offset processing unit 20 according to the second embodiment.

In FIG. 11, the sample adaptive offset processing unit 20 according to the second embodiment includes a linear filter processing unit 11, a linear filter coefficient storage unit 12, and a peripheral pixel selection unit 23.

Since the linear filter processing unit 11 and the linear filter coefficient storage unit 12 are the same or corresponding components as those in the first embodiment, detailed description thereof will be omitted here.

The peripheral pixel selection unit 23 inputs pattern type information, selects peripheral pixels to be used in the linear filter processing according to a pattern corresponding to the pattern type information, and gives the peripheral pixels to the linear filter processing unit 11.

(B-2) Operation of the Second Embodiment Next, the operation of the sample adaptive offset processing according to the second embodiment will be described in detail with reference to the drawings.

In the second embodiment, as shown in FIGS. 10A and 10B, for example, a plurality of reference patterns of peripheral pixels such as a cross-shaped pattern and an X-shaped pattern are provided. As the pattern, not only the two patterns illustrated in FIGS. 10 (A) and 10 (B) but also other patterns may be used.

The pattern type information of which reference pattern to use is, for example, as a linear filter type as in the method of specifying a plurality of reference patterns in edge offset (EO) as SAO types such as EO0, EO1, ... Signaling is performed so that there are a plurality of LO type SAO types such as LO0, LO1, ....

In the video decoding device 2 of the second embodiment, the entropy decoding unit 204 extracts the pattern type information of the linear filter included in the decoded SAO type information, supplies the pattern type information to the peripheral pixel selection unit 23, and at the same time, The filter coefficient in the case of the linear filter type SAO is supplied to the linear filter coefficient storage unit 12.

_{The peripheral pixel selection unit 23 sets the pixels adjacent to the target pixel c to the peripheral pixels ni} (n ₀ ) according to the reference pattern of the pattern type information from the entropy decoding unit 204 (see, for example, FIGS. 10A and 10B). , N ₁ , n ₂ , n ₃ ) and supplies it to the linear filter processing unit 11.

Linear filter processing unit 11 refers to the peripheral pixel n _i from the peripheral pixel selection unit 23, using the filter coefficients of the linear filter coefficient storage unit 12, similarly to the first embodiment, the equation (3) Perform the following processing.

In the video coding apparatus 1 of the second embodiment, the filter coefficient determination unit 120 performs the filter coefficient design process as in the first embodiment for each of the plurality of available patterns.

The filter coefficient determination unit 120 designs the filter coefficients for each of the plurality of available pattern types, and uses the obtained filter coefficients for the linear filter SAO type, and other band offset (BO) and edge offset (EO). ) Etc. to select the SAO type. As described in the first embodiment, the SAO type selection method here evaluates, for example, the filter result of each SAO type and performs selection based only on image quality or selection by rate distortion optimization processing. ..

When the SAO type using the linear filter processing is selected by the selection of the SAO type, the filter coefficient determination unit 120 supplies the filter coefficient for the SAO type to the linear filter coefficient storage unit 12, and the filter coefficient determination unit 120 , The pattern type information of the selected linear filter is supplied to the peripheral pixel selection unit 23.

In the sample adaptive offset processing unit 20 of the in-loop filter unit 108, the peripheral pixel selection unit 23 sets the pixels adjacent to the target pixel c to the peripheral pixels _ni (n) according to the pattern type information of the linear filter from the filter coefficient determination unit 120. _It is selected as 0, n ₁ , n ₂ , n ₃ ) and supplied to the linear filter processing unit 11.

Linear filter processing unit 11 refers to the peripheral pixel n _i from the peripheral pixel selection unit 23, using the filter coefficients of the linear filter coefficient storage unit 12, similarly to the first embodiment, the equation (3) Perform the following processing.

Further, the entropy coding unit 104 entropy-codes the pattern type information from the filter coefficient determining unit 120 as a SAO type together with the linear filter coefficient, multiplexes the coded stream, and outputs the information.

(B-3) Effect of Second Embodiment As described above, according to the second embodiment, the following effects can be obtained.

In addition to the effect of being able to apply the optimum linear filter as in the first embodiment, it is also possible to select from a plurality of linear filter patterns, so that image quality deterioration can be further reduced and coding efficiency is further improved. Highly encoded stream can be generated.

(C) Other Embodiments The present invention is not limited to the first and second embodiments described above, and can be applied to various other coding and decoding processes.

Prediction processing and conversion quantization processing related to coding other than sample adaptation offset are not limited to the configuration as described in the above-described embodiment, but seem to be a combination of various coding tools and in-loop filtering processing. It can also be used for various coding processes.

Regarding sample adaptive offset processing, not only the combination with BO and EO similar to HEVC, but also the combination with SAO type of only a part of these and the filter of the processing type different from BO and EO, these SAO type and linear filter It may be configured so that it can be selected from the types of SAO processing.

Linear expression used in the filtering process (3) is not limited to this format may be used linear combination to become such various arithmetic expression of the pixel value of the target pixel c and the peripheral pixel n _i.

As the reference pattern, the patterns shown in FIGS. 10 (A) and 10 (B) are illustrated, but other reference patterns may be used. A pattern may be used in which the number of pixels to be referred to is not limited to 4 pixels.

Further, the present invention can be implemented as a device having the above configuration or as a program for realizing the above processing.

1 ... Video coding device, 101 ... Difference processing unit, 102 ... Conversion unit, 103 ... Quantization unit, 104 ... Entropy coding unit, 105 ... Inverse quantization unit, 106 ... Inverse conversion unit, 107 ... Addition unit, 108 ... In-loop filter unit, 109 ... Reference image buffer, 110 ... Inter prediction unit, 111 ... Intra prediction unit, 112 ... Switching unit, 120 ... Filter coefficient determination unit,
2 ... Video decoding device, 204 ... Entropy decoding unit, 205 ... Inverse quantization unit, 206 ... Inverse conversion unit, 207 ... Addition unit, 208 ... In-loop filter unit, 209 ... Reference image buffer, 210 ... Inter prediction unit, 211 … Intra prediction part, 212… Switching part,
10 and 20 ... sample adaptive offset processing unit, 11 ... linear filter processing unit, 12 ... linear filter coefficient storage unit, 23 ... peripheral pixel selection unit.

Claims (29)

In a coding device provided with a pixel value correction means for correcting the pixel value of a target pixel in the processing unit area of the reconstructed image, and encoding using the image obtained by the pixel value correction means and the input image.
The pixel value correction means
A linear filter coefficient storage unit that stores linear filter coefficients and
It has a linear correction processing unit that performs linear correction processing based on the pixel values of the target pixel and the pixel values of its peripheral pixels with reference to the linear filter coefficient.
The linear correction processing unit performs linear correction processing for obtaining the corrected pixel value c'according to the formula (A) by referring to the pixel values ni of a plurality of peripheral pixels with respect to the pixel value c of the target pixel. An encoding device, characterized in that it does.
c'= Clip1 (c + ((Σiai (ni-c) + b + r) >> s)) ... (A)
Here, ai and b are linear filter coefficients, s and r are shift amounts s and rounding r = 1 << (s-1) for performing fixed minority operations in integer operations, and Clip1 is a pixel value value. Is a function that clips to a range of valid pixel values. A filter coefficient determining means for determining a linear filter coefficient based on the above input image and the reconstructed image before correction is provided.
The coding apparatus according to claim 1, wherein the linear correction processing unit performs linear correction processing with reference to the determined linear filter coefficient. The coding apparatus according to claim 2, wherein the filter coefficient determining means obtains ai and b by using a least squares method in a process of determining a linear filter coefficient. The coding apparatus according to claim 2 or 3 , further comprising a coding means for multiplexing a linear filter coefficient determined by the filter coefficient determining means into a coded stream. The coding apparatus according to any one of claims 1 to 4 , wherein the linear correction processing unit performs linear correction processing using only ai among the linear filter coefficients. The coding apparatus according to any one of claims 1 to 5 , wherein the linear correction processing unit performs linear correction processing without executing clipping processing. When the linear correction processing unit refers to a plurality of peripheral pixels outside the screen or outside the screen division area for which reference is prohibited depending on the setting, the target pixel is excluded from the target of the linear correction processing. The encoding device according to any one of claims 1 to 6. When the linear correction processing unit refers to the outside of the screen or the outside of the screen division area where reference is prohibited depending on the setting among a plurality of peripheral pixels, the linear correction processing is executed by omitting the correction term for the peripheral pixels that are not referenced. The coding apparatus according to any one of claims 1 to 7. The coding apparatus according to any one of claims 1 to 8 , wherein the linear correction processing unit sets the value of the shift amount s to a value whose setting can be changed. The coding apparatus according to any one of claims 1 to 9 , wherein the plurality of peripheral pixels include at least four pixels adjacent to each other on the top, bottom, left, and right of the target pixel. The coding apparatus according to any one of claims 1 to 10 , wherein the plurality of peripheral pixels include at least four pixels adjacent to the upper left, upper right, lower left, and lower right of the target pixel. The pixel value correction means can also execute one or a plurality of pixel value correction processes other than the linear correction process, and select one of the linear correction process and the one or a plurality of pixel value correction processes. The coding apparatus according to any one of claims 1 to 10. The pixel value correction means trades the distortion of the image quality with respect to the input pixels of the processing results of the plurality of pixel value correction processes, the type of the pixel value correction process, and the code amount required for coding the linear filter coefficient or the correction value. The coding apparatus according to claim 12, further comprising performing rate distortion optimization processing for evaluating off to determine the type of pixel value correction processing. The pixel value correction means has a peripheral pixel selection unit that selects a plurality of peripheral pixels for the target pixel according to each pattern type information based on a plurality of reference patterns of peripheral pixels used in the linear correction process.
The coding apparatus according to any one of claims 1 to 13, wherein the linear correction processing unit performs linear correction processing based on a plurality of selected peripheral pixels and pixel values of the target pixels. .. 14. The coding according to claim 14, further comprising a coding means for multiplexing the linear filter coefficient determined by the filter coefficient determining means and / or the pattern type information of the reference pattern into a coded stream. Device. In a decoding device provided with a pixel value correction means that decodes the input coded stream and corrects the pixel value of the target pixel in the processing unit area of the reconstructed image.
The pixel value correction means
It has a linear correction processing unit that performs linear correction processing based on the pixel values of the target pixel and the pixel values of its peripheral pixels with reference to the linear filter coefficient included in the coded stream.
The linear correction processing unit performs linear correction processing for obtaining the corrected pixel value c'according to the formula (A) by referring to the pixel values ni of a plurality of peripheral pixels with respect to the pixel value c of the target pixel. A decoding device characterized by performing.
c'= Clip1 (c + ((Σiai (ni-c) + b + r) >> s)) ... (A)
Here, ai and b are linear filter coefficients, s and r are shift amounts s and rounding r = 1 << (s-1) for performing fixed minority operations in integer operations, and Clip1 is a pixel value value. Is a function that clips to a range of valid pixel values. The decoding device according to claim 16, wherein the linear correction processing unit performs linear correction processing using only ai among the linear filter coefficients. The decoding device according to claim 16 or 17, wherein the linear correction processing unit performs linear correction processing without executing clipping processing. When the linear correction processing unit refers to a plurality of peripheral pixels outside the screen or outside the screen division area for which reference is prohibited depending on the setting, the target pixel is excluded from the target of the linear correction processing. The decoding device according to any one of claims 16 to 18. When the linear correction processing unit refers to the outside of the screen or the outside of the screen division area where reference is prohibited depending on the setting among a plurality of peripheral pixels, the linear correction processing is executed by omitting the correction term for the peripheral pixels that are not referenced. The decoding device according to any one of claims 16 to 19. The decoding device according to any one of claims 16 to 20, wherein the linear correction processing unit sets the value of the shift amount s to a value whose setting can be changed. The decoding device according to any one of claims 16 to 21, wherein the plurality of peripheral pixels include at least four pixels adjacent to each other on the top, bottom, left, and right of the target pixel. The decoding device according to any one of claims 16 to 22, wherein the plurality of peripheral pixels include at least four pixels adjacent to the upper left, upper right, lower left, and lower right of the target pixel. The pixel value correction means can also execute one or a plurality of pixel value correction processes other than the linear correction process, and select one of the linear correction process and the one or a plurality of pixel value correction processes. The decoding device according to any one of claims 16 to 23. It has a peripheral pixel selection unit that selects a plurality of peripheral pixels for the target pixel according to the pattern type information of the reference pattern included in the coded stream.
The decoding device according to any one of claims 16 to 24, wherein the linear correction processing unit performs linear correction processing based on a plurality of selected peripheral pixels and pixel values of the target pixels. In a coding method provided with a pixel value correction means for correcting the pixel value of a target pixel in the processing unit area of the reconstructed image and encoding using the image obtained by the pixel value correction means and the input image.
The pixel value correction means
Memorize the linear filter coefficient and
With reference to the linear filter coefficient, linear correction processing is performed based on the pixel value of the target pixel and the pixel value of its peripheral pixels.
The pixel value correction means refers to the pixel values ni of a plurality of peripheral pixels with respect to the pixel value c of the target pixel, and performs a linear correction process for obtaining the corrected pixel value c'according to the formula (A). A coding method characterized by performing.
c'= Clip1 (c + ((Σiai (ni-c) + b + r) >> s)) ... (A)
Here, ai and b are linear filter coefficients, s and r are shift amounts s and rounding r = 1 << (s-1) for performing fixed minority operations in integer operations, and Clip1 is a pixel value value. Is a function that clips to a range of valid pixel values. In a decoding method including a pixel value correction means for decoding an input coded stream and correcting a pixel value in a processing unit area of a reconstructed image.
The pixel value correction means
Referring to the linear filter coefficients included in the coded stream, on the basis of the pixel value of the Target pixel and the pixel values of the peripheral pixels, performs linear correction process,
The pixel value correction means refers to the pixel values ni of a plurality of peripheral pixels with respect to the pixel value c of the target pixel, and performs a linear correction process for obtaining the corrected pixel value c'according to the formula (A). A decoding method characterized by performing.
c'= Clip1 (c + ((Σiai (ni-c) + b + r) >> s)) ... (A)
Here, ai and b are linear filter coefficients, s and r are shift amounts s and rounding r = 1 << (s-1) for performing fixed minority operations in integer operations, and Clip1 is a pixel value value. Is a function that clips to a range of valid pixel values. Computer,
In a coding program that functions as a pixel value correction means that corrects the pixel value of the target pixel in the processing unit area of the reconstructed image and encodes the image obtained by the pixel value correction means and the input image. ,
A linear filter coefficient storage unit that stores linear filter coefficients and
With reference to the linear filter coefficient, it functions as a linear correction processing unit that performs linear correction processing based on the pixel values of the target pixel and the pixel values of its peripheral pixels.
The linear correction processing unit performs linear correction processing for obtaining the corrected pixel value c'according to the formula (A) by referring to the pixel values ni of a plurality of peripheral pixels with respect to the pixel value c of the target pixel. An encoding program characterized by performing.
c'= Clip1 (c + ((Σiai (ni-c) + b + r) >> s)) ... (A)
Here, ai and b are linear filter coefficients, s and r are shift amounts s and rounding r = 1 << (s-1) for performing fixed minority operations in integer operations, and Clip1 is a pixel value value. Is a function that clips to a range of valid pixel values. Computer,
In a decoding program provided with means for decoding an input encoded stream and functioning as a pixel value correction for correcting a pixel value in a processing unit area of a reconstructed image.
Referring to the linear filter coefficients included in the coded stream, on the basis of the pixel value of the Target pixel and the pixel value of the surrounding pixel, to function as a linear correction processing unit that performs linear correction process,
The linear correction processing unit performs linear correction processing for obtaining the corrected pixel value c'according to the formula (A) by referring to the pixel values ni of a plurality of peripheral pixels with respect to the pixel value c of the target pixel. A decoding program characterized by performing.
c'= Clip1 (c + ((Σiai (ni-c) + b + r) >> s)) ... (A)
Here, ai and b are linear filter coefficients, s and r are shift amounts s and rounding r = 1 << (s-1) for performing fixed minority operations in integer operations, and Clip1 is a pixel value value. Is a function that clips to a range of valid pixel values.

Images