JP7229682B2 - LOOP FILTER CONTROL DEVICE, IMAGE CODING DEVICE, IMAGE DECODING DEVICE, AND PROGRAM - Google Patents

Description

The present invention relates to image processing technology, and more particularly to a loop filter control device, image coding device, image decoding device, and program.

In recent years, the number of cameras capable of easily acquiring depth information together with a two-dimensional image has been increasing. A typical example is Microsoft's Kinect, which uses an infrared sensor. It is expected that it will be applied.

Also, as a conventional two-dimensional encoding method, AVC/H. 264 and HEVC/H. H.265, in which a deblocking filter is implemented to reduce coding distortion (block distortion) caused by quantization processing (see, for example, Non-Patent Document 1). In addition, a sample adaptive offset (SAO) process is performed after the application of the deblocking filter process. These filters are called loop filters, and by using these filters, it is possible to realize visual image quality improvement and to prevent propagation of image quality degradation to predicted images.

Further, Patent Literature 1 describes a loop filter control device that sets the strength of a deblocking filter using depth information added to a two-dimensional moving image. Specifically, such a loop filter control device uses depth information contained in two adjacent blocks to determine whether the two adjacent blocks belong to the same object (subject). Then, when it is determined that the blocks belong to the same object, the filter strength of the deblocking filter for the block boundary between the two adjacent blocks is set to be increased.

Further, Patent Literature 2 describes an image encoding device that makes block noise generated when encoding a viewpoint image inconspicuous. Specifically, for the depth image, the distribution of pixel values is analyzed according to the encoding unit when the viewpoint image is encoded, and based on the analysis results, the deblocking filter applied when encoding the viewpoint image Determine strength.

Recommendation ITU-T H. 265, (12/2016), "High efficiency video coding", International Telecommunications Union

JP 2014-143515 A JP 2013-110532 A

However, in the loop filter control device described in Patent Document 1, even if two adjacent blocks belong to the same object and the block boundary between the two adjacent blocks is an edge region, the block Setting the filter strength of the deblocking filter to the boundary too high. As a result, the edge region becomes blurred, resulting in degradation of image quality.

As described above, the loop filter control device described in Patent Document 1 has room for improvement in terms of increasing the edge region determination accuracy.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a loop filter control device, an image coding device, an image decoding device, and a program, which are capable of increasing the accuracy of edge region determination and appropriately controlling loop filter processing.

A loop filter control device according to a first feature is a loop filter control device for controlling a loop filter of an image processing device that processes a two-dimensional image to which depth information is added in units of blocks, wherein the depth information a first edge determination unit for determining whether or not the block boundary in the two-dimensional image is an edge region based on the above; and determining whether the block boundary is an edge region based on the two-dimensional image. a second edge determination unit for determining; and a control unit for controlling loop filtering of the two-dimensional image based on the determination result of the first edge determination unit and the determination result of the second edge determination unit. is the gist.

Here, the edge region refers to a region in a two-dimensional image in which pixel values change significantly (that is, a region in which pixel values change significantly in a two-dimensional image), and has a certain length. For example, an edge region is a boundary between an object and a background in a two-dimensional image or a boundary between objects in a two-dimensional image, but an edge region can also exist within the same object.

According to the loop filter control device according to the first feature, by performing edge determination based on depth information, for example, an edge region between an object and a background can be correctly determined. In addition, by performing edge determination based on a two-dimensional image, it is possible to accurately determine edge areas within the same object and edge areas between objects having the same depth, so it is possible to improve the determination accuracy of edge areas. . Therefore, the loop filter control device according to the first feature can control loop filter processing more appropriately.

In the loop filter control device according to the first feature, the loop filter processing includes deblocking filter processing, the control unit is determined not to be an edge region by the first edge determination unit, and the second A deblocking filter control unit may be provided that applies the deblocking filter processing only to block boundaries determined by the edge determination unit not to be edge regions.

According to this loop filter control device, it is possible to appropriately limit the block boundaries to which the deblocking filtering process is applied, so that the amount of computation for the deblocking filtering process can be reduced.

In the loop filter control device according to the first feature, the second edge determination unit determines that the block boundary is an edge region only for block boundaries determined by the first edge determination unit not to be an edge region. It may be determined whether

According to this loop filter control device, the second edge determination unit does not have to perform edge determination for all block boundaries, but only for some block boundaries. can be reduced.

In the loop filter control device according to the first feature, the loop filter processing includes sample adaptive offset processing, and the control unit performs the sample adaptive offset processing based on at least the determination result of the first edge determination unit. A mode selector may be provided for selecting a mode from a plurality of modes including edge offset and band offset.

According to such a loop filter control device, the mode of sample adaptive offset processing can be appropriately selected.

An image coding device according to a second feature is summarized in including the loop filter control device according to the first feature.

An image decoding device according to a third feature is summarized in including the loop filter control device according to the first feature.

The gist of a program according to a fourth aspect is to cause a computer to function as the loop filter control device according to the first aspect.

Advantageous Effects of Invention According to the present invention, it is possible to provide a loop filter control device, an image encoding device, an image decoding device, and a program that can appropriately control loop filter processing by increasing the accuracy of edge region determination.

1 is a diagram showing the configuration of an image encoding device according to an embodiment; FIG. It is a figure which shows the structure of the image decoding apparatus which concerns on embodiment. It is a figure which shows the block boundary which performs deblocking filter processing which concerns on embodiment. It is a figure showing an example of edge offset processing concerning an embodiment. It is a figure which shows the structure of the loop filter control part which concerns on embodiment. It is a figure which shows the operation example of the loop filter control part which concerns on embodiment.

An image encoding device and an image decoding device according to an embodiment will be described with reference to the drawings. An image encoding device and an image decoding device according to the present embodiment encode and decode two-dimensional moving images represented by MPEG. In this embodiment, depth information is added to such a two-dimensional moving image. Each of the image encoding device and the image decoding device according to the present embodiment corresponds to an image processing device that processes a two-dimensional image to which depth information is added in units of blocks.

In addition, in the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

<1. Configuration of image encoding device>
FIG. 1 is a diagram showing the configuration of an image encoding device 1 according to this embodiment.

As shown in FIG. 1, the image coding apparatus 1 includes a block division unit 100, a subtraction unit 110, a transform/quantization unit 120, an entropy coding unit 130, an inverse quantization/inverse transform unit 140, It comprises a synthesis unit 150 , a loop filter 160 , a loop filter control unit 170 , a memory 180 and a prediction unit 190 .

The block division unit 100 divides an input image in units of frames (or pictures) constituting a moving image into small block-shaped regions, and outputs the blocks obtained by the division to the subtraction unit 110 . The block size is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of the block is not limited to square, and may be rectangular. A block is a unit for encoding by the image encoding device 1 and a unit for decoding by the image decoding device 2 . HEVC/H. In the case of H.265, the block division unit 100 divides an input image into blocks called coding tree units (CTUs), and then divides each CTU into blocks called coding units (CUs).

The subtraction unit 110 calculates a prediction residual indicating a pixel-by-pixel difference between the block input from the block division unit 100 and a prediction image (prediction block) obtained by predicting the block by the prediction unit 190 . . Specifically, the subtraction unit 110 calculates a prediction residual by subtracting each pixel value of the predicted image from each pixel value of the block, and outputs the calculated prediction residual to the transformation/quantization unit 120 .

The transform/quantization unit 120 performs orthogonal transform processing and quantization processing on a block-by-block basis. The transform/quantization unit 120 includes a transform unit 121 and a quantization unit 122 .

The transform unit 121 performs orthogonal transform on the prediction residual input from the subtraction unit 110 to calculate transform coefficients, and outputs the calculated transform coefficients to the quantization unit 122 . Orthogonal transformation refers to, for example, discrete cosine transform (DCT), discrete sine transform (DST), Karhunen-Loeve transform (KLT), and the like.

The quantization unit 122 quantizes the transform coefficients input from the transform unit 121 using a quantization parameter (Qp) and a quantization matrix to generate quantized transform coefficients. A quantization parameter (Qp) is a parameter commonly applied to each transform coefficient in a block, and is a parameter that determines coarseness of quantization. A quantization matrix is a matrix having, as elements, quantization values for quantizing each transform coefficient. The quantization unit 122 outputs the generated quantized transform coefficient information and quantization control information to the entropy coding unit 130 and the inverse quantization/inverse transform unit 140 .

The entropy coding unit 130 performs entropy coding on the quantized transform coefficients input from the quantization unit 122, performs data compression to generate coded data (bitstream), and converts the coded data into image coding. output to the outside of the conversion device 1. Huffman code, CABAC (Context-based Adaptive Binary Arithmetic Coding), or the like can be used for entropy coding. Note that the entropy coding unit 130 receives control information about filtering from the loop filter 160 and receives control information about prediction from the prediction unit 190 . The entropy coding unit 130 also entropy codes these control information.

The inverse quantization/inverse transform unit 140 performs inverse quantization processing and inverse orthogonal transform processing on a block-by-block basis. The inverse quantization/inverse transform unit 140 includes an inverse quantization unit 141 and an inverse transform unit 142 .

The inverse quantization unit 141 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 . Specifically, the inverse quantization unit 141 restores the transform coefficients by inversely quantizing the quantized transform coefficients input from the quantization unit 122 using the quantization parameter (Qp) and the quantization matrix. and outputs the restored transform coefficients to the inverse transform unit 142 .

The inverse transform unit 142 performs inverse orthogonal transform processing corresponding to the orthogonal transform processing performed by the transform unit 121 . For example, when the transform unit 121 performs discrete cosine transform, the inverse transform unit 142 performs inverse discrete cosine transform. The inverse transform unit 142 performs inverse orthogonal transform on the transform coefficients input from the inverse quantization unit 141 to restore prediction residuals, and outputs the restored prediction residuals, which are the restored prediction residuals, to the synthesizing unit 150. do.

The synthesizing unit 150 synthesizes the restored prediction residual input from the inverse transform unit 142 with the predicted image input from the predicting unit 190 on a pixel-by-pixel basis. The synthesizing unit 150 adds each pixel value of the restored prediction residual and each pixel value of the predicted image to reconstruct (decode) the block, and outputs the reconstructed image for each block to the loop filter 160 . Such reconstructed images are sometimes called decoded images (or local decoded images).

The loop filter 160 performs loop filtering, which is filtering as post-processing, on the reconstructed image input from the synthesizing unit 150 , and outputs the reconstructed image after the filtering to the memory 180 . Loop filter 160 also outputs information about loop filter processing to entropy coding section 130 .

The loop filter 160 includes a deblocking filter unit 161 that performs deblocking filtering and an SAO unit 162 that performs sample adaptive offset (SAO) processing. Deblocking filter processing is processing for reducing signal degradation caused by processing in units of blocks, and is processing for smoothing signal gaps at boundaries between adjacent blocks. On the other hand, the SAO process is an image quality improvement filter process employed in HEVC, for example, and divides each pixel into categories according to the relative relationship between the pixels in the block and the adjacent pixels, and improves the image quality for each category. In this process, an offset value is calculated for each pixel belonging to the same category, and the offset value is assigned to each pixel belonging to the same category. A specific example of SAO processing will be described later.

Loop filter control section 170 corresponds to a loop filter control device that controls loop filter 160 . Details of the loop filter control unit 170 will be described later.

Memory 180 stores the reconstructed image input from loop filter 160 . The memory 180 stores reconstructed images in units of frames. The memory 180 outputs the stored reconstructed image to the prediction section 190 .

The prediction unit 190 performs prediction on a block-by-block basis. The prediction unit 190 includes an intra prediction unit 191 , an inter prediction unit 192 and a switching unit 193 .

The intra prediction unit 191 generates an intra prediction image by referring to the decoded reference pixels adjacent to the block to be predicted in the reconstructed image stored in the memory 180, and sends the generated intra prediction image to the switching unit 193. Output. In addition, the intra prediction unit 191 selects an optimal intra prediction mode to be applied to the target block from a plurality of predefined intra prediction modes, and performs intra prediction using the selected intra prediction mode.

The inter prediction unit 192 uses the reconstructed image stored in the memory 180 as a reference image, calculates a motion vector by a technique such as block matching, and generates an inter prediction image by inter prediction for predicting a block to be predicted. , and outputs the generated inter-predicted image to the switching unit 193 . The inter prediction unit 192 selects an optimum inter prediction method from inter prediction using a plurality of reference images (typically, bi-prediction) and inter prediction using one reference image (unidirectional prediction), Perform inter prediction using the selected inter prediction method.

The switching unit 193 switches between the intra prediction image input from the intra prediction unit 191 and the inter prediction image input from the inter prediction unit 192 , and outputs either prediction image to the subtraction unit 110 and the synthesis unit 150 .

<2. Configuration of image decoding device>
FIG. 2 is a diagram showing the configuration of the image decoding device 2 according to this embodiment.

As shown in FIG. 2, the image decoding device 2 includes an entropy code decoding unit 200, an inverse quantization/inverse transform unit 210, a synthesizing unit 220, a loop filter 230, a loop filter control unit 240, and a memory 250. , and a prediction unit 260 .

The entropy code decoding unit 200 decodes the encoded data generated by the image encoding device 1 and outputs the quantized transform coefficients to the inverse quantization/inverse transform unit 210 . The entropy code decoding unit 200 also acquires control information related to prediction (intra prediction and inter prediction) and outputs the acquired control information to the prediction unit 260 . Further, entropy code decoding section 200 acquires control information relating to loop filter processing, and outputs the acquired control information to loop filter 230 .

The inverse quantization/inverse transform unit 210 performs inverse quantization processing and inverse orthogonal transform processing on a block-by-block basis. The inverse quantization/inverse transform unit 210 includes an inverse quantization unit 211 and an inverse transform unit 212 .

The inverse quantization unit 211 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 of the image encoding device 1 . The inverse quantization unit 211 restores and restores the transform coefficients by inversely quantizing the quantized transform coefficients input from the entropy code decoding unit 200 using the quantization parameter (Qp) and the quantization matrix. The transform coefficients are output to the inverse transform unit 212 .

The inverse transform unit 212 performs inverse orthogonal transform processing corresponding to the orthogonal transform processing performed by the transform unit 121 of the image encoding device 1 . The inverse transform unit 212 performs inverse orthogonal transform on the transform coefficients input from the inverse quantization unit 211 to restore prediction residuals, and outputs the restored prediction residuals (restored prediction residuals) to the synthesizing unit 220 . do.

The synthesizing unit 220 reconstructs (decodes) the original block by synthesizing the prediction residual input from the inverse transform unit 212 and the predicted image input from the predicting unit 260 on a pixel-by-pixel basis. is output to the loop filter 230 .

The loop filter 230 performs the same loop filtering process as the loop filter 160 of the image encoding device 1 performs on the reconstructed image input from the synthesizing unit 220 based on the control information input from the entropy code decoding unit 200. loop filter processing is performed, and the reconstructed image after the filter processing is output to the memory 250 . The loop filter 230 includes a deblocking filter unit 231 that performs deblocking filter processing and an SAO unit 232 that performs SAO processing.

Loop filter control section 240 corresponds to a loop filter control device that controls loop filter 230 . Details of the loop filter control unit 240 will be described later.

Memory 250 stores the reconstructed image input from loop filter 230 . The memory 250 stores reconstructed images in units of frames. The memory 250 outputs the reconstructed image (decoded image) in units of frames to the outside of the image decoding device 2 .

The prediction unit 260 performs prediction on a block-by-block basis. The prediction unit 260 includes an intra prediction unit 261 , an inter prediction unit 262 and a switching unit 263 .

The intra prediction unit 261 refers to the reconstructed image stored in the memory 250, performs intra prediction according to the control information input from the entropy coding/decoding unit 200 to generate an intra prediction image, and generates the intra prediction image. Output to the switching unit 263 .

The inter prediction unit 262 performs inter prediction for predicting a prediction target block using the reconstructed image stored in the memory 250 as a reference image. The inter prediction unit 262 performs inter prediction according to the control information (motion vector information, etc.) input from the entropy code decoding unit 200 to generate an inter prediction image, and outputs the generated inter prediction image to the switching unit 263 .

The switching unit 263 switches between the intra prediction image input from the intra prediction unit 261 and the inter prediction image input from the inter prediction unit 262 and outputs one of the prediction images to the synthesis unit 220 .

<3. Example of deblocking filter processing>
As an example of deblocking filtering, HEVC/H. Deblocking filtering in H.265 will now be described. Here, the deblocking filter unit 161 of the image encoding device 1 will be described as an example, but the deblocking filter unit 231 of the image decoding device 2 also performs the same operation.

FIG. 3 is a diagram showing block boundaries on which deblocking filtering is performed. As shown in FIG. 3, the block size on which the deblocking filtering unit 161 performs deblocking filtering is 8×8 pixels. The deblocking filter unit 161 first obtains a boundary strength Bs (Boundary Strength) value indicating the strength of the smoothing process for each block. The Bs value is either 0, 1, or 2.

Table 1 shows how to determine the Bs value.

As shown in FIG. 3 and Table 1, the Bs value is 2 when block P or Q is a block for intra prediction. If blocks P and Q are inter-prediction blocks and satisfy at least one of the following conditions, the Bs value is set to 1; otherwise, the Bs value is set to 0.
- Block P or Q contains significant (non-zero) orthogonal transform coefficients and is the boundary of transform unit TU.
- The absolute value of the difference between the motion vectors of blocks P and Q is 1 pixel or more.
• The number of motion vectors or reference images for blocks P and Q are different.

The deblocking filter unit 161 does not perform filtering when the Bs value is 0. The vertical block boundary shown in FIG. 3 will be described below as an example.

Also, when performing deblocking filtering, the deblocking filter unit 161 applies a strong filter when all of the following expressions (1) to (6) are satisfied, and applies a weak filter in other cases. .

Here, the values of the thresholds β and _tC change according to the average value _Qav of the quantization parameters of the adjacent blocks P and Q. FIG.

Note that when the deblocking filter unit 161 performs deblocking filtering, the amount of processing is very large.

<4. Example of SAO processing>
As an example of SAO processing, HEVC/H. SAO processing in H.265 will be described. Here, the SAO unit 162 of the image encoding device 1 will be described as an example, but the SAO unit 232 of the image decoding device 2 also performs the same operation.

The SAO unit 162 performs SAO processing for each CTU and adds an offset value for each pixel to improve image quality. SAO has two modes: edge offset for performing offset according to four types of edge directions, and band offset for applying offset to pixel value ranges divided into 32 types of bands. SAO processing can be switched between edge offset, band offset, or no application for each CTU.

Edge offset processing is processing for adjusting the pixel value for each pixel according to a certain pixel c and two pixels b and c adjacent thereto. As shown in FIG. 4, there are four pixel array candidates (classes) for the arrangement of two adjacent pixels, and a selection is made from these classes. Four absolute offset values are given to the selected class for each of four types of categories (category 1 to category 4) determined by the relative relationship with two adjacent pixels. Categories 1 and 2 add the offset to the pixel value, while categories 3 and 4 subtract the offset from the pixel value.

Band offset processing divides the gradation of pixel values into 32 bands, and adjusts the pixel values of pixels belonging to four bands in which the pixel values are continuous, using offset values set for each band. processing.

According to the edge offset, since the pixel series is smoothed by adding and subtracting the offset value, the ringing distortion can be alleviated by suppressing the fluctuation around the edge. On the other hand, according to the band offset, gradation deterioration can be alleviated by correcting specific continuous gradation. By selecting each mode (edge offset, band offset) with high precision. Further improvement in image quality is expected.

<5. Configuration of Loop Filter Control Unit>
FIG. 5 is a diagram showing the configuration of the loop filter control section 170 of the image encoding device 1 according to this embodiment. The loop filter control section 240 of the image decoding device 2 has the same configuration as the loop filter control section 170 of the image encoding device 1 .

As shown in FIG. 5 , loop filter control section 170 includes first edge determination section 171 , second edge determination section 172 , and control section 173 .

Depth information is input to the first edge determination unit 171 . Depth information is a numerical representation of the distance (depth) to each object in the reconstructed image in units of pixels. However, the depth information is not limited to representing the depth in units of pixels, and may be information representing the depth in units of groups consisting of a plurality of pixels.

The first edge determination unit 171 determines whether or not the block boundary in the reconstructed image is an edge region based on the input depth information, and sends the determination result to the second edge determination unit 172 and the control unit 173. Output. Here, the edge region means a region in which the pixel values in the image change greatly (that is, a region in which the pixel values change greatly in the image), and has a certain length. For example, an edge region is a boundary between an object in an image and a background or a boundary between objects in an image, but an edge region can also exist within the same object. Since the edge area is a boundary area that originally exists in the image, if the deblocking filter process is performed on such an edge area, the edge will be blurred and the image quality will be degraded.

Note that each of the first edge determination unit 171 and the second edge determination unit 172 targets the block boundaries of blocks of the same size as the block size (8×8 pixels) for which the deblocking filtering unit 161 performs deblocking filtering. Determine edge regions. That is, the positions of the block boundaries to be determined as edge regions by the first edge determination unit 171 and the second edge determination unit 172 are the same.

For example, the first edge determination unit 171 calculates the degree of change in the depth information of the plurality of pixels based on the depth information of each of the plurality of pixels near the block boundary of two adjacent blocks. , it is determined that the block boundary between the two blocks is an edge area. Alternatively, the first edge determination unit 171 obtains a representative value of depth information for each of two adjacent blocks, and if the difference between the depth representative values of each block is equal to or greater than a predetermined value, the block of the two blocks is determined. A boundary may be determined to be an edge region. The first edge determination unit 171 transmits information indicating the position of the block boundary determined to be an edge region and/or information indicating the position of the block boundary determined not to be an edge region to the second edge determination unit 172 and the control unit 173. Output.

The second edge determination unit 172 determines whether or not the block boundary in the reconstructed image is an edge region based on the reconstructed image (two-dimensional image) input from the combining unit 150, and controls the determination result. Output to unit 173 . In this embodiment, the second edge determination unit 172 determines whether the block boundary is an edge region only for the block boundary determined by the first edge determination unit 171 not to be an edge region.

Specifically, the second edge determination unit 172 calculates the degree of variation in the pixel values of the plurality of pixels based on the pixel values (luminance values) of the plurality of pixels near the block boundary of two adjacent blocks. If the degree of variation is large, it is determined that the block boundary between the two blocks is an edge area. For example, the second edge determination unit 172 determines that the block boundary between two blocks P and Q (see FIG. 3) is an edge area when the following formula (7) is not satisfied.

Here, the value of the threshold β varies according to the average value Q _av of the quantization parameters of adjacent blocks P and Q. FIG.

Then, the second edge determination unit 172 sends information indicating the position of the block boundary determined to be the edge region and/or the information indicating the position of the block boundary determined not to be the edge region to the second edge determination unit 172 and the control unit. 173.

The control unit 173 controls loop filter processing on the reconstructed image based on the determination result of the first edge determination unit 171 and the determination result of the second edge determination unit 172 . Specifically, the control unit 173 includes a deblocking filter control unit 173 a that controls the deblocking filter unit 161 and an SAO mode selection unit 173 b that selects the mode in the SAO unit 162 .

The deblocking filter control unit 173a applies a deblocking filter only to block boundaries that are determined not to be edge regions by the first edge determination unit 171 and are determined not to be edge regions by the second edge determination unit 172. Apply processing.

That is, the deblocking filter control unit 173a controls the deblocking filter unit 161 so as not to apply deblocking filter processing to the block boundaries determined by the first edge determination unit 171 to be edge regions. Also, the deblocking filter control unit 173a controls the deblocking filter unit 161 so as not to apply deblocking filter processing to block boundaries determined by the second edge determination unit 172 to be edge regions.

The SAO mode selection unit 173b selects a sample adaptive offset processing mode from among a plurality of modes including edge offset and band offset, based on at least the determination result of the first edge determination unit 171. FIG.

For example, the SAO mode selection unit 173b selects edge offset as the SAO processing mode for the CTU when the proportion of the block boundaries in the CTU occupied by edge regions is greater than a predetermined value (eg, 50%). On the other hand, the SAO mode selection unit 173b selects the band offset as the SAO processing mode for the CTU when the ratio of the edge region among the block boundaries in the CTU is equal to or less than a predetermined value, or You may choose not to apply the SAO process.

<6. Operation example of the loop filter control section>
FIG. 6 is a diagram showing an operation example of the loop filter control section 170 of the image encoding device 1 according to this embodiment. The loop filter control section 240 of the image decoding device 2 operates similarly to the loop filter control section 170 of the image encoding device 1 .

As shown in FIG. 6, in step S1, the first edge determination unit 171 determines (acquires) whether or not a block boundary in a reconstructed image (two-dimensional image) is an edge region based on depth information. and outputs the determination result to the second edge determination unit 172 and the control unit 173 .

If the block boundary is determined to be an edge region based on the depth information (step S2: YES), the deblocking filter control unit 173a of the control unit 173 applies deblocking filter processing to the block boundary in step S3. The deblocking filter unit 161 is controlled not to (turn off).

On the other hand, if the block boundary is not determined to be an edge region based on the depth information (step S2: NO), in step S4, the second edge determination unit 172 determines the block boundary based on the reconstructed image (two-dimensional image). It determines (detects) whether or not the boundary is an edge region, and outputs the determination result to the control unit 173 .

If the block boundary is determined to be an edge region based on the reconstructed image (step S5: YES), the deblocking filter control unit 173a does not apply deblocking filter processing to the block boundary in step S3 ( control the deblocking filter unit 161 to turn off.

On the other hand, if the block boundary is not determined to be an edge region based on the reconstructed image (step S5: NO), the deblocking filter control unit 173a performs deblocking filtering on the block boundary in step S6. The deblocking filter unit 161 is controlled to apply (turn on).

Next, in step S7, the SAO mode selection unit 173b of the control unit 173 determines, based on the determination result of the first edge determination unit 171, that the proportion of the block boundaries in the CTU occupied by edge regions is a predetermined value (for example, 50 %).

If the proportion of the block boundary in the CTU occupied by the edge region is greater than a predetermined value (eg, 50%), in step S8, the SAO mode selection unit 173b selects edge offset as the SAO processing mode for the CTU. .

On the other hand, if the edge offset is not selected, in step S9, the SAO mode selection unit 173b selects the SAO processing mode for the CTU based on other criteria. For example, based on the determination result of the second edge determination unit 172, the SAO mode selection unit 173b determines whether the proportion of the block boundary in the CTU occupied by the edge region is greater than a predetermined value (eg, 50%). is determined, and if it is greater than a predetermined value, the edge offset is selected (step S8), and if it is less than or equal to the predetermined value, the band offset is selected (step S10) or SAO processing is not applied ( step S11).

<7. Summary of Embodiments>
As described above, the loop filter control unit 170 includes a first edge determination unit 171 that determines whether a block boundary in a two-dimensional image (reconstructed image) is an edge region based on depth information; Based on the dimensional image, the second edge determination unit 172 determines whether the block boundary is an edge region, and based on the determination result of the first edge determination unit 171 and the determination result of the second edge determination unit 172, and a control unit 173 that controls the loop filtering process for the two-dimensional image.

By performing edge determination based on depth information, for example, an edge region between an object and a background can be determined correctly. In addition, by performing edge determination based on a two-dimensional image, it is possible to accurately determine edge areas within the same object and edge areas between objects having the same depth, so it is possible to improve the determination accuracy of edge areas. . Therefore, loop filter processing can be controlled more appropriately.

Depth information has the feature of accurately capturing the shape of an object, and an area where the depth information greatly changes can be regarded as an edge area. For the edge area on the boundary between the object and the background, the edge area is accurately detected by the edge determination based on the depth information, and the detection accuracy of the edge area is higher when the depth information is used.

On the other hand, for edge regions within the same object, it is considered that edge regions are accurately detected by edge determination based on a two-dimensional image, and that using a two-dimensional image is more accurate than using depth information. .

In this embodiment, deblocking filter processing is not applied to block boundaries determined to be edge regions based on depth information. For block boundaries that are not determined to be edge regions based on depth information, edge determination is performed based on the two-dimensional moving image. Not applicable. Then, deblocking filter processing is applied only to block boundaries that have not been determined to be edge areas in neither edge determination based on depth information nor edge determination based on a two-dimensional moving image. As a result, it is possible to reduce the amount of arithmetic processing for the deblocking filter process while realizing highly accurate edge region detection.

Furthermore, by selecting the SAO mode based on the edge information acquired based on the depth information, the SAO mode can be appropriately selected.

<8. Other Embodiments>
In the embodiments described above, HEVC/H. Although deblocking filtering and SAO processing in H.265 have been described as examples, the present invention also applies to HEVC/H.265. It is not limited to H.265, and can be applied to other encoding schemes such as next-generation image encoding schemes.

In the above-described embodiment, an example of controlling whether the deblocking filter control unit 173a applies deblocking filtering (that is, deblocking filter ON/OFF) has been described, but the deblocking filter control unit 173a Control over whether to apply a strong filter or a weak filter may be provided.

A program that causes a computer to execute each process performed by the image encoding device 1 and a program that causes a computer to execute each process that the image decoding device 2 performs may be provided. The program may be recorded on a computer readable medium. A computer readable medium allows the installation of the program on the computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be, for example, a recording medium such as CD-ROM or DVD-ROM. Alternatively, the image encoding device 1 may be configured as a semiconductor integrated circuit (chipset, SoC) by integrating a circuit that executes each process performed by the image encoding device 1 . Similarly, the image decoding device 2 may be configured as a semiconductor integrated circuit by integrating a circuit for executing each process performed by the image decoding device 2 .

Although the embodiments have been described in detail with reference to the drawings, the specific configuration is not limited to the above, and various design changes can be made without departing from the spirit of the invention.

1: image coding device 2: image decoding device 100: block division unit 110: subtraction unit 120: quantization unit 121: transformation unit 122: quantization unit 130: entropy coding unit 140: inverse transformation unit 141: inverse quantization Unit 142: Inverse transform unit 150: Synthesis unit 160: Loop filter 161: Deblocking filter unit 162: SAO unit 170: Loop filter control unit 171: First edge determination unit 172: Second edge determination unit 173: Control unit 173a: Deblocking filter control unit 173b : SAO mode selection unit 180 : Memory 190 : Prediction unit 191 : Intra prediction unit 192 : Inter prediction unit 193 : Switching unit 200 : Entropy code decoding unit 210 : Inverse transform unit 211 : Inverse quantization unit 212 : inverse transform unit 220 : synthesis unit 230 : loop filter 231 : deblocking filter unit 232 : SAO unit 240 : loop filter control unit 250 : memory 260 : prediction unit 261 : intra prediction unit 262 : inter prediction unit 263 : switching unit

Claims (6)

A loop filter control device for controlling a loop filter of an image processing device that performs block-by-block processing on a two-dimensional image to which depth information is added,
a first edge determination unit that determines whether a block boundary in the two-dimensional image is an edge region based on the depth information;
a second edge determination unit that determines whether the block boundary is an edge region based on the two-dimensional image;
a control unit that controls loop filter processing for the two-dimensional image based on the determination result of the first edge determination unit and the determination result of the second edge determination unit ;
The second edge determination unit determines whether or not the block boundary is an edge region only for a block boundary determined not to be an edge region by the first edge determination unit. Loop filter controller. The loop filtering includes deblocking filtering,
The control unit
A deblocking filter that applies the deblocking filter processing only to block boundaries that are determined not to be edge regions by the first edge determination unit and are determined not to be edge regions by the second edge determination unit. The loop filter control device according to claim 1, further comprising a control section. The loop filtering includes sample adaptive offset processing,
The control unit
4. A mode selection unit that selects a mode of the sample adaptive offset processing from among a plurality of modes including edge offset and band offset, based on at least a determination result of the first edge determination unit. 3. The loop filter control device according to 1 or 2 . An image coding device comprising the loop filter control device according to claim 1 . An image decoding device comprising the loop filter control device according to claim 1 . A program that causes a computer to function as the loop filter control device according to claim 1 .

Images