KR102072204B1 - Apparatus and method of improving quality of image - Google Patents

Abstract

An apparatus and method for improving image quality of an image may include a gradient covariance matrix calculator configured to calculate a gradient covariance matrix from a color image, and the calculated gradient covariance matrix. A kernel arithmetic unit for calculating a kernel using a gradient covariance matrix, and a weighted average calculator for restoring pixels of the color image by calculating a weighted average from the computed kernel.

Description

Apparatus and method for improving image quality {APPARATUS AND METHOD OF IMPROVING QUALITY OF IMAGE}

The following embodiments disclose the technical idea of an apparatus and method for improving image quality that can be used in the field of producing, compressing, transmitting, and displaying an image.

3D image compression system is a system for compressing color video (depth video, depth map). On the other hand, the compression of the color image can be efficiently compressed in the same way as H.264 / AVC, H.264 / MVC, HEVC, but the depth image is significantly different from the color image.

Existing compression (encoding) standards for video compression include H.261, H.263, MPEG-1, MPEG-2, MPEG-4, H.264, and High Efficiency Video Coding (HEVC).

Existing compression standards are slightly different but generally consist of similar structures including motion estimation and compensation, transform encoding, and entropy encoding.

In particular, H.264 and HEVC are known to minimize block boundary distortion in the reconstructed image, thereby improving overall encoding efficiency by enabling more accurate prediction in motion estimation and compensation as well as subjective picture quality improvement.

The deblocking filter shows a good performance in a low bitrate image, but has little performance in a high quality image, or rather has a problem of degrading encoding performance.

In addition, the adaptive loop filter (ALF), which has been recently adopted in the compression standard, is a filter that minimizes the error between the reconstructed picture and the original picture, and when applied like a deblocking filter in a high quality picture, the coding efficiency is improved. It works.

A general adaptive loop filter is a reconstruction filter based on a Wiener filter.

Recently, a method of improving the objective picture quality by providing an adaptive loop filter after the deblocking filter has been proposed.

An apparatus for improving image quality according to an embodiment may include: a gradient covariance matrix calculator configured to calculate a gradient covariance matrix from a color image; and calculating a kernel using the calculated gradient covariance matrix. A kernel calculator may include a weighted average calculator configured to calculate a weighted average from the calculated kernel and to restore pixels of the color image.

The gradient covariance matrix calculator, according to an embodiment, calculates a gradient matrix of the color image, calculates a gradient matrix of a depth image corresponding to the color image, and calculates the calculated color image. The gradient covariance matrix may be calculated from a gradient matrix of and a gradient matrix of the calculated depth image.

The gradient covariance matrix calculator according to an embodiment may add the gradient matrix of the calculated color image and the gradient matrix of the calculated depth image to calculate the gradient covariance matrix. Can be calculated

According to an embodiment, the kernel calculator may calculate a kernel expressing a weight between a center pixel and an adjacent pixel by using the calculated gradient covariance matrix.

According to an embodiment, an apparatus for improving image quality may include: a gradient covariance matrix calculator configured to calculate a gradient covariance matrix from a depth image; and using the calculated gradient covariance matrix to adjoin a center pixel. A kernel calculating unit calculating a kernel expressing weights between pixels, and a weighted average calculating unit calculating a weighted average from the calculated kernels and restoring pixels of the depth image, wherein the gradient covariance matrix calculator comprises: the depth image Compute a gradient matrix of, calculate a gradient matrix of the color image corresponding to the depth image, the gradient matrix of the calculated depth image and the gradient of the calculated color image The gradient ball from a matrix One can calculate the acid matrix (gradient covariance matrix).

The gradient covariance matrix calculator according to an embodiment may add the gradient matrix of the calculated depth image and the gradient matrix of the calculated color image to calculate the gradient covariance matrix. Can be calculated

According to an embodiment, an image quality improvement method includes calculating a gradient covariance matrix from a color image in a gradient covariance matrix calculator, and calculating, by the kernel calculator, the calculated gradient covariance matrix. Computing a kernel using the step, and the weight average calculation unit, calculating a weighted average from the calculated kernel, it may include the step of restoring the pixel of the color image.

The calculating of the gradient covariance matrix according to an embodiment may include calculating a gradient matrix of the color image, and a gradient matrix of a depth image corresponding to the color image. And calculating the gradient covariance matrix from the calculated gradient matrix of the color image and the calculated gradient matrix of the depth image. .

The calculating of the gradient covariance matrix according to an embodiment may include adding the gradient matrix of the calculated color image and the gradient matrix of the calculated depth image to the gradient. The method may include calculating a gradient covariance matrix.

Computing the kernel according to an embodiment may include calculating a kernel expressing a weight between a center pixel and an adjacent pixel by using the calculated gradient covariance matrix.

FIG. 1 is a diagram illustrating gradients of a color image and a depth image in a 3D image.
2 is a block diagram illustrating an image quality improving apparatus according to an exemplary embodiment.
3 is a block diagram illustrating an embodiment in which an image quality improving apparatus operates at an in-loop position of an encoder.
4 is a block diagram illustrating an embodiment in which an image quality improving apparatus operates at an in-loop position of a decoder.
5 is a block diagram illustrating an embodiment in which an image quality improving apparatus operates at a post-processing position of a decoder.
6 is a flowchart illustrating a method of improving image quality according to an embodiment.
FIG. 7 is a flowchart illustrating a process of calculating a gradient covariance matrix in detail in the embodiment of FIG. 6.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings.

In describing the embodiments, when it is determined that detailed descriptions of related known functions or configurations may unnecessarily obscure the gist of the embodiments, the detailed description thereof will be omitted. In addition, terms used in the present specification are terms used to properly express the embodiments, which may vary depending on a user, an operator's intention, or a convention of a corresponding technology. Therefore, definitions of terms should be made based on the contents throughout the specification. Like reference numerals in the drawings denote like elements.

1 illustrates gradients 111 and 121 of a color image 110 and a depth image 120 in a 3D image.

The color image 110 is expressed by classifying the boundary between the object and the background, and the gradient 111 that distinguishes the object from the background may be represented by an eigenvalue and an eigenvector.

The depth image 120 represents the distance (depth) of each pixel on the z-axis, and may be expressed as a depth value corresponding to the color image 110.

Therefore, the depth image 120 may be represented by the gradient 121 corresponding to the gradient 111 of the color image 110 on the z-axis.

In order to express a 3D image, a color image 110 and a depth image 120 are required.

The image quality improving apparatus and method according to the exemplary embodiment reflects the gradient 121 of the depth image 120 corresponding to the color image 110 on the gradient 111 of the color image 110, and thus the color image 110. The image quality of the color image 110 may be improved by restoring a pixel of the color image 110.

According to another exemplary embodiment, an apparatus and a method for improving image quality reflect a gradient image 111 of a color image 110 corresponding to the depth image 120 to a gradient 121 of a depth image 120. The image quality of the depth image 120 may be improved by reconstructing the pixel of 120.

Therefore, by using the apparatus and method for improving image quality according to an embodiment, the image quality of a deteriorated color image or a depth image may be improved in the fields of producing, compressing, transmitting, and displaying a color image or a depth image.

2 is a block diagram illustrating an image quality improving apparatus according to an exemplary embodiment.

The apparatus 200 for improving image quality according to an embodiment may include a gradient covariance matrix calculator 210, a kernel calculator 220, and a weighted average calculator 230.

The gradient covariance matrix calculator 210 may calculate a gradient covariance matrix from a color image.

Covariance can be interpreted as a value indicating the degree of correlation between two random variables, independent of the variance indicating the degree of discreteness of the variable.

If the value of one of the two variables tends to rise, if the other value also correlates with the tendency to rise, then the value of the covariance is positive and, conversely, when the value of one of the two variables tends to rise However, if other values tend to fall, the value of the covariance is negative.

The gradient covariance matrix calculator 210 may calculate a gradient covariance matrix by calculating a correlation of a gradient matrix by reflecting the characteristics of the covariance.

In detail, the gradient covariance matrix calculator 210 may calculate a gradient covariance matrix using a gradient matrix of a color image and a gradient matrix of a depth image.

For example, the gradient covariance matrix calculator 210 may calculate a gradient covariance matrix by reflecting a correlation between a gradient matrix of a color image and a gradient matrix of a depth image. .

More specifically, the gradient covariance matrix calculator 210 may calculate a gradient matrix of the color image.

In addition, the gradient covariance matrix calculator 210 may calculate a gradient matrix of a depth image corresponding to the color image.

The gradient covariance matrix calculator 210 may calculate the gradient covariance matrix from the calculated gradient matrix of the color image and the gradient matrix of the calculated depth image.

For example, the gradient covariance matrix calculator 210 may generate a first gradient matrix obtained by summing a gradient matrix of the calculated color image and a gradient matrix of the calculated depth image.

In addition, the gradient covariance matrix calculator 210 may generate a second gradient matrix having a pre-matrix relationship with the first gradient matrix.

The gradient covariance matrix calculator 210 may multiply the generated first gradient matrix and the second gradient matrix to finally output a gradient covariance matrix.

A gradient matrix calculated by the gradient covariance matrix calculator 210 from a color image or a depth image may be represented by Equation 1.

[Equation 1]

Here, G may be interpreted as a gradient matrix.

V ^T can be interpreted as a transpose matrix of a 1 * 1 matrix [V ₁ V ₂ ].

S ₁ and S ₂ may be interpreted as eigenvalues, and V ₁ and V ₂ may be interpreted as eigenvectors.

A gradient matrix calculated by the gradient covariance matrix calculator 210 from a color image or a depth image may be represented by Equation 1.

[Equation 2]

는 색상 영상의 그래디언트 행렬과 깊이 영상의 그래디언트 행렬을 모두 반영한 제1 그래디언트 행렬로 해석될 수 있다.

May be interpreted as a first gradient matrix that reflects both a gradient matrix of a color image and a gradient matrix of a depth image.

는 그래디언트 공분산 행렬 계산부(210)가 계산하는 색상 영상의 그래디언트 행렬(gradient matrix)로 해석될 수 있다.

May be interpreted as a gradient matrix of a color image calculated by the gradient covariance matrix calculator 210.

는 그래디언트 공분산 행렬 계산부(210)가 계산하는 깊이 영상의 그래디언트 행렬(gradient matrix)로 해석될 수 있다.

May be interpreted as a gradient matrix of a depth image calculated by the gradient covariance matrix calculator 210.

Next, the gradient covariance matrix calculator 210 may calculate a gradient covariance matrix by calculating a first gradient matrix and a second gradient matrix that is a transpose matrix of the first gradient matrix, using Equation 3.

[Equation 3]

는 그래디언트 공분산 행렬(gradient covariance matrix)로 해석될 수 있다.here,

Can be interpreted as a gradient covariance matrix.

는 색상 영상의 그래디언트 행렬(gradient matrix)과 깊이 영상의 그래디언트 행렬(gradient matrix)을 모두 반영한 제1 그래디언트 행렬로 해석될 수 있다.

May be interpreted as a first gradient matrix that reflects both a gradient matrix of a color image and a gradient matrix of a depth image.

는 제1 그래디언트 행렬의 전치 행렬인 제2 그래디언트 행렬로 해석될 수 있다.

May be interpreted as a second gradient matrix that is a transpose matrix of the first gradient matrix.

The kernel operator 220 according to an exemplary embodiment may calculate a kernel using a gradient covariance matrix calculated by the gradient covariance matrix calculator 210.

More specifically, the kernel operator 220 according to an embodiment uses the gradient covariance matrix calculated by the gradient covariance matrix calculator 210 to calculate a kernel expressing a weight between a center pixel and an adjacent pixel. can do.

The kernel calculating unit 220 according to an embodiment may calculate the kernel by using Equation 4.

[Equation 4]

는 커널로 해석될 수 있고,

는 전체 합을 1로 만들기 위한 정규화 상수(normalization constant)로 해석될 수 있다.here,

Can be interpreted as a kernel,

Can be interpreted as a normalization constant to make the total sum to 1.

는 제1 그래디언트 행렬과 제2 그래디언트 행렬로부터 연산된 그래디언트 공분산 행렬(gradient covariance matrix)로 해석될 수 있다.

May be interpreted as a gradient covariance matrix calculated from the first gradient matrix and the second gradient matrix.

Next, the weight average calculator 230 may calculate a weight average from the kernel calculated by the kernel calculator 220.

In other words, the weighted average calculator 230 may generate a reconstructed pixel for the color image by calculating a weighted average.

To this end, the weighted average calculator 230 may calculate a weighted average, that is, a reconstructed pixel, by using Equation (5).

[Equation 5]

는 가중치 평균, 즉, 복원된 픽셀로 해석될 수 있다.here,

Can be interpreted as the weighted average, i.e., the reconstructed pixel.

는 커널로 해석될 수 있다.

Can be interpreted as a kernel.

는 입력되는 픽셀들로 해석될 수 있다.

May be interpreted as input pixels.

The weighted average calculator 230 may restore the pixels of the color image by using the calculated weighted average to improve the image quality of the image.

As a result, when the image quality improving apparatus 200 is used, the compression ratio may be improved by improving the image quality of the image degraded by image compression.

The image quality improving apparatus 200 according to an exemplary embodiment may calculate each gradient matrix from an input color image and a depth image corresponding to the color image, and calculate a gradient covariance matrix from the calculated gradient matrix.

In addition, the kernel may be calculated from the calculated gradient covariance matrix, and the weighted average may be calculated from the calculated kernel to improve the image quality of the input color image.

The image quality improving apparatus 200 according to another exemplary embodiment may calculate each gradient matrix from an input depth image and a color image corresponding to the depth image, and calculate a gradient covariance matrix from the calculated gradient matrix.

In addition, the kernel may be calculated from the calculated gradient covariance matrix, and the weighted average may be calculated from the calculated kernel to improve the quality of the input depth image.

To this end, the gradient covariance matrix calculator 210 may calculate a gradient covariance matrix from a depth image.

The kernel calculator 220 may calculate a kernel expressing a weight between a center pixel and an adjacent pixel by using the calculated gradient covariance matrix.

The weighted average calculator 230 may reconstruct the pixel of the depth image by calculating a weighted average from the calculated kernel.

The gradient covariance matrix calculator 210 may calculate a gradient matrix of a depth image, and calculate a gradient matrix of a color image corresponding to the depth image.

In addition, the gradient covariance matrix calculation unit 210 according to an embodiment may generate a gradient covariance matrix from the calculated gradient matrix of the depth image and the gradient matrix of the calculated color image. Can be calculated

In detail, the gradient covariance matrix calculator 210 may calculate a gradient matrix of an input depth image.

In addition, the gradient covariance matrix calculator 210 may calculate a gradient matrix of the color image corresponding to the depth image.

The gradient covariance matrix calculator 210 may calculate a gradient covariance matrix from the calculated gradient matrix of the depth image and the gradient matrix of the calculated color image.

For example, the gradient covariance matrix calculator 210 may generate a first gradient matrix obtained by adding the calculated gradient matrix of the depth image and the calculated gradient matrix of the color image.

In addition, the gradient covariance matrix calculator 210 may generate a second gradient matrix having a pre-matrix relationship with the first gradient matrix.

The gradient covariance matrix calculator 210 may multiply the generated first gradient matrix and the second gradient matrix to finally output a gradient covariance matrix.

3 is a block diagram illustrating an embodiment in which an image quality improving apparatus operates at an in-loop position of an encoder.

The apparatus for improving image quality according to an embodiment may be implemented as some components of the in-loop filter 305 of the encoder 300 of the 3D image compression system.

The encoder 300 according to an embodiment may include a predictor 301, a transform and quantizer 302, an entropy coding unit 303, an inverse quantization and inverse transform unit 304, an in-loop filter 305, and a picture buffer ( 306, and a motion estimation and compensation unit 307.

The encoder 300 of the 3D image compression system according to an embodiment may perform preprocessing on at least one color image or depth image input.

In detail, the encoder 300 to which the image quality improving apparatus is applied may perform a preprocessing filtering function to improve the compression ratio of the color image or the depth image by improving the image quality of the color image or the depth image.

The prediction unit 301 largely performs intra prediction and inter prediction.

The prediction image output from the prediction unit 301 and the difference image output from the transform and quantization unit 302 are combined with the input image to generate a compressed image.

The encoder 300 of the 3D image compression system performs a reconstruction filter on the compressed image by the in-loop filter 305, stores the resultant image in the picture buffer 306, and transmits additional information to the entropy coding unit 303. I can deliver it.

In this process, the encoder 300 of the 3D image compression system may perform inverse quantization and inverse transformation on the output of the transform and quantization unit 302 using the inverse quantization and inverse transform unit 304.

The motion estimation and compensation unit 307 estimates and compensates for the motion of the input image, thereby reducing noise of the input image.

4 is a block diagram illustrating an embodiment in which an image quality improving apparatus operates at an in-loop position of a decoder.

The apparatus for improving image quality according to an embodiment may be implemented as some components of an in-loop filter of a decoder of a 3D image compression system.

In other words, as shown in FIG. 4, the image quality improving apparatus according to the embodiment may be included in the decoder 400 of the 3D image compression system in the form of an in-loop filter.

The decoder 400 of the 3D image compression system according to an embodiment includes an entropy decoding unit 401, an inverse quantization and inverse transform unit 402, a motion estimation and compensation unit 403, a picture buffer 404, and an Loop filter 405 may be included.

The decoder 400 of the 3D image compression system may receive a bit stream transmitted from the encoder of the compression system in the entropy decoding unit 401 and obtain additional information from the received bit stream.

In addition, the decoder 400 of the 3D image compression system may reconstruct the color image or the depth image using the obtained additional information by the inverse quantization and inverse transform unit 402, and the reconstructed color image or the depth image may be The in-loop filter 405 may be stored in the picture buffer 404.

The in-loop filter 405 to which the image quality improving apparatus is applied may perform a preprocessing filtering function to improve the compression ratio of the color image or the depth image by improving the quality of the color image or the depth image.

The motion estimation and compensation unit 403 estimates and compensates for the motion of the input image, thereby reducing noise of the input image.

5 is a block diagram illustrating an embodiment in which an image quality improving apparatus operates at a post-processing position of a decoder.

The image quality improving apparatus may be implemented as some components of a post filter of a decoder of a 3D image compression system.

The image quality improving apparatus may be implemented as some components of a post filter of a decoder of a 3D image compression system.

In other words, as shown in FIG. 4, the image quality improving apparatus according to the embodiment may be included in the decoder 500 of the 3D image compression system in the form of a post filter.

The decoder 500 of the 3D image compression system according to an embodiment includes an entropy decoding unit 501, an inverse quantization and inverse transform unit 502, a motion estimation and compensation unit 503, a picture buffer 504, and a post. It may include a filter 505.

The decoder 500 of the 3D image compression system may receive a bit stream transmitted from the encoder of the compression system in the entropy decoding unit 501 and obtain additional information from the received bit stream.

In addition, the decoder 500 of the 3D image compression system may reconstruct the color image or the depth image using the obtained additional information by the inverse quantization and inverse transform unit 502, and the reconstructed color image or the depth image may be It may be input to the picture buffer 504.

Also, the reconstructed color image or depth image may be generated as an output image through the post filter 505.

The post filter 505 to which the image quality improving apparatus is applied may perform a preprocessing filtering function to improve the compression ratio of the color image or the depth image by improving the image quality of the color image or the depth image.

The motion estimation and compensation unit 503 estimates and compensates for the motion of the input image, thereby reducing noise of the input image.

6 is a flowchart illustrating a method of improving image quality according to an embodiment.

According to an embodiment, the image quality improvement method may calculate a gradient covariance matrix from a color image (step 601).

In order to calculate a gradient covariance matrix, an image quality improving method according to an embodiment may calculate a gradient matrix of a color image.

In addition, the image quality improving method according to an embodiment may calculate a gradient matrix of a depth image corresponding to the color image.

The image quality improving method according to an embodiment may calculate a gradient covariance matrix from a calculated gradient matrix of a color image and a calculated gradient matrix of a depth image.

Specifically, in order to calculate a gradient covariance matrix, an image quality improvement method according to an embodiment may include a gradient matrix of a calculated color image and a gradient matrix of a calculated depth image. The summation can be used to calculate the gradient covariance matrix.

According to an embodiment, an image quality improvement method may calculate a kernel using a calculated gradient covariance matrix (step 602).

In detail, the image quality improving method according to an embodiment may calculate a kernel expressing a weight between a center pixel and an adjacent pixel by using a gradient covariance matrix calculated to calculate a kernel.

According to an exemplary embodiment, an image quality improvement method may calculate a weighted average from the computed kernel, and restore pixels of the color image (step 603).

According to another exemplary embodiment, an image quality improvement method may calculate a gradient matrix from an input depth image and a color image corresponding to the depth image, and calculate a gradient covariance matrix from the calculated gradient matrix.

In addition, the image quality improvement method may improve the quality of the input depth image by calculating a kernel from the calculated gradient covariance matrix and calculating a weighted average from the calculated kernel.

To this end, the image quality improvement method may calculate a gradient covariance matrix from a depth image.

In addition, the image quality improving method may restore the pixels of the depth image by calculating a weighted average from the calculated kernel.

FIG. 7 is a flowchart illustrating a process of calculating a gradient covariance matrix in detail in the embodiment of FIG. 6.

According to an embodiment, the image quality improving method may calculate a gradient matrix of a color image (step 701).

Next, according to an embodiment, the image quality improving method may calculate a gradient matrix of a depth image corresponding to a color image (step 702).

In addition, the image quality improvement method may calculate a gradient covariance matrix from the calculated gradient matrix of the color image and the calculated gradient matrix of the depth image (step 703).

For example, the image quality improvement method may generate a first gradient matrix obtained by adding a calculated gradient matrix of a color image and a calculated gradient matrix of a depth image.

In addition, the image quality improving method may generate a second gradient matrix having a pre-matrix relationship with the first gradient matrix.

The image quality improvement method may multiply the generated first gradient matrix and the second gradient matrix to finally output a gradient covariance matrix.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or, even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

200: image quality improvement device
210: gradient covariance matrix calculation unit
220: kernel operation unit
230: weighted average calculation unit

Claims (11)

A gradient covariance matrix calculator configured to calculate a gradient covariance matrix from a gradient matrix of a color image and a gradient matrix of a depth image corresponding to the color image;
A kernel calculator configured to calculate a kernel using the calculated gradient covariance matrix; And
A weighted average calculation unit which calculates a weighted average from the calculated kernel and restores pixels of the color image.
Image quality improvement apparatus comprising a. The method of claim 1,
The gradient covariance matrix calculator,
A gradient matrix of the color image is calculated, a gradient matrix of a depth image corresponding to the color image is calculated, a gradient matrix of the color image and a gradient matrix of the depth image. and an image quality improvement apparatus calculating the gradient covariance matrix based on the summation of the gradient matrix. The method of claim 2,
The gradient covariance matrix calculator,
The first gradient matrix is generated by summing a gradient matrix of the color image and a gradient matrix of the depth image, and generating a second gradient matrix having a total matrix relation with the first gradient matrix. 1. The apparatus of claim 1, wherein the gradient covariance matrix is calculated by operating with a gradient matrix. The method of claim 1,
The kernel operation unit,
And a kernel expressing a weight value between a center pixel and an adjacent pixel by using the calculated gradient covariance matrix. A gradient covariance matrix calculator for calculating a gradient covariance matrix from a depth image;
A kernel calculator configured to calculate a kernel expressing a weight value between a center pixel and an adjacent pixel by using the calculated gradient covariance matrix; And
A weighted average calculator configured to reconstruct the pixel of the depth image by calculating a weighted average from the calculated kernel.
Including,
The gradient covariance matrix calculator,
Calculating a gradient matrix of the depth image, calculating a gradient matrix of a color image corresponding to the depth image, a gradient matrix of the calculated depth image, and the calculated color And an image quality improvement device for calculating the gradient covariance matrix from a gradient matrix of an image. The method of claim 5,
The gradient covariance matrix calculator,
And calculating the gradient covariance matrix by summing the gradient matrix of the calculated depth image and the gradient matrix of the calculated color image. Calculating, by a gradient covariance matrix calculator, a gradient covariance matrix from a gradient matrix of a color image and a gradient matrix of a depth image corresponding to the color image;
Computing a kernel in the kernel operation unit using the calculated gradient covariance matrix; And
Restoring a pixel of the color image by calculating a weighted average from the calculated kernel in a weighted average calculator
Image quality improvement method comprising a. The method of claim 7, wherein
Computing the gradient covariance matrix,
Calculating a gradient matrix of the color image;
Calculating a gradient matrix of a depth image corresponding to the color image; And
Calculating the gradient covariance matrix based on a result of summing a gradient matrix of the color image and a gradient matrix of the depth image;
Image quality improvement method comprising a. The method of claim 8,
Calculating a gradient covariance matrix based on the summation,
The first gradient matrix is generated by summing a gradient matrix of the color image and a gradient matrix of the depth image, and generating a second gradient matrix having a total matrix relationship with the first gradient matrix. 1 computing the gradient covariance matrix by computing with a gradient matrix
Image quality improvement method comprising a. The method of claim 7, wherein
Computing the kernel,
Calculating a kernel expressing a weight between a center pixel and an adjacent pixel by using the calculated gradient covariance matrix
Image quality improvement method comprising a. A computer-readable recording medium having recorded thereon a program for performing the method of any one of claims 7 to 10.

Images