KR20140128041A - Apparatus and method of improving quality of image - Google Patents

Abstract

The present invention relates to an apparatus and a method for improving quality of an image. The apparatus for improving quality of an image according to an embodiment of the present invention comprises: a gradient covariance matrix calculating unit which calculates a gradient covariance matrix from a color image; a kernel calculating unit which calculates a kernel by using the calculated gradient covariance matrix; and a weighted average calculating unit which calculates a weighted average from the calculated kernel and restores pixels of the color image.

Description

[0001] APPARATUS AND METHOD OF IMPROVING QUALITY OF IMAGE [0002]

The following embodiments disclose technical ideas on an image quality enhancement apparatus and method that can be used in the field of producing, compressing, transmitting, and displaying images.

The 3D image compression system compresses color video and depth video. On the other hand, compression for color images can be efficiently compressed by the same methods as H.264 / AVC, H.264 / MVC, and HEVC, but the depth characteristics are significantly different from those of color images.

The existing compression (encoding) standards for video compression are H.261, H.263, MPEG-1, MPEG-2, MPEG-4, H.264 and HEVC (High Efficiency Video Coding).

Although the existing compression standards vary slightly, they generally consist of similar structures including motion estimation and compensation, transcoding, and entropy encoding.

In particular, H.264 and HEVC are known to minimize the block boundary distortion existing in the reconstructed image, thereby improving not only the subjective image quality but also more precise prediction in the motion estimation and compensation process, thereby improving the overall coding efficiency.

The deblocking filter has a good performance at a low bit rate image, but has little performance at a high image quality, or degrades a coding performance.

The adaptive loop filter (ALF) adopted in recent compression standards minimizes errors between the reconstructed image and the original image. When applied as a deblocking filter in a high-quality image, the adaptive loop filter (ALF) It is effective.

A typical adaptive loop filter was a Wiener filter based restoration filter.

Recently, an adaptive loop filter has been proposed to improve the objective image quality in the next stage of the deblocking filter.

The apparatus for improving image quality according to an exemplary embodiment includes a gradient covariance matrix calculation unit for calculating a gradient covariance matrix from a color image and a gradient covariance matrix calculation unit for calculating a gradient covariance matrix using the gradient covariance matrix And a weighted average calculator for calculating a weighted average from the calculated kernel and restoring the pixels of the color image.

The gradient covariance matrix calculation unit may calculate a gradient matrix of the color image, calculate a gradient matrix of the depth image corresponding to the color image, calculate the gradient matrix of the color image, The gradient covariance matrix may be calculated from a gradient matrix of the input depth image and a gradient matrix of the calculated depth image.

The gradient covariance matrix calculation unit may calculate the gradient covariance matrix by summing a gradient matrix of the calculated color image and a gradient matrix of the calculated depth image, Can be calculated.

The kernel operation unit may calculate a kernel expressing a weight between a center pixel and a neighboring pixel using the calculated gradient covariance matrix.

The apparatus for enhancing image quality according to an exemplary embodiment includes a gradient covariance matrix calculation unit for calculating a gradient covariance matrix from a depth image, a gradient covariance matrix calculation unit for calculating a gradient covariance matrix using a gradient covariance matrix, And a weighted average calculator for calculating a weighted average from the calculated kernel and restoring pixels of the depth image, wherein the gradient covariance matrix calculator calculates the weighted average of the depth image A gradient matrix of the depth image is calculated, a gradient matrix of the color image corresponding to the depth image is calculated, and a gradient matrix of the calculated depth image and a gradient matrix of the calculated color image are calculated. From the gradient matrix, One can calculate the acid matrix (gradient covariance matrix).

The gradient covariance matrix calculation unit may calculate the gradient covariance matrix by summing a gradient matrix of the calculated depth image and a gradient matrix of the calculated color image, Can be calculated.

According to an embodiment of the present invention, there is provided a method of improving image quality, comprising the steps of: calculating a gradient covariance matrix from a color image in a gradient covariance matrix calculation unit; calculating a gradient covariance matrix using the calculated gradient covariance matrix; Calculating a weighted average from the calculated kernel and restoring the pixels of the color image in the weighted average calculation unit.

The step of calculating the gradient covariance matrix according to an exemplary embodiment may include the steps of calculating a gradient matrix of the color image, a gradient matrix of the depth image corresponding to the color image, And calculating the gradient covariance matrix from the gradient matrix of the calculated color image and the gradient matrix of the calculated depth image (step < RTI ID = 0.0 > .

The step of calculating the gradient covariance matrix according to an exemplary embodiment may include calculating a gradient covariance matrix by summing a gradient matrix of the calculated color image and a gradient matrix of the calculated depth image, And calculating a gradient covariance matrix.

The calculating the kernel according to an exemplary embodiment may include calculating a kernel representing a weight between a center pixel and a neighboring pixel using the calculated gradient covariance matrix.

1 is a diagram showing a gradient of each of a color image and a depth image in a three-dimensional image.
2 is a block diagram illustrating an image quality enhancement apparatus according to an exemplary embodiment of the present invention.
3 is a block diagram illustrating an embodiment in which an image quality improving apparatus operates in an in-loop position of an encoder.
4 is a block diagram illustrating an embodiment in which an image quality improving apparatus operates in an in-loop position of a decoder.
5 is a block diagram illustrating an embodiment in which the image quality improving apparatus operates in a post-processing position of the decoder.
FIG. 6 is a flowchart illustrating an image quality improvement method according to an exemplary embodiment.
FIG. 7 is a flowchart illustrating a process of calculating a gradient covariance matrix in the embodiment of FIG. 6 in more detail.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

In the following description of the embodiments, detailed description of known functions and configurations incorporated herein will be omitted when it may make the gist of the embodiments unnecessarily obscure. The terminologies used herein are terms used to properly represent embodiments, which may vary depending on the user, the intention of the operator, or the practice of the field to which the technology belongs. Accordingly, the definitions of the terms should be based on the contents throughout the specification. Like reference symbols in the drawings denote like elements.

FIG. 1 is a diagram illustrating gradients 111 and 121 of a color image 110 and a depth image 120, respectively, in a three-dimensional image.

The color image 110 is represented by a color separation with respect to a boundary between an object and a background. The gradient 111 that distinguishes an object from a background can be expressed by an eigenvalue and an eigenvector.

The depth image 120 represents the distance (depth) of each pixel on the z-axis, and may be expressed by a depth value corresponding to the color image 110.

Accordingly, the depth image 120 may be represented by a gradient 121 corresponding to the gradient 111 of the color image 110 on the z-axis.

In order to represent a three-dimensional image, a color image 110 and a depth image 120 are required.

An apparatus and method for enhancing image quality according to an embodiment of the present invention may include a step of generating a color image 110 by reflecting a gradient 121 of a depth image 120 corresponding to a color image 110 in a gradient 111 of a color image 110, The image quality of the color image 110 can be improved.

The apparatus and method for enhancing image quality according to another embodiment reflects the gradient 111 of the color image 110 corresponding to the depth image 120 in the gradient 121 of the depth image 120, 120 can be restored and the image quality of the depth image 120 can be improved.

Accordingly, by using the apparatus and method for improving image quality according to an exemplary embodiment, it is possible to improve the quality of degraded color images or depth images in the field of producing, compressing, transmitting, and displaying color or depth images.

2 is a block diagram illustrating an image quality enhancement apparatus according to an exemplary embodiment of the present invention.

The apparatus 200 for improving image quality according to an exemplary embodiment may include a gradient covariance matrix calculation unit 210, a kernel calculation unit 220, and a weighted average calculation unit 230.

The gradient covariance matrix calculation unit 210 according to an exemplary embodiment may calculate a gradient covariance matrix from the color image.

The covariance can be interpreted as a value indicating the degree of correlation of two random variables independent of the variance indicating the degree of discreteness of the variable.

If one of the two variables tends to rise, the other value tends to rise, if the covariance value is positive, and conversely if one of the two variables tends to rise If the other value tends to fall, the covariance value is negative.

The gradient covariance matrix calculation unit 210 may calculate a gradient covariance matrix by calculating a correlation of gradient matrices by reflecting characteristics of the covariance.

Specifically, the gradient covariance matrix calculation unit 210 may calculate a gradient covariance matrix using a gradient matrix of the color image and a gradient matrix of the depth image.

For example, the gradient covariance matrix calculation unit 210 may calculate a gradient covariance matrix by reflecting a correlation between a gradient matrix of a color image and a gradient matrix of a depth image .

More specifically, the gradient covariance matrix calculation unit 210 according to an exemplary embodiment may calculate a gradient matrix of a color image.

Also, the gradient covariance matrix calculation unit 210 may calculate a gradient matrix of the depth image corresponding to the color image.

The gradient covariance matrix calculation unit 210 may calculate the gradient covariance matrix from the gradient matrix of the calculated color image and the gradient matrix of the calculated depth image.

For example, the gradient covariance matrix calculation unit 210 may generate a first gradient matrix that is a sum of a gradient matrix of the calculated color image and a gradient matrix of the calculated depth image.

In addition, the gradient covariance matrix calculation unit 210 may generate a second gradient matrix having a transpose relation with the first gradient matrix.

The gradient covariance matrix calculation unit 210 may multiply the generated first gradient matrix and the second gradient matrix to finally output a gradient covariance matrix.

The gradient matrix calculated by the gradient covariance matrix calculation unit 210 from the color image or the depth image can be expressed by Equation (1).

[Equation 1]

Here, G can be interpreted as a gradient matrix.

V ^T can be interpreted as a transpose matrix of a 1 * 1 matrix [V ₁ V ₂ ].

S ₁ , S ₂ can be interpreted as eigenvalues, and V ₁ , V ₂ can be interpreted as eigenvectors.

The gradient matrix calculated by the gradient covariance matrix calculation unit 210 from the color image or the depth image can be expressed by Equation (1).

&Quot; (2) "

는 색상 영상의 그래디언트 행렬과 깊이 영상의 그래디언트 행렬을 모두 반영한 제1 그래디언트 행렬로 해석될 수 있다.

May be interpreted as a first gradient matrix reflecting both the gradient matrix of the color image and the gradient matrix of the depth image.

는 그래디언트 공분산 행렬 계산부(210)가 계산하는 색상 영상의 그래디언트 행렬(gradient matrix)로 해석될 수 있다.

May be interpreted as a gradient matrix of the color image calculated by the gradient covariance matrix calculation unit 210. [

는 그래디언트 공분산 행렬 계산부(210)가 계산하는 깊이 영상의 그래디언트 행렬(gradient matrix)로 해석될 수 있다.

Can be interpreted as a gradient matrix of the depth image calculated by the gradient covariance matrix calculation unit 210. [

Next, the gradient covariance matrix calculation unit 210 can calculate a gradient covariance matrix by calculating a first gradient matrix and a second gradient matrix, which is a transpose matrix of the first gradient matrix, using [Equation 3].

&Quot; (3) "

는 그래디언트 공분산 행렬(gradient covariance matrix)로 해석될 수 있다.here,

Can be interpreted as a gradient covariance matrix.

는 색상 영상의 그래디언트 행렬(gradient matrix)과 깊이 영상의 그래디언트 행렬(gradient matrix)을 모두 반영한 제1 그래디언트 행렬로 해석될 수 있다.

May be interpreted as a first gradient matrix reflecting both the gradient matrix of the color image and the gradient matrix of the depth image.

는 제1 그래디언트 행렬의 전치 행렬인 제2 그래디언트 행렬로 해석될 수 있다.

May be interpreted as a second gradient matrix, which is a transpose matrix of the first gradient matrix.

The kernel calculator 220 according to an embodiment can calculate a kernel using a gradient covariance matrix calculated by the gradient covariance matrix calculator 210. [

More specifically, the kernel computing unit 220 computes a kernel expressing a weight between a center pixel and a neighboring pixel using a gradient covariance matrix calculated by the gradient covariance matrix computing unit 210 can do.

The kernel operation unit 220 according to the embodiment can calculate the kernel using Equation (4).

&Quot; (4) "

는 커널로 해석될 수 있고,

는 전체 합을 1로 만들기 위한 정규화 상수(normalization constant)로 해석될 수 있다.here,

Can be interpreted as a kernel,

Can be interpreted as a normalization constant to make the sum to be one.

는 제1 그래디언트 행렬과 제2 그래디언트 행렬로부터 연산된 그래디언트 공분산 행렬(gradient covariance matrix)로 해석될 수 있다.

May be interpreted as a gradient covariance matrix computed from the first gradient matrix and the second gradient matrix.

Next, the weighting average calculator 230 according to an embodiment may calculate the weighted average from the kernel computed in the kernel computing unit 220. [

In other words, the weighting average calculator 230 according to the embodiment can generate the restored pixel for the color image by calculating the weighted average.

To this end, the weighted average calculator 230 may calculate the weighted average, i.e., the reconstructed pixel, using Equation (5).

&Quot; (5) "

는 가중치 평균, 즉, 복원된 픽셀로 해석될 수 있다.here,

Can be interpreted as a weighted average, i. E., Reconstructed pixel.

는 커널로 해석될 수 있다.

Can be interpreted as a kernel.

는 입력되는 픽셀들로 해석될 수 있다.

Can be interpreted as input pixels.

The weighted average calculator 230 according to an exemplary embodiment may recover the pixels of the color image using the calculated weighted average to improve the image quality of the image.

Thus, by using the image quality improving apparatus 200, it is possible to improve the image quality of the image degraded by the image compression to improve the compression ratio.

The image quality enhancement apparatus 200 according to an exemplary embodiment may calculate each gradient matrix from the inputted color image and the depth image corresponding to the color image, and calculate a gradient covariance matrix from the calculated gradient matrix.

Also, the kernel can be calculated from the calculated gradient covariance matrix, and the weighted average can be calculated from the kernel to be computed, thereby improving the quality of the input color image.

The image quality enhancement apparatus 200 according to another embodiment may calculate each gradient matrix from the inputted depth image and the color image corresponding to the depth image and calculate a gradient covariance matrix from the calculated gradient matrix.

Further, the kernel can be calculated from the calculated gradient covariance matrix, and the weighted average can be calculated from the calculated kernel to improve the image quality of the input depth image.

For this, the gradient covariance matrix calculation unit 210 according to an embodiment can calculate a gradient covariance matrix from the depth image.

The kernel operation unit 220 may calculate a kernel representing a weight between a center pixel and a neighboring pixel using a gradient gradient covariance matrix.

The weighted average calculator 230 may calculate a weighted average from the computed kernel to restore pixels of the depth image.

The gradient covariance matrix calculation unit 210 may calculate a gradient matrix of the depth image and a gradient matrix of the color image corresponding to the depth image.

In addition, the gradient covariance matrix calculation unit 210 may calculate a gradient covariance matrix from a gradient matrix of the calculated depth image and a gradient matrix of the calculated color image, Can be calculated.

In particular, the gradient covariance matrix calculation unit 210 according to an exemplary embodiment may calculate a gradient matrix of an input depth image.

Also, the gradient covariance matrix calculation unit 210 may calculate a gradient matrix of the color image corresponding to the depth image.

The gradient covariance matrix calculation unit 210 may calculate a gradient covariance matrix from a gradient matrix of the calculated depth image and a gradient matrix of the calculated color image.

For example, the gradient covariance matrix calculation unit 210 may generate a first gradient matrix that is a sum of a gradient matrix of the calculated depth image and a gradient matrix of the calculated color image.

In addition, the gradient covariance matrix calculation unit 210 may generate a second gradient matrix having a transpose relation with the first gradient matrix.

The gradient covariance matrix calculation unit 210 may multiply the generated first gradient matrix and the second gradient matrix to finally output a gradient covariance matrix.

3 is a block diagram illustrating an embodiment in which an image quality improving apparatus operates in an in-loop position of an encoder.

The image quality enhancement apparatus according to an exemplary embodiment may be implemented as a component of the in-loop filter 305 of the encoder 300 of the three-dimensional image compression system.

The encoder 300 according to one embodiment includes a predictor 301, a transform and quantization unit 302, an entropy coding unit 303, an inverse quantization and inverse transform unit 304, an in-loop filter 305, 306, and a motion estimation and compensation unit 307.

The encoder 300 of the three-dimensional image compression system according to an exemplary embodiment may perform preprocessing on at least one color image or depth image input thereto.

Specifically, the encoder 300 to which the image quality improving apparatus according to an exemplary embodiment of the present invention is applied may perform a preprocessing filtering function for improving a color image or a depth image by improving the quality of a color image or a depth image.

The predicting unit 301 largely performs intra prediction and inter prediction.

The prediction image output from the predicting unit 301 and the difference image output from the transforming and quantizing unit 302 are combined with the input image to generate a compressed image.

The encoder 300 of the three-dimensional image compression system performs restoration filtering on the compressed image by the in-loop filter 305, stores the resultant image in the picture buffer 306, and outputs the additional information to the entropy coding unit 303 .

In this process, the encoder 300 of the three-dimensional image compression system can perform inverse quantization and inverse transformation on the output of the transform and quantization unit 302 using the inverse quantization and inverse transform unit 304. [

The motion estimation and compensation unit 307 estimates and compensates for the motion of the input image, thereby reducing the noise of the input image.

4 is a block diagram illustrating an embodiment in which an image quality improving apparatus operates in an in-loop position of a decoder.

The apparatus for improving image quality according to an exemplary embodiment may be implemented as a component of an in-loop filter of a decoder of a three-dimensional image compression system.

In other words, as shown in FIG. 4, the apparatus for improving image quality according to an embodiment may be included in the decoder 400 of the three-dimensional image compression system in the form of an in-loop filter.

The decoder 400 of the three-dimensional image compression system according to one embodiment includes an entropy decoding unit 401, an inverse quantization and inverse transform unit 402, a motion estimation and compensation unit 403, a picture buffer 404, And a loop filter 405.

The decoder 400 of the three-dimensional image compression system can receive the bitstream transmitted from the encoder of the compression system in the entropy decoding unit 401 and obtain the additional information from the received bitstream.

In addition, the decoder 400 of the three-dimensional image compression system can restore the color image or the depth image using the obtained additional information in the inverse quantization and inverse transform unit 402, May be stored in the picture buffer 404 via the in-loop filter 405. [

The in-loop filter 405 to which the image quality improving apparatus according to an exemplary embodiment of the present invention is applied may perform a preprocessing filtering function for improving the quality of the color image or the depth image to improve the compression rate of the color image or the depth image.

The motion estimation and compensation unit 403 estimates and compensates for the motion of the input image, thereby reducing the noise of the input image.

5 is a block diagram illustrating an embodiment in which the image quality improving apparatus operates in a post-processing position of the decoder.

The apparatus for improving image quality according to an exemplary embodiment may be implemented as a component of a post filter of a decoder of a three-dimensional image compression system.

The apparatus for improving image quality according to an exemplary embodiment may be implemented as a component of a post filter of a decoder of a three-dimensional image compression system.

In other words, as shown in FIG. 4, the image quality enhancement apparatus according to an embodiment may be included in the decoder 500 of the three-dimensional image compression system in the form of a post filter.

The decoder 500 of the three-dimensional image compression system according to one embodiment includes an entropy decoding unit 501, an inverse quantization and inverse transform unit 502, a motion estimation and compensation unit 503, a picture buffer 504, And a filter 505.

The decoder 500 of the three-dimensional image compression system can receive the bitstream transmitted from the encoder of the compression system in the entropy decoding unit 501 and obtain the additional information from the received bitstream.

In addition, the decoder 500 of the three-dimensional image compression system can restore the color image or the depth image using the obtained additional information in the inverse quantization and inverse transform unit 502, May be input to the picture buffer 504.

In addition, the restored color image or depth image may be generated as an output image through the post filter 505.

The post filter 505 to which the image quality improving apparatus according to an embodiment is applied may perform a pre-filtering function to improve the color image or the depth image by improving the quality of the color image or the depth image.

The motion estimation and compensation unit 503 estimates and compensates for the motion of the input image, thereby reducing the noise of the input image.

FIG. 6 is a flowchart illustrating an image quality improvement method according to an exemplary embodiment.

An image quality enhancement method according to an exemplary embodiment may calculate a gradient covariance matrix from a color image (step 601).

In order to calculate a gradient covariance matrix, an image quality enhancement method according to an exemplary embodiment may calculate a gradient matrix of a color image.

In addition, the image quality improvement method according to an exemplary embodiment may calculate a gradient matrix of a depth image corresponding to a color image.

The method of improving image quality according to an exemplary embodiment may calculate a gradient covariance matrix from a gradient matrix of the color image and a gradient matrix of the calculated depth image.

Specifically, in order to calculate a gradient covariance matrix, a gradient matrix of the calculated color image and a gradient matrix of the calculated depth image are computed according to an embodiment of the present invention. The gradient covariance matrix can be calculated by summing.

The image quality improvement method according to an exemplary embodiment may calculate a kernel using a gradient gradient covariance matrix (step 602).

In particular, the image quality improvement method according to an exemplary embodiment may calculate a kernel representing a weight between a center pixel and a neighboring pixel using a gradient covariance matrix calculated to calculate the kernel.

The method of improving image quality according to an exemplary embodiment may calculate a weighted average from the calculated kernel to restore pixels of the color image (step 603).

According to another embodiment of the present invention, each gradient matrix may be calculated from an input depth image and a color image corresponding to the depth image, and a gradient covariance matrix may be calculated from the calculated gradient matrix.

Also, the image quality improvement method can improve the image quality of the input depth image by calculating the kernel from the calculated gradient covariance matrix and calculating the weighted average from the computed kernel.

To this end, the image quality enhancement method can calculate a gradient covariance matrix from the depth image.

Also, the image quality improvement method can restore the pixels of the depth image by calculating the weighted average from the computed kernel.

FIG. 7 is a flowchart illustrating a process of calculating a gradient covariance matrix in the embodiment of FIG. 6 in more detail.

The method of improving image quality according to an exemplary embodiment may calculate a gradient matrix of a color image (step 701).

Next, an image quality enhancement method according to an exemplary embodiment may calculate a gradient matrix of a depth image corresponding to a color image (step 702).

Also, the image quality improving method may calculate a gradient covariance matrix from the gradient matrix of the calculated color image and the gradient matrix of the calculated depth image (step 703).

For example, the image enhancement method may generate a first gradient matrix that is a sum of a gradient matrix of the calculated color image and a gradient matrix of the calculated depth image.

In addition, the image enhancement method may generate a second gradient matrix having a first matrix matrix and a transposed matrix matrix.

The image quality improving method may multiply the generated first gradient matrix and the generated second gradient matrix to finally output a gradient covariance matrix.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

200: Image quality improvement device
210: gradient covariance matrix calculation unit
220: kernel operation unit
230: weighted average calculation unit

Claims (11)

A gradient covariance matrix calculation unit for calculating a gradient covariance matrix from the color image;
A kernel operation unit for operating a kernel using the calculated gradient covariance matrix; And
Calculating a weighted average from the calculated kernel, and restoring pixels of the color image;
And an image quality enhancement device. The method according to claim 1,
Wherein the gradient covariance matrix calculator comprises:
A gradient matrix of the color image is calculated, a gradient matrix of a depth image corresponding to the color image is calculated, and a gradient matrix of the calculated color image and a calculated gradient of the calculated depth Wherein the gradient covariance matrix is calculated from a gradient matrix of the image. 3. The method of claim 2,
Wherein the gradient covariance matrix calculator comprises:
And the gradient covariance matrix is calculated by summing a gradient matrix of the calculated color image and a gradient matrix of the calculated depth image. The method according to claim 1,
The kernel computation unit,
And calculates a kernel representing a weight between a center pixel and a neighboring pixel using the calculated gradient covariance matrix. A gradient covariance matrix calculation unit for calculating a gradient covariance matrix from the depth image;
A kernel computation unit for computing a kernel expressing a weight between a center pixel and a neighboring pixel using the calculated gradient covariance matrix; And
Calculating a weighted average from the calculated kernel, and restoring pixels of the depth image;
Lt; / RTI >
Wherein the gradient covariance matrix calculator comprises:
A gradient matrix of the depth image is calculated, a gradient matrix of the color image corresponding to the depth image is calculated, and a gradient matrix of the calculated depth image and a gradient matrix of the calculated color Wherein the gradient covariance matrix is calculated from a gradient matrix of the image. 6. The method of claim 5,
Wherein the gradient covariance matrix calculator comprises:
And the gradient covariance matrix is calculated by summing a gradient matrix of the calculated depth image and a gradient matrix of the calculated color image. Calculating, in the gradient covariance matrix calculation unit, a gradient covariance matrix from the color image;
Calculating a kernel in the kernel operation unit using the calculated gradient covariance matrix; And
Calculating a weighted average from the computed kernel and restoring pixels of the color image in a weighted average calculation unit
The image quality enhancement method comprising: 8. The method of claim 7,
Wherein the step of calculating the gradient covariance matrix comprises:
Calculating a gradient matrix of the color image;
Calculating a gradient matrix of a depth image corresponding to the color image; And
Calculating a gradient covariance matrix from a gradient matrix of the calculated color image and a gradient matrix of the calculated depth image;
The image quality enhancement method comprising: 9. The method of claim 8,
Wherein the step of calculating the gradient covariance matrix comprises:
Calculating a gradient covariance matrix by summing a gradient matrix of the calculated color image and a gradient matrix of the calculated depth image;
The image quality enhancement method comprising: 8. The method of claim 7,
The step of computing the kernel comprises:
Calculating a kernel representing a weight between a center pixel and a neighboring pixel using the calculated gradient covariance matrix
The image quality enhancement method comprising: A computer-readable recording medium having recorded thereon a program for performing the method according to any one of claims 7 to 10.

Images