KR100889935B1 - Method and apparatus for sharpening blurred images by radial distortion compensation - Google Patents

Abstract

A sharpening method and an apparatus thereof consisting of the digital image output unit outputting a lens distortion compensation image inputted through the digital video input part of an image are provided to obtain clear image by removing blur phenomenon of lens distortion by using outline adaptive clear method. A main outline direction calculator(104) selects one direction between the angular orientation and one direction the major contour line. The difference value of the major contour line direction is saved. The first radio frequency band output unit(108) finds the high frequency substitution filtering value from the difference value of the major contour line direction. A second high frequency substitution output unit(110) finds the general high frequency substitution filtering value. A lens distortion compensation parameter calculator(112) produces the lens distortion correction parametric value. The adaptive digital filtering part(114) adaptively mixes the high frequency substitution filtering value and the general high frequency substitution filtering value according to the contour line deviation from direction. The calculated lens correction parameter value is reflected and the digital filtering value is obtained.

Description

METHOD AND APPARATUS FOR SHARPENING BLURRED IMAGES BY RADIAL DISTORTION COMPENSATION}

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for sharpening a digital image in which blurring occurs due to correction of a lens distorted image, and in particular, an edge of a corrected digital image is ringed through a digital image processing technique. A method and apparatus for sharpening a blurred image with lens distortion correction suitable for obtaining a sharp image free from phenomena or artifacts.

The present invention is derived from the research conducted as part of the IT growth engine technology development project of the Ministry of Information and Communication and the Ministry of Information and Communication Research and Development. [Task Management Number: 2006-S-044-02, Title: Multicore CPU and MPU-based Crot Platform game technology development].

In order to produce a magnificent screen when producing high-resolution video content, a camera often photographs an image at full aperture. In this case, the peripheral portion of the input image is severely distorted by camera lens distortion having a short focal length. Therefore, it is necessary to correct the distortion of the peripheral part of the input image. For this correction, the input image is interpolated again according to the lens distortion variable. In this process, the image blur occurs.

Such a method of sharpening the contour in a blurred still image or a moving image has already been applied to a plurality of inventions such as Korean Application No. 10-2003-0069837 and Korean Application No. 10-2002-7003343. The contour sharpening method proposed in the above-described conventional invention determines whether or not to sharpen the contour area at a predetermined threshold. In addition, when the sharpening of the contour is performed by the threshold value set by the user as a result of the determination, the sharpening portion and the non-sharpening portion are not smoothly connected, so that the degradation may be easily detected in the corrected image. There is still a wave or staircase that occurs as a side effect.

To solve this problem, the patent of Korean Patent Application No. 10-2004-0094267 uses a finite impulse response (FIR) digital filter to obtain a corrected image with sharp contours without deterioration. It describes that the outline can be made clear without any phenomenon or step phenomenon.

In the case of performing the sharpening of the contour on the blurred image by the correction of the camera lens distortion with the finite impulse response digital filter proposed in the prior art operating as described above, the portion where the lens distortion correction is applied and the distortion correction is less Because there is no consideration of the applied part, the wave or stair phenomenon is more prominent in the part where the distortion correction is applied, so it is difficult to sharpen the blurred image of the lens distortion corrected by the general outline sharpening method proposed in the above patent. Had a problem.

Accordingly, the present invention provides a method for sharpening a blurred image by lens distortion correction, which can remove a blur caused by a digital lens distorted by a camera lens through digital filtering reflecting a lens distortion correction parameter and perform contour sharpening. And an apparatus.

In addition, the present invention by using the lens distortion correction parameter and the contour adaptive digital filter to eliminate the blurring of the image generated during lens distortion correction to eliminate the blurring of the image generated during image correction without the wave or step phenomenon to obtain a clear image The present invention provides a method and apparatus for sharpening a blurred image by correcting lens distortion.

According to an embodiment of the present invention, a first step of obtaining a difference value corresponding to each direction using the input digital filter and the brightness of an image, and obtaining a difference value in the main contour direction and the main contour direction among them, A second step of obtaining a high frequency band filtering value according to the main contour direction obtained in the first step, a third step of obtaining a general high frequency band filtering value, and using the digital filter and the brightness of the image in each direction. A fourth step of calculating a deviation of the main contour direction with respect to the corresponding difference value; a fifth step of calculating a lens distortion correction parameter value; and the high frequency band filtering value h1 and the general high frequency force filtering value h2. Adaptive mixing according to the deviation of the main contour direction with respect to the difference value corresponding to each direction, the lens distortion correction parameter And a sixth step of obtaining a final digital filtering value by reflecting the data value.

An embodiment of the present invention is a device for sharpening a blurred image by lens distortion correction comprising a digital image output unit for outputting a lens distortion correction image input through a digital image input unit, the digital image input through the digital image input unit And a digital image processor configured to calculate a main contour direction of the image to perform contour sharpening of the input image.

In the present invention, the effects obtained by the representative ones of the disclosed inventions will be briefly described as follows.

According to the present invention, a sharp image can be obtained by eliminating blurring in an image in which lens distortion is corrected without side effects such as ripples or staircases by using the contour adaptive sharpening method considering the lens distortion correction parameter. .

In addition, it is possible to produce a clear video content by removing the blur generated in the process of performing distortion correction of the camera lens when producing a video content through a high-resolution (HD class) camera input image that requires high definition image quality. Most of the video content to which the present invention is applied will be general live-action video, but even when producing video content using CG (Computer graphic) technology, a real-life video is used as a background image to insert a CG object, or even animation content is produced. Often take photorealistic images as reference images. Video content using CG technology includes theater movies, CG animations, and demo videos to promote game titles. By utilizing the present invention in these fields, there is an effect that can produce a clear HD video content without deterioration of image quality.

Hereinafter, the operating principle of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, when it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. Terms to be described later are terms defined in consideration of functions in the present invention, and may be changed according to intentions or customs of users or operators. Therefore, the definition should be made based on the contents throughout the specification.

The present invention is to remove the blur caused by the digital lens reflecting the lens distortion correction parameter by correcting the digital image distorted by the camera lens and to perform contour sharpening.

1 is a block diagram showing the structure of a digital image processing unit according to a preferred embodiment of the present invention, while FIG. 4 is an arbitrary digital filter composed of a 3 by 3 mask according to a preferred embodiment of the present invention, and FIG. Shows the intensity of the digital input image in the 3 by 3 area according to the preferred embodiment of the present invention, and FIGS. 6A to 6D show respective directions (0 °, 45 °, 90 °, 135 °) shows a digital filter showing a contour value, and FIG. 7 is a diagram showing a general high frequency digital filter used in the second high frequency band calculating unit according to the preferred embodiment of the present invention.

Referring to FIGS. 1 and 4 to 7, the present invention provides a digital image input unit 102 for receiving a lens distortion corrected digital image, and reflects a lens distortion correction parameter indicating a degree of correction of the input image. And a digital image output unit 116 for displaying the processed image to the user.

An image provided to the digital image input unit 102 may be a digital image obtained from a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) or a digital image generated using a tool such as Mental Ray or RenderMan. The digital image has been corrected for lens distortion.

The digital image processing unit 100 includes a digital filter, and processes digital distortion-corrected digital images input through the digital image input unit 102. The digital image processing unit 100 extracts and extracts brightness of an image from the input digital image. A high frequency band filtering value is calculated according to the main contour direction calculation unit 104 and the main contour direction calculation unit 104 which calculate the difference value in the main contour direction and the main contour direction based on the brightness of the image. A first high frequency band calculating unit 108, a second high frequency band calculating unit 110 for obtaining a general high frequency band filtering value, a contour direction deviation calculating unit 106 for obtaining a deviation in the contour direction, and a lens distortion correction parameter value for calculating An adaptive digital fill that performs adaptive digital filtering through the lens distortion correction parameter calculator 112 and the digital filter. And a ring (114).

4 to 6D, the first high frequency band calculator calculates a difference value d _i corresponding to each direction by using the digital filter input through the main contour direction calculator 104 and the brightness of the image. The method obtained by (108) is shown in Equation 1 below.

In Equation 1, for example, h (0, 0) is a filter at a corresponding coordinate (0, 0), and Y (0, 0) is a brightness value at a corresponding coordinate (0, 0).

6A to 6B, the difference values obtained in the respective directions (0 °, 45 °, 90 °, 135 °) are defined as d ₁ , d ₂ , d ₃ , and d ₄ , respectively, and the main contour direction and The difference value can be obtained as shown in Equation 2.

The first high frequency band calculator 108 determines the difference value of the direction having the largest value among the difference values obtained in the respective directions obtained from the main contour direction calculation unit 104 as the main contour direction.

6A is a digital filter applied at 0 °, FIG. 6B is a digital filter applied at 90 °, FIG. 6C is a digital filter applied at 135 °, and FIG. 6D is a digital filter applied at 45 °. Therefore, when FIG. 4 is applied to FIGS. 6A to 6D, it can be expressed as follows.

First, when FIG. 4 is applied to the digital filter of FIG. 6A showing the contour value at 0 °, the output value of the digital filter is as follows. h (0,0) = 1, h (0,1) = 1, h (0,2) = 1, h (1,0) = 0, h (1,1) = 0, h (1,2 ) = 0, h (2, 0) = -1, h (2, 1) = -1, h (2, 2) = -1 can be represented.

4 is applied to the digital filter of FIG. 6B showing the contour value at 90 °, the output value of the digital filter is as follows. h (0,0) = 1, h (0,1) = 0, h (0,2) = -1, h (1,0) = 1, h (1,1) = 0, h (1, 2) = -1, h (2, 0) = 1, h (2, 1) = 0, h (2, 2) = -1 can be represented.

When FIG. 4 is applied to the digital filter of FIG. 6C showing the contour value at 135 °, the output value of the digital filter is as follows. h (0,0) = 1, h (0,1) = 1, h (0,2) = 0, h (1,0) = 1, h (1,1) = 0, h (1,2 ) = -1, h (2, 0) = 0, h (2, 1) = -1, h (2, 2) = -1 can be represented.

In addition, when FIG. 4 is applied to the digital filter of FIG. 6D representing the contour value at 45 °, the output value of the digital filter is as follows. h (0,0) = 0, h (0,1) = 1, h (0,2) = 1, h (1,0) = -1, h (1,1) = 0, h (1, 2) = 1, h (2, 0) = -1, h (2, 1) = -1, h (2, 2) = 0 can be represented.

The first high frequency band calculating unit 108 obtains a difference value of the direction having the largest value among the difference values for each direction obtained from the main contour direction calculating unit 104 as shown in Equation 2, and Using the high frequency band filtering value h1 calculated by the first high frequency band calculating unit 108 as shown in Equation 3, the center point 1 and 1 of the shift point in the main contour direction and the line coordinate of the desired point Can be expressed using the brightness value. Equation 3 is as follows.

In addition, the second high frequency band calculator 110 receives the digital distortion corrected digital image from the digital image input unit 102 to obtain a general high frequency band filtering value. The general high frequency band filtering value h2 performed by the second high frequency band calculating unit 110 may be expressed by Equation 4 as follows. At this time, the digital filter used is as shown in FIG.

As such, when the processes of Equation 3 processed by the first high frequency band calculating unit 108 and Equation 4 processed by the second high frequency band calculating unit 110 are completed, the contour direction deviation calculating unit 106 is completed. ) Calculates the variance values dw and VAR for the difference value D corresponding to each direction using the digital filter obtained by the main contour direction calculating unit 104 and the brightness of the image. The dispersion value performed by the contour direction deviation calculator 106 is obtained through a process as shown in Equation 5, and Equation 5 is expressed as follows.

In Equation 5, the E value means an expected value, that is, an average.

The lens distortion correction parameter calculating unit 112 is a value indicating the degree to which lens distortion correction is applied and has a larger value from the center of the corrected image toward the periphery, and the lens distortion correction parameter r is expressed as in Equation 6 below. do.

Where k is any constant and the value can be, for example, 0.05. In addition, c _x , and c _y represent center points of the input image. For example, when the input image has a resolution of 640x480, c _x , has a value of 320, and c _y has a value of 240. x is an arbitrary x direction position in an input image, and 0 <= x <= 640. Similarly, y is an arbitrary position in the input image and has a range of 0 <= y <= 320.

As shown in FIG. 6, the adaptive digital filtering unit 114 calculates a contour direction deviation calculator from the filtered value filtered by the first high frequency band calculator 108 and the value filtered by the second high frequency band calculator 110. Adaptive mixing according to the main contour direction deviation calculated by Equation 5 at (106), and reflects the lens distortion correction parameter calculated from the lens distortion correction parameter calculation unit 112 to obtain the final digital filtering value. Acquire.

The process of obtaining a brightness value and a parameter for adaptively mixing the filtered value filtered from the first high frequency band calculator 108 and the value filtered by the second high frequency band calculator 110 according to the main contour direction deviation It can be represented by the following Equation 7, and Equation 7 is as follows.

Where k is any constant and the value may be, for example, 0.025.

After obtaining the main contour direction and considering the deviation of the main contour direction, the high frequency band filtering value according to the main contour direction is mixed with the general high frequency band filtering value, and then considering the lens distortion correction parameter (r). As shown in the figure, the brightness of the output image Yout (i, j) at each point and a, which is a coefficient used for adaptive filtering, are obtained. Here, a is a parameter for adjusting the internal value of the high frequency band filtering value h1 and the general high frequency band filtering value h2 of the difference value in the direction having the largest value. In practice, a is a value for determining whether to use the directional high frequency filtering value h1 of FIG. 5 or the general Laplacian high frequency filtering value h2 of FIG. 6 according to the area of the input image. This ratio value is used to implement the contour adaptive sharpening method.

That is, in the case of general high frequency band filtering, the 2D convolution sum of the filter of FIG. 7 and the input image of FIG. 5 is obtained. For example, if the input image Y in the 3x3 region has a value of [[10 20 30] [10 20 20] [20 10 10]] as follows, the general high frequency band filter [[-0.25 -0.25 -0.25] of FIG. The sum of convolutions with [-0.25 1 -0.25] [-0.25 -0.25 -0.25] is as follows. (-0.25 * 10) + (-0.25 * 20) + (-0.25 * 30) + (-0.25 * 10) + (1 * 20) + (-0.25 * 20) + (-0.25 * 20) + (- 0.25 * 10) + (-0.25 * 10) = -12.5

In addition, the high frequency band filtering value h1 is obtained by calculating the main outline direction in Equations 1 and 2, and is calculated through Equation 3. When di is obtained in Equation 1, as in general high frequency band filtering, d1, d2, d3, d4, and 4 main contour directions of high frequency filters are respectively applied to the input image Y and the four direction high frequency filters of FIGS. 6A to 6D. The filtering value is obtained by adding the two-dimensional convolution.

2 is a flowchart illustrating an operation procedure for sharpening a blur of a lens distortion corrected image according to an exemplary embodiment of the present invention.

Referring to FIG. 2, in operation 200, the main contour direction calculating unit 104 extracts the brightness of an image from the lens distortion-corrected digital image input from the digital image input unit 102, and uses the digital filter and the brightness of the extracted image. The difference values d _i and D corresponding to the respective directions are obtained, and the difference values in the main outline direction d and the main outline direction are obtained therefrom. In this case, the contour direction is divided into 0 °, 45 °, 90 ° and 135 °, and the contour direction can be obtained through Equation 2 above.

According to the main contour direction d obtained in step 202, the first high frequency band calculator 108 obtains the high frequency band filtering value h1 using Equation 3 described above, and in step 204, the second high frequency band. The band calculator 110 obtains a general high frequency band filtering value h2.

In operation 206, the contour direction deviation calculator 106 uses the digital filter input through the main contour direction calculator 104 and the brightness of the image to determine the deviation of the main contour direction with respect to the difference value D in each direction ( dw).

Subsequently, in step 208, the lens distortion correction parameter is calculated through the lens distortion correction parameter calculating unit 112. In step 210, the adaptive digital filtering unit 114 determines the high frequency band filtering value h1 obtained in step 202 and in step 204. The obtained general high frequency band filtering value (h2) is adaptively mixed according to the deviation (dw) of the main contour direction with respect to the difference value (D) corresponding to each direction obtained in step 206, and then the lens distortion correction parameter r ) To obtain the final digital filtering value. In this case, the a value obtained in Equation 7 is used to adaptively mix.

3 is a flowchart illustrating an operation procedure for calculating a main contour direction according to FIG. 2 of the present invention.

Referring to FIG. 3, in step 300, the difference value d1 in the horizontal direction is calculated. In this case, the difference value d1 in the horizontal direction is represented by an angle and corresponds to FIG. 6A.

In step 302, the difference value d2 in the vertical direction is calculated. In this case, the difference value d2 in the vertical direction is represented by an angle and corresponds to FIG. 6B.

In step 304, the difference value d3 in the diagonal direction is calculated. Here, when the difference value d3 in the diagonal direction is displayed to be angled, it corresponds to 135 ° and corresponds to FIG. 6C.

In step 306, the difference value d4 in the opposite diagonal direction is calculated. Here, when the difference value d4 in the reverse diagonal direction is represented by an angle, it corresponds to 45 ° and corresponds to FIG. 6D.

Thereafter, in step 308, the direction of the difference value having the maximum value among the difference values in each direction is determined as the main contour direction.

As described above, the present invention corrects the digital image distorted by the camera lens, thereby eliminating the blurring generated through digital filtering reflecting the lens distortion correction parameter, and performing contour sharpening.

Meanwhile, in the detailed description of the present invention, specific embodiments have been described, but various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the scope of the following claims, but also by those equivalent to the scope of the claims.

1 is a block diagram showing the structure of a digital image processing unit according to an embodiment of the present invention;

2 is a flowchart illustrating an operation procedure for sharpening a blur of a lens distortion corrected image according to an exemplary embodiment of the present invention;

3 is a flowchart illustrating an operation procedure for calculating a main contour direction according to FIG. 2 of the present invention;

4 illustrates an arbitrary digital filter comprised of a 3 by 3 mask according to a preferred embodiment of the present invention;

5 is a diagram illustrating brightness of a digital input image in a 3 by 3 region according to an embodiment of the present invention;

6a to 6d are diagrams showing digital filters showing contour values for respective directions according to a preferred embodiment of the present invention;

FIG. 7 illustrates a general high frequency digital filter used in a second high frequency band calculating unit according to an exemplary embodiment of the present invention. FIG.

<Brief description of symbols for the main parts of the drawings>

100: digital image processing unit # 102: digital image input unit

104: main contour direction calculating section 106: contour direction deviation calculating section

108: first high frequency band calculating unit 110: second high frequency band calculating unit

112: adaptive digital filtering unit

114: lens distortion correction parameter calculation unit

116: digital video output unit

Claims (12)

delete A device for sharpening a blurred image by lens distortion correction comprising a digital image output unit for outputting a lens distortion correction image input through a digital image input unit, And a digital image processing unit configured to calculate a main contour direction of the digital image input through the digital image input unit and perform sharpening of the contour of the input image. It includes a digital filter, extracts the brightness of the input image, using the digital filter and the brightness of the extracted image to obtain a difference value corresponding to each direction of 0 °, 45 °, 90 ° 135 °, A main contour direction calculation unit which selects one of the directions as a main contour direction and obtains a difference value of the main contour direction; A first high frequency band calculator for obtaining a high frequency band filtering value from the difference value in the main contour direction; A second high frequency band calculating unit obtaining a general high frequency band filtering value; A lens distortion correction parameter calculator configured to calculate a lens distortion correction parameter value; The high frequency band filtering value filtered from the first high frequency band filter and the general high frequency band filtering value filled by the second high frequency band calculating unit are adaptively mixed according to a contour direction deviation, and the calculated lens correction parameter value And an adaptive digital filtering unit for obtaining a digital filtering value by reflecting the reflection. The method of claim 2, The first high frequency band calculator An apparatus for sharpening a blurred image by lens distortion correction, characterized by obtaining a difference value in each direction using the following equation.

Here, di denotes a difference value in each direction, h (i, j) is a filter at a corresponding coordinate, and Y (i, j) is a brightness value at a corresponding coordinate. The method of claim 2, The main contour direction is, The first high-frequency band calculator determines the direction having the largest value among the difference values of the respective directions as the main contour direction, wherein the image distortion blurred by lens distortion correction. The method of claim 2, The first high frequency band calculator, The high frequency band filtering value (h1) is obtained by using the following equation using the difference value in the main contour direction and the brightness value of the coordinate of the center point of the filter.

Where d is the difference value in the direction of the main contour, and Y (1,1) is the brightness value at the center point of the filter. The method of claim 2, The adaptive digital filtering unit, Obtaining a, which is a coefficient used for adaptive filtering, by using the following formula, and using a, the internal frequency of the high frequency band filtering value h1 and the general high frequency band filtering value h2 in the direction having the largest value A device for sharpening a blurred image by lens distortion correction, characterized by filtering by adjusting a value.

Where k is an arbitrary constant. The method of claim 2, The lens distortion correction parameter calculator, The lens distortion correction parameter value, characterized in that for calculating the lens distortion correction parameter value through the following equation.

Here, k is an arbitrary constant, c _x , and c _y represent a center point of the input image, x is an arbitrary x-direction position in the input image, and y is an arbitrary y-direction position in the input image. Obtaining a difference value corresponding to each direction using the input digital filter and the brightness of the image, and obtaining a difference value in the main contour direction and the main contour direction among them; Obtaining a high frequency band filtering value according to the main contour direction obtained in the first step; Obtaining a general high frequency band filtering value; A fourth step of obtaining a deviation of the main contour direction with respect to the difference value corresponding to each direction by using the brightness of the digital filter and the image; Calculating a lens distortion correction parameter value; The high frequency band filtering value h1 and the general high frequency band filtering value h2 are adaptively mixed according to the deviation of the main contour direction with respect to the difference value corresponding to each direction, and reflects the lens distortion correction parameter value. And a sixth step of obtaining a final digital filtering value. The method of claim 8, The main contour direction is, A method of sharpening a blurred image by lens distortion correction, wherein the direction having the largest value among the difference values in each direction is determined as the main contour direction. The method of claim 8, The method, In order to perform the adaptive mixing, a coefficient, which is a coefficient used for adaptive filtering, is obtained by the following equation, and the high frequency band filtering value h1 and the general high frequency band in the direction having the largest value are obtained using the a. A method of sharpening a blurred image by lens distortion correction, characterized in that the filtering is performed by adjusting the internal value of the filtering value (h2).

Where k is an arbitrary constant. The method of claim 8, Determining the main contour direction of the first step, A first calculating step of calculating a difference value in the horizontal direction; A second calculating step of calculating a difference value in the vertical direction; A third calculating step of calculating a difference value in a diagonal direction; A fourth calculating step of calculating a difference value in the reverse diagonal direction; And a first determining step of determining the direction of the difference value having the maximum value among the difference values in each direction as the main contour direction. The method of claim 8, The fifth step of calculating the lens distortion correction parameter value, A method of sharpening a blurred image by lens distortion correction, which is calculated through the following equation.

Here, k is an arbitrary constant, c _x , and c _y represent a center point of the input image, x is an arbitrary x-direction position in the input image, and y is an arbitrary y-direction position in the input image.

Images