KR100969150B1 - Method and apparatus for image restoration based on psf parameter estimation - Google Patents

Abstract

Disclosed are a method and apparatus for image reconstruction based on PSF parameter estimation. The image reconstruction method includes extracting a zero zero point with respect to an input image; Estimating a point spread function (PSF) parameter using the near-missive point; And reconstructing the input image by using the PSF parameter.

Motion blur, PSF (point spread function), deblurring, near zero point, blur angle, blur length

Description

TECHNICAL AND APPARATUS FOR IMAGE RESTORATION BASED ON PSF PARAMETER ESTIMATION

The present invention relates to an image reconstruction system, and more particularly, to an image reconstruction method and apparatus for reconstructing a motion blurred image using a point spread function (PSF) parameter.

When shooting images with the camera, the inadvertent action of the photographer may cause a so-called motion blur phenomenon, which makes the image invisible and blurry, which is one of the main factors that degrade the quality of the image. It is becoming.

Such a motion blurred image may be represented by a convolution operation of the original image and the motion blur point spread function (PSF). Once the PSF is configured, it is possible to restore the motion blur image using an image restoration algorithm.

Recently, many methods of estimating PSF parameters using periodic properties appearing in the frequency domain of an image have been used. As one method, the PSF parameter may be estimated by binarizing the main smear lines having the brightest values in the Fourier Spectrum of the linear motion blur image. Alternatively, the PSF parameters can be estimated by sharpening and binarizing the Fourier spectral image.

However, these methods have problems that can cause data loss during the transformation or binarization of the Fourier spectrum image, and this problem is a factor that can reduce the accuracy of PSF parameter estimation. The PSF estimation error can have a significant effect on the quality of the reconstructed image because propagation of the error occurs during reconstruction.

The present invention provides a method and apparatus for minimizing data loss during image restoration and restoring to an image close to the original through more accurate estimation of PSF parameters.

The present invention provides an image reconstruction method and apparatus capable of more accurately estimating a PSF parameter using a periodic pattern of a motion blur image in a frequency domain.

In accordance with another aspect of the present invention, there is provided a method of restoring an image, the method comprising: extracting a zero zero point from an input image; Estimating a point spread function (PSF) parameter using the near-missive point; And reconstructing the input image by using the PSF parameter.

Here, the extracting the zero myopic point may include converting the input image into a frequency domain, and extracting a zero myopic point on two-dimensional coordinates of the frequency domain. In the extracting the zero dead point, the v-axis approximation center with respect to the first axis and the u-axis approximation dead center with respect to the second axis may be extracted from a dark line of the frequency domain.

The estimating of the PSF parameter may include calculating a shake angle of the input image by using a property of a right triangle formed by connecting the origin of the two-dimensional coordinates with the v-axis approximation point and the u-axis approximation point. Calculating a shake size with respect to the input image by using the shake angle and an orthogonal distance between the sides facing the origin in the right triangle and configuring the shake angle and the shake size as the PSF parameters. It may include a step.

PSF parameter estimation method according to an embodiment of the present invention, the method comprising the steps of: extracting the zero-missing point with respect to the input image; Estimating the shaking angle using the zero near dead point; Estimating the size of the shake using the shake angle; And configuring the shake angle and the shake size as PSF parameters for restoring the input image.

The extracting the zero dead point may include converting the input image into a frequency domain, and extracting the zero near zero dead point and the second axis from the dark axis of the frequency domain on two-dimensional coordinates. The method may include extracting a u-axis approximation dead point for the U-axis.

The estimating of the shaking angle may include using a property of a right triangle formed by connecting the origin of the two-dimensional coordinates with the v-axis approximation point and the u-axis approximation point, and estimating the shake size. May use an orthogonal distance between the shake angle and a side facing the origin in the right triangle.

An image reconstructing apparatus according to an embodiment of the present invention includes: a zero-missive point extractor extracting zero-missive points with respect to an input image; A PSF estimator for estimating a PSF parameter using the zero near point; And an image restoration unit which restores the input image using the PSF parameter.

According to the present invention, it is possible to provide a high quality image by reconstructing an image close to the original by increasing the accuracy of PSF parameter estimation and improving the deblurring performance of the image.

Hereinafter, a method of estimating a point spread function (PSF) parameter, an image decompression method based on PSF parameter estimation, and an image decompression device according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

The present invention can provide a PSF parameter estimation technique capable of more accurately estimating the PSF parameter using the periodicity of the motion blurred image in the frequency domain.

1 is a flowchart illustrating the entire process of an image restoration method according to an embodiment of the present invention.

The image reconstruction method of the present invention may include the following process.

The image reconstruction method of the present invention receives a motion blur image (hereinafter referred to as an "input image") and converts the input image into a frequency domain (S101).

In this case, in the step S101 of converting the input image into the frequency domain, the input image may be transformed into the frequency domain by using a Discrete Fourier Transform (DFT). When the Discrete Fourier Transform is performed on the input image, periodicity in the frequency domain can be confirmed.

As described above, the step S101 of converting the input image into the frequency domain will be described in more detail with reference to FIG. 2.

The image reconstruction method of the present invention may estimate the PSF parameter using the periodicity of the input image (S102). In this case, the PSF is a function having a shake angle θ and a shake size L as a factor, and may mean a function for expressing a shake amount of the input image.

As described above, the step of estimating the PSF parameter (S102) will be described in more detail with reference to FIG. 3.

The image reconstruction method of the present invention may reconstruct an image through deblurring of a motion blur image by repeatedly deconvolving the PSF parameter and the input image (S103).

2 is a diagram for describing a periodic characteristic of an input image.

The step S101 of converting the input image into the frequency domain may include the following process.

The input image including the motion blur phenomenon may be generated through a convolution operation of the original image and the motion blur PSF, and may be defined as in Equation 1.

In this case, g (x, y) corresponds to an input image, f (x, y) corresponds to an original image, h (x, y) corresponds to a motion blur PSF (point spread function), and * corresponds to a convolution operator.

The motion blur PSF refers to a function representing a spreading degree of a point, and may be expressed in a shaking direction and a degree. That is, the motion blur PSF parameter may be composed of a blur angle and a blur length, and may be defined as in Equation 2.

In this case, L is a shake size and θ corresponds to a shake angle.

A discrete Fourier transform may be performed on the input image in order to confirm a periodic property of the input image in the frequency domain, and the input image that has undergone the discrete Fourier transform may be defined as Equation 3 below.

Where G (u, v) is the Fourier transform of g (x, y), F (u, v) is the Fourier transform of f (x, y), and H (u, v) is h (x, y) Corresponds to the Fourier transform.

Referring to FIG. 2, (a) corresponds to G (u, v), (b) corresponds to F (u, v), and (c) corresponds to H (u, v), and the periodicity of each image through a Fourier transform can confirm. Here, H (u, v) may have a different pattern according to the shaking angle and the shaking size. At this time, the angle of the dark lines appears in a direction orthogonal to the direction of the shake angle θ as the shake angle θ changes. In addition, the distance between two neighboring dark portions may vary according to the change in the shake size (L), and the gap becomes narrower as the shake size (L) is larger and wider as the shake size (L) is smaller.

The image reconstruction method of the present invention may estimate the PSF parameter using the periodic pattern of the input image.

3 is a flowchart illustrating a specific process of estimating a PSF parameter.

Referring to FIG. 3, estimating the PSF parameter (S102) may include the following process.

The estimating of the PSF parameter (S102) extracts a near zero point in the frequency domain of the input image (S301).

In order to find the direction of the dark portion in the frequency domain of the input image, a method of finding the zero near point for each axis using two-dimensional (u-axis, v-axis) coordinates of the frequency domain may be used. In this case, the zero-missive point may mean a point closest to the zero point with respect to the u-axis and the v-axis in the dark portion.

4 to 6 are diagrams for explaining a process of extracting the zero dead point for the u-axis and the v-axis. FIG. 4 illustrates an example of a spectrum of an input image, FIG. 5 illustrates values on a v-axis in which the input image of FIG. 4 is projected in one dimension, and FIG. 6 illustrates a minimum value distribution of neighboring points of FIG. 5. It is shown.

In order to extract the v-axis approximation point 401 of the input image shown in FIG. 4, the periodicity of the input image is measured by projecting a value on the v-axis in one dimension as shown in FIG. 5. As shown in Fig. 5, the change in the gradient of each coordinate value is not constant. Therefore, in order to find the near zero dead point, a method using a change in the slope of the minimum value may be used instead of a simple inflection point.

In FIG. 5, when the process of selecting the minimum value 501 among three adjacent points is performed in the direction of the corresponding axis, the portion represented by the lowest value section 601 in FIG. 501), and this value corresponds to the v-axis zero near point. In the same way, the u-axis approximation dead center can be found.

In FIG. 3, the estimating of the PSF parameter (S102) may estimate the shaking angle θ of the input image by using the extracted v-axis approximation and u-axis approximation points (S302). .

The v-axis approximation dead point and the u-axis approximation dead point extracted in step S301 may be expressed as shown in FIG. 7, where A corresponds to the v-axis approximation dead point and B corresponds to the u-axis approximation dead point. .

As shown in FIG. 7, the shaking angle of the input image using the property of a right triangle formed by connecting the origin O, the v-axis approximation point A, and the u-axis approximation point B, θ), and the shaking angle θ may be defined as shown in Equation 4.

3, in the estimating of the PSF parameter (S102), the shake size (L) of the input image may be estimated using the calculated shake angle θ (S303).

The shake size (L) may be calculated using the shake angle (θ) and the orthogonal distance (d) between the sides facing the origin (O) in the right triangle of FIG. 7, can be defined as shown in Equation 5 have.

Here, d corresponds to a blur distance and N corresponds to an image size.

The blur distance d may mean an orthogonal distance between adjacent dark portions in the frequency domain of the input image, and in the present invention, an orthogonal distance between the first period at the origin O of the right triangle may be used. have. A shake size L of the input image may be calculated based on a ratio of the blur distance d and the size N of the input image.

The shake angle θ and the shake size L calculated as described above may be configured as the parameters of the motion blur PSF, and then the image may be reconstructed through the deconvolution operation of the PSF parameter and the input image.

8 is a diagram illustrating an internal configuration of an image reconstruction device according to an embodiment of the present invention.

Referring to FIG. 8, an image reconstruction device according to an embodiment of the present invention may include an image converter 801, a zero dead point extractor 802, a PSF estimator 803, and an image reconstructor 804. Can be.

The image converter 801 may convert the input image into the frequency domain through a Discrete Fourier Transform (DFT) to check the periodic property of the input image in the frequency domain. That is, when the discrete Fourier transform is performed on the input image, the periodic pattern of the input image may be checked.

The zero dead point extractor 802 may extract zero dead points on two-dimensional coordinates, that is, the u-axis and the v-axis, in order to find a dark direction in the frequency domain of the input image. The direction of the dark portion of the frequency domain may be found by using a method of finding the zero near point in the periodic pattern of the input image. The zero dead point extraction unit 802 may extract the zero dead point on the u axis and the v axis by using a gradient change of a minimum value among the coordinate values of the u axis and the v axis.

The PSF estimator 803 may estimate the shake angle and the shake size of the input image as parameters of the motion blur PSF using the zero near point. The PSF estimator 803 may include an angle estimator, a magnitude estimator, and a PSF component.

The angle estimating unit may estimate the shaking angle of the input image by using a property of a right triangle formed by connecting the origin of the two-dimensional coordinates, the u-axis approximation point and the v-axis approximation point. The magnitude estimator may estimate the shake size of the input image using the shake angle and an orthogonal distance between the sides facing the origin in the right triangle. The PSF component may configure the shake angle and the shake size indicating the shake degree of the input image as parameters of the motion blur PSF.

The image reconstructor 804 may reconstruct the input image using the PSF parameter configured with the shake angle and the shake size. The image reconstructor 804 may reconstruct an image by debluring the input image by repeatedly performing a deconvolution operation on the PSF parameter and the input image.

Therefore, the present invention can find the periodic pattern of the motion blur image by using the method of extracting the zero near point without modification of the original data, and can reconstruct the image by accurately estimating the parameter of the motion blur PSF. That is, since the accuracy of the PSF parameter estimation is closely related to the image reconstruction performance, the image reconstruction performance can be improved by minimizing the propagation of errors generated during the image reconstruction process.

The image restoration method according to the present invention may be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

1 is a flowchart illustrating the entire process of an image restoration method according to an embodiment of the present invention.

2 is a diagram for describing a periodic feature of a motion blur image.

3 is a flowchart illustrating a specific process of estimating a PSF parameter.

4 to 6 are diagrams for explaining a process of extracting the near-missive point of the motion blur image.

7 is a diagram for explaining a process of estimating a PSF parameter.

8 is a diagram illustrating an internal configuration of an image reconstruction device according to an embodiment of the present invention.

<Explanation of symbols for main parts of the drawings>

801: an image converter

802: Younggeun dead point extraction unit

803: PSF estimator

804: image restoration unit

Claims (19)

Extracting a near zero point, which is a point closest to a zero point with respect to each axis of two-dimensional coordinates among the dark portions of the input image in the frequency domain of the input image; Estimating a point spread function (PSF) parameter using the near-missive point; And, Restoring the input image using the PSF parameter; Estimating the PSF parameter, Estimating a shaking angle of the input image using the zero near dead point; Estimating a shake size of the input image using the shake angle; And configuring the shake angle and the shake size with the PSF parameters. The method of claim 1, Extracting the zero dead point, Converting the input image into a frequency domain; And extracting a zero myopic point on two-dimensional coordinates of the frequency domain. The method of claim 2, Extracting the zero near point on the two-dimensional coordinates, And a v-axis approximation center with respect to a first axis and a u-axis projection point with respect to a second axis in a dark line of the frequency domain. The method of claim 3, The v-axis near-missive point and u-axis near-missive point, Extracting by using a gradient change of the minimum of the coordinate values on the two-dimensional coordinates. delete The method of claim 3, Estimating the PSF parameter, Calculating a shaking angle with respect to the input image by using a property of a right triangle formed by connecting the origin of the two-dimensional coordinates with the v-axis approximation point and the u-axis approximation point; Calculating a shake size of the input image by using an orthogonal distance between the shake angle and a side facing the origin in the right triangle; And configuring the shake angle and the shake size with the PSF parameters. Extracting a zero myopic point that is a point closest to a zero point with respect to each axis of two-dimensional coordinates among the dark portions of the input image in the frequency domain of the input image; Estimating the shaking angle using the zero near dead point; Estimating the size of the shake using the shake angle; And, Configuring the shake angle and the shake size as PSF parameters for restoring the input image. PSF parameter estimation method comprising a. The method of claim 7, wherein Extracting the zero dead point, Converting the input image into a frequency domain; Extracting a v-axis approximation point for a first axis and a u-axis approximation point for a second axis in a dark portion of the frequency domain on two-dimensional coordinates. The method of claim 8, The v-axis near-missive point and u-axis near-missive point, PSF parameter estimation method for extracting using the change in the slope of the minimum value of the coordinate values on the two-dimensional coordinates. The method of claim 8, Estimating the shaking angle, And using the properties of a right triangle formed by connecting the origin of the two-dimensional coordinates with the v-axis approximation and u-axis approximation. The method of claim 10, Estimating the size of the shaking, And orthogonal distance between the shake angle and the side facing the origin in the right triangle. Extracting a near zero point, which is a point closest to a zero point with respect to each axis of two-dimensional coordinates among the dark portions of the input image in the frequency domain of the input image; Estimating a point spread function (PSF) parameter using the near-missive point; And, Restoring the input image using the PSF parameter; Estimating the PSF parameter, Estimating a shaking angle of the input image using the zero near dead point; Estimating a shake size of the input image using the shake angle; And configuring the shake angle and the shake size with the PSF parameters, the computer readable recording medium having recorded a program for executing the steps. An approximate near point extraction unit for extracting a near zero point, which is a point closest to a zero point for each axis of two-dimensional coordinates among the dark portions of the input image in the frequency domain of the input image; A PSF estimator for estimating a PSF parameter using the zero near point; And, And an image restoration unit for restoring the input image using the PSF parameter. The PSF estimator, An angle estimating unit estimating a shaking angle of the input image using the zero near point; A size estimator for estimating a shake size of the input image using the shake angle; And a PSF component configured to configure the shake angle and the shake size using the PSF parameters. The method of claim 13, And an image converter configured to convert the input image into a frequency domain. The zero-missive point extraction unit, And extracting a zero myopic point on two-dimensional coordinates of the frequency domain. The method of claim 14, The zero-missive point extraction unit, And a v-axis approximation center with respect to a first axis and a u-axis projection point with respect to a second axis in a dark portion of the frequency domain. The method of claim 15, The zero-missive point extraction unit, And extracting the v-axis approximation and u-axis approximation points using a change in the slope of a minimum value among coordinate values on the two-dimensional coordinates. delete The method of claim 15, The angle estimating unit, And calculating a shake angle of the input image by using a property of a right triangle formed by connecting the origin of the two-dimensional coordinates with the v-axis approximation point and the u-axis approximation point. The method of claim 18, The size estimating unit, And a shake size of the input image is calculated using the orthogonal distance between the shake angle and a side facing the origin in the right triangle.

Images