KR101644411B1 - Apparatus and method for restorating of high resolution image - Google Patents

Abstract

The present invention relates to a high-resolution image restoration apparatus and a method thereof, and a high-resolution image restoration apparatus according to an embodiment of the present invention includes a plurality of first images for the same target object in a threshold range for predetermined sub- A determination unit for determining one reference image and at least one candidate image to be processed, and for determining a sub-pixel displacement difference for horizontal, vertical, and rotational components between the reference image and the at least one candidate image, A second image generator for generating a second image having pixel coordinates composed of pixel information of the reference image and the at least one candidate image using a displacement difference; And a final reconstruction image generation unit for generating the reconstructed image.
Accordingly, more precise high-resolution images can be obtained by estimating the motion of the vertical, horizontal, and rotational components between a plurality of low-resolution images.

Description

[0001] APPARATUS AND METHOD FOR RESTORATING HIGH RESOLUTION IMAGE [0002]

The present invention relates to a high-resolution image restoration apparatus and method, and more particularly, to a technique for restoring a single high-resolution image using a plurality of low-resolution images.

With the expansion of communication and multimedia technologies, the demand for high-resolution images is increasing in the market. Here, the meaning of 'increase resolution' means not only an increase in the number of image pixels but also an increase in resolution It means to express it finely. Resolution is evaluated primarily using ISO (International Organization for Standardization) resolution charts. It is based on the resolution of the device, using how precise and precise the line-to-line resolution displayed on the display or print device And the resolution can be determined.

In recent years, digital signal processing has attracted a great deal of attention because it can save the cost of designing and manufacturing while maximizing its effect. This is largely divided into interpolation and high resolution reconstruction techniques. First, interpolation generates a single high-resolution image using a single low-resolution image. When a new pixel data value other than the original image pixel is determined to fill the high-resolution coordinates, It is a concept to generate new data. However, this increased frequency does not result in a revolutionary improvement in resolution due to a linear increase rather than a complete reconstruction of high frequency components.

Next, a method of generating a single high resolution image using a plurality of low resolution images in a so-called super / high resolution reconstruction method. This is similar to the actual data because all the data constituting the high resolution are extracted from the low resolution data. That is, not all data for constructing a high-resolution image is processed but extracted from low-resolution images, so that the actuality of inter-pixel data is very high, and resolution of the restored image is increased, so that a detailed portion can be expressed. However, the computational complexity is so high that it is difficult to commercialize and it is applied to some digital cameras.

Research related to high-resolution reconstruction can be divided into those related to registration techniques and those associated with restoration or a combination of these. Most studies focus on registration. The reason is that registration is the first step, but the result is quite different depending on the result. However, since the registered result is composed of a plurality of sampling frequency components in relation to the restoration of the rear end, techniques for changing it to a single sampling frequency are being studied.

It is difficult to find the sub-pixel displacement when applied to the actual camera image, so that the image quality may be lower than that of the high resolution image by the interpolation method. In fact, the results of Bi-cubic interpolation and the reconstruction of high-resolution images by Bi-cubic interpolation method show that the result of Bi-cubic interpolation is better There are many cases. Some algorithms even show a grid in the reconstructed image at high resolution, and the registration result can not be resolved at high resolution.

BACKGROUND ART [0002] Techniques that serve as a background of the present invention are disclosed in Korean Patent Registration No. 10-0781552 (Nov. 27, 2007).

According to an aspect of the present invention, there is provided a high-resolution image reconstruction apparatus and method for reconstructing a high-resolution image by estimating motion of vertical, horizontal, and rotational components among a plurality of low-resolution images.

The apparatus for reconstructing a high resolution image according to an embodiment of the present invention includes a determination unit that determines one reference image and at least one candidate image included in a threshold range for a predetermined subpixel value among a plurality of first images for the same target object Vertical and rotational components between the reference image and the at least one candidate image, and using the estimated sub-pixel displacement difference to calculate the difference between the reference image and the at least one candidate A second image generator for generating a second image having pixel coordinates composed of pixel information of the image, and a final reconstruction image generator for generating a final reconstruction image by correcting the sampling information of the second image.

The second image is an image having a higher resolution than the first image.

In addition, the reference image may be an image having the largest number of candidate images among the candidate images satisfying the following equation among the first images:

Here, TH is a threshold for a predetermined sub-pixel value, F ' _k (u) is a sub-pixel value of the first image, and F _l (u) is a sub-pixel value of the reference image.

Also, the second image generator may estimate the sub-pixel displacement difference with respect to horizontal, vertical, and rotational components by performing Fourier transform on the reference image and the candidate image, respectively, and comparing the phase and the degree of correlation.

In addition, the sub-pixel displacement difference can be estimated by Fourier transforming the following equation:

Here, x1 is a sub-pixel value of a horizontal component, x2 is a sub-pixel value of a vertical component, and R is a sub-pixel value of a rotation component.

In addition, the final reconstructed image generation unit may apply at least one of a first iterative back projection, a deblurring, and an antialiasing to the second image to correct the second image. have.

A method for reconstructing a high resolution image using an image restoration apparatus according to another embodiment of the present invention includes reconstructing a single reference image included in a threshold range for a predetermined subpixel value among a plurality of first images for the same target object, Vertical and rotational components between the reference image and the at least one candidate image, determining a candidate image of the reference image and the at least one candidate image using the estimated sub-pixel displacement difference, Generating a second image having pixel coordinates composed of pixel information of the at least one candidate image; and correcting the sampling information of the second image to generate a final reconstructed image.

Accordingly, more precise high-resolution images can be obtained by estimating the motion of the vertical, horizontal, and rotational components between a plurality of low-resolution images. In addition, the reliability of the data can be increased by using the Fourier transform to estimate the motion of the rotation component between the plurality of low-resolution images.

1 is a configuration diagram of a high-resolution image restoration apparatus according to an embodiment of the present invention;
FIG. 2 is a flowchart of a high-resolution image restoration method using the high-resolution image restoration apparatus according to FIG. 1;
FIG. 3 is an exemplary diagram for explaining determination of coordinates of a high-resolution image using pixel information of a plurality of low-resolution images in the high-resolution image restoration method according to FIG. 2;
FIG. 4 is a diagram for explaining a graph for identifying a rotation component of a low-resolution image in the high-resolution image reconstruction method according to FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The terms used are terms selected in consideration of the functions in the embodiments, and the meaning of the terms may vary depending on the user, the intention or the precedent of the operator, and the like. Therefore, the meaning of the terms used in the following embodiments is defined according to the definition when specifically defined in this specification, and unless otherwise defined, it should be interpreted in a sense generally recognized by those skilled in the art.

FIG. 1 is a block diagram of a high-resolution image reconstruction apparatus according to an embodiment of the present invention, and FIG. 2 is a flowchart of a high-resolution image reconstruction method using the high-resolution image reconstruction apparatus shown in FIG.

1 and 2, a high-resolution image restoration apparatus 100 according to an embodiment of the present invention includes a high-resolution image restoration apparatus 100, a second image generation unit 120, and a final restoration image generation unit 130).

First, the high resolution image restoration apparatus 100 determines one reference image and at least one candidate image corresponding to a threshold range for a predetermined sub-pixel value among a plurality of first images for the same target object (S210). Here, the plurality of first images are low-resolution images for the same target object, and the number of the first images may be set differently according to the user setting. Also, the reference image and the candidate image mean an image used to register a plurality of first images into one high-resolution image. It is important to determine the reference image and the candidate image because the reference image and the candidate image are restored to a single high resolution image.

In addition, the high resolution image restoration apparatus 100 may set an image having the largest number of candidate images among the candidate images satisfying the following Equation (1) among the first images as a reference image.

In Equation (1), TH represents a threshold for a predetermined sub-pixel value, F ' _k (u) represents a sub-pixel value of the first image, and F _l (u) represents a sub-pixel value of the reference image. Here, the threshold value may be set to a value satisfying the registration condition of the high-resolution image according to the user setting. Of the candidate images satisfying these conditions among the first images, an image As a reference image.

Next, the second image generation unit 120 generates sub-pixel displacement of the reference image and the candidate image using the sub-pixel displacement for the horizontal, vertical, and rotational components between the reference image and the at least one candidate image, (Step S220). Here, the second image means a high-resolution intermediate restored image of the first image. The second image generator 120 estimates motion using a sub-pixel displacement difference between the reference image and the candidate image. In this case, each of the reference image and the candidate image preferably has sub-pixel information. The motion estimation between the reference image and the candidate image can estimate the motion, for example, on a block-by-block basis.

The second image generator 120 performs Fourier transform on each of the reference image and the candidate image to compare the phase and the correlation to generate a subpixel displacement difference for horizontal, Can be estimated. In the case of motion estimation in the frequency domain or the spatial domain, it is possible to estimate the motion on the plane unlike the case in which only the displacement for the horizontal and vertical components is considered or only the rotation component is considered.

Also, the second image generator 120 may estimate the subpixel displacement difference by performing Fourier transform on the following equation (2).

In Equation (2), x1 is a sub-pixel value of a horizontal component, x2 is a sub-pixel value of a vertical component, and R is a sub-pixel value of a rotation component. The Fourier transform for Equation (2) can be expressed as Equation (3) below.

In addition, the absolute value for the mathematical expression 3 can be expressed by the following equation (4).

For example, in the case of using Equation (4), in order to estimate the motion of the rotation component, the candidate image is rotated by a predetermined angle, and then a phase of the original image and a correlation value The value for the rotational motion can be calculated.

Next, the final reconstruction image generation unit 130 generates the final reconstruction image by correcting the sampling information of the second image (S230). Here, the final restored image means a high-resolution final restored image of the first image. The final reconstructed image generating unit 130 may correct the second image by applying at least one of first iterative back projection, deblurring, and anti-aliasing to the second image .

The second image generated by the second image generating unit 120 has pixel coordinates composed of the reference image and the pixel information of the at least one candidate image among the plurality of first images that are low resolution images. In this case, it is difficult to accurately locate the coordinates of the high resolution image. That is, one high-resolution image should be composed of a single sampling frequency, and the second image obtained by registering the first image has a sampling frequency which is not constant or has a plurality of sampling frequencies. Accordingly, when the aliasing that violates the sampling theory occurs, such as when a plurality of sampling frequencies are included in the second image, the final reconstructed image generating unit 130 generates the corrected image.

FIG. 3 is an exemplary diagram for explaining determination of coordinates of a high-resolution image using pixel information of a plurality of low-resolution images in the high-resolution image restoration method according to FIG.

Referring to FIG. 3, the first image 310 includes a reference image 311 and a plurality of candidate images 312. In this case, each reference image 311 and a plurality of candidate images 312 contain subpixel information. In the case of increasing the resolution twice as much as the vertical and horizontal components with respect to the first image 310, a total of four low resolution images are required. In this case, one reference image 311 may be combined with the pixel information of the three candidate images 312 to generate one second image 320. Since the second image 320, which is a higher resolution image than the first image 310, is vertically and horizontally doubled compared to the first image 310, one reference image 311 and three The pixel information of the candidate images 312 is located.

FIG. 4 is a diagram for explaining a graph for identifying a rotation component of a low-resolution image in the high-resolution image reconstruction method according to FIG.

Referring to FIG. 4, there is shown a Fourier transformed graph of an original image and an image rotated by 25 degrees thereto. (a) shows the phase difference between the original image and the rotated image when the original image and the rotated image are Fourier-transformed, respectively. The rotation information of the rotated image can be obtained through the phase difference. (a) In the graph, since the phase difference between the original image and the rotated image is 25 °, it can be seen that the rotated image is rotated by 25 ° with respect to the original image.

(B) shows the degree of correlation between the original image and the rotated image when the original image and the rotated image are Fourier-transformed, respectively. (b) graph shows the highest correlation at 25 °. Therefore, it can be seen that the rotated image is an image rotated by 25 degrees from the original image.

As described above, according to the embodiment of the present invention, more precise high-resolution images can be obtained by estimating the motion of the vertical, horizontal, and rotational components between a plurality of low-resolution images. In addition, the reliability of the data can be increased by using the Fourier transform to estimate the motion of the rotation component between the plurality of low-resolution images.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, Therefore, the present invention should be construed as a description of the claims which are intended to cover obvious variations that can be derived from the described embodiments.

100: High resolution image restoration device
110:
120: second image generating unit
130: Final reconstruction image generation unit
310: 1st image
311: Reference image
312: Candidate image
320: 2nd image

Claims (12)

A determination unit that determines one reference image and at least one candidate image included in a threshold range for a predetermined sub-pixel value among a plurality of first images for the same target object;
Vertical and rotational components between the reference image and the at least one candidate image, and using the estimated sub-pixel displacement difference to calculate pixel information of the reference image and the at least one candidate image A second image generating unit for generating a second image having a pixel coordinate formed by the second image generating unit; And
And a final reconstruction image generation unit for generating a final reconstruction image by correcting the sampling information of the second image,
Wherein the second image generator comprises:
A sub-pixel displacement difference between horizontal, vertical, and rotational components is estimated by performing Fourier transform on each of the reference image and the candidate image,
Wherein the sub-pixel displacement difference is estimated by Fourier transforming the following equation:

Here, x1 is a sub-pixel value of a horizontal component, x2 is a sub-pixel value of a vertical component, and R is a sub-pixel value of a rotation component. The method according to claim 1,
Wherein the second image is an image having a higher resolution than the first image. The method according to claim 1,
Wherein the reference image is set as an image having the largest number of candidate images among the candidate images satisfying the following equations among the first images:

Here, TH is a threshold for a predetermined sub-pixel value, F ' _k (u) is a sub-pixel value of the first image, and F _l (u) is a sub-pixel value of the reference image. delete delete The method according to claim 1,
Wherein the final restored image generation unit comprises:
And correcting the second image by applying at least one of a first iterative back projection, a deblurring, and an anti-aliasing to the second image. A high-resolution image restoration method using an image restoration device,
Determining one reference image and at least one candidate image included in a threshold range for a predetermined sub-pixel value of a plurality of first images for the same target object;
Vertical and rotational components between the reference image and the at least one candidate image, and using the estimated sub-pixel displacement difference to calculate pixel information of the reference image and the at least one candidate image Generating a second image having pixel coordinates comprised of < RTI ID = 0.0 > a < / RTI > And
And correcting the sampling information of the second image to generate a final restored image,
Wherein the generating the second image comprises:
A sub-pixel displacement difference between horizontal, vertical, and rotational components is estimated by performing Fourier transform on each of the reference image and the candidate image,
Wherein the subpixel displacement difference is estimated by Fourier transforming the following equation:

Here, x1 is a sub-pixel value of a horizontal component, x2 is a sub-pixel value of a vertical component, and R is a sub-pixel value of a rotation component. 8. The method of claim 7,
Wherein the second image is a higher resolution image than the first image. 8. The method of claim 7,
Wherein the reference image is set as an image having the largest number of candidate images among the candidate images satisfying the following equations among the first images:

Here, TH is a threshold for a predetermined sub-pixel value, F ' _k (u) is a sub-pixel value of the first image, and F _l (u) is a sub-pixel value of the reference image. delete delete 8. The method of claim 7,
Wherein the generating the final reconstructed image comprises:
Wherein at least one of a first iterative back projection, a deblurring, and an anti-aliasing is applied to the second image to correct the second image.

Images