KR102440459B1 - Method and Apparatus for Image Resolution Enhancement - Google Patents

Abstract

The present invention relates to a method and apparatus for improving image resolution, the method comprising the steps of performing modulation transfer function compensation (MTF) on an input image, and Catmull-Rom (CR) splines on the MTF-compensated image (Spline) enlarging an image by applying an interpolation technique, applying a B-spline interpolation technique to an image enlarged by applying the CR spline interpolation technique to enlarge it to a predetermined resolution, and expanding the image to a predetermined resolution and performing deconvolution on the image to improve sharpness. According to the present invention, it is possible to increase the resolution by 1.4 times without distortion of the original image or degradation of performance. In particular, in image processing, the same physical concept as actually rotating an image by 45° can be applied by enlarging and rearranging the image according to the characteristics of the camera using image information of four adjacent points without rotating the image by 45°. Therefore, there is an advantage that the processing speed can be increased while improving the resolution without distortion of the image.

Description

Method and Apparatus for Image Resolution Enhancement

The present invention relates to a method and apparatus for improving image resolution.

The resolution of an image captured by a satellite equipped with an optical camera may be expressed as in Equation 1 below.

[Equation 1]

Resolution (GSD) = (distance of satellite and target) x (camera pixels) / camera focal length

Theoretically, if the focal length of the camera is long, the resolution can be improved even with a camera with the same aperture, but in practice, the diameter ( The ratio between D) and the camera length (focal length F) has to be limited within a certain value, and the recently created world-class satellites have a focal length and camera diameter ratio (F/D) of around 15-20. It is a trend that is being created with

The ratio of resolution according to the diameter and altitude of a satellite camera has been regarded as an almost natural value that cannot be changed. Although it is possible to acquire images with a resolution of 50 cm at an altitude of 500 km even with a camera with an aperture, the resolution obtained in this way does not change.

For a satellite image acquired to have a specific resolution, various methods or techniques have been developed and proposed to improve the resolution through terrestrial processing, but the image is inevitably distorted during processing.

On the other hand, the concept of obtaining a good image by dividing by the square root using diagonal information (about 1.4 times resolution improvement) is known.

1 is a diagram provided to explain a resolution improvement technique using diagonal line information.

A brief introduction to these concepts is as follows.

1. First, magnify the image by 1.4 times.

2. Resampling by rotating the image by 45°.

3. Convert image data into frequency domain to remove noise.

Through the above process, it is possible to obtain a 1.414 times improved image in a rhombus shape without distortion by using the diagonal information of the image.

However, if the above concept is applied as it is, the process of rotating and rearranging the image by 45° takes a lot of computation time, and rather, distortion of the image is inevitable. In addition, it was not easy to find a suitable method that can be applied in practice to remove noise by converting to the frequency domain.

The technical problem to be solved by the present invention is to provide an apparatus and method capable of improving resolution without image distortion.

An image resolution improvement method according to the present invention for solving the above technical problem includes performing modulation transfer function compensation (MTF) compensation on an input image, and Catmull-Rom (CR) spline on the MTF-compensated image ) Enlarging an image by applying an interpolation technique, applying the B-spline interpolation technique to an image enlarged by applying the CR spline interpolation technique, and enlarging the image to a predetermined resolution, and to the image enlarged to the predetermined resolution performing deconvolution to improve sharpness.

The function used when applying the CR spline interpolation method and the B-spline interpolation method can use the camera characteristic function obtained in the calibration process of the satellite image.

An apparatus for improving image resolution according to the present invention for solving the above technical problem includes an MTF compensator for performing modulation transfer function compensation (MTF) compensation on an input image, and a Catmull-Rom (CR) for the MTF-compensated image. An image magnifying unit for enlarging an image by applying a spline interpolation technique, a resampling unit for enlarging an image to a predetermined resolution by applying a B-spline interpolation technique to an image enlarged by applying the CR spline interpolation technique, and the predetermined and a deconvolution unit for improving sharpness by performing deconvolution on an image enlarged to a resolution.

A computer-readable recording medium according to another embodiment of the present invention records a program for executing any one of the methods in a computer.

According to the present invention, it is possible to increase the resolution by 1.4 times without distortion of the original image or degradation of performance. In particular, in image processing, the same physical concept as actually rotating an image by 45° can be applied by enlarging and rearranging the image according to the characteristics of the camera using image information of four adjacent points without rotating the image by 45°. Therefore, there is an advantage that the processing speed can be increased while improving the resolution without distortion of the image.

1 is a diagram provided to explain a resolution improvement technique using diagonal line information.
2 is a block diagram of an apparatus for improving image resolution according to an embodiment of the present invention.
3 shows the coefficients of the CR spline interpolation technique applied according to an embodiment of the present invention.
4 shows the coefficients of the B-spline interpolation technique applied according to an embodiment of the present invention.
5 and 6 illustrate the results of processing by applying the image resolution improvement method according to the present invention to an image obtained by photographing a calibration edge target.
7 shows a comparison between the multipurpose No. 3A original image and the 38 cm image obtained by applying the resolution improvement method according to the present invention.

Then, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them.

2 is a block diagram of an apparatus for improving image resolution according to an embodiment of the present invention.

Referring to FIG. 2 , the image resolution improving apparatus according to the present invention includes an image input unit 110 , an MTF compensator 120 , an image enlarger 130 , a resampling unit 140 , and a deconvolution unit 150 . can do.

The image input unit 110 may receive an image that has not been processed with modulation transfer function compensation (MTF).

The MTF compensator 120 may perform MTF compensation on the image input to the image input unit 110 . MTF compensation is to enhance the MTF of an image, and is performed in two stages: MTF measurement and image restoration. For MTF compensation, accurate measurement of MTF should be preceded. MTF is measured using the edge method and the pulse method according to the input signal of the image.

The double edge method is a method of finding a boundary in an image based on where the contrast is clearly distinguished in the image, and estimating the MTF through the edge spread function (ESF), which is the response of the system to the boundary. Based on the edge detected in the original image, the ESF is obtained along the perpendicular passing through each point, the ESF is differentiated to obtain an LSF (Line Spread Function), and the LSF is Fourier Transformed to obtain the MTF.

Since the MTF compensation and the part for acquiring the ESF and the LSF in this process are already well known to those skilled in the art, a detailed description thereof will be omitted.

The image enlarger 130 may enlarge the image by applying a Catmull-Rom (CR) spline interpolation technique to the image. When the image to be processed by the image magnifier 130 is a satellite image, the image may be enlarged to 1.414 times the satellite design resolution, ie, constant zooming.

The resampling unit 140 may resampling the image by applying a B-spline interpolation technique to the image, thereby enlarging the image to a target (to be performed) resolution.

The function used when applying the CR spline interpolation method and the B-spline interpolation method can use the camera characteristic function of the satellite camera confirmed through satellite image calibration. The characteristic function of the satellite camera can be obtained through the calibration process using the MTF, ESF, and LSF obtained in the MTF compensation process.

Obtaining a camera characteristic function in the calibration process of a satellite image, and determining a function for applying the CR spline interpolation method and the B-spline interpolation method using this is well known to those skilled in the art, and thus detailed description thereof will be omitted.

The deconvolution unit 150 may perform deconvolution on an image to which the CR spline interpolation technique and the B-spline interpolation technique are applied to improve sharpness. To this end, the deconvolution unit 150 may use an MFT kernel that improves sharpness.

3 shows the coefficients of the CR spline interpolation technique applied according to an embodiment of the present invention, and FIG. 4 shows the coefficients of the B spline interpolation technique applied according to an embodiment of the present invention.

As described above, the function applied to the CR spline interpolation method and the B spline interpolation method can use the camera characteristic function of the satellite camera confirmed through the calibration.

5 and 6 illustrate the results of processing by applying the image resolution improvement method according to the present invention to an image obtained by photographing a calibration edge target.

5 and 6, the image resolution improvement method according to the present invention is applied to the satellite image of the multi-purpose 3A (KOMPSAT-3A) which photographed the edge targets of Salon of France and Baotou of China, and ESF in the image for confirmation , LSF, and MTF values for normalized frequencies were measured.

When the resolution is improved by using the method according to the present invention, the properties appearing have ESF and LSF characteristics similar to the functions of the CR spline interpolation technique and the B spline interpolation technique applied to the present invention, and the resolution is improved without reducing the MTF. It could be confirmed as in FIGS. 3 and 4 that the

7 shows a comparison between the multipurpose No. 3A original image and the 38 cm image obtained by applying the resolution improvement method according to the present invention.

In FIG. 7, the left side is the multipurpose No. 3A original image, and the right side is a 38 cm image obtained by applying the resolution improvement method according to the present invention. As described above, referring to FIG. 7 , it can be confirmed that the resolution can be improved by 1.414 times without lowering the MTF according to the method according to the present invention.

The embodiments described above may be implemented by a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the apparatus, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. may be permanently or temporarily embody in The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Claims (5)

performing MTF compensation (modulation transfer function compensation) on the input image;
Enlarging the image to a size of a predetermined magnification by applying a Catmull-Rom (CR) spline interpolation technique to the MTF-compensated image;
The step of applying the B-spline interpolation method to the enlarged image by applying the CR spline interpolation method to enlarge it to a predetermined resolution, and
performing deconvolution on the image enlarged to the predetermined resolution to improve sharpness
including,
The function used when applying the CR spline interpolation technique and the B-spline interpolation technique is an image that is an LSF (Line Spread Function) obtained as a camera characteristic function of the camera that captured the input image in the process of performing the MTF compensation. How to improve resolution.
In claim 1,
wherein the predetermined magnification is 1.414.
A computer-readable recording medium in which a program for executing the method according to claim 1 or 2 is recorded on a computer.
MTF compensation unit for performing MTF compensation (modulation transfer function compensation) on the input image;
An image magnifier that enlarges the image to a size of a predetermined magnification by applying a Catmull-Rom (CR) spline interpolation technique to the MTF-compensated image;
A resampling unit that enlarges the image enlarged by applying the CR spline interpolation technique to a predetermined resolution by applying the B-spline interpolation technique, and
A deconvolution unit for improving sharpness by performing deconvolution on the image enlarged to the predetermined resolution
including,
The function used when applying the CR spline interpolation technique and the B-spline interpolation technique is an image that is an LSF (Line Spread Function) obtained as a camera characteristic function of the camera that captured the input image in the process of performing the MTF compensation. resolution enhancement device.
In claim 4,
and the predetermined magnification is 1.414.

Images