JP6817784B2 - Super-resolution device and program - Google Patents

Description

The present invention relates to a super-resolution device and a program for super-resolution processing an image to be super-resolutiond.

Conventionally, a method of increasing the pixel density and performing registration super-resolution by aligning the same moving objects in a plurality of frames has been known. This alignment is performed with a decimal pixel accuracy, and the pixel density can be increased from the pixel information of the same object whose sample position is deviated.

For example, Patent Document 1 discloses a method for increasing image resolution in which the number of pixels constituting an image frame is increased according to the direction in which the subject on the input image frame moves by synthesizing n image frames. Has been done. Further, in Patent Document 2, motion vectors are calculated from images of two or more consecutive frames by using motion information between frame images recorded in the motion image data, and calculation by cumulative addition and direction conversion is performed. An image processing method that enables highly accurate alignment without complicated calculations is disclosed.

Japanese Unexamined Patent Publication No. 2008-109375

Masayuki Tanaka, Masatoshi Okutomi, "Super-Resolution Technology from Multiple Images", Proceedings of the 15th Image Sensing Symposium (SSII2009) (Invited Talk), pp.OS4-02-1-23, Jan.2009. Y Matsuo and S. Sakaida, “A Super-resolution Method Using Spatio-Temporal Registration of Multi-Scale Components in Consideration of Color-Sampling Patterns of UHDTV Cameras”, Proceedings of IEEE ISM, pp. 311-314, Dec. 2015

In the prior art, when the alignment accuracy is high, a high-quality super-resolution image can be obtained. However, the alignment accuracy tends to be low when the frame rate is low, the object moves in a complicated (irregular) manner, or occlusion occurs. Further, in the stationary region, in principle, the improvement of the pixel density cannot be improved by the alignment.

Therefore, a single super-resolution image is frequency-decomposed, and similar objects are aligned between the super-resolution image and its low-frequency component image, and the super-resolution image is aligned at the aligned location. And its low-frequency component image are similar, so the super-resolution high-frequency component that exceeds the Nyquist frequency of the super-resolution image is assumed to be similar to the high-frequency component of the super-resolution image, and the high-frequency component is assigned to the super A method of generating a resolution image can be considered. However, this method requires a huge amount of time for alignment.

An object of the present invention made in view of such circumstances is a super-resolution capable of generating a high-speed and high-definition super-resolution image even if the frame image includes a region having a low correlation between frames or a stationary region. To provide image equipment and programs.

In order to solve the above problems, the super-resolution device according to the present invention is a super-resolution device that generates a super-resolution image by super-resolution processing the super-resolution image to be super-resolution image. On the other hand, the frequency decomposition unit that performs frequency decomposition processing of a plurality of layers and decomposes the low frequency component image and the high frequency component image for each layer, and the low frequency component image for each divided block that divides the super-resolution image. A hierarchical alignment unit that determines a similar block having the highest degree of similarity to the divided block and generates alignment information indicating the position of the similar block, and the same spatial phase as the similar block according to the alignment information. A super-resolution high-frequency component image in which the high-frequency component of the super-resolution image is estimated is generated by allocating a block in the high-frequency component image of the position to a high-frequency component exceeding the sampling frequency of the super-resolution image. A resolution high-frequency component image generation unit and a frequency reconstruction unit that generates a super-resolution image by performing frequency reconstruction using the super-resolution high-frequency component image as a low-frequency component and the super-resolution high-frequency component image as a high-frequency component. , The hierarchical alignment unit is characterized in that the similar blocks are determined in order from the low frequency component image having the largest number of layers.

Further, in the super-resolution apparatus according to the present invention, the super-resolution high-frequency component image generation unit uses the power of the block in the high-frequency component image at the same spatial phase position as the similar block to be the power of the super-resolution image. After correction based on the above, the super-resolution high-frequency component image is generated by allocating to a high-frequency component exceeding the sampling frequency of the super-resolution image.

Further, in the super-resolution device according to the present invention, when determining a similar block, the hierarchical alignment unit sets the entire range as a search range for the low-frequency component image of the highest layer, and other than the highest layer. The low-frequency component image of the hierarchy is characterized in that the search range is determined based on the position of a similar block having one larger number of layers.

Further, in the super-resolution device according to the present invention, the frequency resolution unit is characterized in that the super-resolution image is subjected to a wavelet decomposition process of a plurality of layers accompanied by decimation.

Further, in order to solve the above problems, the program according to the present invention is characterized in that the computer functions as the super-resolution device.

According to the present invention, registration super-resolution can be performed between similar objects in the same frame or between similar objects between a plurality of frames, and the frame image includes a region having low correlation between frames and a stationary region. However, high-speed and high-definition super-resolution image quality can be obtained.

It is a block diagram which shows the structural example of the super-resolution apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows an example of the wavelet transform processing in the frequency decomposition part of the super-resolution apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows an example of the alignment process in the alignment part of the super-resolution apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows an example of the allocation processing in the super-resolution high-frequency component image generation part of the super-resolution apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows an example of the frequency reconstruction processing in the frequency reconstruction part of the super-resolution apparatus which concerns on 1st Embodiment of this invention. It is a block diagram which shows the structural example of the 1st super-resolution high-frequency component image generation part in the super-resolution apparatus which concerns on 2nd Embodiment of this invention. It is a figure which shows an example of the gain correction processing of the 1st super-resolution high-frequency component image generation part in the super-resolution apparatus which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the 2nd super-resolution high-frequency component image generation part in the super-resolution apparatus which concerns on 2nd Embodiment of this invention. It is a figure which shows an example of the gain correction processing of the 2nd super-resolution high-frequency component image generation part in the super-resolution apparatus which concerns on 2nd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First Embodiment)
FIG. 1 is a block diagram showing a configuration example of a super-resolution device according to the first embodiment of the present invention. In the example shown in FIG. 1, the super-resolution device 1 includes a frequency resolution unit 11, a hierarchical alignment unit 12, a super-resolution high-frequency component image generation unit 13, and a frequency reconstruction unit 14. The super-resolution device 1 performs super-resolution processing on the super-resolution image (original image) I to generate a super-resolution image ^ISR .

The frequency decomposition unit 11 inputs the super-resolution image I and the frequency decomposition parameter. The frequency decomposition parameter includes a parameter that specifies a filter used for the frequency decomposition process and a parameter that specifies the number of layers (decomposition order) n. Then, the frequency decomposition unit 11 performs frequency decomposition processing of a plurality of layers on the super-resolution image I based on the frequency decomposition parameters, and decomposes each layer into a low frequency component image and a high frequency component image. Then, the generated low-frequency component image (LL ⁿ ) of each layer is output to the hierarchical alignment unit 12, and the high-frequency component image (LH ⁿ , HL ⁿ , HH ⁿ ) of each decomposition rank is super-resolution high-frequency component image. Output to the generation unit 13.

The frequency decomposition process is performed by, for example, wavelet transform using a Cohen-Daubechies-Feauveau (CDF) 9/7 wavelet. The frequency decomposition unit 11 may perform frequency decomposition of a plurality of frame images by using a plurality of filters having different characteristics in order to increase the candidates for the reference image of the super-resolution image. Further, the frequency decomposition unit 11 may perform frequency decomposition on a plurality of floors after enlarging the input video. The magnifying process can be performed using a linear magnifying filter such as the Lanczos 3 filter. The magnification m and the number of layers n can be arbitrary values when the super-resolution image is enlarged in the horizontal direction and the vertical direction by m times each and then the nth-order frequency is decomposed.

FIG. 2 is a diagram showing an example of the wavelet transform process in the frequency decomposition unit 11. In the example shown in FIG. 2, the frequency decomposition unit 11 performs a third-order (n = 3) wavelet transform process with decimation on the super-resolved image I, and the wavelet-resolved image (first-order wavelet decomposition image W1, second floor). Wavelet decomposition image W2 and third-order wavelet decomposition image W3) are generated. The processing can be speeded up by performing the wavelet decomposition processing of a plurality of layers accompanied by decimation.

The hierarchical alignment unit 12 inputs the super-resolution image I and the low-frequency component image LL ^{n of} each layer generated by the frequency decomposition unit 11. The hierarchical alignment unit 12 divides the super-resolution image I into division blocks B _{i and j} of a predetermined size (a × a pixel), and divides each division block B _{i and j} in the low frequency component image. The similar blocks Mi _{and j} having the highest degree of similarity (correlation) with the blocks B _{i and j} are determined by block matching. Then, the alignment information indicating the positions of the similar blocks Mi _{and j} is generated and output to the super-resolution high-frequency component image generation unit 13. Here, the subscript i is a horizontal index, and j is a vertical index.

The hierarchical alignment unit 12 determines similar blocks Mi _{and j} in order from the low-frequency component image having the largest number of layers n. For example, in the case of n = 3, n = 3 similar blocks M _i in the low-frequency component _image, after determining _j ^3, similar blocks M _i of the low frequency in the component image n = _2, the a _j ² was determined, Finally, the similar blocks Mi _{and j} ¹ in the low frequency component image of n = ¹ are determined. In this way, similar blocks M _i, by using the location of the _j ³ estimates similar blocks M _i, the position of the _j ^2, similar blocks M _i, similar to use of the position of the _j ² blocks M _{i Since} the position of _{, j} ¹ can be estimated, the time required for alignment can be shortened.

For example, when the hierarchical alignment unit 12 determines similar blocks Mi _{, j} , the entire range of the low-frequency component image of the highest layer is set as the search range, and the low-frequency component image of the layer other than the highest layer is set as the search range. The search range is determined based on the position of a similar block having one larger number of layers.

FIG. 3 is a diagram showing an example of the hierarchical alignment process in the hierarchical alignment unit 12. In this example, the number of layers n = 3. The hierarchical alignment unit 12 uses the super-resolution image I as a reference frame image and the low-frequency component image LL ³ as a reference frame image, and sets the entire low-frequency component image LL ³ as the search range for the divided blocks B _{1 and 1.} performs block matching to determine the divided blocks _{B 1, 1} a highest similarity similar blocks _{M 1, 1} ³ the low frequency within component image LL ^3.

Next, the hierarchical alignment unit 12 performs block matching on the divided blocks B _{1 and 1 using the} super-resolution image I as a reference frame image and the low-frequency component image LL ² as a reference frame image, and performs block matching on the low-frequency component images. Within LL ² , the similar blocks M _1, ¹² having the highest degree of similarity to the divided blocks B ₁ , ₁ are determined. In this case, the search range is, for example, a predetermined size centering double position in the horizontal and vertical directions corresponding coordinates to the central position of the analogous block M _1,1 ³ (b × b pixels, b> a) The range of.

Next, the hierarchical alignment unit 12 performs block matching on the divided blocks B _{1 and} ¹ using the super-resolution image I as a reference frame image and the low-frequency component image LL ¹ as a reference frame image, and performs block matching on the low-frequency component images. Within LL ¹ , a similar block M _1,1 ¹ having the highest degree of similarity to the divided blocks B ₁ , ¹ is determined. At this time, the search range has a predetermined size (b × b pixels, b> a) centered on a position in which the coordinates corresponding to the center positions of similar blocks M _1, ¹² are doubled in both the horizontal and vertical directions. The range of. The above processing is performed for {B _{i, j} | ∀i, j}, and the alignment information indicating the positions of similar blocks Mi _{, j} ⁿ is output to the super-resolution high-frequency component image generation unit 13.

As shown in FIG. 3, the reason why the hierarchical alignment unit 12 aligns with a low-frequency component image having a size different from that of the super-resolution image I is that the images have similar shapes and different sizes in the same frame. This is because, when is present, it is possible to align between similar images and allocate the high frequency component.

The super-resolution high-frequency component image generation unit 13 contains high-frequency components (LH ⁿ , HL ⁿ , HH ⁿ ) of the frequency-resolved image generated by the frequency-resolving unit 11 and alignment information generated by the hierarchical alignment unit 12. And enter. The super-resolution high-frequency component image generation unit 13 first prepares a zero matrix having the same size as the super-resolution image I as super-resolution high-frequency components (LH ^SR , HL ^SR , HH ^SR ). Then, according to the alignment information, the blocks in the high-frequency component images (LH ⁿ , HL ⁿ , HH ⁿ ) at the same spatial phase position as the similar blocks Mi _{, j} ⁿ exceed the sampling frequency of the super-resolution image I. A super-resolution high-frequency component image that estimates the high-frequency component of the super-resolution image I by allocating it to an unknown high-frequency component is generated and output to the frequency reconstruction unit 14.

FIG. 4 is a diagram showing an example of allocation processing in the super-resolution high-frequency component image generation unit 13. In this example, an example of allocating to similar blocks M _1, ¹¹ is shown. The super-resolution high-frequency component image generation unit 13 exceeds the sampling frequency of the super-resolution image I to the high-frequency component images (LH ¹ , HL ¹ , HH ¹ ) of the first-order wavelet decomposition image W1 according to the alignment information. Allocate to the fractional pixel positions of unknown super-resolution high-frequency components (LH ^SR , HL ^SR , HH ^SR ). Here, when allocating, it is assumed that the alignment information of the same phase position is followed. This means that if the divided blocks B _{i and j} of the super-resolution image I are similar to the similar blocks M _{i and j} of the low-frequency component image LL ⁿ , the super-resolution is in the same phase as the divided blocks B _{i and j.} This is because the block of the image high-frequency image is also likely to be similar to the block in the high-frequency component image (LH ⁿ , HL ⁿ , HH ⁿ ) having the same phase position as the similar blocks Mi _{and j} .

After the above allocation, the super-resolution high-frequency component image generation unit 13 weights the super-resolution high-frequency component images (LH ^SR , HL ^SR , HH ^SR ) according to the distance of the allocated pixels, and ML (Maximum Likelihood). ) The reconstruction process may be performed by estimation, MAP (Maximum A Posterior) estimation, or the like. MAP estimation is a method of estimating and reconstructing a high-resolution image that maximizes posterior probabilities when a low-resolution image is a condition. For more information on MAP estimation, see, for example, E. Levitan and G. Herman: "A maximum a posteriori probability expectation maximization algorithm for image reconstruction in emission tomography", IEEE Transactions on Medical Imaging, vol. 6, no. 3, pp. See 185-192, Sep. 1987.

The frequency reconstruction unit 14 inputs the super-resolution image I, the super-resolution high-frequency component image generated by the super-resolution high-frequency component image generation unit 13, and the frequency reconstruction parameter. The frequency reconstruction parameters include parameters that indicate a filter to be used in the frequency reconstruction process. The frequency reconstruction parameter indicates, for example, the same filter as the filter used in the frequency decomposition unit 11. The frequency reconstruction unit 14 performs frequency reconstruction using the super-resolution image I as a low-frequency component and the super-resolution high-frequency component image (LH ^SR , HL ^SR , HH ^SR ) as a high-frequency component, and performs the super-resolution image I. Generate ^SR and output it to the outside.

FIG. 5 is a diagram showing an example of the frequency reconstruction process in the frequency reconstruction unit 14. In this example, the first-order wavelet reconstruction is performed to generate a super-resolution image ^ISR .

As described above, in the super-resolution device 1 according to the present embodiment, the frequency-resolving unit 11 performs a plurality of layers of frequency-resolving processing on the super-resolution image I, and the layered alignment unit 12 performs a multi-layer frequency decomposition process. Hierarchical alignment is performed between the super-resolution image I and the low-frequency component image LL ⁿ to generate alignment information indicating the positions of similar blocks Mi _{and j} ⁿ . Next, the super-resolution high-frequency component image generation unit 13 creates a block in the high-frequency component image (LH ⁿ , HL ⁿ , HH ⁿ ) at the same spatial phase position as the similar blocks Mi _{, j} ⁿ according to the alignment information. A super-resolution high-frequency component image (LH ^SR , HL ^SR , HH ^SR ) is generated by allocating to a high-frequency component exceeding the sampling frequency of the super-resolution image I. Then, the frequency reconstruction unit 14 performs frequency reconstruction using the super-resolution image I as a low-frequency component and the super-resolution high-frequency component image (LH ^SR , HL ^SR , HH ^SR ) as a high-frequency component to perform super-resolution. Generate image ^ISR .

A computer can be preferably used to function as the super-resolution device 1 described above, and such a computer provides a program describing processing contents for realizing each function of the super-resolution device 1 of the computer. It can be realized by storing it in a storage unit and reading and executing this program by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

In this way, the super-resolution device 1 and its program perform alignment with the frequency-resolved images of other frames close to the super-resolution image, so that similar objects in the same frame can be aligned with each other. Registration super-resolution can be performed between similar objects between a plurality of frames, and high super-resolution image quality can be obtained even in a stationary region or a region with low alignment accuracy. Further, since the hierarchical alignment unit 12 performs the hierarchical alignment, the time required for the alignment can be shortened and the processing speed can be increased.

(Second Embodiment)
Next, the super-resolution device according to the second embodiment of the present invention will be described. The super-resolution device 2 of the second embodiment is different from the super-resolution device 1 of the first embodiment in the processing of the super-resolution high-frequency component image generation unit 13. Since other configurations are the same as those of the first embodiment, the description thereof will be omitted.

The super-resolution high-frequency component image generation unit 13 of the super-resolution apparatus 2 according to the second embodiment is similar to the above-mentioned similar blocks M _{i, j} ⁿ (similar to blocks B _{i, j} in the super-resolution image I to be super-resolution image I). After correcting the power of the block in the high-frequency component image at the same spatial phase position as the block in the low-frequency component image LL ⁿ ) based on the power of the super-resolution image I, the sampling frequency of the super-resolution image I Super-resolution high-frequency component images (LH ^SR , HL ^SR , HH ^SR ) are generated by allocating to high-frequency components exceeding. Therefore, it is possible to prevent over-emphasis of the high frequency component of the super-resolution image ^ISR . Hereinafter, two examples (13-1 and 13-2) will be described as the form of the super-resolution high-frequency component image generation unit 13.

FIG. 6 is a diagram showing a configuration example of the super-resolution high-frequency component image generation unit 13-1. The super-resolution high-frequency component image generation unit 13 includes a gain correction unit 131-1 and an image allocation unit 132-1.

The gain correction unit 131-1 calculates the RMS (root mean square) of the power of the high frequency component images (LH ⁿ , HL ⁿ , HH ⁿ ) generated by the frequency decomposition unit 11. The high frequency component image ^{(LH n, HL n, HH} n) based on the super-resolution high-frequency component image which is a high-frequency component image of the super-resolved image ^{I SR (LH SR, HL SR} , HH SR) power of The estimated RMS, which is an estimated value of the RMS, is calculated. Then, gain correction is performed so that the RMS of the power of the high frequency component image (LH ⁿ , HL ⁿ , HH ⁿ ) becomes equal to the estimated RMS, and a gain corrected high frequency component image (cLH ⁿ , cHL ⁿ , cHH ⁿ ) is generated. , Is output to the image allocation unit 132-1.

FIG. 7 is a diagram showing an example of the gain correction process in the gain correction unit 131-1. In this example, the number of layers n = 3. FIG. 7 (a) is a power spectrum of the super-resolution image I, and FIG. 7 (b) is an estimated power spectrum of the super-resolution image ^ISR . The shape of the power spectrum of the super-resolution high-frequency component image (LH ^SR , HL ^SR , HH ^SR ) assumes that the power spectrum in the band of the super-resolution image I is gently continuous. Therefore, the power spectrum of the super-resolution high-frequency component image (LH ^SR , HL ^SR , HH ^SR ) is obtained by function approximation of the power spectrum of {LH ⁿ , HL ⁿ , HH ⁿ | ⁿ ∈ 1, 2, 3}. The RMS of the power of the super-resolution high-frequency component image (LH ^SR , HL ^SR , HH ^SR ) can be estimated by using the inserted value.

Similar to the super-resolution device 1, the image allocation unit 132-1 prepares a 0 matrix having the same size as the super-resolution image I as super-resolution high-frequency components (LH ^SR , HL ^SR , HH ^SR ) and positions them. accordance combined information, similar blocks _{M i} in the low-frequency component image LL _^n, the gain correction high-frequency component image of the same spatial phase position and ^{j n (cLH n, cHL n} , cHH n) blocks in, the super-resolution image A super-resolution high-frequency component image that estimates the high-frequency component of the super-resolution image I by allocating it to an unknown high-frequency component that exceeds the sampling frequency of I is generated and output to the frequency reconstruction unit 14.

FIG. 8 is a diagram showing a configuration example of the super-resolution high-frequency component image generation unit 13-2. The super-resolution high-frequency component image generation unit 13-2 includes a gain correction unit 131-2 and an image allocation unit 132-2.

The gain correction unit 131-2 includes blocks B _{i, j} in the super-resolution image I, similar blocks M _{i, j} ⁿ in the low frequency component image LL ⁿ similar to the blocks B _{i, j} , and similar blocks. M _i, the high frequency component image of the same spatial phase position and ^{j n (LH n, HL n} , HH n) block _{a i in,} and a ^{j n,} the block _{a i,} block was corrected ^{j n} _{cA i,} ^{j n} Is generated and output to the image allocation unit 132-2.

FIG. 9 is a diagram showing an example of the gain correction process in the gain correction unit 131-2. The powers of blocks B _{i, j,} similar blocks M _{i, j} ⁿ , and blocks A _{i, j} ⁿ are P (B _{i, j} ), P (Mi _{, j} ⁿ ), P (A _{i, j} ⁿ ), respectively. Then, the gain correction coefficient C is obtained by the following equation (1). Then, the block obtained by multiplying each pixel of the block A _{i, j} ⁿ by the correction coefficient C is defined as the block cA _{i, j} ⁿ . In this way, the gain correction unit 131-2 can correct the gain for each block A _{i, j} ⁿ by a simple calculation.

The image allocation unit 132-2 prepares a 0 matrix having the same size as the super-resolution image I as super-resolution high-frequency components (LH ^SR , HL ^SR , HH ^SR ), and blocks cA _{i, j} according to the alignment information. ⁿ is assigned to an unknown high-frequency component exceeding the sampling frequency of the super-resolution image I to generate a super-resolution high-frequency component image in which the high-frequency component of the super-resolution image I is estimated, and the frequency reconstruction unit 14 Output to. Although the gain correction unit 131-2 and the image allocation unit 132-2 have been described separately for convenience, these processes may be performed together.

As described above, in the super-resolution device 2 according to the present embodiment, the frequency-resolving unit 11 performs a plurality of layers of frequency-resolving processing on the super-resolution image I, and the hierarchical alignment unit 12 performs super-resolution processing. Hierarchical alignment is performed between the resolution image I and the low-frequency component image LL ⁿ to generate alignment information indicating the positions of similar blocks Mi _{and j} ⁿ . Next, the super-resolution high-frequency component image generation unit 13 applies the power of the block in the high-frequency component image (LH ⁿ , HL ⁿ , HH ⁿ ) at the same spatial phase position as the similar blocks Mi _{, j} ⁿ to the super-resolution. After correction based on the power of the image image I, the super-resolution high-frequency component images (LH ^SR , HL ^SR , HH ^SR ) are assigned to the high-frequency components exceeding the sampling frequency of the super-resolution image I according to the alignment information. To generate. Then, the frequency reconstruction unit 14 performs frequency reconstruction using the super-resolution image I as a low-frequency component and the super-resolution high-frequency component image (LH ^SR , HL ^SR , HH ^SR ) as a high-frequency component to perform super-resolution. Generate image ^ISR .

A computer can be preferably used to function as the above-mentioned super-resolution device 2, and such a computer provides a program describing processing contents for realizing each function of the super-resolution device 2. It can be realized by storing it in a storage unit and reading and executing this program by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

The super-resolution device 2 and its program are gain-corrected in consideration of the spectral power difference between the high-frequency component image (LH ⁿ , HL ⁿ , HH ⁿ ) and the super-resolution high-frequency component image (LH ^SR , HL ^SR , HH ^SR ). I do. Therefore, the high-frequency component of the super-resolution image ^ISR is not overemphasized, and the image quality of the super-resolution image ^ISR can be further improved.

Although the above embodiments have been described as typical examples, it will be apparent to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be construed as being limited by the above embodiments, and various modifications and modifications can be made without departing from the scope of claims. For example, it is possible to combine the plurality of constituent blocks described in the embodiment into one, or to divide one constituent block into one.

1 Super-resolution device 11 Frequency resolution unit 12 Hierarchical alignment unit 13, 13-1, 13-2 Super-resolution high-frequency component image generation unit 131-1, 131-2 Gain correction unit 132-1, 132-2 Image Allocation part 14 Frequency reconstruction part

Claims (5)

It is a super-resolution device that generates a super-resolution image by super-resolution processing the super-resolution image.
A frequency decomposition unit that performs multiple layers of frequency decomposition processing on the super-resolution image and decomposes each layer into a low-frequency component image and a high-frequency component image.
A hierarchy that determines a similar block having the highest degree of similarity to the divided block in the low-frequency component image for each divided block obtained by dividing the super-resolution image, and generates alignment information indicating the position of the similar block. Mold alignment part and
According to the alignment information, a block in the high-frequency component image having the same spatial phase position as the similar block is assigned to a high-frequency component exceeding the sampling frequency of the super-resolution image, and the high-frequency component of the super-resolution image is assigned. Super-resolution high-frequency component image generator that generates the estimated super-resolution high-frequency component image,
A frequency reconstruction unit that generates a super-resolution image by performing frequency reconstruction using the super-resolution image as a low-frequency component and the super-resolution high-frequency component image as a high-frequency component is provided.
The layered alignment unit is a super-resolution device characterized in that similar blocks are determined in order from a low-frequency component image having a large number of layers. The super-resolution high-frequency component image generation unit corrects the power of the block in the high-frequency component image at the same spatial phase position as the similar block based on the power of the super-resolution image, and then performs the super-resolution. The super-resolution device according to claim 1, wherein the super-resolution high-frequency component image is generated by allocating to a high-frequency component exceeding the sampling frequency of the image. When determining similar blocks, the hierarchical alignment unit sets the entire range as the search range for the low-frequency component images of the highest layer, and the number of layers is 1 for the low-frequency component images of layers other than the highest layer. The super-resolution device according to claim 1 or 2, wherein the search range is determined based on the position of a larger similar block. The super-resolution device according to any one of claims 1 to 3, wherein the frequency-resolving unit performs a wavelet-resolving process of a plurality of layers accompanied by decimation on the super-resolution image. A program for operating a computer as the super-resolution device according to any one of claims 1 to 4.

Images