JP2015197818A - Image processing apparatus and method of the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate high-resolution image having high sharpness from a low resolution image.SOLUTION: A high-resolution image generation part 104 generates a high-resolution image X from a low resolution image Y through reconfiguration type super-resolution. A high range emphasis part 105 performs high range emphasis of the high-resolution image on the basis of a high-resolution image Xinitially generated by the high-resolution image generation part 104 and the current high-resolution image X.

Description

  The present invention relates to image processing for generating a high resolution image from a low resolution image.

  A super-resolution technique is known as a technique for generating a high-resolution image from a low-resolution image. There are two major super-resolution techniques: reconfiguration-type super-resolution (for example, Non-Patent Document 1) and learning-type super-resolution (for example, Non-Patent Document 2).

  In reconstruction super-resolution, first, an appropriate initial high-resolution image is generated from the original low-resolution image by interpolation enlargement processing, etc., and the initial high-resolution image is captured using imaging information (blur, down-sampling, motion). Generate a low resolution image. Then, an error between the low resolution image generated from the initial high resolution image and the original low resolution image is calculated, and the initial high resolution image is updated so as to reduce the error.

  Reconstruction-type super-resolution is a technique that exhibits a very high effect when there are a plurality of similar image data (having minute positional deviations) as input image data. However, if there is only one input image data, it is difficult to perform accurate alignment even if there are multiple input image data, or if the constraints such as the band are not satisfied, the reconstruction type super-resolution is Only images with low sharpness can be generated.

  On the other hand, learning-type super-resolution performs super-resolution processing using learning data generated in advance. The learning data is composed of, for example, a correspondence between small resolution units of a low resolution image and a high resolution image. The learning data is generated as a correspondence relationship in which, for example, an ideal image (high resolution image) and an assumed input image (low resolution image) created by simulation or the like are compared in units of small areas. Learning super-resolution can generate a high-resolution image from one low-resolution image.

  According to learning-type super-resolution, an ideal high-resolution image can be generated if the input image data exactly matches the learning data, but noise is generated otherwise. For this reason, learning is performed using abstraction, which increases versatility, while generating a high-resolution image with a lower sharpness than an ideal image.

  Therefore, a technique has been proposed for generating a high-resolution image that has no noise and is as sharp as possible from one image data. For example, the technique of Patent Document 1 proposes a reconfigurable super-resolution process that focuses on the fact that the same luminance change continues along the contour of the subject. However, since the technique proposed in Patent Document 1 uses the property that the same luminance change is spatially continuous along the contour of the subject, an image that can obtain an effect is limited to a part of images having many edge portions. .

JP 2007-310837 A

Sung C. P., Min K. P. `` Super-Resolution Image Reconstruction: A Technical Overview '' IEEE Signal Proc. Magazine, Vol. 26, No. 3, pp. 21-36, 2003 Freeman, W.T., Jones, T.R., Pasztor, E.C.``Example-based super-resolution '' Computer Graphics and Applications, IEEE, 2002 S. Daly “The Visible Differences Predictor: Analgorithm for the assessment of image fidelity” in A. B. Watson, editor, Digital Image and Human Vision, 179-206, Cambridge, MA, MIT Press, 1993

  An object of the present invention is to generate a high-resolution image with high sharpness from a low-resolution image.

  The present invention has the following configuration as one means for achieving the above object.

  Image processing according to the present invention generates a high-resolution image from a low-resolution image by reconstruction super-resolution, and based on the initially generated high-resolution image and the current high-resolution image, high-frequency enhancement of the high-resolution image I do.

  According to the present invention, a high-resolution image with high sharpness can be generated from a low-resolution image.

1 is a block diagram illustrating a configuration example of an image processing apparatus according to an embodiment. The block diagram which shows the structural example of an image process part. The flowchart explaining the image process of an image process part. The flowchart explaining the image process of an image process part. The figure which shows the relationship between a matrix and the size of low-resolution data. FIG. 6 is a block diagram illustrating a configuration example of an image processing unit according to the second embodiment. The flowchart explaining the image process of an image process part. The flowchart explaining the image process of an image process part. FIG. 10 is a block diagram illustrating a configuration example of an image processing unit according to the third embodiment. The block diagram which shows the structural example of a high region component calculation part. The flowchart explaining the image process of an image process part.

  Hereinafter, image processing according to an embodiment of the present invention will be described in detail with reference to the drawings.

[Device configuration]
The block diagram of FIG. 1 shows a configuration example of the image processing apparatus of the embodiment.

  The image input unit 101 inputs image data of a low resolution image (hereinafter, low resolution data). In the image processing unit 102, the image input unit 101 generates image data with higher resolution (hereinafter referred to as high resolution data) from the low image data. The image output unit 103 outputs the high resolution data generated by the image processing unit 102.

  The low-resolution data is supplied from an imaging device such as a digital camera, a recording medium such as a memory card, a server device or a user device connected via a wired or wireless network. The image input unit 101 inputs low resolution data from a designated supply source in accordance with a user instruction.

  The output destination of the high resolution data is a display device such as a liquid crystal panel, a recording medium such as a memory card, a server device or a user device connected via a wired or wireless network. The image output unit 103 outputs high-resolution data to a specified output destination according to a user instruction.

  The image processing apparatus according to the embodiment is realized by supplying a program for executing the processing functions of the image input unit 101, the image processing unit 102, and the image output unit 103 to a computer device such as a personal computer (PC). That is, the image processing apparatus is realized by the CPU of the PC executing the program using the RAM as a work memory.

[Image processing unit]
A configuration example of the image processing unit 102 is shown in the block diagram of FIG.

The low resolution data input from the image input unit 101 is stored and held in the memory unit 201. The initial image generation unit 202 generates an initial image of high resolution data (hereinafter, “initial high resolution image X ₀ ”) from the low resolution data.

  The image degradation unit 203 generates a degraded image obtained by degrading the high-resolution image X in accordance with the degradation condition given from the imaging information holding unit 204. The deterioration condition is downsampling, which is a deterioration process due to the limitation of the point spread function (PSF) of the optical system and the number of sensor pixels in the imaging apparatus. The deterioration condition of downsampling is given by the reduction ratio (1 / Sx, 1 / Sy) at the time of downsampling (Sx> 0, Sy> 0). Although details of the deterioration condition will be described later, the deterioration condition is preset.

  The evaluation value calculation unit 205 calculates a difference between the low resolution data held by the memory unit 201 and the deteriorated image data generated by the image deterioration unit 203 as an evaluation value E. Based on the evaluation value E, the convergence determination unit 206 determines whether or not the update of the high resolution image X has converged.

  The update amount calculation unit 207 is an update for correcting the current high-resolution image X by a maximum a posteriori estimation method (hereinafter referred to as a MAP estimation method), which is a kind of reconstruction type super-resolution. Calculate the amount. The high frequency emphasis determination unit 209 performs high frequency emphasis on the current high resolution image X, or ends the image processing of the image processing unit 102 and obtains the image data (high resolution data) of the current high resolution image X. Determine whether to output. Note that the high-frequency emphasis enhances components having a high spatial frequency of the image data, thereby enhancing the sharpness of the image.

The difference calculation unit 210 calculates a difference (a high frequency component described later) between the current high resolution image X and the initial high resolution image X _{0 in} order to perform high frequency emphasis. The image update unit 208 adds the update amount calculated by the update amount calculation unit 207 or the difference (high frequency component) calculated by the difference calculation unit 210 to the current high resolution image X to generate an updated high resolution image X To do.

  As described above, the initial image generation unit 202, the image degradation unit 203, the imaging information holding unit 204, the evaluation value calculation unit 205, the convergence determination unit 206, the update amount calculation unit 207, and the image update unit 208 are included in the high resolution image generation unit 104. Configure. The high frequency emphasis determination unit 209 and the difference calculation unit 210 constitute a high frequency emphasis unit 105.

● Degradation conditions As degradation conditions for the imaging device, the PSF of the optical system of the imaging device and downsampling, which is a degradation process due to the limitation on the number of sensor pixels, are handled. Assuming that the captured image (low resolution data) is Y, the degradation condition acts as defined by equation (1).
Y = DB · X (1)
Here, the matrix B is a square matrix that performs degradation processing by PSF,
Matrix D is a matrix reflecting the image reduction magnification of the downsampling process.
X is high resolution data.

  FIG. 4 shows that the sizes of the matrices B and D depend on the size of the low resolution data. Assuming that the size of the low-resolution data is the number of horizontal pixels W and the number of vertical pixels H, and the image reduction ratio during downsampling is (1 / Sx, 1 / Sy), the size of the matrix described in equation (1) is shown in Fig. 4. It is shown.

  Since X, which is high-resolution data, is handled as a vector description, the matrix size is (H × W) rows × 1 column. The matrix B that performs the degradation process by PSF is a square matrix of (H × W) rows × (H × W) columns. The size of the matrix D representing the downsampling process is (H × 1 / Sx × W × 1 / sy) rows × (H × W) columns.

Super-resolution processing The image processing (super-resolution processing) of the embodiment uses the MAP estimation method disclosed in Non-Patent Document 2. The MAP estimation method is a process for estimating high resolution data as an optimization problem that maximizes the posterior probability by using prior information on the high resolution data.

According to the MAP estimation method, a regularization term for determining a priori probability or a solution for the square error between low-resolution data and degraded image data obtained by degrading the estimated high-resolution data using Equation (1). The high resolution data is estimated by minimizing the evaluation function to which is added. In the embodiment, as shown in Expression (2), high-resolution data is generated by minimizing an evaluation function that does not include a regularization term.
X ^ = arg _x min [|| Y-DBX || ² ]… (2)
Where X ^ is the estimated high resolution data.

[|| Y-DBX || ² ] on the right side of the equation (2) is a term for calculating an absolute difference value between the low resolution data Y and the image estimated through the deterioration process, and the equation (3) is an equation. This is a mathematical formula that extracts the right side of (2) and uses it as an evaluation function during image processing.
E = || Y-DBX || ² … (3)

  The image processing of the image processing unit 102 will be described with reference to the flowcharts of FIGS. 3A and 3B.

Initial image generation unit 202 inputs the one low-resolution data Y (S301), creates an initial high resolution image X ₀ required MAP estimation (S302). That is, the initial image generation unit 202 performs an interpolation process on the input low resolution data Y to create an initial high resolution image X _{0 in} which the horizontal size of the image is enlarged by Sx times and the vertical size is enlarged by Sy times. For example, a bicubic method is used for the interpolation processing, but other interpolation processing may be used.

Next, the high frequency emphasis determination unit 209 sets an initial evaluation value Eold (S303), and the image degradation unit 203 acquires degradation conditions from the imaging information holding unit 204 (S304). As described above, the degradation conditions handled in this embodiment are PSF and downsampling. Subsequently, the image degradation unit 203 creates a degraded image Y ′ corresponding to the low resolution data Y from the high resolution image X based on the degradation condition (S305). The deteriorated image Y ′ is created by calculating the following equation based on the equation (1).
Y '= DB · X (4)

  Next, the evaluation value calculation unit 205 calculates the evaluation value E using Equation (3) (S306), and the convergence determination unit 206 determines whether or not the update of the high resolution image X has converged, in other words, the MAP estimation process It is determined whether or not to end (S307). The end determination of the MAP estimation process is performed by comparing a preset threshold value Th1 and the evaluation value E. If the evaluation value is less than the threshold value (E <Th1), the MAP estimation process is ended. If the evaluation value is equal to or greater than the threshold (E ≧ Th1), the MAP estimation process is continued.

When E ≧ Th1, the update amount calculation unit 207 calculates the update amount ΔX necessary for reducing the difference value shown in Expression (3) from the evaluation value E and the current high-resolution image X using the following expression (S308). ).
ΔX = ∂E / ∂X =-(DB) ^T [Y-DB ・ X ^]… (5)

Next, the image update unit 208 updates the high resolution image X with the update amount ΔX using the following equation (S309), and returns the process to step S305.
X = X + ηΔX (6)
Here, η is a parameter that determines the width of the update.

Thus, initial high resolution image X ₀ the MAP estimation process starts as a high-resolution image X, the determination condition E <Th1 in step S307 is repeated Update processing high resolution image X until fully. Note that the processing from step S304 to S309 corresponds to the MAP estimation processing.

On the other hand, when the determination condition E <Th1 is satisfied in step S307, the high frequency emphasis determination unit 209 compares the current evaluation value E with the evaluation value Eold (previous evaluation value) according to the following equation to obtain high resolution data Is determined whether or not to be output (S310).
E / Eold <Th2 (7)
Here, Th2 is a predetermined threshold value.

  As shown in Expression (7), when the ratio E / Eold between the current evaluation value and the previous evaluation value is less than the threshold, the high frequency emphasis determination unit 209 determines whether the current high resolution image X image data (high resolution data ) Is output (S314), and the image processing of the image processing unit 102 is terminated.

  On the other hand, when Expression (7) is not satisfied, the high frequency emphasis determination unit 209 updates the evaluation value Eold with the current evaluation value E (S311). Subsequently, the difference calculation unit 210 calculates a difference d between the current high resolution image X input from the high frequency emphasis determination unit 209 and the initial high resolution image (S312). The difference d corresponds to the high frequency component recovered by the MAP estimation process.

The image update unit 208 corrects the current high-resolution image X based on the high-frequency component (difference d) to generate a high-resolution image X that emphasizes the high-frequency component (S313), and the process proceeds to step 304. return. Note that the processing from steps S311 to S313 is referred to as “high frequency emphasis processing”.
X = X + d (8)

  In this way, the high frequency component recovered by the MAP estimation process is emphasized, and the MAP estimation process is performed again. By this processing, a solution in which high frequency components are more emphasized can be obtained, and high-definition high-resolution data can be generated.

  In the above, an example in which high-resolution data is generated using the MAP method has been described, but other high-resolution image generation methods may be used. For example, evaluation value convergence methods such as a projection onto convex sets (POCS) method, a POCS-maximum likelihood (POCS-ML) method, and a back projection method are included.

  The image processing according to the second embodiment of the present invention will be described below. Note that the same reference numerals in the second embodiment denote the same parts as in the first embodiment, and a detailed description thereof will be omitted.

  In the first embodiment, the example in which the convergence of the MAP estimation process and the determination of the high frequency emphasis process are performed based on the evaluation value E has been described. In the second embodiment, an example in which the convergence of the MAP estimation process and the determination of the high frequency enhancement process are performed based on the number of repetitions of a series of processes from the generation of the deteriorated image Y ′ to the update of the high resolution image X will be described.

  A block diagram of FIG. 5 shows a configuration example of the image processing unit 102 of the second embodiment. In the image processing unit 102 of the second embodiment, the difference from the configuration of the image processing unit 102 of the first embodiment shown in FIG. 2 is that the evaluation value calculation unit 205 is replaced with a counter unit 211 that counts the number of iterations of the MAP estimation process. It is a point. That is, the image processing unit 102 according to the second embodiment determines the convergence of the MAP estimation process and the high frequency enhancement process based on the number of iterations counted by the counter unit 211. If the number of iterations of the MAP estimation process is N, the number of updates of the high-resolution image X is N-1, and the counter unit 211 counts the number of updates of the high-resolution image X, and the MAP estimation process is performed based on the number of updates. Convergence may be determined.

  The high-resolution image generation unit 104 includes an initial image generation unit 202, an image degradation unit 203, an imaging information holding unit 204, a convergence determination unit 206, an update amount calculation unit 207, an image update unit 208, and a counter unit 211. . The high frequency emphasizing unit 105 includes a high frequency emphasizing determining unit 209, a difference calculating unit 210, and a counter unit 211.

  Image processing of the image processing unit 102 will be described with reference to the flowcharts of FIGS. 6A and 6B. Note that the same reference numerals are given to processes that are substantially the same as the processes of the first embodiment, and detailed description thereof is omitted.

  When the initial high-resolution image generation by the initial image generation unit 202 (S302) and the acquisition of the deterioration condition by the image deterioration unit 203 (S304) are completed, the counter unit 211 initializes two counters (S601). One counter is a first counter (count value t1) that counts the number of MAP estimation processes, and the other counter is a second counter (count value t2) that counts the number of high-frequency emphasis processes. , Both are initialized to zero (t1 = 0, t2 = 0).

  Next, the convergence determination unit 206 compares the count value t1 of the first counter with a predetermined threshold value Th3, and determines whether or not to end the MAP estimation process (S602). If t1> Th3, the MAP estimation process is terminated, and if t1 ≦ Th3, the MAP estimation process is continued.

  When t1 ≦ Th3, after the MAP estimation processing (S305, S308, S309) is executed, the counter unit 211 counts up the first counter (t1 ++) (S603), and the processing returns to step 602. If t1> Th3, the MAP estimation process is terminated, and the high frequency emphasis determination unit 209 compares the count value t2 of the second counter with a predetermined threshold Th4 to determine whether to output high resolution data. Determination is made (S604).

  When t2> Th4, the high frequency emphasis determination unit 209 outputs the image data (high resolution data) of the current high resolution image X (S314), and ends the image processing of the image processing unit 102. When t2 ≦ Th4, the counter unit 211 counts up the second counter (t2 ++) and initializes the count value of the first counter (t1 = 0) (S605). Thereafter, high frequency emphasis processing (S311 and S312) is performed, and the processing returns to step S602.

  In this way, the number of MAP estimation processing and high frequency enhancement processing can be controlled by two preset threshold values. Therefore, it is possible to generate an optimal high-resolution image according to the calculation resource of the image processing apparatus.

  Hereinafter, image processing according to the third embodiment of the present invention will be described. Note that the same reference numerals in the third embodiment denote the same parts as in the first and second embodiments, and a detailed description thereof will be omitted.

In Examples 1 and 2, the difference calculation unit 210, an example was described of calculating the current high-resolution image X and the difference of the initial high-resolution image X ₀ as a high-frequency component. In the third embodiment, an example will be described in which a database (DB) prepared in advance is used for calculating the high frequency component.

  A configuration example of the image processing unit 102 according to the third embodiment is shown in the block diagram of FIG. The difference between the image processing unit 102 of the third embodiment and the configuration of the image processing unit 102 of the first embodiment shown in FIG. 2 is that the difference calculation unit 210 is replaced with a high frequency component calculation unit 212.

  The high-resolution image generation unit 104 includes an initial image generation unit 202, an image degradation unit 203, an evaluation value calculation unit 205, an imaging information holding unit 204, a convergence determination unit 206, an update amount calculation unit 207, and an image update unit 208. Is done. The high frequency emphasis unit 105 includes a high frequency emphasis determination unit 209 and a high frequency component calculation unit 210.

  The block diagram of FIG. 8 shows a configuration example of the high frequency component calculation unit 212.

The high frequency estimation unit 801 receives the current high resolution image X from the high frequency characteristic determination unit 209, receives the initial high resolution image X ₀ from the initial image generation unit 202, and stores the high frequency component DB 806 prepared in advance. To estimate high-frequency components. That is, the image feature amount extraction unit 804 extracts a feature amount (for example, a mid-range component) corresponding to the high-frequency component. Feature amount may be extracted from the initial high-resolution image X ₀ or current high-resolution image X, or may be extracted from both images. The high frequency component search unit 805 searches the high frequency component DB 806 for a feature value closest to the feature value extracted by the image feature value extraction unit 804, and acquires a high frequency component associated with the detected feature value.

  The high frequency component acquired by the above method includes noise when the high frequency component DB 806 is insufficiently constructed or when the search by the high frequency component searching unit 805 is inaccurate. In order to obtain a good high-resolution image with less noise, the noise sensation calculation unit 802 and the correction processing unit 803 perform processing to reduce the noise component from the high frequency component estimated by the high frequency estimation unit 801.

  The noise sensation calculation unit 802 inputs the current high resolution image X and the high frequency component estimated by the high frequency estimation unit 801, and uses, for example, the method described in Non-Patent Document 3 to sense the high frequency component noise sensation. calculate. The noise feeling calculated by the noise feeling calculation unit 802 is a map of visual differences between images in consideration of the masking effect. Thereby, when the high frequency component calculated by the high frequency component calculation unit 211 is added to the current high resolution image X, it is possible to know how much noise component (noise feeling) exists in which region.

  The correction processing unit 803 performs correction processing for suppressing the noise component of the high frequency component estimated by the high frequency estimating unit 801 based on the noise sense calculated by the noise sense calculating unit 802. This correction process may be performed by multiplying a high frequency component by a correction coefficient that is inversely proportional to the magnitude of the noise feeling, or may be a gain adjustment that cuts only a high frequency component with a large noise feeling.

  The image processing of the image processing unit 102 will be described with reference to the flowchart of FIG. Note that the processing in steps S301 to S311, S313, and S314 is the same as the processing in the first embodiment, and detailed description thereof is omitted. That is, instead of the process (S312) of calculating the high frequency component d in the first embodiment, the processes of steps S901 to S903 are performed.

  When the update of the evaluation value Eold (S311) is completed, based on the high frequency component estimation by the high frequency estimation unit 801 (S901), the noise calculation by the noise calculation unit 802 (S902), and the noise processing by the correction processing unit 803 Correction processing (S903) is performed. Thereafter, as in the first embodiment, the image update unit 208 corrects the high resolution image (S313), and the process returns to step S305.

  In the third embodiment, the high frequency component DB 806 prepared in advance for the calculation of the high frequency component is used. In normal learning-type super-resolution, the result of adding the high-frequency components obtained from such DBs is output as it is, so that the high-frequency components generally contain as little noise as possible. Abstract learning at the expense of In the third embodiment, the high frequency component estimated by the high frequency estimation unit 801 is subjected to processing for removing a noise-sensitive component by the noise calculation unit 802 and the correction processing unit 803. Furthermore, the high-resolution image obtained by the addition of the high-frequency component by the image update unit 208 is also reduced in noise by the MAP estimation process performed again, and the high-frequency component DB 806 is not ideal. Resolution data can be generated.

  As described above, even when the high-frequency component DB 806 is insufficiently constructed, a high-resolution image with high sharpness and little noise can be generated. Of course, if there is a high-frequency component DB 806 suitable for input low-resolution data, an ideal high-resolution image can be restored. Therefore, it is possible to generate a high-resolution image (high resolution data) with a high sharpness for an arbitrary input image (low-resolution data).

[Modification]
Depending on the degree of coincidence between the feature quantity obtained by the image feature quantity extraction unit 804 and the closest feature quantity existing in the high-frequency component DB 806, the high-frequency component estimation by the high-frequency estimation unit 801 and the difference calculation unit 210 The calculation of the high frequency component by may be switched. In other words, the high frequency component is estimated if the matching degree is high, and the high frequency component is calculated if the matching degree is low. Naturally, the difference calculation unit 210 in the second embodiment can be replaced with a high frequency component calculation unit 211.

[Other Examples]
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various recording media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

102 ... Image processing unit, 104 ... High resolution image generation unit, 105 ... High frequency enhancement unit

Claims (16)

Generating means for generating a high-resolution image from a low-resolution image by reconstruction type super-resolution;
An image processing apparatus comprising: an enhancement unit that performs high-frequency enhancement of a high-resolution image based on a high-resolution image initially generated by the generation unit and a current high-resolution image. The generation means includes first determination means for determining whether or not the update of the high-resolution image in the reconstruction type super-resolution has converged,
2. The image processing apparatus according to claim 1, wherein the enhancement unit includes a second determination unit that determines whether to perform the high-frequency enhancement when it is determined that the update has converged. The generating means includes
First generation means for generating a high resolution image from a low resolution image;
Second generation means for generating a degraded image obtained by degrading the high-resolution image;
Evaluation means for calculating an evaluation value of the deteriorated image based on the low resolution image;
First calculating means for calculating an update amount of the high-resolution image;
Updating means for updating the high-resolution image based on the update amount;
The first determination unit determines whether the update has converged based on the evaluation value, and the second generation unit, the evaluation unit, and the first calculation until it is determined that the update has converged. 3. The image processing apparatus according to claim 2, wherein the processing of the means and the updating means is repeated.   4. The image processing apparatus according to claim 3, wherein the evaluation unit calculates a difference between the low resolution image and the deteriorated image as the evaluation value.   5. The image processing apparatus according to claim 4, wherein the first calculation unit calculates the update amount from the evaluation value and a current high resolution image.   6. The image processing apparatus according to claim 3, wherein the first determination unit determines that the update has converged when the evaluation value becomes less than a first threshold value. The enhancement means includes second calculation means for calculating a high frequency component of the high resolution image,
The second determination means determines whether to perform the high frequency emphasis based on the evaluation value,
When it is determined that the high frequency emphasis is performed, the updating unit updates the current high resolution image based on the high frequency component, and the processing of the generating unit is repeated until it is determined that the high frequency emphasis is not performed. 7. The image processing apparatus according to any one of claims 3 to 6, wherein   8. The image processing apparatus according to claim 7, wherein the second determination unit holds the evaluation value in the previous determination as the previous evaluation value, and performs the determination based on the current evaluation value and the previous evaluation value. .   The second determination means determines that the high-frequency emphasis is not performed when a ratio between the current evaluation value and the previous evaluation value is less than a second threshold, and the current high-resolution image is determined as the low-resolution image. 9. The image processing apparatus according to claim 8, wherein the image processing apparatus outputs a high-resolution image corresponding to the image.   10. The second calculation unit according to claim 7, wherein the second calculation unit calculates a difference between the high-resolution image generated by the first generation unit and the current high-resolution image as the high-frequency component. The image processing apparatus described in 1. The second calculating means includes
Means for extracting feature quantities from the high resolution image generated by the first generation means and / or the current high resolution image;
Means for searching for a high frequency component corresponding to the feature amount with reference to a database;
Means for calculating a noise component from the current high resolution image and the high frequency component obtained by the search;
10. The image processing apparatus according to claim 7, further comprising a unit that corrects the high frequency component based on the noise component.   12. The image processing according to claim 7, wherein the updating unit generates the updated high-resolution image by adding the update amount or the high-frequency component to a current high-resolution image. apparatus.   3. The image processing apparatus according to claim 2, wherein the first determination unit performs the determination based on the number of times the high-resolution image is updated, and the second determination unit performs the determination based on the number of times of high-frequency emphasis. . Generate a high-resolution image from a low-resolution image by reconstruction super-resolution,
An image processing method for performing high-frequency emphasis on a high-resolution image based on the initially generated high-resolution image and the current high-resolution image.   14. A program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 13.   A computer-readable recording medium on which the program according to claim 15 is recorded.

Images