JP2009087185A - Image restoring apparatus and method for the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a restored image with high accuracy, even when the photographed image is severely deteriorated by improving the accuracy for estimating properties of an original image from the deteriorated image. <P>SOLUTION: Wavelet conversion images g1, g2, ..., gM are generated, by sectioning the deteriorated image "g" obtained by imaging into a plurality of frequency band components; then intermediate restored images h1+h2, h1+h2+h3, ..., are generated sequentially by using each wavelet conversion image g1, g2, ..., gM in a stepwise manner from low-frequency band components to high-frequency band components. By finally generating the restored image "h" containing all frequency band components, the intermediate restored image is derived with high accuracy from the low-frequency band component which receives less influence from the image deterioration. By obtaining an estimated parameter for the original image "f", using that intermediate restored image with accuracy, the restored image "h" can be obtained with accuracy, even when the deteriorated image "g" is severely deteriorated. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image restoration apparatus and method, and in particular, when a captured image is deteriorated due to an influence of camera shake or defocusing at the time of imaging, an image that is not affected by camera shake or defocusing is estimated and restored from the deteriorated image. The present invention relates to an image restoration apparatus and method.

  Conventionally, when shooting a landscape or a person with an imaging device such as a digital camera or a camera mounted on a mobile phone, an image with degraded image quality (hereinafter referred to as “ May be obtained). In order to restore an image close to the original image (hereinafter referred to as “restored image”) by estimating an image without deterioration (hereinafter referred to as “original image”) from such a degraded image, various image restorations are performed. An algorithm is considered.

As one of image restoration algorithms based on digital signal processing, there is an algorithm that assumes the property of an original image corresponding to a captured degraded image and performs image restoration based on the property. A typical example is an image restoration algorithm using a Kalman filter (see, for example, paragraph [0034] of Patent Document 1).
JP 2006-101447 A

In the image restoration algorithm using the Kalman filter, it is necessary to estimate the property of the original image before performing restoration processing on the degraded image. That is, it is necessary to first calculate an estimation parameter representing the nature of the original image and then perform image restoration processing using the estimation parameter. Here, there are the following two methods for estimating the properties of the original image.
1. The properties of the original image are statistically obtained in advance using a large number of images.
2. The properties of the original image are estimated from the degraded image.

  However, the first method has a drawback in that only the images for which statistical properties have been obtained in advance can be restored, so that the shooting target is limited. On the other hand, the second method has an advantage that the object to be photographed is not limited, but if the degree of deterioration is severe, the difference between the original image and the deteriorated image becomes large, and therefore the property of the original image cannot be estimated accurately (original Image estimation parameters cannot be calculated accurately). For this reason, there is a problem that the estimation accuracy of the restored image is deteriorated.

  The present invention has been made to solve such problems, and improves the accuracy of estimating the properties of the original image from the deteriorated image, and obtains a restored image with high accuracy even when the captured image is severely deteriorated. The purpose is to be able to.

  In order to solve the above-described problem, in the present invention, a degraded image obtained by imaging is divided into a plurality of frequency band components. Then, an intermediate restoration image is generated while sequentially estimating the properties of the original image of the corresponding frequency band component by using the degraded image subjected to frequency division from the low frequency band component to the high frequency band component step by step. A property of the original image is estimated from the intermediate restored image, and a restored image including all frequency band components is generated using the estimation parameter obtained thereby.

  According to the present invention configured as described above, even if the image obtained by imaging is severely deteriorated, the low-frequency region is less affected by the deterioration than the high-frequency region. The intermediate restored image can be obtained with high accuracy by obtaining the intermediate restored image. Then, since the estimated parameter of the original image is obtained using the accurate intermediate restored image obtained in this way, the estimated parameter of the original image can be obtained with high accuracy, and the restored image can be obtained with high accuracy. As described above, even when the captured image is severely deteriorated, the accuracy of estimating the property of the original image from the deteriorated image is improved, and the restored image can be obtained with high accuracy.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, in the present embodiment, an example will be described in which an original image without deterioration is estimated from a deteriorated image in which image quality deterioration has occurred due to camera shake, and a restored image close to the original image is generated. Hereinafter, the original deteriorated image obtained by imaging is denoted by g, the original image is denoted by f, and the restored image is denoted by h. Note that the shutter speed of the digital camera in which the image restoration apparatus according to the present embodiment is implemented is relatively fast and the camera shake is a constant linear motion (the camera shake amount is L and the camera shake direction is θ), and the gyro sensor or the like. Therefore, it is assumed that the camera shake amount L and the camera shake direction θ can be measured.

  FIG. 1 is a block diagram illustrating a functional configuration example of the image restoration apparatus 100 according to the present embodiment. As shown in FIG. 1, the image restoration apparatus 100 according to the present embodiment includes a wavelet transform unit 1, a transformed image synthesis unit 2, a system parameter estimation unit 3, an observation noise estimation unit 4, and an image restoration unit 5. Yes.

  The wavelet transform unit 1 corresponds to the frequency band dividing means of the present invention, and divides the deteriorated image g obtained by imaging into a plurality of frequency band components (M is an arbitrary integer equal to or greater than 2). In the following, the degraded images of the M frequency band components generated by the wavelet transform (hereinafter referred to as “wavelet transform images”) are denoted as g1, g2,..., GM in order from the low frequency band to the high frequency band. . Further, corresponding to the frequency bands of the wavelet transformed images g1, g2,..., GM, the frequency band components of the original image f are f1, f2,. Are denoted as h1, h2,..., HM.

In this case, the following equation holds.
g = g1 + g2 +... + gM
f = f1 + f2 + ... + fM
h = h1 + h2 + ... + hM
Here, the symbol + indicates a synthesis process. That is, it is shown that when the M wavelet transform images g1, g2,.

  The transformed image composition unit 2 corresponds to the degraded image composition means of the present invention, and the M wavelet transformed images g1, g2,..., GM divided by the wavelet transform unit 1 are extracted from the low frequency band components. Stepwise synthesis into high frequency band components. Specifically, the converted image composition unit 2 generates (M−1) composite deteriorated images g1 + g2, g1 + g2 + g3,..., G1 + g2 + g3 +.

  As will be described in detail below, the feature of this embodiment is that the original image f is stepwise from a low frequency band to a high frequency band using a plurality of frequency-divided wavelet transform images g1, g2,. This is to generate the restored image h by the Kalman filter while estimating the property of the corresponding frequency band component (determining the estimation parameter). In the image restoration method using the Kalman filter, a state space model including a system model and an observation model is defined, and a restored image h is obtained based on the state space model.

Here, a known algorithm of an image restoration method using a Kalman filter will be described. The image restoration process is performed for each line in the camera shake direction θ. For example, as shown in FIG. 2, when the camera shake direction θ is a horizontal direction, and each pixel value on one line of the camera shake direction θ in the original image f and the deteriorated image g is x _n and y _n , The system model and the observation model of the state space model are defined as the following (Expression 1) and (Expression 2), respectively.

  When the camera shake direction θ is an oblique direction that is neither horizontal nor vertical, each pixel value on one line in the camera shake direction θ is interpolated from the pixel values of a plurality of pixels around the position on the line. Ask for. Alternatively, the entire degraded image g is rotated by θ so that the camera shake direction θ becomes the horizontal direction, the restored image h is obtained using the rotated degraded image g, and then the restored image h is rotated backward by θ. You may do it.

In the system model, as shown in FIG. 3, from a plurality of adjacent pixel values in a certain range on the original image f (hereinafter referred to as “reference pixel values”), a plurality of adjacent pixels in another range Is estimated (hereinafter referred to as “estimated pixel value”). Specifically, the system model is a model for obtaining a matrix X _n of estimated pixel values by multiplying a matrix X _n−1 of reference pixel values by a prediction parameter matrix A and adding a system noise matrix W _n. ing.

The observation model is a model for obtaining the pixel value y _n of the degraded image g by multiplying the matrix X _n representing each pixel value of the original image f by a known blur function matrix C and adding the observation noise ν _n. It has become.

The prediction parameter matrix A and the system noise matrix W _n in the above system model are estimation parameters for estimating the property of the original image f. In the image restoration method using the Kalman filter, these estimated parameters are obtained, and the restored image h is obtained by sequentially estimating the pixel values _Xn of the original image f using the obtained estimated parameters.

The above is a known algorithm for an image restoration method using a Kalman filter. In contrast, in the present embodiment, in the system model, instead of the virtual original image f (degraded image g), the wavelet transform image g1 in the lowest frequency band or the low frequency band to the high frequency band is restored step by step. The prediction parameter matrix A and the system noise matrix W _n are obtained using the intermediate restored images h1, h2,. Further, in the observation model, the observation noise ν _n is obtained using the wavelet transformed images g1, g2,..., GM instead of the deteriorated image g.

  When the camera shake amount L is for two pixels (L = 2), the above-described (Expression 1) and (Expression 2) are expressed as the following (Expression 3) and (Expression 4), respectively. In this case, four estimation parameters of the prediction parameters α and β, the system noise w, and the observation noise ν are obtained, and the restored image h is obtained from these estimation parameters. Hereinafter, for simplification of description, an example in which L = 2 is described.

The system parameter estimation unit 3 shown in FIG. 1 obtains the prediction parameters α and β and the system noise w based on the system model shown in (Expression 3). At this time, the system parameter estimation unit 3 obtains the prediction parameters α ₁ and β ₁ and the system noise w ₁ using the wavelet transform image g1 in the lowest frequency band in the first stage. In the next stage, the system parameter estimation unit 3 uses the first intermediate restored image h1 + h2 restored based on the prediction parameters α ₁ and β ₁ and the system noise w _{1 and the} like, and the prediction parameters α ₂ and β ₂ and the system The noise w ₂ is obtained.

In the next stage, the system parameter estimation unit 3 uses the second intermediate restored image h1 + h2 + h3 restored based on the prediction parameters α ₂ and β ₂ and the system noise w _{2 and the} like, and the prediction parameters α ₃ and β ₃ and the system determine the noise _{w 3.} The system parameter estimation unit 3 repeatedly performs this operation, so that finally the prediction parameters α _M-1 (= α), β _M-1 (= β) and the system necessary for obtaining the restored image h are obtained. The noise w _M-1 (= w) is obtained.

The observation noise estimation unit 4 obtains the observation noise ν based on the observation model shown in (Expression 4). At this time, the observation noise estimator 4 obtains the observation noise ν ₁ by using the synthesized degraded image g1 + g2 of the wavelet transform image g1 in the lowest frequency band and the wavelet transform image g2 on the one-step high frequency side at the first stage. In the next stage, the observation noise estimation unit 4 obtains the observation noise ν ₂ by using the synthesized degraded image g1 + g2 + g3, which is also synthesized with the wavelet transform image g3 on the one-step high frequency side.

Further, in the next stage, the observation noise estimation unit 4 obtains the observation noise ν ₃ by using the synthesized degraded image g1 + g2 + g3 + g4 that is also synthesized with the wavelet transform image g4 on the one-step high frequency side. The observation noise estimation unit 4 repeats this operation to finally obtain the observation noise ν _M−1 (= ν) necessary for obtaining the restored image h.

  The image restoration unit 5 uses the estimated parameters obtained in a stepwise manner in the system parameter estimation unit 3 and the observation noise estimation unit 4 as described above, so that the intermediate restoration images h1, h2,. Are obtained step by step, and finally a restored image h is obtained.

At the initial stage, the image restoration unit 5 includes the first prediction parameters α ₁ and β ₁ and the system noise w ₁ obtained by the system parameter estimation unit 3, and the first observation noise ν ₁ obtained by the observation noise estimation unit 4. Are applied to (Equation 3) and (Equation 4) to obtain the first intermediate restored image h1 + h2 from the synthesized degraded image g1 + g2 generated by the converted image synthesis unit 2. The intermediate restored image h1 + h2 obtained here is input to the system parameter estimation unit 3.

In the next stage, the image restoration unit 5 uses the second prediction parameters α ₂ and β ₂ and the system noise w ₂ obtained by the system parameter estimation unit 3 and the second prediction parameter α ₂ obtained by the observation noise estimation unit 4. of by applying the observation noise [nu ₂ and (equation 3) and (equation 4), obtaining an intermediate restored image h1 + h2 + h3 of the second synthetic degraded image g1 + g2 + g3 generated by the conversion image synthesizing unit 2. The intermediate restored image h1 + h2 + h3 obtained here is input to the system parameter estimation unit 3.

  By repeating this operation, the image restoration unit 5 finally obtains the final restored image h1 + h2 +... + HM = h from the synthesized degraded image g1 + g2 +... + GM generated by the converted image synthesis unit 2. Ask. In this way, the restored image h obtained by the image restoration unit 5 is output from the image restoration device 100 to the outside as a final generated image.

  FIG. 4 is a block diagram illustrating an example in which the functions of the system parameter estimation unit 3, the observation noise estimation unit 4, and the image restoration unit 5 illustrated in FIG. Here, as an example, a case where the degraded image g is divided into three frequency bands (M = 3) will be described as an example. In FIG. 4, components having the same functions as those shown in FIG. 1 are denoted by the same reference numerals.

  In FIG. 4, the wavelet transform unit 1 generates three wavelet transform images g1, g2, and g3 by dividing the input degraded image g into three frequency band components. The transformed image synthesis unit 2 synthesizes the three wavelet transformed images g1, g2, and g3 divided by the wavelet transformation unit 1 step by step from a low frequency band component to a high frequency band component. Specifically, the converted image composition unit 2 generates two composite deteriorated images, g1 + g2 and g1 + g2 + g3.

The intermediate system parameter estimation unit 3-1 uses the wavelet transform image g1 that is a degraded image of the lowest frequency band component among the three frequency band components divided by the wavelet transform unit 1, and uses the wavelet transform image g1 Prediction parameters α ₁ and β ₁ and system noise w ₁ which are intermediate estimation parameters representing the properties of the intermediate image f1 + f2 including the frequency band component on the higher-frequency side are obtained.

Further, the intermediate observation noise estimation unit 4-1 generates a synthesized degraded image g1 + g2 that is the synthesized degraded image generated by the converted image synthesizing unit 2 and includes the lowest frequency band component and the frequency band component on the one-step high frequency side. Using this, the observation noise ν ₁ is obtained. The intermediate system parameter estimation unit 3-1 and the intermediate observation noise estimation unit 4-1 constitute intermediate parameter estimation means of the present invention.

The intermediate image restoration unit 5-1 corresponds to the intermediate restored image generation unit of the present invention, and includes prediction parameters α ₁ , obtained by the intermediate system parameter estimation unit 3-1 and the intermediate observation noise estimation unit 4-1. Using β ₁ , system noise w _1, and observation noise ν ₁ , an intermediate restored image h 1 + h 2 including the lowest frequency band component and the frequency band component on the one-step high frequency side is generated.

The system parameter estimation unit 3-2 uses the intermediate restored image h1 + h2 generated by the intermediate image restoration unit 5-1, and uses the intermediate restored image h1 + h2 as a prediction parameter α that is an estimation parameter representing the properties of the original image f1 + f2 + f3 = f. ₂ (= α), β ₂ (= β) and system noise w ₂ (= w) are obtained.

In addition, the observation noise estimation unit 4-2 is an image generated by the converted image combining unit 2 and includes a frequency band component included in the combined deterioration image g1 + g2 and a frequency band component on a higher frequency side than that of the frequency band component. An observation noise ν ₂ (= ν) is obtained using the image g1 + g2 + g3 (a degraded image g before wavelet transform may be used). The system parameter estimation unit 3-2 and the observation noise estimation unit 4-2 constitute a parameter estimation unit of the present invention.

  The image restoration unit 5-2 corresponds to the restored image generation unit of the present invention, and the prediction parameters α and β obtained by the system parameter estimation unit 3-2 and the observation noise estimation unit 4-2, the system noise w. Then, using the observation noise ν, a restored image h1 + h2 + h3 = h including all frequency band components is generated.

  4 shows an example in the case of M = 3, but in the case of M = 4, between the intermediate image restoration unit 5-1, the system parameter estimation unit 3-2, and the observation noise estimation unit 4-2. In addition, another set of intermediate system parameter estimation unit, intermediate observation noise estimation unit and intermediate image restoration unit are inserted. The intermediate system parameter estimation unit and the intermediate observation noise estimation unit inserted during this period constitute the upper intermediate parameter estimation unit of the present invention, and the inserted intermediate image restoration unit constitutes the upper intermediate restored image generation unit of the present invention. .

In this case, the intermediate system parameter estimator of the upper intermediate parameter estimator uses the intermediate restored image h1 + h2 generated by the intermediate image restoration unit 5-1, and uses the intermediate restored image h1 + h2 to have a frequency band component and one step further therefrom. Prediction parameters α ₂ and β ₂ and system noise w ₂ representing the properties of the upper intermediate image f1 + f2 + f3 including the frequency band component on the high frequency side are obtained.

In addition, the observation noise estimation unit of the upper intermediate parameter estimation unit generates the frequency band component of the image generated by the converted image synthesis unit 2 and included in the synthesized degraded image g1 + g2, and the frequency band component on the higher frequency side by one step. The observed noise ν ₂ is obtained by using the combined degraded image g1 + g2 + g3.

The intermediate restored image h1 + h2 has an intermediate image restoration unit corresponding to the upper intermediate restored image generation means, using the prediction parameters α ₂ and β ₂ , the system noise w ₂ and the observation noise ν ₂ obtained as described above. A high-order intermediate restored image h1 + h2 + h3 including a frequency band component and a frequency band component on the higher frequency side by one stage is generated.

Then, the system parameter estimation unit 3-2 uses the upper intermediate restored image h1 + h2 + h3 generated by the upper intermediate image restoration unit, and is a prediction parameter that is an estimation parameter representing the properties of the original image f1 + f2 + f3 + f4 = f including all frequency band components. α ₃ (= α), β ₃ (= β) and system noise w ₃ (= w) are obtained.

Further, the observation noise estimation unit 4-2 uses the synthesized degraded image g1 + g2 + g3 + g4 generated by the transformed image synthesis unit 2 (may use the degraded image g before wavelet transform), and observes noise ν ₃ (= ν )

  Further, the image restoration unit 5-2 uses the prediction parameters α and β, the system noise w, and the observation noise ν obtained by the system parameter estimation unit 3-2 and the observation noise estimation unit 4-2, and uses all the frequency band components. A restored image h1 + h2 + h3 + h4 = h is generated.

  On the other hand, when M = 2, the system parameter estimation unit 3-2, the observation noise estimation unit 4-2, and the image restoration unit 5-2 are unnecessary, and the restored image h1 + h2 output from the intermediate image restoration unit 5-1 The restored image h is the final product. In this case, the wavelet transform unit 1 divides the degraded image g obtained by imaging into two frequency band components. The system parameter estimator 3-1 and the observation noise estimator 4-1 use the wavelet transformed image g1 of the low frequency band component and the synthesized degraded image g1 + g2 (may be the degraded image g) generated by the transformed image synthesizer 2. Then, estimation parameters α, β, w, and ν representing the properties of all frequency band components are obtained. Then, the intermediate image restoration unit 5-1 generates a restored image h including all frequency band components using the estimation parameters α, β, w, and ν obtained as described above.

  Next, the operation of the image restoration apparatus 100 according to the present embodiment configured as described above will be described. FIG. 5 is a flowchart illustrating an operation example of the image restoration apparatus 100 according to the present embodiment. Note that the flowchart of FIG. 5 illustrates an operation example (an operation example of the image restoration apparatus 100 illustrated in FIG. 4) in the case where the camera shake amount L = 2 and the frequency division number M = 3.

  In FIG. 5, the wavelet transform unit 1 divides the input degraded image g into three frequency band components by wavelet transform (step S1). Further, the converted image combining unit 2 generates two combined deteriorated images g1 + g2, g1 + g2 + g3 using the three wavelet converted images g1, g2, and g3 divided by the wavelet converting unit 1 (step S2).

Next, the intermediate system parameter estimation unit 3-1 uses the wavelet transform image g1 of the lowest frequency band component generated by the wavelet transform unit 1, and uses the prediction parameter α ₁ that is an intermediate estimation parameter representing the nature of the intermediate image f1 + f2. , Β ₁ and system noise w ₁ are obtained. Further, the intermediate observation noise estimation unit 4-1 obtains an observation noise ν ₁ that is an intermediate estimation parameter by using the synthesized degraded image g1 + g2 generated by the converted image synthesis unit 2 (step S3).

Then, the intermediate image restoration unit 5-1 uses the intermediate estimation parameters α ₁ , β ₁ , w ₁ , and ν ₁ obtained by the intermediate system parameter estimation unit 3-1 and the intermediate observation noise estimation unit 4-1. An intermediate restored image h1 + h2 is obtained from (Expression 3) and (Expression 4) (Step S4).

  Next, the system parameter estimation unit 3-2 uses the intermediate restored image h1 + h2 generated by the intermediate image restoration unit 5-1, and uses prediction parameters α, β representing properties of the original image f including all frequency band components, and The system noise w is obtained. Further, the observation noise estimation unit 4-2 obtains the observation noise ν by using the synthesized degraded image g1 + g2 + g3 generated by the converted image synthesis unit 2 (step S5).

  Then, the image restoration unit 5-2 uses (Equation 3) and (Equation 4) using the estimated parameters α, β, w, and ν obtained by the system parameter estimation unit 3-2 and the observation noise estimation unit 4-2. ) To generate a restored image h including all frequency band components (step S6).

  As described above in detail, the image restoration apparatus 100 according to the present embodiment generates the wavelet transform images g1, g2,..., GM by dividing the degraded image g obtained by imaging into a plurality of frequency band components. . Then, the intermediate restored images h1 + h2, h1 + h2 + h3,... Are sequentially estimated using the wavelet transform images g1, g2,. Are sequentially generated, and finally a restored image h including all frequency band components is generated.

  According to this embodiment configured as described above, even if the degree of deterioration of the deteriorated image g obtained by imaging is severe, if only the low frequency band component is extracted, the influence of the deterioration is small. The intermediate restored image can be obtained with high accuracy by obtaining the intermediate estimated parameter from the image and generating the intermediate restored image. Then, since the estimated parameter of the original image f is obtained using the accurate intermediate restored image obtained in this way, the estimated parameter of the original image f can be obtained with high accuracy. Thereby, even when image quality degradation of the captured degraded image g is severe, the accuracy of estimating the property of the original image f from the degraded image g can be improved, and the restored image h can be obtained with high accuracy.

  In the above embodiment, the case where the shutter speed of the digital camera is relatively fast and the camera shake is a linear motion has been described as an example. However, even when the camera shake is not a linear motion, it can be applied by applying a known technique. Is possible. For example, as shown in FIG. 6, image restoration can be performed by sequentially performing image restoration processing in the −45 °, 90 °, 45 °, and 0 ° directions.

  In the above-described embodiment, an example in which the original image f is estimated from the degraded image g in which image quality degradation has occurred due to camera shake and the restored image h is generated has been described. The same applies to the case where the restored image h is generated from the deteriorated image.

  Moreover, although the said embodiment demonstrated the example which uses wavelet transformation as a process for dividing the degradation image g into a several frequency band component, this invention is not limited to this. For example, it is possible to perform frequency division using processing such as Fourier transform.

  In addition, each of the above-described embodiments is merely an example of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. In other words, the present invention can be implemented in various forms without departing from the spirit or main features thereof.

It is a block diagram which shows the function structural example of the image restoration apparatus by this embodiment. It is a figure which shows each pixel value on 1 line of the camera shake direction in an original image and a degradation image. It is explanatory drawing of a system model. FIG. 2 is a block diagram illustrating an example in which functions of a system parameter estimation unit, an observation noise estimation unit, and an image restoration unit illustrated in FIG. 1 are disassembled in detail. It is a flowchart which shows the operation example of the image restoration apparatus by this embodiment. It is a figure for demonstrating the image restoration method in case a camera shake direction is not a linear motion.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Wavelet transformation part 2 Conversion image composition part 3 System parameter estimation part 4 Observation noise estimation part 5 Image restoration part 3-1 Intermediate system parameter estimation part 4-1 Intermediate observation noise estimation part 5-1 Intermediate image restoration part 3-2 System Parameter estimation unit 4-2 Observation noise estimation unit 5-2 Image restoration unit

Claims (3)

Frequency band dividing means for dividing a degraded image obtained by imaging into a plurality of frequency band components;
A deteriorated image synthesizing means for stepwise synthesizing degraded images of a plurality of frequency band components divided by the frequency band dividing means from a low frequency band component to a high frequency band component;
Of the plurality of frequency band components divided by the frequency band dividing means, a deteriorated image of the lowest frequency band component and an image generated by the deteriorated image synthesizing means, the lowest frequency band component and one step higher frequency side than that Intermediate parameter estimation means for obtaining an intermediate estimation parameter representing a property of the lowest frequency band component and a frequency band component on the one-step high frequency side from the lowest frequency band, using a synthetic degradation image including the frequency band component of
An image generated by the degraded image synthesizing means, the synthesized degraded image including the lowest frequency band component and a frequency band component at one stage higher than the lowest frequency band component, and the intermediate estimation parameter obtained by the intermediate parameter estimating means An intermediate restored image generating means for generating an intermediate restored image including the lowest frequency band component and a frequency band component on the one-step high frequency side from the lowest frequency band;
The intermediate restored image generated by the intermediate restored image generation unit, the image generated by the degraded image synthesizing unit, the frequency band component of the intermediate restored image, and the frequency band component at a higher frequency side than that Parameter estimation means for obtaining an estimation parameter representing the properties of all frequency band components using a synthesized degraded image including:
An image generated by the degraded image synthesizing means, including a synthesized degraded image including the frequency band component of the intermediate restoration image and a frequency band component on the higher frequency side of the intermediate restored image, and the parameter estimation means An image restoration apparatus comprising: a restored image generation unit that generates a restored image including all the frequency band components using an estimation parameter. The intermediate restored image generated by the intermediate restored image generation unit, the image generated by the degraded image synthesizing unit, the frequency band component of the intermediate restored image, and the frequency band component at a higher frequency side than that An upper intermediate parameter estimating means for obtaining an upper intermediate estimation parameter representing the property of the frequency band component of the intermediate restored image and the frequency band component on the higher frequency side of the intermediate restored image using the synthesized degraded image including:
An image generated by the degraded image synthesizing means, which is obtained by the synthesized degraded image including the frequency band component of the intermediate restored image and the frequency band component on the higher frequency side by one stage and the upper intermediate parameter estimating means. Using the upper intermediate estimation parameter, the upper intermediate restored image generating means for generating the upper intermediate restored image including the frequency band component of the intermediate restored image and the frequency band component on the higher frequency side than that of the intermediate restored image,
In place of the processing according to claim 1, the parameter estimation unit is the upper intermediate restored image generated by the upper intermediate restored image generating unit and the image generated by the degraded image synthesizing unit, Using the synthesized degraded image including the frequency band component of the intermediate restored image and the frequency band component on the higher frequency side one step from that, obtain an estimation parameter representing the properties of all frequency band components,
In place of the processing according to claim 1, the restored image generating means is an image generated by the degraded image synthesizing means, the frequency band component of the upper intermediate restored image, and one step further on the higher frequency side. 2. The restored image including all the frequency band components is generated using the synthesized degraded image including the frequency band component and the upper intermediate estimation parameter obtained by the upper intermediate parameter estimating unit. The image restoration apparatus described. A first step of dividing a degraded image obtained by imaging into a plurality of frequency band components;
Among the plurality of frequency band components, a degraded image of the lowest frequency band component and a synthesized degraded image including the lowest frequency band component and a frequency band component on one higher frequency side than the lowest frequency band component, A second step of obtaining an intermediate estimation parameter representing the nature of the frequency band component one step higher than the lowest frequency band;
Using the synthesized degraded image including the lowest frequency band component and the frequency band component on the one-step high frequency side, and the intermediate estimation parameter, the lowest frequency band component and the frequency band component on the one-step high frequency side from the lowest frequency band A third step of generating an intermediate restored image including:
First, an estimated parameter representing the properties of all frequency band components is obtained using the intermediate restored image and a synthesized deteriorated image including the frequency band component of the intermediate restored image and the frequency band component on the higher frequency side of the intermediate restored image. 4 steps,
Generating a restored image including all the frequency band components by using the synthesized degraded image including the frequency band component of the intermediate restored image and the frequency band component on the higher frequency side of the intermediate restored image and the estimated parameter; And an image restoration method.