JP2014099048A - Image restoration device and image restoration method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce operation amount in restoration processing when restoring a deteriorated color image using a known PSF.SOLUTION: In an image restoration device 3, an original image estimation section 23 evaluates the plausibility of an estimated original image based on a difference between a pixel value of a deteriorated photographed image and an estimated pixel value of the deteriorated image of the estimated original image and a PSF and based on a function for evaluating the sparsity in gradient of the pixel values in the estimated original image. The function for evaluating the sparsity is subjected to a weighting using a weight parameter. By using a weight parameter which is updated in a restoration processing of one representative color channel for restoration processing of an auxiliary color channel, the operation amount in restoration processing in the auxiliary color channel is reduced.

Description

  The present invention relates to an image restoration apparatus and an image restoration method for restoring a deteriorated image, and more particularly, to an image restoration apparatus and an image restoration method suitable for restoring blurring or the like of an image generated in a photographing process.

  In shooting using a digital still camera or the like, there may be a case where an image is blurred due to out-of-focus or camera shake. The degraded image in which this blur occurs is generated by convolution of the original image (ideal image) and a point spread function (hereinafter referred to as “PSF (Point Spread Function)”) representing a spatial filter that degrades the original image. You can think of it. Conventionally, as a method for restoring an original image from a deteriorated image, a non-blind method (Non-Blind Deconvolution) is known in which an original image is restored using a known PSF obtained theoretically or actually measured. Inverse filters and Wiener filters that perform deconvolution are known.

  Further, as the non-blind method, there is known an LR (Lucy-Richardson) method (see Non-Patent Document 1) that performs maximum likelihood estimation of an original image using a Bayesian theorem and a stochastic statistical method. Furthermore, a method for improving the restoration accuracy of the original image by repeating the estimation process of the original image that requires a predetermined iterative process as in the LR method by the IRLS algorithm (hereinafter referred to as “IRLS application”). Method ") (see Non-Patent Document 2).

W.H.Richardson, "Bayesian-Based Iterative Method of Image Restoration", Journal of Optical Society of America, Vol.62, No.1,1972 A. Levin, et al. "Deconvolution using natural image priors", http://groups.csail.mit.edu/graphics/CodedAperture/SparseDeconv-LevinEtAl07.pdf, (a supplemental document of "Image and Depth from a Conventional Camera with a Coded Aperture ", ACM SIGGRAPH 2007)

  However, when restoring a color-degraded image by the conventional technique described in Non-Patent Document 2, it is necessary to perform an iterative process for estimating an original image for each color channel. There is a problem that the processing time and the processing load increase due to the enormous volume.

  The present invention has been devised in view of such problems of the prior art, and it is possible to reduce the amount of computation of restoration processing when restoring a color-degraded image using a known PSF. An object of the present invention is to provide an image restoration apparatus and an image restoration method.

  The image restoration device of the present invention is an image restoration device that restores a color degradation image based on a known point spread function, an image input unit to which the degradation image is input, and an original image before degradation of the degradation image An original image estimation unit that acquires an estimated original image for each color channel, and a restored image composition unit that generates a restored image by synthesizing the estimated original image for each color channel, The original image estimation unit evaluates the likelihood of the estimated original image based on an evaluation index including a predetermined parameter for one representative color channel selected from the plurality of color channels, while the auxiliary image other than the representative color channel For the color channel, using the value of the parameter when the estimated original image for the representative color channel is evaluated to be likely And evaluating the likelihood of the estimated original image based on the evaluation index.

  As described above, according to the present invention, when a color deteriorated image is restored using a known PSF, there is an excellent effect that the amount of computation of the restoration process can be reduced.

1 is a block diagram showing a schematic configuration of an image processing system including an image restoration apparatus according to the present invention. The flowchart which shows the flow of the image restoration method by the image restoration apparatus shown in FIG. The flowchart which shows the detail of the decompression | restoration process (ST103) regarding the representative channel in FIG. The flowchart which shows the detail of the decompression | restoration process (ST104) regarding the auxiliary channel in FIG.

  A first invention made to solve the above problem is an image restoration device for restoring a color degraded image based on a known point spread function, an image input unit to which the degraded image is input, and the degradation An original image estimation unit that acquires an estimated original image for each color channel by estimating an original image before image degradation, and a restored image that generates a restored image by combining the estimated original image for each color channel The original image estimation unit evaluates the likelihood of the estimated original image based on an evaluation index including a predetermined parameter for one representative color channel selected from the plurality of color channels, For auxiliary color channels other than the representative color channel, the estimated original image for the representative color channel is evaluated as being likely. Using the value of the parameter, and evaluating the likelihood of the estimated original image based on the evaluation index.

  According to the image restoration apparatus of the first invention, in the first invention, the likelihood of the estimated original image for the representative color channel is evaluated when the color deteriorated image is restored using the known PSF. Since the auxiliary color channel processing is performed based on the parameters updated in the processing, it is possible to reduce the amount of computation of the restoration processing.

  In the second invention, the evaluation index includes a difference between a pixel value of the input deteriorated image and a pixel value of the deteriorated image estimated based on the estimated original image and the point spread function, and the estimated original. A function for evaluating the sparsity of the gradient of the pixel value in the image, and the parameter is relative to the difference for each pixel in the evaluation of the estimated original image by weighting the function for evaluating the sparsity. The original image estimation unit performs an iterative process for sequentially updating the estimated original image until the estimated original image is evaluated as being likely for the representative color channel, and The weight parameter is updated based on the finally updated estimated original image, and the iterative process is newly performed based on the updated weight parameter. A series of processes to be started is repeated a predetermined number of times, and for the auxiliary color channel, the estimated original image is evaluated until the estimated original image is evaluated as being likely based on the weight parameter finally updated for the representative color channel. A repetitive process for sequentially updating images is performed.

  According to the image restoration apparatus of the second invention, when restoring a color-degraded image using a known PSF by the IRLS application method, the auxiliary color channel is updated based on the weight parameter updated in the processing of the representative color channel. Since the process is performed, it is possible to omit a repetition step of the iterative process in the auxiliary color channel, and it is possible to reduce the calculation amount of the restoration process.

  According to a third aspect, in the first or second aspect, the representative color channel is a color channel having the largest number of high frequency components among the plurality of color channels.

  According to the image restoration apparatus of the third aspect of the invention, since a color channel having a high importance can be selected as a representative color channel based on the magnitude of the high frequency component, even when the repetition step of the iterative process in the auxiliary color channel is omitted. A highly accurate restoration process is possible.

  According to a fourth invention, in any one of the first to third inventions, the degraded image is an RGB color image, the representative color channel is a G channel, and the auxiliary color channel is an R channel. And B channel.

  According to the image restoration device of the fourth aspect of the invention, the G channel (green) with high visibility is designated as the representative channel, so that even when the repetition step of the iterative process in the R channel and the B channel is omitted, high accuracy is achieved. Can be restored.

  In the fifth invention, the evaluation index includes a difference between a pixel value of the input deteriorated image and a pixel value of the deteriorated image estimated based on the estimated original image and the point spread function, and the estimated original. A function for evaluating the sparsity of the gradient of the pixel value in the image, and the parameter is relative to the difference for each pixel in the evaluation of the estimated original image by weighting the function for evaluating the sparsity. A weight parameter that determines a priority degree, and the original image estimation unit performs an iterative process for sequentially updating the estimated original image until the estimated original image is evaluated as being likely for the representative color channel, and finally The weight parameter is updated based on the updated estimated original image, and the iteration process is updated based on the updated weight parameter. A series of processes starting from the above is repeated a predetermined number of times, and the estimated original image is evaluated as being likely based on the weight parameter obtained from the estimated original image finally updated for the representative color channel for the auxiliary color channel. Until then, the iterative process for sequentially updating the estimated original image is performed.

  According to the image restoration apparatus of the fifth aspect of the present invention, when restoring a color-degraded image using a known PSF by the IRLS application method, the weight obtained from the estimated original image finally updated for the representative color channel Since the auxiliary color channel processing is performed based on the parameters, it is possible to omit the repetition step of the iterative processing in the auxiliary color channel, and it is possible to reduce the calculation amount of the restoration processing. In particular, since the final restored image of the representative channel is used, improvement in restoration accuracy in the auxiliary color channel can be expected.

  The sixth invention is an image restoration method for restoring a color deteriorated image based on a known point spread function, an image input step in which the deteriorated image is input, and an original image before the deterioration of the deteriorated image. An original image estimation step for obtaining an estimated original image for each color channel by estimating the estimated original image for each color channel, and a restored image combining step for generating a restored image by synthesizing the estimated original image for each color channel, In the original image estimation step, the difference between the pixel value of the input degraded image and the pixel value of the degraded image estimated based on the estimated original image and the point spread function, and the gradient of the pixel value in the estimated original image Based on a function for evaluating sparsity, the likelihood of the estimated original image is evaluated, and the function for evaluating sparsity includes the evaluation of the estimated original image. Weighting is performed using a weight parameter that determines a relative priority with respect to the difference for each pixel, and in the original image estimation step, the estimated original image is selected for one representative color channel selected from a plurality of color channels. Until the estimated original image is evaluated to be plausible, and the weighting parameter is updated based on the finally updated estimated original image, and the updated weighting parameter is updated. Based on the weight parameter finally updated with respect to the representative color channel, the estimated original image is repeatedly generated for the auxiliary color channel other than the representative color channel by repeating a series of processes for newly starting the iterative process based on the predetermined number of times. Until the estimated original image is sequentially updated until An image restoration method, characterized by.

  According to the image restoration method of the sixth aspect of the present invention, when restoring a color-degraded image using a known PSF by the IRLS application method, the auxiliary color channel is based on the weight parameter updated in the restoration process of the representative color channel. Therefore, it is possible to omit the repetitive step of the iterative process in the auxiliary color channel, and it is possible to reduce the calculation amount of the reconstructing process.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a schematic configuration of an image processing system provided with an image restoration apparatus according to the present invention. The image processing system 1 performs a restoration process of a captured deteriorated image (an image in which a blur caused by a focal blur or camera shake at the time of shooting) is obtained, and acquires an original image. An imaging device 2 that generates an image (here, an RGB image) and an image restoration device 3 that performs restoration processing of the degraded image when the digital color image input from the imaging device 2 is a degraded image are provided. ing.

  The imaging device 2 is composed of a digital still camera, but other known devices such as a digital video camera can be used as the imaging device 2 as long as at least a subject image can be recorded as digital image data.

  The image restoration device 3 includes a computer system having an image processing function, and an image input unit 11 to which image data transmitted from the imaging device 2 is input via an interface (not shown), and input image data (degraded image). An image restoration unit 12 that restores the original image from the image, a display data generation unit 13 that generates display data for display display from the restored original image, and a display unit 14 that includes a display that displays the original image to the user. And a recording unit 15 including a storage device that records the original image.

  The image input unit 11 stores the image data from the imaging device 2 in a memory (not shown), and outputs the image data to the image restoration unit 12 as necessary. The image restoration unit 12 includes a PSF acquisition unit 21 that acquires a theoretically or actually measured known PSF, and a plurality of color channels (color information separated for each component. Here, R channel, G A representative channel selection unit 22 that selects one representative channel from the channel and the B channel, and an estimated original image (hereinafter referred to as “degraded image”) for each color channel. , Which is referred to as “estimated original image”), and a restored image combining unit 24 that generates a restored image by synthesizing the estimated original image for each color channel.

  Here, the PSF acquisition unit 21 acquires a PSF based on camera shake information detected by a sensor (such as an acceleration sensor) (not shown) included in the imaging device 2. However, the present invention is not limited to this, and the PSF may be acquired by other known methods.

  At least a part of the processing functions of the above-described units of the image restoration device 3 can be realized by software processing in which a program such as an image processing application is executed by a CPU (not shown). However, these processing functions are not limited to those based on software processing, and may be realized using hardware capable of executing similar processing.

  FIG. 2 is a flowchart showing the flow of the image restoration method by the image restoration apparatus. In the image restoration device 3, color image data photographed by the imaging device 2 is sequentially input to the image input unit 11 (ST101). When image data is input, the representative channel selection unit 22 acquires information on the spatial frequency in the input image data, and selects a color channel (for example, an R channel in a reddish degraded image) having the highest proportion of high-frequency components. A representative color channel is selected (ST102). At the same time, color channels other than the representative color channel (eg, G channel and B channel) are selected as auxiliary color channels.

  Note that the selection of the representative color channel in step ST102 is not necessarily performed every time image data is input, and the same representative color channel can be used for a series of moving image data and the like. Alternatively, the user can designate the representative color channel to be selected in advance. For example, the user can designate the G channel as a representative channel in consideration of the high visibility of green.

  Next, in the image restoration device 3, the original image estimation unit 23 performs an original image restoration process for the representative color channel selected in step ST102 (ST103). This representative color channel restoration process is performed according to the conventional IRLS application method. Subsequently, the original image estimation unit 23 performs original image restoration processing for each auxiliary color channel (ST104). Although details will be described later, in step ST104 for each auxiliary color channel, the restoration process is made efficient by using the processing result of step ST103 for the representative color channel. Thereafter, in the image restoration device 3, the restored image synthesis unit 26 synthesizes the estimated original images of the respective color channels output in steps ST103 and ST104 (ST105), and the synthesized color restored image is displayed on the display unit 14. At the same time, it is recorded in the recording unit 15 (ST106).

Here, the principle of restoring a deteriorated image by the above-described IRLS application method will be briefly described. The original image x before degradation and the captured degradation image y can be expressed by the following equation (1) using an N × N convolution matrix C _f based on f which is a known PSF.

  In the IRLS application method, in order to estimate the original image x satisfying the above equation (1), x that maximizes the posterior probability as shown in the following equation (2) is obtained.

  Here, P (y | x) can be expressed as the following equation (3), assuming that the noise included in the degraded image is Gaussian noise with variance η that follows the same probability distribution.

Furthermore, the prior probability P (x) of x can be expressed as the following equation (4) using the set of filters g _k . Here, regarding a predetermined function ρ, i represents a pixel, and g _{i, k} represents a k-th filter centered on the pixel i.

  By taking the logarithm of the above equations (2) to (4), an evaluation value for evaluating the likelihood of the estimated original image x can be expressed as the following equation (5).

In the above equation (5), the first term represents the difference (residual) between the image obtained by degrading the estimated original image x by PSF and the captured image (degraded image y), and the second term represents the estimated original image. It is a function for evaluating how sparsity (the property that there are many 0 components) the gradient of the luminance value (pixel value) in the image x is. That is, the difference between C _f x obtained by convolving f (PSF) into x (original image) and y (degraded image) is small (evaluation index 1), and the gradient distribution of x (original image) is sparse. (Evaluation index 2) is defined as an evaluation value, and x that minimizes the evaluation value is obtained. Note that the weighting coefficient w (= αη ² ) added to the evaluation index 2 in equation (5) is a scalar value that determines the relative priority with respect to the evaluation index 1.

  Therefore, x is calculated so that the differentiation of the above equation (5) becomes zero. If the sub-expression of the differentiation with respect to x in the equation (5) is expressed by the following equation (6), it can be reduced to a problem solved for the linear equation Ax = b. ) Makes it possible to estimate x.

In the above equation (5), the weighting factor w uniformly acts on the entire image. When the weighting factor w is increased, the luminance gradient of the restored image becomes gentle (that is, the sharpness is lowered). On the other hand, when the weighting coefficient w is reduced (closer to 0), the luminance gradient of the restored image tends to be steep (that is, sharpness becomes higher), but the random noise component of the original image is emphasized or the contour is increased. There is an adverse effect that ripple ringing noise occurs around. Therefore, in the restored image, the degree of restoration is weakened for pixels with a large luminance gradient (ie, emphasizing sparsity), and the degree of restoration is increased for pixels with a small luminance gradient (ie, sparsity is neglected). Further, a weight matrix W _k (weight parameter) with a weight for each pixel is further introduced so that the restoration degree can be appropriately changed for each pixel.

On the other hand, the magnitude of the luminance gradient for each pixel of the restored image is unknown before the restoration process, and there is no point in diverting the gradient of the captured degraded image. The image restoration step is repeated a plurality of times by introducing the IRLS algorithm, and the restoration image quality is improved by calculating the weight matrix W _k from the restored image obtained in each step.

Therefore, the term for evaluating the sparsity of the luminance gradient in equation (6) is multiplied by the weight matrix W _k to obtain the following equation (7), and the linear equation A′x = b is solved.

FIG. 3 is a flowchart showing details of the restoration process (ST103) regarding the representative channel in FIG. When the image restoration unit 12 acquires luminance information and the like for the representative channel (for example, G channel) in the color image data, the original image estimation unit 23 initializes the values of the weight matrix W _k in the above equation (7) ( Further, an initial solution for the estimated original image x is set (ST202). Here, the initial solution is set as x = y (captured degraded image).

  Next, the original image estimation unit 23 calculates a residual r (= A′x−b) based on the above equation (7) (ST203), and in subsequent steps ST204 to ST209, it is estimated by iterative processing based on the CG method. Iterate until the residual r becomes sufficiently small while updating the original image x sequentially (that is, until the finally updated estimated original image x is evaluated as likely).

In step ST204, the residual r is evaluated depending on whether or not ρ represented by the following equation (8) is equal to or less than a predetermined threshold (a value that can be regarded as substantially 0).
ρ = r ^T r (8)

When the original image estimation unit 23 determines in step ST204 that the residual r is not small (ρ exceeds the threshold) (No), in the subsequent step ST205, the prediction vector (corrected) represented by the following equation (9) is corrected. An update ratio (correction coefficient) β of (direction vector) p is calculated. Here, ρ _bak is the previous value of ρ. In the first iteration process, the update ratio β is not calculated and is set to the prediction vector p = r (initial value).
β = ρ / ρ _bak (9)

In the subsequent step ST206, the prediction vector p is updated as shown in the following equation (10). Here, p _bak is the previous value of p. In the first iteration process, this step ST206 is omitted (that is, p = r).
p = r + βp _bak Formula (10)

In subsequent step ST207, an update ratio (correction coefficient) α for the estimated solution of the estimated original image x is calculated as shown in the following equation (11). Here, q = A′p.
α = ρ / (p ^T q) Equation (11)

In the subsequent step ST208, the estimated solution of the estimated original image x is updated based on the prediction vector p and the update ratio α as shown in the following equation (12). Here, x _bak is the previous value of x.
x = x _bak + αp (12)

In the subsequent step ST209, as shown in the following equation (13), the residual r is updated based on the update ratio α, and the process returns to step ST204 to evaluate the updated residual r.
r = r _bak + αq (13)

  If the original image estimation unit 23 repeats the iterative process of steps ST204 to ST209 and finally determines that the residual r is sufficiently small (Yes) in step ST204, the original image estimation unit 23 ends the first step ST1 of the IRLS algorithm and performs step Proceed to ST210.

In step ST210, based on the result of the first step ST1 described above, a weight matrix W _k is calculated by the following equation (14). Here, a and ε are constants. Further, D _k = C _gk x, and the final value of the first step ST1 is used for x.
W _k = (max {| D _k |, ε}) ^a-2 Expression (14)

  Thereafter, as the second step ST2 of the IRLS algorithm, steps ST211 and ST212 similar to the steps ST202 and ST203 are performed, and further, steps ST213 to 218 which are the same iterative processes as the steps ST204 to ST209 are performed.

  When the original image estimation unit 23 repeats the iterative processing of steps STST213 to 218 and finally determines that the residual r is sufficiently small (Yes) in step ST213, the original image estimation unit 23 ends the second step ST2 of the IRLS algorithm and performs step Proceed to ST219.

In step ST219, similar to step ST210, a new weight matrix W _k is calculated based on the result of the second step ST2. Thereafter, as the third step ST3 of the IRLS algorithm, steps ST220 and 221 similar to steps ST211 and 212 are performed, and further, steps ST222 to 227 which are the same iterative processes as steps ST213 to ST218 are performed.

  If the original image estimation unit 23 finally determines that the residual r is sufficiently small (Yes) in step ST222, the original image estimation unit 23 ends the third step ST3 of the IRLS algorithm, and represents the estimated solution of the final estimated original image x. A restored image relating to the color channel is output (ST228). Here, an example (steps ST1 to ST3) in which the number of repeated steps is three has been shown, but the present invention is not limited to this, and the number of steps can be changed.

FIG. 4 is a flowchart showing details of the restoration process (ST104) for the auxiliary channel in FIG. In the restoration process related to the auxiliary channel, information on the weight matrix W _k of the restoration process related to the representative channel is used. First, the original image estimation unit 23 is finally updated in the restoration process for the representative channel described above together with the luminance information about the auxiliary channel (for example, the R channel or the B channel) in the color image data (in FIG. 3). Information on the weight matrix W _k (calculated in step ST219) is acquired (ST301).

  Thereafter, the image restoration unit 12 sets an initial solution of the estimated original image x related to the auxiliary channel (step ST302), similarly to step ST220 in FIG. 3 described above, and subsequently, similarly to step ST221 and the like in FIG. A residual r is calculated. Thereafter, steps ST304 to ST309, which are the same iterative processes as steps ST222 to ST227 in FIG.

  If the original image estimation unit 23 finally determines that the residual r is sufficiently small (Yes) in step ST304, the original image estimation unit 23 does not repeat the three steps as in the case of the representative channel described above, but performs the single step. The estimated solution of the estimated original image x is output as a restored image (ST310).

  In step ST301, the weight matrix Wk calculated in step ST219 in FIG. 3 is acquired. Alternatively, the restored image output in ST228 is acquired, and the restored image is used to perform the same process as ST219 described above. The weight matrix Wk calculated by the method may be used. The restored image from which the weight matrix Wk calculated in ST219 is calculated is an estimated image obtained in the second step ST2 of the representative channel. On the other hand, since the estimated image obtained in ST3 of the same channel (the restored image output in ST228) has better restoration accuracy than the estimated image in ST2, the estimated image in ST3 is used as a calculation source for the auxiliary channel weight matrix. By doing so, it can be expected that the weight matrix can be made more suitable.

As described above, in the image restoration device 3 and the image restoration method thereof, the weight matrix W _k updated in the restoration process of the representative color channel when the color deteriorated image is restored using the known PSF by the conventional IRLS application method. Since the auxiliary color channel restoration process is performed using the above, it is possible to omit the repetition step of the repetition process in the auxiliary color channel, and to reduce the calculation amount of the restoration process.

As mentioned above, although this invention was demonstrated based on specific embodiment, these embodiment is an illustration to the last, Comprising: This invention is not limited by these embodiment. For example, in the above-described embodiment, an example using an RGB image is shown, but the present invention is not limited to this, and other color models (for example, YUV) can also be used. Also in this case, one color channel (for example, Y channel) is selected as the representative channel, and the other channels (for example, U channel and V channel) are selected as auxiliary channels. Further, the parameters diverted in the auxiliary color channel restoration process (parameters calculated in the representative color channel restoration process) are not limited to the weight matrix W _k of the above embodiment, and the amount of computation of the restoration process can be reduced. Various modifications are possible as long as possible. Note that the components of the image restoration apparatus and the image restoration method according to the present invention described in the above embodiments are not necessarily all necessary, and can be appropriately selected as long as they do not depart from the scope of the present invention. is there.

  The image restoration apparatus and the image restoration method according to the present invention can reduce the amount of computation of restoration processing when restoring a color-degraded image using a known PSF. The present invention is useful as an image restoration apparatus and an image restoration method for restoring blur and the like.

DESCRIPTION OF SYMBOLS 1 Image processing system 2 Imaging device 3 Image restoration device 11 Image input part 12 Image restoration part 21 PSF acquisition part 22 Representative channel selection part 23 Original image estimation part

Claims (6)

An image restoration device that restores a color-degraded image based on a known point spread function,
An image input unit to which the degraded image is input;
An original image estimation unit for obtaining an estimated original image for each color channel by estimating an original image before deterioration of the deteriorated image;
A restored image synthesis unit that generates a restored image by synthesizing the estimated original image for each color channel;
The original image estimation unit evaluates the likelihood of the estimated original image based on an evaluation index including a predetermined parameter for one representative color channel selected from the plurality of color channels, while other than the representative color channel For the auxiliary color channel, using the value of the parameter when the estimated original image for the representative color channel is evaluated to be likely, the likelihood of the estimated original image is evaluated based on the evaluation index. A featured image restoration device. The evaluation index includes a difference between a pixel value of the input deteriorated image and a pixel value of the deteriorated image estimated based on the estimated original image and the point spread function, and a sparse gradient of a pixel value in the estimated original image A function for evaluating sex,
The parameter is a weight parameter that determines a relative priority with the difference for each pixel in the evaluation of the estimated original image by weighting a function for evaluating the sparsity,
The original image estimation unit performs an iterative process of sequentially updating the estimated original image until the estimated original image is evaluated as being likely for the representative color channel, and finally the updated estimated original image The weight parameter is updated based on the updated weight parameter, and a series of processes for newly starting the iterative process is repeated a predetermined number of times based on the updated weight parameter, and the auxiliary color channel is finally updated for the representative color channel. The image restoration apparatus according to claim 1, wherein an iterative process for sequentially updating the estimated original image is performed until the estimated original image is evaluated as being likely based on the weighted parameter.   The image restoration apparatus according to claim 1, wherein the representative color channel is a color channel having the largest number of high-frequency components among the plurality of color channels.   4. The degraded image according to claim 1, wherein the degraded image is an RGB color image, the representative color channel is a G channel, and the auxiliary color channel is an R channel and a B channel. The image restoration apparatus described. The evaluation index includes a difference between a pixel value of the input deteriorated image and a pixel value of the deteriorated image estimated based on the estimated original image and the point spread function, and a sparse gradient of a pixel value in the estimated original image A function for evaluating sex,
The parameter is a weighting parameter that determines a relative priority with the difference for each pixel in the evaluation of the estimated original image by weighting a function for evaluating the sparsity.
The original image estimation unit performs an iterative process of sequentially updating the estimated original image until the estimated original image is evaluated as being likely for the representative color channel, and finally the updated estimated original image The weight parameter is updated based on the updated weight parameter, and a series of processes for newly starting the iterative process is repeated a predetermined number of times based on the updated weight parameter, and the auxiliary color channel is finally updated for the representative color channel. 2. The iterative process of sequentially updating the estimated original image is performed until the estimated original image is evaluated as being likely based on a weight parameter obtained from the estimated original image. Image restoration device. An image restoration method for restoring a deteriorated color image based on a known point spread function,
An image input step in which the deteriorated image is input;
An original image estimation step for obtaining an estimated original image for each color channel by estimating an original image before deterioration of the deteriorated image;
A restored image synthesis step of creating a restored image by synthesizing the estimated original image for each color channel;
In the original image estimation step, the difference between the pixel value of the input deteriorated image and the pixel value of the deteriorated image estimated based on the estimated original image and the point spread function, and the gradient of the pixel value in the estimated original image On the basis of a function for evaluating the sparsity of the estimated original image,
In the evaluation of the estimated original image, the function for evaluating the sparsity is weighted by a weight parameter that determines a relative priority with the difference for each pixel.
Furthermore, in the original image estimation step, for one representative color channel selected from a plurality of color channels, an iterative process for sequentially updating the estimated original image is performed until the estimated original image is evaluated as being likely. Then, the weight parameter is updated based on the finally updated estimated original image, and a series of processes for newly starting the iterative process based on the updated weight parameter is repeated a predetermined number of times, except for the representative color channel For the auxiliary color channel, iterative processing for sequentially updating the estimated original image is performed until the estimated original image is evaluated as being likely based on the weight parameter finally updated for the representative color channel. An image restoration method characterized by the above.

Images