JPH11272866A - Image restoring method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image restoring method which can restore a more accurate ideal image. SOLUTION: After initial values of an (n)-dimensional PSF model (h) are set in steps 102 to 107, an ideal image model F for a frequency space is corrected in steps 108 to 112 with an image G converted into the frequency space and the PSF model H, limitations of a real space are imposed on an ideal image model (f) converted into the real space, and a PSF model H for the frequency space is corrected in steps 113 to 117 with an image G converted into the frequency space and the ideal image model F and limited so that respective frequency component values of the PSF model H in the frequency space do not exceed corresponding component values of a predetermined distribution; and a process for imposing the limitation of the real space is repeated for the PSF model (h) converted into the real space until specific conditions are met.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image restoration method for improving the quality of an image acquired by an optical device such as a microscope or a confocal microscope.

[0002]

2. Description of the Related Art The image quality of an image obtained by observing an object with an optical device is deteriorated by an amount corresponding to the blur of the optical device as compared with the original object.
That is, the intensity distribution g of the image is represented by the convolution (conv) of the luminance distribution f of the original object and the point spread function (PSF) h of the optical device.
noise) was added to the noise.

[0003]

(Equation 1) And can be generally expressed.

[0004] Techniques for removing blur caused by an optical device from this image by numerical calculation and obtaining an ideal image include "restoration" or deconvolution of the image.
solution).

[0005] When a specimen is photographed using an optical microscope, an optical sectioning method is known as a method for obtaining a three-dimensional image (= laminated image) while displacing the specimen at equal intervals in the depth direction. Have been. In this case, since the PSF of the optical microscope is much wider in the depth direction than in the horizontal direction, each image in the stacked image does not become a cross-sectional image that accurately reflects the luminance distribution of the sample, but from above and below. Leaking blur is superimposed.

Therefore, conventionally, predetermined calculation processing has been performed in order to remove blur from the laminated image based on the PSF of the optical microscope and to "restore" an ideal three-dimensional image. The PSF used in the calculation here is a theoretical value or a measured value.

However, in the case of an optical microscope, the actual PSF
And the theoretical value do not always match, and it is also difficult to accurately measure the PSF. Therefore, an image restoration technique called “blind decovolution” has been conventionally studied. This restores both the ideal image and the PSF from one image, and has the advantage that the ideal image can be restored even if the PSF is unknown.

[0008] Blind deconvolution is generally performed by an algorithm called an alternative method. Here, in the alternating method, first, an ideal image luminance distribution model f (hereinafter, referred to as an ideal image model f) and a point spread function model h (hereinafter, a PSF model h)
), And then alternately correct the two models so that the convolution of the two models becomes the measured actual image intensity distribution g (hereinafter, referred to as image g). , In the process, while adding appropriate constraints to the model,
The two models are optimized by repeating such alternate correction several times as an optimization loop. Hereinafter, the model of the real space is described by lowercase letters (g, f, h), and the model of the frequency space is described by uppercase letters (G, F, H).

FIG. 4 shows a flowchart for explaining such an alternating method. In this case, step 401
In step 402, the actual image g measured by the optical device is read, and in step 402, the initial value of the ideal image model f is given to the generally measured actual image g. A Dirac delta function or a blurred geometric optical PSF is given.

Next, in step 403, the actual image g and the ideal image model f are converted from the real space to the frequency space. This is to efficiently calculate the convolution as a product in the frequency space. The transform is generally a fast Fourier transform (F
FT). Then, in step 404, a restriction condition and an end condition are given. That is, since the ideal image and the PSF are light intensity distributions, they do not take negative values. Therefore, it is set so that the non-negative restriction is applied to the two models f and h in the real space. This restriction is performed in steps 409 and 414 described later. Further, the spatial frequency of the PSF has an upper limit, that is, a band limit. However, since the value is unknown, the value is set by estimating from the spectrum of the image. FIGS. 5A and 5B
(C) is a one-dimensional illustration of the relationship between the object, PSF and image spectra for simplicity. Of these, the spectrum 201 of the object shown in FIG. 7A generally contains a higher frequency component than the spectrum 202 of the PSF of the optical apparatus shown in FIG. Therefore, the spectrum 204 of the measured image shown in FIG.
Band limit 20 of the same image as band limit 203 of
This is because it is considered to have five. In the case of a three-dimensional image, 3
Set the axis band limit. This band limit is executed in step 412 described later. In addition, the termination condition is generally the number of repetitions of the optimization loop, which is determined in step 415 described later.

In step 405, the PSF model h is converted from the real space to the frequency space, and in step 406, the point spread function model H in the frequency space (hereinafter referred to as the PSF model H).
The ideal image luminance distribution model F in frequency space (hereinafter, referred to as an ideal image model F) is modified so that the product of the two images becomes the actual image G. Here, the following equation has been proposed as an update equation for correcting the ideal image model F.

[0012]

(Equation 2)

At step 407, a restriction condition of the frequency space with respect to the ideal image model F is added. At step 408, the ideal image model F is transformed from the frequency space to the real space. This is for applying the non-negative restriction of the real space in step 409 to the ideal image model f. The conversion here is generally performed by a fast inverse Fourier transform (IFFT).

Next, at step 409, a non-negative limit is applied to the ideal image model f. That is, a negative element value is replaced with 0, and the other element values are left as they are. Further, there is an example in which correction is performed so that the total sum of the ideal image model f does not change.

Similarly, in steps 410 to 414,
Step 40 after replacing the ideal image model and the PSF model
The same processing as in steps 5 to 409 is performed to modify the PSF model. Note that the restriction condition of the frequency space for the PSF model H in step 412 is a band limit. That is, a component having a frequency higher than the band limit of each axis is set to 0.
And leave the other components as they are. In addition, as the restriction condition for the PSF model h in the real space in step 414, there is an example in which the sum of element values is normalized to 1 in addition to the non-negative restriction. This is to satisfy the energy conservation law in image formation.

If it is determined in step 415 that the number of repetitions of steps 405 to 414, which are the optimization loop, has reached the number set in step 404, the process proceeds to step 416, and the process ends. This step 4
In step 16, steps 405 to 414 which are optimization loops
Is output as an ideal image obtained by removing blur from the actual image g.

FIG. 6A shows the band limit added in step 412 to the PSF model H.
become that way. ξ, η, ζ are three axes x, y, z in the real space.
Are coordinate axes in the frequency space corresponding to. Also,

[0018]

Embedded image Is the band limit of each axis determined from the spectrum of the image. The band limit of the three-dimensional frequency space is

[0019]

Embedded image 6 is a window function of 1 inside the rectangular parallelepiped 301 or the ellipsoid 302 and 0 outside the rectangular solid 301 or the ellipsoid 302 (FIG. 6A shows only ξ, η, ζ> 0, which is １ ／ of the whole). The rest is also symmetric.) This is PS
By multiplying the F model H, a band limit is added.

There is an example in which the band limit for the PSF model H is determined from the outer shape of the PSF theoretical value based on the diffraction theory. This will be described with reference to FIG. Here shows a typical example applied to the image restoration of wide-field fluorescence microscopy, half B _Z of the radius B _r and height of the cylinder 311, from the value of each common microscopic lateral resolution and depth of focus,

[0021]

(Equation 3) It is decided as follows. The two cones 312 having the apexes in contact with each other are so-called “missing cones”, which correspond to the fact that the wide-field microscope has no resolving power in the z direction (depth direction). The band limit function is defined as 1 inside the cylinder 311 excluding the inside of the two cones 312, and as 0 otherwise (ie, inside the two cones 312 and outside the cylinder 311). By multiplying this by the PSF model H, a band limit is applied.

There is also an example in which an outer shape is restricted on the PSF model h in the real space, which will be described with reference to FIG. This is also an example applied to image restoration of a normal wide-field fluorescence microscope, and two cones 323 lacking a vertex are used.
To include the main part of the PSF of the wide-field microscope

[0023]

(Equation 4) It has been decided. d is set equal to the height (depth) of the image to be restored. By defining a window function of 1 inside the two cones 323 and 0 outside the two cones 323 and multiplying by the PSF model h, the real space is restricted (step 4 shown in FIG. 4).
14) is being executed.

[0024]

Incidentally, blind deconvolution is not appropriate (ill-posed).
This is an inverse problem, and the uniqueness of the solution is not guaranteed. That is, there are countless combinations of the ideal image and the PSF that form one actual image. Applying "pressure" to converge the two models to the optimal combination from among them is the role of the constraint on the models.

The ideal image model of the two models depends on the measured actual image and is naturally not constant. On the other hand, the PSF model can be expected to be substantially unchanged if the conditions of the optical devices used for the measurement are the same. Therefore, in blind deconvolution, it is important to set what kind of limiting conditions are set for the PSF model. If the PSF model is optimized for a distribution close to the actual PSF,
This is because an ideal image model that has been optimized so that convolution with it becomes an actual image should also be an accurate ideal image. The aforementioned constraints on the PSF model are to achieve this goal.

The conventional limiting conditions for the PSF model in the frequency space all limit only the frequency, and the component values are not manipulated. In other words, each of the limiting conditions is to multiply a window function of 1 in a certain area and 0 in other areas. Therefore, only the frequency outside the region is cut, and the value of the frequency component inside the region is stored as it is. The fact that such a conventional method does not have sufficient "pressure" to optimize the two models will be described with reference to FIGS. In addition, these drawings
For simplicity, they are all shown in one dimension.

The spectrum 204 of the image shown in FIG. 5C measured by the optical instrument is the product of the spectrum 201 of the object shown in FIG. 5A and the spectrum 202 of the PSF of the optical instrument shown in FIG. It has become. FIG. 7 shows two examples of combinations of an ideal image and a PSF that are likely to be restored by blind deconvolution from the spectrum 204 of this image. In this case, FIG. 5A shows the spectrum 204 of the image shown in FIG. 5C by reference numeral 401, and FIG.
And (c) show the actual PSF of the optical device shown in FIG.
The spectrum 402 close to the spectrum 202 of FIG.
An ideal image spectrum 40 corresponding to a portion smaller than the hand limit in the object spectrum 201 shown in FIG.
4 is restored. FIGS. 7B and 7C show a spectrum 405 of a PSF in which frequency components below the band limit are constant, and a spectrum 401 of an image.
The spectrum 407 of the ideal image which has not changed from the original image is restored. And PS in both combinations
The product of F and the ideal image becomes an image spectrum 401.

However, the conventional restriction condition that restricts only the frequency allows the PSF spectrum 405 as in the latter case. These are extreme examples, and there are countless combinations that can be restored. Among them, there is no guarantee that the combination as shown in FIGS. 7B and 7C will be optimized under the restriction conditions other than the band limit (such as non-negative restriction in the real space).

The above-described method in which the outer shape of the PSF model h in the real space is limited is effective in this respect. If this restriction condition is converted into a frequency space and considered, it is understood that the pressure becomes a pressure that brings the PSF model h closer to the actual distribution than when only the band limit in the frequency space is used. However, the PSF in the real space does not have a clear boundary like the band limit in the frequency space, and strictly extends to infinity. In addition, the PSF of an optical device is generally deteriorated from a theoretical value due to an aberration of an optical system or an influence of a measurement environment. In the case of real space, it corresponds to an increase in the spread of the PSF.

Therefore, it is difficult to appropriately limit the PSF model to the real space. Restricting the real-space PSF model to an area smaller than the real space requires a PSF in the frequency space.
This is equivalent to assuming the model h to have a wider bandwidth than it actually is.
That is, the two models H to be restored are shown in FIG.
This is because the pressure can be closer to those shown in FIGS. 7B and 7C than in FIG. (In FIG. 6 (c) and (Equation 7), the reason why the restriction area of the PSF is enlarged is considered to be related to the situation in this area.) In addition, there is a problem relating to the execution of numerical calculation. When certain frequency components of the modified ideal image model F are extremely smaller than the corresponding frequency components of the actual image, the corresponding frequency components of the PSF model H modified based on them have extremely large singular values. Can be. The restriction by the conventional window function leaves such a singular value. However, when dealing with large data such as a three-dimensional image by numerical calculation,
Such a singular value causes a calculation error due to a digit loss of a computer or the like. Specifically, it appears as distortion and noise of the ideal image model f and the PSF model h.

The two models restored by blind deconvolution are also affected by the initial values. In such a nonlinear optimization problem, a solution (model) may not converge or converge to an erroneous solution unless the initial value is appropriate. Above all, if starting from an inappropriate initial value, wasteful time is required until convergence. The initial value of the conventional PSF model h is not always optimal in this respect. The present invention has been made in view of the above circumstances, and has as its object to provide an image restoration method that can restore a more accurate ideal image.

[0032]

According to the first aspect of the present invention,
In an image restoration method for obtaining an n-dimensional ideal image f in which blurring due to a point spread function has been removed from an n-dimensional image g obtained as a convolution of an object and a point spread function (PSF), After setting the initial value of the PSF model h,
By the image G converted into the frequency space and the PSF model H,
The ideal image model F in the frequency space is modified, the real image model f converted into the real space is restricted, and the real image is restricted by the image G and the ideal image model F converted into the frequency space. The PSF model H is modified and the PS in the frequency space is
A process of restricting each frequency component value of the F model H so as not to exceed a corresponding component value of a predetermined distribution and imposing a real space restriction on the PSF model h converted into the real space is performed. The process is repeatedly executed until a predetermined end condition is satisfied, and a final ideal image model f is generated as an ideal image.

According to a second aspect of the present invention, in the first aspect, the predetermined distribution is a PSF theoretical value in a frequency space of a device that has acquired the image g or a distribution similar thereto.

According to a third aspect of the present invention, an n-dimensional ideal image f obtained by removing a blur caused by a point spread function from an n-dimensional image g obtained as a convolution of an object and a point spread function (PSF) is obtained. In an image restoration method for obtaining an n-dimensional PS
After setting the initial value of the F model h, the ideal image model F in the frequency space is corrected by the image G converted into the frequency space and the PSF model H, and each frequency component value of the ideal image model F in the frequency space is An image G and an ideal image converted to a frequency space are restricted by restricting the component values so as not to be smaller than a corresponding component value of a predetermined distribution, and restricting the real space to the ideal image model f converted to the real space. The PSF model H in the frequency space is modified by the model F, the PSF model H in the frequency space is restricted so as not to have a component exceeding a predetermined frequency, and the PSF model h converted into the real space is used. A process for adding a restriction on the real space is repeatedly executed until a predetermined end condition is satisfied, and a final ideal image model f is generated as an ideal image.

As a result, in the present invention, blind deconvolution is performed under the following restriction conditions and initial values of the model. First, a new measure is taken to limit the PSF model H in the frequency space so as not to exceed a predetermined distribution. This distribution is a PSF theoretical value or a distribution according to it. This PSF theoretical value is obtained by separate calculation or measurement depending on the optical instrument used for measuring the object and the measurement conditions. Alternatively, means for applying an equivalent restriction condition to the ideal image model F in the frequency space is newly provided. Second, as the initial value of the PSF model h, the above-described theoretical value of the PSF or a distribution similar thereto is used.

The limitation of the present invention on the PSF model H in the frequency space acts so that the PSF model h and the ideal image model f are optimized to more accurate distribution. The first reason is that the distribution of the PSF generally has a clear upper limit called a band limit in the frequency space, so that unlike the real space where the distribution of the PSF extends infinitely,
Precise restriction conditions can be set.

Second, the actual PSF in the frequency space
Generally does not exceed the component value of the corresponding theoretical value. The PSF that has deteriorated due to the aberration of the optical system or the measurement environment appears in a form in which a certain frequency component is lower than a theoretical value in a frequency space. Therefore, P
The restriction not to exceed the distribution of the theoretical SF value acts to allow only the degraded PSF. This works so that the two models converge on the degraded PSF and the corresponding ideal image, and does not hinder the "pressure" due to other limiting conditions. On the other hand, it is impossible to realize the same restriction in the real space. This is because the degradation of the PSF may increase or decrease the intensity at any point in space.

Third, the PSF model is stricter and more appropriate than a conventional window function based on band limits. The limitation of the present invention is that the distribution of the PSF model is
Apply "pressure" to approach SF. As a result, the accuracy of the restored ideal image is improved, and FIG.
The risk of converging to an abnormal model such as (b ') and (c') is reduced.

Further, the limitation of the present invention on the PSF model or the ideal image model in the frequency space operates to remove a singular value which causes a calculation error. On the other hand, the initial value of the PSF model according to the present invention acts to speed up the convergence of the model. This is because the distribution is closer to the actual PSF than the conventional initial value and includes the actual PSF. Also, along with the limiting conditions of the present invention,
This has the effect of reducing the risk of converging on an abnormal model.

[0040]

Embodiments of the present invention will be described below with reference to the drawings. (First Embodiment) The first embodiment shows an example in which accurate PSF theoretical values are used and applied to an image of an epifluorescence confocal laser scanning microscope (EFCLSM).

FIG. 1 shows a flow of image processing according to the present invention, which was measured by an optical section method using EFCLSM.
This is a program that reads a two-dimensional image (laminated image) file, performs image processing on it, and displays an ideal image from which blur has been removed, or outputs it as an image file. However,
Since the contents other than the image processing are general contents, the description is omitted.

In this case, first, in step 101, a three-dimensional image file is read and assigned to the real part of the three-dimensional complex data array of the actual image g. Here, 0 is substituted for the imaginary part. Also, since the fast Fourier transform (FFT) is performed later, if the number of data points on each of the x, y, and z axes is not a power of 2, the missing data points are padded with zeros and the number of points is raised to a power of 2. Keep it. The actual image g also becomes the initial value of the ideal image model f, but the substitution is performed in step 103 described later. In addition, the sum Σg of all the element values of the actual image g is obtained for use in step 112 later.

Next, at step 102, the PSF model h
Is prepared as a three-dimensional complex data array having the same size as the actual image g, and the EFCL having no aberration is used as an initial value.
The PSF theoretical value of SM / h (where / h indicates a vector h; the same applies hereinafter) is calculated by the following equation, and is substituted into the real part of the PSF model h, and 0 is substituted into the imaginary part. .

[0044]

(Equation 5) Here, a is a transmission function of the light receiving pinhole of the EFCLSM, and is represented by the following equation.

[0045]

(Equation 6)

[0046]

(Equation 7) Further, normalization as in the following equation is performed so that the sum of all element values of the PSF model h becomes 1.

[0047]

(Equation 8)

Next, in step 103, the real image g in the real space is converted into the real image G in the frequency space by FFT, and subsequently, in step 104, the ideal image model F is replaced with the real image G. A three-dimensional complex number data array having the same size is prepared, and the value of the actual image G in the frequency space is substituted as its initial value.

Then, in step 105, a limiting condition for the PSF model ratio in the frequency space is given. in this case,
Since the theoretical value / h of the PSF has been substituted into the PSF model h in the real space in step 101, this is converted into the PSF model H in the frequency space by FFT, and it is converted into the data array / H (here, / H indicates a vector H. The same applies to the following.) However, since the PSF theoretical value / H in the frequency space has a band limit, components of frequencies higher than that are ignored, and the data array / H is composed only of components within the band limit. Further, since the theoretical PSF theoretical value / h with no aberration is symmetric in each axis direction with respect to the origin, it is expressed as F
H converted by FT has only a real part value. Therefore, the data array H is a real number array including elements within the band limit.

In step 106, the ideal image model f and PS
The non-negative restriction of the real space is set to be applied to the F model h. This restriction is applied to steps 112 and 117 described later.
Executed in Further, in step 107, the number of repetitions N of the optimization loop is set as an end condition.

Next, at step 108, the PSF of the real space
The model h is transformed into a PSF model H in the frequency space by FFT. However, the first time is not necessary because it has already been converted. Then, in step 109, each component of the ideal image model F is corrected by an update expression based on the following expression so that the product of the corrected PSF model H and the actual image G becomes the actual image G.

[0052]

(Equation 9)

In step 110, a restriction condition of the frequency space with respect to the ideal image model F is added, but this is not applied in the first embodiment. In step 111, the ideal image model F in the frequency space is converted into the ideal image model f in the real space by IFFT.

Then, in step 112, a non-negative restriction is imposed on the ideal image model f in the real space. This is performed in the following procedure. First, element values having a negative real part of the ideal image model f are replaced with 0, and the other element values are replaced with the absolute value of a complex number and the imaginary part is set to 0. Next, normalization such as the following equation is performed using Σg obtained in step 101 so that the total sum of the ideal image model f does not change.

[0055]

(Equation 10)

In step 113, the ideal image model f in the real space is converted into the ideal image model F in the frequency space by FFT.
At step 114, the PSF model H in the frequency space is set so that the product of the corrected ideal image model F and the actual image G is obtained.
Are corrected by an update equation based on the following equation.

[0057]

[Equation 11]

At step 115, the limiting condition set at step 105 is added to the PSF model H in the frequency space. This is performed in the following procedure. First, the absolute value of each component of the PSF model H is compared with the corresponding component of the limiting condition H. And only when the former is larger,

[0059]

(Equation 12) , The components of the PSF model H in the frequency space are replaced. When there is no corresponding component of the limiting condition H, that is, the component of the PSF model H having a frequency higher than the band limit of each axis is replaced with 0.

In step 116, the PSF model H in the frequency space is converted into a PSF model h in the real space by IFFT. Then, in step 117, a non-negative restriction is imposed on the PSF model h in the real space. This is performed in the following procedure. First, the element value of the PSF model h having a negative real part is set to 0.
For the other element values, the real part is replaced with the absolute value of the complex number, and the imaginary part is set to 0. Next, (Expression 11) is normalized so that the sum of all element values of the PSF model h becomes 1.

Thereafter, in step 118, the number of repetitions of steps 108 to 117, which is the optimization loop, is 107
When it is determined that the number of times N set in has been reached, the process proceeds to step 119, and the process ends. In step 119, the ideal image model f repeatedly corrected in steps 108 to 117, which is an optimization loop, is output as an ideal image obtained by removing blur from the actual image g, and the image processing for restoration is completed.

Accordingly, in this way, an effective method is obtained for blind deconvolution by all the alternating methods, irrespective of the updating formula, other restriction conditions, termination conditions, and the like.

In this embodiment, step 10
(Equation 2) is used as the update equation in 9 and 114, but it may be replaced with (Equation 3) to (Equation 5). In addition,
When an update equation such as (Equation 5) is used, it is not necessary to set an initial value of the ideal image model. (Second Embodiment) This second embodiment uses a theoretical value covering the actual PSF, and uses the P value of the first embodiment.
The limiting conditions for the initial value of the SF model h and the PSF model H in the frequency space are changed as follows.

An epi-fluorescence confocal laser scanning microscope (EFC)
The resolving power of LSM) becomes maximum when the diameter of the light receiving pinhole is made infinitely small. Conversely, P in the frequency space
When the diameter of the light receiving pinhole is increased from infinity to small, the frequency component value of each SF becomes equal to or less than the corresponding frequency component value when the diameter of the light receiving pinhole is reduced to infinity. Therefore,
Also when restoring a three-dimensional image measured with an arbitrary light receiving pinhole diameter, the theoretical value of the PSF when the diameter of the light receiving pinhole is made infinitely small can be used as a limiting condition for the PSF model H in the frequency space. That is, instead of (Equation 8) described in the first embodiment,

[0065]

(Equation 13) May be used. Further, this value can be used as an initial value of the PSF model h.

In this way, the accuracy of the ideal image restored from the first embodiment described above decreases, but the PSF
The effect of simplifying the preparation of the initial value of the model h and the limiting condition for the PSF model H in the frequency space can be expected. (Third Embodiment) In the third embodiment, an approximate value including the actual PSF is used, and the P value of the first embodiment is used.
The limiting conditions for the initial value of the SF model h and the PSF model H in the frequency space are changed as follows.

In this case, if it is assumed that a rough distribution of the PSF theoretical value is known in advance, a distribution including the rough distribution can be used. That is, this is shown in FIG.
Explaining with reference to (b), since the distribution 408 including the PSF theoretical value is a distribution in which each frequency component has a value equal to or higher than the PSF theoretical value 402, such a distribution is expressed by a simple approximate expression. It may be expressed and used instead of (Equation 8).

In this way, the preparation of the initial values of the PSF model h and the limiting conditions for the PSF model H in the frequency space is further simplified. Although the accuracy of the restored ideal image is lower than that of the first embodiment, the accuracy of the ideal image can be made higher than that of the conventional restoration using only the band limit. (Fourth Embodiment) This fourth embodiment uses a PSF
Are used to modify the initial values of the PSF model h and the limiting conditions for the PSF model H in the frequency space as follows.

In this case, in an optical device in which it is difficult to calculate the theoretical value of the PSF, or in an optical device in which the actual value of the PSF is largely different from the theoretical value, the theoretical value of the first embodiment is replaced with the theoretical value as shown in (Equation 8). The measured value of PSF is used for the measurement. For example, a method is known in which the PSF of a fluorescence microscope is measured by treating minute fluorescent beads as point light sources. However, the measured PSF
It is necessary to pay attention to the fact that the actual PSF may be deteriorated due to a measurement error, and it is necessary to appropriately modify the actual PSF to include the actual PSF as described in the third embodiment. . (Fifth Embodiment) The fifth embodiment uses a PSF.
In the first to fourth embodiments, the limiting condition for the PSF model H in the frequency space improves the accuracy of the restored ideal image and reduces the calculation error. It has the effect of removing the singular value that causes it. Among them, an example in which at least the latter effect is obtained will be described with reference to FIG. 1 and FIG. 2 described above.

In this case, step 105 'shown in FIG. 2A is replaced with step 105' shown in FIG. 1 and step 11 shown in FIG.
5 'and 115 "are used.

Then, in step 105 ', a band limit is set as a restriction condition for the PSF model H in the frequency space, similarly to the prior art. Then, in the optimization loop shown in FIG. 1, in step 115 ', a band limit is added to the PSF model H in the frequency space, and in step 115 ", the absolute value of each component is limited to 1 or less. , Only those components whose absolute value is greater than 1

[0072]

[Equation 14] Replace as

In this way, since each frequency component of the PSF of the optical device is generally 1 or less, the limiting condition in the fifth embodiment is to exclude a PSF model that is obviously wrong, and Does not unduly limit the PSF model. Then, only extremely large singular values can be removed. (Sixth Embodiment) The sixth embodiment is an example in which the ideal image model in the frequency space is restricted. The restriction condition on the PSF model H in the frequency space according to the fifth embodiment is substantially equivalent to that. Since this can be realized also by the restriction condition for the ideal image model F in the frequency space, this will be described with reference to FIGS.

In this case, step 105 'shown in FIG. 2A is replaced by step 105' shown in FIG. 2A, and step 115 'shown in FIG.
(Here, except for step 115 ″), and in place of step 110 described in FIG. 1, FIG.
Step 110 'shown in FIG.

Then, also in this case, step 105 '
Then, a band limit is set as a restriction condition for the PSF model H in the frequency space in the same manner as in the related art, and then, in the optimization loop shown in FIG.
At 0 ', the absolute value of each component of the ideal image model F in the frequency space is limited so as not to be smaller than the absolute value of the corresponding component of the actual image G in the frequency space. That is, only the component whose absolute value is smaller than the actual image G

[0076]

(Equation 15) Replace as

Then, in step 115 ', a band limit is added to the PSF model H in the frequency space. In this way, each frequency component of the image is generally smaller than the corresponding component of the original object,
The restriction condition in the embodiment does not unduly restrict the ideal image model. Then, as in the fifth embodiment, the next P
It is possible to prevent occurrence of an extremely large singular value in the modification of the SF model H (step 114).

In each of the above embodiments, the fluorescence confocal microscope or the fluorescence microscope has been described as an example. However, it is needless to say that the present invention can be applied to all images formed by convolution as shown in (Equation 1). The image to be restored is not limited to the three-dimensional image as described above, but may be in other dimensions. For example, in order to restore a two-dimensional image, the two-dimensional ideal image model and the PSF model may be optimized for the two-dimensional actual image g by the same method as described above. Although the PSF theoretical value or a distribution similar thereto is used as the initial value of the PSF model h, the initial value may be given by a conventional impulse or the like. Even in such a case, the accuracy of the restored ideal image is improved as compared with the related art due to the limitation of the present invention on the PSF model in the frequency space.

The present invention includes the following inventions. (1) In the image restoring method according to claim 1, the predetermined distribution is such that a component equal to or lower than a predetermined frequency is one.
Where the other components have a distribution of zero. (2) In the image restoration method according to the first aspect, the theoretical value of the PSF of the device that has acquired the image g or a distribution similar thereto is set as the initial value of the PSF model h. (3) In the image restoration method according to the first aspect, the restriction on the real space for the ideal image model f and the PSF model h is a non-negative restriction. (4) In the image restoration method according to the first aspect, the termination condition is that a predetermined number of repetitive processes have been performed. (5) In the image restoration method according to the first aspect, the device for acquiring the image g is a confocal microscope. (6) In the image restoration method according to the first aspect, the apparatus for acquiring the image g is a microscope. (7) In the image restoration method according to claim 3, the predetermined distribution is an image G in a frequency space. (8) In the image restoration method according to claim 3, as the initial value of the PSF model h, a theoretical value of the PSF of the device that has acquired the image g or a distribution similar thereto is set. (9) In the image restoration method according to the third aspect, the restriction on the real space for the ideal image model f and the PSF model h is a non-negative restriction. (10) In the image restoring method according to the third aspect, the termination condition is that a predetermined number of repetitive processes have been performed. (11) In the image restoration method according to the third aspect, the device for acquiring the image g is a confocal microscope. (12) In the image restoration method according to the third aspect, the apparatus for acquiring the image g is a microscope.

[0080]

As described above, according to the present invention, the accuracy of an ideal image restored by blind deconvolution can be improved, and the calculation time required for convergence to the ideal image can be reduced. it can.

[Brief description of the drawings]

FIG. 1 is a flowchart for explaining first to sixth embodiments of the present invention.

FIG. 2 is a flowchart used for describing fifth and sixth embodiments of the present invention.

FIG. 3 is a flowchart used to explain a sixth embodiment of the present invention.

FIG. 4 is a flowchart for explaining a conventional image restoration method.

FIG. 5 is a view for explaining how to determine a band limit in the image restoration method.

FIG. 6 is a view for explaining a method of restricting a PSF model.

FIG. 7 is a view for explaining a difference between the conventional technique and the present invention.

[Explanation of symbols]

 201: Object spectrum, 202: PSF spectrum, 203: PSF band limit, 204: Image spectrum, 205: Image band limit, 401: Image spectrum, 402, 405: PSF spectrum, 403, 406 ... PSF band limit, 404, 407: spectrum of ideal image, 408: distribution including PSF theoretical value.

Claims (3)

[Claims] 1. An image restoration method for obtaining, from an n-dimensional image g acquired as a convolution of an object and a point spread function (PSF), an n-dimensional ideal image f in which blurring due to the point spread function has been removed. In the above, after setting the initial value of the n-dimensional PSF model h, by using the image G and the PSF model H converted to the frequency space,
The ideal image model F in the frequency space is corrected, the real image is converted into the real space, and the real space is restricted. By the image G and the ideal image model F converted into the frequency space,
The PSF model H in the frequency space is modified so that each frequency component value of the PSF model H in the frequency space is restricted so as not to exceed a corresponding component value of a predetermined distribution, and the PSF model h converted to the real space An image restoration method characterized by repeatedly executing a process of imposing a restriction on a real space on a target image until a predetermined end condition is satisfied, and generating a final ideal image model f as an ideal image. 2. The image restoration method according to claim 1, wherein the predetermined distribution is a theoretical PSF value in a frequency space of a device that has acquired the image g or a distribution similar thereto. 3. An image restoration method for obtaining an n-dimensional ideal image f in which blurring due to a point spread function has been removed from an n-dimensional image g obtained as a convolution of an object and a point spread function (PSF). In the method, after setting an initial value of an n-dimensional PSF model h, by using the image G and the PSF model H converted to the frequency space,
The ideal image model F in the frequency space is modified, and each frequency component value of the ideal image model F in the frequency space is limited so as not to be smaller than a corresponding component value of a predetermined distribution. The image model f is restricted in the real space, and the image G converted to the frequency space and the ideal image model F
The PSF model H in the frequency space is corrected. The PSF model H in the frequency space is restricted so as not to have a component exceeding a predetermined frequency. An image restoration method characterized by repeatedly executing a process for adding a restriction until a predetermined end condition is satisfied, and generating a final ideal image model f as an ideal image.