CA1319415C - Iterative image restoration device - Google Patents
Iterative image restoration deviceInfo
- Publication number
- CA1319415C CA1319415C CA000592676A CA592676A CA1319415C CA 1319415 C CA1319415 C CA 1319415C CA 000592676 A CA000592676 A CA 000592676A CA 592676 A CA592676 A CA 592676A CA 1319415 C CA1319415 C CA 1319415C
- Authority
- CA
- Canada
- Prior art keywords
- image
- viewed
- factor
- point
- subject
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
ITERATIVE IMAGE RESTORATION DEVICE
Abstract of the Disclosure An image restoration device restores images (7) which are viewed through an optical member (9) and stored in a data processor (13). Processing means (15) associated with the data processor (13) iteratively determine, for each point in a viewed image, a factor which minimizes noise and distortion at that point. The factor is iteratively determined through a division operation of a transform of a first function of a response function of the optical member (9) and a transform of a second function of the response function. Preferably, the transform is a Fourier transform and the response function is the point spread function of the optical member (9).
The processing means displays the restored image on a suitable display unit (17) (e.g. a CRT).
Abstract of the Disclosure An image restoration device restores images (7) which are viewed through an optical member (9) and stored in a data processor (13). Processing means (15) associated with the data processor (13) iteratively determine, for each point in a viewed image, a factor which minimizes noise and distortion at that point. The factor is iteratively determined through a division operation of a transform of a first function of a response function of the optical member (9) and a transform of a second function of the response function. Preferably, the transform is a Fourier transform and the response function is the point spread function of the optical member (9).
The processing means displays the restored image on a suitable display unit (17) (e.g. a CRT).
Description
11 3 ~ g ~ 1 r3 ITER~TIVE IMAGE RESTORATION DEVICE
Description Background of the Invention Various observation enhancing devices exist for viewing a subject. In general, the device transforms the subject and provides a distorted image which is viewed. More accurately, the transformed subject is further distorted by noise in electronics involved, optical noise from surrounding light, and/or quantum noise of random photons.
Thus, the viewed image is a trans~ormed, imperfect version of the subject as illustrated in Figure 1.
The subject 7 is mathematically represented as a function f. The observation enhancing device 9 transforms or distorts the subject 7 by a function k. The generated image 11 treferenced as g) is viewed as an array of measured data points gi which includes a noise or measurement error factor ~ .
The distorted image is then stated mathematically as gi ~ki(x)f(x)dx + ~ i That is, each point or pixel i in the viewed image g involves a sum of the light contributed from other neighboring points x on the subject 7 and an 3~
~ 3 ~
unknown noise or measurement error ~ i. The sum is over the set D of viewed points image.
In accordance with this contribution, a 3-D
subject introduces a further complication. In addition to light contributed from neighboring points on the same plane, a 2-D slice or a view of a plane through the 3-D subject is blurred by light contributed to the viewed plane from neighboring planes. Hence, the distortion becomes a more complex problem.
Various devices and schemes have been developed to define and subsequently reverse such distortion to provide what is called a restored image of the subject. Some of the schemes involve a matrix of measured points which define the distortion and noise or measurement error. The matrix is inverted to provide a restored image. These schemes, how-ever, are typically cumbersome and unsuitable for images with hundreds of data points or more.
Summary of the Invention An object of the present invention is to provide a method and device for correcting optical distortion and for providing a restored image, especially one from a viewed image of hundreds of data points.
A further object of the present invention is to provide such a method and device which involves a division operation at each pixel instead of an inversion of a matrix.
~ 3 In particular, the device of the present invention includes an optical viewing member through which the subject is viewed, a data processor for receiving and storing a series of viewed images of the subject as viewed through the optical viewing member, processing means associated with the data processor for restoring the viewed images, and display means for providing a view of the restored images. The processing means restores the viewed images by iteratively determining, for each point in a viewed image, a factor which minimizes noise and distortion at that point. The factor is iteratively determined through a division operation, in a trans-form domain, of functions of the response function of the optical viewing member. In a preferred embodiment, the factor is iteratively determined through a division operation in a Fourier transform domain of the response function.
More specifically, the factor is iteratively determined by the relationship C 1 = C -where c is the factor;
n is the numbar of iterations of c; and = inverse Fourier transform of / (H H + ~ +~));
~ 3 ~ 3 where ~ ' is the Fourier transform of ~', ~' = (M + ~ I) c - g;
M is a matrix of viewed image points along orthogonal axes i and j, and defined as Mij = 5ki(x) kj(x) dx D(c) D(c) is the set of viewed image points in which ki(x) ci 0, and ki(x) is the response function of the optical viewing member at image pixel i over surrounding points x;
kj(x) is the response function at image pixel j ;
and ~ are constants (~ 0);
I is the Identity matrix;
g is an array of measured values of the points in the image;
H is the Fourier transform of h(x) which is another form of the response function k.(x); and *
H is the Fourier transform of h(x) in which h(x) = h(-~).
The h(x) form of the response function ki(x) may be the point spread function of the optical viewing member such that ~ 3 ~
ki(x) = h(yi-x) where i is a point of the viewed image; and Yi is the corresponding point on the subject.
In accordance with a particular embodiment s of the invention there is provided an image restoration device comprising:
an optical viewing member for providing a view of a subject, the optical viewing member responding to each point of the subject according to a response function to provide a viewed image o~ the subject;
a data processor for receiving from the optical viewing member and storing a series of viewed images of the subject as viewed through the optical viewing member;
computer processing means coupled to the data processor for responding to the viewed images . and restoring the viewed images to substantially noiseless, undistorted images, the computer processing means restoring each viewed image by iteratively determining for each point in a viewed image a factor which minimizes noise and distortion at that point, the factor being iteratively determined, according to a formulation of a 2s mathematical definition of the factor, by performance of transform calculations including a mathematical division operation in a transform domain of functions which include the response function of the optical viewing member to determine the factor, transforms of the response function being denominators in the division operation, the division operation enabling avoidance of performance of a matrix inversion operation, and with the deter-mined factors for the points of a viewed image the 3s computer processing means minimizing noise and dis-- 5a - ~ 3 ~ V~
tortion at each point of the image such that a sub-stantially noiseless undistorted image is formed;
and display means coupl~d to the computer 5 processin~ means ~or providing a view of the sub-stantially noiseless, undistorted images in a non-transform domain.
In accordance with a further embodiment of the invention there is provided an image restoration 0 device comprising:
(i) an optical viewing member for providing a view of a subject, the optical viewing member responding to each point of the subject according to a response function to provide a viewed image of the subjecti (ii) a data processor for receiving from the optical viewing member and storing a series of viewed images of the subject as viewed through the optical viewing member;
(iii) computer processing means associated with the data processor for restoring the viewed images to substantially noiseless, undistorted images, the computer processing means restoring each viewed image by:
2s (a) iteratively determining for each point in a viewed image a factor which minimizes noise and distortion at that point, the factor being iteratively determined according to a formulation of a mathematical definition of the factor, by performance of transform calculations including a division operation in a transform domain of functions which include the response function of the optical viewing member to determine the factor;
.~
.~ ~ .~. .
- 5b ~
the factor being iteratively determined by the relationship Cn+l = Cn ~ "
where c is the factor;
n is the number of iterations of c; and A = inverse Fourier transform of (~ /(H H ~ a + ~));
where ~' is the Fourier transform of ~', ~' = (M ~ ~I) c - g;
M is a matrix of viewed image points i,j and defined as Mij = 5 ki(x) kj(x) dx D(c) D(c) is the set of viewed image points in which ~ki(x) Ci > 0, and ki(x) is the response function of the optical viewing member at image point i over surrounding points x;
~ and ~ are constants (20);
I is the Identity matrix;
g is an array of measured values of the points in the image;
H is the Fourier transform of h(x) which 2s is another form of the response function ki(x); and H* is the Fourier transform of h(x) in which ~(x) = h(-x); and (b) with the determined factors of a viewed image, minimizing noise and distortion at each point of the viewed image such that a substantially noiseless, undistorted image is formed; and (iv) display means coupled to the computer processing means for providing a view of the sub-stantially noiseless undistorted images in a non-transform domain.
,J."i~L
~` ,r - 5c -From a different aspect and in accordance with a particular embodiment of the invention there is provided a method of restoring an image of a subject, comprising the steps of:
s viewing the subject through an optical viewing system having a response to each point of the subject according to a response function, during the viewing the optical viewing system providing viewed images of the subject;
receiving in a data processor a series of viewed images of the subject as viewed through the optical viewing member;
restoring each viewecl image in the data processor by:
(a) iteratively determining, for each point in said viewed image, a factor which minimizes noise and optical distortion, the factor being iteratively determined according to a formulation of a mathematical definition of the factor which allows 20 performance of transform calculations including a division calculation with a transform of the response function of the optical viewing member as a denominator in the division calculation to determine the factor, such that performance of matrix 25 inversion is avoidable., (b) with the determined factors of the viewed image, minimizing noise and distoxtion at each point in said viewed image by manipulating said points, and (c) forming a substantially noiseless, undistorted image from the manipulated points of the viewed image; and displaying on display means coupled to the data processor each substantially noiseless and 35 undistorted image for each viewed image restored by the data process.
`J
, .
_ 5~ _ ~ 3~ 3 In accordance with a further particular embodiment of the second aspect, there is provided a method of restoring an image of a subject, comprising the steps of:
s viewing the subject through an optical viewing system having a response to each point of the subject according to a response function, during the viewing the optical viewing system providing viewed images of the subject;
receiving in a data processor a series of viewed images of the subject as viewed through the optical viewing member;
restoring each viewed image in the data processor by:
(a) iteratively determlning, for each point in a viewed image, a factor which minimizes noise and optical distortion, the factor being iteratively determined according to a formulation of a mathematical definition of the factor which allows performance of transform calculations including division with a transform of the response function of the optical viewing member to determine the factor, the factor being iteratively determined by 2s the relationship cn+l = Cn ~ Q
where c is the factox;
n is the number of iterations of c; and a = inverse Fourier transform of (~ /(H H + ~ + ~));
where ~' is the Fourier transform of ~', ~' = (M + ~I) c - gi M is a matrix of viewed image points i,j 3s and defined as Mij = ~ ki(X) kj(x) dx D(c) - 5e - ~3~
D(C) iS the set of viewed image points in which ~ki(x) ci > 0, and ki(x) is the response function of the optical viewing member at image point i over surrounding points x;
~ and ~ are constants (20);
I is the Identity matrix;
g is an array of measured values of the points in the image;
H is the Fourier transform of h(x) which is another form of the response function ki(x); and H* is the Fourier transform of h(x) in which h(x) = h(-x);
( b) with the determined factors of a viewed image, minimizing noise and distortion at each point in said viewed image by manipulating said points;
and (c) forming a substantially noiseless, 20 undistorted image from the manipulated points of the viewed image.
Brief Description of the Drawin~s The foregoing and other objects, features and advantages of the invention will be apparent 25 from the following more particular description of a preferred embodiment of the invention, as illus-trated in the accompanying drawings. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
Figure l is an illustration of the distortion and adaition of noise to a subject viewed through an optical system.
Figure 2 is a block diagram of an 3s embodiment of the present invention.
.~i~..~
_ 5f _ ~3~
Detailed Description of the Preferred Embodiment In general, the present invention provides a viewing system as shown in Figure 2. The user views a desired subject 7 through an optical system 5 9 such as a microscope, a telescope, a camera, a system of lenses and the like, or a combination thereof. The viewed image of the subject is captured by a data processor 13 which is connected to the optical system 9. The data processor 13 receives and stores in a first memory 19 a series of viewed images of .~
.~
~3~f~ j the subject 7 as the user views the subject through the optical system 9. Computer or other processing means 15 are associated with the data processor 13.
The processing means 15 restores the viewed images, which have been received by the data processor 13 and stored in first memory 19, by manipulating the measured data points of the received images in a second memory 5. A mathematical discussion of this manipulation is presented later. The restored images are thereafter displayed by suitable means, such as CRT 17 connected to the data processor 13.
A particular embodiment of the present inven-tion involves fluorescence microscopy, where cell structures are illuminated by fluorescent dyes and are viewed through an optical microscope to provide an image of the distribution of specific molecules and ions in the cell. 3-D information about the cell is obtained by optical sectioning. This requires focusing the microscope through a series of 128 planes of about 0.25 micrometers apart. Each of the 128 2-D images is acquired by a CCD camera mounted on the microscope and stored as a 128 x 128 image in a data processor. The stored images have been blurred by distortion and noise as previously stated. The distortion and blurring is removed by a deconvolution process as discussed in "3-D Molecular Distribution in Living Cells by Deconvolution of Optical Sectîons Using Light Microscopy", by Walter Carrington and Kevin Fogarty, reprinted in the Proceedings o~ the Thirteenth Northeast ~ 3 ~
Bioengineering Conference, Philadelphia, PA, March 12, 1987.
In sum, the deconvolution process described in the Carrington/Fogarty article solves S the sum of a least square fit of the measured data points and a regularization or smoothing term. In addition, the solution is constrained to non-negative values because the viewed subject is known to have a non-negative density. The solution is 10 also determined by an iterative division of Fourier transforms of functions of the point spread function, or more generally the response function of the optical system 9. In the iterative division, the viewed image is expressed as a convolution of 15 the response function and the function describing the subject 7.
The convolution is preferably performed by a high speed computer or an array processor con-nected to the data processor in a second memory 5 as shown in Figure 2.
By way of background, manipulation of stored images to provide a restored image has been accomplished in many ways. The simplest method is a direct inversion of the mathematical statement which s is illustrated in Figure l.
g = h*f + ~
where the inverse is given in the Fourier domain by ~ 3 ~ 3 F = (l/H) (G+ ~) This is not an accurate method because the inversion magnifies the noise in the data excessively.
Another method uses a conventional least square approach in which measurement errors are minimized.
The least square solution is expressed as a normalization of the difference of the transformed image (as known by the function k) and the viewed (i.e. measured~ image g.
!l~f - gl! 2 In addition, a linear smoothing operation enables high frequency oscillations in the direct solution to be damped to a lower amount of high frequency oscillations. This is accomplished by adding to the least s~lare solution a smoothing factor of f ( x)1 2dx Hence, a known equation used to describe the distorted image is ~ (f) = l~kf - gll 2 +~ f(x)~ 2dx Equation 1 This equation is solved for the minimum function f.
In the present invention a further constraint is imposed on the solution of Equation 1. This constraint is where f is greater than or equal to ~3~3~
zero, because the subject is known to be non-negative. A trivial calculus approach to solving Equation 1 for the minimum f, by setting the derivative of Equation 1 equal to zero, is not possible since f is a function and f(x) is constrained to be non-negative. The f(x) 70 that minimizes Equation 1 is known, however, to have the form f(x) = max(~ ki(x)ci, 0) at a point x in the subject. The ~pplicants then take this form of f and substitute it into Equation 1 so that ~ (f) of Equation 1 is a function of ci.
In this form, Equation 1 is then differentiated with respect to c. The derivative of Equation 1, (i.e. ~'(f)) is then translated into terms of c and set equal to zero. The c which satisfies that relationship defines the f that minimizes Equation 1. The problem can then be expressed in the form kf - g ~ ~c = 0 Equation 2 where c is a 3D vector of the ci's.
In prior art methods, Equation 2 was solved for f by Newton's method. In the present invention, an auxiliary function is defined by integrating Equation 2 such that the derivative of the auxiliary 2S function is zero to satisfy Equation 2 and then a ~ 3~9~
variation of Newton's method is applied to Equation 2 written in terms of the defined auxiliary function. Such an auxiliary function ~ (c) is stated as follows:
~;(c) = ~c (M +,~ I)c - c~g where c+ is the transpose of c.
Next, by Newton~s method c has the approximate form cn~l cn ~ Equation 3 where ~ c) 1 ~'(c) Equation 4 where ~" (c) = (M +~ I) c - g, is the gradient of (~! (C) and ~ " (c) = M +~ I, known as the Hessian matrix.
In the present invention a damped Newton's method is used to replace~'' with another matrix A, such that A = B ~ I Equation 5 where B is a matrix of i,j coordinates over all space defined as ~ 3 '~t 9 ~
Bi~ = JJJki(x)kj(x)dx which in terms of the point spread function h of the optical system may be written as Bij rh(yi-x) h(yj-x)dx or B = ~h(yi-xk)h(Yj xk) Equation 6 in Equation 3 is then replaced by ~ = (A+~ Equation 7 It is noted that Bij differs from Mij by having a domain over all points in space instead of over a lO set of viewed image points D(c).
Solving for ~ from~ = (A ~ is then as follows.
A = ( A+ ~
(A+~ , Equation 8 Equation 8 is then expressed as a convolution which is easier to multiply than the matrices of data of prior art schemes. A "convolution" as used here is defined to follow the statement that:
A summation, over points L, of the product of a function of variables m and L and a function of ~ 3~
L, equals the convolution of the two functions at a point m.
From Equations 5 and 8 (B ~ = y~
B~ + ~ +~ ' Equation 9 From Equation 6 a) i ~. ~, h (yi-xk) h (Yj -Xk) ~ j ~ h ( yi-Xk) h ( Xk Yj ) ~ j where h(x) = h(-x); then by the definition of convolution ( )i ~ h(yi-Xk) ((h*~)(Xk)) = (h * ~*~)) (Yi) = h* ~*~ (Yi) Equation 10 where * indicates a convolution.
Substituting Equation 10 into Equation 9 h * h * *~+ ~ ' Equation 11 ~ 3 ~
Equation 11 is then solved for ~by transforming each side; dividing the the transform of ~' by the sum of the transforms of h * ~,cX and/\; and calculating the inverse transform of this quotient.
In particular, a Fourier transform is employed, although other transforms are suitable, such that:
(h*h*~ ~ + (~ )~+ (~a) = (~ ~
h~ + ~ ~ +
(h~ ~A + ~+ ~
( y~
~ = = Equation 12 1 n ~\
(h~' h~+ ~+ ~) H H ~ +~
where ~ indicates Fourier transform;
' is the Fourier transform of ~'; and H , the Fourier transform of ~, is the complex conjugate of H, the Fourier transform of h.
Applying the inverse Fourier transform to Equation 12 provides = Inverse Fourier Transform of (~ ~(HH*~+~)) Equation 13 The value of ~ from Equation 13 is substituted into E~uation 3 to define a new c (i.e. cn+l) where a first cn (i.e. cO) i5 arbitrarily chosen. A
series of new c's is iteratively defined in a ~ 3 similar fashion using Equations 3 through 13 and the foregoing procedure. The iteration is stopped at a final value for c when ~ approaches 0;
~' gets small;
length of ~' approaches O;
length of g ~increases instead of continues to decrease;
or a combination thereof.
Once a final value for c is established, the f(x) ~ 0, that minimizes Equation 1 at a point i in the image g is defined from f(x~ = max ~ki(x)c, 0).
The newly defined f(x) is substituted into Equation 1, and Equation 1 is used to provide the sought restored image from point to point.
While the invention has been particularly shown and described with reference to embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Description Background of the Invention Various observation enhancing devices exist for viewing a subject. In general, the device transforms the subject and provides a distorted image which is viewed. More accurately, the transformed subject is further distorted by noise in electronics involved, optical noise from surrounding light, and/or quantum noise of random photons.
Thus, the viewed image is a trans~ormed, imperfect version of the subject as illustrated in Figure 1.
The subject 7 is mathematically represented as a function f. The observation enhancing device 9 transforms or distorts the subject 7 by a function k. The generated image 11 treferenced as g) is viewed as an array of measured data points gi which includes a noise or measurement error factor ~ .
The distorted image is then stated mathematically as gi ~ki(x)f(x)dx + ~ i That is, each point or pixel i in the viewed image g involves a sum of the light contributed from other neighboring points x on the subject 7 and an 3~
~ 3 ~
unknown noise or measurement error ~ i. The sum is over the set D of viewed points image.
In accordance with this contribution, a 3-D
subject introduces a further complication. In addition to light contributed from neighboring points on the same plane, a 2-D slice or a view of a plane through the 3-D subject is blurred by light contributed to the viewed plane from neighboring planes. Hence, the distortion becomes a more complex problem.
Various devices and schemes have been developed to define and subsequently reverse such distortion to provide what is called a restored image of the subject. Some of the schemes involve a matrix of measured points which define the distortion and noise or measurement error. The matrix is inverted to provide a restored image. These schemes, how-ever, are typically cumbersome and unsuitable for images with hundreds of data points or more.
Summary of the Invention An object of the present invention is to provide a method and device for correcting optical distortion and for providing a restored image, especially one from a viewed image of hundreds of data points.
A further object of the present invention is to provide such a method and device which involves a division operation at each pixel instead of an inversion of a matrix.
~ 3 In particular, the device of the present invention includes an optical viewing member through which the subject is viewed, a data processor for receiving and storing a series of viewed images of the subject as viewed through the optical viewing member, processing means associated with the data processor for restoring the viewed images, and display means for providing a view of the restored images. The processing means restores the viewed images by iteratively determining, for each point in a viewed image, a factor which minimizes noise and distortion at that point. The factor is iteratively determined through a division operation, in a trans-form domain, of functions of the response function of the optical viewing member. In a preferred embodiment, the factor is iteratively determined through a division operation in a Fourier transform domain of the response function.
More specifically, the factor is iteratively determined by the relationship C 1 = C -where c is the factor;
n is the numbar of iterations of c; and = inverse Fourier transform of / (H H + ~ +~));
~ 3 ~ 3 where ~ ' is the Fourier transform of ~', ~' = (M + ~ I) c - g;
M is a matrix of viewed image points along orthogonal axes i and j, and defined as Mij = 5ki(x) kj(x) dx D(c) D(c) is the set of viewed image points in which ki(x) ci 0, and ki(x) is the response function of the optical viewing member at image pixel i over surrounding points x;
kj(x) is the response function at image pixel j ;
and ~ are constants (~ 0);
I is the Identity matrix;
g is an array of measured values of the points in the image;
H is the Fourier transform of h(x) which is another form of the response function k.(x); and *
H is the Fourier transform of h(x) in which h(x) = h(-~).
The h(x) form of the response function ki(x) may be the point spread function of the optical viewing member such that ~ 3 ~
ki(x) = h(yi-x) where i is a point of the viewed image; and Yi is the corresponding point on the subject.
In accordance with a particular embodiment s of the invention there is provided an image restoration device comprising:
an optical viewing member for providing a view of a subject, the optical viewing member responding to each point of the subject according to a response function to provide a viewed image o~ the subject;
a data processor for receiving from the optical viewing member and storing a series of viewed images of the subject as viewed through the optical viewing member;
computer processing means coupled to the data processor for responding to the viewed images . and restoring the viewed images to substantially noiseless, undistorted images, the computer processing means restoring each viewed image by iteratively determining for each point in a viewed image a factor which minimizes noise and distortion at that point, the factor being iteratively determined, according to a formulation of a 2s mathematical definition of the factor, by performance of transform calculations including a mathematical division operation in a transform domain of functions which include the response function of the optical viewing member to determine the factor, transforms of the response function being denominators in the division operation, the division operation enabling avoidance of performance of a matrix inversion operation, and with the deter-mined factors for the points of a viewed image the 3s computer processing means minimizing noise and dis-- 5a - ~ 3 ~ V~
tortion at each point of the image such that a sub-stantially noiseless undistorted image is formed;
and display means coupl~d to the computer 5 processin~ means ~or providing a view of the sub-stantially noiseless, undistorted images in a non-transform domain.
In accordance with a further embodiment of the invention there is provided an image restoration 0 device comprising:
(i) an optical viewing member for providing a view of a subject, the optical viewing member responding to each point of the subject according to a response function to provide a viewed image of the subjecti (ii) a data processor for receiving from the optical viewing member and storing a series of viewed images of the subject as viewed through the optical viewing member;
(iii) computer processing means associated with the data processor for restoring the viewed images to substantially noiseless, undistorted images, the computer processing means restoring each viewed image by:
2s (a) iteratively determining for each point in a viewed image a factor which minimizes noise and distortion at that point, the factor being iteratively determined according to a formulation of a mathematical definition of the factor, by performance of transform calculations including a division operation in a transform domain of functions which include the response function of the optical viewing member to determine the factor;
.~
.~ ~ .~. .
- 5b ~
the factor being iteratively determined by the relationship Cn+l = Cn ~ "
where c is the factor;
n is the number of iterations of c; and A = inverse Fourier transform of (~ /(H H ~ a + ~));
where ~' is the Fourier transform of ~', ~' = (M ~ ~I) c - g;
M is a matrix of viewed image points i,j and defined as Mij = 5 ki(x) kj(x) dx D(c) D(c) is the set of viewed image points in which ~ki(x) Ci > 0, and ki(x) is the response function of the optical viewing member at image point i over surrounding points x;
~ and ~ are constants (20);
I is the Identity matrix;
g is an array of measured values of the points in the image;
H is the Fourier transform of h(x) which 2s is another form of the response function ki(x); and H* is the Fourier transform of h(x) in which ~(x) = h(-x); and (b) with the determined factors of a viewed image, minimizing noise and distortion at each point of the viewed image such that a substantially noiseless, undistorted image is formed; and (iv) display means coupled to the computer processing means for providing a view of the sub-stantially noiseless undistorted images in a non-transform domain.
,J."i~L
~` ,r - 5c -From a different aspect and in accordance with a particular embodiment of the invention there is provided a method of restoring an image of a subject, comprising the steps of:
s viewing the subject through an optical viewing system having a response to each point of the subject according to a response function, during the viewing the optical viewing system providing viewed images of the subject;
receiving in a data processor a series of viewed images of the subject as viewed through the optical viewing member;
restoring each viewecl image in the data processor by:
(a) iteratively determining, for each point in said viewed image, a factor which minimizes noise and optical distortion, the factor being iteratively determined according to a formulation of a mathematical definition of the factor which allows 20 performance of transform calculations including a division calculation with a transform of the response function of the optical viewing member as a denominator in the division calculation to determine the factor, such that performance of matrix 25 inversion is avoidable., (b) with the determined factors of the viewed image, minimizing noise and distoxtion at each point in said viewed image by manipulating said points, and (c) forming a substantially noiseless, undistorted image from the manipulated points of the viewed image; and displaying on display means coupled to the data processor each substantially noiseless and 35 undistorted image for each viewed image restored by the data process.
`J
, .
_ 5~ _ ~ 3~ 3 In accordance with a further particular embodiment of the second aspect, there is provided a method of restoring an image of a subject, comprising the steps of:
s viewing the subject through an optical viewing system having a response to each point of the subject according to a response function, during the viewing the optical viewing system providing viewed images of the subject;
receiving in a data processor a series of viewed images of the subject as viewed through the optical viewing member;
restoring each viewed image in the data processor by:
(a) iteratively determlning, for each point in a viewed image, a factor which minimizes noise and optical distortion, the factor being iteratively determined according to a formulation of a mathematical definition of the factor which allows performance of transform calculations including division with a transform of the response function of the optical viewing member to determine the factor, the factor being iteratively determined by 2s the relationship cn+l = Cn ~ Q
where c is the factox;
n is the number of iterations of c; and a = inverse Fourier transform of (~ /(H H + ~ + ~));
where ~' is the Fourier transform of ~', ~' = (M + ~I) c - gi M is a matrix of viewed image points i,j 3s and defined as Mij = ~ ki(X) kj(x) dx D(c) - 5e - ~3~
D(C) iS the set of viewed image points in which ~ki(x) ci > 0, and ki(x) is the response function of the optical viewing member at image point i over surrounding points x;
~ and ~ are constants (20);
I is the Identity matrix;
g is an array of measured values of the points in the image;
H is the Fourier transform of h(x) which is another form of the response function ki(x); and H* is the Fourier transform of h(x) in which h(x) = h(-x);
( b) with the determined factors of a viewed image, minimizing noise and distortion at each point in said viewed image by manipulating said points;
and (c) forming a substantially noiseless, 20 undistorted image from the manipulated points of the viewed image.
Brief Description of the Drawin~s The foregoing and other objects, features and advantages of the invention will be apparent 25 from the following more particular description of a preferred embodiment of the invention, as illus-trated in the accompanying drawings. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
Figure l is an illustration of the distortion and adaition of noise to a subject viewed through an optical system.
Figure 2 is a block diagram of an 3s embodiment of the present invention.
.~i~..~
_ 5f _ ~3~
Detailed Description of the Preferred Embodiment In general, the present invention provides a viewing system as shown in Figure 2. The user views a desired subject 7 through an optical system 5 9 such as a microscope, a telescope, a camera, a system of lenses and the like, or a combination thereof. The viewed image of the subject is captured by a data processor 13 which is connected to the optical system 9. The data processor 13 receives and stores in a first memory 19 a series of viewed images of .~
.~
~3~f~ j the subject 7 as the user views the subject through the optical system 9. Computer or other processing means 15 are associated with the data processor 13.
The processing means 15 restores the viewed images, which have been received by the data processor 13 and stored in first memory 19, by manipulating the measured data points of the received images in a second memory 5. A mathematical discussion of this manipulation is presented later. The restored images are thereafter displayed by suitable means, such as CRT 17 connected to the data processor 13.
A particular embodiment of the present inven-tion involves fluorescence microscopy, where cell structures are illuminated by fluorescent dyes and are viewed through an optical microscope to provide an image of the distribution of specific molecules and ions in the cell. 3-D information about the cell is obtained by optical sectioning. This requires focusing the microscope through a series of 128 planes of about 0.25 micrometers apart. Each of the 128 2-D images is acquired by a CCD camera mounted on the microscope and stored as a 128 x 128 image in a data processor. The stored images have been blurred by distortion and noise as previously stated. The distortion and blurring is removed by a deconvolution process as discussed in "3-D Molecular Distribution in Living Cells by Deconvolution of Optical Sectîons Using Light Microscopy", by Walter Carrington and Kevin Fogarty, reprinted in the Proceedings o~ the Thirteenth Northeast ~ 3 ~
Bioengineering Conference, Philadelphia, PA, March 12, 1987.
In sum, the deconvolution process described in the Carrington/Fogarty article solves S the sum of a least square fit of the measured data points and a regularization or smoothing term. In addition, the solution is constrained to non-negative values because the viewed subject is known to have a non-negative density. The solution is 10 also determined by an iterative division of Fourier transforms of functions of the point spread function, or more generally the response function of the optical system 9. In the iterative division, the viewed image is expressed as a convolution of 15 the response function and the function describing the subject 7.
The convolution is preferably performed by a high speed computer or an array processor con-nected to the data processor in a second memory 5 as shown in Figure 2.
By way of background, manipulation of stored images to provide a restored image has been accomplished in many ways. The simplest method is a direct inversion of the mathematical statement which s is illustrated in Figure l.
g = h*f + ~
where the inverse is given in the Fourier domain by ~ 3 ~ 3 F = (l/H) (G+ ~) This is not an accurate method because the inversion magnifies the noise in the data excessively.
Another method uses a conventional least square approach in which measurement errors are minimized.
The least square solution is expressed as a normalization of the difference of the transformed image (as known by the function k) and the viewed (i.e. measured~ image g.
!l~f - gl! 2 In addition, a linear smoothing operation enables high frequency oscillations in the direct solution to be damped to a lower amount of high frequency oscillations. This is accomplished by adding to the least s~lare solution a smoothing factor of f ( x)1 2dx Hence, a known equation used to describe the distorted image is ~ (f) = l~kf - gll 2 +~ f(x)~ 2dx Equation 1 This equation is solved for the minimum function f.
In the present invention a further constraint is imposed on the solution of Equation 1. This constraint is where f is greater than or equal to ~3~3~
zero, because the subject is known to be non-negative. A trivial calculus approach to solving Equation 1 for the minimum f, by setting the derivative of Equation 1 equal to zero, is not possible since f is a function and f(x) is constrained to be non-negative. The f(x) 70 that minimizes Equation 1 is known, however, to have the form f(x) = max(~ ki(x)ci, 0) at a point x in the subject. The ~pplicants then take this form of f and substitute it into Equation 1 so that ~ (f) of Equation 1 is a function of ci.
In this form, Equation 1 is then differentiated with respect to c. The derivative of Equation 1, (i.e. ~'(f)) is then translated into terms of c and set equal to zero. The c which satisfies that relationship defines the f that minimizes Equation 1. The problem can then be expressed in the form kf - g ~ ~c = 0 Equation 2 where c is a 3D vector of the ci's.
In prior art methods, Equation 2 was solved for f by Newton's method. In the present invention, an auxiliary function is defined by integrating Equation 2 such that the derivative of the auxiliary 2S function is zero to satisfy Equation 2 and then a ~ 3~9~
variation of Newton's method is applied to Equation 2 written in terms of the defined auxiliary function. Such an auxiliary function ~ (c) is stated as follows:
~;(c) = ~c (M +,~ I)c - c~g where c+ is the transpose of c.
Next, by Newton~s method c has the approximate form cn~l cn ~ Equation 3 where ~ c) 1 ~'(c) Equation 4 where ~" (c) = (M +~ I) c - g, is the gradient of (~! (C) and ~ " (c) = M +~ I, known as the Hessian matrix.
In the present invention a damped Newton's method is used to replace~'' with another matrix A, such that A = B ~ I Equation 5 where B is a matrix of i,j coordinates over all space defined as ~ 3 '~t 9 ~
Bi~ = JJJki(x)kj(x)dx which in terms of the point spread function h of the optical system may be written as Bij rh(yi-x) h(yj-x)dx or B = ~h(yi-xk)h(Yj xk) Equation 6 in Equation 3 is then replaced by ~ = (A+~ Equation 7 It is noted that Bij differs from Mij by having a domain over all points in space instead of over a lO set of viewed image points D(c).
Solving for ~ from~ = (A ~ is then as follows.
A = ( A+ ~
(A+~ , Equation 8 Equation 8 is then expressed as a convolution which is easier to multiply than the matrices of data of prior art schemes. A "convolution" as used here is defined to follow the statement that:
A summation, over points L, of the product of a function of variables m and L and a function of ~ 3~
L, equals the convolution of the two functions at a point m.
From Equations 5 and 8 (B ~ = y~
B~ + ~ +~ ' Equation 9 From Equation 6 a) i ~. ~, h (yi-xk) h (Yj -Xk) ~ j ~ h ( yi-Xk) h ( Xk Yj ) ~ j where h(x) = h(-x); then by the definition of convolution ( )i ~ h(yi-Xk) ((h*~)(Xk)) = (h * ~*~)) (Yi) = h* ~*~ (Yi) Equation 10 where * indicates a convolution.
Substituting Equation 10 into Equation 9 h * h * *~+ ~ ' Equation 11 ~ 3 ~
Equation 11 is then solved for ~by transforming each side; dividing the the transform of ~' by the sum of the transforms of h * ~,cX and/\; and calculating the inverse transform of this quotient.
In particular, a Fourier transform is employed, although other transforms are suitable, such that:
(h*h*~ ~ + (~ )~+ (~a) = (~ ~
h~ + ~ ~ +
(h~ ~A + ~+ ~
( y~
~ = = Equation 12 1 n ~\
(h~' h~+ ~+ ~) H H ~ +~
where ~ indicates Fourier transform;
' is the Fourier transform of ~'; and H , the Fourier transform of ~, is the complex conjugate of H, the Fourier transform of h.
Applying the inverse Fourier transform to Equation 12 provides = Inverse Fourier Transform of (~ ~(HH*~+~)) Equation 13 The value of ~ from Equation 13 is substituted into E~uation 3 to define a new c (i.e. cn+l) where a first cn (i.e. cO) i5 arbitrarily chosen. A
series of new c's is iteratively defined in a ~ 3 similar fashion using Equations 3 through 13 and the foregoing procedure. The iteration is stopped at a final value for c when ~ approaches 0;
~' gets small;
length of ~' approaches O;
length of g ~increases instead of continues to decrease;
or a combination thereof.
Once a final value for c is established, the f(x) ~ 0, that minimizes Equation 1 at a point i in the image g is defined from f(x~ = max ~ki(x)c, 0).
The newly defined f(x) is substituted into Equation 1, and Equation 1 is used to provide the sought restored image from point to point.
While the invention has been particularly shown and described with reference to embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (16)
1. An image restoration device comprising:
an optical viewing member for providing a view of a subject, the optical viewing member responding to each point of the subject according to a response function to provide a viewed image of the subject;
a data processor for receiving from the optical viewing member and storing a series of viewed images of the subject as viewed through the optical viewing member;
computer processing means coupled to the data processor for responding to the viewed images and restoring the viewed images to substantially noiseless, undistorted images, the computer processing means restoring each viewed image by iteratively determining for each point in a viewed image a factor which minimizes noise and distortion at that point, the factor being iteratively determined, according to a formulation of a mathematical definition of the factor, by performance of transform calculations including a mathematical division operation in a transform domain of functions which include the response function of the optical viewing member to determine the factor, transforms of the response function being denominators in the division operation, the division operation enabling avoidance of performance of a matrix inversion operation, and with the deter-mined factors for the points of a viewed image the computer processing means minimizing noise and dis-tortion at each point of the image such that a sub-stantially noiseless undistorted image is formed;
and display means coupled to the computer processing means for providing a view of the sub-stantially noiseless, undistorted images in a non-transform domain.
an optical viewing member for providing a view of a subject, the optical viewing member responding to each point of the subject according to a response function to provide a viewed image of the subject;
a data processor for receiving from the optical viewing member and storing a series of viewed images of the subject as viewed through the optical viewing member;
computer processing means coupled to the data processor for responding to the viewed images and restoring the viewed images to substantially noiseless, undistorted images, the computer processing means restoring each viewed image by iteratively determining for each point in a viewed image a factor which minimizes noise and distortion at that point, the factor being iteratively determined, according to a formulation of a mathematical definition of the factor, by performance of transform calculations including a mathematical division operation in a transform domain of functions which include the response function of the optical viewing member to determine the factor, transforms of the response function being denominators in the division operation, the division operation enabling avoidance of performance of a matrix inversion operation, and with the deter-mined factors for the points of a viewed image the computer processing means minimizing noise and dis-tortion at each point of the image such that a sub-stantially noiseless undistorted image is formed;
and display means coupled to the computer processing means for providing a view of the sub-stantially noiseless, undistorted images in a non-transform domain.
2. An image restoration device as claimed in claim 1 wherein the factor is iteratively determined through a division operation in a Fourier transform domain of the response function.
3. An image restoration device as claimed in claim 1 wherein the factor is iteratively determined by the relationship Cn+1 = Cn - .DELTA.
where c is the factor;
n is the number of iterations of c; and .DELTA. inverse Fourier transform of (? /(H H* + .alpha. + .lambda.));
where ?' is the Fourier transform of ?', ?' = (M + .alpha.I) c - g;
M is a matrix of viewed image points i,j and defined as Mij = ? ki(x) kj(x) dx D(c) D(c) is the set of viewed image points in which .SIGMA.ki(x) ci > 0, and ki(x) is the response function of the optical viewing member at image point i over surrounding points x;
.alpha. and .lambda. are constants (?0);
I is the Identity matrix;
g is an array of measured values of the points in the image;
H is the Fourier transform of h(x) which is another form of the response function ki(x); and H* is the Fourier transform of ?(x) in which ?(x) = h(-x).
where c is the factor;
n is the number of iterations of c; and .DELTA. inverse Fourier transform of (? /(H H* + .alpha. + .lambda.));
where ?' is the Fourier transform of ?', ?' = (M + .alpha.I) c - g;
M is a matrix of viewed image points i,j and defined as Mij = ? ki(x) kj(x) dx D(c) D(c) is the set of viewed image points in which .SIGMA.ki(x) ci > 0, and ki(x) is the response function of the optical viewing member at image point i over surrounding points x;
.alpha. and .lambda. are constants (?0);
I is the Identity matrix;
g is an array of measured values of the points in the image;
H is the Fourier transform of h(x) which is another form of the response function ki(x); and H* is the Fourier transform of ?(x) in which ?(x) = h(-x).
4. An image restoration device as claimed in claim 3 wherein h(x) is the point spread function of the optical viewing member such that ki(x) = h(Yi - x) where I is a point of the viewed image; and Yi is the corresponding point on the subject.
5. An image restoration device as claimed in claim 1 wherein the series of viewed images received by the data processor from the optical viewing member includes a series of 2-D images of different planes through a 3-D image of the subject.
6. An image restoration device as claimed in claim 1 where the formulation of the mathematical definition of the factor includes a related matrix which replaces a Hessian matrix of the factor such that the division operation replaces a matrix inversion operation and the factor is iteratively determinable in a transform domain of the response function of the optical viewing member.
7. A method of restoring an image of a subject, comprising the steps of:
viewing the subject through an optical viewing system having a response to each point of the subject according to a response function, during the viewing the optical viewing system providing viewed images of the subject;
receiving in a data processor a series of viewed images of the subject as viewed through the optical viewing member;
restoring each viewed image in the data processor by:
(a) iteratively determining, for each point in said viewed image, a factor which minimizes noise and optical distortion, the factor being iteratively determined according to a formulation of a mathematical definition of the factor which allows performance of transform calculations including a division calculation with a transform of the response function of the optical viewing member as a denominator in the division calculation to determine the factor, such that performance of matrix inversion is avoidable., (b) with the determined factors of the viewed image, minimizing noise and distortion at each point in said viewed image by manipulating said points, and (c) forming a substantially noiseless, undistorted image from the manipulated points of the viewed image; and displaying on display means coupled to the data processor each substantially noiseless and undistorted image for each viewed image restored by the data process.
viewing the subject through an optical viewing system having a response to each point of the subject according to a response function, during the viewing the optical viewing system providing viewed images of the subject;
receiving in a data processor a series of viewed images of the subject as viewed through the optical viewing member;
restoring each viewed image in the data processor by:
(a) iteratively determining, for each point in said viewed image, a factor which minimizes noise and optical distortion, the factor being iteratively determined according to a formulation of a mathematical definition of the factor which allows performance of transform calculations including a division calculation with a transform of the response function of the optical viewing member as a denominator in the division calculation to determine the factor, such that performance of matrix inversion is avoidable., (b) with the determined factors of the viewed image, minimizing noise and distortion at each point in said viewed image by manipulating said points, and (c) forming a substantially noiseless, undistorted image from the manipulated points of the viewed image; and displaying on display means coupled to the data processor each substantially noiseless and undistorted image for each viewed image restored by the data process.
8. A method as claimed in claim 7 wherein the factor is iteratively determined through a division operation with a Fourier transform of the response function.
9. A method as claimed in claim 7 wherein the factor is iteratively determined by the relationship Cn+1 = Cn - .DELTA.
where c is the factor;
n is the number of iterations of c; and .DELTA. = inverse Fourier transform of (?/(H H* + .alpha. + .lambda.));
where ?' is the Fourier transform of ?', ?' = (M + .alpha.I) c - g;
M is a matrix of viewed image points i,j and defined as Mij = ? ki(x) kj(x) dx D(c) D(c) is the set of viewed image points in which .SIGMA.ki(x) ci > O, and ki(x) is the response function of the optical viewing member at image point i over surrounding points x;
.alpha. and .lambda. are constants (?0);
I is the Identity matrix;
g is an array of measured values of the points in the image;
H is the Fourier transform of h(x) which is another form of the response function ki(x); and H* is the Fourier transform of ?(x) in which ?(x) = h(-x).
where c is the factor;
n is the number of iterations of c; and .DELTA. = inverse Fourier transform of (?/(H H* + .alpha. + .lambda.));
where ?' is the Fourier transform of ?', ?' = (M + .alpha.I) c - g;
M is a matrix of viewed image points i,j and defined as Mij = ? ki(x) kj(x) dx D(c) D(c) is the set of viewed image points in which .SIGMA.ki(x) ci > O, and ki(x) is the response function of the optical viewing member at image point i over surrounding points x;
.alpha. and .lambda. are constants (?0);
I is the Identity matrix;
g is an array of measured values of the points in the image;
H is the Fourier transform of h(x) which is another form of the response function ki(x); and H* is the Fourier transform of ?(x) in which ?(x) = h(-x).
10. A method as claimed in claim 9 wherein h(x) is the point spread function of the optical viewing member such that ki(x) = h(yi -x) where i is a point of the viewed image; and Yi is the corresponding point on the subject.
11. A method as claimed in claim 7 wherein the step of receiving in a data processor a series of viewed images of the subject includes receiving series of images of parallel planes through a 3-D
image of the subject as viewed through the optical viewing member.
image of the subject as viewed through the optical viewing member.
12. A method as claimed in claim 7 wherein the step of restoring each viewed image includes a formulation of a mathematical definition of the factor which replaces a Hessian matrix of the factor with a related matrix such that a division operation replaces a matrix inversion operation and the factor is iteratively determinable in a transform domain of the response function of the optical viewing member.
13. An image restoration device comprising:
(i) an optical viewing member for providing a view of a subject, the optical viewing member responding to each point of the subject according to a response function to provide a viewed image of the subject;
(ii) a data processor for receiving from the optical viewing member and storing a series of viewed images of the subject as viewed through the optical viewing member;
(iii) computer processing means associated with the data processor for restoring the viewed images to substantially noiseless, undistorted images, the computer processing means restoring each viewed image by:
(a) iteratively determining for each point in a viewed image a factor which minimizes noise and distortion at that point, the factor being iteratively determined according to a formulation of a mathematical definition of the factor, by performance of transform calculations including a division operation in a transform domain of functions which include the response function of the optical viewing member to determine the factor;
the factor being iteratively determined by the relationship Cn+1 = Cn - .DELTA.
where c is the factor;
n is the number of iterations of c; and .DELTA. = inverse Fourier transform of (?'/(H H* + .alpha. + .lambda.));
where ?' is the Fourier transform of ?', ?' = (M + .alpha.I) c - g;
M is a matrix of viewed image points i,j and defined as Mij = ? ki(x) kj(x) dx D(c) D(c) is the set of viewed image points in which .SIGMA.ki(x) Ci > 0, and ki(x) is the response function of the optical viewing member at image point i over surrounding points x;
.alpha. and .lambda. are constants (?0);
I is the Identity matrix;
g is an array of measured values of the points in the image;
H is the Fourier transform of h(x) which is another form of the response function ki(x); and H* is the Fourier transform of ?(x) in which ?(x) = h(-x); and (b) with the determined factors of a viewed image, minimizing noise and distortion at each point of the viewed image such that a substantially noiseless, undistorted image is formed; and (iv) display means coupled to the computer processing means for providing a view of the sub-stantially noiseless undistorted images in a non-transform domain.
(i) an optical viewing member for providing a view of a subject, the optical viewing member responding to each point of the subject according to a response function to provide a viewed image of the subject;
(ii) a data processor for receiving from the optical viewing member and storing a series of viewed images of the subject as viewed through the optical viewing member;
(iii) computer processing means associated with the data processor for restoring the viewed images to substantially noiseless, undistorted images, the computer processing means restoring each viewed image by:
(a) iteratively determining for each point in a viewed image a factor which minimizes noise and distortion at that point, the factor being iteratively determined according to a formulation of a mathematical definition of the factor, by performance of transform calculations including a division operation in a transform domain of functions which include the response function of the optical viewing member to determine the factor;
the factor being iteratively determined by the relationship Cn+1 = Cn - .DELTA.
where c is the factor;
n is the number of iterations of c; and .DELTA. = inverse Fourier transform of (?'/(H H* + .alpha. + .lambda.));
where ?' is the Fourier transform of ?', ?' = (M + .alpha.I) c - g;
M is a matrix of viewed image points i,j and defined as Mij = ? ki(x) kj(x) dx D(c) D(c) is the set of viewed image points in which .SIGMA.ki(x) Ci > 0, and ki(x) is the response function of the optical viewing member at image point i over surrounding points x;
.alpha. and .lambda. are constants (?0);
I is the Identity matrix;
g is an array of measured values of the points in the image;
H is the Fourier transform of h(x) which is another form of the response function ki(x); and H* is the Fourier transform of ?(x) in which ?(x) = h(-x); and (b) with the determined factors of a viewed image, minimizing noise and distortion at each point of the viewed image such that a substantially noiseless, undistorted image is formed; and (iv) display means coupled to the computer processing means for providing a view of the sub-stantially noiseless undistorted images in a non-transform domain.
14. An image restoration device as claimed in claim 13 wherein h(x) is the point spread function of the optical viewing member such that ki(x) = h(yi -X) where i is a point of the viewed image; and yi is the corresponding point on the subject.
15. A method of restoring an image of a subject, comprising the steps of:
viewing the subject through an optical viewing system having a response to each point of the subject according to a response function, during the viewing the optical viewing system providing viewed images of the subject;
receiving in a data processor a series of viewed images of the subject as viewed through the optical viewing member;
restoring each viewed image in the data processor by:
(a) iteratively determining, for each-point in a viewed image, a factor which minimizes noise and optical distortion, the factor being iteratively determined according to a formulation of a mathematical definition of the factor which allows performance of transform calculations including division with a transform of the response function of the optical viewing member to determine the factor, the factor being iteratively determined by the relationship Cn+1 = Cn - .DELTA.
where c is the factor;
n is the number of iterations of c; and .DELTA. = inverse Fourier transform of (?1 /(H H* + .alpha. .lambda.)) where ?1 is the Fourier transform of ?1, ?1 = (M + .alpha.I) c - g;
M is a matrix of viewed image points i,j and defined as Mij = ? ki(x) kj(x) dx D(c) D(c) is the set of viewed image points in which .SIGMA.ki(x) Ci > 0, and ki(x) is the response function of the optical viewing member at image point i over surrounding points x;
.alpha. and .lambda. are constants (? 0) I is the Identity matrix;
g is an array of measured values of the points in the image;
H is the Fourier transform of h(x) which is another form of the response function ki(x); and H* is the Fourier transform of ?(x) in which ?(x) = h(-x);
(b) with the determined factors of a viewed image, minimizing noise and distortion at each point in said viewed image by manipulating said points;
and (c) forming a substantially noiseless, undistorted image from the manipulated points of the viewed image.
viewing the subject through an optical viewing system having a response to each point of the subject according to a response function, during the viewing the optical viewing system providing viewed images of the subject;
receiving in a data processor a series of viewed images of the subject as viewed through the optical viewing member;
restoring each viewed image in the data processor by:
(a) iteratively determining, for each-point in a viewed image, a factor which minimizes noise and optical distortion, the factor being iteratively determined according to a formulation of a mathematical definition of the factor which allows performance of transform calculations including division with a transform of the response function of the optical viewing member to determine the factor, the factor being iteratively determined by the relationship Cn+1 = Cn - .DELTA.
where c is the factor;
n is the number of iterations of c; and .DELTA. = inverse Fourier transform of (?1 /(H H* + .alpha. .lambda.)) where ?1 is the Fourier transform of ?1, ?1 = (M + .alpha.I) c - g;
M is a matrix of viewed image points i,j and defined as Mij = ? ki(x) kj(x) dx D(c) D(c) is the set of viewed image points in which .SIGMA.ki(x) Ci > 0, and ki(x) is the response function of the optical viewing member at image point i over surrounding points x;
.alpha. and .lambda. are constants (? 0) I is the Identity matrix;
g is an array of measured values of the points in the image;
H is the Fourier transform of h(x) which is another form of the response function ki(x); and H* is the Fourier transform of ?(x) in which ?(x) = h(-x);
(b) with the determined factors of a viewed image, minimizing noise and distortion at each point in said viewed image by manipulating said points;
and (c) forming a substantially noiseless, undistorted image from the manipulated points of the viewed image.
16. A method as claimed in claim 15 wherein h(x) is the point spread function of the optical viewing member such that ki(x) = h(Yi - X) where i is a point of the viewed image; and yi is the corresponding point on the subject.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US16413788A | 1988-03-04 | 1988-03-04 | |
| US164,137 | 1998-09-30 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA1319415C true CA1319415C (en) | 1993-06-22 |
Family
ID=22593136
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA000592676A Expired - Fee Related CA1319415C (en) | 1988-03-04 | 1989-03-03 | Iterative image restoration device |
Country Status (1)
| Country | Link |
|---|---|
| CA (1) | CA1319415C (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10757390B2 (en) | 2016-08-05 | 2020-08-25 | Interdigital Ce Patent Holdings | Method for obtaining at least one sub-aperture image being associated with one view |
-
1989
- 1989-03-03 CA CA000592676A patent/CA1319415C/en not_active Expired - Fee Related
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10757390B2 (en) | 2016-08-05 | 2020-08-25 | Interdigital Ce Patent Holdings | Method for obtaining at least one sub-aperture image being associated with one view |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US5047968A (en) | Iterative image restoration device | |
| Tsang | Quantum limit to subdiffraction incoherent optical imaging | |
| Fienup | Space object imaging through the turbulent atmosphere | |
| Brown et al. | The shear power spectrum from the COMBO-17 survey | |
| Hanke et al. | Restoration of atmospherically blurred images by symmetric indefinite conjugate gradient techniques | |
| Zhou et al. | Image pre-filtering for measurement error reduction in digital image correlation | |
| JPH10509817A (en) | Signal restoration method and apparatus | |
| CN111174912B (en) | Snapshot type dispersion ambiguity-resolving hyperspectral imaging method | |
| JP4499331B2 (en) | System and method for recovering wavefront phase information | |
| Cantale et al. | Firedec: a two-channel finite-resolution image deconvolution algorithm | |
| Liu | Augmented Lagrangian method for total generalized variation based Poissonian image restoration | |
| Grama et al. | Computation of full-field strains using principal component analysis | |
| Debarnot et al. | Learning low-dimensional models of microscopes | |
| Bardsley et al. | Blind iterative restoration of images with spatially-varying blur | |
| Soulez | A “learn 2D, apply 3D” method for 3D deconvolution microscopy | |
| Lazzaretti et al. | Weighted-celo sparse regularisation for molecule localisation in super-resolution microscopy with Poisson data | |
| Biggs | Accelerated iterative blind deconvolution | |
| Verveer et al. | Acceleration of the ICTM image restoration algorithm | |
| Bangun et al. | Wigner distribution deconvolution adaptation for live ptychography reconstruction | |
| CA1319415C (en) | Iterative image restoration device | |
| Schneider et al. | Galaxy-galaxy-galaxy lensing: Third-order correlations between the galaxy and mass distributions in the Universe | |
| Veran et al. | Deconvolution method for accurate astrometry and photometry on adaptive optics images of stellar fields | |
| Beinert et al. | Total Variation‐Based Reconstruction and Phase Retrieval for Diffraction Tomography with an Arbitrarily Moving Object | |
| US20100189377A1 (en) | Method of estimating at least one deformation of the wave front of an optical system or of an object observed by the optical system and associated device | |
| US5043930A (en) | Digital simulation model for forward looking infrared (FLIR) sensors |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MKLA | Lapsed |