JP2003255231A - Optical imaging system and optical image data processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an optical imaging system having a spatial resolution in a level of 50 nm and an optical image data processing method, which enable easy and stable scanning optical microscopic observation of high spatial resolution. <P>SOLUTION: Arithmetics of integrating a two-dimensional light intensity distribution reconstituted by inverse arithmetics taking a dot image distribution function as a seed function on the basis of a two-dimensional light intensity distribution of each light radiation area obtained by a two-dimensional light detector and constituting a two-dimensional light intensity distribution of an overall observation area are executed in a computer provided in a data processing system to display the light intensity distribution of the overall observation area as an image. In executing the arithmetics, each of optical signal intensity data constituting the two-dimensional light intensity distribution of each light radiation area obtained by the two-dimensional light detector is multiplied by the least square filter based on the dot image distribution function to correct the signal light intensity, and then the two-dimensional light intensity distribution is reconstituted. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an optical imaging system and an optical image data processing method, and more particularly to an optical imaging system and optical image data processing capable of observing with a scanning optical microscope at high spatial resolution. Regarding the method.

[0002]

2. Description of the Related Art A scanning optical microscope has been used for a long time as a general-purpose device for observing and measuring the shape and size of a specimen sample, and in particular, it has a high resolution and contrast for semiconductor samples and biological samples. When the observation is required, a confocal laser scanning microscope with a spatial resolution of about 200 nm is widely used.

FIG. 7 is a diagram for explaining a configuration example of a conventional confocal laser scanning microscope system. This confocal laser scanning microscope system is an optical system having a configuration used in a general scanning optical microscope. System 700 and data processing system 72
And the observation sample 7 on the sample stage 701.
02 is mounted, and the light emitted from the laser 707 as a light source is condensed by the lenses 708 and 709, and then the pinhole 7
The desired spot diameter is set at 10 and a parallel beam is formed by the lens 711, then the optical path is changed to the lower side by the half mirror 713 and converged by the objective lens 703 to make the observation sample 7
02.

The fluorescence or the like emitted from the observation sample 702 by this light irradiation is used for the objective lens 703 and the half mirror 71.
2, and the photomultiplier tube 70 which is a photodetector after being condensed by the lens 704 and passing through the pinhole 705.
The light intensity is detected by 6. In addition, this optical system 700
Is equipped with a drive mechanism (not shown) for moving the sample stage 701 in the XY plane.
It is possible to obtain two-dimensional mapping data by performing observation while driving in the XY plane of 01.

The light intensity data detected by the photomultiplier tube 706 is stored in a data processing system 72 including a computer.
0 is sequentially recorded as fluorescence intensity values (g _{x, y} ) for each observation point, and an observation data matrix R _g corresponding to two-dimensional mapping of a predetermined observation area is formed. Then, the observation fluorescence intensity is displayed on the display screen when the data acquisition of the fluorescence intensity values at all the observation points is completed, and the observation data matrix R _g is used as the display image data matrix R _i to form an image. A two-dimensional image of the sample 702 is reproduced.

Incidentally, the spot light scanning device 712 is arranged in the optical path of the irradiation light instead of the above-mentioned method of obtaining the two-dimensional mapping by moving the sample stage 701 in the XY plane.
It is also possible to change the light irradiation point on the observation sample 702 to obtain a two-dimensional mapping by synchronizing the drive of the spot light scanning device 712 with the observation point signal transmitted from the data processing system 720. Good.

Further, for example, in Japanese Unexamined Patent Publication No. 2001-91847, there is also an invention of a microscope capable of reliably exhibiting a super-resolution effect by an annular illumination by avoiding a reduction in resolution due to diffraction by an annular mask. It is disclosed.

Further, in order to meet the recent demand for development of a high-resolution optical microscope for biotechnology research, a mesh-like standing irradiation light is emitted by irradiating light from four sides of an observation sample.
luminescence) on the observation sample and image processing the fluorescent image obtained from the sample.
HELM (harmonic) with 0 nm resolution
excitation light microsc
has been reported by Frohn et al. (PN
AS, 7232-7236, Vol. 97, No. Thirteen
(2000)).

[0009]

However, in the confocal laser scanning microscope of the conventional configuration, only the integrated intensity of the signal light emitted from the sample area irradiated with light is captured as information from the observation sample and image processing is performed. Therefore, the spatial resolution is determined by the spot size of the light irradiated on the sample, and there is a problem that the improvement of the spatial resolution as a microscope is naturally limited.

Further, the HELM described above employs a special and complicated optical system for forming the mesh-shaped standing irradiation light, which makes the microscope main body expensive and causes a slight vibration or sample ( Also, as a result of the drift of the sample holder) having a great influence on the spatial resolution, there is also a problem that it is insufficient in terms of simplicity and stability in general-purpose use.

The present invention has been made in view of the above problems, and an object thereof is to enable easy and stable observation of a scanning optical microscope with high spatial resolution.
An object of the present invention is to provide an optical imaging system having a spatial resolution of 0 nm level and an optical image data processing method.

[0012]

In order to achieve such an object, the present invention according to claim 1 irradiates an observation sample with light emitted from a light source as spot light to perform the observation. An optical imaging system including an optical system for receiving signal light from a sample and a data processing system for reproducing the signal light as an observation sample image, wherein the optical system is configured to detect a sample observation position. The data processing means is provided with an observation position setting means for changing in a two-dimensional plane and a light detection means capable of recognizing a two-dimensional light intensity distribution of signal light obtained from the spot light irradiation region. Storage means for storing the two-dimensional light intensity distribution of the signal light obtained by the detection means in association with each light irradiation region on the observation sample;
Based on the two-dimensional light intensity distribution for each light irradiation region obtained by the light detecting means, the two-dimensional light intensity distribution reconstructed by the inverse calculation using the point spread function as a seed function is integrated to obtain 2 of the entire observation region. The present invention is characterized by comprising a calculation means for forming a three-dimensional light intensity distribution and a display means for displaying the light intensity distribution of the entire observation region formed by the calculation means as an image.

The invention described in claim 2 is the same as claim 1
In the optical imaging system described in the paragraph 1, the arithmetic means provided in the data processing means uses a point spread function for each optical signal intensity data forming the two-dimensional light intensity distribution for each light irradiation region obtained by the light detecting means. It is characterized in that the signal light intensity is corrected by applying a Wiener inverse filter to reconstruct a two-dimensional light intensity distribution.

The invention described in claim 3 is the same as claim 1
Alternatively, in the optical imaging system according to the second aspect, the pixel interval of the photodetector included in the optical system is set to be smaller than the cutoff spatial frequency of the point spread function determined by the optical system.

The invention described in claim 4 is the same as that of claim 1.
The optical imaging system according to any one of items 1 to 3, wherein an annular stop is provided in the optical path of the optical system.

The invention according to claim 5 is the same as claim 1.
The optical imaging system according to any one of items 1 to 4 is characterized in that the optical system is a confocal optical system, and the light detecting means is provided at a position conjugate with the real image plane of the observation sample.

Further, the invention according to claim 6 is the same as claim 5
In the optical imaging system described in the item [1], the optical system includes a pulse laser as a light source, and two-photon excitation of fluorescent molecules contained in the observation sample is possible.

According to a seventh aspect of the present invention, there is provided a data processing method for an optical image, which comprises a step of storing a two-dimensional light intensity distribution of signal light in association with each light irradiation region on an observation sample, and each of the steps. A step of reconstructing the two-dimensional light intensity distribution by an inverse operation using a point spread function as a seed function based on the two-dimensional light intensity distribution for each light irradiation region; And displaying the light intensity distribution of the entire observation area as an image.

The invention described in claim 8 is the invention according to claim 7.
In the optical image data processing method described in (1), the signal light intensity is corrected by applying a Wiener inverse filter based on a point spread function to each light signal intensity data forming the two-dimensional light intensity distribution for each light irradiation region. And reconstructing a two-dimensional light intensity distribution.

According to a ninth aspect of the present invention, there is provided a program for causing a computer to execute the program.
Alternatively, each step described in 8 is executed.

Further, the invention according to claim 10 is a computer-readable recording medium, characterized in that the program according to claim 9 is recorded.

[0022]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

(System Configuration) FIG. 1 is a diagram for explaining the configuration of the optical imaging system of the present invention. This optical imaging system includes an optical system 100 having a configuration used in a general scanning optical microscope and data. A processing system 120 and an observation sample 102 on a sample stage 101.
And the light emitted from the laser 106, which is a light source, is condensed by the lenses 107 and 108, and then the pinhole 109
To a desired spot diameter, a parallel beam is formed by the lens 110, the optical path is changed downward by the half mirror 112, and the observation sample 102 is converged by the objective lens 103.
Light up. The light source to be used is not particularly limited, but in addition to an ordinary laser light source, a laser having a pulse width of several hundred femtoseconds or less, and various high-luminance light sources such as a high pressure mercury lamp and a xenon arc lamp may be used.

The fluorescence or the like emitted from the observation sample 102 by this light irradiation is the objective lens 103 and the half mirror 11.
2 and the two-dimensional distribution of the light intensity is detected by the two-dimensional photodetector 105 such as CCD which is condensed by the lens 104. That is, in the conventional confocal laser scanning microscope shown in FIG. 7, only the light intensity of the optical signal from the observation sample is monitored by the photodetector, whereas in the optical imaging system of the present invention. , The light intensity distribution in the XY plane having a spatial spread corresponding to the light spot size on the observation sample is monitored. The two-dimensional photodetector 105 may be a photodiode array, a two-dimensional photomultiplier tube, or the like, in addition to the CCD. Further, for the reason described below, in the optical system of the optical imaging system of the present invention, the signal light having a spatial spread from the observation sample 102 is provided without providing a pinhole on the light entrance side of the two-dimensional photodetector 105. All are made to enter into the two-dimensional photodetector 105.

In the optical system 100, the sample stage 101 is moved in the XY direction.
A two-dimensional mapping data can be obtained by providing a driving mechanism (not shown) for making the movable in a plane, and performing observation while driving the sample stage 101 in the XY plane. The sample stage 101
In place of the above-described method for obtaining a two-dimensional mapping by moving in the XY plane, the spot light scanning device 111 is provided in the optical path of the irradiation light, and the driving of the spot light scanning device 111 is performed by the data processing system 120. Two-dimensional mapping may be obtained by changing the light irradiation point on the observation sample 102 by synchronizing with the observation point signal transmitted from.

As such a spot light scanning method, a stage using a piezo element as a drive mechanism is used, a galvano mirror inserted in the optical path of the irradiation light is driven, or it is arranged in the optical path. Various methods such as driving the mirror with a piezo element can be used. Further, if necessary, it is possible to optimize the spatial resolution by providing an annular stop at an appropriate optical path position such as between the lenses 107 and 108 in FIG.

The spatial distribution data of the light intensity detected by the two-dimensional photodetector 105 in this manner is sent to the data processing system 120 equipped with a computer and the fluorescence intensity distribution g _{x, y (s)} for each observation point is obtained. _{, T)} are sequentially recorded,
An observation image data matrix R _g corresponding to a two-dimensional mapping of a predetermined observation area is formed. Here, the fluorescence intensity distribution g
_{x, y (s, t)} means the light intensity at the coordinates (s, t) in the beam spot plane at the coordinates (x, y) of the observation point. The data analysis described below is executed at the time when the acquisition of the fluorescence intensity distribution data at all the observation points is completed, and the restored image of the entire sample is reconstructed based on the restored image data matrix R _f after deconvolution to display the image. The data matrix R _i is calculated to reproduce the two-dimensional image of the observation sample 102.

Although the optical imaging system configured as a reflection type optical system has been described with reference to FIG. 1, it may be configured as a transmission type optical imaging system as shown in FIG. In addition, each component of the optical system in FIG.
The same reference numerals as the components described in FIG. 1 are attached.

(Principle of Optical Imaging) FIG. 3 shows the present invention.
Of optical imaging adopted in the optical imaging system of
FIG. 3A is a diagram for explaining the principle, and FIG.
Two objects in the area ₁~ D_ThreeExist only apart
Optical signal strength obtained from each of these objects when
3B is a diagram for explaining the spatial distribution of FIG.
Detects the optical signal generated in this way and displays it as an image
It is a figure for demonstrating the process of reconstructing.

As shown in FIG. 3 (a), the signal intensity obtained from a light-irradiated object has a maximum at a position corresponding to the center of a spatially spread object and is asymptotic while vibrating toward both ends. There is also a "tail" portion having signal strength in a region where an object should not exist originally.
Then, FIG. 3 two objects spacing _(a 1) ~ FIG 3 _(a 3)
When the distance between the centers of these light intensities decreases with the distance between the objects, when the distances d ₁ , d ₂ , and d ₃ are gradually narrowed as shown in FIG. Occurs, and at the overlapped portion, interference of which degree changes depending on the object interval occurs.

In the conventional confocal scanning optical microscope, since only one light intensity information obtained by corresponding to one irradiation light spot is detected, the above-mentioned corresponding to an area where an observed object should not exist originally is detected. The light intensity in the "tail" area of the object is also detected as an optical signal from the object, which causes "blurring" of the contour of the microscope image and also includes important information for improving the spatial resolution of the microscope. The degree of interference at the overlapping portion of the "tail" of the optical signal that should have existed was not targeted for analysis at all.

In the optical imaging system of the present invention, the degree of interference at the overlapping portion of the "tails" of the optical signal is treated as an object of analysis. Therefore, the optical system of the optical imaging system of the present invention will be described with reference to FIG. As did
A pinhole is not provided on the light entrance side of the two-dimensional photodetector, and all the signal light having a spatial spread from the observation sample is
It is configured to be incident on the three-dimensional photodetector, and the analysis of the optical signal proceeds in the following procedure.

That is, when the sample is irradiated with light (FIG. 3 (b ₁ )), the observed image g (x, y) of the optical microscope obtained from the sample is a point image distribution under the condition that noise can be ignored. A device function called a function h (x, y) is a convolution of the true sample image f (x, y).
The observed image g (x, y) is given by (Equation 1) (FIG. 3 (b ₂ )).

[0034]

[Equation 1]

Therefore, the point image distribution function h (x, y) is used as a seed function to perform an inverse operation (deconvolution) on the observed image g (x, y) (FIG. 3 (b ₃ )) to obtain a true value. Estimate the sample image f (x, y) (FIG. 3 (b ₄ ))
Therefore, it is possible to improve the spatial resolution (resolution). Here, if the boundary condition is unclear, an error occurs due to the execution of the deconvolution operation. Therefore, in the optical imaging system of the present invention, spot light is used as the illumination light of the sample to clarify the boundary condition. It is supposed to irradiate a limited area.

(Specific Contents of Data Processing) A data processing method of an optical image in the optical imaging system of the present invention will be described below.

(1. Acquisition of Observation Image Data) When the observation sample is irradiated with laser light as a beam spot having a spatial spread, fluorescence, reflected light, or transmission from the observation sample at the light irradiation position (x, y). Light (depending on the configuration of the optical system) is detected by a two-dimensional photodetector having the number of pixels σ · τ (horizontal σ pixel, vertical τ pixel), and image data g _{x, y} (s,
t) (where 1 ≦ s ≦ σ and 1 ≦ t ≦ τ). Following this, the irradiation position of the spot light is moved in the x direction by a fixed minute distance, and the next observation image data g
Record _{x + 1, y} (s, t). Here, the resolution is improved by setting the step size (sample feeding) to a minute distance equal to or smaller than the spot size (about a fraction).

This procedure is repeated N times (where N = (transverse direction ξ
Times) × (vertical direction ψ times)) Iteratively scans the entire surface of the target observation sample to obtain the observation image data matrix R _g (ξ · ψ matrix) expressed by (Equation 2). .

[0039]

[Equation 2]

(However, 1≤x≤ξ, 1≤y≤ψ, 1≤s
≦ σ, 1 ≦ t ≦ τ) Image processing using a computer is executed on the observation image data matrix R _g thus obtained.

[0041] (2. Recovery calculation of the image data matrix) observed image data matrix each image data included in the R _g g
_x, with respect to _{y (s,} t), perform the deconvolution that the point spread function as a seed function, restored image data f _x the true value has been _recovered, calculates _{y (s,} t), ₍ The restored image data matrix R _f corresponding to the observed image data matrix R _g shown in Expression 3) is obtained.

[0042]

[Equation 3]

(However, 1≤x≤ξ, 1≤y≤ψ, 1≤s
≦ sigma, 1 according to ≦ t ≦ τ) This method, image data _{g x,} y (s, t) from the restored image data _{f x,} y (s, spot light when calculating the t) with deconvolution calculation Since the irradiation range of 1 can be accurately given as a boundary condition, calculation with less error becomes possible. Here, the above-described deconvolution is executed on either the Fourier plane or the real image plane, and as an example of the algorithm, observed image data g (x, y)
Fourier transform G (u, v) of the point spread function h
A method of inverse transform by dividing by Fourier transform H (u, v) of (x, y), Wiener inverse filter W (u,
v), Harris's method, and non-linear optimization method.

Among these deconvolution algorithms, the observed image data g (x, y) is Fourier transformed G (u, v) and this is Fourier transformed H (u, v) of the point spread function h (x, y). ), The inverse transformation method uses the convolution theorem to calculate F (u, v) (= G (u, v)
/ H (u, v)) and inverse Fourier transform of this F (u, v) to obtain the restored image data f (x, y). In this method, when H (u, v) has a value close to zero, F (u, v) has a very large value.
When G (u, v) contains noise, the noise is amplified and it is difficult to obtain satisfactory restored image data f (x, y).

Winner inverse filter W (u, v)
In the method using, the filter function is given by (Equation 4).

[0046]

[Equation 4]

Here, H ^t (u, v) is the transposed matrix of H (u, v), and c is the power spectrum ratio of signal and noise.

Then, based on this W (u, v), F
(U, v) = G (u, v) · W (u, v) F
(U, v) is obtained, and the restored image data f (x, y) is obtained by inverse Fourier transforming this F (u, v).

In the optical system of the optical imaging system of the present invention, the range of the image observed for irradiating the sample with the spot light is limited to the spot light irradiation region. Therefore, the constraint condition on the spread of the object in the Harris hyperimage method is satisfied, and deconvolution by the Harris method is possible. In the Harris super-resolution method, the observed image is sampled more finely than the cutoff frequency of the measurement system, and after interpolation with the sink function, inverse matrix calculation is performed to widen the band,
By this processing, the high frequency component deteriorated by the optical system can be recovered and the resolution is improved.

In deconvolution using the non-linear optimization method, the true luminance value is iteratively modified so that the square error between the true luminance value and the value obtained by convolving the point spread function and the measured value is minimized. Generally, since the brightness information of an image does not have a negative value, the true brightness value is obtained by using an optimization method such as the steepest descent method or the conjugate gradient method with this as a constraint condition. According to this method, since there is no divergence of the solution due to noise, there is a great merit that the most appropriate brightness value can be obtained.

(3. Reconstruction of Restoration Image) Using each restoration image data f _{x, y} (s, t) included in the restoration image data matrix R _f thus obtained, (Equation 5) Reconstruct the recovered image R _{i of the} entire sample as noted.

[0052]

[Equation 5]

(However, 1 ≦ m ≦ μ, 1 ≦ n ≦ ν, i
_{m, n} are restored image R _i of coordinates (m, n) the luminance value of the pixels of the two-dimensional light beam detector corresponding to) restore the image data matrix R _f overall recovery image of the sample from R _i
As a method of reconstructing, the recovery image data f _{x, y} (s, t) included in the recovery image data matrix R _f is, for example,
Method using the central value of
There is a method of obtaining a restored image R _{i of the} entire sample by shifting _{x, y} (s, t) by the number of pixels (Δσ, Δτ) corresponding to the scanning steps in the x direction and the y direction, and superimposing and averaging. .

[0054] Among them, the restored image data f _x included in the restored image data matrix R _f, according to the method of using the center value of _{y (s,} t), first, in the recovery image data matrix R _f, xi] = μ, ψ = ν, x = m, y = n Distant, each restored image data f included in the restored image data matrix _{R f}
The luminance value of the pixel at the center of _{m, n} (s, t) is given as i _{m, n} as in (Equation 6).

[0055]

[Equation 6]

Using this i _{m, n} , a restored image R _{i of the} entire sample is obtained as in (Equation 7).

[0057]

[Equation 7]

On the other hand, the restored image data f _{x, y} (s, t)
Is the number of pixels corresponding to the scanning steps in the x and y directions (Δ
sigma, according to the method of obtaining a restored image R _i of the entire sample by averaging overlapped by shifting one by .DELTA..tau), restored image R _i of the entire sample is defined by equation (8).

[0059]

[Equation 8]

(However, 1≤m≤μ, 1≤n≤ν, im _{, n}
The luminance value of the pixel located at the coordinate (m, n) of the restored image R _i of the whole sample) where the number of pixels of the restored image R _i of the entire sample (mu (lateral direction), [nu (vertical direction)) is , Μ = Δσ · (ξ−1), respectively
+ Σ and ν = Δτ · (ψ-1) + τ.
Here, Δσ is the number of pixels corresponding to one horizontal scanning step, ξ is the number of horizontal scanning, and σ is each restored image data f.
_{x, y} (s, t) horizontal pixel count, Δτ is 1 vertical pixel
The number of pixels corresponding to one scanning step, ψ is the number of scans in the vertical direction, and τ is each restored image data f _{x, y} (s,
t) is the number of pixels in the vertical direction.

[0061] The following is a luminance value of the pixels of the two-dimensional light beam detector corresponding to the coordinates (m, n) of the restored image R _i i
A specific calculation procedure for obtaining _{m and n} is σ / Δσ =
Ns and τ / Δτ = Nt are integers, and Δσ and Δτ
Both are integers (however, σ and τ are each restored image data f
_{The case of x, y} (s, t) horizontal and vertical pixels) will be described as an example.

First, as shown in FIG. 1, the number of pixels corresponding to the step size during scanning (horizontal direction Δσ, vertical direction Δ).
τ) Place the restored images on top of each other while shifting them. At this time, when Ns sheets are overlapped in the lateral direction with reference to the restored image data f _{x, y} (s, t), for example, f _{x, y} (s, t)
The average value of t) with the other Ns images is as in (Equation 9),

[0063]

[Equation 9]

In the same manner as above, N is also set in the vertical direction.
Taking into consideration the average when t images are overlapped by shifting by the scanning step size, (Equation 10) is obtained as a whole.

[0065]

[Equation 10]

In this way, the restored image R _{i of the} entire sample is obtained.
The brightness value at each coordinate of is given by (Equation 11),

[0067]

[Equation 11]

Ns ≦ m ≦ μ−Ns and Nt ≦ n ≦ ν
-In Nt,

[0069]

[Equation 12]

(However, roundup (a / b) means obtaining the quotient of a / b by rounding up, and mod (a / b) means obtaining the remainder of a / b).

Further, if (Equation 12) is expanded so as to be satisfied in all image regions (1≤m≤μ and 1≤n≤ν) including the edge portion, (Equation 13) is obtained.

[0072]

[Equation 13]

(However, min (a, b, c) is the minimum value of a, b, c, and 1 ≦ s <μ, 1 ≦ t ≦ ν, 1 ≦ x ≦ ξ and 1
F _{x, y} (s, t) = 0 except for ≤ y ≤ ψ)

(Embodiment) An embodiment of the optical imaging system of the present invention will be described below.

(Embodiment 1) FIGS. 4 to 6 are views obtained by using computer simulation in order to explain the appearance of an observed image obtained by using the optical imaging system of the present invention. FIG. 6 is a diagram for explaining an image in each calculation process from obtaining an observation image to an image, and FIG. 5 is a diagram for comparing each of a sample image, an observation image, and a restored image, and FIG. FIG. 6 is a diagram for explaining a comparison of resolutions of an observed image and a restored image.

In FIG. 4, the sample image (a), the image (b) obtained by multiplying the sample image, the image (c) obtained by performing the convolution operation of the point spread function (PSF), and the convolution. Observation image (d) obtained by adding images, spot light intensity distribution image (e), point spread function (PSF)
(F) and the image (g) of the noise component are shown,
In addition, a line profile of the signal intensity obtained from each of these images is also shown.

The sample image of FIG. 4 (a) is an original image of the sample being observed, and here, two square points are arranged at a predetermined distance. The intensity distribution of spot light (FIG. 4 (e)) is applied to this sample image to obtain the fluorescence (or scattered light) intensity distribution from the sample.
(B) is obtained.

Then, the convolution operation is executed by the PSF shown in FIG. 4 (f) to obtain the convolved image of FIG. 4 (c). As can be seen from this convolved image, convolution of the PSF causes blurring around the point that is the observation sample, and some of the two points that should have been originally separated overlap and meet. Further, in the observed image (FIG. 4 (d)), the noise component is superimposed, so that the entire image becomes dull.

FIG. 5 shows a sample image (FIG. 5 (a)), an observed image (FIG. 5 (b)), and a restored image (FIG. 5 (c)).
And the line profiles of signal strength corresponding to these images are shown. The restored image is an image restored using the conjugate gradient method. As shown in FIG. 5B, in the observed image, in addition to the dull image containing a lot of noise components, the images of two originally separated square points are fused. In the restored image, these points are completely separated and reproduced as an image extremely close to the sample image, and the signal intensity distribution is close to the original distribution.

FIG. 6 is a diagram for explaining the result of the comparison of the resolutions of the observed image and the restored image when the interval between the two points of the observed sample is changed. This comparison is shown in FIG.
As shown in (a), the ratio of the signal intensity (luminance) P at the position corresponding to each point and the signal intensity (luminance) V at the position corresponding to the center between the two points (sample image reproduction ratio = V / P). Therefore, if the sample image is completely restored in the restored image, the brightness V at the position where the observation sample does not exist becomes zero, and the sample image reproduction ratio also becomes zero. The sample image reproduction ratio defined in this way has a noise of 0.1%.
(FIG. 6B), 1.0% (FIG. 6C), 10. %
The simulation is performed under each of the conditions (FIG. 6D).

As can be seen from this result, the sample image reproduction ratio is zero when the sample interval is 100 nm under any noise condition, and the sample image is completely restored if the sample interval is more than this interval. Is possible. Further, it is concluded from the restored image that the resolution, which is the limit at which two points can be recognized separately, is defined by the sample interval that gives a sample image reproduction ratio of 0.8 or less, and is a resolution of about 40 nm. On the other hand, the resolution of the observed image is determined to be about 90 nm, which means that the resolution is more than doubled.

In actual measurement, an image corresponding to the above-mentioned simulation result is obtained for each sample scan, but noise is reduced by setting the sample scan step sufficiently finely and integrating these images. It is possible to improve the spatial resolution.

(Embodiment 2) A data processing procedure of an optical image in the optical imaging system of the present invention based on a two-photon excitation confocal microscope will be described below.

An infrared pulse laser having a pulse width of about 100 femtoseconds is used as a light source, and the laser light has a magnification of 1
An objective lens of No. 00 condensed the spot light having a diameter of about 200 nm and irradiated it on the sample.

This irradiation of light excites the fluorescent molecules in the observation sample to generate fluorescence, and the fluorescence is generated through a collector lens with a magnification of 10 to obtain a single pixel size of 6.7 μm × 6.7 μ.
It was detected by a two-dimensional photodetector (cooled CCD) consisting of m square pixels. In such an optical system, the spot size of the fluorescence on the CCD is about 200 μm in diameter, and the light spot on the CCD is projected in the range of about 30 pixels × 30 pixels. Therefore, the fluorescence luminance in the range of 64 pixels × 64 pixels was recorded with 16-bit accuracy.

Here, the fluorescence image matrix g _{x, y} (s, t) of the fluorescent molecule excited by the spot light is defined as in (Equation 14).

[0087]

[Equation 14]

However, each element of the matrix means the brightness value of each pixel, s and t mean the coordinates of the pixel in the fluorescence image, and 0 ≦ s ≦ 64 and 0 ≦ t ≦ 64. Moreover, x and y mean the coordinates of the irradiation position of the spot light.

Fluorescence image matrix g recorded by the cooled CCD
_{x, y} (s, t) was stored in the hard disk via the computer. 50 nm for two-dimensional mapping
The spot light on the sample was sequentially moved with the step size of, and a range of about 5 μm square was scanned. As a result, 100 × 100 sheets (10,000 sheets in total) are recorded in the hard disk as captured images, and the observed image data matrix R _g is _calculated based on these images as in (Equation 15).

[0090]

[Equation 15]

However, 0 ≦ x ≦ 100, 0 ≦ y ≦ 100
Here, (x, y) is the coordinates of the irradiation position of the spot light on the sample. Observation image data matrix R obtained in this way
Restored image data f _{x, y} (s, t) by executing deconvolution calculation by a non-linear optimization method for each of the elements of _g (fluorescence image matrix g _{x, y} (s, t)) Is calculated, and the restored image data matrix R _f is calculated as in (Expression 16).

[0092]

[Equation 16]

Finally, the restored image R _{i of the} entire sample is calculated based on this restored image data matrix R _f . Specifically, the restored image data f that is each element of the restored image data matrix
The recovered image R _i of the sample was reconstructed by shifting _{x, y} (s, t) by the number of pixels corresponding to the step size and calculating the average. The theoretical minimum step size in this case is determined by the size of the pixel of the two-dimensional photodetector and the optical magnification, and in the case of the present embodiment is about 6.7 nm.

(Embodiment 3) A data processing procedure of an optical image in the optical imaging system of the present invention based on a transmission type confocal microscope will be described below.

The light emitted from the light source was condensed as a light spot with a diameter of about 200 nm through a pinhole and a condenser lens, and was irradiated onto the sample. CC which is provided with a square pixel having a pixel size of 6 μm for transmitting light from the sample corresponding to this light spot through an objective lens and a collector lens.
Projected onto D. The spot size on the CCD is about 200 μm, which corresponds to about 34 pixels × 34 pixels. Therefore, the range of 64 × 64 pixels is defined by (Equation 17)
Was recorded as an observed image matrix g _{x, y} (s, t).

[0096]

[Equation 17]

However, each element of the matrix means the luminance value of each pixel, and in this case, it is the intensity value of the transmitted light. Further, s and t mean pixel coordinates (0 ≦ s ≦ 64, 0 ≦ t ≦ 6.
4) and (x, y) mean the coordinates of the spot light irradiation position on the sample.

Observation image matrix g read from CCD
_{x, y} (s, t) is sent to the dedicated CPU, and the deconvolution calculation for calculating the restored image from the observed image and the restored image R _i of the entire sample based on the calculated restored image The calculation of the brightness value I _{x, y at} the coordinates (x, y) is executed. Note that I _{x, y} is the restored image data f _x,
Using the value of the central part of _y (s, t), I _{x, y} = f _{x, y}
The value of I _{x, y} calculated from (32, 32) was recorded on the hard disk, and the scanning of the light spot was executed.

For the scanning of the spot light on the sample, a sample stage equipped with a piezo controller capable of being driven with an accuracy of 1 to 5 nm in the x and y directions was used, and a 5 μm square area was scanned with a step size of 10 nm.
Therefore, 500 points in x direction x 500 points in y direction (total 2500 points
An observation image at (00 point) was obtained, and the luminance value of the restored image was obtained based on this to reconstruct the restored image R _{i of the} entire sample.
Note that the theoretical spatial resolution in the case of performing image reconstruction by the Jacobian method in this embodiment is about 50 nm.

[0100]

As described above, according to the present invention,
Based on the two-dimensional light intensity distribution for each light irradiation area obtained by the two-dimensional photodetector, the two-dimensional light intensity distribution reconstructed by the inverse calculation using the point spread function as a seed function is integrated to Since the two-dimensional light intensity distribution is configured, an optical imaging system having a spatial resolution of 50 nm level and an optical image data processing method capable of easily and stably performing scanning optical microscope observation with high spatial resolution are provided. It becomes possible to provide.

[Brief description of drawings]

FIG. 1 is a diagram for explaining a configuration of a reflective optical imaging system of the present invention.

FIG. 2 is a diagram for explaining the configuration of a transmission type optical imaging system of the present invention.

FIG. 3 is a diagram for explaining the principle of optical imaging adopted in the optical imaging system of the present invention, in which (a) is a case where two objects are present in the irradiation light spot area at a distance d _{1 to} d _3. It is a diagram for explaining the spatial distribution of the optical signal intensity obtained from each of these objects,
(B) is a figure for demonstrating the process which detects an optical signal and is reconstructed as an image.

FIG. 4 is a diagram for explaining an image in each calculation process from a sample image to an observation image.

FIG. 5 is a diagram for comparing a sample image, an observed image, and a restored image.

FIG. 6 is a diagram for explaining comparison of resolutions of an observed image and a restored image.

FIG. 7 is a diagram for explaining a configuration example of a conventional confocal laser scanning microscope system.

[Explanation of symbols]

100 optical system 101 sample stage 102 observation sample 103 Objective lens 104, 107, 108, 110 lenses 105 two-dimensional photodetector 106 laser 109 pinhole 111 spot light scanning device 112 half mirror 120 data processing system

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Masahiro Yamada             1-1-1 Higashi 1-1-1 Tsukuba City, Ibaraki Prefecture             Inside the Tsukuba Center, National Institute of Advanced Industrial Science and Technology F term (reference) 2F065 AA51 DD04 FF04 GG04 HH04                       JJ26 LL00 LL13 LL28 LL30                       MM11 PP24 QQ16 QQ18 QQ24                       QQ31                 2H052 AA08 AB24 AC04 AC15 AC34                       AD16 AD32 AD35 AF06 AF14                       AF21 AF25

Claims (10)

[Claims] 1. An optical system for irradiating an observation sample with light emitted from a light source as spot light to receive signal light from the observation sample, and data for reproducing the signal light as an observation sample image. An optical imaging system including a processing system, wherein the optical system includes an observation position setting unit for changing a sample observation position in a two-dimensional plane, and signal light obtained from an irradiation region of the spot light. And a light detecting unit capable of recognizing a two-dimensional light intensity distribution, wherein the data processing system is configured to detect the signal light obtained by the light detecting unit.
Storage means for storing a three-dimensional light intensity distribution corresponding to each light irradiation area on the observation sample, and a point image distribution based on the two-dimensional light intensity distribution for each light irradiation area obtained by the light detection means A calculation means for integrating the two-dimensional light intensity distributions reconstructed by the inverse calculation using the function as a seed function to form a two-dimensional light intensity distribution of the entire observation area; and a total observation area formed by the calculation means. An optical imaging system comprising: a display unit for displaying the light intensity distribution as an image. 2. The calculation means included in the data processing means is a Wiener inverse filter based on a point spread function for each optical signal intensity data constituting the two-dimensional light intensity distribution for each light irradiation region obtained by the light detection means. The optical imaging system according to claim 1, wherein the signal light intensity is corrected by multiplying by to reconstruct a two-dimensional light intensity distribution. 3. The pixel interval of the light detection means provided in the optical system is set to be smaller than the cut-off spatial frequency of the point spread function determined by the optical system. Optical imaging system. 4. The optical imaging system according to claim 1, further comprising a ring stop in the optical path of the optical system. 5. The optical system according to claim 1, wherein the optical system is a confocal optical system, and the light detecting means is provided at a position conjugate with the real image plane of the observation sample. Optical imaging system. 6. The optical imaging system according to claim 5, wherein the optical system is provided with a pulse laser as a light source and is capable of two-photon excitation of fluorescent molecules contained in the observation sample. 7. A data processing method for an optical image, the step of storing a two-dimensional light intensity distribution of signal light in association with each light irradiation region on an observation sample, and two-dimensional for each light irradiation region. A step of reconstructing a two-dimensional light intensity distribution by an inverse operation using a point spread function as a seed function based on the light intensity distribution, and integrating the reconstructed two-dimensional light intensity distributions And a step of displaying the distribution as an image. 8. The signal light intensity is corrected by applying a least-squares filter by a point spread function to each optical signal intensity data forming a two-dimensional light intensity distribution for each light irradiation region,
The optical image data processing method according to claim 7, further comprising the step of reconstructing a three-dimensional light intensity distribution. 9. A program for causing a computer to execute the steps according to claim 7 or 8. 10. A computer-readable recording medium in which the program according to claim 9 is recorded.