JPH0442116A - Image data correcting method for scanning type microscope - Google Patents

Abstract

PURPOSE:To prevent an output image from being distorted owing to irregularity of the scanning of illumination light by measuring the current actual scanning position of the illumination light every time image data is sampled and replacing the obtained image data with data regarding an ideal sampling point. CONSTITUTION:Every time the image data is sampled, the current actual scanning position A(m,n) (m, n: sampling order in main scanning and subscanning directions, and m, n=1, 2, 3...) of the illumination light and the image data B(m,n) obtained as to an ideal sampling point AH(m,n) is substituted with data regarding the ideal sampling point closest to the actual illumination light scanning position A(m,n). Thus, the image data is substituted and then each digital image data obtained by sampling the output of a photodetector can be utilized as the data on the ideal sampling point closest to the actual illumination light scanning position whatever irregularity is generated in the illumination light scanning.

Description

DETAILED DESCRIPTION OF THE INVENTION (Industrial Application Field) The present invention relates to a method for correcting image data of a scanning microscope, and more particularly, to an image data correction method capable of preventing distortion of an output image due to uneven scanning of illumination light. It is about the method.

(Prior Art) Conventionally, illumination light is converged into a minute light spot, and this light spot is scanned two-dimensionally on a sample, and at that time, the light that has passed through the sample or the light that has been reflected there is detected by a photodetector. Optical scanning microscopes are known that detect electrical signals that carry an enlarged image of a sample. In addition, Japanese Patent Application Publication No. 6
2-217218 discloses an example of this optical scanning microscope.

In many cases, this optical scanning microscope periodically samples the output of a photodetector to obtain digital image data. This sampling is performed with a strictly defined period so that the brightness of each of the many sampling points arranged in a grid on the sample can be determined.

(Problem to be solved by the invention) The above optical scanning microscope? In an Ik mirror, especially when the scanning speed is increased, blurring of the illumination light scanning mechanism and fluctuations in the scanning speed are likely to occur. If this unevenness occurs in illumination light scanning, even if the sampling period is strictly defined as described above, the obtained digital image data will be distorted relative to each other on the sample (that is, not aligned in a grid pattern). It becomes something that carries point information. If a microscopic image is output based on such image data, the image will naturally be distorted.

The present invention has been made in view of the above-mentioned circumstances, and an object of the present invention is to provide an image data correction method for a scanning microscope that can prevent distortion of output images due to uneven scanning of illumination light. It is something.

(Means and effects for solving the problem) The method for correcting image data of a scanning microscope according to the present invention scans a sample in the main and sub-scanning directions with illumination light as described above, and uses the light from the sample at that time. In a scanning microscope that detects with a photodetector and periodically samples the output of this photodetector to obtain digital image data, each time the image data is sampled, the actual illumination light scanning position A at that time is determined. (m, n) [m, n indicate the sampling order in the main and sub-scanning directions, respectively, m, n=
1.2.3, . A
(m, n) is characterized in that it is replaced with the ideal sampling point closest to (m, n).

If the image data is replaced as described above, no matter how uneven the illumination light scanning is, each digital image data obtained by sampling the photodetector output will be most closely aligned with the actual illumination light scanning position. This will be used as data about ideal sampling points that are close to each other.

Note that when such image data replacement is performed, a plurality of image data may be obtained for one sampling point, or conversely, image data may be lost for a certain sampling point. Therefore, in the former case, for example, the average value of a plurality of image data may be used as the final image data for that sampling point. Also, in the latter case,
For example, image data regarding sampling points above, below, to the left, and to the right of a sampling point where image data is missing may be averaged, and the average value may be used as data at the data missing sampling point.

(Example) Hereinafter, the present invention will be described in detail based on an example shown in the drawings.

FIG. 1 shows an example of a transmission type confocal scanning microscope for carrying out the method of the present invention, and FIG. 2 shows in detail the scanning mechanism used therein. As shown in FIG. 1, a laser diode 5 emits illumination light 11.
is integrally held on the movable table 15. The moving stage 15 also includes a collimator lens 16 and an objective lens 1.
A light transmitting optical system 18 consisting of 7 and a light receiving optical system 21 consisting of an objective lens 19 and a condensing lens 20 are fixed with their optical axes aligned with each other.

A sample stage 22, which is separate from the moving stage 15, is arranged between these optical systems 18.2I. A photodetector 9 is fixed to the movable table 15 below the light receiving optical system 21. As this photodetector, for example, a photodiode or the like is used. Further, in front of the photodetector 9 (upper side in the figure), a pinhole plate 8 having a pinhole 8a is disposed.

Laser light (illumination light) emitted from laser diode 5
11 is made into parallel light by the weight meter lens 16, then condensed by the objective lens 17, and then directed to the sample stage 22.
The light converges into a minute light spot P on the sample 23 placed on the surface (on the surface or inside it).

The luminous flux of each transmitted light 11' that has passed through the sample 23 is made into parallel light by the objective lens I9 of the light receiving optical system 2I, and then condensed by the condensing lens 20 to form a point image Q. This point image Q is detected by the photodetector 9. This photodetector 9 outputs a signal S indicating the degree of budding of each part of the sample 23 irradiated with the light spot P.

Next, regarding the two-dimensional scanning of the light point P of the illumination light 11, the second
This will be explained with reference to the drawings. The movable table 15 is supported by the pedestal 32 so as to be movable in the direction of arrow X.

That is, one end portions of two guide rods 40, 40 are fixed to the pedestal 32, and these guide rods 40, 40 are loosely fitted into two guide holes 4L 41 provided in the movable table 15. A laminated piezo element 33 is interposed between the movable table 15 and the pedestal 32. This laminated piezo element 33 receives driving power from a piezo element drive circuit 34 and causes the movable table 15 to reciprocate at high speed in the direction of arrow X.

On the other hand, the sample stand 22 is mounted on the stand 32 with 1. On the other hand, it is supported so as to be movable in the direction of arrow Y, which is perpendicular to the direction of arrow X. That is, one ends of two guide rods 45.45 are fixed to the pedestal 32, and these guide rods 45.45 are loosely fitted into the two guide holes 4B, 4B provided in the sample stage 22. . A laminated piezo element 47 is interposed between the sample stage 22 and the pedestal 32. This laminated piezo element 47 receives drive power from a piezo element drive circuit 48 and moves the sample stage 22 back and forth in the direction of arrow Y at high speed. Thereby, the sample stage 22 is moved relative to the moving stage 15, and the light spot P sub-scans the sample 23 in the Y direction perpendicular to the main scanning direction X.

By scanning the light spot P two-dimensionally over the sample 23 as described above, an analog signal S carrying a two-dimensional image of the sample 23 is obtained. This signal S is periodically sampled and made into pixel-divided digital image data. Generation of this image data will be explained in detail later.

Note that a synchronization signal is inputted to the piezo element drive circuits 34 and 48 from a control circuit (not shown), whereby main and sub-scanning of the light spot P is synchronized.

Next, signal processing will be explained. As shown in FIG. 3, the output S of the photodetector 9 is amplified by an amplifier 50 and then converted to digital image data Sd by an A/D converter 51. A/D in this A/D converter 51
The sampling frequency of the D conversion and the timing of sampling are determined by the sampling clock S1 input from the sampling clock generation circuit 52.

On the other hand, as shown in FIG. 1, the mount 32 includes a displacement sensor 53 that detects the displacement of the moving table 15 in the X direction (displacement in the main scanning direction), and the displacement sensor 53 that detects the displacement of the sample table 22 in the Y direction (displacement in the sub scanning direction). A displacement sensor 54 is attached to detect the displacement in the direction. The analog outputs Sx and Sy of these displacement sensors 53 and 54 are amplified by amplifiers 55 and 56, respectively, and sent to the A/D converter 51 as displacement signals indicating the displacement of the illumination light spot P in the main scanning direction and the sub-scanning direction. is input.

The A/D converter 51 receives the displacement signals Sx and Sy at the same timing as the sampling of the output S of the photodetector 9.
to sample. The digital displacement signal Dx%Dy and digital image data Sd thus obtained are first stored in the memory 58 through the calculation section 57.

2. If the main and sub-scans of the illumination light spot P are completely ideal and no scanning unevenness occurs, the sampling of the photodetector output S is such that the illumination light spot P is on the sample 23. This is done every time ideal sampling points (indicated by black circles in FIG. 4) arranged in a grid are passed. In other words, the digital image data Sd consists of each ideal sampling point AH (m, n) [
m and n indicate the sampling order in the main and sub-scanning directions, respectively;
m%.

Image data B for =1, 2, 3,...]
It becomes a set of (m, n).

If there is unevenness in the main and sub-scanning of the illumination light point P, each image data B (m, n) is not necessarily about the ideal sampling point AM (m, n), but all images The data B (m, n) is temporarily associated with the ideal sampling point AH (m, n) and stored in the -memory 58 at each address corresponding to the values of m and n, respectively.

On the other hand, the digital displacement signal D x s DY is the light point P when each image data B (m, n) is sampled.
This shows the displacement in the main scanning direction and the displacement in the sub-scanning direction.

Therefore, the calculation unit 57 calculates the actual scanning position A' (m) of the light spot P when each image data B (m, n) is sampled, based on these both signals Dx and Dy.

Find n). The actual scanning position A' thus obtained
Information indicating (m, n) is stored in the memory 58 at each address associated with the values of m and n, respectively.

Next, a description will be given of correction of the image data B (m, n) in order to deal with scanning unevenness of the illumination light spot P. First, the calculation unit 57 reads the actual scanning position A' from the memory 58.
(m, n) are read out sequentially, and each scanning position A' (m
, n), subtract the position of the light point P at the start of scanning to find the absolute (that is, the DC component of the displacement) actual scanning position A (m, n) in the illumination light scanning system. .

The calculation unit 57 stores ideal sampling points AH (m, n) of chain saws m and n in its internal memory,
This ideal sampling point AM (m.

n) and the above actual scanning position A (m, n) deviation E
Find (m, n). This deviation E (m, n) is determined in the main scanning direction and the sub-scanning direction. In other words, as shown in FIG. 4, for example, ideal sampling points AH (1°n), (2, n), (3
, n), . Each point A (1, n) whose scanning position is indicated by a white circle, (
2, n), (3, n), ...... - For example, the deviation E (5, n) is X (5, n) in the main scanning direction and X (5, n) in the sub scanning direction. is Y (5, n
).

Next, based on the deviation E (m, n), the calculation unit 57 generates an array argument to be corrected indicating which ideal sampling point AH the actual scanning position A (m, n) is closest to. seek. That is, the above deviation E (5, n)
To explain with an example, the ideal sampling point AH(m, n
), that is, the 1-pixel interval, is L, then the main scanning direction array argument k to be corrected and the subscanning direction array argument to be corrected are determined as follows. Note that such integer conversion may be performed, for example, by rounding off the value to the first decimal place.

In the example of A(5,n) in FIG. 4, the array arguments obtained in the above manner are k-0 and 1-+1, respectively.

In other words, image data B (5, n) is shifted by 0 pixels in the main scanning direction (that is, no movement) and by +1 pixel in the sub-scanning direction from the ideal sampling point AH (5, n).
It is determined that the image data should be replaced as the image data of the ideal sampling point A)l (5, n+1).

The calculation unit 57 calculates all the image data B as described above.
Find the array argument k, 1 to be corrected for (m, n), and add image data B according to 1 to the array argument.
Replace (m, n). Image data B after this replacement
' (m, n) are stored in the memory 58 shown in FIG. 3, read out from there as appropriate, and converted into analog data in the D/A converter 59, for example, in an image reproducing device such as a CRT display device or an optical scanning recording device. 60 for reproduction of the microscopic image of the sample 23.

Note that the timing of writing and reading image data into the memory 58 and the timing of D/A conversion in the D/A converter 59 are defined by the aforementioned sampling clock S1.

If the image data B (m, n) is replaced as described above, no matter what unevenness occurs in illumination light scanning, the replaced image data B' (m, n) will match the actual illumination. This will be utilized as data about the ideal sampling point closest to the optical scanning position. Therefore, this image data B' (m.

The microscope image reproduced based on n) can be free from distortion by eliminating the influence of the illumination light scanning unevenness.

In the example of FIG. 4, for example, if the (n-1)th main scan is performed along the dashed line, the array argument kl to be corrected for image data B(5, n-1) is
! hani-0,1-0, and this image data B(5,n
-1) as is, the ideal sampling point AH(5, n
1). Therefore, if things continue as they are, the image data B for the ideal sampling point A)l (5, n) will be missing.

Therefore, in such a case, for example, ideal sampling points A above, below, left and right of ideal sampling point AH (5, n)
) Each image data B' (4゜n), B' related to I
(6,n) ,B' (5,n+1) ,B'(
5, n-1) and use the average value as image data regarding the ideal sampling point AH (5, n). Furthermore, if image data B° does not exist with respect to the ideal sampling points AH on the upper, lower, left, and right sides, for example, the average value of all the image data B' (m, n) after replacement is calculated as the ideal sampling point An (5 , n).

Moreover, when image data B (m, n) is replaced, contrary to the above, one ideal sampling point A) I (
A plurality of image data B'(m, n) may be obtained for m, n). In such a case, for example, the plurality of image data B' (m.

The average value of n) may be determined and the average value may be finally used as the image data for that sampling point.

Note that in the above example, the array argument to be corrected is
J is calculated each time illumination light scanning is performed, but if the reproducibility is high, the array argument kS1 to be corrected is calculated only once in advance and stored in memory. They may be read out and used every time the illumination light is scanned.

Further, the method of the above embodiment was applied to a transmission type scanning microscope, but the method of the present invention is not limited to such a scanning microscope, but can be applied to a reflection type scanning microscope or a scanning fluorescence microscope. It is similarly applicable to .

Furthermore, the method of the present invention is applicable not only to a scanning microscope having the illumination light scanning mechanism as described above, but also to a scanning microscope in which the illumination light sub-scanning is also performed by moving the movable stage 15, or a scanning microscope arranged outside the movable stage 15. Other types of illumination, such as scanning microscopes in which illumination light is guided from an illumination light source to a light transmission optical system via an optical fiber, and scanning microscopes in which illumination light scanning is performed by deflecting the illumination light beam with an optical deflector. The present invention can be similarly applied to a scanning microscope equipped with an optical scanning mechanism.

Further, in the above embodiment, image data replacement correction is performed only in the main scanning direction and sub-scanning direction of illumination light, but image data replacement correction is also performed in the optical axis direction (arrow Z direction). Of course, it is also possible.

(Effects of the Invention) As explained in detail above, in the image data correction method for a scanning microscope according to the present invention, each time image data is sampled, the actual illumination light scanning position at that time is measured, and each ideal Since the image data obtained regarding the sampling point is replaced with that regarding the ideal sampling point closest to the actual illumination light scanning position, even if the illumination light scanning is uneven, each digital image obtained The image data will be utilized as data about the ideal sampling point closest to the actual illumination light scanning position. Therefore, according to this method, it is possible to eliminate the influence of illumination light scanning unevenness and reproduce a distortion-free microscope image.

[Brief explanation of drawings]

FIG. 1 is a schematic front view showing an example of a scanning microscope for carrying out the method of the present invention, FIG. 2 is a perspective view showing main parts of the scanning microscope, and FIG. FIG. 4, a block diagram showing an electric circuit, is an explanatory diagram for explaining the method of the present invention. 5... Laser diode 9... Photodetector 11...
- Illumination light 11°...Transmitted light 15-°Movement table 16...Collimator lens l7.
19... Objective lens 18... Light transmission optical system 20.
...Condensing lens 21... Light receiving optical system 22...
・Sample stand 23... Sample 32... Mount
33.47...Laminated piezo element 34.4
8... Piezo element drive circuit 50.55.56...Amplifier 51...A/D converter 52...Sampling clock generation circuit 53.5
4...Displacement sensor 57...Calculating unit 58...Memory 59...D/A converter 60...Image reproducing device Fig.

Claims (1)

[Claims] A sample is scanned in the main and sub-scanning directions with illumination light, the light from the sample at that time is detected by a photodetector, and the output of this photodetector is periodically sampled to generate digital data. In a scanning microscope that obtains image data, each time the image data is sampled, the actual illumination light scanning position A (m, n) at that time [m and n indicate the sampling order in the main and sub-scanning directions, respectively; m, n=1, 2, 3
. A method for correcting image data of a scanning microscope, characterized in that image data is replaced with data related to ideal sampling points.