JP2004170572A - Confocal microscopic system having image connecting function, light quantity eccentricity correcting method and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a confocal microscopic system capable of yielding a synthetic image whose luminance irregularity is restrained to be little by correcting the eccentricity of the light quantity of an original image in the case of generating the synthetic image in a wide range by connecting a plurality of images acquired by shifting the measuring range of the surface of a sample. <P>SOLUTION: The confocal microscopic system 1 is equipped with a correction amount storage means 98 for storing the correction amount of the eccentricity of the light quantity caused by optical aberration calculated based on the image obtained by measuring the mirror surface sample having uniform light reflectance on every pixel position, and a light quantity eccentricity correction means 97 for performing the correction of the eccentricity of the light quantity based on the correction amount read out from the means 98 for the image of the surface of the sample obtained by ordinary measurement on every pixel position. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention provides a confocal microscope system having an image stitching mode for stitching a plurality of images generated while shifting a measurement range of a sample surface to generate a synthesized image in a wide range, and to reduce uneven brightness of a synthesized image. And a computer program for realizing the method.
[0002]
[Prior art]
A confocal microscope scans a sample surface with light, receives light from the sample surface with a light receiving element via a confocal optical system, and processes the received light information to obtain an ultra-depth image (a very deep focal depth). Image) is displayed on the screen. When the distance between the focus of the objective lens and the sample is changed in the optical axis direction (that is, the height direction of the sample), the amount of light incident on the light receiving element via the confocal optical system, that is, the amount of received light, changes When the surface is focused, the amount of received light is maximized. Therefore, height information of each point on the sample surface is obtained from the distance between the focus of the objective lens and the sample when the maximum amount of received light is obtained.
[0003]
The obtained height distribution information of the sample surface is displayed on the screen of the display device by, for example, three-dimensional display. Alternatively, the height distribution can be replaced with a luminance distribution or a color distribution and displayed on a screen. A CRT (cathode ray tube) or an LCD (liquid crystal display) is used as a display device, and a controller for control, a display device, a console, and the like are connected to the confocal microscope to form a confocal microscope system.
[0004]
Also, while changing the distance between the focus of the objective lens and the sample at a predetermined pitch in the direction of the optical axis, the maximum light receiving amount (light receiving amount at the time of focusing) at each point on the sample surface is obtained, and the maximum light receiving amount at each point is obtained. By connecting the quantities, a black and white image of the sample surface with a very deep depth of focus can be obtained. This image is a super-depth image.
[0005]
Also, if the distance between the focus of the objective lens and the sample is fixed so that the maximum amount of received light can be obtained at any given pixel of interest, the amount of received light of the pixel of the portion of the pixel of interest and the portion where the height difference is large becomes significantly smaller, A black-and-white image is obtained in which only the portion having the same height as the target pixel is bright. This image is called a slice image.
[0006]
Further, there is a confocal microscope provided with a non-confocal optical system that shares some optical axes with the confocal optical system. By receiving light from a sample irradiated with white light by a color imaging element via a non-confocal optical system, a color image of the sample surface in the same range as the ultra-depth image can be obtained. This color image has a small depth of focus unlike the super-depth image, but is useful when observing the outline of the sample surface or when positioning the sample to obtain a confocal image of a target portion. Further, by performing a synthesizing process in which the luminance signal of the color image is replaced with the luminance signal of the super-depth image, a color image having a deep focal depth (this is referred to as a color super-depth image) can be obtained.
[0007]
The confocal microscope as described above has a drawback that the field of view is narrow like a normal optical microscope. That is, the higher the optical magnification, the narrower the range of the sample surface that can be observed (measured) as one visual field (hereinafter, referred to as a measurement range). In order to cover such a defect, a plurality of (for example, 3 × 3) measurement results (for example, confocal images) measured while shifting the measurement range are joined to form a wide range (for example, an area nearly 9 times larger). May be generated. Such joining of a plurality of images can be realized using the same technique as the processing of joining image files obtained by a digital camera.
[0008]
[Problems to be solved by the invention]
Since the confocal microscope is also a device having an optical system, it has a problem of optical aberration. This problem appears as, for example, a phenomenon that the periphery of the obtained super-depth image is slightly darker than the center. This phenomenon is referred to as light amount eccentricity.
[0009]
Due to the characteristics of human eyes, a state where the peripheral portion of the image is slightly darker than the central portion does not cause any discomfort. In addition, the optical system has been improved to reduce the optical aberration, and the eccentricity of the light amount due to the optical aberration is almost inconspicuous.
[0010]
However, as described above, when a plurality of images are stitched together to obtain an image of a wide range of the sample surface, a dark portion at a peripheral portion of each image becomes a center of a newly synthesized wide image. It is located at or near the part, which causes a sense of discomfort.
[0011]
In view of the above problems, the present invention corrects the eccentricity of the light amount of the original image and corrects the luminance when connecting a plurality of images obtained by shifting the measurement range of the sample surface to generate a composite image of a wide range. An object of the present invention is to provide a confocal microscope system capable of obtaining a synthesized image with less unevenness. It is also an object of the present invention to provide a method for such light amount eccentricity correction and a computer program for realizing the method.
[0012]
[Means for Solving the Problems]
A confocal microscope system according to the present invention receives light from a sample surface within a measurement range determined by an optical magnification by a light receiving element via a confocal optical system, and processes an image of the sample surface obtained by processing the received light information. A confocal microscope system having an image stitching mode for stitching a plurality of images generated while shifting a measurement range to generate a single composite image, wherein a mirror-surface sample having a uniform light reflectance is obtained. Correction amount storage means for storing, for each pixel position, a correction amount of light amount eccentricity caused by optical aberration calculated based on an image obtained by measurement, and correction for an image of a sample surface obtained by normal measurement Light amount eccentricity correction means for executing correction of light amount eccentricity based on the correction amount read from the amount storage means for each pixel position.
[0013]
Further, the light amount eccentricity correction method of the confocal microscope system according to the present invention uses a confocal microscope system to generate an image of a sample surface within a measurement range determined by the optical magnification thereof, and to generate a plurality of images generated while shifting the measurement range. A correction method for reducing the eccentricity of the amount of light in a combined image caused by optical aberrations when combining images to generate one combined image, comprising the steps of: (a) using a mirror sample having a uniform light reflectance; Calculating a correction amount of each pixel and storing the same in a storage unit so that the luminance of each pixel in the image obtained when the measurement is performed is equal to that of the pixel; (C) performing a correction based on the correction amount read from the storage means for the luminance of the pixel, and (c) outputting the pixel after performing the correction in step (b) for all the pixels as a measurement result. Or characterized in that it comprises a step of storing.
[0014]
According to the confocal microscope system and the light amount eccentricity correction method as described above, when a plurality of images are joined together to obtain a wide range image of the sample surface, the light amount of each of the original plurality of images is Since the eccentricity correction eliminates or reduces the eccentricity of the light amount between the central portion and the peripheral portion, the uneven brightness in the combined image obtained by connecting the central portions and the peripheral portions is eliminated or reduced, and the eccentricity correction becomes inconspicuous.
[0015]
In a preferred embodiment, in step (a) of the above method, the image obtained when the specular sample is measured is subjected to median filtering to remove noise, and thereafter, the correction amount of each pixel is calculated. As a result, even if the mirror sample has small dust or scratches, erroneous correction due to the dust or scratch is avoided.
[0016]
Further, the computer program according to the present invention receives light from a sample surface within a measurement range determined by an optical magnification by a light receiving element via a confocal optical system, and processes an image of the sample surface obtained by processing the received light information. A program to be executed by a computer connected to a confocal microscope system having an image stitching mode for stitching a plurality of images generated while shifting a measurement range to generate one composite image. Receiving data of an image obtained when measuring a specular sample having a high light reflectance from the confocal microscope system, and calculating a correction amount of each pixel so that the luminance of each pixel in the image is equal. And transmitting the calculated correction amount data of each pixel to the confocal microscope system.
[0017]
In a preferred embodiment, the computer program further comprises a step of removing noise by performing a median filter process on the image received from the confocal microscope system, and a correction amount of each pixel is calculated using the image after the median filter process. You.
[0018]
By using a computer program as described above, a computer system capable of high-speed processing can be connected to the confocal microscope system, and the correction amount for light amount eccentricity correction can be calculated. In particular, it is possible to execute the median filter processing that requires a long processing time at high speed.
[0019]
Such a computer program is supplied in a state of being stored in a computer-readable storage medium such as a CD-ROM, and is installed in a computer from the storage medium and executed.
[0020]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0021]
FIG. 1 shows a schematic configuration of a confocal microscope system according to an embodiment of the present invention. The confocal microscope system 1 includes a confocal microscope having a confocal optical system 2 and a non-confocal optical system 3, a laser driving circuit 44, a first AD converter 41, a CCD driving circuit 43, and a second AD converter of the confocal microscope. 42, a controller including an objective lens moving mechanism 40, a control unit 46 using a microcomputer, etc., and a display device 47 and an input device (console) 48 connected to the controller.
[0022]
First, the confocal optical system 2 of the confocal microscope and its signal processing will be described. The confocal optical system 2 includes a light source 10 for irradiating the sample wk with monochromatic light (for example, laser light), a first collimating lens 11, a polarizing beam splitter 12, a quarter-wave plate 13, a horizontal deflection device 14a, and a vertical deflection. The device 14b includes a first relay lens 15, a second relay lens 16, an objective lens 17, an imaging lens 18, a pinhole plate 9, a light receiving element 19, and the like.
[0023]
As the light source 10, for example, a semiconductor laser that emits red laser light is used. Laser light emitted from the light source 10 driven by the laser drive circuit 44 passes through the first collimator lens 11, the optical path is bent by the polarization beam splitter 12, and passes through the 波長 wavelength plate 13. Thereafter, the light is deflected in the horizontal (horizontal) direction and the vertical (vertical) direction by the horizontal deflection device 14a and the vertical deflection device 14b, passes through the first relay lens 15 and the second relay lens 16, and is sampled by the objective lens 17. The light is focused on the surface of the sample wk placed on the stage 30.
[0024]
The horizontal deflection device 14a is constituted by a resonant (resonant type) scanner, and the vertical deflection device 14b is constituted by a galvano (electromagnetic) scanner. By deflecting the laser light in both the horizontal and vertical directions, the surface of the sample wk is scanned with the laser light. For convenience of description, the horizontal direction is referred to as an X direction, and the vertical direction is referred to as a Y direction. The objective lens 17 is driven in the Z direction (optical axis direction) by the objective lens moving mechanism 40. Thus, the distance between the focus of the objective lens 17 and the sample wk in the optical axis direction (that is, the height direction of the sample wk) can be changed.
[0025]
However, the distance in the optical axis direction between the focal point of the objective lens 17 and the sample wk can be changed by another method. For example, instead of driving the objective lens 17 in the Z direction, the sample stage 30 may be driven in the Z direction. Alternatively, a configuration in which the focal point of the objective lens 17 is moved in the Z direction by inserting a lens whose refractive index changes between the objective lens 17 and the sample wk is also possible.
[0026]
In the confocal microscope of the present embodiment, the objective lens 17 can be electrically moved in the Z-axis direction via the objective lens moving mechanism 40 by a control signal from the control unit 46, and the sample stage 30 is provided with a stage manual operation mechanism. It can be displaced in the X direction, the Y direction and the Z direction by a manual operation via 31. In addition, the objective lens 17 can be moved up and down via the control unit 46 and the objective lens moving mechanism 40 by a key operation of the input device 48 (for example, an operation of an up / down key).
[0027]
The laser light reflected by the sample wk reverses the above optical path. That is, the light passes through the objective lens 17, the second relay lens 16, and the first relay lens 15, and again passes through the quarter-wave plate 13 via the horizontal deflection device 14a and the vertical deflection device 14b. As a result, the laser light passes through the polarization beam splitter 12 and is condensed by the imaging lens 18. The condensed laser light passes through the pinhole of the pinhole plate 9 disposed at the focal position of the imaging lens 18 and enters the light receiving element 19. The light receiving element 19 is composed of, for example, a photomultiplier tube (photomultiplier tube) or a photodiode, and converts the amount of received light into an electric signal. The electric signal corresponding to the amount of received light is provided to the first AD converter 41 via an output amplifier and a gain control circuit (not shown), and is converted into a digital value.
[0028]
The height (depth) information of the sample wk can be acquired by the confocal optical system 2 configured as described above. The principle will be briefly described below.
[0029]
As described above, when the objective lens 17 is driven in the Z direction (optical axis direction) by the objective lens moving mechanism 40, the relative distance in the optical axis direction between the focal point of the objective lens 17 and the sample wk changes. When the focal point of the objective lens 17 is focused on the surface of the sample wk, the laser light reflected on the surface of the sample wk is condensed by the imaging lens 18 via the above optical path, and almost all the laser light is It passes through the pinhole of the pinhole plate 9. Therefore, at this time, the amount of light received by the light receiving element 19 is maximized. Conversely, when the focus of the objective lens 17 is shifted from the surface of the sample wk, the laser light condensed by the imaging lens 18 focuses on a position shifted from the pinhole plate 9, so that a part of the laser beam is focused. Only light can pass through the pinhole. As a result, the amount of light received by the light receiving element 19 is significantly reduced.
[0030]
Therefore, if the amount of light received by the light receiving element 19 is detected at any point on the surface of the sample wk while driving the objective lens 17 in the Z direction (optical axis direction), the objective lens 17 when the amount of received light is maximized is detected. (Distance in the optical axis direction between the focal point of the objective lens 17 and the sample wk) can be uniquely obtained as height information.
[0031]
In practice, each time the objective lens 17 is moved by one step (one pitch), the surface of the sample wk is scanned by the horizontal deflection device 14a and the vertical deflection device 14b to obtain the amount of light received by the light receiving element 19. When the objective lens 17 is moved in the Z direction from the lower end to the upper end of the measurement range in the Z direction, light reception amount data that changes according to the Z direction position is obtained for each point (pixel) in the scanning range.
[0032]
FIG. 2 is a graph illustrating an example of the amount of received light that changes according to the position of the objective lens 17 in the Z direction. Based on such received light amount data, the maximum received light amount and the Z-direction position at that time are obtained for each point (pixel). Therefore, the distribution of the surface height of the sample wk on the XY plane is obtained. This process is executed by the control unit 46 using a microcomputer.
[0033]
The obtained surface height distribution information can be displayed on the monitor screen of the display device 47 by several methods. For example, the height distribution (surface shape) of the sample can be displayed three-dimensionally by three-dimensional display. Alternatively, the height data can be converted into luminance data to be displayed as a two-dimensional distribution of brightness. By converting the height data into color difference data, the height distribution can be displayed as a color distribution.
[0034]
Further, a surface image (monochrome image) of the sample wk can be obtained from a luminance signal using the received light amount obtained for each point (pixel) in the XY scanning range as luminance data. If a luminance signal is generated using the maximum amount of received light in each pixel as luminance data, an ultra-deep image with an extremely deep focal depth focused at each point having a different surface height can be obtained. When the height (Z-direction position) at which the maximum amount of received light is obtained at an arbitrary pixel of interest is fixed, the amount of received light of a pixel having a large height difference from the pixel of interest is significantly small, and An image in which only portions having the same height are bright (ie, a slice image) is obtained.
[0035]
Next, the non-confocal optical system 3 provided in the confocal microscope and its signal processing will be described. The non-confocal optical system 3 includes a white light source 20, a second collimating lens 21, a first half mirror 22, a second half mirror 23, and a color for irradiating the sample wk with white light (illumination light for capturing a color image). It includes a CCD (image sensor) 24 and the like. In addition, the non-confocal optical system 3 shares the objective lens 17 of the confocal optical system 2, and the optical axes of the two optical systems 1 and 2 partially match.
[0036]
For example, a white lamp is used as the white light source 20, but a special light source may not be provided, and natural light or indoor light may be used. The white light emitted from the white light source 20 passes through the second collimating lens 21, the optical path is bent by the first half mirror 22, and is focused on the surface of the sample wk placed on the sample stage 30 by the objective lens 17. .
[0037]
The white light reflected by the sample wk passes through the objective lens 17, the first half mirror 22, and the second relay lens 16, is reflected by the second half mirror 23, enters the color CCD 24, and forms an image. The color CCD 24 is provided at a position conjugate with or close to the pinhole of the pinhole plate 9 of the confocal optical system 2. The color image picked up by the color CCD 24 is read out by the CCD drive circuit 43, and its analog output signal is given to the second AD converter 42 and converted into a digital value. The color image thus obtained is displayed on the monitor screen of the display device 47 as an enlarged color image for observing the sample wk.
[0038]
Further, the super-depth image obtained by the confocal optical system 2 and the normal color image obtained by the non-confocal optical system 3 are combined to obtain a deep color super-depth with a focal depth substantially in focus at all pixels. Images can also be generated and displayed. For example, it is possible to generate a color super-depth image by a synthesis process in which a luminance signal forming a color image obtained by the non-confocal optical system 3 is replaced with a luminance signal of an ultra-depth image obtained by the confocal optical system 2. it can. Similarly, a color slice image can be generated by a synthesis process in which a luminance signal forming a color image obtained by the non-confocal optical system 3 is replaced with a luminance signal of a slice image obtained by the confocal optical system 2. .
[0039]
The controller including the control unit 46 is also in charge of the processing regarding the color image as described above. An input device 48 such as a console (operation console) and a display device 47 such as a CRT (cathode ray tube) or an LCD (liquid crystal display device) are connected to the controller.
[0040]
The user can set various measurement parameters using the input device 48 according to the guidance displayed on the screen of the display device 47. For example, a moving range (measurement range) and a moving pitch of the objective lens 17 in the Z direction are set. Alternatively, by setting the light receiving sensitivity (PMT gain) of the light receiving element 19 and the amount of attenuation by the ND filter according to the light reflectance of the surface of the sample wk, etc., the super-depth image or the slice image displayed on the screen is appropriate. Adjust so that the brightness (brightness) is high. Further, the shutter speed, gain, and white balance for obtaining a color image by the color CCD 24 are set.
[0041]
The confocal microscope system 1 (controller) of the present embodiment is also provided with a communication interface for connecting an external computer system such as a personal computer. By connecting an external computer system in which dedicated software for controlling the confocal microscope system 1 is installed to the confocal microscope system 1, processing of acquired image information and height distribution information of the sample wk can be performed seamlessly. It is possible to do.
[0042]
FIG. 3 is a block diagram illustrating an example of a hardware configuration in which an external computer system 50 is connected to a controller of the confocal microscope system 1. The external computer system 50 includes a display device 51 such as a CRT or LCD, a keyboard 52, a mouse (or another pointing device) 53, a communication interface 54 such as an RS232C, a USB (universal serial bus), IEEE1394, and a processing device (CPU). 55, a main memory 56 as a semiconductor storage medium, a fixed disk device 57 as an auxiliary storage device, and a removable disk device 58.
[0043]
The dedicated software for controlling the confocal microscope system 1 is supplied in a state stored in a storage medium 59 such as a CD-ROM, and is supplied from the storage medium 59 by a removable disk device 58 such as a CD-ROM drive. It is read and installed in the fixed disk device 57. The dedicated software installed in the fixed disk device 57 is loaded into the main memory 56 and executed by the processing device 55.
[0044]
The processing executed by such dedicated software includes processing for setting measurement conditions of the confocal microscope system 1 and image joining processing for joining images obtained as a result of measurement to generate a composite image. include. Further, a light amount eccentricity correction process for reducing luminance unevenness of a combined image obtained by the image joining process is also included. Next, as an example of the image joining process, a manual joining mode for manually designating a plurality of images to be joined and their positions will be described.
[0045]
FIG. 4 is a diagram illustrating an example of a screen display displayed on the display device 51 in the manual joining mode. A lower right area 61 of the screen display 60 is an area for displaying a plurality of images to be combined. An area 62 on the upper right of the screen display 60 is an area for displaying a combined image obtained by moving a plurality of images one by one and connecting them using a pointing device. An area 63 on the left side of the screen display 60 is an area for setting a joining condition.
[0046]
FIG. 5 is an enlarged view of an area 63 for setting a joining condition in the screen display 60 of FIG. In the upper block 70, conditions for creating a patchwork data file (ie, manually creating a composite image file) are set. In the sub-block 71, a reference data type (that is, a type of information used for image synthesis) is selected. Light intensity data, R, G, and B can be selected from four types. The light quantity data means the light reception quantity data of each pixel constituting the above-described super-depth image. R, G, and B represent red, green, and blue, respectively, which are the color components of each pixel in the color image obtained by the above-described non-confocal optical system.
[0047]
In the sub-block 72, the data to be saved (that is, the type of the composite image) can be selected. It is possible to select whether or not to save each of the four types of data, that is, a color raw image, light amount data, color data, and height data. The color raw image means a color image obtained by a non-confocal optical system, the light amount data means the above-mentioned super-depth image, the color data means the above-mentioned color ultra-depth image, and the height data means the above-mentioned color super-depth image It means an image of the distribution information of the surface height.
[0048]
In the sub-block 73, it is possible to select whether to perform the automatic overlay position correction (ON) or not (OFF). The automatic overlapping position correction means image processing for automatically performing fine adjustment of the overlapping position so that the overlapping portions of two adjacent images coincide with each other. Such image processing is a known technique.
[0049]
In the sub-block 74, it is possible to select whether or not to save the above setting conditions as a creation condition setting file. When the "condition file creation" button is clicked with the mouse 53, a creation condition setting file is created and saved. Thereafter, when generating a composite image under the same condition, the condition setting can be easily performed by reading the corresponding creation condition setting file.
[0050]
The “file creation” button 75 is a button that is pressed last when creating a file of a composite image according to a procedure described later. The “end” button 76 is a button pressed when terminating the patchwork data file creation mode.
[0051]
In the lower block 77, a plurality of images and the like (that is, images displayed on the screen) to be used for generating a composite image are selected. The “select folder” button 78 is pressed when specifying a folder (directory) in which a plurality of images and the like used for generating a composite image are stored.
[0052]
In the sub-block 79, a display file type can be selected. One can be selected from a normal measurement data file, a patchwork data file, and a creation condition setting file. The normal measurement data file is a plurality of images used for generating a composite image. The patchwork data file is a file of the generated composite image. It should be noted that the file of the composite image in the middle of reaching the target composite image is also included in the patchwork data file. The creation condition setting file is a setting condition file stored as described above.
[0053]
In the sub-block 80, a display data type (that is, a type of a plurality of images used for generating a composite image) can be selected. One of four types of data, that is, a color raw image, light amount data, color data, and height data can be selected. The color raw image means a color image obtained by a non-confocal optical system, the light amount data means the above-described super-depth image, the color data means the color ultra-depth image, and the height data means the surface height. It means an image of distribution information. When the images are manually joined by an operation described later, the accuracy of the manual joining is increased by selecting the type of the image having the highest visibility (different depending on the sample). For example, in the case of a sample having a characteristic uneven shape that is unlikely to appear in a pattern pattern on the surface, using an image obtained by processing height data improves visibility and improves joining accuracy.
[0054]
FIG. 6 is a flowchart illustrating an example of a procedure for manually joining images. Although this flowchart includes both the operation by the user and the processing executed by the processing device 55 according to the program, the distinction will be clarified in the following description.
[0055]
In step # 101, the user executes the measurement while moving the sample stage at an appropriate pitch in the X direction and the Y direction, and acquires and stores a plurality of images having different measurement ranges. Note that it is necessary to determine the moving pitch of the sample stage so that the ends of adjacent images overlap each other.
[0056]
In step # 102, the user starts the manual splicing mode, and sets the splicing conditions as described above in the third area 63 (see FIG. 5) of the screen display shown in FIG. 4 (step # 103). . In the following step # 104, a plurality of images used for combining (joining) are displayed in the first area 61 of the screen display shown in FIG. In the display example of FIG. 4, images 66 to 69 of the display data type (light amount data) specified in the sub-block 80 of the file stored in the folder specified in the third area 63 are stored in the first area 61. Is displayed. The display data type specified in the sub-block 80 can be changed and set even during the execution of the joining.
[0057]
In the next step # 105, the user drags a plurality of images one by one with the mouse 53 to move to the second area 62, and drops the images at the joint positions. In the display example of FIG. 4, a composite image 64 obtained by connecting three images 66 to 68 displayed in the first area 61 is displayed in the second area 62. Then, a state is displayed in which the fourth image 69 displayed in the first area 61 is dragged to the second area 62 to join the image 65 to the composite image 64.
[0058]
After arranging all the images, in the next step # 106, the user instructs execution of the combination processing. That is, the “file creation” button 75 shown in FIG. 5 is pressed.
[0059]
When the execution of the combining process is instructed, the processing device 55 checks whether or not the automatic superposition correction is turned on in the sub-block 73 shown in FIG. 5 (step # 107). Executes the overlay position automatic correction (step # 108). In the last step # 109, generation, display and storage of a composite image file are executed. One or more synthesized image (data) files selected in the sub-block 72 shown in FIG. 5 are generated and stored. Similarly to the type of image displayed in the first area 61, a composite image of the type specified in the sub-block 80 is displayed in the second area 62.
[0060]
Next, a description will be given of a method and a process for correcting the eccentricity of the light amount of the original image to obtain a composite image with less luminance unevenness in the image joining process as described above. The light amount eccentricity here refers to a phenomenon in which the peripheral portion of an image becomes darker (luminance decreases) than the central portion due to optical aberration. Such light amount eccentricity is almost inconspicuous in general, but when a composite image is generated by connecting a plurality of images, the dark portion may be conspicuous because it is located at or near the center of the composite image.
[0061]
Therefore, in the confocal microscope system 1 of the present embodiment, a predetermined specular sample (test sample) is measured, and a correction amount is calculated and stored based on the light amount eccentricity obtained from the measurement result, and the actual correction amount is calculated. The light amount eccentricity correction processing for correcting the light amount eccentricity in the image such as the ultra-depth image obtained from the measurement of the sample is executed.
[0062]
FIG. 7 is a flowchart of a process of calculating and storing a correction amount in the light amount eccentricity correction process. First, in step # 201, the specular sample is measured, and the obtained image is displayed on the screen. This mirror sample is a test sample for correcting decentering of light amount having a mirror surface manufactured by evaporating a metal on a substrate, for example. It is manufactured so that adhesion of dust and scratches do not occur as much as possible. Therefore, an image (for example, an ultra-depth image) obtained by the measurement of the specular sample becomes a uniform white image, but the peripheral portion of the image is darker than the central portion due to the optical aberration. Therefore, the correction amount (correction coefficient) of each pixel may be calculated so that the luminance of each pixel of the obtained image is equal (for example, equal to the maximum luminance).
[0063]
However, it is very difficult to make the surface of the mirror sample in a perfect state without any dust or scratches (that is, noise). Therefore, the influence of such noise is removed by median filter processing. The median filter processing is a filter processing in which an intermediate value is extracted from the values of a plurality of (odd × odd) pixels centered on the target pixel and is set as the value of the target pixel. The above-described processing is sequentially executed for each pixel constituting the image. By this median filtering, noise whose value changes rapidly can be removed. In order to effectively remove noise by median filter processing, it is necessary to appropriately set the size of the median filter (odd number × odd number of pixels). If the size of the median filter is large, the processing time is long. If the size is too small, the noise is not completely removed.
[0064]
Therefore, in step # 202, the user specifies the size of the median filter on the screen. FIG. 8 shows how the size of the median filter is specified. When the user designates the rectangular area 82 by dragging the mouse 53 so as to surround dust and flaws included in the image obtained by the measurement of the specular sample, the processing device 55 determines the size of the designated rectangular area 82. (Number of pixels) is calculated and a message dialog box 84 is displayed.
[0065]
If the user clicks the OK button 85 with the mouse 53, the processing device 55 executes the above-described median filter processing in the next step # 203, and displays the processed image on the screen. When the user clicks the cancel button 86, the rectangular area 82 can be specified again.
[0066]
When the median filter process is executed and the display image is updated, the user visually checks whether noise has been removed from the image (step # 204). If the noise still remains, the process returns to step # 202 and the process is repeated from the designation of the size of the median filter. This time, the rectangular area 82 is designated so as to surround the noise with a size larger than the previous time.
[0067]
When the noise is removed in this way, the processing device 55 calculates the correction amount (correction coefficient) of each pixel in the next step # 205. The correction amount is calculated so that the luminance of all pixels is equal. For example, if there is a pixel having the maximum luminance in the center of the image and the luminance of the target pixel in the peripheral part is 90% of the maximum luminance, the correction coefficient of the pixel is 100/90 (rounded to 1.1). I do. The correction coefficients are calculated for all the pixels, and in the next step # 206, the processing device 55 stores the correction amount of each pixel. The storage location may be, for example, the fixed disk device 57 of the computer system 50, but is usually stored on the controller side of the confocal microscope system 1 via the communication interface 54.
[0068]
FIG. 9 is a flowchart of the correction process in the actual measurement in the light amount eccentricity correction process. As an example, a case of a correction process in measurement of an ultra-deep image is shown. This processing is mainly executed by the controller (control unit 46) of the confocal microscope system 1. In step # 301, the control unit 46 measures the sample and acquires an ultra-deep image.
[0069]
The obtained super-depth image is temporarily stored, and in the next step # 302, the correction amount for each pixel (position) stored in the processing shown in FIG. 7 is read, and the luminance of each pixel is corrected according to the correction amount. I do. When the correction is completed for all the pixels (Yes in Step # 303), the corrected super-depth image is stored as the measurement result (Step # 304). When the computer system 50 is connected to the confocal microscope system 1, the corrected super-depth image is transmitted to the computer system 50 as a measurement result and displayed on the display device 51.
[0070]
FIG. 10 is a diagram showing functional blocks of the system configuration according to the embodiment of the present invention. However, this block diagram shows a configuration example in a case where images are automatically joined. The light amount eccentricity correction method according to the present invention can be applied not only to the above-described manual joining of images but also to the case of automatically joining images.
[0071]
In the configuration of FIG. 10, a computer system 50 is connected to the confocal microscope system 1, and the computer system 50 includes an image joining unit 88, an image information storage unit 89, a light amount eccentricity calculation unit 90, a maximum noise designation unit 91, A median filter processing unit 92 and a communication control unit 93 are provided. The confocal microscope system 1 (controller) includes a communication control unit 94, a stage driving unit 95, an imaging / measuring unit 96, a light amount eccentricity correcting unit 97, and a correction amount storing unit 98.
[0072]
In measurement of an ultra-deep image or the like, a measurement result (for example, ultra-deep image data) acquired by the imaging / measuring unit 96 of the confocal microscope system 1 is transmitted to the computer system 50 via the communication control unit 94, and the image information is acquired. It is stored in the storage means 89 (a storage area provided in the fixed disk device 57).
[0073]
The image joining means 88 composed of dedicated software of the computer system 50 operates the stage driving means 95 of the confocal microscope system 1 via the communication control means 93, and shifts the measurement range by a predetermined positioning to obtain an ultra-deep image. And the like are acquired from the confocal microscope system 1 and stored in the image information storage unit 89. After that, the plurality of image data read from the image information storage unit 89 are connected according to the data of the amount of movement of the stage 30 by the stage driving unit 95. At this time, fine adjustment of the overlapping position is performed so that the overlapping portion of the two adjacent images coincides as described in the manual joining process. The composite image obtained by the joining process is stored in the image information storage unit 89.
[0074]
If necessary, the user measures the specular sample as described above, and calculates and stores the correction amount of the light amount eccentricity correction. The light amount eccentricity amount calculating means 90 constituted by dedicated software of the computer system 50 calculates the correction amount of each pixel (position) by the above-described processing. At this time, the median filter processing unit 92 performs a process of removing noise using a median filter of a size determined from the area specified by the maximum noise specification unit 91 (corresponding to the above-described rectangular area 82). .
[0075]
The correction amount of each pixel (position) calculated by the light amount eccentricity amount calculation unit 90 is transmitted to the confocal microscope system 1 via the communication control unit 93 and stored in the correction amount storage unit 98. Once the correction amount is stored in the correction amount storage means 98, the measurement result (image data) obtained by the imaging / measurement means 96 in the subsequent measurement is read from the correction amount storage means 98 by the light amount eccentricity correction means 97. The image data is corrected according to the issued correction amount, and the corrected image data is transmitted to the computer system 50.
[0076]
As described above, the embodiments of the present invention have been described including the modified examples as appropriate. However, the present invention is not limited to the above embodiments, and can be implemented in various forms.
[0077]
For example, in the above-described embodiment, dedicated software for controlling the confocal microscope system 1 is installed in the external computer system 50, and the external computer system 50 calculates a correction amount for the light amount eccentricity correction and performs a median filter. Although the processing is executed, such processing may be executed on the confocal microscope system 1 side.
[0078]
In general, when the correction amount is calculated and the median filter processing is executed by the external computer system 50, the advantage that the processing speed is faster can be obtained. On the other hand, if such processing is performed on the confocal microscope system 1 side, there is no need to exchange data regarding the light amount eccentricity correction with the external computer system 50. Further, even when the measurement is executed or the measurement result is displayed by using the display device 47 and the input device 48 provided as standard equipment in the confocal microscope system 1 without connecting the external computer system 50, the light amount eccentricity correction method of the present invention is also used. Can be applied.
[0079]
Although the confocal microscope of the above embodiment is a reflection type microscope, the present invention can be applied to a transmission type confocal microscope. In the case of a transmission microscope, laser light of a confocal optical system and white light of a non-confocal optical system are emitted from the back surface of the sample. The light source of the confocal optical system may include not only a monochromatic light source including a laser light source but also a light source including a plurality of wavelengths. The light source of the non-confocal optical system can be replaced by natural light or room light.
[0080]
【The invention's effect】
As described above, according to the confocal microscope system, the light amount eccentricity correction method, and the computer program of the present invention, when performing a joining process of a plurality of images to obtain an image of a wide range of the sample surface, Since the light amount eccentricity of the central portion and the peripheral portion is eliminated or reduced by the light amount eccentricity correction for each of the plurality of original images, the uneven brightness in the combined image obtained by connecting them is less noticeable.
[Brief description of the drawings]
FIG. 1 is a diagram showing a schematic configuration of a confocal microscope system according to an embodiment of the present invention.
FIG. 2 is a graph illustrating an example of a light receiving amount that changes according to a Z-direction position of an objective lens.
FIG. 3 is a block diagram illustrating a hardware configuration example in which an external computer system is connected to a controller of the confocal microscope system.
FIG. 4 is a diagram illustrating an example of a screen display displayed on a display device in a manual joining mode.
FIG. 5 is an enlarged view of a region for setting a joining condition in the screen display of FIG. 4;
FIG. 6 is a flowchart illustrating an example of a procedure for manually joining images.
FIG. 7 is a flowchart of a process of calculating and storing a correction amount in the light amount eccentricity correction process.
FIG. 8 is a diagram showing how a size of a median filter is specified.
FIG. 9 is a flowchart of a correction process in an actual measurement in the light amount eccentricity correction process.
FIG. 10 is a diagram showing functional blocks of a system configuration according to the embodiment of the present invention.
[Explanation of symbols]
Reference Signs List 1 confocal microscope system 2 confocal optical system 19 light receiving element 47, 51 display device 59 storage medium 60 screen display 97 light amount eccentricity correction means 98 correction amount storage means wk sample

Claims (6)

Light from the sample surface within the measurement range determined by the optical magnification is received by the light receiving element via the confocal optical system, and an image of the sample surface obtained by processing the received light information is generated, and the measurement range is shifted. A confocal microscope system having an image stitching mode for stitching a plurality of images generated while generating a single composite image,
Correction amount storage means for storing, for each pixel position, a correction amount of light amount eccentricity caused by optical aberration calculated based on an image obtained by measuring a mirror-finished sample having uniform light reflectance,
Light amount eccentricity correction means for executing, for each pixel position, light amount eccentricity correction based on the correction amount read from the correction amount storage means for an image of the sample surface obtained by normal measurement. Confocal microscope system. When a confocal microscope system is used to generate an image of the sample surface within a measurement range determined by the optical magnification and combine a plurality of images generated while shifting the measurement range to generate one composite image A correction method for reducing the light amount eccentricity of the composite image due to the optical aberration,
(A) calculating a correction amount of each pixel and storing the same in a storage unit such that the luminance of each pixel in an image obtained when a mirror sample having uniform light reflectance is measured is equal;
(B) correcting the luminance of each pixel in the image obtained by the normal measurement based on the correction amount read from the storage unit;
(C) outputting or storing pixels as measurement results after executing the correction of step (b) for all pixels, and a light amount eccentricity correction method for a confocal microscope system. 3. The method according to claim 2, wherein in the step (a), a correction amount of each pixel is calculated after removing noise by performing a median filter process on an image obtained when the specular sample is measured. Eccentricity correction method of the confocal microscope system of the above. Light from the sample surface within the measurement range determined by the optical magnification is received by the light receiving element via the confocal optical system, and an image of the sample surface obtained by processing the received light information is generated, and the measurement range is shifted. A program to be executed by a computer connected to a confocal microscope system having an image stitching mode for stitching a plurality of images generated while generating one composite image,
Receiving data of an image obtained when measuring a specular sample having a uniform light reflectance from the confocal microscope system;
Calculating a correction amount of each pixel so that the luminance of each pixel in the image is equal;
Transmitting the calculated data of the correction amount of each pixel to the confocal microscope system. The image processing apparatus according to claim 1, further comprising: removing noise by performing a median filter process on the image received from the confocal microscope system, wherein a correction amount of each pixel is calculated using the image after the median filter process. 4. The computer program according to 4. A computer-readable storage medium on which the computer program according to claim 4 is recorded.

Images