JP2009282356A - Microscope apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce anxiety feeling and uncomfortable feeling which user senses when performing automatic measurement with a microscope accompanied by automatic setting of a measuring condition. <P>SOLUTION: A controller 111 controls a scanning confocal microscope 100, and sequentially acquires a slice image for an observation surface for each height of a sample 101 over an acquired range in the height direction. All focal images of the sample 101 are prepared by updating, whenever a slice image is newly acquired, the maximum luminance image acquired by setting, as the luminance value of the pixel at the position, the larger one of the luminance value of the pixel constituting a newly acquired slice image and the maximum value of luminance values of pixels that are at the same position as the former pixel and constitute respective already acquired slice images. A progress situation calculation section 117 calculates a value showing the acquiring progress situation of the slice images until the present time based on the value showing the preparing situation of all focal images. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique of a microscope, and more particularly, to a technique of automatic measurement performed using a microscope.

As an apparatus for acquiring an omnifocal image or a three-dimensional image of a sample, for example, there is a scanning confocal microscope. An outline of the configuration of the scanning confocal microscope is shown in FIG.
In FIG. 8, the light emitted from the point light source 801 passes through the half mirror 802, and then converges on one point on the sample 804 to be observed by the objective lens 803 that has been favorably corrected for aberrations. The sample 804 is illuminated. The reflected light from the sample 804 at this time is reflected by the half mirror 802, and then the reflected light from other than the focal point is cut by the pinhole plate 805, and only the reflected light that has passed through the pinhole is a photodetector. 806 is detected. Here, a two-dimensional image can be obtained from the amount of light detected by the photodetector 806 when the sample 804 is illuminated by two-dimensional scanning with irradiation light from the point light source 801.

  In FIG. 8, the reflected light from the position (i), which is a position deviated from the focusing position of the objective lens 803, cannot pass through the pinhole plate 805 and is not detected by the photodetector 806. Therefore, with the configuration shown in FIG. 8, it is possible to obtain an image only at the condensing position of the objective lens 803, in other words, an image only at the in-focus position.

  The confocal optical system as shown in FIG. 8 has a shallow depth of focus. For this reason, there is no problem when the observation surface of the sample 804 is planar, but when the observation surface has a different height (having irregularities), the focus position on the observation surface An image cannot be obtained for a region outside the range. That is, when the sample 804 is at the condensing position of the objective lens 803, the output (detected light amount value) of the photodetector 806 is maximized, and light detection is performed as the relative position between the objective lens 803 and the sample 804 increases from this position. The output of the device 806 decreases rapidly.

  Accordingly, as shown in FIG. 9, a non-planar sample 900 having observation planes A, B, and C each having a height of a, b, and c (where b> a> c) is shown in the illustrated Z. When observing in the direction, for example, even if the observation surface A is focused, the observation images of the observation surfaces B and C cannot be obtained.

  By the way, in such a case, within the range from the height c to the height b, the sample 900 is moved while moving either the sample 900 or the objective lens 803 in the Z-axis direction (optical axis direction) at a predetermined pitch. A plurality of slice images (confocal images) are acquired. Then, when the maximum output value of the photodetector 806 is extracted for each pixel position constituting each slice image, and the extracted light quantity value is arranged at each pixel position, an omnifocal image of the sample 900 is formed. be able to. Further, by obtaining height information (relative distance between the sample 900 and the objective lens 803) in the optical axis direction when the output of the photodetector 806 is maximized for each pixel position, a three-dimensional image of the sample 900 is obtained. Can also be configured.

  When obtaining such an omnifocal image or a three-dimensional image, measurement conditions such as a measurement range (for example, a range of b in FIG. 9) and a measurement pitch (movement pitch in relative movement between the sample 900 and the objective lens 803), etc. And a plurality of slice images are acquired under the measurement conditions. Accordingly, depending on the setting contents of the measurement range and the measurement pitch value, the measurement time may become very long. In such a case, the user may feel uneasy about whether the measurement is in progress or uncomfortable about not knowing when the measurement is finished.

As a technique for reducing such anxiety and discomfort, for example, in Patent Document 1, measurement conditions are manually set in advance, and the remaining time until the end of measurement is calculated based on the set measurement conditions. A technique of displaying and displaying is disclosed.
JP 2000-330026 JP

  However, when the apparatus automatically performs not only the measurement operation but also the measurement condition setting, the measurement end position is not always determined at the start of the measurement. For this reason, in the technique of Patent Document 1 on the assumption that manual setting is performed, the remaining time until the end of measurement cannot be calculated and displayed.

  Further, during the measurement operation in which the apparatus automatically sets the measurement conditions, it is possible to display the number of captured images (number of steps) from the start of measurement to the present. However, simply displaying the number of steps allows the user to know that the measurement is in progress, but not when the acquisition is complete, especially when the measurement time is long. The user will feel strong discomfort.

  The present invention has been made in view of the above-described problems, and a problem to be solved is to reduce anxiety and discomfort that a user feels when performing automatic measurement with a microscope that automatically sets measurement conditions. That is.

  A microscope apparatus according to one aspect of the present invention includes a slice image acquisition unit that sequentially acquires slice images of an observation surface for each height of an observation sample over an acquisition range in the height direction, and the slice image acquisition unit includes: The luminance value of the pixel constituting the newly acquired slice image and the maximum value of the luminance values of the pixels at the same position as the pixel and constituting each of the already acquired slice images, The omnifocal which creates the omnifocal image of the observation sample by updating the maximum luminance image obtained by setting the larger one as the luminance value of the pixel at the position, each time the slice image is newly acquired. An image creation means, an end determination means for judging the end of acquisition of the slice image over the acquisition range based on the slice image acquired by the slice image acquisition means most recently, and the acquisition of the slice image over the acquisition range of the slice image. A progress status calculating means for calculating a value indicating the progress status of acquisition until the end of the acquisition based on a value indicating the creation status of the omnifocal image by the omnifocal image creating means; and the progress status calculating means Output means for outputting the progress status indicated by the calculated value.

  In the above-described microscope apparatus according to the present invention, the value indicating the creation status of the omnifocal image by the omnifocal image creation means is a luminance that is greater than or equal to a predetermined reference luminance level among the pixels constituting the maximum luminance image. The progress status calculating means can be configured to calculate a ratio of the frequency with respect to a predetermined first reference pixel number as a value indicating the progress status.

  Further, in the above-described microscope apparatus according to the present invention, the progress status calculation unit is configured to calculate a value indicating the progress status based on the luminance value of the slice image most recently acquired by the slice image acquisition unit. can do.

  At this time, the value indicating the creation status of the omnifocal image by the omnifocal image creation means is the frequency of the pixels constituting the maximum brightness image whose brightness is equal to or higher than a predetermined reference brightness level, and The progress status calculation means includes the ratio of the frequency to the predetermined first reference pixel number, and the frequency of the pixels constituting the slice image most recently acquired by the slice image acquisition means that have a luminance equal to or lower than a predetermined reference luminance level. The sum with the ratio to the predetermined second reference pixel number can be calculated as a value indicating the progress status.

  Further, in the microscope apparatus according to the present invention described above, the end determination unit has a predetermined number of pixels whose luminance is equal to or lower than a predetermined reference luminance level among pixels constituting the slice image most recently acquired by the slice image acquisition unit. When it is determined that the second reference pixel number is exceeded, it is possible to make a determination that the acquisition is completed.

  A microscope apparatus according to another aspect of the present invention includes a slice image acquisition unit that sequentially acquires slice images of an observation surface for each height of an observation sample over an acquisition range in the height direction, and the slice image. Each time a new image is acquired, the inclination of the change in the luminance value of the pixel at the same position constituting each acquired slice image is obtained, and the maximum luminance value in the pixel is determined for each pixel based on the change in the inclination. An omnifocal image creating unit that creates an omnifocal image of the observation sample by setting the detected maximum luminance value as the luminance value of each pixel, and the slice image obtained by the slice image acquiring unit End determination means for determining the end of acquisition over the acquisition range based on the number of pixels for which the maximum luminance value has been found, and acquisition of the slice image over the acquisition range by the slice image acquisition means A progress status calculating means for calculating a value indicating a progress status of the acquisition of the slice image up to the present time based on the number of pixels in which the maximum value of the luminance is found, and the progress status calculating means. Output means for outputting the progress status indicated by the value.

  In the above-described microscope apparatus according to the present invention, the progress calculation means calculates a ratio of the number of pixels in which the maximum value of the luminance is found with respect to a predetermined reference pixel number as a value indicating the progress. Can be configured to.

  According to the present invention, it is possible to reduce anxiety and discomfort felt by the user when performing automatic measurement with a microscope that automatically sets measurement conditions.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
First, FIG. 1 will be described. FIG. 1 shows the configuration of a scanning confocal microscope system according to the first embodiment of the present invention.

  The scanning confocal microscope 100 includes an objective lens 102, an objective lens switching unit 103, a Z position control unit 104, a half mirror 105, a pinhole plate 106, a photodetector 107, a two-dimensional scanning unit 108, a mirror 109, and laser illumination. Part 110 is provided. The scanning confocal microscope 100, the controller 111, and the output device 112 constitute a scanning confocal microscope system.

  The objective lens switching unit 103 switches the objective lens 102 disposed on the optical axis of the scanning confocal microscope 100. The Z position control unit 104 controls the Z position of the objective lens 102 (the position in the optical axis direction of the scanning confocal microscope 100). The pinhole plate 106 cuts off reflected light from other than the focal point. The photodetector 107 detects light and outputs a signal indicating the amount of light. The two-dimensional scanning unit 108 two-dimensionally scans the laser light on a plane perpendicular to the traveling direction. The laser illumination unit 110 irradiates the sample (observation sample) 101 by exciting the laser light. The controller 111 acquires an observation image of the sample 101 and Z position information of the objective lens 102. The output device 112 displays the acquired observation image of the sample 101 and the like.

  Laser light emitted from the laser illumination unit 110 is reflected by the mirror 109 and introduced into the two-dimensional scanning unit 108. The two-dimensional scanning unit 108 emits the laser light by two-dimensional scanning. The laser light emitted from the two-dimensional scanning unit 108 passes through the half mirror 105 and is then collected by the objective lens 102 to illuminate the surface of the sample 101. The reflected light from the surface of the sample 101 at this time passes through the objective lens 102 and is then reflected by the half mirror 105 and travels toward the pinhole plate 106. The pinhole plate 106 cuts the reflected light from other than the condensing point among the reflected light. The reflected light that has passed through the pinhole plate 106 is detected by a photodetector 107, and a signal indicating the amount of light is output. Based on this signal, an image (confocal image) of the sample 101 that is sliced on a plane perpendicular to the optical axis direction is optically obtained.

  Here, the Z position control unit 104 drives the objective lens 102 in the optical axis direction, and sequentially acquires confocal images at each Z position while changing the relative distance of the objective lens 102 with respect to the sample 101. An omnifocal image or a three-dimensional image can be obtained by extracting the maximum luminance at the same pixel for each pixel from the plurality of acquired confocal images. Note that the movement range of the Z position at this time (change range of the relative distance of the objective lens 102 with respect to the sample 101) is referred to as a measurement range.

  The controller 111 includes an MPU (arithmetic processing unit), a main memory, an input device, an interface device, and a storage device, and an output device 112 is connected thereto. Note that the MPU, main memory, input device, interface device, and storage device are not shown in FIG.

  The MPU performs operation control of the entire scanning confocal microscope system of FIG. 1 by executing a control program stored in a storage device, for example. By executing the control program, at least the luminance frequency counting unit 115, It functions as a maximum luminance frequency counting unit 116, a progress situation calculation unit 117, and an image capture end determination unit 118.

  The main memory is a semiconductor memory used as a work memory by the MPU as needed, and functions as at least a confocal image memory 113, a maximum luminance image memory 114, and an operation parameter memory 119.

The input device is a mouse device or a keyboard device, for example, and acquires various instructions from the user as inputs.
The interface device manages the exchange of various data with each component of the scanning confocal microscope system.

The storage device is, for example, a hard disk device, and stores various control programs and data.
The output device 112 is a display device, for example, and displays an operation screen, an observation image, and the like under the control of the MPU.

The confocal image memory 113 stores a confocal image of the sample 101 generated by the MPU based on the signal output from the photodetector 107.
The maximum luminance image memory 114 is obtained by extracting the maximum luminance value for each pixel at the same position from a plurality of confocal images acquired while moving the Z position of the objective lens 102 from the measurement start position to the current position. The maximum luminance image that can be obtained is stored.

The luminance frequency counting unit 115 counts the number of pixels having a luminance value equal to or lower than the reference luminance level A among the pixels constituting the confocal image stored in the confocal image memory 113.
The maximum luminance frequency counting unit 116 counts the number of pixels having a luminance value equal to or higher than the reference luminance level B among the pixels constituting the maximum luminance image stored in the maximum luminance image memory 114.

The progress status calculation unit 117 calculates the progress status from the start to the end of the image capturing operation by the scanning confocal microscope 100.
The image capturing end determination unit 118 determines whether to end the image capturing operation by the scanning confocal microscope 100.

  The operation parameter memory 119 is used in the automatic measurement operation by the scanning confocal microscope system such as the reference luminance levels A and B described above and the number of reference pixels used in the determination by the image capturing end determination unit 118. Various operation parameters are stored.

The scanning confocal microscope system shown in FIG. 1 is configured as described above.
Next, a control process performed by the controller 111 shown in FIG. 1 will be described. In this control process, the controller 111 itself sets the measurement range described above to generate an omnifocal image of the sample 101, and displays the progress status from the start to the end of the confocal image capturing operation of the sample 101 for that purpose. Do. In the controller 111, the MPU reads out and executes a control program stored in advance in the storage device, thereby realizing the control process described below.

First, FIG. 2 will be described. FIG. 2 is a flowchart showing a first example of control processing performed by the controller 111.
First, the user operates the input device of the controller 111 while observing the image of the sample 101 displayed on the output device 112 to give a predetermined instruction to the Z position control unit 104, thereby changing the Z position of the objective lens 102. Move to the lower limit of the measurement range. The MPU acquires the Z position at this time as the setting of the measurement start position. In addition, the manual setting of the measurement end position (the upper limit position of the measurement range) that is conventionally performed at this time is not performed here.

Thereafter, the user operates the input device of the controller 111 to give a predetermined instruction to start the control process of FIG.
When this control process is started, first, in S201, a process of reading the above-described operation parameter from the operation parameter memory 119 is performed. The operation parameters read out by this process are the maximum upper limit position and the maximum lower limit position that can be taken with respect to the Z position of the objective lens 102, the above-described measurement pitch, two reference luminance levels used in the processes of S205 and S206, which will be described later, and will be described later. These are the two reference pixel numbers used in the processes of S209 and S210, respectively. These operation parameters may be fixed values set in advance, but may be set by the user and stored in the operation parameter memory 119 before the start of the control process.

  In S202, the process of capturing the confocal image of the sample 101 is performed at the current Z position, and the obtained confocal image (slice image with respect to the observation surface for each Z position of the sample 101) is associated with information indicating the Z position. Thus, processing to be stored in the confocal image memory 113 is performed. In the confocal image capturing process of the sample 101, the output signal from the light detector 107 is acquired, and the correspondence between the light amount (luminance value) represented by the acquired signal and the constituent pixels of the image is determined in two. This is performed based on the scanning position when the signal is acquired in the two-dimensional scanning of the laser beam by the two-dimensional scanning unit 108. By doing so, a confocal image of the sample 101 is obtained.

  Subsequent to S202, in S203, a process for creating a maximum luminance image is performed. In this process, the brightness value of the confocal image newly stored in the confocal image memory 113 and the brightness value of the image already stored in the maximum brightness image memory 114 (each of the already acquired slice images are configured. A comparison is made for each pixel at the same position. Here, for a pixel having a higher luminance value in the confocal image, the luminance value of the pixel is updated to that of the confocal image and stored in the maximum luminance image memory 114. Therefore, when the maximum luminance values stored in the maximum luminance image memory 114 are arranged for each pixel, a maximum luminance image is obtained. If the update of the maximum luminance image is repeated for the confocal image with respect to the observation plane for each Z position of the sample 101 acquired over the acquisition range in the Z direction, the maximum luminance image becomes the omnifocal image of the sample 101. Become.

  When the brightness value stored in the maximum brightness image memory 114 is updated, information indicating the Z position at the time of capturing the confocal image from which the brightness value is obtained is also stored in the maximum brightness image memory 114 for each pixel. Try to remember.

  In S204, for each of the confocal image stored in the confocal image memory 113 in the processing of S202 and the maximum luminance image stored in the maximum luminance image memory 114 in the processing of S203, the luminance value and the number of pixels of the constituent pixels. A process of creating a histogram indicating the relationship between the Examples of histograms for confocal images and histograms for maximum luminance images created by the processing of S204 are shown in FIGS. 3A and 3B, respectively.

  In S205, a process of counting the frequency of pixels having a luminance value equal to or lower than the reference luminance level A, which is one of the operation parameters described above, is performed from the histogram created for the confocal image in the process of S204 (see FIG. 3A). The MPU of the controller 111 functions as the luminance frequency counting unit 115 by executing the processing of S205, and the count result at this time is referred to as the luminance frequency for the confocal image acquired by the latest processing of S202. Is done.

  In S206, a process of counting the frequency of pixels having a luminance value equal to or higher than the reference luminance level B, which is one of the operation parameters described above, is performed from the histogram created for the maximum luminance image by the process of S204 (see FIG. 3B). The MPU of the controller 111 functions as the maximum luminance frequency counting unit 116 by executing the process of S206, and the count result at this time is referred to as the maximum luminance frequency.

  In S <b> 207, processing for calculating the progress status from the start to the end of the image capturing operation by the scanning confocal microscope 100 is performed. This progress is calculated by obtaining the value of the following equation.

[Progress status] = [Maximum luminance frequency] / [First reference pixel number] × 100 (%)
Here, the first reference pixel number is a value indicating the number of pixels constituting the omnifocal image of the sample 101, and is one of the operation parameters described above. That is, the progress situation is calculated as a ratio of the maximum luminance frequency to the first reference pixel number. For example, if the maximum luminance is obtained for all the pixels of the first reference pixel number, the maximum luminance image at that time is the omnifocal image of the sample 101, and thus the progress status is “100%”.

Note that the MPU of the controller 111 functions as the progress calculation unit 117 by executing the processing of S207.
In S208, a process of displaying the progress status calculated by the process of S207 on the output device 112 is performed. The display of the progress information on the output device 112 is performed, for example, as shown in the progress bar display shown in FIG. Visually indicated by length. Here, even if the value of the progress calculation result exceeds “100%”, the display of the progress information is terminated at “100%”.

  Instead of displaying the progress information as shown in FIG. 4A, as shown in FIG. 4B, the value of the calculation result of the current progress situation may be displayed as a numerical value. May be notified to the user.

  In S209, as the first end determination process of the measurement range, a process of determining whether or not the maximum luminance frequency acquired by the process of S206 has exceeded the above-described first reference pixel number is performed. Here, when it is determined that the maximum luminance frequency exceeds the first reference pixel number (when the determination result is Yes), the process proceeds to S210. On the other hand, when it is determined that the maximum luminance frequency does not exceed the first reference pixel number (when the determination result is No), the process proceeds to S211.

  In S210, as the second end determination process of the measurement range, a process of determining whether the luminance frequency acquired by the process of S205 exceeds a preset second reference pixel number is performed. Here, the second reference pixel number indicates whether or not the confocal image acquired by the process of S202 is sufficiently dark so that it can be considered that the confocal image is no longer used for generating the omnifocal image of the sample 101. It is a threshold value used for determination, and is one of the operation parameters described above. Here, when it is determined that the luminance frequency exceeds the second reference pixel number (when the determination result is Yes), it is determined that acquisition of the confocal image of the sample 101 is completed, and the process is performed in S212. Proceed. On the other hand, when it is determined that the luminance frequency does not exceed the second reference pixel number (when the determination result is Yes), the process proceeds to S211.

Note that the MPU of the controller 111 functions as the image capture end determination unit 118 by executing the processes of S209 and S210.
In S211, a predetermined instruction is given to the Z position control unit 104 to move the Z position of the objective lens 102 upward by the measurement pitch, and then the process returns to S202 and the above-described process is performed again. . By repeating the process of S202 in this way, confocal images for the observation surface for each height of the sample 101 are sequentially acquired over the acquisition range in the Z direction (height direction).

  In S212, the maximum luminance value for each pixel is read from the maximum luminance image memory 114 for all the pixels, and an all-in-focus image is generated and displayed on the output device 112. Thereafter, the control processing in FIG. . In the process of S212, information on the Z position stored together with the maximum luminance value for each pixel is read from the maximum luminance image memory 114, and this Z position and each pixel on the XY plane (a plane perpendicular to the Z direction) are read. The process of creating a three-dimensional image of the sample 101 and displaying it on the output device 112 may also be performed by associating with the position of.

  When the controller 111 performs the above-described control processing, an omnifocal image of the sample 101 is generated, and the progress status from the start to the end of the operation for capturing the confocal image of the sample 101 is displayed. In addition, at this time, the controller 111 automatically sets the measurement range (the upper limit position of the Z position of the objective lens 102 which is the measurement end position) manually. That is, according to the processing of FIG. 2, the display of the progress status until the end of the measurement can be provided to the user even though the end position of the measurement range is not known because it is not manually set in advance. .

  In the process of FIG. 2, not only the progress status of the omnifocal image generation work but also the progress status of the 3D image generation work of the sample 101 described above can be calculated and displayed. .

  Further, instead of displaying the progress status of the omnifocal image generation work as shown in FIGS. 4A and 4B, for example, as shown in FIG. 4C, the progress status of image capture and the progress status of image generation It is also possible to display them separately.

  Next, FIG. 5 will be described. FIG. 5 is a flowchart showing a second example of the control process performed by the controller 111. Similarly to the first example shown in FIG. 2, the second example generates the omnifocal image of the sample 101 by setting the measurement range described above by the controller 111 itself, and the confocal image of the sample 101 for that purpose. The progress status from the start to the end of the capture operation is displayed. However, the method of calculating the progress status from the start to the end of the image capturing operation by the scanning confocal microscope 100 performed by the progress status calculation unit 117 is different from the first example.

  First, the user operates the input device of the controller 111 while observing the image of the sample 101 displayed on the output device 112 to give a predetermined instruction to the Z position control unit 104, thereby changing the Z position of the objective lens 102. Move to the lower limit of the measurement range. The MPU acquires the Z position at this time as the setting of the measurement start position. In addition, the manual setting of the measurement end position (the upper limit position of the measurement range) that is conventionally performed at this time is not performed here.

Thereafter, the user operates the input device of the controller 111 to give a predetermined instruction to start the control process of FIG.
When the control process of FIG. 5 is started, first, in S301, a process of reading the above-described operation parameter from the operation parameter memory 119 is performed. The operation parameters read out by this processing are the maximum upper limit position and the maximum lower limit position that can be taken with respect to the Z position of the objective lens 102, the measurement pitch described above, two reference luminance levels used in the processing of S305 and S306 described later, and described later. The numbers of reference pixels are two in the processes of S309 and S310, respectively. These operation parameters may be fixed values set in advance, but may be set by the user and stored in the operation parameter memory 119 before the start of the control process.

  In S302, the process of capturing the confocal image of the sample 101 is performed at the current Z position, and the obtained confocal image (slice image with respect to the observation surface for each Z position of the sample 101) is associated with information indicating the Z position. Thus, processing to be stored in the confocal image memory 113 is performed. In the confocal image capturing process of the sample 101, the output signal from the light detector 107 is acquired, and the correspondence between the light amount (luminance value) represented by the acquired signal and the constituent pixels of the image is determined in two. This is performed based on the scanning position when the signal is acquired in the two-dimensional scanning of the laser beam by the two-dimensional scanning unit 108. By doing so, a confocal image of the sample 101 is obtained.

  Subsequent to S302, in S303, a process for creating a maximum luminance image is performed. In this process, the brightness value of the confocal image newly stored in the confocal image memory 113 and the brightness value of the image already stored in the maximum brightness image memory 114 (each of the already acquired slice images are configured. A comparison is made for each pixel at the same position. Here, for a pixel having a higher luminance value in the confocal image, the luminance value of the pixel is updated to that of the confocal image and stored in the maximum luminance image memory 114. Therefore, when the maximum luminance values stored in the maximum luminance image memory 114 are arranged for each pixel, a maximum luminance image is obtained. If the update of the maximum luminance image is repeated for the confocal image with respect to the observation plane for each Z position of the sample 101 acquired over the acquisition range in the Z direction, the maximum luminance image becomes the omnifocal image of the sample 101. Become.

  When the brightness value stored in the maximum brightness image memory 114 is updated, information indicating the Z position at the time of capturing the confocal image from which the brightness value is obtained is also stored in the maximum brightness image memory 114 for each pixel. Try to remember.

  In S304, for each of the confocal image stored in the confocal image memory 113 in the processing of S302 and the maximum luminance image stored in the maximum luminance image memory 114 in the processing of S303, the luminance value and the number of pixels of the constituent pixels. A process for creating a histogram as shown in FIG. 3A and FIG. 3B showing the relationship between and in the same manner as in the first example is performed.

  In S305, a process of counting the frequency of pixels having a luminance value equal to or lower than the reference luminance level A, which is one of the operation parameters described above, is performed from the histogram created for the confocal image in S304 (see FIG. 3A). The MPU of the controller 111 functions as the luminance frequency counting unit 115 by executing the process of S305, and the count result at this time is referred to as the luminance frequency for the confocal image acquired by the latest process of S302. Is done.

  In S306, processing is performed to count the frequency of pixels having a luminance value equal to or higher than the reference luminance level B, which is one of the operation parameters described above, from the histogram created for the maximum luminance image by the processing in S304 (see FIG. 3B). The MPU of the controller 111 functions as the maximum luminance frequency counting unit 116 by executing the process of S306, and the counting result at this time is referred to as the maximum luminance frequency.

  In S307, processing for calculating the progress status from the start to the end of the image capturing operation by the scanning confocal microscope 100 is performed. This progress is calculated by obtaining the value of the following equation.

[Progress status A] = [maximum luminance frequency] / [first reference pixel number] × 50 (%)
[Progress status B] = [Luminance frequency] / [Second reference pixel number] × 50 (%)
[Progress] = [Progress A] + [Progress B] (%)
Here, the first reference pixel number is a value indicating the number of pixels constituting the omnifocal image of the sample 101, and is one of the operation parameters described above. In addition, the second reference pixel number is determined when the confocal image acquired by the process of S302 is sufficiently dark that it can be considered that the confocal image is no longer used for generating the omnifocal image of the sample 101. Luminance frequency, one of the operating parameters described above.

Note that the MPU of the controller 111 functions as the progress calculation unit 117 by executing the processing of S307.
In S308, a process of displaying the progress status calculated by the process of S307 on the output device 112 is performed. As in the first example described above, the progress information output device 112 displays the current progress calculation result value as in the progress bar display shown in FIG. Visually shown relative to “100%” indicating completion. Here, even if the value of the progress calculation result exceeds “100%”, the display of the progress information is terminated at “100%”. As shown in FIG. 4B, the value of the current progress calculation result may be displayed as a numerical value, and further, the numerical value may be notified to the user together with the display.

  In S309, as the first end determination process of the measurement range, a process of determining whether or not the maximum luminance frequency acquired by the process of S306 has exceeded the above-described first reference pixel number is performed. Here, when it is determined that the maximum luminance frequency exceeds the first reference pixel number (when the determination result is Yes), the process proceeds to S310. On the other hand, when it is determined that the maximum luminance frequency does not exceed the first reference pixel number (when the determination result is No), the process proceeds to S311.

  In S310, as the second end determination process of the measurement range, a process of determining whether or not the luminance frequency acquired by the process of S305 exceeds the above-described second reference pixel number is performed. Here, when it is determined that the luminance frequency exceeds the second reference pixel number (when the determination result is Yes), it is determined that the acquisition of the confocal image of the sample 101 is completed, and the process proceeds to S312. Proceed. On the other hand, when it is determined that the luminance frequency does not exceed the second reference pixel number (when the determination result is Yes), the process proceeds to S311.

Note that the MPU of the controller 111 functions as the image capture end determination unit 118 by executing the processes of S309 and S310.
In S311, a predetermined instruction is given to the Z position control unit 104 to move the Z position of the objective lens 102 upward by the measurement pitch, and then the process returns to S302 and the above-described process is performed again. . By repeating the processing of S302 in this manner, confocal images for the observation surface for each height of the sample 101 are sequentially acquired over the acquisition range in the Z direction (height direction).

  In S312, the maximum brightness value for each pixel is read from the maximum brightness image memory 114 for all pixels, an all-in-focus image is generated and displayed on the output device 112, and then the control process in FIG. 5 is terminated. . In the processing of S312, information on the Z position stored together with the maximum luminance value for each pixel is read from the maximum luminance image memory 114, and the Z position and each pixel on the XY plane (plane perpendicular to the Z direction) are read. The process of creating a three-dimensional image of the sample 101 and displaying it on the output device 112 may also be performed by associating with the position of.

  When the controller 111 performs the above-described control processing, an omnifocal image of the sample 101 is generated, and the progress status from the start to the end of the operation for capturing the confocal image of the sample 101 is displayed. In addition, at this time, the controller 111 automatically sets the measurement range (the upper limit position of the Z position of the objective lens 102 which is the measurement end position) manually. That is, according to the processing of FIG. 5, the display of the progress status until the end of the measurement can be provided to the user even though the end position of the measurement range is not known because it is not manually set in advance. .

  In the process of FIG. 5, not only the progress status of the omnifocal image generation work but also the progress status of the 3D image generation work of the sample 101 described above can be calculated and displayed. .

  Further, instead of displaying the progress status of the omnifocal image generation work as shown in FIGS. 4A and 4B, for example, as shown in FIG. 4C, the progress status of image capture and the progress status of image generation It is also possible to display them separately.

  In the processing of FIG. 5, a weight (50:50) equal to the value of the progress status A and the value of the progress status B, which is the basis for calculating the progress status, is given. Depending on the case where one of them is emphasized in the acquisition end determination, the weight given to the value of progress status A and the value of progress status B may be different.

  In the process of FIG. 5, instead of displaying the progress status on the output device 112, the progress status A and the progress status B may be displayed separately on the output device 112. Can be provided.

Note that the embodiment of the present invention is not limited to the scanning confocal microscope system, and can be implemented in, for example, a magnification observation microscope such as a video microscope microscope.
Here, FIG. 6 will be described. FIG. 6 shows the configuration of a video microscope microscope system according to the second embodiment of the present invention.

  The video microscope microscope 400 includes an objective lens 402, an objective lens switching unit 403, a Z position control unit 404, a half mirror 405, a photodetector 406, and an LED illumination unit 407. The video microscope microscope 400, the controller 408, and the output device 409 constitute a video microscope microscope system.

  The objective lens switching unit 403 switches the objective lens 402 disposed on the optical axis of the video microscope microscope 400. The Z position control unit 404 controls the Z position of the objective lens 402 (the position in the optical axis direction of the video microscope microscope 400). The photodetector 406 is an image sensor using, for example, a CCD (charge coupled device), detects light, and outputs a signal indicating the amount of light. The LED illumination unit 407 emits an LED (light emitting diode) and irradiates the sample (observation sample) 401 with the light. The controller 408 acquires an observation image of the sample 401 and Z position information of the objective lens 402. The output device 409 displays the acquired observation image of the sample 401 and the like.

  The light emitted from the LED illumination unit 407 passes through the half mirror 405 and then passes through the objective lens 402 to illuminate the sample 401. The reflected light from the surface of the sample 401 at this time passes again through the objective lens 402 and is then reflected by the half mirror 405 to form an image of a predetermined height of the sample 401 on the light receiving surface of the photodetector 406. Image. The light detector 406 detects this light and outputs a signal indicating the amount of light. Based on this signal, an image (slice image) obtained by slicing the sample 401 with a plane perpendicular to the optical axis direction is obtained optically.

  Here, the Z position control unit 404 drives the objective lens 402 in the optical axis direction, and sequentially acquires slice images at each Z position while changing the relative distance of the objective lens 402 with respect to the sample 401. An omnifocal image or a three-dimensional image can be obtained by extracting the maximum luminance at the same pixel for each pixel from the acquired slice images. The movement range of the Z position at this time (change range of the relative distance of the objective lens 402 with respect to the sample 401) is referred to as a measurement range.

  The controller 408 includes an MPU (arithmetic processing unit), a main memory, an input device, an interface device, and a storage device, and an output device 409 is connected thereto. Note that the MPU, main memory, input device, interface device, and storage device are not shown in FIG.

  The MPU performs operation control of the entire video microscope microscope system of FIG. 6 by executing a control program stored in a storage device, for example, and at least the maximum brightness value search unit 411, by executing the control program. It functions as a maximum luminance counting unit 412, a progress situation calculating unit 414, and an image capture end determining unit 415.

The main memory is a semiconductor memory used as a work memory by the MPU as needed, and functions as at least the image memory 410 and the operation parameter memory 413.
The input device is a mouse device or a keyboard device, for example, and acquires various instructions from the user as inputs.

The interface device manages the exchange of various data with each component of the video microscope microscope system.
The storage device is, for example, a hard disk device, and stores various control programs and data.

The output device 409 is a display device, for example, and displays an operation screen, an observation image, and the like under the control of the MPU.
The image memory 410 stores a slice image of the sample 401 generated by the MPU based on the signal output from the photodetector 406.

The maximum luminance value search unit 411 searches for the maximum luminance value for each pixel from the luminance value of the slice image acquired most recently and the luminance value of the slice image already acquired at that time.
The maximum brightness counting unit 412 counts the number of pixels that have found the maximum brightness value.

The operation parameter memory 413 stores various operation parameters used in the automatic measurement operation by the video microscope microscope system.
The progress status calculation unit 414 calculates the progress status from the start to the end of the image capturing operation by the video microscope microscope 400.

The image capturing end determination unit 415 determines whether to end the image capturing operation by the video microscope microscope 400.
The video microscope microscope system shown in FIG. 6 is configured as described above.

  Next, control processing performed by the controller 408 shown in FIG. 6 will be described. In this control process, the controller 408 itself sets the above-described measurement range to generate an omnifocal image of the sample 401, and displays the progress status from the start to the end of the slice image capturing operation for the sample 401 for that purpose. . In the controller 408, the MPU reads out and executes a control program stored in advance in the storage device, thereby realizing the control process described below.

Here, FIG. 7 will be described. FIG. 7 is a flowchart showing the control process performed by the controller 408.
First, the user operates the input device of the controller 408 to give a predetermined instruction to the Z position control unit 404 while observing the image of the sample 401 displayed on the output device 409, thereby changing the Z position of the objective lens 402. Move to the lower limit of the measurement range. The MPU acquires the Z position at this time as the setting of the measurement start position. In addition, the manual setting of the measurement end position (the upper limit position of the measurement range) that is conventionally performed at this time is not performed here.

Thereafter, the user operates the input device of the controller 408 to give a predetermined instruction to start the control process of FIG.
When this control process is started, first, in S501, a process of reading the above-described operation parameter from the operation parameter memory 413 is performed. The operation parameters read out by this processing are the maximum upper limit position and the maximum lower limit position that can be taken for the Z position of the objective lens 402, the above-described measurement pitch, the set number of slice images used in the processing of S503 described later, and S506 described later. These are the two reference pixel numbers used in the above process. These operation parameters may be fixed values set in advance, but may be set by the user and stored in the operation parameter memory 413 before the start of the control process.

  In S502, the slice image capturing process of the sample 401 is performed at the current Z position, and the obtained slice image (slice image with respect to the observation surface for each Z position of the sample 401) is associated with information indicating the Z position. Processing to be stored in the memory 410 is performed. In addition, in the process of acquiring the slice image of the sample 401, the output signal from the photodetector 406 is acquired, and the light amount (luminance value) represented by the acquired signal is associated with the constituent pixels of the image. Is performed based on the position on the light receiving surface of the photodetector 406 that has received the light. By doing so, a slice image of the sample 401 is obtained.

  Subsequent to S502, in S503, search processing for the maximum luminance value is performed. In this process, the maximum luminance value is searched for each pixel from a plurality of slice images stored in the image memory 410, and the number of pixels for which the maximum luminance value has been found is counted. Here, the search for the maximum luminance value for each pixel starts with the most recently acquired slice image and the most recent slice number set in the operation parameter memory 413 among the plurality of slice images already acquired at this time. For each pixel of the minute image, a luminance change curve is created for each pixel at the same position, with the Z position as the horizontal axis and the luminance value as the vertical axis. Then, a change in the slope of the brightness change curve is obtained, and if the sign of the slope is inverted, the brightness value at that time is obtained as the maximum brightness value.

Note that the MPU of the controller 408 functions as the maximum luminance value searching unit 411 and the maximum luminance counting unit 412 by executing the processing of S503.
In S504, processing for calculating the progress status from the start to the end of the image capturing operation by the video microscope microscope 400 is performed. This progress is calculated by obtaining the value of the following equation. This progress is calculated by obtaining the value of the following equation.
[Progress status] = [Number of pixels for which maximum luminance value was found] / [Third reference pixel number]
× 100 (%)
Here, the third reference pixel number indicates that the acquisition of the slice image of the sample 401 has been completed when the maximum luminance value can be found by using what percentage of the pixels constituting the slice image of the sample 401. It is a value that determines whether or not to make a determination, and is one of the operation parameters described above.

Note that the MPU of the controller 408 functions as the progress calculation unit 414 by executing the processing of S504.
In S505, a process of displaying the progress status calculated by the process of S504 on the output device 409 is performed. The display of the progress information on the output device 409 is the same as the display of the progress information on the output device 409, as in the case of the scanning confocal microscope system described above, such as the progress bar display shown in FIG. 4A. The result of the calculation of the progress status is visually shown as a relative length to “100%” indicating the completion of the generation of the omnifocal image. Here, even if the value of the progress calculation result exceeds “100%”, the display of the progress information is terminated at “100%”. As shown in FIG. 4B, the value of the current progress calculation result may be displayed as a numerical value, and further, the numerical value may be notified to the user together with the display.

  In S506, as the third end determination process of the measurement range, a process of determining whether or not the number of pixels for which the maximum luminance value acquired by the process of S503 has been found exceeds the above-described third reference pixel number is performed. . If it is determined that the number of pixels for which the maximum luminance value has been found has exceeded the third reference pixel number (when the determination result is Yes), the process proceeds to S508. On the other hand, when it is determined that the number of pixels for which the maximum luminance value has been found does not exceed the third reference pixel number (when the determination result is No), the process proceeds to S507. Note that the MPU of the controller 408 functions as the image capture end determination unit 415 by executing the processing of S506.

  In S507, a predetermined instruction is given to the Z position control unit 404 to move the Z position of the objective lens 402 upward by the measurement pitch, and then the process returns to S502 and the above-described process is performed again. . By repeating the processing in S502 in this way, slice images for the observation surface for each height of the sample 401 are sequentially acquired over the acquisition range in the Z direction (height direction).

  In step S <b> 508, the maximum luminance value for each pixel is acquired for all pixels from the slice image acquired up to the execution of this process and stored in the image memory 410, and an all-focus image is created and displayed on the output device 409. Processing is performed, and thereafter, the control processing of FIG. 7 is terminated. In the processing of S508, information on the Z position stored together with the maximum luminance value for each pixel is read from the image memory 410, and this Z position and the position of each pixel on the XY plane (a plane perpendicular to the Z direction). , A process of creating a three-dimensional image of the sample 401 and displaying it on the output device 409 may be performed together.

  When the controller 408 performs the above-described control processing, an omnifocal image of the sample 401 is generated, and the progress status from the start to the end of the slice image capturing operation for the sample 401 is displayed. In addition, at this time, the controller 408 automatically sets the measurement range (the upper limit position of the Z position of the objective lens 402 that is the measurement end position) without manually setting the measurement range. That is, according to the processing of FIG. 7, the display of the progress status until the end of the measurement can be provided to the user even though the end position of the measurement range is not known because it is not manually set in advance. .

  In the process of FIG. 7, not only the progress status of the omnifocal image generation work but also the progress status of the 3D image generation work of the sample 401 described above can be calculated and displayed. .

  As described above, according to any of the embodiments of the present invention, in the magnifying observation microscope that creates the omnifocal image and the three-dimensional image by determining the capturing range of the slice image with respect to the observation surface for each height of the observation sample. It is possible to calculate and provide the user with the progress status until the end of image capture for generating an omnifocal image or a three-dimensional image. Therefore, for example, when the measurement time is long, the user may feel uncomfortable, such as anxiety about whether the measurement is in progress or not knowing when the measurement is finished.

As mentioned above, although embodiment of this invention was described, this invention is not limited to each embodiment mentioned above, A various improvement and change are possible within the range which does not deviate from the summary of this invention.
For example, in each embodiment of the present invention, it is possible to display the progress status until the end of all measurements to the user by calculating and displaying the progress status of the generation of the omnifocal image or 3D image. It is.

It is a figure which shows the structure of the scanning confocal microscope system which is 1st embodiment of this invention. It is the figure which showed the processing content of the 1st example of the control processing performed by the controller shown by FIG. 1 with the flowchart. It is a figure which shows the example of the histogram about a confocal image. It is a figure which shows the example of the histogram about a maximum luminance image. It is a figure which shows the example of a display of a progress condition (the 1). It is a figure which shows the example of a display of a progress condition (the 2). It is a figure which shows the example of a display of a progress condition (the 3). It is the figure which showed the processing content of the 2nd example of the control processing performed by the controller shown by FIG. 1 with the flowchart. It is a figure which shows the structure of the video micro microscope system which is 2nd embodiment of this invention. It is the figure which showed the processing content of the control processing performed by the controller shown by FIG. 6 with the flowchart. It is a figure which shows the outline of a structure of a scanning confocal microscope. It is a figure which shows the shape of a non-planar sample.

Explanation of symbols

100 Scanning confocal microscope 101, 401, 804, 900 Sample 102, 402, 803 Objective lens 103, 403 Objective lens switching unit 104, 404 Z position controller 105, 405, 802 Half mirror 106, 805 Pinhole plate 107, 406, 806 Photo detector 108 Two-dimensional scanning unit 109 Mirror 110 Laser illumination unit 111, 408 Controller 112, 409 Output device 113 Confocal image memory 114 Maximum luminance image memory 115 Luminance frequency counting unit 116 Maximum luminance frequency counting unit 117, 414 Progress status calculation unit 118, 415 Image capture end determination unit 119, 413 Operation parameter memory 400 Video microscope microscope 407 LED illumination unit 410 Image memory 411 Maximum brightness value search unit 412 Maximum brightness count unit 8 1 point-like light source

Claims (7)

Slice image acquisition means for sequentially acquiring slice images for the observation surface for each height of the observation sample over an acquisition range in the height direction;
The luminance value of the pixels constituting the slice image newly acquired by the slice image acquisition means and the maximum luminance value of the pixels at the same position as the pixel and constituting each of the already acquired slice images By updating the maximum luminance image obtained by setting the larger value as the luminance value of the pixel at the position, every time the slice image is newly acquired, the omnifocal image of the observation sample is obtained. Omnifocal image creation means to create,
An end determination unit that determines the end of acquisition of the slice image over the acquisition range based on the slice image acquired by the slice image acquisition unit most recently;
Progress status calculation for calculating a value indicating a progress status of acquisition of the slice image up to the end of acquisition over the acquisition range based on a value indicating a creation status of the omnifocal image by the omnifocal image creating means Means,
Output means for outputting the progress status indicated by the value calculated by the progress status calculation means;
A microscope apparatus characterized by comprising: The value indicating the creation status of the omnifocal image by the omnifocal image creation means is the frequency of the pixels constituting the maximum brightness image whose brightness is equal to or higher than a predetermined reference brightness level
The progress calculation means calculates a ratio of the frequency to a predetermined first reference pixel number as a value indicating the progress;
The microscope apparatus according to claim 1.   The microscope apparatus according to claim 1, wherein the progress status calculation unit calculates a value indicating the progress status based further on a luminance value of a slice image most recently acquired by the slice image acquisition unit. The value indicating the creation status of the omnifocal image by the omnifocal image creation means is the frequency of the pixels constituting the maximum brightness image whose brightness is equal to or higher than a predetermined reference brightness level
The progress status calculation means has a ratio of the frequency to a predetermined first reference pixel number and a luminance of a pixel constituting a slice image most recently acquired by the slice image acquisition means is not higher than a predetermined reference luminance level. Calculating the sum of the frequency and the ratio to a predetermined second reference pixel number as a value indicating the progress status;
The microscope apparatus according to claim 3.   The end determination unit determines that the frequency of the pixels constituting the slice image most recently acquired by the slice image acquisition unit is less than a predetermined reference luminance level exceeds a predetermined second reference pixel number. The microscope apparatus according to claim 1, wherein the acquisition is determined to be completed. Slice image acquisition means for sequentially acquiring slice images for the observation surface for each height of the observation sample over an acquisition range in the height direction;
Each time the slice image is newly acquired, the inclination of the luminance value change of the pixel at the same position constituting each acquired slice image is obtained, and the maximum luminance value in the pixel is changed to the inclination change. Omnifocal image creation means for creating an omnifocal image of the observation sample by finding out for each pixel based on the maximum value of the found luminance as the luminance value of each pixel;
End determination means for determining the end of acquisition of the slice image over the acquisition range by the slice image acquisition means based on the number of pixels in which the maximum value of the luminance is found;
The value indicating the progress of acquisition of the slice image up to the current time until the end of acquisition of the slice image over the acquisition range by the slice image acquisition means is set to the number of pixels where the maximum value of the luminance is found. Progress calculation means to calculate based on;
Output means for outputting the progress status indicated by the value calculated by the progress status calculation means;
A microscope apparatus characterized by comprising:   The said progress condition calculation means calculates the ratio of the number of the pixels in which the maximum value of the said brightness | luminance was found with respect to the predetermined | prescribed reference pixel number as a value which shows the said progress condition. Microscope device.