JP2006145793A - Microscopic image pickup system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a microscopic image pickup system switchable whether a high resolution or a high frame rate according to the conditions at the time of observation. <P>SOLUTION: The microscopic image pickup system is equipped with: a movable stage 7 on which a sample is placed; an optical system which makes macroobservation of the sample; an image pickup means 8 picking up an optical image by the optical system; and a control means 9 which controls the resolution of the image picked up by the image pickup means on the basis of the movement of the stage relative to the optical system by the stage moving means. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a microscope image capturing system that acquires a microscope image using an imaging unit in a microscope.

In recent years, there have been an increasing number of imaging means such as CCD cameras and CMOS cameras that can acquire images with a large number of pixels and high resolution. There are also various interface means for transferring image data from the image pickup means to a device such as a personal computer. In particular, many techniques for performing high-speed transfer using a digital interface have been put into practical use. Recently, a high-resolution microscope image is often obtained by using such an imaging means for a microscope, and some have a resolution of 5 million pixels, 10 million pixels or more.
JP-A-6-217180

  However, when using a digital camera or the like having a remarkably large number of pixels, it does not always have a sufficient transfer speed (frame rate). For example, when transferring image data from a CCD camera with 5 million pixels. Only about 5 frames per second can be transferred. Moreover, when it reaches 10 million pixels, it becomes 3 frames or less per second. When a digital camera having a large number of pixels and a low frame rate is used as an imaging means for a microscope, there are the following problems.

  First of all, when displaying a captured image, the frame rate is slow, so that the follow-up to the movement of the optical system or sample is poor. This makes it difficult to focus the microscope and to move the sample to specify the place to be observed in the sample, and the observer feels stress.

  Next, in order to solve the above problems, a camera capable of transferring image data at high speed and increasing the frame rate by cutting out and transferring only a part of the captured image has been invented and put into practical use. However, even if such a camera is used, only a partial image can be obtained and the entire image cannot be seen.

  Further, as another means, there is a method of thinning out the entire image uniformly to reduce the image data and transferring it at high speed so that the entire image can be viewed at high speed. However, even if the image obtained in this way is aligned with the microscope pin, it may not be in focus if a still image captured with the full number of pixels of the camera is acquired in that state. . Therefore, the characteristics of a camera that can obtain a high resolution image with a large number of pixels cannot be utilized.

  In view of the above problems, in the present invention, switching between high-resolution imaging and low-resolution imaging is performed according to the observation conditions, thereby controlling the speed of the frame rate of the image signal output from the imaging means. A microscope imaging system is provided.

  According to the first aspect of the present invention, there is provided a movable stage on which a sample is placed, an optical system for magnifying and observing the sample, and an optical image by the optical system. A microscope image imaging system comprising: an imaging unit that captures an image; and a control unit that controls a resolution of an image captured by the imaging unit based on a movement of the stage relative to the optical system. Can be achieved by providing.

  According to the second aspect of the present invention, the image pickup means can change the resolution of an image obtained by image pickup under the control of the control means. This can be achieved by providing a microscope imaging system according to claim 1.

  According to the third aspect of the present invention, the microscope image capturing system further acquires a focus instruction information for moving the sample to a focus position. Provided with information acquisition means, and when the focus instruction information acquisition means acquires the focus instruction information, the control means compares the image of the image as compared with the case where the focus instruction information is not acquired. It can achieve by providing the microscope image pickup system according to claim 2, wherein the image pickup means is controlled so as to have a relatively high resolution.

  According to the fourth aspect of the present invention, the microscope image pickup system further evaluates whether or not the microscope image pickup system is focused based on the image picked up by the image pickup means. It can achieve by providing the microscope image pick-up system of Claim 1 characterized by including the focus evaluation means to do.

  According to the fifth aspect of the present invention, the imaging means captures an image of a predetermined region in the range of images that can be captured, and the focus evaluation means captures the captured image. This can be achieved by providing the microscope image pickup system according to claim 4 that evaluates whether or not the in-focus state is based on an image of a predetermined area.

  According to the sixth aspect of the present invention, the microscope image capturing system further detects whether or not the sample is moving relative to the optical system. The microscope image capturing system according to claim 1, further comprising a movement detection unit, wherein the control unit controls a resolution of an image captured by the imaging unit based on a detection result by the movement detection unit. Can be achieved by providing.

  According to the seventh aspect of the present invention, when the movement detection unit detects that the sample is moving relative to the optical system, 7. The microscope image pickup system according to claim 6, wherein the control unit controls the image pickup unit so that the resolution of the image is relatively lowered as compared with the case where the image is not moved. Can be achieved.

  According to the eighth aspect of the present invention, when the stage is moved relatively in a direction perpendicular to the optical axis direction of the optical system, the control unit moves the stage. The microscopic image imaging system according to claim 1, wherein the imaging unit is controlled so that the resolution of the image is relatively low as compared with the case where the image is not made.

  According to the ninth aspect of the present invention, when the stage is moved relatively in the optical axis direction of the optical system, the control means is compared with a case where the stage is not moved. Then, the image pickup means is controlled so that the resolution of the image is relatively low. This can be achieved by providing the microscope image pickup system according to claim 1.

  According to the imaging system for a microscope of the present invention, when a high pixel camera is used for a microscope, by switching between high resolution or high frame rate according to the observation conditions, observation and focusing, The sample can be positioned under better conditions, and comfortable and optimal microscope observation can be performed without giving stress to the observer.

<First Embodiment>
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows a microscope image capturing system in which an imaging apparatus is combined with a microscope optical system according to this embodiment. Illumination light emitted from the illumination light source 1 is applied to the surface of the sample 5 via the epi-illumination optical system 2, the half mirror 3, and the objective lens 4, and the reflected light from the sample 5 is again applied to the objective lens 4, the half mirror 3, and the coupled light. The camera 8 is reached via the image optical system 6.

  The sample 5 is placed on a stage 7 that can move in XY directions, and the sample can be moved to a desired observation position. The objective lens 4 can be moved electrically in the direction of the optical axis by the drive unit 12 to thereby focus. Other focusing methods include moving the stage up and down and moving the entire optical system in addition to the objective lens.

  The camera 8 captures a microscope image and outputs an image signal. The image signal is captured by the camera control unit 9 and stored in an image memory (not shown) or displayed on the display unit 10 to be used for observation, analysis, image storage, or the like.

  The image signal is sent to the focus evaluation unit 11 simultaneously with the camera control unit 9, and the focus evaluation unit 11 contrast autofocus (hereinafter, autofocus is abbreviated as AF) based on the obtained image (for example, AF). In-focus evaluation is performed using the hill-climbing AF method), and a focus evaluation value indicating how much in-focus is shifted in which direction is output to the drive unit 12. This focus evaluation value is updated every time the image is updated.

  The drive unit 12 repeats the focusing operation based on the focus evaluation value, and drives the objective lens 4 until the focus position is reached or always follows the focus position. The focusing operation is started and stopped by the focusing instruction unit 13, but the focusing instruction signal is output to the focusing evaluation unit 11, and at the same time as the focusing operation is started and stopped, the camera control It is also output to part 9.

  Upon receipt of the focus instruction signal, the camera control unit 9 instructs the camera 8 on the resolution of the image signal to be output, and the camera 8 outputs an image at the resolution of the instructed image signal. In the present embodiment configured as described above, the resolution of the image output from the camera 8 is changed between the normal observation and the focusing operation.

  When observing a microscope image, the sample is moved using the XY stage 7 until the observation position of the sample is determined. In this case, it is desirable that the frame rate is fast. Generally, about 20 to 30 frames per second are required although it depends on the use scene. Here, the relationship between the resolution and the frame rate will be described with reference to FIG.

  FIG. 2 shows an image (a) captured at high resolution and an image (b) obtained by thinning out pixel data from this image. In the figure, for example, an image (FIG. 2A) captured by a camera of 5 million pixels and 5 frames per second (FIG. 2 (a)) is thinned out in half in each of the horizontal direction and the vertical direction (1 pixel in 2 pixels in each direction (FIG. 2 (b)), the amount of data becomes ¼ and the frame rate becomes about four times, so that 20 frames per second can be realized.

  As described above, if the camera can switch between a mode that outputs everything without thinning out and a mode that thins out to 1/4, when performing normal observation, a mode with a high frame rate is used in the thinning mode to 1/4. The captured image is displayed on the display unit 10. However, even if it is positioned in the X-Y direction in that state and captured in a mode that outputs all the still images to be saved without thinning out, it will not notice that the image is out of focus. There may not be.

  Therefore, in this embodiment, when an observer instructs a focusing operation using the focusing instruction unit 13, a focusing instruction signal is output from the focusing instruction unit 13 to the focusing evaluation unit 11 and the camera control unit 9. The The camera control unit 9 transmits a camera control signal to the camera 8 in order to switch to a mode in which a mode for thinning out the observation for ¼ is switched to a mode that outputs all without receiving a focus instruction signal. Is done.

  The camera 8 receives the camera control signal and switches the output mode. The mode that has been thinned to 1/4 for observation until then becomes a mode that outputs all without thinning. Since the focus evaluation unit 11 performs contrast AF (mountain climbing AF) based on image data from which all pixels are output, it is possible to perform focus evaluation and focus operations with higher accuracy than during observation.

  3 to 5 show a series of processing in the present embodiment. FIG. 3 shows the operation of the operator. 4 and 5 show processing of the microscope image capturing system for each operation of FIG. In FIG. 3, first, the observer moves the stage 7 on which the sample 5 is placed using an operation unit (not shown) of the input unit (step 1, hereinafter, “step” is referred to as “S”). At this time, the operator confirms with the display unit 10 that the desired observation position of the observation sample 5 intersects the optical axis (for example, the observation position is positioned at the center of the display screen of the display unit 10). However, the stage 7 is moved.

When the desired observation position of the observation sample 5 intersects the optical axis, the operator inputs focusing instruction information using the focusing instruction unit 13 to focus (S2).
FIG. 4 shows details of processing on the microscopic image capturing system side in S1 of FIG. First, the camera control unit 9 sets the image signal to be output to the camera 8 so as to be thinned to 1 / n (n is an arbitrary positive number) (hereinafter referred to as a low resolution mode) (S11). The camera 8 captures an image in the low resolution mode, and the image signal is captured by the camera control unit 9 and displayed on the display unit 10 (S12). At this time, since the image signal is thinned out, it can be output from the camera 8 at a high frame rate, so that the displayed image can follow the movement of the stage. Imaging is performed in the low-resolution mode until focusing instruction information is input.

  FIG. 5A shows details of the processing on the microscope image capturing system side in S2 of FIG. First, when the operator inputs focus instruction information through the focus instruction unit 13, a focus instruction signal is output from the focus instruction unit 13 to the focus evaluation unit 11 and the camera control unit 9 (S21). Then, the camera control unit 9 transmits a camera control signal to the camera 8.

  Upon receiving this camera control signal, the camera 8 switches from the mode of thinning to 1 / n for observation (low resolution mode) to the mode of outputting all without thinning (hereinafter referred to as high resolution mode) (S22).

  Then, an image is captured in the high resolution mode (S23). The image signal output by the camera 8 is transmitted to the focus evaluation unit 11. Then, the stage 7 is moved Z1 in the optical axis direction (S24). After the stage 7 is moved, an image is captured again in the high resolution mode (S25), and the image signal output by the camera 8 is transmitted to the focus evaluation unit 11.

  The focus evaluation unit 11 compares the contrast value of the image received in S25 (the frame image of S25) with the contrast value of the image captured in S23 (the frame image of S23) (S26). If the contrast value of the frame image in S25 ≧ the contrast value of the frame image in S23 (proceeds to “Yes” in S26), the movement amount ΔZ in the optical axis direction is set to “Z1” (S27). If the contrast value of the frame image in S25 <the contrast value of the frame image in S23 (goes to “No” in S26), the movement amount ΔZ in the optical axis direction is set to “−Z1” (S28). . Then, the stage 7 is moved in the optical axis direction by the movement amount ΔZ by controlling the drive unit 12 (S29).

Next, an image is captured in the high resolution mode (S291). The image signal output by the camera 8 is transmitted to the focus evaluation unit 11.
The focus evaluation unit 11 compares the contrast value of the currently received image (current frame image) with the contrast value of the previously captured image (previous frame image) (S292). In the case where the contrast value of the current frame image ≧ the contrast value of the previous frame image (proceed to “Yes” in S292), the stage 7 is moved slightly in the optical axis direction by controlling the drive unit 12 (ΔZ). The image is moved (S293), and the process of S291 to S293 is repeated while the contrast value of the current frame image ≧ the contrast value of the previous frame image.

  If the contrast value of the current frame image is smaller than the contrast value of the previous frame image (goes to “No” in S292), the stage 7 is returned to the imaging position of the previous frame image (S294). ).

  The image captured in the high resolution mode in this way is accumulated in an image memory (not shown) or displayed on the display unit 10 to be used for observation, analysis, image storage, or the like.

  FIG. 5B shows a change in contrast value as the stage 7 moves in FIG. 5A. The vertical axis represents the contrast value of the captured image. The higher the vertical axis, the higher the contrast value. The horizontal axis indicates the amount of movement of the stage 7 in the optical axis direction. It is shown that the stage 7 moves upward as the horizontal axis moves to the right. p1 to p6 indicate positions captured by moving the stage 7 respectively.

  The case of passing through S27 in FIG. 5A is, for example, a case where the stage moves in one direction toward the apex p5 (focus point), such as p3 and p4, when the initial stage position is p2.

  The case of passing through S28 in FIG. 5A is a case where, for example, when the first stage position is p2, the subsequent stage 7 moves to p1 in the direction opposite to the in-focus position. In this case, in order to change the moving direction of the stage 7, a negative value is substituted for the moving amount ΔZ. Thereby, p2-> p1-> p2-> p3-> ... and the stage 7 can move toward the in-focus position (moves along the arrow in FIG. 5B).

  As described above, according to the present embodiment, it is possible to observe at a comfortable frame rate without giving stress to the observer during normal observation, and when performing autofocus (contrast AF) using a camera image, Since the maximum resolution of the camera is used, focusing can be performed with high accuracy.

  FIG. 6 is a diagram for explaining a modification of the present embodiment. In order to perform in-focus evaluation at high speed, an image of the entire area (see FIG. 6A) is not output at the time of in-focus evaluation. For example, as shown in FIG. An image of only the region is output, and the above-described focus evaluation is performed on this region. When the in-focus position is reached, an image of the entire area at that position is obtained with high resolution. In this way, since only a part of the high resolution image is used at the time of focusing, it is possible to perform image transfer at high speed while being a high resolution image. This modification is effective when it is only necessary to focus on a predetermined area in the observed image (when the area to be noticed is limited).

  As for the interface between the camera and the camera control unit, the image signal and the control signal may have different forms or may be combined with the same interface. Typical examples of the different ones include an analog interface, a digital interface such as LVDS, a camera link, and the like. Typical examples of those that also serve as the same interface include IEEE 1394 and USB. This embodiment does not limit these interfaces, and any interface can be used as long as the same effect can be obtained.

<Second Embodiment>
Next, a second embodiment of the present invention will be described with reference to the drawings.
FIG. 7 is a microscope image capturing system in the present embodiment. The same components as those in the first embodiment are denoted by the same reference numerals, and description thereof is omitted. An image signal of the camera 8 is captured by the camera control unit 9 and stored in an image memory (not shown) or displayed as an image on the display unit 10 to be used for observation or analysis or image storage.

  The image signal is also taken into the movement detection unit 21 to detect a relative movement between the stage 7 on which the sample 5 is placed and the objective lens 4 (or the entire optical system). This movement detection by the movement detection unit 21 is performed by image processing. For movement detection by this image processing, a method of simply detecting a change in luminance value, a method of pattern recognition, or the like can be considered. As a method of detecting a change in luminance value, for example, a method of acquiring a luminance value difference between frame images and determining that the stage is moving when the difference exceeds a threshold value can be considered. As a pattern recognition method, for example, after recognizing the contour of each object existing in a certain frame image, the stage moves depending on whether or not the coordinates of the contour of the object have changed between frame images. What is determined to be

  When the movement detection unit 21 detects the relative movement between the sample 5 and the objective lens 4 (or the entire optical system), the movement detection unit 21 outputs a control signal to the camera control unit 9. Control the resolution.

  In the present embodiment configured as described above, the resolution of the image output from the camera 8 is changed depending on whether the sample is stopped or moved with respect to the optical system. The case where the sample 5 is moved with respect to the optical system is, for example, a case where focusing is performed, in which case the sample 5 moves in the Z direction. In another case, the observation position of the sample is searched. In this case, the sample moves in the XY direction. In any case, it is desired that the frame rate of the camera 8 be fast because the image is desired to follow faster while moving.

  When the sample 5 is stopped with respect to the optical system, the above focusing is completed, the observation position is determined, and the microscope image is observed or the still image is captured and stored. Although the frame rate is not so necessary, it is desired that the desired resolution (in many cases, as high a resolution as possible) is obtained.

  Accordingly, when the movement detection unit 21 determines that the stage 5 is moving, it outputs a control signal to the camera control unit 9 to capture an image with an increased camera frame rate (in this case, the resolution is lowered). ). Conversely, when the movement detection unit 21 determines that the sample 5 is stopped, it outputs a control signal to the camera control unit 9 to increase the resolution of the camera 8 (in this case, the frame rate is reduced).

  FIG. 8 shows processing for switching the resolution based on whether or not the stage is moving in the present embodiment. First, the movement detector 21 detects the movement of the stage 5 (S31). As described above, this detection is performed by subjecting the image captured by the camera 8 to image processing (for example, a method of simply detecting a change in luminance value, a method of pattern recognition, or the like).

  If it is determined that the stage 5 is moving (proceed to “Yes” in S32), a control signal is output to the camera control unit 9, the camera 8 is set to the low resolution mode (S33), and the frame rate is set. Is picked up (S34).

  If it is determined that the stage 5 is stopped (going to “No” in S32), a control signal is output to the camera control unit 9, and the camera 8 is set to the high resolution mode (S35). An image is picked up (an image signal is output at a low frame rate) (S36).

  As described above, according to the embodiment of the present invention, a high frame rate is obtained when moving a sample for which a high frame rate is desired, and a high resolution image is obtained when the sample for which high resolution is desired is stopped. It is done.

  As a modified example of the above embodiment, for example, the movement of the sample is not performed by image processing, but a linear scale is arranged on each of the X, Y, and Z movement axes, and the actual movement amount of each axis is detected. There is also a method of doing in. In this case, there is an advantage that the movement can be reliably detected even when there is no contrast and the movement cannot be detected by image processing, such as a mirror surface. Further, a sensor for monitoring the movement of the stage may be provided, and it may be determined whether or not the sample has moved based on the detection result of the sensor.

  As another modification, for example, there is a combination of an autofocus device (for example, an active AF device using laser light) instead of the movement detection unit. By switching the resolution of the camera based on the autofocus instruction signal from the autofocus device, it is possible to observe at a high resolution during observation, and an image can be obtained at a high frame rate when moving in the Z-axis direction. In this case, functions such as image processing and a linear scale are not necessary, and there is a merit that can be realized with a simple configuration.

It is a figure which shows the outline | summary of the microscope image imaging system in 1st Embodiment. It is a figure which shows the image imaged by the high resolution in 1st Embodiment, and the image which thinned out pixel data from this image. It is a figure which shows operation of the operator in 1st Embodiment. It is a figure which shows the detail of the process by the side of the microscope image pick-up system in S1 of FIG. It is a figure which shows the detail of the process by the side of the microscope image pick-up system in S2 of FIG. It is a figure which shows the change of the contrast value accompanying the movement of the stage 7 in FIG. 5A. It is a figure for demonstrating the modification of 1st Embodiment. It is a figure which shows the outline | summary of the microscope image pick-up system in 2nd Embodiment. It is a figure which shows the process which switches the resolution based on whether the stage in 2nd Embodiment is moving.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Illumination light source 2 Epi-illumination optical system 3 Half mirror 4 Objective lens 5 Sample 6 Imaging optical system 7 Stage 8 Camera 9 Camera control part 10 Display part 11 Focus evaluation part 12 Drive part 13 Focus instruction part 21 Movement detection part

Claims (9)

A movable stage on which the sample is placed;
An optical system for magnifying and observing the sample;
Imaging means for capturing an optical image by the optical system;
Control means for controlling the resolution of an image captured by the imaging means based on the movement of the stage relative to the optical system;
A microscope image capturing system comprising:   The microscope imaging system according to claim 1, wherein the imaging unit can change a resolution of an image obtained by imaging under the control of the control unit. The microscope image capturing system further includes focusing instruction information acquisition means for acquiring focusing instruction information for moving the sample to a focusing position,
When the focusing instruction information acquisition unit acquires the focusing instruction information, the control unit relatively increases the resolution of the image compared to the case where the focusing instruction information is not acquired. The microscope image pickup system according to claim 2, wherein the image pickup unit is controlled to do so. The microscope image capturing system according to claim 1, further comprising: a focus evaluation unit that evaluates whether or not the image is focused based on the image captured by the image capturing unit. Imaging system. The imaging means captures a predetermined area in a range of images that can be captured,
The microscope image capturing system according to claim 4, wherein the focusing evaluation unit evaluates whether or not focusing is performed based on the captured image of the predetermined region. The microscope image capturing system further includes movement detection means for detecting whether or not the sample is moved relative to the optical system,
The microscope image capturing system according to claim 1, wherein the control unit controls a resolution of an image captured by the imaging unit based on a detection result by the movement detecting unit.   When the movement detection unit detects that the sample is moving relative to the optical system, the control unit determines the resolution of the image as compared with the case where the sample is not moved. The microscope image pickup system according to claim 6, wherein the image pickup unit is controlled to be relatively low.   When the stage is moved relatively in a direction perpendicular to the optical axis direction of the optical system, the control unit is configured to make the resolution of the image relatively low compared to the case where the stage is not moved. The microscope image pickup system according to claim 1, wherein an image pickup unit is controlled. When the stage is moved relatively in the optical axis direction of the optical system, the control means controls the imaging means so as to make the resolution of the image relatively lower than when the stage is not moved. The microscope image pickup system according to claim 1.

Images