JP2005352100A - Microscope device, image display method, and image display program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a microscope device improving operability by making a first image of a region image being the whole image of a specimen or a reference, making a second image of a region to be noticed smaller than the image and overlapping them to be displayed, and providing a display means for catching change of the specimen image while confirming the whole specimen, and to provide an image display method carried out in the microscope device and an image display program. <P>SOLUTION: The microscope device comprises an image information acquiring means for acquiring image information by irradiating the specimen with a laser beam and detecting observed light from the specimen, an image information storing means for storing image information acquired by the image information acquiring means as first image information, and an image information display means for superposing and displaying image information acquired by the image acquiring means to the first image information stored to the image information acquiring means as second image information. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a microscope apparatus, an image display method, and an image display program that display on a screen an observation image of a sample obtained by scanning a sample with light through an optical system, and in particular, an observation image obtained at the start of observation. The present invention relates to a microscope apparatus, an image display method, and an image display program that can superimpose and display an observation image that continues to be scanned while narrowing down the area.

  Conventionally, a laser scanning microscope irradiates laser light while scanning in the X-axis and Y-axis directions of a sample via a scanning optical system and an objective lens, and transmits transmitted light from the sample, reflected light, or fluorescence generated in the sample. Detected again by the detector through the objective lens and the scanning optical system, two-dimensional luminance information of transmitted light, reflected light, or fluorescence is obtained. Further, by displaying this luminance information in correspondence with the XY scanning position as a two-dimensional luminance distribution on a display or the like, it is also possible to observe a fluorescent image, a transmission image, or a reflection image of the sample.

To obtain an observation image with such a laser scanning microscope,
1. The sample is irradiated with parallel light, the observation position of the sample and the objective magnification are determined, and the focus is adjusted.
2. One or a plurality of laser beams to be irradiated are selected, and each intensity and the order of irradiation are determined.
3. Determine the detection wavelength and detection sensitivity of one or more photodetectors.
4). Determine the range to scan.
5. Set the scanning speed and resolution.
6). Laser beam is continuously scanned to adjust the focus of the sample.
Etc. work occurs. Although the above-described work flow is shown simply, there are actually many more setting items, and in order to set these items to optimum values and obtain an ideal image, there are many operators. Need experience.

Work 1 above. A technique is disclosed in which the image obtained in step 1 and the observation image obtained by actually starting the operation are displayed in a superimposed manner (see, for example, Patent Document 1).
The microscope main body described in Patent Document 1 has an optical path that is divided into an optical path related to a scanning microscope and an optical path related to an imaging device, and stores an image obtained by the scanning microscope as a first image and captures an image. The image obtained by the apparatus is stored as a second image, the shift amount of these stored images is corrected, and these images are displayed in a superimposed manner.
JP-A-11-231223

  However, this amount of deviation is caused by, for example, a problem caused by a difference between the range scanned by the point light source of the scanning microscope and the range detected by the imaging device. The minimum unit of the light source is a point, and the detection range of the imaging device basically corresponds to a two-dimensional square surface. Therefore, in order to perform processing for overlapping images while always managing such information, it is enormous. A storage device that handles data and a data arithmetic processor are required. Furthermore, in order to display such information as a moving image, it is required to increase the processing capacity of the processor, and there is a problem that a large-capacity storage device must be installed in order to keep a history of each video.

  Moreover, in the conventional microscope main body like the above-mentioned patent document 1, since the optical path of a scanning microscope apparatus and an imaging device is divided into two, the weak fluorescence detected with a scanning microscope may reduce. There was a problem that there was.

  In addition, these series of operations follow changes in the desired range for the entire sample in real time, and a means for visually indicating the number of cells and the degree of dispersion in the entire sample that can be confirmed with a scanning microscope is specified. There was no problem.

  In many scanning microscopes, a method of displaying an image showing the whole image as a still image in a separate window is used, and the image superposition technique is generally performed by post-processing. there were.

  The present invention has been made in view of the above-described drawbacks of the prior art, and is executed in the microscope apparatus that can capture a change in the sample image while checking the entire sample and has improved operability. It is an object to provide an image display method and an image display program.

The present invention employs the following configuration in order to solve the above problems.
In the operation of obtaining the second image, which is the region of interest on the sample smaller than the first image, by using the entire image of the sample obtained by laser scanning or the reference region image as the first image, the second image During acquisition, these images are displayed in a superimposed manner, and these images have a function of displaying in real time as a two-dimensional or three-dimensional fluorescence history screen.

  Further, the scanning microscope has a function of changing the second image in real time with the first image as a background in response to enlargement, reduction, movement, or rotation during acquisition of the second image.

  That is, according to one aspect of the present invention, the microscope apparatus of the present invention includes an image information acquisition unit that acquires image information by irradiating a sample with a laser beam and detecting observation light from the sample. Image information storage means for storing image information acquired by the image information acquisition means as first image information, and image information acquired by the image acquisition means in the first image information stored in the image information acquisition means. And image information display means for superimposing and displaying the second image information.

  In the microscope apparatus of the present invention, the image information display unit displays the second image information acquired by the image acquisition unit with time, in real time.

  In the microscope apparatus of the present invention, the image information display means displays a plurality of second image information acquired by the image acquisition means with the passage of time so that the display position is shifted so that the change history can be understood. It is characterized by.

  Moreover, according to one aspect of the present invention, the microscope apparatus of the present invention stores an imaging unit that acquires image information by imaging a sample, and image information acquired by the imaging unit as first image information. Image information storage means, image information acquisition means for acquiring image information by irradiating the sample with a laser beam and detecting observation light from the sample, and first image information stored in the storage means And image information display means for superimposing and displaying the image information acquired by the image acquisition means as second image information.

  In the microscope apparatus of the present invention, the observation region of the sample for acquiring the second image information is a part of the observation region of the sample for acquiring the first image information. Features.

In the microscope apparatus of the present invention, the image information acquisition unit acquires image information by performing point scanning, line scanning, or two-dimensional scanning.
Moreover, according to one aspect of the present invention, the image display method of the present invention is an image display method executed by a microscope apparatus, which irradiates a sample with a laser beam and detects observation light from the sample. The acquired image information is stored in the memory as the first image information, and the acquired image information is superimposed as the second image information on the first image information stored in the memory. It is characterized by displaying.

  Moreover, according to one aspect of the present invention, the image display method of the present invention is an image display method executed by a microscope apparatus, wherein image information is acquired by imaging a sample, and the image information acquired by the imaging is acquired. Is stored in the memory as first image information, the image information is acquired by irradiating the sample with a laser beam, and detecting the observation light from the sample, and the first image information stored in the memory In addition, the image information acquired by the detection is superimposed and displayed as second image information.

  Moreover, according to one aspect of the present invention, the image display program of the present invention is an image display program for causing a microscope apparatus to execute, and irradiates a sample with a laser beam to detect observation light from the sample. A procedure for acquiring the image information, a procedure for storing the acquired image information in the memory as the first image information, and the first image information stored in the memory with the acquired image information as the first image information. 2 is a computer-executable image display program for executing the procedure of superimposing and displaying the image information as the second image information.

  Moreover, according to one aspect of the present invention, the image display program of the present invention is an image display program for causing a microscope apparatus to execute the image display program, the procedure for acquiring image information by imaging a sample, and the above imaging A procedure for storing the acquired image information in the memory as first image information, a procedure for acquiring image information by irradiating the sample with a laser beam and detecting observation light from the sample, and storing in the memory A computer-executable image display program for executing a procedure of superimposing and displaying image information acquired by the detection as second image information on the stored first image information.

  According to the present invention, the entire image of the sample or the reference region image is set as the first image, the region of interest that is smaller than the image is set as the second image, and these are superimposed and displayed. By providing a display means capable of capturing the change in the sample image while confirming the above, it is possible to improve the operability of the scanning microscope.

Embodiments of the present invention will be described below with reference to the drawings.
First, a first embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a diagram for explaining the configuration of a scanning microscope to which the present invention is applied.

  In FIG. 1, a scanning microscope 100 includes a laser oscillator 1 that can be provided one or more, a beam expander 2 that expands a laser beam (illumination light) from the laser oscillator 1, a laser beam guided to a sample 9, An optical path splitting element 3 such as a beam splitter or a dichroic mirror that separates fluorescence or reflected light from the sample 9 and a laser beam, a galvano mirror 4, a galvano mirror 5, a pupil projection lens 6, an imaging lens 7, an objective lens 8, and an optical path split A confocal aperture 10, a wavelength selection element 11, a light intensity detector 12, and a stage 13 are provided to obtain a confocal effect of fluorescence or reflected light from the sample 9 divided by the element 3.

  The light that has passed through the confocal aperture 10 is guided to the light intensity detector 12 via a wavelength selection element 11 such as a dichroic mirror or an interference filter. A plurality of wavelength selection elements 11 can be provided, and a plurality of light intensity detectors 12 can be provided by providing a plurality.

  Next, the laser beam guided from the laser oscillator 1 to the sample 9 is guided to the galvanometer mirror 4 and the galvanometer mirror 5 after passing through the optical path splitting element 3. By installing the galvanometer mirror 4 and the galvanometer mirror 5, two-dimensional scanning of the laser beam becomes possible, and the two-dimensionally scanned laser beam passes on the sample 9 via the pupil projection lens 6, the ligation lens 7, and the objective lens 8. Scan.

  The stage 13 has a function of being electrically movable in the three-dimensional direction, and the galvanometer mirror 4 and the galvanometer mirror 5 can scan the laser beam in an arbitrary point or line or in the two-dimensional direction.

These electrically driveable parts are controlled by a computer which is a control unit (not shown).
The operation of the computer includes control of the stage 13, driving and coordinate control of the galvanometer mirrors 4 and 5, dimming and ON / OFF of the laser oscillator 1, accumulation of light intensity information sent from the light intensity detector 12, and galvanometer mirror 4 By treating the position indicated by the coordinates of 5 and the light intensity information as pixels on the coordinates, the image is displayed on the display as an image to which point or line or 2D or 3D color information is added.

  In this computer, a GUI (Graphical User Interface) for operating the scanning microscope 100 is mounted, and it is desirable that the GUI is designed to be user-friendly.

It is also necessary to distribute control functions in consideration of system balance so that the load on the computer does not increase.
Furthermore, instead of using a so-called computer, a measuring instrument or control box specialized for the mounting function of the scanning microscope 100 may be provided. For example, in the case of a computer, the display for displaying thumbtack information is two-dimensional, but if this is changed to an output device capable of three-dimensional display such as a hologram, a three-dimensional sample image can be expressed. .

Next, a process for performing fluorescence observation in the scanning microscope 100 having such a laser scanning optical system will be described.
FIG. 2 is a flowchart showing a flow of a process for performing fluorescence observation.

  First, in step S21, the sample 9 is set on the stage 13, and in step S22, the sample 9 is irradiated with light from the light source, so that the stage 13 is roughly adjusted, and the sample 9 is focused.

Next, in step S23, the observation optical path is changed to the laser scanning side, and the laser scanning optical system is operated using a computer.
Specifically, first, coordinates for scanning the galvanometer mirror 4 and the galvanometer mirror 5 are set from the laser scanable range, and the scan range, resolution, and speed are set. Next, the fluorescent probe dyeing the sample 9 is selected, and one or a plurality of laser oscillators 1 for exciting the fluorescent probe are selected. The laser oscillator 1 is set to irradiate laser light only during the scanning range of the setting process. This is because when the galvanometer mirror 4 and the galvanometer mirror 5 are in a line or quadrangle raster scan (a scanning method in which an operation to get on the first line every line is generated), the galvanometer mirror 4 and the galvanometer mirror 5 are retraced. A period arises. During this blanking period, it is necessary to control the laser light to be turned off so that unnecessary discoloration traces do not remain on the sample 9. In many cases, an acousto-optic element is used to turn on / off the laser oscillator 1, but recently, the laser itself can be turned on / off by using a semiconductor laser.

  Next, the detection wavelength region of the fluorescent probe is set, and the light intensity detector 12 for detecting the detection wavelength is determined. If there is one light intensity detector 12, no particular setting is required. For the light intensity detector 12, a photodiode or a photomultiplier tube is used.

Next, a color representing the light intensity is selected. In general, the light intensity is expressed by a pseudo color, and this color may be a single color or a complex color that changes depending on the light intensity.
As described above, when the minimum setting of the scanning microscope is completed, the sample 9 is continuously scanned with laser light, and the specimen 9 is focused.

  Next, when the sample 9 is focused in step S23, a first image is acquired in step S24. The first image is desirably an entire image of the sample 9, but may be an arbitrary size depending on the application. In the first embodiment, the first thumbtack is an overall image of the sample 9. For this purpose, the range in which the galvanometer mirror 4 and the galvanometer mirror 5 are shaken is set to the maximum. If necessary, the setting value of each function is changed, and then the number of scans is set to one to acquire the first image.

  Next, in step S25, a region of interest on the first image for performing measurement such as fading or recovery of the fluorescent probe or wavelength change due to stimulation is set in detail. The image obtained by this becomes the second image.

Next, how these first image and second image are displayed will be described.
3 to 8 are diagrams showing display examples.
FIG. 3 is a display example in which only the first image (shaded portion in the figure) is displayed, and FIG. 4 shows the second image (black circle part in the drawing) obtained by dot scanning as the first image. FIG. 5 is a display example in which a second image (black line portion in the figure) obtained by line scanning is displayed superimposed on the first image. 6 is a display example in which a second image (black rectangular area portion in the figure) obtained by two-dimensional scanning is displayed superimposed on the first image.

  By using such an expression method, in the fluorescence observation of the cells, it is possible to observe the state of fading, recovery or restoration of the fluorescence intensity of the second image while comparing it with the first thumbtack.

  7 is a display example in which the second image obtained by the two-dimensional scanning is displayed so as to be superimposed on the first image, as in FIG. 6, and FIG. 8 shows the image displayed in FIG. It is the example of a display displayed changing the angle.

  As described above, when the scanning range on the sample 9 is changed, it is possible to observe the sample 9 while always being aware of where the user is looking by projecting the following second thumbtack.

  Although the display example shown in FIG. 8 deals with rotation, it is also possible to follow the movement or enlargement or reduction of the scanning range. The second image that has been moved, enlarged, reduced, or rotated may be displayed so as to leave a locus on the first image line, whereby the latest state of the sample 9 is left as a history in the display image. be able to.

FIG. 9 is a diagram illustrating a display example in which the second image is displayed so as to overlap in the time axis direction every time two-dimensional scanning is completed.
The expression method as shown in FIG. 9 can be expressed as a fluorescence intensity image when the second image is obtained by dot scanning or line scanning. In addition, since the number of scans can be visually confirmed, it also serves as an index for preventing an excessive number of scans.

  In the first embodiment, by using the display method as described above, the change of the sample 9 can be visually recognized in real time, and the operability of the scanning microscope is improved. The display means that can be expressed can be freely selected by the user. Therefore, a conventional means for displaying only the second image is also provided at the same time.

Next, a second embodiment of the present invention will be described with reference to FIGS.
In the second embodiment, the first image in the first embodiment described above is an image obtained from the imaging device.

FIG. 10 is a diagram for explaining the configuration of a microscope system to which the present invention is applied.
The microscope system 200 includes a light source 14 for irradiating the sample 9 with parallel light, an optical path switching optical component 19 that divides the light from the light source 14 into an observation optical path and an objective optical path, and an enlargement ratio when observing the sample 9. An objective lens 8 for determining the image, a stage 13 for moving the sample 9 in a three-dimensional direction, an eyepiece 16 for confirming the image of the sample 9 with the naked eye, and a lens barrel 17 for guiding light to the imaging device 18 An image pickup device 18 having a function of acquiring an image of the sample 9 obtained by irradiating light from the light source 14 and sending it to a computer as a control unit, and an optical device having an optical path for guiding laser light of a scanning microscope Member 15. Note that the stage 13 is electrically driven and is controlled by a computer as a control unit.

A process for performing fluorescence observation in the microscope system 200 having such a horizontal configuration will be described with reference to FIG.
FIG. 11 is a flowchart showing a flow of a process for performing fluorescence observation in the second embodiment.

In FIG. 11, the same steps as those in FIG. 2 are denoted by the same reference numerals as those in FIG.
First, in step S21, the sample 9 is set on the stage 13, and in step S22, the sample 9 is irradiated with light from the light source, so that the stage 13 is roughly adjusted, and the sample 9 is focused. At this time, the operability can be improved if a function of automatically focusing, that is, an autofocus function is implemented, preferably while automatically moving the stage 13 from the image of the imaging device 18.

Next, in step S111, the video of the imaging device 18 is captured from the GUI screen of the computer, and this is used as the first image.
Next, in step S23, the observation optical path is changed to the laser scanning side, the laser scanning optical system is operated, and laser light is irradiated to readjust the stage 13 in order to acquire an image by the confocal effect. At this time, it is desirable to perform focus adjustment by selecting, from the first image, a region where there is no problem even if the sample 9 is damaged by fading due to laser light as the focus adjustment range. As a result, even if it is difficult to adjust the focus in laser scanning, it is possible to reduce damage in the range of interest of the sample 9 to be observed. However, as a condition, the image obtained by the imaging device 18 must be capable of managing one-to-one coordinates with the image obtained by scanning, and must store an information amount that does not impair the resolution even if the image is enlarged or reduced. is there.

  Next, in step S25, a region of interest on the first image for performing measurement such as fading or recovery of the fluorescent probe or wavelength change due to stimulation is set in detail. The image obtained by this becomes the second image.

Then, the first image acquired in step S111 becomes a reference screen as shown in FIG. 3, and the second image can be displayed in an overlapping manner as shown in FIGS.
Further, if the work procedure shown in FIG. 11 can be executed only by operating the computer that is the control unit, the manual work is reduced, and it is possible to escape from the danger that the dangerous laser beam is mistakenly viewed with eyes. .

  The embodiments of the present invention have been described above with reference to the drawings. However, the microscope apparatus to which the present invention is applied is limited to the above-described embodiments and the like as long as the function is executed. Needless to say, a single device, a system composed of a plurality of devices, an integrated device, or a system in which processing is performed via a network such as a LAN or WAN may be used. .

  It can also be realized by a system comprising a CPU, ROM or RAM memory connected to the bus, input device, output device, external recording device, medium driving device, portable recording medium, and network connection device. That is, a ROM or RAM memory, an external recording device, and a portable recording medium that record software program codes for realizing the systems of the above-described embodiments are supplied to the microscope device, and the computer of the microscope device performs the program. Needless to say, this can also be achieved by reading and executing the code.

  In this case, the program code itself read from the portable recording medium or the like realizes the novel function of the present invention, and the portable recording medium or the like on which the program code is recorded constitutes the present invention. .

  Examples of portable recording media for supplying program codes include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, DVD-RAMs, magnetic tapes, and non-volatile memories. Various recording media recorded through a network connection device (in other words, a communication line) such as a card, a ROM card, electronic mail or personal computer communication can be used.

  In addition, the functions of the above-described embodiments are realized by executing the program code read out on the memory by the computer, and the OS running on the computer is actually executed based on the instruction of the program code. The functions of the above-described embodiments are also realized by performing part or all of the process.

  Furthermore, a program code read from a portable recording medium or a program (data) provided by a program (data) provider is provided in a function expansion board inserted into a computer or a function expansion unit connected to a computer. The CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments are also performed by the processing. Can be realized.

  That is, the present invention is not limited to the embodiments described above, and can take various configurations or shapes without departing from the gist of the present invention.

It is a figure for demonstrating the structure of the scanning microscope to which this invention is applied. It is a flowchart which shows the flow of the process which performs the fluorescence observation in 1st Embodiment. FIG. 10 is a first diagram illustrating a display example. It is FIG. (2) which shows a display example. It is FIG. (3) which shows a display example. It is FIG. (4) which shows a display example. It is FIG. (5) which shows a display example. It is FIG. (6) which shows a display example. It is a figure which shows the example of a display which superimposes and displays a 2nd image in a time-axis direction whenever a two-dimensional scan is completed. It is a figure for demonstrating the structure of the microscope system to which this invention is applied. It is a flowchart which shows the flow of the process which performs the fluorescence observation in 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Laser oscillator 2 Beam expander 3 Optical path dividing element 4 Galvanometer mirror 5 Galvanometer mirror 6 Pupil projection lens 7 Imaging lens 8 Objective lens 9 Sample 10 Confocal aperture 11 Wavelength selection element 12 Light intensity detector 13 Stage 14 Light source 15 Optical member Reference Signs List 16 eyepiece 17 lens barrel 18 imaging device 19 optical path switching optical component 100 scanning microscope 200 microscope system

Claims (10)

Image information acquisition means for acquiring image information by irradiating a sample with a laser beam and detecting observation light from the sample;
Image information storage means for storing the image information acquired by the image information acquisition means as first image information;
Image information display means for superimposing and displaying the image information acquired by the image acquisition means as second image information on the first image information stored in the image information acquisition means;
A microscope apparatus comprising:   The microscope apparatus according to claim 1, wherein the image information display unit displays the second image information acquired by the image acquisition unit over time in real time.   The image information display means displays a plurality of pieces of second image information acquired by the image acquisition means with the passage of time so that the change history can be understood. Microscope equipment. Imaging means for acquiring image information by imaging a sample;
Image information storage means for storing image information acquired by the imaging means as first image information;
Image information acquisition means for acquiring image information by irradiating the sample with a laser beam and detecting observation light from the sample;
Image information display means for superimposing and displaying the image information acquired by the image acquisition means as second image information on the first image information stored in the storage means;
A microscope apparatus comprising:   The observation area of the sample for acquiring the second image information is a part of the observation area of the sample for acquiring the first image information. The microscope apparatus according to any one of the above.   The microscope apparatus according to claim 1, wherein the image information acquisition unit acquires image information by performing point scanning, line scanning, or two-dimensional scanning. An image display method executed by a microscope apparatus,
By irradiating a sample with a laser beam and detecting observation light from the sample, image information is obtained,
Storing the acquired image information in a memory as first image information;
An image display method comprising: superimposing and displaying the acquired image information as second image information on the first image information stored in the memory. An image display method executed by a microscope apparatus,
Obtain image information by imaging the sample,
The image information acquired by the imaging is stored in a memory as first image information,
By irradiating the sample with a laser beam and detecting observation light from the sample, image information is obtained,
An image display method, wherein the image information acquired by the detection is superimposed and displayed as second image information on the first image information stored in the memory. An image display program for causing a microscope apparatus to execute,
A procedure for obtaining image information by irradiating a sample with a laser beam and detecting observation light from the sample,
Storing the acquired image information in a memory as first image information;
A procedure for superimposing and displaying the acquired image information as second image information on the first image information stored in the memory;
A computer-executable image display program for executing the program. An image display program for causing a microscope apparatus to execute,
A procedure for acquiring image information by imaging a sample;
A procedure for storing image information acquired by the imaging in a memory as first image information;
A procedure for obtaining image information by irradiating the sample with a laser beam and detecting observation light from the sample,
A procedure for superimposing and displaying the image information acquired by the detection as the second image information on the first image information stored in the memory;
A computer-executable image display program for executing the program.

Images