JP4480976B2 - Scanning optical microscope apparatus, control method therefor, and program - Google Patents

Description

  The present invention relates to a technique used in a scanning optical microscope that irradiates a multiple-stained fluorescent sample with light having a plurality of different wavelengths and observes a plurality of excited fluorescence corresponding to each wavelength.

There is a scanning optical microscope that detects a plurality of fluorescence generated by exciting a sample multiple-stained with different fluorescent dyes with a plurality of lights having different wavelengths (see, for example, Patent Document 1).
In addition, when light with different wavelengths is sent to a microscope optical path deflected by at least two coordinates, and sequentially aiming and irradiating the observation position of the sample at time intervals, switching of each point on at least one surface of the sample There is a scanning optical microscope that irradiates light every scanning line or every scanning line and detects reflected light or radiated light emitted from the irradiated position to form an image of a scanning surface (for example, see Patent Document 2).

In addition, when the operator sets / registers a desired image area (mask image) on the frame image and scans the entire sample, confocal scanning is performed to acquire an image while irradiating excitation light only on the set area. There is a scanning optical microscope.
In addition, the excitation wavelength and the optical path setting for guiding each excitation wavelength and each fluorescence excited by the excitation wavelength to the corresponding photodetector are synchronized with the scanning for each frame scanning or for each line scanning. Alternatively, there is a confocal scanning optical microscope that switches during light reception by one pixel and detects the fluorescence of each of the specimens stained with two or more fluorescent dyes with a corresponding photodetector (for example, Patent Document 3). reference).
JP 2002-131645 A JP 2000-35400 A JP-A-10-206745

  In the scanning optical microscope described above, it is necessary for the operator to sequentially set the wavelength of light to be irradiated for each irradiation point, each scanning line, or each region of interest on an image. For example, a plurality of regions of interest within one screen. In such a case, it takes a long time to set the irradiation region and the wavelength of the light to be irradiated.

  In addition, when an operator sets an irradiation area on an image using an area creation tool called ROI (Region Of Interest) or the like, irradiation is performed not only on the cell of interest but also on the surrounding area. It will be set as an area. That is, since it is difficult to set only the target cell as the irradiation region, the unnecessary region is irradiated with the excitation light, resulting in useless fluorescence fading.

  Further, in the conventional scanning optical microscope, as a method of scanning by changing the wavelength for each region of interest, the entire image is scanned, and when the scanning is performed on the region of interest, on / off control of excitation light or Wavelength switching control was performed. In this method, since a region other than the region of interest where image acquisition is not performed is also scanned, it takes a long time to acquire an image.

  By the way, as an application using a biological microscope, there is FRAP (Fluorescence Recovery After Photobleaching) observation that observes the movement of things in a cell using fluorescence fading. In FRAP, protein molecules or lipid membranes of interest are labeled with a fluorescent dye in advance, and after the sample is partially faded using light such as laser light, the fluorescence of the faded portion is Observe the state of recovery by the replacement of the pigment with its surroundings by diffusion. This operation of fading is called bleaching processing.

  When the bleaching process is performed using the conventional scanning optical microscope described above, the area other than the attention area is also scanned, so that the intracellular fluorescence recovery action occurs during the area scanning other than the bleaching area. It is difficult to perform the bleaching process efficiently.

  In view of the above problems, it is an object of the present invention to enable efficient acquisition of an accurate image of a region of interest that is an observation target.

A scanning optical microscope apparatus according to one aspect of the present invention includes a light source that generates irradiation light to be irradiated on a sample, a scanning unit that two-dimensionally scans the irradiation light, and the irradiation light by two-dimensional scanning by the scanning unit. Detecting means for detecting the intensity of light coming from the sample when the sample is irradiated, and a frame image showing an image of the sample based on the intensity of light detected by the detecting means A frame image generating means for generating, and an observation target image for extracting an area representing an image to be observed from the frame image based on a result of automatic determination processing based on the luminance of the pixels constituting the frame image A position corresponding to the region extracted by the region extracting unit and the observation target image region extracting unit is used for the irradiation light by two-dimensional scanning by the scanning unit. When the irradiation is to be performed, the light source is controlled so that the irradiation light is generated by the light source, and the irradiation light is detected by two-dimensional scanning by the scanning means at a position that does not correspond to the extracted region. The light source control means for controlling the light source so that the light source does not generate the irradiation light when it is irradiated, and the irradiation light generated by the light source controlled by the light source control means is applied to the sample. The above-described problem is solved by comprising observation target image generation means for generating an image to be observed based on the intensity of light detected by the detection means when irradiated.

  In the above-described configuration, the luminance of the pixels constituting the frame image corresponds to the intensity of light detected by the detecting means, and an area composed of pixels in which the intensity of light meets a predetermined condition is true. Therefore, the region of interest is set more accurately than when using the region creation tool as described above. Furthermore, according to the above-described configuration, only the observation target region is irradiated with the irradiation light, and the other regions are not irradiated with the irradiation light. Therefore, useless fluorescent fading does not occur in the peripheral region of the observation target region. Therefore, accurate acquisition of the image of the attention area that is the observation target is efficiently performed.

In the scanning optical microscope apparatus according to the present invention described above, the light source generates a plurality of irradiation lights having different wavelengths, and the detection means detects the intensity of the light for each wavelength of the light, and The observation target image region extraction unit extracts a region in which the image to be observed is represented from the frame image for each wavelength of light coming from the sample, and the light source control unit extracts the observation target image region. The control based on the region extracted for each wavelength of the light by the means may be performed for each of a plurality of irradiation lights generated by the light source.

According to this configuration, since a unique observation target region is extracted for each wavelength of irradiation light, useless fluorescence fading due to a difference in wavelength of irradiation light in a region around the observation target region may occur. Absent.
In the scanning optical microscope apparatus according to the present invention described above, when the region is extracted by the observation target image region extraction unit, the scanning is performed so that the irradiation light scans only the position corresponding to the region. A scanning control means for controlling the means may be further included.

According to this configuration, scanning of the position corresponding to the region of the frame image other than the observation target region is omitted, so that the time required for acquiring the image of the observation target region is shortened and the bleaching process is efficiently performed. Will be able to.
In this configuration, the scanning unit includes a main scanning unit that performs main scanning of the irradiation light in the direction of the first axis, and the irradiation light that is subjected to the main scanning by the main scanning unit is one line. Sub-scanning means for performing sub-scanning of the irradiation light having a width corresponding to the one line in the direction of the second axis intersecting the first axis when the scanning is completed. A region extracted by the observation target image region extracting unit when the main scanning of one line for the first region extracted by the image region extracting unit is completed, and a region different from the first region When the second area is on the same line as the line on which the main scanning is performed, the scanning control means performs the main scanning of one line for the second area on the main scanning means. The sub-scan after the sub-scan It may be configured to perform control to be performed by the stage.

  According to this configuration, the number of flyback operations, which is an operation of moving the irradiation position of the irradiation light to the scanning start position of the next main scanning while moving the irradiation position by one main scanning in the sub-scanning direction, is reduced. Accordingly, the time required for acquiring the image of the observation target region is further reduced, and the bleaching process can be performed more efficiently.

  In this configuration, the scanning control means uses the main scanning means so that the amount of scanning by the sub scanning performed by the sub scanning means is minimized while the scanning means performs one two-dimensional scanning. The main scanning and the sub-scanning direction by the sub-scanning means can be rotated.

According to this configuration, since the number of flyback operations is further reduced, the time required for acquiring the image of the observation target region is further reduced, and the bleaching process can be performed more efficiently. Become.
In the configuration described above, the scanning unit includes a main scanning unit that performs main scanning of the irradiation light in the direction of the first axis, and the irradiation light in the direction of the second axis that intersects the first axis. A plurality of areas extracted by the observation target image area extracting means, the scanning control means only performs one line for each of the areas. The control for causing the main scanning means to perform the main scanning continuously for the plurality of areas and the control for causing the sub-scanning means to perform the sub-scan for one line for each of the areas are alternately repeated. Can be configured.

According to this configuration, the simultaneity of acquisition time of images generated for each observation target region is improved.
According to another aspect of the present invention, there is provided a scanning optical microscope apparatus comprising: a light source that generates irradiation light for irradiating a sample; scanning means that two-dimensionally scans the irradiation light; and two-dimensional scanning performed by the scanning means. Detection means for detecting the intensity of light coming from the sample when the irradiation light is applied to the sample, and a frame in which an image of the sample is represented based on the intensity of the light detected by the detection means Frame image generation means for generating an image, and observation for extracting an area representing an image to be observed from the frame image based on the result of automatic determination processing based on the luminance of the pixels constituting the frame image A target image region extracting unit and a position corresponding to the region extracted by the observation target image region extracting unit are illuminated by two-dimensional scanning by the scanning unit. When the light is to be irradiated, the light source is controlled so that the light is generated by the light source, and a position that does not correspond to the extracted region is irradiated by the two-dimensional scanning by the scanning means. The above-mentioned problem is solved by having a light source control means for controlling the light source so that the light source does not generate the irradiation light when the light is irradiated.

According to this configuration, since the optimum excitation light is selected for the reagent and the state of the cell, efficient selection of the excitation light for the bleaching process can be easily performed.
According to another aspect of the present invention, there is provided a control method for a scanning optical microscope apparatus, in which light arriving from a sample detected when the sample is irradiated with the irradiation light generated by the light source while being two-dimensionally scanned. Based on the result of the automatic determination process based on the intensity of the sample, the region representing the image to be observed is extracted from the frame image representing the image of the sample, and the two-dimensional scanning is performed. When the irradiation light irradiates a position corresponding to the region extracted by the extraction, the irradiation light is generated by the light source, and the irradiation light being two-dimensionally scanned is extracted in the region extracted by the extraction. When irradiating a non-corresponding position, the light source is controlled so that the light is not generated by the light source, and is generated by the controlled light source. Irradiation light so as to produce an image that is to the observation target based on the intensity of the incoming light from the sample to be detected when it is irradiated to the sample.

By doing so, the same actions and effects as those of the scanning optical microscope apparatus according to the present invention described above can be obtained, so that the aforementioned problems are solved.
In addition, even if it is a program which makes a computer perform each of the control method of the scanning optical microscope apparatus based on this invention mentioned above, the subject mentioned above is solved by running this program with a computer.

  As described above, according to the present invention, there is an effect that accurate acquisition of an image of a region of interest that is an observation target can be efficiently performed.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 shows the configuration of a scanning optical microscope apparatus for carrying out the present invention.
In FIG. 1, a computer 2 is connected to the scanning optical microscope main body 1. The computer 2 is a CPU that controls the operation of the entire scanning optical microscope apparatus by executing a control program, a main memory that the CPU uses as a work memory if necessary, and exchanges various data with the scanning optical microscope main body 1. It is a computer with a very standard configuration having an interface unit to be managed and an auxiliary storage device such as a hard disk device for storing various programs and image data. The operation of the scanning optical microscope main body 1 is controlled by the computer 2.

  Further, the computer 2 includes an operation panel 3 provided with various operators operated to give various instructions to the computer 2 from the observer, a light source 4 that generates light to irradiate the observation sample 106, A wavelength switching device 5 that switches the wavelength of light irradiated on the observation sample 106 and an image monitor 6 that displays various information generated by the computer 2 are connected.

The configuration and operation of the scanning optical microscope main body 1 will be described.
The light generated by the light source 4 and guided to the scanning optical microscope main body 1 via the wavelength switching device 5 enters the excitation wavelength switching device 101. The excitation wavelength switching device 101 is, for example, a photometric dichroic mirror that reflects light of a specific wavelength and transmits light of other wavelengths.

The light having a specific wavelength reflected by the excitation wavelength switching device 101 is then incident on the scanning unit 102.
The scanning unit 102 includes a galvanometer mirror for scanning in the X-axis direction that is the main scanning direction and a galvanometer mirror for scanning in the Y-axis direction that is the sub-scanning direction, and a light source according to the scanning control signal sent from the computer 2. The light from 4 is scanned two-dimensionally in the X-axis direction and the Y-axis direction. In addition, a scanning control end signal is output to the computer 2 for each scanning of one line by the X-axis direction scanning in the XY-axis direction scanning.

The light scanned from the scanning unit 102 in the XY-axis direction is irradiated as spot light on the observation sample 106 placed on the stage 105 through the objective lens 104 attached to the revolver 103 for observation. The sample 106 is scanned.
One unit is configured to detect the intensity of light of a specific wavelength by the photometric dichroic mirror 107a, the barrier filter 108a, and the photoelectric converter 109a. Therefore, the scanning optical microscope body 1 is provided with a total of three detectors.

  Light obtained from the observation sample 106 due to the above-described spot light, for example, reflected light or fluorescence generated in the observation sample 106 is returned to the incident optical path, passes through the excitation dichroic mirror 101, and is measured by the photometric dichroic mirror 107a, The wavelength is selected by passing through a photometric wavelength switching device consisting of 107b and 107c and barrier filters 108a, 108b and 108c. When the observer gives an instruction to the computer 2, light having a wavelength desired by the observer is selected by this photometric wavelength switching device.

  Light having a wavelength selected by the photometric wavelength switching device is incident on photoelectric converters 109a, 109b, and 109c such as a photomultiplier. From the photoelectric converters 109a, 109b, and 109c, analog electric signals corresponding to the intensity of incident light are output. This electrical signal is converted into digital data corresponding to the intensity of light by an analog / digital converter (not shown) provided in the computer 2 and stored in a memory of the computer 2. By processing this digital data by the computer 2, an image relating to the observation sample 106 is displayed on the image monitor 6.

  Here, the light source 4 is configured to generate at least two wavelengths of light, and the wavelength switching device 5 includes an AOTF (Acousto-Optic Tunable Filter) and an AOM (Acousto-Optic Modulator: An optical element such as an Acousto-Optic Modulator) or an EOM (Electro-Optic Modulator) is provided, and the irradiation wavelength of the light irradiating the observation sample 106 is switched at high speed under the control of the computer 2. To be able to. Note that a plurality of semiconductor lasers may be provided as the light source 4 and the wavelength switching device 5, and laser light generated by each semiconductor laser may be switched at high speed under the control of the computer 2. In the present embodiment, a 488 nm blue laser and a 543 nm green laser are provided.

  Hereinafter, a procedure for observing fluorescence generated in the observation sample 106 using the apparatus shown in FIG. 1 will be described.

First, the computer 2 controls the light source 4 and the wavelength switching device 5 to simultaneously irradiate the observation sample 106 with light of a plurality of wavelengths, in this embodiment, blue laser light and green laser light, and obtain a frame image (reference image). To do.
Next, as shown in FIG. 2, the acquired frame image is separated into channel images, that is, images created based on the detection result by a single detector. In the example of FIG. 2, a blue laser light generated based on a frame image (image of (a)) obtained by irradiation with blue laser light and green laser light, for example, based on a signal detected by the photoelectric converter 109a. Excited by the green laser light generated based on the image containing the image region stained with the reagent excited in step (the image of channel 1 shown in (b)) and the signal detected by the photoelectric converter 109b. The image is separated into an image (image of channel 2 shown in (c)) including an image region stained with a reagent.

  Note that when a frame image is acquired by simultaneously irradiating light of a plurality of wavelengths, the fluorescence on the long wavelength side is not only detected as the fluorescence emitted by the first reagent among the plurality of reagents stained on the observation sample 106. There may be a fluorescent crosstalk phenomenon that also enters a detector that detects the fluorescence emitted by the second reagent. As a method of avoiding this crosstalk phenomenon, in the scanning of the observation sample 106, the same line is scanned once with different excitation light without simultaneously irradiating light of a plurality of wavelengths. A line-sequential scanning method is known in which fluorescence detection is performed only with a detector corresponding to detection of fluorescence generated by excitation light. Therefore, when the above-described problem occurs, for example, by performing this line sequential scanning, it is possible to acquire a clear frame image in which fluorescence crosstalk is suppressed, and from the frame image to the channel image. Separation can also be performed appropriately.

  Next, the brightness (fluorescence intensity) of each pixel of the obtained channel image is compared with a preset threshold value. As a result of the comparison of the fluorescence intensity, when the intensity exceeding the threshold is measured, the irradiation of the excitation light corresponding to the channel image is set to the on state, and when the fluorescence intensity falls below the threshold, the excitation light Set the irradiation to the off state.

  This comparison process described above is performed on all the pixels of all channel images, and it is set whether or not each pixel in the image frame is irradiated with each excitation light. In the excitation light scanning performed thereafter, when the position of the observation sample 106 corresponding to the pixel in the image frame is scanned, only the excitation light set for the pixel in the above-described processing is irradiated. To.

  Conventionally, in the setting of the excitation light irradiation position for an acquired image that has been performed using an area creation tool such as ROI, an area as shown in FIG. On the other hand, the excitation light is turned on while the excitation light is turned off when scanning the hatched region (mask region) in FIG. For this reason, excitation light is irradiated also to the area | region other than the cell which is paying attention, and the useless fluorescence was generated.

  On the other hand, according to the method of the first embodiment described above, it is possible to create a mask region as shown in FIG. 3B, which is difficult to create with the region creation tool. In other words, since the fluorescence intensity is compared for each pixel and the setting is made so that only the target cell is irradiated with the excitation light, unnecessary fluorescence is not generated.

The mask generation described above is performed for each channel image.
Next, a cell observation method performed using the mask generated in this manner will be described with reference to FIG.
In the example of the observation sample 106 shown in FIG. 4, the cell A is stained with a reagent that generates fluorescence with a blue laser, and the cell B is stained with a reagent that generates fluorescence with a green laser.

  When mask generation is performed on the observation sample 106 as described above, the blue laser is set to the on state and the green laser is set to the off state at the position of the point 501 where only the cell A exists. Similarly, at the point 502 where only the cell B exists, the blue laser is set to the off state and the green laser is set to the on state.

On the other hand, at the point 503 where the cell A and the cell B overlap, both the blue laser and the green laser are set to the on state. In the region where neither the cell A nor the cell B exists, both the blue laser and the green laser are set to the off state.
The excitation wavelength switching device and the photometric wavelength switching device are switched to an appropriate optical path at high speed in synchronization with the irradiation of the excitation light to the observation sample 106 performed according to the above settings. For these devices that switch the optical path at high speed, the above-described acoustic elements such as AOTF or AOM / EOM, a grading device for spectroscopy, and the like may be used.

  The switching of the optical path by these devices can be uniquely determined in advance by the fluorescent material introduced into the sample. Examples of combinations of reagents (fluorescent substances), dichroic mirrors, and barrier filters for irradiated light when using a dichroic mirror turret for excitation as the excitation wavelength switching device and a dichroic mirror and barrier filter for photometry as the photometric wavelength switching device [Table 1].

  As described above, by automatically setting the wavelength of the light irradiated to the observation sample 106 and the region on the observation sample 106 that scans the light in the apparatus shown in FIG. Is done. In addition, an excitation light path and a photometry optical path that are optimal for the wavelength of the light to be irradiated are automatically constructed, and scanning and fluorescence measurement of the observation sample 106 are performed while these optical paths are switched at high speed. An image without fluorescence crosstalk that can be generated when light is simultaneously irradiated is created by the computer 2 from the measurement result.

In addition, since the optimum excitation light is selected for the reagent and cell state by comparing the fluorescence intensity between channels, it is easy to efficiently select the excitation light for bleaching in experiments such as FRAP. You can do it.
Note that it is not always necessary to generate an image in order to increase the efficiency of the bleaching process.

  Next, FIG. 5 will be described. This figure is a flowchart showing the processing contents of the mask generation processing described above. This process is performed by the CPU of the computer 2 executing a predetermined program, and is started when the process of separating the acquired frame image for each channel image is completed.

The following processing is performed for each channel image.
First, in S101, a process for causing the computer 2 to set the fluorescence wavelength to be observed is performed.
In S102, one pixel constituting the channel image that is the current processing target is extracted, and the luminance value of that pixel is examined. In the subsequent processing of S103, it is determined whether or not the luminance value of this pixel exceeds a preset luminance value threshold value. If the determination result is Yes, in S104, the pixel of the observation sample 106 is handled. The irradiation of excitation light when scanning of the position to be performed is set to an on state. On the other hand, if the determination result in S103 is No, the irradiation of excitation light when scanning of the position corresponding to the pixel of the observation sample 106 is performed in S105 is set to an off state. The excitation light irradiation setting is an operation setting of the light source 4 and the wavelength switching device 5, and light for generating fluorescence corresponding to the channel image that is the current processing target (in this embodiment, a blue laser). Light or green laser light).

  After the processing of S104 or S105 is finished, in S106, it is determined whether or not the pixel that was the target of the processing of S102 described above is the last pixel that constitutes the channel image that is the current processing target. It is determined whether or not the processing from S102 to S105 has been performed for all the pixels of the channel image that is the processing target, and if the determination result is Yes, the mask generation processing is terminated. On the other hand, if this determination result is No, the processing returns to S102 and the above-described processing is repeated.

The above processing is the mask generation processing, and the CPU of the computer 2 performs this processing for each channel image, thereby generating a mask for each excitation light.
In the first embodiment described above, the excitation light irradiation is turned on / off based on the result of the comparison between the fluorescence intensity and the threshold value of each channel image pixel, but instead, For example, as shown in FIG. 6, a pixel distribution indicating the relationship between the gradation value indicating the luminance (fluorescence intensity) of each pixel and the number of pixels of the pixel indicating the gradation value is obtained, and this pixel distribution is obtained. When the position of the observation sample 106 corresponding to a pixel belonging to the threshold value range is automatically determined as a threshold value, the excitation value corresponding to the channel image is automatically determined as a threshold value. The light irradiation is set to the on state, and when the position of the observation sample 106 corresponding to the pixel not belonging to the threshold range is scanned, the excitation light irradiation corresponding to the channel image is set to the off state. It may be. Further, based on the state of the acquired frame image or channel image, the maximum value and the minimum value of the threshold range automatically set as described above may be adjusted to values desired by the observer.

Next, a method for observing fluorescence generated in the observation sample 106 more efficiently than Example 1 using the apparatus shown in FIG. 1 will be described.
In the conventional fluorescence observation of the same type, an operation of scanning the entire screen as shown in FIG. 7A with respect to a frame image (reference image) obtained by irradiating a plurality of wavelengths (for example, a blue laser and a green laser). The scanning unit 102 is configured to perform the excitation light irradiation only when the position of the solid-line arrow portion shown in the figure is scanned. Therefore, when regions of interest serving as the center of sample observation are scattered on the same image, such as regions 601 and 602 in the figure, only the scanning unit 102 operates unnecessarily without irradiating light. There was a period.

  In addition, when fluorescence observation is performed by the above-described method while performing cell stimulation operations such as bleaching on the region 601, the area around the region 601 is scanned while the unnecessary region is being scanned. Since the fluorescence recovery occurs due to the movement of the protein, etc., the second irradiation with the excitation light to the subsequent region 601 will bleach the cells whose fluorescence recovery has progressed. In such bleaching processing, it takes a long time for the bleaching to converge, which is one of the factors that reduce the work efficiency for observation.

Therefore, in the second embodiment, when the attention area is scattered on the same image, the image is partially scanned and only the attention area is scanned, thereby enabling efficient image acquisition and cell stimulation. To do.
First, the observer registers the shape, size, and position of the set attention area in the computer 2.

Next, for each channel image obtained by separating the frame images, the fluorescence intensity of each image line in the selected area is compared, and the on / off of the excitation light and the optical path are automatically set in the same manner as in the first embodiment. To do. The region of interest is automatically generated by this processing.
Next, scanning of the observation sample 106 is started. Then, the scanning unit 102 is controlled according to the registered position information, and as shown in FIG. 6B, the light irradiation position moves to the scanning start position of the attention area 603, and the area 603 is changed according to the registered shape and size information. Scanning is performed, and image acquisition processing for the region 603 is performed.

When the scanning of the area 603 is completed, the scanning unit 102 is controlled to move the light irradiation position to the scanning start position of the attention area 604, and this time, the area 604 is scanned according to the registered shape and size information. Image acquisition processing is performed.
Thereafter, when the scanning of the area 604 is completed, the image acquisition process ends.

When scanning is repeated to perform bleaching processing or time series observation, the scanning unit 102 is controlled to move the light irradiation position to the scanning start position of the attention area 603 after the scanning of the area 604 is completed. Then, the above-described scanning is started again.
By scanning only the region of interest as described above, the time required to operate only the scanning mechanism without being irradiated with the excitation light can be shortened, so that the image acquisition time can be shortened. .

  Further, by performing the bleaching process by such scanning, the next bleaching process is started before the recovery of the fluorescence is started or in a state where the fluorescence is slightly recovered, so that the bleaching process is performed as shown in FIG. Convergence time is shortened, and bleaching can be performed efficiently.

Next, a method for more efficiently observing the fluorescence generated in the observation sample 106 using the apparatus shown in FIG. 1 will be described.
In the above-described second embodiment, the method of performing the scanning so as to irradiate only the region of interest in the observation sample 106 has been described. Such a method is expected to improve the image acquisition efficiency particularly effectively when the Y coordinates (sub-scanning positions) of different attention areas 701 and 702 do not overlap at all as shown in FIG. 9A, for example. it can.

  However, for example, as shown in FIG. 9B, when the Y coordinates of different attention areas 703 and 704 overlap, the above-described method scans the attention area 704 after the scanning of the attention area 703 is completed. Therefore, the flyback operation, that is, the operation of moving the light irradiation position in the sub-scanning direction by one main scanning line to the scanning start position of the next main scanning is performed in the X-axis direction (main scanning direction) of the overlapping portion. ) Occurs twice per line. Therefore, the image acquisition time becomes longer by the overlapped flyback operation.

Therefore, in the third embodiment, a method for efficiently scanning only the attention area will be described without being influenced by the positional relationship of the attention area.
First, the observer registers the shape, size, and position of the set attention area in the computer 2.

Next, the positions and sizes of all the attention areas registered in the computer 2 are compared, and as shown in FIGS. 9C and 9D, a scan path suitable for the arrangement state of the attention areas is constructed. Done.
As a condition for this path construction, when the Y coordinate does not overlap between the areas, the attention area is individually scanned as shown in FIG. 9C, and once the scanning in the attention area is completed, the scanning is temporarily performed. The operation of the scanning unit 102 is controlled so as to be interrupted and to move the scanning position to the scanning start position of the next scanning region. Note that the scanning procedure in this case is the same as that in the second embodiment.

  On the other hand, as shown in FIG. 9B, when there are overlapping Y coordinates in the attention areas 703 and 704, the same Y is obtained when scanning is performed for each area as in the method of the second embodiment. Since the flyback operation occurs twice on the main scanning line of the coordinates, in order to avoid this problem, first, as shown in FIG. When the scanning is completed and the one-line scanning is completed, the scanning with respect to the attention area 703 is temporarily stopped, and the scanning unit 102 is moved so that the irradiation position of the light moves to the scanning start position of the second line of the attention area 704 shown in FIG. The second line shown in the figure of the attention area 704 is scanned.

Thereafter, when the scanning of the second line of the attention area 704 is completed, the scanning with respect to the attention area 704 is temporarily interrupted, and this time the light irradiation position moves to the scanning start position of the third line of the attention area 703 shown in FIG. Thus, the operation of the scanning unit 102 is controlled.
Thereafter, by repeating the above-described control processing for the scanning unit 102, occurrence of overlapping flyback operations at the same Y coordinate is eliminated, so that the image acquisition time that has been a concern with the technique of the second embodiment described above. No delay occurs.

  Next, FIG. 10 will be described. FIG. 2 is a flowchart showing the processing contents of the control processing of the scanning unit 102 performed by the CPU of the computer 2. By performing this processing by the CPU, efficient scanning of only the region of interest in the observation sample 106 described above is performed.

In this process, the observer registers the attention area information indicating the shape, size, and position of the attention area in the computer 2, and then, for each channel image obtained by separating the frame images as in the first embodiment. It is started after the excitation light and the optical path are automatically set.
First, in S201, an instruction is given to the scanning unit 102 to operate the galvanometer mirror for X-axis direction scanning and the galvanometer mirror for Y-axis direction scanning, and based on the attention area information registered by the observer, The irradiation position is moved to the scanning start position of the attention area A which is the first attention area in the frame image.

In S202, an instruction is given to the scanning unit 102 to operate the galvanometer mirror for scanning in the X-axis direction which is the main scanning direction, and only the range of the attention area A is determined in the main scanning direction based on the attention area information registered by the observer. Is scanned by one line.
In S203, the observer determines whether or not the attention area B, which is the second attention area in the frame image, is present at the current position of the galvanometer mirror for Y-axis direction scanning that is the sub-scanning direction. If the determination result is Yes, the process proceeds to S204. If the determination result is No, the process proceeds to S205.

In S204, an instruction is given to the scanning unit 102 to operate the galvanometer mirror for scanning in the X-axis direction, and based on the attention area information registered by the observer, only the range of the attention area B is lighted by one line in the main scanning direction. Let it scan.
In S205, an instruction is given to the scanning unit 102 to operate the Y-axis direction scanning galvanometer mirror, and the main scanning line by the X-axis direction scanning galvanometer mirror is moved by one line in the sub-scanning direction.

  In S206, it is determined based on the attention area information registered by the observer whether or not the attention area A exists in a place where light can be irradiated at the current position of the galvanometer mirror for Y-axis direction scanning. If Yes, the process returns to S202, and the processes after the scanning of the attention area A are repeated. On the other hand, if the determination result is No, the process proceeds to S207.

  In S207, it is determined based on the attention area information registered by the observer whether or not the attention area B exists at a position where light can be irradiated at the current position of the galvanometer mirror for Y-axis direction scanning. If Yes, the process returns to S204, and the processes after scanning the attention area B are repeated. On the other hand, if this determination result is No, this control process is terminated.

  The above processing is the control processing of the scanning unit 102, and the CPU of the computer 2 performs this processing, so that only the region of interest in the observation sample 106 is scanned regardless of whether or not there is region overlap in the main scanning direction. Done.

Next, a method for more efficiently observing the fluorescence generated in the observation sample 106 using the apparatus shown in FIG. 1 will be described.
When performing the scanning shown in the technique of the third embodiment described above, the number of times of the flyback operation described above is reduced if the number of scans in the Y-axis direction (the number of sub-scans) is reduced, so that the image acquisition time is shortened. Can do. Therefore, for example, when the attention regions 801 and 802 are arranged on the frame image as shown in FIG. 11A, the galvanometer mirror for X-axis direction scanning and the galvanometer for Y-axis direction scanning of the scanning unit 102 are used. A rotation scanning method in which the waveform of a signal applied to the mirror is controlled to rotate the scanning direction is used, and in this example, the main scanning direction and the sub-scanning direction are changed by performing a 90 ° clockwise rotation. Scanning can be performed as shown in (b). Thereafter, scanning may be performed according to the method of the third embodiment described above.

Here, FIG. 12 will be described. FIG. 3 is a flowchart showing the processing contents of the scanning direction determination process performed by the CPU of the computer 2. By performing this process by the CPU, an efficient scanning direction in the observation sample 106 is determined.
In this process, the observer registers the attention area information indicating the shape, size, and position of the attention area in the computer 2, and then, for each channel image obtained by separating the frame images as in the first embodiment. It is started after the excitation light and the optical path are automatically set.

  First, in S301, based on the information on the shape and size of the attention area, the main scanning distance at this time (the longest distance scanned by one line scanning in the main scanning direction with respect to one or more attention areas) and the sub-scanning are performed. Comparison is made with the distance (the longest distance scanned by one line scanning in the sub-scanning direction with respect to one or more regions of interest), and the direction in which the distance is long is temporarily determined as the main scanning direction.

In S302, the main scanning direction is changed to be rotated by + 1 °.
In S303, the sub-scanning distance after the change in the process of the previous step is checked, and it is checked whether or not this sub-scanning distance is minimized. If the determination result is Yes, the main scanning direction and the sub-scanning direction are determined at the angle of the frame image at this time, and the scanning direction determination process is terminated. On the other hand, if this determination result is No, the processing returns to S301 and the above-described processing is repeated.

The process so far is the scanning direction determination process. After the completion of this process, the fluorescence generated in the observation sample 106 is observed by the method of Example 3 described above.
By performing the rotation scanning method as described above, the sub-scanning number h ′ shown in FIG. 11B is reduced compared to the sub-scanning number h shown in FIG. The number of flyback operations is reduced and the image acquisition time is shortened.

  Next, another method for observing fluorescence generated in the observation sample 106 using the apparatus shown in FIG. 1 will be described. This technique is intended to reduce the shift in the acquisition time of the image of each region of interest, and is suitable for measuring the propagation speed of calcium in nerve cells.

This method will be described with reference to FIG.
First, when there are a plurality of attention areas in the acquired image, the observer registers the set shape, size, and position of the attention area in the computer 2. In the example of FIG. 13A, it is assumed that the first attention area 901, the second attention area 902, and the third attention area 903 are registered, and the first attention area 901 uses blue laser light of 488 nm. Scanning is performed, and scanning using green laser light of 543 nm is performed in the second region of interest 902 and the third region of interest 903.

  At this time, for example, in the method according to the second embodiment described above, as shown in FIG. 13B, first, the light irradiation position is moved to the scanning start position of the first region of interest 901 and the laser beam scanning of 488 nm is started. Then, the first attention area 901 is scanned according to the shape and size information registered in the computer 2. When the scanning of the first region of interest 901 is completed, the light irradiation position is moved to the scanning start position of the second region of interest 902, and the wavelength switching device 5 switches the laser light to that of 543 nm, The second region of interest 902 is scanned. Thereafter, when the scanning of the second region of interest 902 is completed, the light irradiation position is moved to the scanning start position of the third region of interest 903, and the third region of interest 903 is scanned.

  Since such a scanning procedure is to acquire an image of another region of interest after acquiring an image of one region of interest, the image acquisition time differs for each other region of interest. . Therefore, it is not suitable when it is desired to observe images obtained at approximately the same time for each region of interest. For example, when acquiring images of a plurality of attention areas in order to measure the propagation speed of calcium in nerve cells, if there is a time lag of image acquisition for each attention area, the overall propagation state is captured. It is difficult.

  On the other hand, in this embodiment, as shown in FIG. 13C, first, the light irradiation position is moved to the scanning start position of the first region of interest 901, and one line scanning is performed with the laser beam of 488 nm. Subsequently, the light irradiation position is moved to the scanning start position of the second region of interest 902, the laser light is switched to that of 543 nm, one line scanning of the second region of interest 902 is performed, and further the scanning of the third region of interest 903 is performed. One line scan of the third region of interest 903 is performed by moving the light irradiation position to the start position.

  Here, the laser beam is switched again to that of 488 nm, and the light irradiation position is moved to the scanning start position of the second line of the first region of interest 901 to perform one line scanning. Next, the laser beam is switched again to that of 488 nm, and the second line of the second region of interest 902 and the second line of the third region of interest 903 are continuously performed.

  In this way, by adopting a method in which the attention area is scanned one line and then moved to the next attention area and scanned one line at a time, the simultaneity of the imaging time of each attention area is improved. As a result, the phenomenon of the observation sample 106 that changes simultaneously in a plurality of regions of interest, such as the state of propagation of calcium in nerve cells, can be captured as an image almost simultaneously in the plurality of regions of interest.

  Here, FIG. 14 will be described. FIG. 2 is a flowchart showing the processing contents of the control processing of the scanning unit 102 performed by the CPU of the computer 2. By performing this process by the CPU, scanning of the observation sample 106 that improves the synchronization of the imaging time of each region of interest described above is performed.

  In this process, the observer registers the attention area information indicating the shape, size, and position of the attention area in the computer 2, and then, for each channel image obtained by separating the frame images as in the first embodiment. It is started after the excitation light and the optical path are automatically set. In the present embodiment, it is assumed that these attention areas are numbered consecutively in the ascending order of the first, second,...

In FIG. 14, first, in S <b> 301, processing for substituting the number of regions of interest set for the observation sample 106 into a variable N is performed.
In S302, a process of substituting the initial value “1” for the variable n is performed.
In S303, processing for substituting the variable Kn for the number of scans in the Y-axis direction (sub-scanning direction) necessary for scanning the entire n-th region of interest is performed.

  In S304, the value obtained by adding “1” to the current value of the variable n is newly substituted into the variable n. In the subsequent S305, the value of the variable n is larger than the value of the variable N described above. A process of determining whether or not has been completed is performed. If the determination result is Yes, the process proceeds to S306. If No, the process returns to S303 and the above-described process is repeated.

  In S306, out of these attention areas set for the observation sample 106, the one having the maximum width in the Y-axis direction (sub-scanning direction) is extracted, and the entire extracted attention area is scanned. Therefore, processing for substituting the variable K for the number of scans in the Y-axis direction (number of sub-scans) necessary for this is performed.

In S307, a process of substituting the initial value “1” for the variable k is performed.
In S308, a process of substituting the initial value “1” for the variable n is performed.
In S309, a process for determining whether or not the value of the variable k has become larger than the value of the variable Kn described above, that is, whether or not scanning of the entire area of the nth region of interest has been completed is performed. The processes of S310 and S311 are performed only when the scanning portion remains (the result of this determination process is No). That is, in S310, an instruction is given to the scanning unit 102 to operate the galvanometer mirror for scanning in the X-axis direction and the galvanometer mirror for scanning in the Y-axis direction, and based on the attention area information registered by the observer, A process of moving the irradiation position to the scanning start position of the kth line of the nth region of interest in the frame image is performed. In subsequent S311, further instructions are given to the scanning unit 102 to scan in the X-axis direction which is the main scanning direction. The galvanometer mirror is operated, and based on the attention area information registered by the observer, only the range of the attention area A is optically scanned by one line in the main scanning direction. Note that the CPU also gives an instruction to the wavelength switching device 5 when executing the processing of S311 to switch the laser light to one used for scanning the nth region of interest.

  In S312, a process of adding the value obtained by adding “1” to the current value of the variable n to the variable n is performed again. In subsequent S313, the value of the variable n is larger than the value of the variable N described above. A process of determining whether or not has been completed is performed. If the determination result is Yes, the process proceeds to S314. If No, the process returns to S308 and the above-described process is repeated.

  In S314, the value obtained by adding “1” to the current value of the variable k is substituted for the variable k. In S315, the value of the variable k is larger than the value of the variable K described above. A process of determining whether or not has been completed is performed. Here, if this determination result is Yes, this control process is terminated. On the other hand, if this determination result is No, the process returns to S307 and the above-described process is repeated.

The above processing is the control processing of the scanning unit 102, and the CPU of the computer 2 performs this processing, whereby the observation sample 106 that improves the simultaneity of the imaging time of each region of interest described above is scanned.
In addition to the first to fifth embodiments described above, a control program for causing the CPU of the computer 2 to perform the processes shown in the flowcharts in FIGS. 5, 10, 12, and 14 is created. Even if the program is recorded on a computer-readable recording medium and the program is read from the recording medium into the computer 2 and executed by the CPU, the fluorescence generated in the observation sample 106 is observed with the apparatus shown in FIG. be able to. Examples of the recording medium from which the recorded control program can be read by a computer include, for example, a storage device such as a ROM or a hard disk device provided as an internal or external accessory device in the computer, a flexible disk, and an MO (magneto-optical disk) Portable recording media such as CD-ROM and DVD-ROM can be used.

  The recording medium may be a storage device provided in a computer functioning as a program server connected to a computer system via a communication line. In this case, a transmission signal obtained by modulating a carrier wave with a data signal representing a control program is transmitted from the program server to a computer system through a communication line as a transmission medium, and the computer system transmits the received transmission signal. The control program can be executed by the CPU by demodulating and reproducing the control program.

  In addition, the present invention is not limited to the above-described embodiments, and various improvements and changes can be made without departing from the scope of the present invention.

It is a figure which shows the structure of the scanning optical microscope apparatus which implements this invention. It is a figure explaining separation to a channel picture of a frame picture. It is a figure explaining the effect by the production | generation of the mask image by the method of Example 1. FIG. It is a figure explaining the method of the cell observation performed using the apparatus shown in FIG. It is a flowchart which shows the processing content of a mask production | generation process. It is a figure which shows an example of the setting method of the threshold value used as the setting reference | standard of excitation light reflection. FIG. 6 is a diagram illustrating Example 2. 6 is a graph showing the effect of Example 2. FIG. 6 is a diagram for explaining a third embodiment. It is a flowchart which shows the processing content of a scanning unit control process. FIG. 10 is a diagram for explaining a fourth embodiment. It is a flowchart which shows the processing content of a scanning direction determination process. FIG. 10 is a diagram for explaining a fifth embodiment. 10 is a flowchart showing the processing content of a scanning unit control process in Embodiment 5.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Scanning optical microscope body 2 Computer 3 Operation panel 4 Light source 5 Wavelength switching device 6 Image monitor 101 Excitation wavelength switching device 102 Scanning unit 103 Revolver 104 Objective lens 105 Stage 106 Observation sample 107a, 107b, 107c Photometric dichroic mirror 108a, 108b , 108c Barrier filter 109a, 109b, 109c Photoelectric converter

Claims (9)

A light source for generating irradiation light for irradiating the sample;
Scanning means for two-dimensionally scanning the irradiation light;
Detecting means for detecting the intensity of light coming from the sample when the irradiation light is irradiated onto the sample by two-dimensional scanning by the scanning means;
Frame image generation means for generating a frame image in which an image of the sample is represented based on the intensity of light detected by the detection means;
An observation target image region extraction unit that extracts a region in which an image to be observed is represented from the frame image based on a result of an automatic determination process based on the luminance of the pixels constituting the frame image;
Control of the light source so as to generate the irradiation light at the light source when the irradiation light is irradiated by the two-dimensional scanning by the scanning means at a position corresponding to the region extracted by the observation target image region extraction means. And controlling the light source so that the light source does not generate the irradiation light when the irradiation light irradiates a position not corresponding to the extracted region by two-dimensional scanning by the scanning means. Light source control means for performing,
Observation that generates an image to be observed based on the intensity of light detected by the detection means when the sample is irradiated with irradiation light generated by the light source controlled by the light source control means A target image generating means;
A scanning optical microscope apparatus characterized by comprising: The light source generates a plurality of irradiation lights having different wavelengths,
The detection means detects the intensity of the light for each wavelength of the light,
The observation target image region extraction unit extracts a region where an image to be observed is represented from the frame image for each wavelength of light coming from the sample,
The light source control unit performs the control based on the region extracted for each wavelength of the light by the observation target image region extraction unit for each of a plurality of irradiation lights generated by the light source.
The scanning optical microscope apparatus according to claim 1.   When the region is extracted by the observation target image region extraction unit, the image processing device further includes a scanning control unit that controls the scanning unit so that the irradiation light scans only the position corresponding to the region. The scanning optical microscope apparatus according to claim 1. The scanning means includes
Main scanning means for performing main scanning of the irradiation light in the direction of the first axis;
Sub-scanning means for performing sub-scanning of the irradiation light in the direction of a second axis intersecting with the first axis;
Have
When the main scanning of one line for the first region extracted by the observation target image region extraction unit is completed, the first region is the region extracted by the observation target image region extraction unit. When the second area, which is another area, is present on the same line as the line on which the main scanning has been performed, the scanning control unit performs the main scanning of one line for the second area. Performing control for causing the sub-scanning means to perform the sub-scanning after the main-scanning means is performed.
The scanning optical microscope apparatus according to claim 3.   The scanning control unit is configured to perform main scanning by the main scanning unit and the sub scanning so that a scanning amount by the sub scanning performed by the sub scanning unit is minimized while the scanning unit performs one two-dimensional scanning. 5. The scanning optical microscope apparatus according to claim 4, wherein the direction of sub-scanning by the means is rotated. The scanning means includes
Main scanning means for performing main scanning of the irradiation light in the direction of the first axis;
Sub-scanning means for performing sub-scanning of the irradiation light in the direction of a second axis intersecting with the first axis;
Have
When there are a plurality of regions extracted by the observation target image region extracting unit, the scanning control unit continues the main scanning of only one line for each of the regions and continues to the main region. Alternately repeating the control to be performed by the scanning means and the control to cause the sub-scanning means to perform the sub-scan for the one line for each of the regions;
The scanning optical microscope apparatus according to claim 3. A light source for generating irradiation light for irradiating the sample;
Scanning means for two-dimensionally scanning the irradiation light;
Detecting means for detecting the intensity of light coming from the sample when the irradiation light is irradiated onto the sample by two-dimensional scanning by the scanning means;
Frame image generation means for generating a frame image in which an image of the sample is represented based on the intensity of light detected by the detection means;
An observation target image region extraction unit that extracts a region in which an image to be observed is represented from the frame image based on a result of an automatic determination process based on the luminance of the pixels constituting the frame image;
Control of the light source so as to generate the irradiation light at the light source when the irradiation light is irradiated by the two-dimensional scanning by the scanning means at a position corresponding to the region extracted by the observation target image region extraction means. And controlling the light source so that the light source does not generate the irradiation light when the irradiation light irradiates a position not corresponding to the extracted region by two-dimensional scanning by the scanning means. Light source control means for performing,
A scanning optical microscope apparatus characterized by comprising: Based on the result of automatic determination processing based on the intensity of light coming from the sample detected when the sample is irradiated with the irradiation light generated by the light source in two dimensions, an image of the sample is represented. Extract the area where the image to be observed is represented from the frame image
When the irradiation light being two-dimensionally scanned irradiates a position corresponding to the region extracted by the extraction, the irradiation light is generated by the light source, and the irradiation light being two-dimensionally scanned is extracted. When the position that does not correspond to the region extracted by is to be irradiated, the light source is controlled so that the irradiation light is not generated in the light source,
Generating an image to be observed based on the intensity of light coming from the sample detected when the sample is irradiated with the irradiation light generated by the controlled light source;
A control method for a scanning optical microscope apparatus, comprising: A program for causing a computer to control a scanning optical microscope apparatus,
Based on the result of automatic determination processing based on the intensity of light coming from the sample detected when the sample is irradiated with the irradiation light generated by the light source in two dimensions, an image of the sample is represented. A process for extracting an area in which an image to be observed is represented from a frame image,
When the irradiation light being two-dimensionally scanned irradiates a position corresponding to the region extracted by the extraction, the irradiation light is generated by the light source, and the irradiation light being two-dimensionally scanned is extracted. A process of controlling the light source so as not to generate the irradiation light in the light source when irradiating a position that does not correspond to the region extracted by
A process of generating an image to be observed based on the intensity of light coming from the sample detected when the sample is irradiated with irradiation light generated by the controlled light source;
A program that causes a computer to perform

Images