WO2015087629A1

WO2015087629A1 - Image acquisition device and image acquisition method

Info

Publication number: WO2015087629A1
Application number: PCT/JP2014/078520
Authority: WO
Inventors: 順一坂上; 田中　健二; 和田　成司; 山根　健治
Original assignee: ソニー株式会社
Priority date: 2013-12-10
Filing date: 2014-10-27
Publication date: 2015-06-18

Abstract

Provided are an image acquisition device and an image acquisition method such that it is possible to acquire a clear image while a laser light source is disposed inside the housing body of the image acquisition device for the purpose of reducing the size of the image acquisition device. The image acquisition device (1) comprises: a light source (2) which emits a laser beam; a measuring unit (3) which is disposed within the same housing body as the light source (2), scans a sample (S) with the laser beam, receives the laser light and measures the intensity of the light to be measured from the sample (S); and a control unit (6) which generates an image of the sample (S) on the basis of the distribution of the measured intensity of the measured light. The measuring unit (3) measures the intensity of the laser beam which is emitted from the light source (2). On the basis of the measured intensity of the laser beam, the control unit (6) executes a control of the intensity of the laser beam which is emitted from the light source, and/or a correction of the distribution of the measured intensity of the measured light.

Description

Image acquisition apparatus and image acquisition method

The present disclosure relates to an image acquisition device and an image acquisition method.

Various microscopes are used in the fields of living organisms and biotechnology to observe the physiological reaction and morphology of living cells. Among these various types of microscopes are microscopes that observe an object by observing a fluorescent / phosphorescent phenomenon from living or non-living measurement data. For example, Patent Document 1 discloses an example of a fluorescence microscope.

In recent years, it has become possible to shorten the wavelength of a laser used as a light source, and as such a fluorescence microscope, a two-photon excitation microscope using a two-photon absorption process for material excitation has come to be used. It is coming.

Such a fluorescence microscope, for example, scans a measurement material with excitation light output from a laser light source and creates an intensity distribution of fluorescence from the measurement material, thereby generating an image of the measurement material based on the intensity distribution. To do.

JP 2012-8261 A

The laser light source used for the light source of the fluorescence microscope as described above is generally a large-sized one, and is used in a configuration that is externally attached to the microscope as a separate housing from the microscope. Tends to increase in size.

On the other hand, in recent years, laser light sources that are smaller than conventional ones have appeared, such as semiconductor lasers that utilize semiconductor recombination emission. Therefore, the miniaturization of the image acquisition device itself is being considered by providing a small laser light source such as a semiconductor laser in the housing of an image acquisition device that requires a light source such as a fluorescence microscope.

However, when the laser light source is provided in the housing of the image acquisition device, the laser light source is affected by heat generated by each device provided in the same housing, and the laser intensity changes. Appears as noise on the image.

Therefore, the present disclosure proposes a new and improved image acquisition apparatus and image acquisition method capable of downsizing the apparatus by providing a laser light source in the same housing and acquiring a clear image. To do.

According to the present disclosure, a light source that emits laser light, and a sample that is provided in the same housing as the light source, scans the sample with the laser light, and receives the laser light to measure the intensity of light to be measured from the sample. And a control unit that generates an image of the sample based on the measured intensity distribution of the measurement target light, and the measurement unit measures the intensity of the laser light emitted from the light source. The control unit performs at least one of control of the intensity of the laser light emitted from the light source and correction of the intensity distribution of the measured light to be measured based on the measured intensity of the laser light. An image acquisition device is provided for execution.

Further, according to the present disclosure, the sample is scanned with the laser light emitted from the light source provided in the same housing, the intensity of the measurement target light generated from the sample is received by receiving the laser light, and the processor Is based on the measured intensity distribution of the measurement target light, generating an image of the sample, measuring the intensity of the laser light emitted from the light source, and measuring the intensity of the laser light. Based on this, there is provided an image acquisition method including control of the intensity of laser light emitted from the light source and at least one of correction of the measured intensity distribution of the measurement target light.

As described above, according to the present disclosure, there is provided an image acquisition apparatus and an image acquisition method capable of downsizing the apparatus by providing a laser light source in the same housing and acquiring a clear image. The

Note that the above effects are not necessarily limited, and any of the effects shown in the present specification, or other effects that can be grasped from the present specification, together with or in place of the above effects. May be played.

FIG. 3 is an explanatory diagram illustrating an example of a schematic configuration of an image acquisition device according to a first embodiment of the present disclosure. It is explanatory drawing for demonstrating fluorescence. It is explanatory drawing for demonstrating the principle of a confocal microscope. It is explanatory drawing for demonstrating the principle of a confocal microscope. It is explanatory drawing for demonstrating the difference between the fluorescence by 1 photon excitation and the fluorescence by 2 photon excitation. It is a graph for demonstrating the light absorption characteristic of a biological body. It is explanatory drawing for demonstrating a two-photon excitation fluorescence microscope. It is explanatory drawing for demonstrating the sample scanning system of a microscope. It is a system diagram showing an example of a schematic system configuration of an image acquisition device according to a comparative example. It is explanatory drawing which showed an example of the structure of the optical system of the image acquisition apparatus which concerns on a comparative example. It is explanatory drawing for demonstrating an example of a structure of a microscope unit. It is explanatory drawing for demonstrating an example of a function structure of the image acquisition apparatus which concerns on a comparative example. It is a schematic diagram which shows the structure of the light source concerning the embodiment in detail. It is a characteristic view which shows the state which raised the peak power of the laser by intermittent light emission. FIG. 3 is an explanatory diagram illustrating an example of a configuration of an optical system of the image acquisition device according to the first embodiment of the present disclosure. It is an explanatory view for explaining an example of a functional configuration of the image acquisition device according to the embodiment. It is explanatory drawing for demonstrating the detailed function structure of the distribution information generation part and system control part which concern on the embodiment. It is explanatory drawing for demonstrating an example of operation | movement of the two-dimensionalization process part which concerns on the embodiment. It is explanatory drawing for demonstrating the outline | summary of the correction process which concerns on the same embodiment. It is the figure which showed an example of the file format of the RAW file which concerns on the same embodiment. It is explanatory drawing for demonstrating the principle of correction | amendment of intensity distribution. It is a flowchart for demonstrating the flow of the process concerning correction | amendment of intensity distribution. It is explanatory drawing for demonstrating the principle of intensity | strength control of the emitted light from a light source. It is a flowchart for demonstrating the flow of the process which concerns on the intensity | strength control of the emitted light from a light source. It is explanatory drawing for demonstrating the principle of intensity | strength control of the emitted light from the light source at the time of warm-up. 5 is a flowchart showing a flow of a series of processes related to image display of the information processing apparatus according to the embodiment. It is the flowchart which showed an example of the process regarding the noise correction of the information processing apparatus which concerns on the embodiment. It is explanatory drawing which showed an example of the structure of the optical system of the image acquisition apparatus which concerns on 2nd Embodiment of this indication. It is an explanatory view for explaining an example of a functional configuration of the image acquisition device according to the embodiment. It is explanatory drawing for demonstrating the detailed function structure of the distribution information generation part and system control part which concern on the embodiment. It is a figure for demonstrating an example of the designation | designated method of observation object. It is the figure which showed an example of the file format of the RAW file which concerns on the same embodiment. 5 is a flowchart illustrating an example of a flow of a series of operations of the image acquisition apparatus according to the embodiment. It is explanatory drawing for demonstrating the principle of wavelength control of the laser beam radiate | emitted from a light source. It is explanatory drawing for demonstrating the principle of wavelength control of the laser beam radiate | emitted from a light source. It is explanatory drawing for demonstrating the principle of wavelength control of the laser beam radiate | emitted from a light source. It is a flowchart for demonstrating the flow of the process which concerns on the wavelength control of the laser beam radiate | emitted from a light source. It is explanatory drawing for demonstrating an example of a function structure of the image acquisition apparatus which concerns on 3rd Embodiment of this indication. It is explanatory drawing for demonstrating the detailed function structure of the distribution information generation part and system control part which concern on the embodiment. It is an explanatory view for explaining an outline of an image acquisition device concerning a 4th embodiment of this indication. It is explanatory drawing for demonstrating the principle of the correction process which concerns on the same embodiment. It is explanatory drawing for demonstrating the detailed function structure of the distribution information generation part and system control part which concern on the embodiment. It is the figure which showed an example of the file format of the RAW file which concerns on the same embodiment. It is an example of the hardware constitutions of an image acquisition apparatus.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be made in the following order.
1. 1. First embodiment 1.1. Outline of image acquisition apparatus 1.2. About Microscope 1.2.1. Microscope type: Confocal microscope 1.2.2. Microscope type: Two-photon excitation microscope 1.3. Image acquisition apparatus according to comparative example 1.3.1. Configuration of optical system 1.3.2. Configuration of microscope unit 1.3.3. Functional configuration of image acquisition apparatus 1.4. Problems of image acquisition device according to comparative example 1.5. Configuration of image acquisition apparatus 1.5.1. Overview 1.5.2. Configuration of light source 1.5.3. Configuration of optical system 1.5.4. Functional configuration of image acquisition apparatus 1.6. RAW file format 1.7. Details of correction processing 1.7.1. Principle of correction 1.7.2. Flow of operations related to correction 1.8. Details of laser light intensity control 1.8.1. Principle of strength control 1.8.2. Flow of operation related to intensity control 1.9. Control of laser beam intensity during warm-up 1.10. Operation of information processing apparatus 1.11. Summary 2. Second Embodiment 2.1. Outline of image acquisition apparatus 2.2. Configuration of image acquisition apparatus 2.2.1. Configuration of optical system 2.2.2. Functional configuration of image acquisition apparatus 2.3. RAW file format 2.4. Flow of operation of image acquisition device 2.5. Details of wavelength control 2.5.1. Principle of wavelength control: When observing a sample with multiple observation wavelengths 2.5.2. One mode of wavelength control: when observing a sample with a single observation wavelength 2.6. Summary 3. Third embodiment 3.1. Outline of image acquisition device 3.2. Configuration of image acquisition apparatus 3.2.1. Configuration of optical system 3.2.2. Functional configuration of image acquisition apparatus 3.3. Summary 4. Fourth embodiment 4.1. Outline of image acquisition device 4.2. Configuration of image acquisition device 4.3. RAW file format 4.4. Summary 5. Hardware configuration Summary

<1. First Embodiment>
[1.1. Overview of image acquisition device]
First, an overview of an image acquisition device according to the first embodiment of the present disclosure will be described with reference to FIG. FIG. 1 is an explanatory diagram illustrating an example of a schematic configuration of an image acquisition device according to the first embodiment of the present disclosure.

As shown in FIG. 1, the image acquisition device 1 according to the present embodiment includes a light source 2, a measurement unit 3, a control unit 6, and an I / F (Interface) 7.

The image acquisition device 1 according to the present embodiment scans the sample S with light emitted from the light source 2 and measures the fluorescence emitted from the sample S by the measurement unit 3, thereby measuring the sample based on the measured fluorescence. The image of S is acquired. In addition, like the fluorescence in this case, the light obtained from the sample S corresponds to an example of “measurement target light”.

Particularly, in the image acquisition device 1 according to this embodiment, at least the light source 2 and the measurement unit 3 are provided in the image acquisition device 1 (that is, in the same housing). For example, in the example illustrated in FIG. 1, the light source 2, the measurement unit 3, the control unit 6, and the I / F 7 are provided in the image acquisition device 1.

The measurement unit 3 includes a microscope unit 4 that functions as a microscope, which will be described later, and a scanning system (detection system) 5 that performs scanning of the sample S and detection of fluorescence emitted from the sample S.

Also, the control unit 6 controls the operation of the light source 2 and the measurement unit 3, and converts the light measured (detected) by the measurement unit 3 into an image. The control unit 6 controls the operation of the light source 2 based on the light measured (detected) by the measurement unit 3 so that the light source 2 operates stably.

The I / F 7 is an interface for the image acquisition device 1 to transmit / receive information to / from a user or another device. For example, the image acquisition apparatus 1 is connected to an external network via the I / F 7 (for example, a communication interface), and is generated by the control unit 6 in the information processing apparatus 800 connected via the external network. An image may be output.

Details of the light source 2, the measurement unit 3, the control unit 6, and the I / F 7 of the image acquisition device 1 according to the present embodiment will be separately described later as “1.5. Configuration of the image acquisition device”.

Note that the image acquisition device 1 according to the present embodiment does not necessarily have other configurations (for example, the control unit 6 and the I / F 7) as long as at least the light source 2 and the measurement unit 3 are provided in the same housing. The image acquisition apparatus 1 may not be provided. For example, an information processing device capable of transmitting and receiving information between the light source 2 and the image acquisition device 1 including the measurement unit 3 is separately provided, and the control unit 6 and the I / F 7 are provided in the information processing device. Also good. Of course, it goes without saying that the control unit 6 and the I / F 7 may be provided on the information processing apparatus 800 side.

[1.2. About the microscope]
Here, prior to the detailed description of the image acquisition device 1 according to the present embodiment, FIGS. 2 to 8 show a microscope used as the microscope unit 4 of the measurement unit 3 in the image acquisition device 1 according to the present embodiment. Details will be described with reference to FIG.

Fluorescence emitted from the sample S can be cited as one of the phenomena occurring in the sample S of interest in the present embodiment. In the following, first, fluorescence will be briefly described with reference to FIG. FIG. 2 is an explanatory diagram for explaining fluorescence.

When light of a predetermined wavelength is irradiated to a molecule that constitutes the measurement sample (or adheres to the measurement sample), the electrons in the molecule correspond to the ground state using the energy of the irradiated light. May shift from the energy level to the energy level corresponding to the excited state. The light irradiated at this time is called excitation light. When a molecule in the ground state is excited to produce a singlet excited state, the excited electron moves to one of the energy levels corresponding to the singlet excited state. , Moving to lower energy levels while releasing energy by internal conversion. When electrons in the excited state return to the ground state, energy may be emitted as light, and the light emitted at this time is the fluorescence of interest in the present embodiment.

<< 1.2.1. Microscope type: Confocal microscope >>
As one of optical microscopes used for observing such fluorescence, there is a confocal microscope as shown in FIGS. Hereinafter, the principle of the confocal microscope will be briefly described with reference to FIGS. 3 and 4. 3 and 4 are explanatory diagrams for explaining the principle of the confocal microscope.

The confocal fluorescence microscope shown in FIG. 3 uses a laser beam as excitation light, guides the laser beam to a measurement sample (fluorescence specimen), and guides the fluorescence generated at the focal plane of the measurement sample to a detector. It has a configuration like this. Here, the laser beam used as the excitation light can be regarded as a point light source by passing through the pinhole A, and the laser beam passes through the dichroic mirror and the objective lens and is projected onto the fluorescent specimen. In the fluorescent specimen, fluorescence is generated by the energy of the projected laser beam, and the emitted fluorescence is collected by the objective lens and guided to the detector by the dichroic mirror. A pinhole B is installed immediately before the detector, and the fluorescence that has passed through the pinhole B is detected by a detector such as a photomultiplier tube (PMT).

Here, the wavelength of the laser beam used as the excitation light can be appropriately selected according to, for example, the type of fluorescent dye used for staining the measurement sample, and is not limited to a specific wavelength. .

In such a confocal fluorescence microscope, the installation position of the pinhole A, the projection position of the point light source (focal plane of the measurement sample), and the installation position of the pinhole B are optically conjugate with each other. This three-point conjugate relationship is said to be confocal.

At this time, as shown in FIG. 4, the fluorescence emitted from the focal plane (the focused surface) of the objective lens is collected by the objective lens and passes through the confocal pinhole (pinhole B in FIG. 3). Fluorescence from out-of-focus parts can pass through the confocal pinhole, although it can pass through. As a result, with the confocal fluorescence microscope, it is possible to obtain luminance information of only the portion of the measurement sample in focus. Therefore, a two-dimensional image (optical tomographic image) of only a focused portion can be constructed by scanning the plane (sample surface) of the measurement sample in the vertical direction and the horizontal direction. In addition, the scanning of the sample surface is repeated while changing the focal position, and fluorescence from measurement samples at different depth positions (depth positions) is accumulated to collect a set of optical tomographic images (3 Dimensional enlarged image group) can be obtained.

<< 1.2.2. Microscope types: Two-photon excitation microscope >>
There is a two-photon excitation microscope as another method capable of obtaining a three-dimensional image. FIG. 5 is an explanatory diagram for explaining the principle of two-photon excitation. The diagram on the left side of FIG. 5 is an explanatory diagram showing the principle of normal fluorescence explained earlier. Excitation light is generated by exciting a molecule with excitation light having a certain wavelength (in the figure, excitation light having a wavelength of 350 nm). Longer wavelength fluorescence (fluorescence with a wavelength of 500 nm in the figure) is emitted. Such a mechanism of fluorescence generation is so-called fluorescence generation by one-photon excitation because it is a mechanism in which fluorescence is generated when a molecule is excited by the interaction between one photon and a molecule.

On the other hand, as shown in the diagram on the right side of FIG. 5, while a molecule is excited to a virtual level by one photon, the molecule is further excited by another photon. Fluorescence may occur when molecules in the excited state are excited to the transition to the ground state. Such a mechanism of fluorescence generation is called fluorescence generation by two-photon excitation because fluorescence is generated when the molecule is excited by the interaction between two photons and the molecule. By utilizing fluorescence generation by two-photon excitation, it becomes possible to generate fluorescence having a wavelength shorter than that of the excitation light (in the example of FIG. 5, fluorescence having a wavelength of 500 nm using infrared light having a wavelength of 700 nm as excitation light). Has occurred.)

In order for two-photon excitation to be established, a molecule that collided with the first photon collided with another photon within an extremely short time of about 1.0 × 10 ⁻¹⁶ seconds when it was excited to a virtual level. Therefore, a high photon density is required, and a laser light source capable of outputting a high peak power is used. Further, since the emitted fluorescence signal is very weak compared to the one-photon excitation process, it is required to use an optical system with little loss and a sensitive detector.

In spite of these points, the main reason for the demand for a two-photon excitation fluorescence microscope is that the fluorescence wavelength in the infrared band of about 700 nm to 1000 nm used for two-photon excitation is water or water as shown in FIG. This is because it has a wavelength range called “biological window” that easily penetrates the living body without being absorbed by hemoglobin. Because of such high transmittance of excitation light, it can be observed only to a depth of about 100 μm with a confocal fluorescence microscope, whereas it can be observed to a depth of 1000 μm (1 mm) with a two-photon excitation fluorescence microscope. It has been broken.

In addition, as shown in FIG. 7, in the one-photon excitation fluorescence microscope, molecules are excited throughout the thickness direction of the sample other than the focal plane, and fluorescence is emitted from the entire thickness direction, whereas two-photon excitation fluorescence is emitted. In the microscope, only the vicinity of the focal plane is excited. This makes it possible to brightly observe deeper portions of the sample, and minimizes fluorescence quenching and light damage even when repeated vertical and horizontal scans are performed at different focal planes. To the limit. Moreover, since phototoxicity can be minimized for the same reason, it becomes possible to observe living cells existing deep in the sample for a long time.

Also, in the two-photon excitation fluorescence microscope, the fluorescence emitted by excitation is derived from a minute region in the sample, and thus a fluorescence image can be obtained by detecting all the light signals. Thereby, in the two-photon excitation fluorescence microscope, the detection optical system in the microscope can be simplified. That is, in the two-photon excitation process, since fluorescence is emitted only from the sample near the focal position, there is no need to cut extra signals using a pinhole as in the confocal fluorescence microscope, and the detector is located near the sample. Is arranged so as to pick up as many fluorescent signals as possible to diffuse in all directions.

<< 1.2.3. Microscope scanning method >>
In the laser scanning microscope as described above, in many cases, a two-dimensional image (by scanning in the X direction and Y direction (vertical direction and horizontal direction) of the measurement sample using two types of galvanometer mirrors ( Optical tomogram). FIG. 8 shows a configuration example of a confocal fluorescence microscope using two types of galvanometer mirrors.

Excitation light emitted from a laser light source passes through an optical system such as a lens and a pinhole provided at a conjugate position, and then passes through a dichroic mirror that transmits the excitation light and reflects fluorescence. The excitation light that has passed through the dichroic mirror passes through an optical system such as a lens, and after the X coordinate is controlled by an X direction galvanometer mirror that controls scanning of the measurement sample in the X direction, the Y direction controls the Y direction scanning. The Y coordinate is controlled by the galvanometer mirror, and the light is condensed to a desired XY coordinate on the measurement sample by the objective lens.

Fluorescence emitted from the measurement sample is reflected by the Y-direction galvanometer mirror and the X-direction galvanometer mirror, follows the same path as the excitation light, and is reflected by the dichroic mirror. The fluorescence reflected by the dichroic mirror is transmitted through a pinhole provided at the conjugate position and then guided to a detector such as a photomultiplier tube.

Here, the two galvanometer mirrors used for controlling the condensing position on the measurement sample have mirrors connected to a rotation axis as schematically shown in FIG. In the galvanometer mirror, the amount of rotation of the rotating shaft is controlled by the magnitude of the input voltage, and the angle at which the mirror surface is facing can be changed at high speed and with high accuracy.

[1.3. Image acquisition apparatus according to comparative example]
Next, in describing the details of the image acquisition device 1 according to the present embodiment, the configuration of the conventional image acquisition device will be described as a comparative example to organize the problems of the image acquisition device 1 according to the present embodiment. For example, FIG. 9 is an explanatory diagram illustrating an example of a schematic configuration of an image acquisition apparatus according to a comparative example. As shown in FIG. 9, the image acquisition device 1w according to the comparative example has a light source 2w provided outside the image acquisition device 1w.

<< 1.3.1. Configuration of optical system >>
Here, with reference to FIG. 10, the configuration of the optical system of the image acquisition device 1 w according to the comparative example is mainly such that the sample S is irradiated with light from the light source 2 w and excited by the light. A description will be given focusing on the part for detecting the fluorescence generated from.

As shown in FIG. 10, the optical system of the image acquisition device 1 w according to the comparative example includes a light source 2, a beam shaping lens 511, a galvano mirror 51,

lenses

513 and 515, a mirror 517, and a dichroic mirror 52. , Objective lens 42, imaging lens 521, emission filter 523, and photodetector 53. In the example shown in FIG. 10, a PMT is used as the photodetector 53. Hereinafter, it is assumed that a PMT is used as the light detector 53, and when “PMT 53” is described, “light detector 53” is indicated.

The excitation light (laser light) emitted from the light source 2 reaches the galvanometer mirror 51 with the beam diameter being expanded by the beam shaping lens 511 while being parallel light. The excitation light that has reached the galvanometer mirror 51 is reflected by the galvanometer mirror 51 and then guided to the dichroic mirror 52 via the lens 513, the mirror 517, and the lens 515. The lens 513, the mirror 517, and the lens 515 are an optical system for guiding the excitation light reflected by the galvanometer mirror 51 to the dichroic mirror 52.

The dichroic mirror 52 transmits the reached excitation light and guides it to the objective lens 42. The objective lens 42 condenses the excitation light on the sample S. The objective lens 42 and the imaging lens 521 enlarge the image of the sample S to a predetermined magnification, and form the enlarged image on the detection surface of the PMT 53.

When the sample S is irradiated with excitation light, a molecule in the sample S is excited by the excitation light and emits fluorescence. The fluorescence is reflected by the dichroic mirror 52 via the objective lens 42 and reaches the imaging lens 521, and is condensed by the imaging lens 521 and forms an image on the detection surface of the PMT 53 via the emission filter 523. For example, the emission filter 523 transmits only the emitted color by absorbing light (external light) other than the color light expanded by the objective lens 42. The colored light image from which the external light is lost is formed on the PMT 53.

<< 1.3.2. Configuration of microscope unit >>
Next, an example of the configuration of the microscope unit 4 will be described with reference to FIG. FIG. 11 is an explanatory diagram for explaining an example of the configuration of the microscope unit 4. As shown in FIG. 14, the microscope unit 4 includes

non-polarizing beam splitters

41 and 44, an objective lens 42, a filter 43, a camera 45, and an eyepiece 46. In the example shown in FIG. 11, each configuration of the scanning system (detection system) 5 w corresponds to the configuration given the same reference numeral in the optical system shown in FIG. 10. In the example shown in FIG. 11, a part of the configuration shown in FIG. 10 is omitted.

Excitation light (laser light) emitted from the light source 2w is guided to the non-polarizing beam splitter 41 in the microscope unit 4 through the scanning system (detection system) 5w. The non-polarizing beam splitter 41 reflects the excitation light guided through the scanning system (detection system) 5 w toward the objective lens 42. The objective lens 42 corresponds to the objective lens 42 in the optical system shown in FIG.

The objective lens 42 condenses the excitation light on the sample S. As a result, a molecule in the sample S is excited by the excitation light and emits fluorescence. Excitation light generated from the sample S is guided to the non-polarizing beam splitter 41 via the objective lens 42. The non-polarizing beam splitter 41 branches the fluorescence guided through the objective lens 42 toward both the scanning system (detection system) 5w and the filter 43.

The filter 43 selectively transmits the emission color guided through the objective lens 42 and the non-polarizing beam splitter 41. As a specific example, an emission filter may be applied as the filter 43. In this case, the filter 43 absorbs light (external light) other than the colored light and transmits only the emission color. The colored light image from which the external light has been lost is guided to the non-polarizing beam splitter 44. The non-polarizing beam splitter 44 branches the colored light image guided through the filter 43 toward both the camera 45 and the eyepiece 46. With such a configuration, the user can observe the image of the sample guided through the eyepiece 46 or can capture the image of the sample with the camera 45.

Further, the fluorescence guided toward the scanning system (detection system) 5w is reflected by the dichroic mirror 52, guided to the PMT 53, and detected by the PMT 53. The processing based on the fluorescence detected by the PMT 53 will be described later as the operation of the control unit 6w.

<< 1.3.3. Functional configuration of image acquisition device >>
Next, the functional configuration of the image acquisition device 1w according to the comparative example will be described with particular attention to the configuration of the control unit 6w and the I / F 7 with reference to FIG.

As shown in FIG. 12, an image acquisition device 1w according to a comparative example has a light source 2 connected to the outside of the device, and excitation light emitted from the light source 2 is transmitted to a scanning system (detection system) 5w and a microscope unit 4. To the sample S. At this time, the operation of the galvanometer mirror 51 is controlled by the system control unit 61 described later, so that the sample S is scanned with the excitation light.

Fluorescence generated from the sample S by the irradiated excitation light is guided to the scanning system (detection system) 5w through the microscope unit 4 and detected by the PMT 53 of the scanning system (detection system) 5w. The PMT 53 converts the detected fluorescence into an electrical signal by a photoelectric effect at a preset sampling rate, and outputs the electrical signal to the control unit 6w as data indicating the intensity of the fluorescence.

The control unit 6w includes a system control unit 61w, a two-dimensional processing unit 62w, a correction processing unit 63w, an image processing unit 64, a RAW image generation unit 65, a storage unit 66, a display control unit 67, a communication And a control unit 68. The I / F 7 includes a condition specifying unit 71, a display unit 72, and a communication unit 73.

The system control unit 61w controls the operation of the light source 2 and the galvanometer mirror 51 based on the measurement conditions specified by the user via the condition specifying unit 71. The condition specifying unit 71 is an input I / F for the user to specify conditions related to measurement and image acquisition (hereinafter, sometimes collectively referred to as “measurement conditions”). For example, the user designates the output of the light source 2 and the scanning condition relating to the scanning on the sample S (for example, the scanned range and the resolution of the generated image) via the condition designating unit 71 as the measurement conditions. Is possible.

The system control unit 61w controls the operation of the galvano mirror 51 based on a scanning condition specified as a measurement condition by the user, and outputs control information indicating the content of the control to a two-dimensionalization processing unit 62w described later.

The two-dimensionalization processing unit 62w sequentially acquires data indicating the intensity of fluorescence detected (measured) by the PMT 53 from the PMT 53 at a preset sampling rate. Further, the two-dimensionalization processing unit 62w sequentially acquires control information indicating the control content of the galvano mirror 51 from the system control unit 61w. Thereby, the two-dimensionalization processing unit 62w recognizes which data on the sample S corresponds to the data indicating the fluorescence intensity acquired from the PMT 53 based on the control information acquired from the system control unit 61w. It becomes possible. That is, the two-dimensionalization processing unit 62w arranges the data indicating the fluorescence intensity sequentially acquired from the PMT 53 based on the control information acquired from the system control unit 61w and converts the data into a two-dimensional data, thereby obtaining the detected fluorescence intensity distribution. Generate.

The intensity distribution of the fluorescence generated by the two-dimensional processing unit 62w is subjected to correction processing such as noise removal by the correction processing unit 63w and is output to the image processing unit 64, for example. The image processing unit 64 generates image data by performing image processing such as compression processing on the intensity distribution subjected to the correction processing.

The image processing unit 64 outputs the generated image data to the display control unit 67, for example. Further, the image processing unit 64 may output the generated image data to the communication control unit 68. Further, the image processing unit 64 may store the generated image data in the storage unit 66.

The RAW image generation unit 65 acquires the fluorescence intensity distribution from the two-dimensional processing unit 62w, and generates a RAW file by shaping the fluorescence intensity distribution as image data (RAW image) into a predetermined file format. . At this time, the RAW image generation unit 65 displays information related to the fluorescence intensity distribution, such as a fluorescence intensity distribution acquisition condition (for example, a shooting condition or a scanning condition such as a parameter at the time of shooting). The acquired information may be associated with the generated RAW file as related information. The related information may be recorded in the RAW file as incidental information of the RAW file, or may be associated as a separate file from the RAW file. In the following, these modes may be collectively referred to simply as “associate”.

The RAW image generation unit 65 outputs the generated RAW file to the communication control unit 68, for example. Further, the RAW image generation unit 65 may store the generated RAW file in the storage unit 66.

The storage unit 66 is a storage unit for storing various control data used in the image acquisition device 1w and data generated in the image acquisition device 1w (for example, image data and RAW files). The storage unit 66 may be configured as a database, for example. Further, the storage unit 66 may be used as a storage area for storing data temporarily created to execute various processes such as image processing.

The display control unit 67 acquires image data from the image processing unit 64 and causes the display unit 72 to display the acquired image data. The display unit 72 is an output I / F for displaying information such as a display. Thereby, the user can check the image of the sample S through the display unit 72.

Also, the display control unit 67 may cause the display unit 72 to display a U / I (User Interface) for operating the image acquisition device 1w. Note that the control data for the display control unit 67 to generate U / I may be stored in advance in the storage unit 66, for example.

The communication control unit 68 controls the operation of the communication unit 73 that performs communication with the external device so that the image acquisition device 1w transmits and receives data to and from the external device such as the information processing device 800 via the network. To do. For example, the communication control unit 68 acquires image data or a RAW file generated in the image acquisition device 1w, and controls the communication unit 73 to acquire the acquired data to the information processing device 800 via the network. Send. Accordingly, for example, the generated image data and RAW file can be processed by the information processing apparatus 800 having higher image processing capability than the image acquisition apparatus 1w.

[1.4. Problem of image acquisition device according to comparative example]
As described above, the image acquisition device 1w according to the comparative example includes the light source 2w outside (that is, externally) the image acquisition device 1w, and includes a series including the light source 2w and the image acquisition device 1w. The system tends to become larger. On the other hand, the image acquisition device 1 according to the present embodiment is downsized by incorporating the light source 2 (that is, the laser module) in the housing.

On the other hand, when the light source 2 is provided in the same housing as the measurement unit 3 and the control unit 6, the light source 2 is provided close to the measurement unit 3 and the control unit 6. For this reason, the operation of the light source 2 becomes unstable due to heat generated by the measurement unit 3 and the control unit 6, for example, the intensity of the laser light output from the light source 2 changes, and the change in the intensity causes noise in the generated image. May become apparent.

In particular, when the observation result of the microscope unit 4 (for example, a two-photon excitation microscope) is imaged as in the image acquisition device 1 according to the present embodiment, an image is generated by scanning the sample S. The imaging speed depends on the control speed of the galvanometer mirror 51 used for scanning and the detection speed of the PMT 53. Due to such a configuration, the shooting speed is often slower than a device that generates an image using an imaging element such as a so-called camera, and the shooting speed tends to decrease in proportion to an increase in image size. Therefore, when the sample S is irradiated with the excitation light emitted from the light source 2 and an image is generated based on the fluorescence from the sample S, there is a thermal variation in the intensity of the laser light emitted from the light source 2. In many cases, it appears as noise on the image.

[1.5. Configuration of image acquisition apparatus]
<< 1.5.1. Overview >>
In view of the above problems, the image acquisition device 1 according to the present embodiment is miniaturized by incorporating the light source 2 in the device (in the same housing), and thermal fluctuation of the laser light emitted from the light source 2. This is to reduce the noise associated with the image and to obtain a clearer image.

Specifically, the image acquisition device 1 according to the present embodiment measures the intensity of the laser light emitted from the light source 2, and based on the measured intensity distribution of the laser light, the laser light emitted from the light source 2 is measured. At least one of intensity control and correction of the generated image is executed.

For example, the image acquisition device 1 according to the present embodiment corrects the generated image based on the measured intensity distribution of the laser beam, thereby eliminating noise associated with the thermal fluctuation of the laser beam that is manifested on the image. It becomes possible to reduce.

Further, the image acquisition device 1 according to the present embodiment can stabilize the laser beam that has become unstable due to thermal fluctuation by controlling the intensity of the laser beam based on the measured intensity distribution of the laser beam. It becomes possible.

Hereinafter, for the image acquisition device 1 according to the present embodiment, first, an example of the configuration of the light source 2 will be described, and then the optical system and functional configuration of the image acquisition device 1 according to the present embodiment will be described. Since the configuration of the microscope unit 4 is the same as that of the image acquisition device 1w according to the comparative example described above, detailed description thereof is omitted.

<< 1.5.2. Structure of light source >>
First, an example of the configuration of the light source 2 used in the image acquisition device 1 according to the present embodiment will be described. In the image acquisition device 1 according to the present embodiment, the light source 2 includes an oscillating unit such as an optical parametric oscillator, and the wavelength of the emitted laser light can be changed by changing the oscillation condition of the oscillating unit. A structured light source is used. Hereinafter, an example of the configuration of the light source 2 will be described with reference to FIG. 13A. FIG. 13A is a schematic diagram showing an example of the configuration of the light source 2 in detail.

The light source 2 includes a MOPA 210 combining a mode-locked oscillation laser and an optical amplifier, and a wavelength conversion module (OPO) 250. The MOPA 210 includes a mode-locked laser section 220, an optical isolator section 230, and an optical amplifier section (SOA section) 240.

In the lower part of FIG. 2, pulse waveforms L1, L2, and L3 of the laser beams output from the mode-locked laser unit 220, the optical amplifier unit 240, and the wavelength conversion module 250, and pulses for intermittent driving described later. Each waveform P1 is shown.

The mode-locked laser unit 220 includes a semiconductor laser 222 and elements of a lens 224, a band-pass filter 226, and a mirror 228 that allow laser light emitted from the semiconductor laser 222 to pass therethrough. The bandpass filter 226 has a function of transmitting light in a certain wavelength range and not allowing light outside the range to pass. An external resonator (spatial resonator) is formed between the mirror at the rear end face of the semiconductor laser 222 and the mirror 228. The laser emitted from the mode-locked laser unit 220 by the path length of the external resonator. The frequency of light is determined. Thereby, it can be made to lock to a specific frequency compulsorily, and the mode of a laser beam can be locked.

The mode-locked laser unit 220 can synchronize a short pulse with a longer period (for example, about 1 GHz) than a normal semiconductor laser by configuring an external resonator. For this reason, the laser beam L1 output from the mode-locked laser unit 220 has a low average power and a high peak, has little damage to the living body, and has a high photon efficiency.

The optical isolator unit 230 is arranged at the subsequent stage of the mode-locked laser unit 220. The optical isolator unit 230 includes an optical isolator 232 and a mirror 234. The optical isolator unit 230 has a function of preventing light reflected by a subsequent optical component or the like from entering the semiconductor laser 222.

The optical amplifier section (SOA section) 240 functions as an optical modulation section that amplifies and modulates the laser light emitted from the semiconductor laser 222, and is disposed at the subsequent stage of the optical isolator section 230. The laser output from the mode-locked laser unit 220 is amplified by the optical amplifier unit 240 because its power is relatively small. The optical amplifier 240 is composed of an SOA (Semiconductor Optical Amp), that is, a semiconductor optical amplifier 242 and an SOA driver 244 that controls the semiconductor optical amplifier 242. The semiconductor optical amplifier 242 is a small and low-cost optical amplifier, and can be used as an optical gate and an optical switch for turning light on and off. In the present embodiment, the laser light emitted from the semiconductor laser 222 is modulated by turning on and off the semiconductor optical amplifier 242.

In the optical amplifier section (SOA section) 240, the laser light is amplified according to the magnitude of the control current (DC). Furthermore, the optical amplifier unit 240 intermittently drives with the control current of the pulse waveform P1 shown in FIG. 2 at the time of amplification, thereby turning on / off the laser light of the pulse waveform L1 with a predetermined period T, and intermittently. Laser light (pulse waveform L2) is output. Thus, by generating a pulse waveform having a desired timing and period, it is possible to synchronize with a control signal included in the system. In this way, for example, in the case of a MOPA type light source, intermittent driving is realized by intermittent driving in the semiconductor optical amplifier 242 (Semiconductor Optical Amplifier abbreviated as SOA) in the rear stage that amplifies the pulse of the oscillation unit in the front stage. I can do it. In the case of a MOPA type light source, the optical amplifier unit 240 (semiconductor optical amplifier 242) functions as an intermittent light emitting unit.

In this embodiment, since MLLD is used, the frequency of the laser light output from the semiconductor laser 222 is, for example, 500 MHz to 1 GHz, and the pulse width is about 0.5 to 1.0 [ps]. As will be described later, by injecting a transmission synchronization signal from the oscillation synchronization signal injection unit 316, in addition to the intermittent drive synchronization by the SOA unit, the oscillation pulse output from the semiconductor laser 222 is also synchronized with the control signal of the system. It is possible. In the case of using TiSa, the oscillation frequency of laser light is about 40 MHz to 80 MHz and the pulse width is about 0.1 to 0.2 [ps], whereas a laser having a higher oscillation frequency can be obtained by using MLLD. It is possible to output light.

In this embodiment, the wavelength of the laser beam output from the optical amplifier unit 240 is 405 nm as an example. Since the wavelength of 405 nm is a wavelength with relatively high absorption, it is converted into a wavelength (about 900 nm to 1300 nm) that reaches the back of the living body and produces a two-photon effect at a high density. Therefore, the laser light output from the optical amplifier unit 240 is input to the subsequent wavelength conversion module 250 and wavelength-converted by the LBO 252 of the wavelength conversion module 250.

As an example, the LBO 252 of the wavelength conversion module 250 converts the input laser beam (pulse waveform L2) into two wavelengths. One of the converted two-wavelength laser light corresponds to signal light, and the other corresponds to idler light. In the light source 2 according to this embodiment, one of the signal light and the idler light is output from the wavelength conversion module 250 to the outside, and is irradiated onto the object (sample S).

In the example shown in FIG. 13A, the long wavelength laser light (pulse waveform L3) of the two converted laser lights is output from the wavelength conversion module 250 as signal light and is applied to the object (sample S). Irradiated. In this case, of the converted two-wavelength laser light, the short-wavelength laser light corresponds to idler light.

By the way, in observation of a living body with a laser microscope, it is effective to increase the peak power by reducing the average power of the laser in order to reduce the damage to the object. Further, since the laser chip constituting the MOPA type light source using the semiconductor laser is small, it is considered that the operation limit is determined by heat generated by a high power load.

In the light source 2 of this embodiment, since intermittent operation is performed by outputting intermittent laser light (pulse waveform L2), the average power is the same as compared with the case where intermittent operation is not performed. The peak when emitting light can be increased. Further, by performing intermittent operation, heat generation due to a high power load can be suppressed.

In this embodiment, it is used as the light source 2 for two-photon excitation, and the light source 2 excites the phosphor with two photons. In particular, in a microscope using a two-photon excitation light source, FOM (= (peak power) ² × pulse width × frequency = peak power × average power) is known as a figure of merit. According to this figure of merit, the output can be increased in proportion to the product of peak power and average power. Therefore, in the observation of a living body with a laser microscope, it is effective to increase the peak power by reducing the average power in order to increase the output while minimizing the damage to the object. For this reason, in this embodiment, the duty (DUTY = pulse width × frequency) ratio is lowered by performing intermittent operation, and the peak power is increased.

FIG. 13B is a characteristic diagram showing a state where the peak power of the laser is increased by intermittent light emission. The upper part of FIG. 13B shows the characteristics in the case of one-photon excitation. The upper left characteristic shows the peak power of continuous emission, and the right characteristic shows the peak power of intermittent emission when the DUTY ratio is 50%. ing. As described above, when the duty ratio of intermittent light emission is set to 50%, the signal intensity (2 × I ₀ ) that is twice as high in the case of intermittent light emission can be output with respect to the signal intensity (I ₀ ) of continuous light emission. it can.

The middle part of FIG. 13B shows the characteristics in the case of two-photon excitation. The characteristics on the left side show the peak power of continuous emission, and the characteristics on the right side show the peak power of intermittent emission when the DUTY ratio is 50%. ing. According to the figure of merit FOM, in the case of two-photon excitation, the figure of merit increases with the square of the peak power. Therefore, in the case of intermittent light emission, the signal intensity of two-photon excitation (= 4 × I ₀ ² ) is four times the signal intensity of continuous light emission (= I ₀ ² ). In addition, the average signal intensity of the two-photon excitation (= 2 × I ₀ ² ) is ^{2 with} respect to the signal intensity of the continuous emission (= I ₀ ² ) in the average signal intensity of the pulse emission point and the pulse non-emission point. Doubled. Therefore, according to the present embodiment, it is possible to increase the peak power and the average signal intensity by performing intermittent driving in the light source 2 of two-photon excitation.

The lower characteristic of FIG. 13B shows a signal obtained by passing the middle characteristic through a band-limited low-pass filter. Since the processing by the band-limited low-pass filter is inserted before the A / D conversion, when the on / off duty ratio is 50% (1/2), the signal amplitude before the A / D conversion becomes 1/2, and as a result In the intermittent light emission of two-photon excitation, twice the signal amplitude can be obtained. Also, when the on / off duty ratio is 1 / n, if the peak power is n times, the signal amplitude obtained by two-photon excitation is n times, so it is desirable that the duty is small. Since the peak power obtained from the light source 2 has an upper limit, it is preferable to select an appropriate value with a duty ratio of 1 or less.

<< 1.5.3. Configuration of optical system >>
Next, with reference to FIG. 14, the configuration of the optical system of the image acquisition device 1 according to the present embodiment is particularly different from the configuration of the optical system of the image acquisition device 1w according to the comparative example described above (see FIG. 10). The explanation will be given focusing on the part.

As described above, in the image acquisition apparatus 1 according to the present embodiment, the light source 2 includes the wavelength conversion module (OPO) 250, and the input laser light (pump light) is converted into laser light having two wavelengths (that is, signal light). Light and idler light) and output. In the image acquisition device 1 according to the present embodiment, one of the signal light and idler light output from the light source 2 is irradiated toward the sample S as excitation light. In the example illustrated in FIG. 14, the signal light is irradiated toward the sample S as excitation light.

In addition, about the structure which irradiates excitation light toward the sample S, it is the same as that of the optical system (refer FIG. 10) of the image acquisition apparatus 1w which concerns on the comparative example mentioned above. That is, the excitation light emitted from the light source 2 is guided to the objective lens 42 via the beam shaping lens 511, the galvano mirror 51, the lens 513, the mirror 517, the lens 515, and the dichroic mirror 52. It is condensed toward the sample S.

When the sample S is irradiated with excitation light, a molecule in the sample S is excited by the excitation light to emit fluorescence, and the fluorescence is emitted from the objective lens 42, the dichroic mirror 52, the imaging lens 521, and the emission. An image is formed on the detection surface of the PMT 53 via the filter 523. At this time, the emission filter 523 absorbs light (external light) other than the color light expanded by the objective lens 42 (that is, only the emitted color is transmitted), and the color light that has lost the external light is absorbed. An image is formed on the PMT 53.

Also, in the image acquisition device 1 according to the present embodiment, a PD (Photo Detector) 54 is provided to detect the intensity of the other signal light and idler light different from the one used as excitation light. In the example shown in FIG. 14, the PD 54 measures the intensity of idler light emitted from the light source 2.

<< 1.5.4. Functional configuration of image acquisition device >>
Next, an example of the functional configuration of the image acquisition apparatus 1 according to the present embodiment will be described with particular attention to the configuration of the control unit 6 with reference to FIG. In the example shown in FIG. 15, each configuration of the scanning system (detection system) 5 corresponds to the configuration given the same reference numeral in the optical system shown in FIG. 14. In the example shown in FIG. 15, a part of the configuration shown in FIG. 14 is omitted.

As shown in FIG. 15, the image acquisition device 1 according to the present embodiment includes a light source 2 inside the device (inside the housing), and excitation light emitted from the light source 2 is scanned with a scanning system (detection system) 5 and a microscope unit. 4 is irradiated to the sample S. At this time, the system controller 61 controls the operation of the galvanometer mirror 51, so that the sample S is scanned with the excitation light. In the following description, in order to make the description easy to understand, it is assumed that the signal light is used as the excitation light among the signal light and idler light emitted from the light source 2.

Fluorescence generated from the sample S by the irradiated excitation light is guided to the scanning system (detection system) 5 through the microscope unit 4 and detected by the PMT 53 of the scanning system (detection system) 5. The PMT 53 converts the detected fluorescence into an electrical signal by a photoelectric effect at a preset sampling rate, and outputs it to the control unit 6 as data indicating the intensity of the fluorescence.

The scanning system (detection system) 5 of the image acquisition apparatus 1 according to the present embodiment includes a PD 54, and the PD 54 measures the intensity of idler light at a preset sampling rate. The PD 54 converts the intensity of the measured idler light into an electric signal, and outputs it to the control unit 6 as data indicating the intensity of the idler light.

In addition, it may replace with PD54 and may provide PD56 which measures the intensity | strength of excitation light (signal light). In this case, a beam splitter 55 is provided on the optical path of the excitation light (signal light) irradiated toward the sample S, and a part of the excitation light (signal light) guided toward the sample S is supplied to the PD 56. You just have to branch towards. In the following description, it is assumed that the scanning system (detection system) 5 is provided with a PD 54 for measuring the intensity of idler light.

The control unit 6 includes a system control unit 61, a distribution information generation unit 62, a correction processing unit 63, an image processing unit 64, a RAW image generation unit 65, a storage unit 66, a display control unit 67, and communication control. Part 68. The I / F 7 includes a condition specifying unit 71, a display unit 72, and a communication unit 73. The operations of the storage unit 66, the display control unit 67, the communication control unit 68, the condition designating unit 71, the display unit 72, and the communication unit 73 are the same as those of the image acquisition device 1w according to the comparative example described above, and thus detailed description thereof will be made. Description is omitted.

The system control unit 61 controls the operations of the light source 2 and the galvanometer mirror 51 based on the measurement conditions specified by the user via the condition specifying unit 71. The condition specifying unit 71 is an input I / F for the user to specify measurement conditions related to measurement and image acquisition. For example, the user designates the output of the light source 2 and the scanning condition relating to the scanning on the sample S (for example, the scanned range and the resolution of the generated image) via the condition designating unit 71 as the measurement conditions. Is possible.

The system control unit 61 controls the operation of the galvano mirror 51 based on the scanning conditions specified by the user as the measurement conditions, and outputs control information indicating the contents of the control to the two-

dimensional processing units

621 and 625 described later. The operation so far is the same as that of the image acquisition device 1w (see FIG. 12) according to the comparative example described above.

The system control unit 61 may control operations of the light source 2 and the galvanometer mirror 51 based on control information indicating measurement conditions determined in advance. For example, when the sample S is known, the wavelength and output of the laser light output from the light source 2 may be controlled based on control information created in advance corresponding to the sample S. Note that these pieces of control information may be stored in advance in the storage unit 66, for example.

Further, the system control unit 61 according to the present embodiment acquires data indicating an intensity distribution based on the intensity of idler light from the distribution information generation unit 62 described later, and controls the operation of the light source 2 based on the acquired data. .

Here, a more detailed configuration of the system control unit 61 will be described with reference to FIG. FIG. 16 is an explanatory diagram for describing detailed functional configurations of the distribution information generation unit 62 and the system control unit 61 according to the present embodiment. As shown in FIG. 16, the system control unit 61 includes an intensity control amount determination unit 611.

The intensity control amount determination unit 611 acquires data indicating an intensity distribution based on the intensity of idler light measured by the PD 54 from the two-dimensionalization processing unit 621 of the distribution information generation unit 62 described later. Further, the intensity control amount determination unit 611 receives a notification of the maximum correction amount width from the intensity correction distribution determination unit 623 of the distribution information generation unit 62 described later. The intensity control amount determination unit 611 monitors the state of the light source 2 (particularly, the change in the intensity of the excitation light) based on the acquired data indicating the intensity distribution of idler light and the maximum width of the correction amount.

Also, the intensity control amount determination unit 611 generates control information for controlling the intensity of the laser light (pump light) emitted from the light source 2 based on the monitoring result. The system control unit 61 controls the intensity of the laser light (pump light) emitted from the light source 2 based on the control information generated by the intensity control amount determination unit 611.

With such a configuration, for example, the system control unit 61 makes the light source 2 stable when the operation of the light source 2 becomes unstable (for example, when the intensity of emitted laser light fluctuates). It is possible to control the intensity of the laser light (pump light) emitted from the light source 2 so as to operate. The details of the control of the light source 2, that is, the intensity control of the laser beam emitted from the light source 2 by the system control unit 61 will be separately described later as “1.8. Details of laser beam intensity control”.

Next, details of the distribution information generation unit 62 will be described with reference to FIGS. 15 and 16. As illustrated in FIG. 16, the distribution information generation unit 62 includes a two-dimensionalization processing unit 621, an intensity correction distribution determination unit 623, and a two-dimensionalization processing unit 625.

The two-dimensionalization processing unit 621 sequentially acquires data indicating the intensity of the idler light detected (measured) by the PD 54 from the PD 54 at a preset sampling rate. Further, the two-dimensionalization processing unit 621 sequentially acquires control information indicating the control content of the galvanometer mirror 51 from the system control unit 61. Thereby, the two-dimensionalization processing unit 621 recognizes which position on the sample S the data indicating the intensity of the idler light acquired from the PD 54 corresponds to based on the control information acquired from the system control unit 61. It becomes possible. That is, the two-dimensionalization processing unit 621 arranges the data indicating the idler light intensity sequentially acquired from the PD 54 based on the control information acquired from the system control unit 61 to two-dimensionalize the detected idler light intensity. Generate a distribution.

Here, refer to FIG. FIG. 17 is an explanatory diagram for describing an example of the operation of the two-dimensionalization processing unit 621. In FIG. 17, reference numeral C <b> 11 schematically shows the control content of the galvanometer mirror 51, i.e., the scan content on the sample S by the excitation light. The two-dimensionalization processing unit 621 two-dimensionalizes the data indicating the idler light intensity sequentially acquired from the PD 54 based on the scanning content indicated by the control information, and generates the intensity distribution D21.

As described above, the intensity of the laser light (pump light) output from the light source 2 changes under the influence of heat generated by each device (for example, the measurement unit 3 and the control unit 6) in the image acquisition device 1. There is a case. At this time, with the change in the intensity of the pump light, the intensity of the signal light and the idler light also changes in the same manner as the pump light. Therefore, a change in the intensity of idler light appears as, for example, the strength of the electrical signal converted by the PD 54.

In the generated intensity distribution D21, the end regions D211a and D211b at which the scanning direction is switched may be deleted as unnecessary portions, and the other regions D210 may be used as effective portions. The unnecessary portions D211a and D211b may be deleted by, for example, the two-dimensionalization processing unit 621 or a correction processing unit 63 described later. Also, the operation when the two-dimensionalization processing unit 625 described later generates an intensity distribution based on the intensity of the fluorescence detected by the PMT 53 is the same as the operation of the two-dimensionalization processing unit 621 described with reference to FIG. It is the same.

The two-dimensionalization processing unit 621 uses the data indicating the generated idler light intensity distribution (hereinafter simply referred to as “idler light intensity distribution”) as an intensity correction distribution determining unit 623 and an intensity control amount determining unit. To 611.

The intensity correction distribution determination unit 623 acquires the intensity distribution of idler light from the two-dimensionalization processing unit 621. Based on the acquired intensity distribution of idler light, the intensity correction distribution determination unit 623 generates correction data for the correction processing unit 63 described later to correct the fluorescence intensity distribution.

Further, the intensity correction distribution determining unit 623 obtains in advance the intensity distribution of idler light during the warm-up period, and calculates the maximum width of the correction amount based on the intensity distribution of idler light during the warm-up period. Also good. The maximum width of the correction amount calculated in this way is emitted from the light source 2 and the correction amount of the fluorescence intensity distribution based on the temporal fluctuation of the number of photons due to the fluctuation of the intensity of the laser emitted from the light source 2. The maximum width (margin) of the control amount of the intensity of laser light (pump light) is shown. Therefore, the intensity correction distribution determining unit 623 may notify the intensity control amount determining unit 611 of the calculated maximum correction amount width.

The intensity correction distribution determination unit 623 outputs the intensity distribution of idler light to the RAW image generation unit 65, and outputs correction data generated based on the intensity distribution of the idler light to the correction processing unit 63.

The two-dimensionalization processing unit 625 sequentially acquires data indicating the intensity of fluorescence detected (measured) by the PMT 53 from the PMT 53 at a preset sampling rate. Further, the two-dimensional processing unit 625 sequentially acquires control information indicating the control contents of the galvanometer mirror 51 from the system control unit 61. Thereby, the two-dimensionalization processing unit 625 recognizes, based on the control information acquired from the system control unit 61, which position on the sample S the data indicating the fluorescence intensity acquired from the PMT 53 corresponds to. Is possible. That is, the two-dimensionalization processing unit 625 arranges data indicating the fluorescence intensity sequentially acquired from the PMT 53 based on the control information acquired from the system control unit 61 and converts the data into a two-dimensional data, thereby obtaining the detected fluorescence intensity distribution. Generate.

The two-dimensionalization processing unit 625 outputs data indicating the generated fluorescence intensity distribution (hereinafter, simply referred to as “fluorescence intensity distribution”) to the correction processing unit 63 and the RAW image generation unit 65.

The correction processing unit 63 acquires the fluorescence intensity distribution from the two-dimensionalization processing unit 625. Further, the correction processing unit 63 acquires correction data generated based on the intensity distribution of idler light from the intensity correction distribution determining unit 623.

The correction processing unit 63 corrects the fluorescence intensity distribution based on the correction data. Here, an outline of correction by the correction processing unit 63 will be described with reference to FIG. It is explanatory drawing for demonstrating the outline | summary of the correction process which concerns on this embodiment. In FIG. 18, reference symbol D <b> 11 schematically indicates the fluorescence intensity distribution generated by the two-dimensional processing unit 625. Reference numeral D21 schematically indicates the intensity distribution of idler light generated by the two-dimensionalization processing unit 621. Reference numeral D10 indicates the fluorescence intensity distribution (that is, the RAW image) after the correction processing.

Note that, as described above, the intensity of the laser light (pump light) output from the light source 2 may change due to the influence of heat generated by each device in the image acquisition device 1, and similarly to the pump light, The intensity of the signal light and idler light also changes. Therefore, the change in the intensity of the signal light is manifested as noise in the fluorescence intensity distribution, and similarly, the change in the intensity of the idler light is manifested as noise in the intensity distribution of the idler light. On the other hand, the signal light and the idler light are output after the pump light is converted by the wavelength conversion module (OPO) 250. Therefore, the intensity change of the signal light and the intensity change of the idler light are always synchronized. That is, the intensity of the signal light and the intensity of the idler light are always in a proportional relationship.

Therefore, in the correction processing unit 63 according to the present embodiment, using such a property, noise that is manifested in the fluorescence intensity distribution based on the change in the intensity of the signal light is generated based on the intensity distribution of the idler light. Correct with the correction data. The correction processing unit 63 outputs the fluorescence intensity distribution corrected by the correction data to the image processing unit 64.

For details of the operation related to the generation of correction data by the intensity correction distribution determination unit 623 and the correction of the fluorescence intensity distribution based on the correction data by the correction processing unit 63, see “1.7. Details of Correction Processing”. Will be separately described later.

The image processing unit 64 acquires the fluorescence intensity distribution corrected by the correction data. The image processing unit 64 generates image data by performing image processing such as compression processing on the corrected fluorescence intensity distribution.

Further, the correction processing for the fluorescence intensity distribution D11 based on the correction data D21 described above may be performed by an external device such as the information processing device 800, for example. Therefore, the RAW image generation unit 65 according to the present embodiment may associate the idler intensity distribution D21 with the RAW file D1 generated with the fluorescence intensity distribution D11 as image data (RAW image).

In this way, by associating the intensity distribution D21 of idler light with the RAW file D1, the external apparatus (for example, the information processing apparatus 800) that has read the RAW file D1 has a laser intensity similar to that of the correction processing unit 63. It is possible to correct the noise associated with the fluctuations.

The configuration of the image acquisition device 1 according to the present embodiment has been described above with reference to FIGS. 13A, 13B, and FIGS. Next, more detailed contents of each component of the image acquisition device 1 according to the present embodiment will be described.

[1.6. RAW file format]
First, an example of the file format of the RAW file D1 according to the present embodiment will be described with reference to FIG. FIG. 19 is a diagram showing an example of the file format of the RAW file D1 according to the present embodiment.

As shown in FIG. 19, the RAW file D1 according to the present embodiment includes, for example, a data area d10, a basic control information area d30, and an extension area d20.

The data area d10 is an area for storing image data (actual data). As shown in FIG. 19, for example, the data area d10 includes a RAW main image d11, a child image IFD (Image File or Directory) d12, and a child image d121. The RAW main image d11 shows a RAW image. In the RAW file D1 according to the present embodiment, the fluorescence intensity distribution D11 is stored as the RAW main image d11. The fluorescence intensity distribution D11 stored as the RAW main image d11 is not limited to one (1 slice). For example, when a plurality of fluorescence intensity distributions D11 created by scanning in the XY direction (plane direction) are acquired along the Z direction (depth direction), the plurality of fluorescence intensity distributions D11 are acquired. May be stored in the RAW file D1 as the RAW main image d11.

In the data area d10, image data that has been subjected to image processing and images that have been subjected to image processing such as enlarged / reduced image data may be separately stored as child images. In this case, for example, a child image IFD d12 for storing child images may be provided, and the child image d121 may be stored in the child image IFD d12. The child image IFD d12 also plays a role of indicating the position (address) of the child image d121 in the RAW file D1.

In the basic control information area d30, information of a predetermined type is recorded as control information. For example, the basic control information area d30 includes a main image and shooting information IFDd31, a set reproduction JPEGd32, and a plaintext part manufacturer note IFDd33. The main image and shooting information IFDd31 is an IFD for storing, for example, image data used for playback and shooting information (for example, meta information such as EXIF (Exchangeable image file format)).

In the main image and shooting information IFDd31, for example, image data used for image reproduction (for example, data subjected to compression processing) is stored as a set reproduction JPEGd32.

Also, the plaintext maker note IFDd33 stores the model information of the image acquisition device 1 and information indicating the format such as encryption as control information. The control information may be stored in advance in the storage unit 66, for example, and the RAW image generation unit 65 may read the control information from the storage unit 66 and embed it in the RAW file D1.

The extended area d20 includes a manufacturer note IFDd21. The manufacturer note IFDd21 is an IFD for storing information not specified in EXIF, such as a camera control mode, and also stores shooting information and control information unique to the image acquisition device 1.

The manufacturer note IFDd21 according to the present embodiment includes, for example, an imaging condition d211, an idler light intensity image d212, a servo signal image d213, a captured image d214 during warm-up, a tampering prevention signal d215, and an automatic tracking adjustment. It includes a subject image d216 and an imaging synchronization signal d217.

The imaging condition d211 is control information indicating conditions that are not defined in EXIF among the imaging conditions when acquiring a RAW image (that is, fluorescence intensity distribution). In the shooting condition d211, control information indicating each condition is stored, for example, in the form of a parameter table.

The idler light intensity image d212 shows the idler light intensity distribution D21. When a plurality of fluorescence intensity distributions D11 are recorded as the RAW main image d11, idler intensity distributions D21 corresponding to the plurality of fluorescence intensity distributions D11 are used as idler light intensity images d212. To be recorded. Further, as the idler light intensity image d212, correction data calculated based on the idler light intensity distribution D21 may be used instead of the idler light intensity distribution D21.

In this way, by storing the idler light intensity image d212 in the RAW file D1, the external device (for example, the information processing device 800) that has read the RAW file D1 can perform the same as the correction processing unit 63 described above. It is possible to correct the RAW main image d11.

The servo signal image d213 is data obtained by two-dimensionalizing a servo signal for controlling the intensity of laser light emitted from the light source 2 by the system control unit 61 based on scanning conditions. By storing such a servo signal image d213 in the RAW file D1, the external device (for example, the information processing device 800) that has read the RAW file D1 causes the fluorescence intensity distribution (that is, the RAW image). It becomes possible to recognize what kind of control was performed on the light source 2 at the time of acquisition. For this reason, for example, the external device performs image processing content (control) on the fluorescence intensity distribution (that is, the RAW image) according to the control content recognized based on the servo signal image d213 in the RAW file D1. Parameter) can be changed as appropriate.

The photographed image d214 at the time of warm-up is an image obtained by two-dimensionalizing the intensity distribution of idler light and the intensity distribution of fluorescence acquired at the time of warm-up. Thus, by recording the captured image d214 at the time of warm-up in the RAW file D1, for example, the external device (for example, the information processing apparatus 800) that has read the RAW file D1 is in the warm-up period. It becomes possible to estimate the change in the intensity of the laser beam. Note that. The captured image d214 at the time of warming up may be recorded in the RAW file D1 at least when shooting is started during the warmup period, and may not be included in other cases. The details of the operation when photographing during the warm-up period will be described later separately as “1.9. Control of laser light intensity during warm-up”.

The falsification preventing signal d215 is a signal (data) for detecting the falsification when at least a part of the data in the RAW file D1 is falsified. As a specific example of the falsification preventing signal d215, there is a sticky bit.

The subject image d216 at the time of automatic tracking adjustment indicates an image acquired as a result of the tracking or enlargement / reduction when, for example, tracking or enlargement / reduction is performed. Instead of the subject image d216 at the time of automatic tracking adjustment, information indicating the contents of control related to tracking and enlargement / reduction may be recorded.

The inter-imaging synchronization signal d217 is a signal indicating the timing of each imaging related to the acquisition of each fluorescence intensity distribution D11 when the plurality of fluorescence intensity distributions D11 are stored in the RAW file D1 as the RAW main image d11. It is.

The file format of the RAW file D1 according to the present embodiment has been described above with reference to FIG. Note that the file format of the RAW file D1 described above is merely an example, and it is needless to say that not all information may be included. Also, image data such as the idler light intensity image d212, the servo signal image d213, and the captured image d214 at the time of warm-up are not necessarily two-dimensionalized if information necessary for reproducing the image data is recorded. There is no need to record it as data. For example, it goes without saying that an array of data corresponding to each pixel of image data may be recorded together with control information indicating scanning conditions (that is, information for making the array of data two-dimensional).

[1.7. Details of correction process]
Next, referring to FIGS. 20 and 21, the control unit 6 according to the present embodiment generates correction data based on the idler light intensity distribution and the operation related to correction of the fluorescent intensity distribution based on the correction data. Details will be described.

<< 1.7.1. Principle of correction >>
First, the principle of correction of the fluorescence intensity distribution in the image acquisition apparatus 1 according to the present embodiment will be described with reference to FIG. FIG. 20 is an explanatory diagram for explaining the principle of intensity distribution correction. As shown in FIG. 20, among the OPO laser modules constituting the light source 2, the pump light frequency output from the MOPA light source (fixed wavelength) is νp, the signal light frequency is νs, and the idler light frequency is νi. .

Here, the energy E per photon of light having a frequency ν is expressed by the following formula 1 when the Planck multiplier h, the speed of light c, and the wavelength λ of light are used.

Here, when the time variation of the number of photons due to the variation of the laser intensity is n (t), the frequency νp of the pump light, the frequency νs of the signal light, and the frequency νi of the idler light are expressed by Equation 2 below. Meets the requirements.

At this time, when excited with signal light having an energy of h * νs * n (t), the spatial fluorescence intensity PMTin (x, y, z) of the fluorescence generated at the coordinates (x, y, z) on the sample S ) Is represented by Equation 3 shown below.

In Equation 3 shown above, S (c / νs) represents the absorption luminous efficiency, and Q (x, y, z) represents the spatial fluorescence concentration distribution of the observation target (ie, sample S). Show.

As shown in Equation 3, in general, the spatial fluorescence intensity PMTin (x, y, z) will remain the time fluctuation n (t) of the number of photons accompanying the laser intensity fluctuation.

On the other hand, the control unit 6 according to the present embodiment estimates the time fluctuation n (t) of the number of photons accompanying the fluctuation of the laser intensity based on the intensity distribution of the idler light detected by the PD 54.

Specifically, the intensity PDin (x, y, z) of the idler light when scanning the coordinates (x, y, z) on the sample S is the signal light and the intensity of the pump light. Since it fluctuates in the same manner, it is expressed by the following equation 4.

The control unit 6 calculates the time variation n (t) based on the equation 4 shown above, and based on the calculated time variation n (t), the spatial fluorescence intensity PMTin (x, y, z shown in the equation 3 ) Is corrected. When the corrected spatial fluorescence intensity is PMTin ′ (x, y, z), the spatial fluorescence intensity PMTin ′ (x, y, z) is expressed by Equation 5 shown below.

In Equation 5 shown above, (h * νi) / PDin (x, y, z) corresponds to the correction data. In other words, the intensity correction distribution determining unit 623 calculates the correction data based on the intensity PDin (x, y, z) for each coordinate (x, y, z) indicated by the acquired intensity distribution of idler light.

<< 1.7.2. Flow of operations related to correction >>
Next, the flow of operations related to generation of correction data based on idler intensity distribution and correction of fluorescence intensity distribution based on the correction data will be described with reference to FIG. FIG. 21 is a flowchart for explaining the flow of processing relating to correction of the intensity distribution. In this description, the warm-up is completed and the processing at the time of observing the sample S will be described. As the pre-stage of the observation of the sample S, the intensity distribution of idler light is acquired in advance during the warm-up period. I will explain it as something to keep.

(Step S101)
The two-dimensionalization processing unit 621 of the distribution information generation unit 62 sequentially acquires data indicating the intensity of the idler light detected (measured) by the PD 54 from the PD 54 at a preset sampling rate. Further, the two-dimensionalization processing unit 621 sequentially acquires control information indicating the control content of the galvanometer mirror 51 from the system control unit 61. The two-dimensionalization processing unit 621 arranges data indicating the idler light intensity sequentially acquired from the PD 54 based on the control information acquired from the system control unit 61 and converts the data into a two-dimensional data, thereby calculating the detected idler light intensity distribution. Generate. At this time, the two-dimensionalization processing unit 621 may discard the unnecessary portion of the idler light intensity distribution and use only the effective portion as the idler light intensity distribution. The two-dimensional processing unit 621 outputs the generated idler light intensity distribution to the intensity correction distribution determining unit 623 and the intensity control amount determining unit 611.

Also, the two-dimensionalization processing unit 625 of the distribution information generation unit 62 sequentially acquires data indicating the intensity of fluorescence detected (measured) by the PMT 53 from the PMT 53 at a preset sampling rate. Further, the two-dimensional processing unit 625 sequentially acquires control information indicating the control contents of the galvanometer mirror 51 from the system control unit 61. The two-dimensionalization processing unit 625 generates the detected fluorescence intensity distribution by arranging the data indicating the fluorescence intensity sequentially acquired from the PMT 53 and making it two-dimensional based on the control information acquired from the system control unit 61. . At this time, the two-dimensionalization processing unit 625 may discard the unnecessary portion of the fluorescence intensity distribution and use only the effective portion as the fluorescence intensity distribution. The two-dimensionalization processing unit 625 outputs the generated fluorescence intensity distribution to the correction processing unit 63 and the RAW image generation unit 65.

(Step S103)
The distribution information generation unit 62 may perform noise removal processing on the intensity distribution of idler light and the intensity distribution of fluorescence generated by the two-dimensionalization processing unit 621 and the two-dimensionalization processing unit 625, respectively. . A mode in which noise removal processing is performed on each of the intensity distribution of idler light and the intensity distribution of fluorescence will be separately described later as a “fourth embodiment”.

(Step S105)
The intensity correction distribution determination unit 623 acquires the maximum value and the average value of the idler light intensity based on the previously acquired intensity distribution of the idler light during the warm-up period, and calculates the maximum correction amount based on the maximum value / average value. Calculate large. The intensity correction distribution determining unit 623 notifies the intensity control amount determining unit 611 of the calculated maximum correction amount width. Thereby, the intensity control amount determination unit 611 estimates the time variation of the number of photons due to the variation of the intensity of the laser emitted from the light source 2, and sets the maximum width (margin) of the control amount of the intensity of the laser beam (pump light). It becomes possible to decide. The details of the laser beam (pump light) intensity control will be described separately later in “1.8. Details of laser beam intensity control”.

(Step S107)
The intensity correction distribution determination unit 623 generates correction data for correcting the fluorescence intensity distribution based on the intensity distribution of idler light acquired from the two-dimensionalization processing unit 621 when the sample S is observed. Specifically, the intensity correction distribution determination unit 623 has (h * νi) / when the intensity of the idler light at the coordinates (x, y, z) on the sample S is PDin (x, y, z). Correction data is generated based on PDin (x, y, z). Here, h represents the Planck constant, and νi represents the frequency of idler light. The intensity correction distribution determination unit 623 outputs the idler light intensity distribution to the RAW image generation unit 65, and outputs correction data generated based on the idler light intensity distribution to the correction processing unit 63.

(Step S109)
The RAW image generation unit 65 generates a RAW file by shaping the fluorescence intensity distribution acquired from the two-dimensionalization processing unit 625 as image data (RAW image) into a predetermined file format. At this time, the RAW image generation unit 65 may generate a RAW file using a series of fluorescence intensity distributions acquired in the Z direction (depth direction) as image data.

(Step S111)
The RAW image generation unit 65 associates the intensity distribution of idler light acquired from the intensity correction distribution determination unit 623 with the generated RAW file. The RAW image generation unit 65, when a series of fluorescence intensity distributions acquired in the Z direction (depth direction) is recorded as image data in the RAW file, The idler intensity distributions corresponding to the intensity distributions are associated together.

As described above, by associating the intensity distribution of idler light with the RAW file, the external apparatus (for example, the information processing apparatus 800) that has read the RAW file D1 changes the laser intensity that is manifested in the fluorescence intensity distribution. Can be corrected.

(Step S113)
The correction processing unit 63 acquires the fluorescence intensity distribution from the two-dimensional processing unit 625. Further, the correction processing unit 63 acquires correction data generated based on the intensity distribution of idler light from the intensity correction distribution determining unit 623. The correction processing unit 63 corrects the fluorescence intensity distribution with the correction data based on the above-described Expression 5.

With such a configuration, the image acquisition device 1 according to the present embodiment corrects the noise that is manifested in the fluorescence intensity distribution based on the change in the intensity of the signal light using the correction data generated based on the intensity distribution of the idler light. It becomes possible.

[1.8. Details of laser light intensity control]
<< 1.8.1. Principle of strength control >>
Next, details of processing related to control of the intensity of laser light (pump light) output from the light source 2 by the control unit 6 according to the present embodiment will be described with reference to FIGS. 22 and 23.

First, the principle of laser beam intensity control in the image acquisition apparatus 1 according to the present embodiment will be described with reference to FIG. FIG. 22 is an explanatory diagram for explaining the principle of intensity control of emitted light from the light source 2. As shown in FIG. 22, among the OPO laser modules constituting the light source 2, the frequency of pump light output from the MOPA light source (fixed wavelength) is νp, the frequency of signal light is νs, and the frequency of idler light is νi. .

For example, in the case where the light source 2w is provided outside the image acquisition device 1w as in the image acquisition device 1w according to the comparative example described above, it is often difficult to perform fine control on the light source 2w. . In such a case, for example, the spatial fluorescence intensity PMTin (x, y, z) is expressed by the following equation in anticipation of the time fluctuation n (t) of the number of photons accompanying the fluctuation of the intensity of the output laser beam. The maximum spatial fluorescence intensity PMTin_max (x, y, z) represented by 6 is designed.

Note that, in Equation 6 shown above, n_max (t) represents the maximum value of the time variation n (t) of the number of photons, and Q_max (x, y, z) represents the observation target (ie, the sample S ) Shows the maximum value of the spatial fluorescence concentration distribution.

That is, when it is difficult to perform fine control on the light source 2w as in the image acquisition device 1w according to the comparative example, the design is performed in anticipation of the maximum value of the time variation n (t). It was difficult to use the entire dynamic range for the density distribution Q (x, y, z).

On the other hand, the control unit 6 according to the present embodiment monitors the fluctuation of the laser intensity based on the intensity distribution of the idler light detected by the PD 54, and controls the laser intensity following the fluctuation of the laser intensity. Do.

Specifically, when the change in laser intensity, which is sufficiently large with respect to the imaging time, is nl (t) and the change within the imaging time is nh (t), the control unit 6 , N (t) = nl (t) + nh (t). In addition, the control unit 6 applies intensity control with a sufficiently short latency tl to the imaging time by applying negative feedback proportional to the intensity of idler light to nl (t). The spatial fluorescence intensity PMTin (x, y, z) at this time is expressed by the following formula 7.

In Expression 7 shown above, when tl is sufficiently short with respect to fluctuation of nl, Expression 7 can be approximated to Expression 8 shown below.

Further, when the correction processing described above is used in combination, in the above equation 8, since it can be regarded as nh << nl, the above equation 8 can be approximated to the following equation 9.

That is, since the spatial fluorescence intensity PMTin (x, y, z) is expressed by the following formula 10, it is necessary to consider the time fluctuation n (t) of the number of photons accompanying the fluctuation of the laser light intensity. It becomes possible to use almost the entire dynamic range.

<< 1.8.2. Flow of operations related to intensity control >>
Next, with reference to FIG. 23, the flow of operations related to the control of the intensity of laser light (pump light) output from the light source 2 will be described. FIG. 23 is a flowchart for explaining a flow of processing relating to intensity control of emitted light from the light source.

(Step S201)
First, individual differences of laser light sources (for example, the semiconductor laser 222 of the mode-locked laser unit 220) used for the light source 2 before the start of measurement are in a reference state determined in advance based on the control parameters of the laser light source. Adjust it. This adjustment is an adjustment based on individual differences of laser light sources, and is different from the adjustment of laser intensity due to thermal fluctuation.

Next, the intensity control amount determination unit 611 of the system control unit 61 receives from the intensity correction distribution determination unit 623 of the distribution information generation unit 62 the maximum width of the correction amount based on the intensity distribution of idler light during the warm-up period (that is, warm The maximum value / average value of the intensity of idler light during the up period is acquired. The intensity control amount determination unit 611 determines the maximum value n_max of the intensity control target of the laser light (pump light) based on the acquired maximum width of the correction amount. As a specific example, when the maximum width of the acquired correction amount is 2, the intensity control amount determination unit 611 reciprocates the maximum width of the correction amount in the dynamic range of the intensity of the laser light (pump light). That is, the maximum value n_max of the intensity control target is determined so as to use a range of 1/2.

(Step S203)
Based on the determined maximum value n_max of the intensity control target, the intensity control amount determination unit 611 calculates the control target maximum value PDin_max of the idler light intensity based on Expression 11 shown below. In Expression 11 shown below, h represents the Planck constant, and νi represents the frequency of idler light.

(Step S205)
In addition, the intensity control amount determination unit 611 is the intensity distribution for one shooting acquired most recently (for example, the immediately preceding shooting) from the intensity distribution of idler light from the two-dimensional processing unit 621 of the distribution information generation unit 62. Based on the above, the average PDin_ave of the idler light intensity is calculated.

(Step S207)
Next, the intensity control amount determination unit 611 evaluates the stability of the laser beam by pattern matching over a plurality of images (for example, for two photographings) acquired most recently from the intensity distribution of the idler light, and sets the weighting coefficient W to 0 <W. It is determined within the range of ≦ 1. Note that W = 1 corresponds to the case where the laser beam is stable, and the laser beam corresponds to a more unstable state as the coefficient W decreases.

As a specific example, the intensity control amount determination unit 611 compares a plurality of idler light intensity distributions by pattern matching, digitizes the difference, and compares the difference with a predetermined threshold value, thereby stabilizing the stability of the laser beam. May be evaluated. In this case, when the calculated difference is equal to or smaller than the threshold value, the intensity control amount determination unit 611 determines that the laser beam is stable and sets W = 1. Further, when the calculated difference exceeds the threshold value, the intensity control amount determination unit 611 determines that the laser beam is in an unstable state, and sets the coefficient W to a predetermined value in a range of 0 <W <1. Set.

As another example, the intensity control amount determination unit 611 may dynamically determine the coefficient W in the range of 0 <W ≦ 1 based on the difference calculated according to the comparison between the intensity distributions of idler light. Good. Specifically, the intensity control amount determining unit 611 sets W to a value closer to 0 when the calculated difference is large (that is, unstable), and when the calculated difference is small (that is, If it is relatively stable, W may be set to a value closer to 1.

(Step S209)
When the coefficient W is determined, the intensity control amount determination unit 611 determines the intensity after control of the laser light emitted from the light source 2 as follows: intensity after control = intensity before control * (PDin_max + 1) / (PDin_ave + 1) * W Calculate based on the formula.

(Step S211)
The system control unit 61 controls the light source 2 so that the intensity of the laser light (pump light) emitted from the light source 2 is controlled based on the intensity after control calculated by the intensity control amount determination unit 611. At this time, it is desirable that the system control unit 61 performs feedback control so that the intensity of the laser beam is controlled within one imaging unit (that is, within imaging for one fluorescence intensity distribution).

(Step S213)
When the system control unit 61 controls the intensity of the laser beam, the intensity control amount determination unit 611 determines whether the laser beam is emitted based on the intensity distribution of idler light emitted from the light source 2 after the intensity of the laser beam (pump light) is controlled. Determine if it is stable.

As described above, the system control unit 61 sequentially determines whether or not the laser light is stable, determines the weighting coefficient W based on the determination result, and determines the laser light (pump light) emitted from the light source 2. ) Execute intensity control.

The system control unit 61 may continuously execute the above-described series of operations during the period in which the light source 2 emits laser light, or periodically (intermittently) at predetermined timings. May be.

When the series of operations described above are periodically executed at predetermined timings, the system control unit 61 may determine whether or not to stop the series of operations in step S213. Specifically, the system control unit 61 repeatedly executes a series of operations according to steps S207 to S211 (that is, monitoring of laser light and feedback processing based on the monitoring result) until the laser light is stabilized (step S213, No). And if a laser beam is stabilized (step S213, Yes), the system control part 61 should just stop a series of operation | movement which concerns on intensity | strength control of a laser beam (pump light).

With such a configuration, the image acquisition device 1 according to the present embodiment is configured so that the light source 2 can be operated even when the operation of the light source 2 becomes unstable due to heat generated by a built-in device (for example, the measurement unit 3 or the control unit 6). By controlling the intensity of the laser light emitted from 2, the light source 2 can be stably operated.

[1.9. Laser intensity control during warm-up]
Next, laser light intensity control during form-up will be described. As described above, the intensity of the laser beam output from the laser module often depends on the temperature. Therefore, when a laser module is used as the light source 2, in many cases, the laser module is used after warming up in order to stabilize the intensity of the output laser light.

On the other hand, the warm-up of the laser module often takes a long time, and this warm-up time may impose a work time related to the observation of the sample S. Therefore, in the image acquisition device 1 according to the present embodiment, by controlling the intensity of the laser beam, the laser beam can be stabilized even in a state where the intensity of the laser beam is unstable, such as during warm-up. The sample S can be observed.

Hereinafter, an example of laser beam intensity control during warm-up of the image acquisition apparatus 1 according to the present embodiment will be described with reference to FIG. FIG. 24 is an explanatory diagram for explaining the principle of intensity control of emitted light from the light source 2 during warm-up. In the example shown in FIG. 24, feedback control during warm-up by the control unit 6 according to the present embodiment is converted into a time response by Laplace conversion.

In the case of the configuration described above with reference to FIG. 22, so-called feedback control based on the proportional component K1 and integral component K2 / s, that is, PI control is performed. However, when the intensity of the laser beam is unstable as in the warm-up period, the feedback control based only on the proportional component K1 and the integral component K2 / s (that is, PI control) may lack responsiveness. .

In particular, during the warm-up period, in the conditional expression of the spatial fluorescence intensity PMTin (x, y, z) shown as Expression 7 described above, with respect to a gradual change nl (t) that is sufficiently large with respect to the imaging time. The latency tl cannot be regarded as sufficiently short. Therefore, for example, a change with periodicity may be manifested in the change in the intensity of the laser light emitted from the light source 2.

Therefore, the image acquisition apparatus 1 according to the present embodiment performs PID control by adding control based on the differential component K3s to the feedback control, and improves responsiveness.

Specifically, the control unit 6 of the image acquisition device 1 calculates and accumulates the transfer function G (s) of the wavelength conversion module (OPO) 250 during the warm-up period, thereby storing the accumulated transfer function G (s ) To calculate the differential component K3s. For example, the control unit 6 calculates an input to the wavelength conversion module (OPO) 250 based on control information for controlling the intensity of the laser light emitted from the light source 2, and calculates the intensity of the detected idler light. The transfer function G (s) may be calculated as the output. Then, the image acquisition device 1 controls the intensity of the laser light emitted from the light source 2 based on the calculated differential component K3s, thereby suppressing the manifestation of the change with periodicity.

As a specific example, the image acquisition apparatus 1 predicts the periodicity of the change in the intensity of the laser light based on the accumulated transfer function G (s) of the OPO, and applies feedback control that is opposite to the predicted period. Thus, a change with periodicity may be suppressed.

With such a configuration, the image acquisition apparatus 1 according to the present embodiment emits light from the light source 2 even when the intensity of the laser light emitted from the light source 2 is unstable as in the warm-up period. It is possible to improve the responsiveness related to the intensity control of the laser beam. Therefore, according to the image acquisition device 1, for example, it is possible to start observation of the sample S even in a state where the warm-up period is not completed (a state where the intensity of the emitted laser light is unstable). Become.

If the observation is started before the completion of the warm-up period, the control amount based on the differential component K3s is larger than the change amount of the periodic change of the emitted laser light, and the influence of the control by the differential component K3s. May remain as noise. Therefore, when the image acquisition device 1 starts observation before the completion of the warm-up period, the fluorescence intensity distribution and the intensity of idler light are used using a known sample such as a fluorescent bead before the start of the observation. You may acquire distribution. In this case, the image acquisition apparatus 1 warms up the fluorescence intensity distribution and the idler intensity distribution acquired in advance to the RAW file created based on the observation result of the sample S into the RAW file. You may associate as a picked-up image of time (refer FIG. 19).

With such a configuration, for example, the external device (for example, the information processing device 800) that has acquired the RAW file estimates a change in the intensity of the laser light based on a captured image associated with the RAW file during warm-up. It becomes possible. Therefore, even when the influence of the control by the differential component K3s remains as noise in the observation result (that is, the fluorescence intensity distribution) in the RAW file, the external device estimates the change in the intensity of the laser light and Noise can be corrected.

As described above, according to the image acquisition device 1 according to the present embodiment, the sample S is observed even in a state where the warm-up period is not completed (a state where the intensity of the emitted laser light is unstable). Can be started. Therefore, according to the image acquisition apparatus 1 according to the present embodiment, the period during which observation is impossible is shortened due to warm-up, and as a result, the observation time of the sample S can be shortened.

[1.10. Operation of information processing apparatus]
Next, an example of processing in which the information processing apparatus 800 generates an image of the sample S based on the RAW file D1 acquired from the image acquisition apparatus 1, performs image processing on the generated image, and presents the image to the user. This will be described with reference to FIGS. 25 and 26. FIG. FIG. 25 is a flowchart showing a flow of a series of processes related to image display of the information processing apparatus 800 according to the present embodiment. FIG. 26 is a flowchart illustrating an example of processing related to noise correction of the information processing apparatus 800 according to the present embodiment. First, referring to FIG.

(Step S301)
First, the information processing apparatus 800 uses a U / I (hereinafter referred to as “U / I”) to specify the RAW file D1 that is the target of image correction and display, the image processing content for the specified RAW file D1, and the parameters of the processing. , Described as “image adjustment U / I”), for example, to the user via a display device.

(Step S303)
In addition, the information processing apparatus 800 displays information specified by the user via the presented image adjustment U / I, for example, the RAW file D1 to be processed, the details of the image processing, and the processing parameters, based on the user operation details (for example, , And the operation content using the operation device).

(Step S305)
The information processing apparatus 800 acquires a RAW file D1 to be processed based on the user's specification. The RAW file D1 may be acquired in advance from the image acquisition apparatus 1 and stored in a storage device or the like, or communication with the image acquisition apparatus 1 is established and acquired from the image acquisition apparatus 1. May be.

(Step S307)
The information processing apparatus 800 analyzes the acquired RAW file D1 based on the file format of the RAW file D1, and extracts each piece of information recorded in the RAW file D1. Thereby, the information processing apparatus 800 extracts, for example, the RAW main image S309 recorded in the RAW file D1, and the shooting information S311 and S321 indicating the conditions at the time of shooting the RAW main image S309. The photographing information S311 and S321 extracted in this way includes, for example, the above-described intensity distribution of idler light. Needless to say, if the target RAW file D1 has already been analyzed, the information processing apparatus 800 does not need to execute the process relating to the analysis of the RAW file D1 again.

The information processing apparatus 800 generates an image of the sample S based on the RAW main image S309 extracted from the RAW file D1 and the shooting information S311 and S321, performs image processing on the generated image, and presents it to the user. Examples of image processing include processing related to various noise corrections. Therefore, hereinafter, the processing relating to the correction of the noise accompanying the fluctuation of the intensity of the laser beam will be described as steps S313 to S315, and the processing different from the noise accompanying the fluctuation of the intensity of the laser beam will be described as steps S323 to S325. A process related to noise correction will be described.

(Step S313)
First, the contents of processing related to correction of noise accompanying fluctuations in the intensity of laser light will be described. In this case, the information processing apparatus 800 first cuts out an image of a region to be processed from the extracted RAW main image S309, and sets a partial image corresponding to the cut out region as a processing target.

Note that the area to be processed may be acquired by the information processing apparatus 800 itself based on user input, for example.

As another example, when the image acquisition device 1 captures the RAW main image S309, the image acquisition apparatus 1 receives the designation of the region to be observed as a user input, and sends control information indicating the region to the RAW file D1 in the photographic information S311. May be recorded as In this case, the information processing apparatus 800 may recognize an area to be processed based on the control information recorded as the shooting information S311 from the RAW file D1.

Of course, it goes without saying that the RAW main image S309 may be processed without performing the cut-out process. In the following processing, the cut-out partial image and the RAW main image S309 itself are not particularly distinguished from each other, and are simply described as “RAW main image S309”.

(Step S315)
The information processing apparatus 800 corrects the noise associated with the fluctuation in the intensity of the laser light that appears on the RAW main image S309 based on the intensity distribution of the idler light extracted as the imaging information S311. The content of this correction process is the same as the content of the process in which the image acquisition device 1 described above calculates correction data based on the intensity distribution of idler light and corrects the fluorescence intensity distribution based on the correction data.

Note that the information processing apparatus 800 may be configured to be able to adjust the content of the correction process based on the parameter of the correction process specified as the user input. As a specific example, the information processing apparatus 800 may be configured to adjust the application amount of correction processing based on the intensity distribution of idler light for the RAW main image S309 based on user input.

As described above, the information processing apparatus 800 generates the adjusted image S317 by correcting the noise associated with the fluctuation of the intensity of the laser light that has been manifested on the RAW main image S309.

(Step S323)
Next, a case where noise correction based on an ε filter is performed will be described as an example of processing related to correction of other noise that is different from noise due to fluctuations in laser light intensity. First, the information processing apparatus 800 cuts out an image of a region to be processed from the extracted RAW main image S309, and sets a partial image corresponding to the cut-out region as a processing target. This process is the same as the process according to step S313 described above.

(Step S323)
Next, the information processing apparatus 800 corrects the RAW main image S309 based on the extracted shooting information S321.

Here, the contents of the processing relating to the correction will be described with reference to FIG. FIG. 26 is a flowchart illustrating an example of processing related to noise correction of the information processing apparatus 800 according to the present embodiment.

(Step S401)
First, the information processing apparatus 800 specifies a pixel serving as a reference for processing from the RAW main image S309, and calculates the level (pixel value) of the pixel as a reference level.

(Step S403)
Next, the information processing apparatus 800 calculates a threshold for weighting for each pixel to be processed. At this time, the information processing apparatus 800 compares pixels whose idler intensity is extremely fluctuating as compared with other pixels due to the intensity distribution of the idler light extracted as the shooting information S311 compared to other pixels. Set the threshold value high.

(Step S405)
When the threshold value is calculated for each pixel to be processed, the information processing apparatus 800 weights the level for each pixel to be processed based on the calculated reference level and the threshold value calculated for each pixel to be processed.

(Step S407)
Further, the information processing apparatus 800 calculates the distance from the reference pixel for each pixel to be processed.

(Step S409)
The information processing apparatus 800 weights the distance for each pixel to be processed based on the distance calculated for each pixel to be processed.

(Step S411)
The information processing apparatus 800 performs an averaging process by applying an ε filter based on the level weighting and the distance weighting calculated for each pixel to be processed.

(Step S413)
As described above, the information processing apparatus 800 performs the above-described series of operations until the noise correction process is performed on a series of pixels as a processing target (NO in step S413). Then, the information processing apparatus 800 generates an adjusted image S327 by performing noise correction processing on a series of pixels (step S413, YES).

(Step S331)
Here, FIG. 25 will be referred to again. When the information processing apparatus 800 generates at least one of the adjusted images S317 and S327, the information processing apparatus 800 presents the generated adjusted image in a predetermined area on the image adjustment U / I and displays the image adjustment U / I. Update. Thereby, the user can check the generated adjusted image via the image adjustment U / I. At this time, the information processing apparatus 800 may store the generated adjusted image in a predetermined storage unit (for example, a storage such as a hard disk).

(Step S333)
The information processing apparatus 800 continues the above-described series of processing relating to image adjustment and presentation of the adjusted image until the user instructs the end of the image adjustment (NO in step S333). At this time, the information processing apparatus 800 may be configured to further execute processing related to image adjustment on the generated adjusted image based on an instruction from the user.

When the user instructs the end of the image adjustment (step S333, YES), the information processing apparatus 800 ends the above-described series of processes relating to the adjustment of the image and the presentation of the image after the adjustment.

Heretofore, an example of processing related to noise correction of the information processing apparatus 800 according to the present embodiment has been described with reference to FIGS. As described above, by associating the RAW file D1 with related information such as the intensity distribution of idler light, in the external apparatus such as the information processing apparatus 800, similarly to the image acquisition apparatus 1, a laser is obtained. It becomes possible to correct noise based on intensity fluctuations.

[1.11. Summary]
As described above, when the signal light is excitation light, the image acquisition device 1 according to the present embodiment corrects the fluorescence intensity distribution based on the excitation light based on the idler light intensity distribution. With such a configuration, the image acquisition device 1 according to the present embodiment corrects noise associated with fluctuations in the intensity of the laser light, for example, even under circumstances where the intensity of the laser light varies, and provides a clear image. Can be obtained.

Further, the image acquisition device 1 according to the present embodiment monitors the intensity of the excitation light emitted from the light source 2 based on the intensity distribution of idler light, and the laser light (pump light) emitted from the light source 2 based on the monitoring result. ) Control the intensity. With such a configuration, the image acquisition apparatus 1 according to the present embodiment can be controlled so that the intensity of the laser beam is stabilized even under a situation where the intensity of the laser beam fluctuates, for example. .

Due to the above-described features, the image acquisition device 1 according to the present embodiment can incorporate the light source 2 in the same housing, so that the image acquisition device 1 itself can be downsized.

Further, in the case of a configuration in which a light source is provided outside as in the image acquisition device 1w according to the comparative example, a laser module used as the light source is provided by, for example, an OEM (Original equipment manufacturer) or the like, and its internal structure is There are many cases where detailed control is difficult due to the black box. For this reason, the image acquisition device 1w according to the comparative example is often used at a low output in order to stably operate the laser module, and the performance of the laser module may not be fully utilized.

In contrast, as described above, the image acquisition device 1 according to the present embodiment controls the intensity of the laser beam and corrects the noise accompanying the fluctuation of the intensity of the laser beam. For this reason, the image acquisition apparatus 1 according to the present embodiment can obtain a clear image even when the output of the laser module is increased and can be used by taking advantage of the performance of the laser module.

<2. Second Embodiment>
[2.1. Overview of image acquisition device]
Next, an image acquisition apparatus according to the second embodiment will be described. First, the problems of the image acquisition apparatus according to the present embodiment will be organized.

Image acquisition apparatuses that require a light source such as a fluorescence microscope often use a fixed-wavelength laser light source that can project an excitation wavelength that is predetermined according to the fluorescent dye used.

However, the fluorescent substrate in the sample may fade over time. In such a case, since the excitation wavelength changes, it is necessary to change the wavelength of the excitation light output from the laser light source. When responding to such a situation by a configuration using a fixed-wavelength laser light source, it is necessary to provide a plurality of types of laser light sources. The device itself tends to be expensive.

In addition, since the excitation wavelength of the fluorescent substrate in the sample changes according to the degree of fading, in a configuration in which a plurality of fixed-wavelength laser light sources are provided, the user can appropriately switch the laser light source to be used to excite the fluorescent substrate. It is necessary to specify the light source, which may reduce convenience. Further, in the case of a configuration in which a plurality of types of laser light sources are switched and used, even if fluorescence can be excited from a sample that has faded, an image with high contrast is not necessarily obtained.

In view of this, the present disclosure proposes a new and improved image acquisition apparatus and image acquisition method capable of obtaining a high-contrast image by a simpler method even in a situation where the excitation wavelength of the sample changes. .

Specifically, the image acquisition apparatus according to the present embodiment uses a laser module capable of changing the wavelength of the output laser light (excitation light) as a light source, and the laser is based on the observed fluorescence intensity distribution. Control the wavelength of light. Thereby, the image acquisition apparatus according to the present embodiment does not require a complicated operation such as an observer (user) controlling the operation of the light source 2 while checking the observation result or switching the light source 2 itself. It is possible to obtain an image with high contrast.

Hereinafter, regarding the image acquisition apparatus according to the present embodiment, the configuration of the optical system will be described first, and then the functional configuration of the image acquisition apparatus will be described. In order to distinguish the image acquisition apparatus according to the present embodiment from the image acquisition apparatus 1 according to the first embodiment, hereinafter, the image acquisition apparatus may be referred to as an “image acquisition apparatus 1a”.

[2.2. Configuration of image acquisition apparatus]
<< 2.2.1. Configuration of optical system >>
First, with reference to FIG. 27, regarding the configuration of the optical system of the image acquisition device 1a according to the present embodiment, in particular, the configuration of the optical system of the image acquisition device 1 according to the first embodiment described above (see FIG. 14). The explanation will focus on the differences. In the present description, as in the first embodiment described above, the description will be made assuming that signal light is emitted toward the sample S as excitation light.

The image acquisition device 1a according to the present embodiment controls the wavelength of laser light (pump light) emitted from the light source 2 based on the intensity of fluorescence (coloring light) detected by the PMT 53. Therefore, it is not always necessary to provide the PD 54 for detecting the intensity of idler light as in the image acquisition device 1 according to the first embodiment described above.

In addition, about the structure which irradiates excitation light toward the sample S, the optical system (refer FIG. 10) of the image acquisition apparatus 1w which concerns on the comparative example mentioned above, or the optical system of the image acquisition apparatus 1 which concerns on 1st Embodiment (refer FIG. This is the same as in FIG. That is, the excitation light emitted from the light source 2 is guided to the objective lens 42 via the beam shaping lens 511, the galvano mirror 51, the lens 513, the mirror 517, the lens 515, and the dichroic mirror 52. It is condensed toward the sample S.

<< 2.2.2. Functional configuration of image acquisition device >>
Next, an example of a functional configuration of the image acquisition device 1a according to the present embodiment will be described with reference to FIG. As shown in FIG. 28, the image acquisition device 1a according to the present embodiment includes a light source 2, a measurement unit 3, a control unit 6a, and an I / F 7. The measurement unit 3 includes a microscope unit 4 and a scanning system (detection system) 5a. The configurations of the light source 2, the microscope unit 4, and the I / F 7 are the same as those of the image acquisition apparatus 1 according to the first embodiment described above. Therefore, in the following, description will be made with particular attention to the configuration of the scanning system (detection system) 5a and the control unit 6a, which is different from the image acquisition device 1 according to the first embodiment. In the example shown in FIG. 28, each configuration of the scanning system (detection system) 5a corresponds to the configuration given the same reference numeral in the optical system shown in FIG. In the example shown in FIG. 28, a part of the configuration shown in FIG. 27 is omitted.

As shown in FIG. 28, the scanning system (detection system) 5a according to the present embodiment does not necessarily include the PD 54 for measuring idler light, and thus the scanning system according to the first embodiment described above. Different from (detection system) 5 (see FIG. 15). Other configurations are the same as those of the scanning system (detection system) 5 according to the first embodiment described above.

That is, the excitation light emitted from the light source 2 is applied to the sample S via the scanning system (detection system) 5 a and the microscope unit 4. The fluorescence generated from the sample S by the irradiated excitation light is guided to the scanning system (detection system) 5a through the microscope unit 4 and detected by the PMT 53 of the scanning system (detection system) 5a. The PMT 53 converts the detected fluorescence into an electrical signal by a photoelectric effect at a preset sampling rate, and outputs it to the control unit 6a as data indicating the intensity of the fluorescence.

The control unit 6a includes a system control unit 61a, a distribution information generation unit 62a, a correction processing unit 63, an image processing unit 64, a RAW image generation unit 65, a storage unit 66, a display control unit 67, and communication control. Part 68. The control unit 6a according to the present embodiment is particularly different from the control unit 6 according to the first embodiment described above (see FIGS. 15 and 16) in the configuration of the system control unit 61a and the distribution information generation unit 62a. .

Here, the detailed configuration of the system control unit 61a and the distribution information generation unit 62a will be described with reference to FIG. As illustrated in FIG. 29, the distribution information generation unit 62a includes a two-dimensionalization processing unit 625. Further, the system control unit 61a includes a wavelength control amount determination unit 613.

The two-dimensionalization processing unit 625 sequentially acquires data indicating the intensity of fluorescence detected (measured) by the PMT 53 from the PMT 53 at a preset sampling rate. Further, the two-dimensional processing unit 625 sequentially acquires control information indicating the control contents of the galvanometer mirror 51 from the system control unit 61. Then, the two-dimensionalization processing unit 625 arranges the data indicating the fluorescence intensity sequentially acquired from the PMT 53 based on the control information acquired from the system control unit 61 and converts the data into a two-dimensional data, thereby obtaining the detected fluorescence intensity distribution. Generate.

The two-dimensionalization processing unit 625 outputs the generated fluorescence intensity distribution to the wavelength control amount determination unit 613, the correction processing unit 63, and the RAW image generation unit 65.

The fluorescence intensity distribution output from the two-dimensional processing unit 625 is subjected to correction processing such as noise removal by the correction processing unit 63 and output to the image processing unit 64. The image processing unit 64 generates image data by performing image processing such as compression processing on the intensity distribution subjected to the correction processing. For example, the generated image data is output to the display control unit 67 and displayed on the display unit 72 by the display control unit 67. As a result, the user can check the image of the sample S via the display unit 72.

The wavelength control amount determination unit 613 acquires the generated fluorescence intensity distribution from the two-dimensional processing unit 625, and the laser light (excitation) emitted from the light source 2 so as to improve the contrast of the fluorescence intensity distribution. The control amount of the wavelength (frequency) of light is determined. When the wavelength control amount determination unit 613 determines the laser light wavelength control amount, the system control unit 61 controls the wavelength of the laser light emitted from the light source 2 based on the control amount.

When the wavelength of the laser light emitted from the light source 2 is controlled, the two-dimensionalization processing unit 625 acquires the fluorescence intensity distribution based on the laser light after the control, and the acquired fluorescence intensity distribution is wavelength-controlled. The data is output to the quantity determining unit 613. The wavelength control amount determination unit 613 evaluates the newly acquired fluorescence intensity distribution, and calculates the control amount of the wavelength of the laser light so as to improve the contrast of the fluorescence intensity distribution.

By repeating the above operations, the wavelength of the laser light emitted from the light source 2 is controlled by the system control unit 61 and the wavelength control amount determination unit 613 so that, for example, the contrast of the fluorescence intensity distribution is maximized. .

The wavelength control amount determination unit 613 may determine the control amount of the wavelength of the laser light by evaluating the contrast for the region designated as the observation target by the user in the acquired fluorescence intensity distribution. .

In this case, the display control unit 67 may display U / I for designating a region in the image data on the display unit 72 together with the image data created based on the fluorescence intensity distribution. Thereby, the user can designate the region to be observed for the image displayed on the display unit 72 via the condition designating unit 71.

For example, FIG. 30 is a diagram for explaining an example of an observation target designation method. In FIG. 30, a reference sign V20 indicates an image displayed on the display unit 72, and a reference sign V21 schematically indicates a region designated as an observation target by the user. Note that the shape of the region V21 is not necessarily limited to the circular shape shown in FIG. 30 as long as at least a part of the region in the image V20 can be designated. For example, the shape of the region V21 may be a rectangular shape or an arbitrary shape designated by the user.

The wavelength control amount determination unit 613 receives, from the condition designation unit 71, information indicating the region V21 in the image designated by the user via the condition designation unit 71 with respect to the image V20 of the sample S displayed on the display unit 72. get. The coordinate system of the image V20 corresponds to the coordinate system of the fluorescence intensity distribution. Therefore, the wavelength control amount determination unit 613 recognizes a region on the fluorescence intensity distribution corresponding to the region V21 designated by the user for the image V20 based on the information indicating the region V21 acquired from the condition designating unit 71. .

When the information indicating the region V21 designated as the observation target by the user is acquired, the wavelength control amount determination unit 613 improves the contrast so that the region V21 on the fluorescence intensity distribution is the target. What is necessary is just to determine the control amount of the wavelength of a laser beam.

Further, the system control unit 61 may be configured to control (adjust) the wavelength of the laser light based on designation from the user. As a specific example, the system control unit 61 causes the user to specify whether or not further adjustment is necessary for an image created based on the fluorescence intensity distribution after wavelength control, and based on the designation, It may be possible to determine whether or not to further control the wavelength of light.

In this case, for example, the display control unit 67 presents image data created based on the fluorescence intensity distribution after wavelength control to the user via the display unit 72, and the system control unit 61 further adjusts. A user designation indicating whether it is necessary is acquired from the condition designating unit 71.

The system control unit 61 may cause the wavelength control amount determination unit 613 to calculate the control amount of the wavelength of the laser light again when receiving a designation from the user that further adjustment is necessary. At this time, the system control unit 61 controls the wavelength control amount determination unit 613 so as to control conditions (for example, parameters such as a threshold value for determining contrast) related to the calculation of the control amount of the wavelength of the laser light. May be.

As another example, the system controller 61 may be operated so that the user can directly or indirectly specify the wavelength of the laser beam. As a specific example, the system control unit 61 receives a specification related to contrast adjustment (for example, increases / decreases the contrast or specifies an adjustment amount) from the user, and determines the wavelength of the laser light based on the specified contrast adjustment content. May be controlled.

As described above, the system control unit 61 can automatically control the wavelength of the laser light so as to improve the contrast of the fluorescence intensity distribution, and also controls the wavelength of the laser light based on an instruction from the user. It is also possible to do. With such a configuration, for example, the system control unit 61 automatically controls the wavelength of the laser beam so that the contrast of the fluorescence intensity distribution is maximized, and then sets the wavelength of the laser beam based on an instruction from the user. It is also possible to operate so as to make fine adjustments.

The detailed contents of the determination of the control amount of the wavelength of the laser light and the wavelength control of the laser light based on the control amount will be separately described later as “2.5. Details of wavelength control”.

The RAW image generation unit 65 acquires the fluorescence intensity distribution from the two-dimensionalization processing unit 625 and generates a RAW file by shaping the fluorescence intensity distribution as image data (RAW image) into a predetermined file format. May be. At this time, the RAW image generation unit 65 displays information related to the fluorescence intensity distribution, such as a fluorescence intensity distribution acquisition condition (for example, a shooting condition or a scanning condition such as a parameter at the time of shooting). The acquired information may be associated with the generated RAW file as related information.

In particular, the RAW image generation unit 65 according to the present embodiment, for example, as shown in FIG. 30, information indicating the region V21 specified by the user in the image V20 or an image in the region V21 as RAW It may be associated with a file.

Further, the RAW image generation unit 65 may associate control information indicating the control amount of the wavelength (frequency) of the laser light (excitation light) determined by the wavelength control amount determination unit 613 with the RAW file as related information. In this way, by associating control information indicating the control amount of the wavelength of the laser light with the RAW file as related information, for example, the external device of the information processing apparatus 800 recognizes the degree of fading of the sample based on the control amount. Is possible.

Note that the RAW image generation unit 65 may separately associate the control information determined based on the automatic adjustment by the system control unit 61 and the control information changed based on the user's instruction with the RAW file as related information. In addition, the RAW image generation unit 65 associates an image based on the fluorescence intensity distribution acquired based on each condition (for example, an image subjected to image processing such as compression) with the RAW file together with the control information. Good. As described above, the RAW image generation unit 65 displays the control information indicating the control amount of the wavelength of the laser light and the image acquired based on the laser light controlled according to the control amount for each of the conditions over a plurality of conditions. May be associated with a RAW file.

The configuration of the image acquisition device 1a according to the present embodiment has been described above with reference to FIGS. Next, more detailed contents of each component of the image acquisition device 1a according to the present embodiment will be described.

[2.3. RAW file format]
First, an example of the file format of the RAW file D1a according to the present embodiment will be described with reference to FIG. FIG. 31 is a diagram showing an example of the file format of the RAW file D1a according to the present embodiment.

As shown in FIG. 31, the RAW file D1 according to the present embodiment includes, for example, a data area d10, a basic control information area d30, and an extension area d20a. The configurations of the data area d10 and the basic control information area d30 are the same as those of the RAW file D1 (see FIG. 19) according to the first embodiment described above, and detailed description thereof is omitted.

As shown in FIG. 31, the expansion area d20 includes a manufacturer note IFDd21. The manufacturer note IFDd21 is an IFD for storing information not specified in EXIF, such as a camera control mode, and also stores shooting information and control information unique to the image acquisition device 1.

The manufacturer note IFDd21 according to the present embodiment includes, for example, a shooting condition d211, an initial shot image d221, a target image d222 specified by the user, a variable range d223 specified by the user, a parameter d224 based on automatic adjustment, and an observation. Parameter d225 designated by the user and an image d226 after automatic adjustment. Note that the shooting condition d211 is the same as that of the RAW file D1 according to the first embodiment described above.

The initial captured image d221 is an image based on the fluorescence intensity distribution acquired before the system control unit 61 controls the wavelength based on the control amount determined by the wavelength control amount determination unit 613. In other words, the initial photographed image d221 shows an image based on the fluorescence intensity distribution acquired using laser light having a wavelength (that is, initial setting) determined in advance according to the sample S to be observed as excitation light. .

The target image d222 designated by the user indicates an image obtained by cutting out a portion corresponding to the area designated as the observation target by the user from the captured image. For example, in the case of the example shown in FIG. 31, the target image d222 designated by the user corresponds to an image obtained by cutting out the portion indicated by the region V21 from the image V20.

The variable range d223 designated by the user is information indicating an area designated as an observation target by the user in the captured image. For example, in the example shown in FIG. 31, the variable range d223 specified by the user corresponds to control information indicating the region V21. Note that the control information indicating the area can be expressed by, for example, coordinates or vectors in the captured image.

The parameter d224 based on automatic adjustment indicates a parameter determined by the system control unit 61 during the automatic adjustment, such as the wavelength of the laser light emitted from the light source 2 determined by automatic adjustment by the system control unit 61. As a specific example, parameter D224 based on automatic adjustment indicates each parameter (for example, the wavelength of laser light) when adjustment is performed so that the contrast of the captured image is maximized. The parameter D224 based on the automatic adjustment is an image based on the fluorescence intensity distribution acquired according to the control of the light source 2 based on the parameter determined by the automatic adjustment by the system control unit 61 (for example, image processing such as compression). May be included.

The parameter d225 designated by the observer indicates the parameter determined by the system control unit 61 based on the designation from the user. The parameter d225 designated by the observer is an image based on the fluorescence intensity distribution acquired in accordance with the control of the light source 2 based on the parameter determined by the user (for example, image processing such as compression has been performed) Image).

In this way, the relevant information indicating the control amount of the wavelength (frequency) of the laser beam (excitation light) (that is, the parameter d224 based on the automatic adjustment or the parameter d225 specified by the observer) is recorded in the RAW file D1a. Thus, for example, the external device of the information processing device 800 can recognize the degree of fading of the sample based on the related information. In addition, by recording an image based on the fluorescence intensity distribution acquired in accordance with the control of the light source 2 based on the parameter, the external device of the information processing apparatus 800 displays the image corresponding to each condition as the image. It is possible to present it to the user together with the acquisition conditions (that is, parameters).

The post-automatic adjustment image d226 shows an image based on the fluorescence intensity distribution acquired in accordance with the control of the light source 2 based on the finally determined parameters. The automatically adjusted image d226 may be an image cut out based on the variable range d223 designated by the user.

The file format of the RAW file D1a according to the present embodiment has been described above with reference to FIG. Note that the file format of the RAW file D1a described above is merely an example, and it is needless to say that not all information may be included.

[2.4. Flow of operation of image acquisition device]
Next, a flow of a series of operations of the image acquisition device 1a according to the present embodiment will be described with reference to FIG. FIG. 32 is a flowchart showing an example of the flow of a series of operations of the image acquisition device 1a according to the present embodiment.

(Step S501)
First, the control unit 6a of the image acquisition apparatus 1a controls the wavelength of the excitation light emitted from the light source 2 according to the sample S to be measured (that is, controls based on the initial setting). Excitation light emitted from the light source 2 based on the control of the control unit 6a is guided toward the sample S by the measurement unit 3 (that is, the scanning system (detection system) 5a and the microscope unit 4). Then, the measurement unit 3 scans the sample S with the excitation light emitted from the light source 2 and detects the fluorescence emitted from the sample S. The measurement unit 3 outputs the fluorescence detection result to the control unit 6a.

The control unit 6a generates a fluorescence intensity distribution based on the fluorescence detection result acquired from the measurement unit 3, and performs predetermined image processing (for example, noise removal processing or compression processing) on the fluorescence intensity distribution to obtain image data. Is generated. Then, the control unit 6a may display a U / I for designating a region in the image data on the display unit 72 together with the generated image data. Thereby, the user can designate the region to be observed for the image displayed on the display unit 72 via the condition designating unit 71.

(Step S503)
The control unit 6a receives the designation of the region to be observed in the image data based on the user input via the condition designating unit 71. Note that the area designated by the user corresponds to a variable range, and an object (part of the sample S) in the area is a target.

(Step S51)
The control unit 6a targets the image in the area designated based on the user input so that the object in the image is clearly displayed (for example, the contrast in the area is maximized). The wavelength of the excitation light emitted from the light source 2 is controlled. Details will be described later separately as “2.5. Details of Wavelength Control”.

(Step S505)
When the wavelength of the excitation light emitted from the light source 2 is controlled, the control unit 6a acquires a fluorescence intensity distribution based on the excitation light after the wavelength control, and generates image data based on the acquired fluorescence intensity distribution. Then, the control unit 6a causes the display unit 72 to display the generated image data. Thereby, the user can check the adjusted image based on the control of the wavelength of the excitation light emitted from the light source 2.

(Step S507)
Note that the image acquisition device 1a according to the present embodiment may receive an instruction from the user as to whether or not further adjustment is necessary for the adjusted image. In this case, for example, when the user instructs that further adjustment is necessary (step S507, NO), the condition (for example, a parameter such as a threshold for determining contrast) is changed. Then, the wavelength of the excitation light is controlled again.

(Step S509)
When the user instructs that no further adjustment is necessary for the adjusted image (step S507, YES), the image acquisition device 1a according to the present embodiment acquires the fluorescence acquired based on the excitation light at that time. RAW file D1 is generated on the basis of the intensity distribution.

The flow of a series of operations of the image acquisition device 1a according to the present embodiment has been described above with reference to FIG. The series of operations of the image acquisition device 1a described above is merely an example, and is not necessarily limited to the operation flow described above. As a specific example, the image acquisition device 1a may be operated so that the user can directly or indirectly specify the wavelength of the excitation light.

[2.5. Details of wavelength control]
<< 2.5.1. Principle of wavelength control: When observing a sample with multiple observation wavelengths >>
Next, the details of the operation related to the control of the wavelength of the laser light emitted from the light source 2 by the control unit 6a according to the present embodiment will be described with reference to FIGS. In this description, with reference to FIGS. 33 to 35, the principle of wavelength control in the image acquisition apparatus 1a according to the embodiment is a case where the composition of a sample is identified by a plurality of (for example, two) observation wavelengths. Explained as an example. The case where the composition of a sample is measured at a single observation wavelength will be separately described later as “2.3.3. One aspect of wavelength control: a case where a sample is observed at a single observation wavelength”.

First, refer to FIG. FIG. 33 is an explanatory diagram for explaining the principle of wavelength control of the laser light emitted from the light source 2, the wavelength of the laser light output from the light source 2, the excitation spectrum and the fluorescence spectrum of the fluorescent dye contained in the sample S An example of the relationship is shown. In the following, two different types of fluorescent dyes F1 and F2 in the sample S are to be observed, and the wavelength of the excitation light for causing the fluorescent dyes F1 and F2 to emit light (hereinafter referred to as “emission wavelength” in some cases). A) is assumed to be λ1 and λ2, respectively.

33, the horizontal axis indicates the wavelength [nm], and the vertical axis indicates the relative efficiency [%]. Reference sign g11 indicates the excitation spectrum of the fluorescent dye F1, and reference sign g12 indicates the fluorescence spectrum of the fluorescent dye F1. Reference sign g21 indicates the excitation spectrum of the fluorescent dye F2, and reference sign g22 indicates the fluorescence spectrum of the fluorescent dye F2.

Also, let the fluorescence relative efficiencies of the fluorescent dyes F1 and F2 at the emission wavelength λ1 determined by the fluorescence wavelength distribution (that is, fluorescence spectrum) g12 be E11 = E (λ1, F1) and E12 = E (λ1, F2). In addition, the fluorescence relative efficiencies of the fluorescent dyes F1 and F2 at the emission wavelength λ2 obtained from the fluorescence wavelength distribution (that is, fluorescence spectrum) g22 are E21 = E (λ2, F1) and E22 = E (λ2, F2).

Further, it is assumed that the light source 2 is configured such that the wavelength of the output laser light can be controlled for each wavelength λs within the band W1 defined between the wavelengths λH to λL.

The control unit 6a of the image acquisition device 1a according to the present embodiment generates a two-color identification image by synthesizing the fluorescence intensity distribution acquired for each of the emission wavelengths λ1 and λ2. For example, FIG. 34 is an explanatory diagram for explaining the principle of wavelength control of the laser light emitted from the light source, and is generated by synthesizing the fluorescence intensity distributions acquired for the respective emission wavelengths λ1 and λ2. An example of a two-color identification image is shown.

34, reference symbol V11a indicates the fluorescence intensity distribution when the emission wavelength is λ1, and reference symbol V12a indicates the fluorescence intensity distribution when the emission wavelength is λ2. Reference numeral V13a indicates an example of a two-color identification image obtained by combining the fluorescence intensity distributions V11a and V12a.

As shown in FIG. 34, when the fluorescence intensity distributions V11a and V12a acquired for the emission wavelengths λ1 and λ2, respectively, are synthesized, the obtained identification image V13a does not necessarily have a high contrast.

Therefore, for example, the control unit 6a (specifically, the system control unit 61a) of the image acquisition device 1a according to the present embodiment controls the emission wavelengths λ1 and λ2 so that the contrast of the identification image V13a is maximized. Thus, the ratio of the fluorescence intensity obtained based on each of the emission wavelengths λ1 and λ2 is adjusted.

In the following, referring to FIG. 35, an example of a process in which the system control unit 61a calculates the contrast of the two-color identification image generated by combining the fluorescence intensity distributions obtained based on the emission wavelengths λ1 and λ2, respectively. While explaining. FIG. 35 is an explanatory diagram for explaining the principle of wavelength control of laser light emitted from the light source 2, and is a diagram for explaining an example of a contrast calculation method when a sample is observed at a plurality of observation wavelengths. It is.

35, reference symbol V11 indicates the fluorescence intensity distribution in the case of the emission wavelength λ1, and reference symbol V12 indicates the fluorescence intensity distribution in the case of the emission wavelength λ2. Reference numeral V13 represents an example of a two-color identification image obtained by combining the fluorescence intensity distributions V11 and V12.

First, the system control unit 61a is based on the fluorescence relative efficiencies E11 = E (λ1, F1), E12 = E (λ1, F2), E21 = E (λ2, F1), and E22 = E (λ2, F2). Then, the initial values of the emission wavelengths λ1 and λ2 are calculated so that the following expression 12 is maximized.

The data indicating the excitation spectrum and the fluorescence spectrum of each of the fluorescent dyes F1 and F2 may be stored in advance on the database, for example, by configuring the storage unit 66 as a database. Thereby, the system control unit 61a calculates the fluorescence relative efficiencies E11, E12, E21, and E22 based on the data indicating the excitation spectrum and the fluorescence spectrum of each of the fluorescent dyes F1 and F2 stored in the storage unit 66 (database). It is possible to pass.

When the initial values of the emission wavelengths λ1 and λ2 are determined as described above, the system control unit 61a controls the light source 2 so that the laser beams having the determined emission wavelengths λ1 and λ2 are output, and the emission wavelengths λ1 and The fluorescence intensity distribution based on each λ2 is acquired. Thereby, fluorescence intensity distributions V11 and V12 based on the initial values of the emission wavelengths λ1 and λ2 are acquired.

Next, the system control unit 61a specifies a coordinate V111 = (x1, y1) having high luminance (for example, maximum luminance) from the acquired fluorescence intensity distribution V11. Similarly, the system control unit 61a specifies a coordinate V121 = (x2, y2) having a high luminance from the acquired fluorescence intensity distribution V12.

Once the coordinates V111 and V121 are specified, the system control unit 61a calculates the luminances L (λ1, x1, y1) and L (λ1, x2, y2) of the coordinates V111 and V121 in the fluorescence intensity distribution V11. Similarly, the system control unit 61a calculates the luminances L (λ2, x1, y1) and L (λ2, x2, y2) of the coordinates V111 and V121 in the fluorescence intensity distribution V12.

The system control unit 61a shows the calculated luminance L (λ1, x1, y1), L (λ1, x2, y2), L (λ2, x1, y1), and L (λ2, x2, y2) as follows: Based on Equation 13, the contrast C (λ1, λ2) is calculated.

As described above, the contrast C (λ1, λ2) of the two-color identification image V13 generated by combining the fluorescence intensity distributions V11 and V13 obtained based on the emission wavelengths λ1 and λ2, respectively, is calculated.

The system control unit 61a repeats the control of the light emission wavelengths λ1 and λ2 and the calculation and evaluation of the contrast C (λ1, λ2) based on the light emission wavelengths λ1 and λ2 after the control, so that the contrast C (λ1, λ2 The light emission wavelengths λ1 and λ2 may be adjusted so that) is maximized.

Next, the flow of controlling the emission wavelengths λ1 and λ2 by the system control unit 61a when observing a sample with a plurality of observation wavelengths will be described with reference to FIG. FIG. 36 is a flowchart for explaining the flow of processing relating to the wavelength control of the laser light emitted from the light source 2, and shows an example in which a sample is observed at a plurality of observation wavelengths.

(Step S511)
First, the system control unit 61a is based on the fluorescence relative efficiencies E11 = E (λ1, F1), E12 = E (λ1, F2), E21 = E (λ2, F1), and E22 = E (λ2, F2). The initial values of the emission wavelengths λ1 and λ2 are calculated. It should be noted that the initial values of the emission wavelengths λ1 and λ2 may be calculated so that the above-described Expression 12 becomes maximum.

(Step S513)
Next, the system control unit 61a performs control so that the wavelength of the excitation light emitted from the light source 2 becomes the calculated emission wavelength λ1. Thereby, the fluorescence detection result based on the controlled excitation light is output from the measurement unit 3a to the distribution information generation unit 62a. Then, the distribution information generation unit 62a generates a fluorescence intensity distribution V11 based on the emission wavelength λ1 based on the acquired fluorescence detection result. The distribution information generation unit 62a outputs the generated fluorescence intensity distribution V11 to the system control unit 61a. As described above, the system control unit 61a acquires the fluorescence intensity distribution V11 based on the controlled excitation light from the distribution information generation unit 62a.

(Step S515)
Further, the system control unit 61a performs control so that the wavelength of the excitation light emitted from the light source 2 becomes the calculated emission wavelength λ2. Thereby, the fluorescence detection result based on the controlled excitation light is output from the measurement unit 3a to the distribution information generation unit 62a. Then, the distribution information generation unit 62a generates a fluorescence intensity distribution V12 based on the emission wavelength λ2 based on the acquired fluorescence detection result. The distribution information generation unit 62a outputs the generated fluorescence intensity distribution V12 to the system control unit 61a. As described above, the system control unit 61a acquires the fluorescence intensity distribution V12 based on the controlled excitation light from the distribution information generation unit 62a.

(Step S517)
The system control unit 61a calculates the contrast C (λ1, λ2) based on the fluorescence intensity distributions V11 and V12 based on the generated emission wavelengths λ1 and λ2, respectively.

(Step S521)
The system control unit 61a continues the above processing until the contrast C (λ1, λ2) reaches the maximum while adjusting the emission wavelengths λ1 and λ2 (step S519, NO), and the contrast C (λ1, λ2) is When the maximum value is reached, the series of processing ends (step S519, YES).

Next, an example of a method in which the system control unit 61a identifies the emission wavelengths λ1 and λ2 that maximize the contrast C (λ1, λ2) will be described.

First, the system control unit 61a uses the case where the emission wavelengths λ1 and λ2 are both initial values as a reference state, and changes the emission wavelength λ1 from the reference state to λs in the plus direction, and changes from λs in the minus direction. A fluorescence intensity distribution V11 is obtained for. Then, the system control unit 61a calculates the contrast C (λ1, λ2) based on each of the acquired fluorescence intensity distributions V11, and identifies the changing direction of the emission wavelength λ1 in which the contrast C (λ1, λ2) increases.

Similarly, the system control unit 61a acquires the fluorescence intensity distribution V12 when the emission wavelength λ2 is changed by λs in the positive direction and when λs is changed by the negative direction from the reference state. Then, the system control unit 61a calculates the contrast C (λ1, λ2) based on each of the acquired fluorescence intensity distributions V12, and specifies the change direction of the emission wavelength λ2 in which the contrast C (λ1, λ2) increases. .

Next, the system control unit 61a changes the light emission wavelengths λ1 and λ2 from the reference state a plurality of times (for example, twice) by λs in the specified direction, and changes the contrast C (λ1, λ2) for each change. calculate. Based on the calculation result of the contrast C (λ1, λ2), the system control unit 61a emits light having a large change amount of the contrast C (λ1, λ2) among the light emission wavelengths λ1 and λ2 (that is, the contrast before and after the control). Emission wavelength with a large difference between C (λ1, λ2) is specified. In the following description, it is assumed that the amount of change in contrast C (λ1, λ2) is larger when the emission wavelength λ1 is changed than when the emission wavelength λ2 is changed.

The system control unit 61a fixes the emission wavelength λ2 side and calculates the contrast C (λ1, λ2) while changing the emission wavelength λ1 by λs in the specified direction, so that the contrast C (λ1, λ2) is maximized. The emission wavelength λ1 is specified.

Next, the system control unit 61a calculates the contrast C (λ1, λ2) while fixing the emission wavelength λ1 side and changing the emission wavelength λ2 by λs in the specified direction, so that the contrast C (λ1, λ2) is obtained. The maximum emission wavelength λ2 is specified.

As described above, the system control unit 61a identifies the emission wavelengths λ1 and λ2 at which the contrast C (λ1, λ2) is maximized. Note that the example described above is merely an example, and it goes without saying that the method is not particularly limited as long as the emission wavelengths λ1 and λ2 that maximize the contrast C (λ1, λ2) can be specified.

As described above, with reference to FIGS. 33 to 36, the details of the operation related to the control of the wavelength of the laser light emitted from the light source 2 by the control unit 6a according to the present embodiment will be described with reference to the composition of the sample at a plurality of observation wavelengths. The case of identifying has been described as an example.

<< 2.5.2. One aspect of wavelength control: When observing data with a single observation wavelength >>
Next, as an aspect of the operation related to the control of the wavelength of the laser light emitted from the light source 2 by the system control unit 61a according to the present embodiment, a case where the material is observed with a single observation wavelength will be described. In this description, the system control unit 61a is described assuming that the fluorescent dye F1 in the sample S is the target and controls the emission wavelength λ1.

In this case, first, the system control unit 61a determines an initial value of the emission wavelength λ1 so that the fluorescence relative efficiency E11 = E (λ1, F1) is maximized. The system control unit 61a may calculate the emission wavelength λ1 that maximizes the fluorescence relative efficiency E11 based on the peak value of the excitation spectrum of the fluorescent dye F1.

Next, the system control unit 61a uses the case where the emission wavelength λ1 is the initial value as a reference state, and changes the fluorescence when the emission wavelength λ1 is changed by λs in the plus direction and when the emission wavelength λ1 is changed by λs from the reference state. The intensity distribution V11 is acquired. Then, the system control unit 61a calculates the contrast of the acquired fluorescence intensity distribution V11 and identifies the changing direction of the emission wavelength λ1 in which the contrast increases. In addition, as a calculation method of the contrast of the fluorescence intensity distribution V11, for example, a so-called general contrast calculation method such as a contrast calculation method based on the maximum value and the minimum value of the pixel values in the intensity distribution V11. It is possible to apply.

When the change direction of the emission wavelength λ1 in which the contrast increases is specified, the system control unit 61a calculates the contrast while changing the emission wavelength λ1 by λs in the change direction, and specifies the emission wavelength λ1 that maximizes the contrast. .

As described above, the system control unit 61a specifies the light emission wavelength λ1 that maximizes the contrast. Note that the example described above is merely an example, and it is needless to say that the method is not particularly limited as long as the emission wavelength λ1 that maximizes the contrast can be specified.

[2.6. Summary]
As described above, the image acquisition device 1a according to the present embodiment uses the laser module that can control the wavelength of the output laser light (excitation light) as the light source 2, and based on the observed fluorescence intensity distribution. The wavelength of the laser beam is controlled. With such a configuration, the image acquisition device 1a according to the present embodiment enables the observer (user) to perform complicated work such as controlling the operation of the light source 2 while checking the observation result, or switching the light source 2 itself. Therefore, it is possible to obtain an image with high contrast.

In addition, the image acquisition device 1a according to the present embodiment allows the laser light to be maximized so that the contrast of the acquired fluorescence intensity distribution is maximized even in a situation where the wavelength of the laser light changes as the temperature rises. It is possible to control the wavelength. For this reason, the image acquisition device 1 according to the present embodiment can stabilize the wavelength of the laser light even if the light source 2 is incorporated in the same housing, for example, and thus the image acquisition device 1a itself can be downsized. Is possible.

<3. Third Embodiment>
[3.1. Overview of image acquisition device]
Next, an image acquisition apparatus according to the third embodiment will be described. In the first embodiment described above, the intensity control and correction of the emitted light from the light source 2 have been described, and in the second embodiment, the wavelength control of the emitted light from the light source 2 has been described. Needless to say. Therefore, in the third embodiment, a configuration in which the image acquisition device 1 according to the first embodiment described above and the image acquisition device 1a according to the second embodiment are combined will be described. Hereinafter, in order to distinguish the image acquisition device according to the present embodiment from the image acquisition device 1 according to the first embodiment and the image acquisition device 1a according to the second embodiment described above, “image May be described as “acquiring device 1b”.

[3.2. Configuration of image acquisition apparatus]
<< 3.2.1. Configuration of optical system >>
First, the configuration of the optical system of the image acquisition device 1b according to the present embodiment will be described. The configuration of the optical system of the image acquisition device 1b according to this embodiment is the same as that of the optical system of the image acquisition device 1 according to the first embodiment shown in FIG.

That is, in the image acquisition device 1b according to the present embodiment, the light source 2 includes a wavelength conversion module (OPO) 250, and the input laser light (pump light) is converted into laser light having two wavelengths (that is, signal light and idler). Light) and output. In the image acquisition device 1b according to the present embodiment, one of the signal light and idler light output from the light source 2 is irradiated toward the sample S as excitation light. In the following description, it is assumed that the signal light is emitted toward the sample S as excitation light.

Moreover, about the structure which irradiates excitation light toward the sample S, the optical system (refer FIG. 10) of the image acquisition apparatus 1w which concerns on the comparative example mentioned above, and the optical system of the image acquisition apparatus which concerns on each embodiment (FIG. 14, FIG. This is the same as in FIG. That is, the excitation light emitted from the light source 2 is guided to the objective lens 42 via the beam shaping lens 511, the galvano mirror 51, the lens 513, the mirror 517, the lens 515, and the dichroic mirror 52. It is condensed toward the sample S.

<< 3.2.2. Functional configuration of image acquisition device >>
Next, an example of a functional configuration of the image acquisition device 1b according to the present embodiment will be described with reference to FIG. As shown in FIG. 37, the image acquisition device 1b according to the present embodiment includes a light source 2, a measurement unit 3, a control unit 6b, and an I / F 7. The measurement unit 3 includes a microscope unit 4 and a scanning system (detection system) 5. The configurations of the light source 2, the microscope unit 4, the scanning system (detection system) 5, and the I / F 7 are the same as those of the image acquisition device 1 according to the first embodiment described above. Therefore, hereinafter, the description will be focused on the configuration of the control unit 6b that is different from the image acquisition device 1 according to the first embodiment. In the example shown in FIG. 37, each configuration of the scanning system (detection system) 5 corresponds to the configuration given the same reference numeral in the optical system shown in FIG. In the example shown in FIG. 37, a part of the configuration shown in FIG. 14 is omitted.

As shown in FIG. 37, the fluorescence generated from the sample S by the irradiated excitation light is guided to the scanning system (detection system) 5 through the microscope unit 4 and detected by the PMT 53 of the scanning system (detection system) 5. The The PMT 53 converts the detected fluorescence into an electrical signal by a photoelectric effect at a preset sampling rate, and outputs the electrical signal to the control unit 6b as data indicating the intensity of the fluorescence.

The PD 54 measures the intensity of idler light at a preset sampling rate. The PD 54 converts the intensity of the measured idler light into an electric signal, and outputs it to the control unit 6 as data indicating the intensity of the idler light. Instead of the PD 54, a PD 56 that measures the intensity of excitation light (signal light) may be provided. As described above, the configurations of the PMT 53, the PD 54, and the PD 56 are the same as those of the image acquisition device 1 according to the first embodiment described above.

As shown in FIG. 37, the control unit 6b includes a system control unit 61b, a distribution information generation unit 62b, a correction processing unit 63, an image processing unit 64, a RAW image generation unit 65, and a storage unit 66. The display control unit 67 and the communication control unit 68 are included. The control unit 6b according to the present embodiment has a configuration in which the configuration of the system control unit 61b and the distribution information generation unit 62b is the control unit 6 (see FIGS. 15 and 16) according to the first embodiment described above, It differs from the control part 6a (refer FIG.28 and FIG.29) which concerns on 2nd Embodiment.

Here, the detailed configuration of the system control unit 61b and the distribution information generation unit 62b will be described with reference to FIG. As illustrated in FIG. 38, the distribution information generation unit 62b includes a two-dimensionalization processing unit 621, an intensity correction distribution determination unit 623, and a two-dimensionalization processing unit 625. The system control unit 61 a includes an intensity control amount determination unit 611 and a wavelength control amount determination unit 613.

The operations of the two-dimensionalization processing unit 621, the intensity correction distribution determining unit 623, and the intensity control amount determining unit 611, and the operation of the system control unit 61b based on the operation of the intensity control amount determining unit 611 are described above. This is the same as the image acquisition device 1 according to the first embodiment (see FIG. 16).

That is, the intensity control amount determination unit 611 acquires the idler light intensity distribution from the two-dimensional processing unit 621, receives the notification of the maximum correction amount width from the intensity correction distribution determination unit 623, and acquires the acquired idler light intensity. Based on the distribution and the maximum width of the correction amount, the state of the light source 2 (particularly, the change in the intensity of the excitation light) is monitored. Further, the intensity control amount determination unit 611 generates control information for controlling the intensity of the laser light (pump light) emitted from the light source 2 based on the monitoring result. And the system control part 61b should just control the intensity | strength of the laser beam (pump light) radiate | emitted from the light source 2 based on the control information which the intensity | strength control amount determination part 611 produced | generated.

The two-dimensionalization processing unit 625, the wavelength control amount determination unit 613, and the operation of the system control unit 61b based on the operation of the wavelength control amount determination unit 613 are the image acquisition device 1a according to the second embodiment described above. (See FIG. 29).

That is, the wavelength control amount determination unit 613 obtains the generated fluorescence intensity distribution from the two-dimensional processing unit 625, and the laser light emitted from the light source 2 so as to improve the contrast of the fluorescence intensity distribution. The control amount of the wavelength (frequency) of (excitation light) is determined. When the control amount of the wavelength of the laser light is determined by the wavelength control amount determination unit 613, the system control unit 61b controls the light source 2 based on the control amount, so that the laser light emitted from the light source 2 is controlled. Control the wavelength.

By repeating the above operation, the wavelength of the laser light emitted from the light source 2 is controlled by the system control unit 61b and the wavelength control amount determination unit 613 so that the contrast of the fluorescence intensity distribution is maximized.

The operation of the RAW image generation unit 65 according to the present embodiment is the same as that of the RAW image generation unit 65 according to the first and second embodiments described above. That is, the RAW image generation unit 65 acquires the fluorescence intensity distribution from the two-dimensionalization processing unit 625, and shapes the fluorescence intensity distribution as image data (RAW image) into a predetermined file format, thereby converting the RAW file. Generate. The RAW image generation unit 65 may acquire information related to the fluorescence intensity distribution from the system control unit 61 and associate the acquired information with the generated RAW file as related information. The related information associated with the RAW file can include, for example, the related information shown in the first embodiment (see FIG. 19) or the related information shown in the second embodiment (see FIG. 31). Needless to say.

The configuration of the image acquisition device 1b according to the present embodiment has been described above with reference to FIGS. 37 and 38.

[3.3. Summary]
As described above, by combining the image acquisition device 1 according to the first embodiment and the image acquisition device 1a according to the second embodiment, the image acquisition device 1b according to the present embodiment may be operated. Is possible. In addition, it cannot be overemphasized that the image acquisition apparatus 1b which concerns on this embodiment has the effect of both the image acquisition apparatus 1 which concerns on 1st Embodiment, and the image acquisition apparatus 1a which concerns on 2nd Embodiment similarly. Yes.

<4. Fourth Embodiment>
[4.1. Overview of image acquisition device]
Next, an image acquisition apparatus according to the fourth embodiment will be described. First, the problems of the image acquisition apparatus according to the present embodiment will be organized with reference to FIGS. 39 and 40. FIG. 39 is a diagram for explaining the outline of the image acquisition apparatus according to the present embodiment. FIG. 40 is an explanatory diagram for explaining the principle of the correction processing according to the present embodiment. Hereinafter, the image processing apparatus according to the present embodiment may be referred to as “image acquisition apparatus 1c” in order to distinguish it from the image acquisition apparatuses according to other embodiments.

The image acquisition device 1 according to the first embodiment and the image acquisition device 1b according to the third embodiment generate correction data based on the intensity distribution of idler light, and generate fluorescence data based on the generated correction data. It corrects the noise caused by the fluctuation in the intensity of the laser beam, which is manifested on the intensity distribution.

On the other hand, since the intensity of fluorescence and idler light are detected separately, noise that does not depend on fluctuations in the intensity of laser light is included in the intensity distribution of the generated fluorescence and the intensity distribution of idler light, respectively. May occur separately. For example, FIG. 39 schematically shows a state in which different noises are generated in the generated fluorescence intensity distribution D11 and the idler intensity distribution D21.

In the case of the example shown in FIG. 39, when the fluorescence intensity distribution D11 is corrected based on the intensity distribution D21 of idler light, the fluorescence intensity distribution D10 after correction processing includes the fluorescence intensity distribution before correction. In addition to the noise generated in D11, the noise generated in the idler light intensity distribution D21 is also superimposed. That is, in the example shown in FIG. 39, the noise associated with the fluctuation in the intensity of the laser light is corrected by the correction process, but the noise generated in each of the fluorescence intensity distribution D11 and the correction data D21 is reduced. Without any actualization, it becomes apparent in the fluorescence intensity distribution D10 after the correction process.

Therefore, in the image acquisition device 1c according to the present embodiment, as shown in FIG. 40, noise correction processing is performed on each of the generated fluorescence intensity distribution D11 and idler light intensity distribution D21. As a result, the image acquisition apparatus 1c corrects the noise that does not depend on the intensity fluctuation of the laser beam, which is manifested in the fluorescence intensity distribution D11 and the idler intensity distribution D21, and corrects the fluorescence intensity distribution D11a and the noise after correction. An idler light intensity distribution D21a is generated.

The image acquisition apparatus 1c according to the present embodiment corrects the intensity distribution D11a of the fluorescence with the intensity distribution D21a of the idler light, so that noise associated with the intensity variation of the laser light that is manifested in the intensity distribution D11a of the fluorescence is obtained. It correct | amends and the fluorescence intensity distribution D10 after a correction process is produced | generated. With such a configuration, the image acquisition device 1c can suppress the occurrence of noise that does not depend on the intensity fluctuation of the laser light, which occurs in each of the fluorescence intensity distribution D11 and the idler intensity distribution D21. It becomes.

Further, the image acquisition device 1c according to the present embodiment generates the RAW file D1 based on the fluorescence intensity distribution D11a and the idler intensity distribution D21a after noise correction. As a result, even when an external device such as the information processing device 800 similarly generates the fluorescence intensity distribution D10 after the correction process, it is possible to suppress the occurrence of noise that does not depend on the intensity fluctuation of the laser light. It becomes. Details of the image acquisition device 1c according to the present embodiment will be described below.

[4.2. Configuration of image acquisition apparatus]
The image acquisition device according to the present embodiment is different from the image acquisition device 1b in the configuration of the distribution information generation unit 62c corresponding to the distribution information generation unit 62b in the image acquisition device 1b according to the third embodiment described above. The configuration is basically the same as that of the image acquisition device 1b. Therefore, hereinafter, description will be given focusing on the configuration of the distribution information generation unit 62c according to the present embodiment with reference to FIG. FIG. 41 is an explanatory diagram for describing detailed functional configurations of the distribution information generation unit 62c and the system control unit 61c according to the present embodiment.

As shown in FIG. 41, the distribution information generation unit 62c according to the present embodiment includes a correction processing unit 627 and a correction processing unit 629, and thus the distribution information generation unit 62b ( Unlike FIG. 38).

In the distribution information generation unit 62c according to the present embodiment, the two-dimensionalization processing unit 621 outputs the generated idler light intensity distribution to the correction processing unit 627.

The correction processing unit 627 acquires the idler light intensity distribution from the two-dimensional processing unit 621, and performs correction processing for noise that does not depend on the intensity fluctuation of the laser light, on the acquired idler light intensity distribution. At this time, the correction processing unit 627 applies a pixel-by-pixel filter to the acquired intensity distribution of idler light.

An example of the pixel-by-pixel filter is a median filter. In this case, the correction processing unit 627 uses, for example, each pixel in the idler light intensity distribution as a reference pixel, and sets the pixel value of the reference pixel as a threshold value to other surrounding pixels (for example, surrounding 8 pixels). Threshold processing is performed on the image. Then, the correction processing unit 627 smoothes the pixel value between the reference pixel and other surrounding pixels.

Note that when the pixel value is smoothed between the reference pixel and other surrounding pixels, the correction processing unit 627 may use a simple average, or smooth each pixel value according to predetermined statistics. Also good.

Further, the correction processing unit 627 may add anisotropy to threshold processing for other surrounding pixels and processing related to smoothing of pixel values between the reference pixel and other surrounding pixels. . As a specific example, the correction processing unit 627 may control a parameter (for example, a threshold value) of the noise removal process so that a stronger noise removal process is performed in the horizontal direction than in the vertical direction. Good.

The correction processing unit 627 outputs the intensity distribution of the idler light subjected to the noise removal processing to the intensity correction distribution determining unit 623 and the intensity control amount determining unit 611. Note that the subsequent processing, that is, the operations of the intensity correction distribution determination unit 623 and the intensity control amount determination unit 611, except for the point that the target is the intensity distribution of the idler light after the noise removal processing, This is the same as the image acquisition apparatus according to the third embodiment.

In the distribution information generation unit 62c according to this embodiment, the two-dimensionalization processing unit 625 outputs the generated fluorescence intensity distribution to the correction processing unit 629.

The correction processing unit 629 acquires the idler light intensity distribution from the two-dimensional processing unit 625, and performs correction processing on noise that does not depend on the intensity fluctuation of the laser light, on the acquired fluorescence intensity distribution. At this time, the correction processing unit 629 applies a frequency unit filter (that is, a frequency filter) to the acquired fluorescence intensity distribution.

Specifically, the correction processing unit 629 uses a predetermined frequency as a threshold and removes a signal having a frequency higher than the threshold as noise (that is, removes a high-frequency component). As described above, the correction processing unit 627 and the correction processing unit 629 perform noise removal processing with different methods on the acquired intensity distributions.

The correction processing unit 629 outputs the fluorescence intensity distribution subjected to the noise removal processing to the wavelength control amount determination unit 613, the correction processing unit 63, and the RAW image generation unit 65. The subsequent processing, that is, the operations of the wavelength control amount determination unit 613, the correction processing unit 63, and the RAW image generation unit 65 are the same as those described above except that the fluorescence intensity distribution after the noise removal processing is targeted. This is the same as the image acquisition device according to the second and third embodiments.

Also, the RAW image generation unit 65 according to the present embodiment generates a RAW file using the fluorescence intensity distribution after noise removal as image data, and relates the idler intensity distribution after noise removal to the RAW file. Needless to say, it is related as information.

The configuration of the image acquisition device 1c according to the present embodiment has been described with particular attention to the operation of the distribution information generation unit 62c, with reference to FIG. Note that the

correction processing units

627 and 629 are not necessarily provided, and only one of them may be provided.

[4.3. RAW file format]
Next, an example of the file format of the RAW file D1c according to the present embodiment will be described with reference to FIG. FIG. 42 is a diagram showing an example of the file format of the RAW file D1c according to the present embodiment.

42, the RAW file D1c according to the present embodiment includes, for example, a data area d10, a basic control information area d30, and an extension area d20c. The configurations of the data area d10 and the basic control information area d30 are the same as those of the RAW file D1 (see FIG. 19) according to the first embodiment described above, and detailed description thereof is omitted.

42, the extension area d20c includes a manufacturer note IFDd21. The manufacturer note IFDd21 is an IFD for storing information not specified in EXIF, such as a camera control mode, and also stores shooting information and control information unique to the image acquisition device 1.

In addition, the manufacturer note IFDd21 according to the present embodiment includes, for example, a shooting condition d211, an initial shot image d221, a target image d222 specified by the user, a variable range d223 specified by the user, and data d231 used for noise separation. And a parameter d232 applied to noise separation and an intensity image of idler light after noise correction. Note that the shooting condition d211, the initial shot image d221, the target image d222 specified by the user, and the variable range d223 specified by the user are the same as those of the RAW file D1a (see FIG. 31) according to the second embodiment described above. .

The idler intensity image d233 after noise correction indicates the idler intensity distribution D21a that has been subjected to noise removal processing by the correction processing unit 627. Instead of the idler intensity distribution D21a, correction data calculated based on the idler intensity distribution D21a may be recorded in the RAW file D1c as an idler intensity image d233 after noise correction.

Data d231 used for noise separation indicates frequency filter data applied to the fluorescence intensity distribution D11 for noise removal. Note that the format of the data d231 used for noise separation is not particularly limited as long as the content of the noise removal processing applied to the fluorescence intensity distribution D11 can be specified. As a specific example, the data d231 used for noise separation may be a histogram indicating the characteristics (frequency characteristics) of the frequency filter.

In this way, by recording the data d231 used for noise separation in the RAW file D1c, the external device (for example, the information processing device 800) that has read the RAW file D1c detects the noise on the RAW main image d11. Correction can be performed in the same manner as in the image acquisition device 1c according to the present embodiment.

The parameter d232 applied to the noise separation is a parameter indicating the type of noise removal processing performed on the correction data D21 based on the intensity distribution of idler light and the contents thereof (for example, information indicating the applied filter and its parameters). . Note that the format of the parameter d232 applied to the noise separation is not particularly limited as long as the type and contents of the noise removal processing performed on the correction data D21 can be specified.

In this way, by recording the parameter d232 applied to noise separation in the RAW file D1c, the external device (for example, the information processing device 800) that has read the RAW file D1c allows the intensity of idler light after noise correction. It is possible to specify the content of the noise removal process performed on the image.

The file format of the RAW file D1c according to the present embodiment has been described above with reference to FIG. Note that the file format of the RAW file D1c described above is merely an example, and it is needless to say that not all information may be included.

[4.4. Summary]
As described above, the image acquisition device 1c according to the present embodiment performs noise correction processing on each of the fluorescence intensity distribution D11 and the idler light intensity distribution D21, so that the fluorescence intensity distribution after noise correction is performed. D11a and idler intensity distribution D21a are generated. Then, the image acquisition device 1c corrects the noise accompanying the intensity fluctuation of the laser light that has become apparent in the fluorescence intensity distribution D11a by correcting the fluorescence intensity distribution D11a with the idler intensity distribution D21a. A fluorescence intensity distribution D10 after processing is generated. With such a configuration, the image acquisition device 1c according to the present embodiment makes manifestation of noise generated in each of the fluorescence intensity distribution D11 and the idler intensity distribution D21 independent of the intensity fluctuation of the laser beam. It becomes possible to suppress.

Also, the image acquisition device 1c according to the present embodiment generates a RAW file D1c based on the fluorescence intensity distribution D11a and the idler intensity distribution D21a after noise correction. As a result, even when an external device such as the information processing device 800 similarly generates the fluorescence intensity distribution D10 after the correction process, it is possible to suppress the occurrence of noise that does not depend on the intensity fluctuation of the laser light. It becomes.

<5. Hardware configuration>
Next, an example of a hardware configuration of the image acquisition device 1 according to the present embodiment will be described with reference to FIG. FIG. 43 is a diagram illustrating an example of a hardware configuration of the image acquisition device 1 according to the present embodiment.

As shown in FIG. 43, the image acquisition apparatus 1 according to the present embodiment includes a processor 901, a memory 903, a storage 905, a light source unit 907, a measurement unit 909, an operation device 911, a display device 913, A communication device 915 and a bus 917 are included.

The processor 901 may be, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), or a SoC (System on Chip), and executes various processes of the image acquisition device 1. The processor 901 can be configured by, for example, an electronic circuit for executing various arithmetic processes. Each configuration included in the control unit 6 described above can be configured by the processor 901.

The memory 903 includes a RAM (Random Access Memory) and a ROM (Read Only Memory), and stores programs and data executed by the processor 901. The storage 905 can include a storage medium such as a semiconductor memory or a hard disk. Note that the storage unit 66 described above can be configured by, for example, the memory 903 and the storage 905.

The light source unit 907 is a unit for irradiating the sample S with excitation light, and corresponds to the light source 2 described above. In the light source unit 907, the intensity and wavelength of the emitted excitation light are controlled by the processor 901.

The measurement unit 909 is a unit that guides the excitation light emitted from the light source unit 907 toward the sample S and detects the fluorescence from the sample S, and corresponds to the measurement unit 3 described above. The measurement unit 909 controls the optical path of the light emitted from the light source unit 907 according to the control of the processor 901, and scans the sample S with the light.

The operation device 911 has a function of generating an input signal for a user to perform a desired operation. The operation device 911 may include an input unit for a user to input information, such as buttons and switches, and an input control circuit that generates an input signal based on an input by the user and supplies the input signal to the processor 901. The above-described condition designating unit 71 can be configured by the operation device 911.

The display device 913 is an example of an output device, and may be a display device such as a liquid crystal display (LCD) device, an organic EL (OLED: Organic Light Emitting Diode) display device, or the like. The display device 913 can provide information by displaying a screen to the user. The display unit 72 described above can be configured by the display device 913.

The communication device 915 is a communication unit included in the image acquisition apparatus 1 and communicates with an external apparatus such as the information processing apparatus 800 via a network. The communication device 915 is an interface for wireless communication, and may include a communication antenna, an RF (Radio Frequency) circuit, a baseband processor, and the like.

The communication device 915 has a function of performing various kinds of signal processing on a signal received from an external device, and can supply a digital signal generated from the received analog signal to the processor 901. The communication unit 73 described above can be configured by the communication device 915.

The bus 917 connects the processor 901, the memory 903, the storage 905, the light source unit 907, the measurement unit 909, the operation device 911, the display device 913, and the communication device 915 to each other. The bus 917 may include a plurality of types of buses.

In addition, it is possible to create a program for causing hardware such as a CPU, ROM, and RAM incorporated in a computer to perform the same functions as the configuration of the image acquisition apparatus 1 described above. A computer-readable storage medium that records the program can also be provided.

<6. Summary>
The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various control examples or modification examples within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

In addition, the effects described in this specification are merely illustrative or illustrative, and are not limited. That is, the technology according to the present disclosure can exhibit other effects that are apparent to those skilled in the art from the description of the present specification in addition to or instead of the above effects.

The following configurations also belong to the technical scope of the present disclosure.
(1)
A light source that emits laser light;
A measurement unit that is provided in the same housing as the light source, scans the sample with the laser light, receives the laser light, and measures the intensity of the measurement target light from the sample;
A control unit that generates an image of the sample based on the measured intensity distribution of the measurement target light;
With
The measurement unit measures the intensity of the laser light emitted from the light source,
The control unit executes at least one of control of the intensity of the laser light emitted from the light source and correction of the measured intensity distribution of the measurement target light based on the measured intensity of the laser light. An image acquisition device.
(2)
The light source includes a laser unit that outputs pump light of a predetermined frequency, and an optical parametric oscillator, and emits signal light and idler light by causing the pump light to enter the optical parametric oscillator.
The measuring unit is
One of the signal light and the idler light is a first light, the other is a second light, and the sample is scanned with the first light,
Receiving the first light, measuring the intensity of the measurement object light generated from the sample and the intensity of the second light,
The control unit controls the intensity of the first light according to the control of the intensity of the pump light based on the measured intensity of the second light, and the intensity distribution of the measured light to be measured. Perform at least one of the corrections,
The image acquisition apparatus according to (1).
(3)
The controller is
Calculating the maximum width of the intensity correction amount based on the maximum value and the average value of the intensity of the second light during the warm-up period of the light source;
Based on the maximum width of the intensity correction amount and the maximum measurable value of the measurement target light, the control target maximum value of the second light is calculated,
Based on the control target maximum value of the second light and the average of the intensity of the second light measured during the scanning period related to the generation of the intensity distribution of the measurement target light, Control the intensity of the pump light,
The image acquisition apparatus according to (2).
(4)
The control unit controls the transfer function of the optical parametric oscillator based on the measured intensity distribution of the second light before the first timing during the warm-up period during the warm-up period of the light source. The information according to (2) or (3), wherein information is stored and the intensity of the pump light after the first timing during the warm-up period is controlled based on differential control according to the stored control information. Image acquisition device.
(5)
The control unit corrects the intensity distribution of the measurement target light measured after the first timing during the warm-up period based on the intensity distribution of the second light accumulated before the first timing. The image acquisition device according to (4).
(6)
The control unit includes the second light accumulated before the first timing in the image generated based on the intensity distribution of the measurement target light measured after the first timing during the warm-up period. The image acquisition device according to (5), wherein information indicating an intensity distribution of the image is associated.
(7)
The control unit calculates an intensity correction distribution based on the intensity distribution of the second light measured during the scanning period related to generation of the intensity distribution of the measurement target light, and based on the calculated intensity correction distribution The image acquisition device according to any one of (2) to (6), wherein the image is generated by correcting an intensity distribution of the measurement target light.
(8)
The control unit performs a noise removal process on the intensity distribution of the second light measured during the scanning period related to the generation of the intensity distribution of the measurement target light, and the noise removal process is performed Based on the intensity distribution of the second light, at least one of the control of the intensity of the first light and the correction of the intensity distribution of the measured light to be measured is executed (2) to (2) The image acquisition device according to any one of 7).
(9)
The control unit performs a noise removal process on the intensity distribution of the measurement target light, and controls the intensity of the first light based on the intensity distribution of the measurement target light subjected to the noise removal process; The image acquisition device according to (8), wherein at least one of the measured intensity distribution correction of the measurement target light is executed.
(10)
The image acquisition device according to (9), wherein the control unit executes a noise removal process for the intensity distribution of the second light and a noise removal process for the intensity distribution of the measurement target light by different methods.
(11)
The control unit removes noise by applying a pixel-based median filter to the intensity distribution of the second light, and applies noise to the intensity distribution of the measurement target light by applying a frequency filter. The image acquisition device according to (10), wherein the image acquisition device is removed.
(12)
The control unit generates information indicating the intensity distribution of the second light measured during the scanning period related to generation of the intensity distribution of the measurement target light based on the intensity distribution of the measurement target light. The image acquisition device according to any one of (2) to (11), which is associated with the image.
(13)
The control unit converts the intensity distribution of the second light measured during the scanning period related to generation of the intensity distribution of the measurement target light into two dimensions based on control information indicating the content of the scanning, The image acquisition device according to (12), wherein information indicating the dimensioned second light intensity distribution is associated with the image generated based on the intensity distribution of the measurement target light.
(14)
The light source is configured to be able to control the wavelength of the laser light,
The image acquisition apparatus according to any one of (1) to (13), wherein the control unit controls the wavelength of the laser light based on the measured intensity distribution of the measurement target light.
(15)
Scanning a sample with a laser beam emitted from a light source provided in the same housing, receiving the laser beam, and measuring the intensity of light to be measured generated from the sample;
A processor generates an image of the sample based on the measured intensity distribution of the measurement target light;
Measuring the intensity of the laser light emitted from the light source;
Based on the measured intensity of the laser light, performing at least one of control of the intensity of the laser light emitted from the light source and correction of the measured intensity distribution of the measurement target light;
An image acquisition method including:

1, 1a, 1b, 1c Image acquisition device 2 Light source 3, 3a Measuring unit 4 Microscope unit 41 Non-polarization beam splitter 42 Objective lens 43 Filter 44 Non-polarization beam splitter 45 Camera 46

Eyepiece

5, 5a Scanning system (detection system)
51 Galvano mirror 52 Dichroic mirror 53 PMT
54 PD
55

Beam splitters

6, 6a,

6b Control units

61, 61a, 61b, 61c System control unit 611 Intensity control amount determination unit 613 Wavelength control

amount determination units

62, 62a, 62b, 62c Distribution information generation unit 621 Two-dimensionalization processing unit 623 Intensity correction distribution determination unit 625 Two-dimensional processing unit 627 Correction processing unit 629 Correction processing unit 63 Correction processing unit 64 Image processing unit 65 RAW image generation unit 66 Storage unit 67 Display control unit 68 Communication control unit 7 I / F
71 Condition Specifying Unit 72 Display Unit 73 Communication Unit 800 Information Processing Device

Claims

A light source that emits laser light;
A measurement unit that is provided in the same housing as the light source, scans the sample with the laser light, receives the laser light, and measures the intensity of the measurement target light from the sample;
A control unit that generates an image of the sample based on the measured intensity distribution of the measurement target light;
With
The measurement unit measures the intensity of the laser light emitted from the light source,
The control unit executes at least one of control of the intensity of the laser light emitted from the light source and correction of the measured intensity distribution of the measurement target light based on the measured intensity of the laser light. An image acquisition device.
The light source includes a laser unit that outputs pump light of a predetermined frequency, and an optical parametric oscillator, and emits signal light and idler light by causing the pump light to enter the optical parametric oscillator.
The measuring unit is
One of the signal light and the idler light is a first light, the other is a second light, and the sample is scanned with the first light,
Receiving the first light, measuring the intensity of the measurement object light generated from the sample and the intensity of the second light,
The control unit controls the intensity of the first light according to the control of the intensity of the pump light based on the measured intensity of the second light, and the intensity distribution of the measured light to be measured. Perform at least one of the corrections,
The image acquisition apparatus according to claim 1.
The controller is
Calculating the maximum width of the intensity correction amount based on the maximum value and the average value of the intensity of the second light during the warm-up period of the light source;
Based on the maximum width of the intensity correction amount and the maximum measurable value of the measurement target light, the control target maximum value of the second light is calculated,
Based on the control target maximum value of the second light and the average of the intensity of the second light measured during the scanning period related to the generation of the intensity distribution of the measurement target light, Control the intensity of the pump light,
The image acquisition device according to claim 2.
The control unit controls the transfer function of the optical parametric oscillator based on the measured intensity distribution of the second light before the first timing during the warm-up period during the warm-up period of the light source. The image acquisition apparatus according to claim 2, wherein information is stored, and the intensity of the pump light after the first timing during the warm-up period is controlled based on differential control corresponding to the stored control information.
The control unit corrects the intensity distribution of the measurement target light measured after the first timing during the warm-up period based on the intensity distribution of the second light accumulated before the first timing. The image acquisition device according to claim 4.
The control unit includes the second light accumulated before the first timing in the image generated based on the intensity distribution of the measurement target light measured after the first timing during the warm-up period. The image acquisition apparatus according to claim 5, wherein information indicating an intensity distribution of the image is associated.
The control unit calculates an intensity correction distribution based on the intensity distribution of the second light measured during the scanning period related to generation of the intensity distribution of the measurement target light, and based on the calculated intensity correction distribution The image acquisition device according to claim 2, wherein the image is generated by correcting an intensity distribution of the measurement target light.
The control unit performs a noise removal process on the intensity distribution of the second light measured during the scanning period related to the generation of the intensity distribution of the measurement target light, and the noise removal process is performed 3. The control unit according to claim 2, wherein at least one of control of the intensity of the first light and correction of the measured intensity distribution of the measurement target light is executed based on the intensity distribution of the second light. Image acquisition device.
The control unit performs a noise removal process on the intensity distribution of the measurement target light, and controls the intensity of the first light based on the intensity distribution of the measurement target light subjected to the noise removal process; The image acquisition device according to claim 8, wherein at least one of the measured intensity distribution correction of the measurement target light is executed.
The image acquisition device according to claim 9, wherein the control unit executes a noise removal process for the intensity distribution of the second light and a noise removal process for the intensity distribution of the measurement target light by different methods.
The control unit removes noise by applying a pixel-based median filter to the intensity distribution of the second light, and applies noise to the intensity distribution of the measurement target light by applying a frequency filter. The image acquisition device according to claim 10, wherein the image acquisition device is removed.
The control unit generates information indicating the intensity distribution of the second light measured during the scanning period related to generation of the intensity distribution of the measurement target light based on the intensity distribution of the measurement target light. The image acquisition apparatus according to claim 2, wherein the image acquisition apparatus is associated with the image.
The control unit converts the intensity distribution of the second light measured during the scanning period related to generation of the intensity distribution of the measurement target light into two dimensions based on control information indicating the content of the scanning, The image acquisition apparatus according to claim 12, wherein information indicating the dimensioned distribution of the second light intensity is associated with the image generated based on the intensity distribution of the measurement target light.
The light source is configured to be able to control the wavelength of the laser light,
The image acquisition apparatus according to claim 1, wherein the control unit controls the wavelength of the laser light based on the measured intensity distribution of the measurement target light.
Scanning a sample with a laser beam emitted from a light source provided in the same housing, receiving the laser beam, and measuring the intensity of light to be measured generated from the sample;
A processor generates an image of the sample based on the measured intensity distribution of the measurement target light;
Measuring the intensity of the laser light emitted from the light source;
Based on the measured intensity of the laser light, performing at least one of control of the intensity of the laser light emitted from the light source and correction of the measured intensity distribution of the measurement target light;
An image acquisition method including: