WO2014020911A1

WO2014020911A1 - Observation device

Info

Publication number: WO2014020911A1
Application number: PCT/JP2013/004640
Authority: WO
Inventors: 大内　由美子
Original assignee: 株式会社ニコン
Priority date: 2012-08-01
Filing date: 2013-07-31
Publication date: 2014-02-06
Also published as: JPWO2014020911A1; JP5939301B2

Abstract

In order to reduce both the luminance irregularity inside a frame and the luminance irregularity between frames, an example of an observation device according to the present invention comprises an illumination optical system for illuminating a sample with laser light from a laser light source, and an imaging unit for repeatedly imaging the sample illuminated by the illumination optical system. The illumination optical system includes a light diffusion element and a control unit for periodically changing the diffusion pattern in the region of the light diffusion element onto which the laser light is irradiated. The control unit performs a setting so that the changes in the diffusion pattern during imaging operation match with each other among a plurality of the repeated imaging operations.

Description

Observation device

The present invention relates to an observation apparatus that illuminates an observation target region with a laser beam.

There is a structured illumination microscope as one of the microscopes using a laser light source. In some structured illumination microscopes, a plurality of modulated images are acquired while illuminating the sample with interference fringes (structured illumination light) by laser light, and a demodulating operation is performed on these modulated images. A super-resolution image (demodulated image) is generated. In this structured illumination microscope, in order to maintain the accuracy of the demodulation operation, the intensity of the interference fringes needs to be uniform at least in the observation target region on the specimen.

However, due to the coherence of the laser beam, there is a possibility that noise interference fringes and speckle noise are superimposed on the interference fringes. If luminance (intensity) unevenness different from the interference fringe pattern (hereinafter simply referred to as “luminance unevenness in the frame”) occurs in the modulated image due to this, the accuracy of demodulation operation is affected. As a known solution to this luminance unevenness, there is a method of inserting a rotating diffuser plate in the optical path of the laser light to lower the temporal coherency of the laser light (see Patent Document 1 etc.).

US Pat. No. 6,925,225

By the way, when the observation target is a biological specimen, it is necessary to shorten the imaging period, so that the exposure time per modulated image may be shortened. However, it has been found that when the exposure time per sheet is short, luminance unevenness between frames is conspicuous and the demodulation operation may fail due to this.

Therefore, an object of the present invention is to provide a laser observation apparatus capable of reducing not only luminance unevenness in a frame but also luminance unevenness between frames.

An example of the observation apparatus of the present invention is an observation apparatus including an illumination optical system that illuminates a sample with a laser beam from a laser light source, and an imaging unit that repeatedly images the sample illuminated by the illumination optical system. The illumination optical system includes a light diffusing element, and further includes a control unit that periodically changes a diffusion pattern of a region irradiated with the laser light in the light diffusing element, and the control unit is configured to capture the image. Settings are made so that changes in the diffusion pattern over time are the same among the plurality of repeated imaging operations.

Note that the control unit may perform the setting so that a repetition cycle of the imaging is an integral multiple of a change cycle of the diffusion pattern.

In addition, the control unit may adjust a repetition cycle of the imaging according to a change cycle of the diffusion pattern.

Further, the control unit may adjust a drive signal to the imaging unit in order to adjust a repetition cycle of the imaging.

In addition, the control unit may adjust a change cycle of the diffusion pattern according to a repetition cycle of the imaging.

The light diffusing element may be a rotatable light diffusing plate, and the light diffusing plate may be rotationally controlled by the control unit.

The light diffusing element may be a light diffusing element having a variable diffusion pattern, and the diffusion pattern may be controlled by the control unit.

In addition, an example of the observation apparatus of the present invention further includes a multimode optical fiber that relays the laser light between the laser light source and the illumination optical system, and the light diffusing element includes the laser light source and the light. You may arrange | position between fibers.

The laser light source includes a first laser light source that emits a first laser light, and a second laser light source that emits a second laser light having a wavelength different from that of the first laser light, and the imaging unit includes: A first image sensor that receives first observation light emitted from the sample in response to the first laser light, and a second image sensor that receives second observation light emitted from the sample in response to the second laser light. The one exposure time of the first image sensor and the one exposure time of the second image sensor may be individually adjustable by the control unit.

Further, the illumination optical system may be a structured illumination optical system that illuminates the sample with interference fringes generated by the laser light.

1 is a configuration diagram of a structured illumination microscope apparatus 1. FIG. 2A is a view of the rotational diffusion plate 109 as viewed from the direction along the optical axis, and FIG. 2B is a cross-sectional view obtained by cutting the rotational diffusion plate 109 along a plane including the optical axis. is there. 3A is a view of the three-way diffraction grating 131 ′ viewed from the direction along the optical axis, and FIG. 3B is a view of the light beam selection member 18 viewed from the direction along the optical axis. is there. 4A is a timing signal generated by the sensor 109B, FIG. 4B is a diagram showing the exposure timing of the first image sensor 351, and FIG. 4C is the second image sensor 352. Is a diagram showing the exposure timing (when the frame period Tf is an integral multiple of the rotation period Tr). FIG. 5A is a diagram showing a region where the laser spot sweeps on the rotating diffusion plate 109 during the exposure period of the first image sensor 351, and FIG. 5B shows the laser spot on the rotating diffusion plate 109. FIG. 8 is a diagram showing a region to be swept during the exposure period of the second image sensor 352 (when the frame period Tf is an integral multiple of the rotation period Tr). 6A is a timing signal generated by the sensor 109B, FIG. 6B is a diagram showing exposure timing of the first image sensor 351, and FIG. 6C is a second image sensor 352. Is a diagram showing the exposure timing (when the frame period Tf is a non-integer multiple of the rotation period Tr). FIG. 7A is a diagram showing a region where the laser spot sweeps on the rotary diffusion plate 109 during the exposure period of the first image sensor 351, and FIG. 7B shows the laser spot on the rotary diffusion plate 109. FIG. 8 is a diagram showing a region to be swept during the exposure period of the second image sensor 352 (when the frame period Tf is a non-integer multiple of the rotation period Tr).

[First Embodiment]
Hereinafter, a structured illumination microscope apparatus will be described as a first embodiment of the present invention.

First, the structure of the structured illumination microscope apparatus 1 will be described.

FIG. 1 is a configuration diagram of the structured illumination microscope apparatus 1. As shown in FIG. 1, the structured illumination microscope apparatus 1 includes a laser unit 100, an optical fiber 11, an illumination optical system 10, an imaging optical system 30, a first imaging element 351, a second imaging element 352, A control device 39, an image storage / arithmetic device 40, and an image display device 45 are provided. The illumination optical system 10 is an epi-illumination type and illuminates the specimen 5 via the first dichroic mirror 7 and the objective lens 6 of the imaging optical system 30.

The laser unit 100 includes a first laser light source 101, a second laser light source 102,

shutters

103 and 104, a mirror 105, a dichroic mirror 106, an acousto-optic variable filter (AOTF) 107, a lens 108, a rotating diffusion plate 109, and a rotating mechanism 109A. , Sensor 109B,

lenses

110 and 111, and FC connector 112 are provided.

Each of the first laser light source 101 and the second laser light source 102 is a coherent light source, and the emission wavelengths thereof are different from each other. Here, it is assumed that the wavelength λ1 of the first laser light source 101 is longer than the wavelength λ2 of the second laser light source 102 (λ1> λ2).

When laser light having a wavelength λ 1 (first laser light) emitted from the first laser light source 101 enters the dichroic mirror 106 through the shutter 103 and the mirror 105, the dichroic mirror 106 is reflected. On the other hand, when the laser beam having the wavelength λ2 (second laser beam) emitted from the second laser light source 102 enters the beam splitter 106 via the shutter 104, the laser beam passes through the dichroic mirror 106 and is integrated with the first laser beam. The

Laser light (at least one of the first laser light and the second laser light) emitted from the dichroic mirror 106 is transmitted through the AOTF 107, the lens 108, the rotation diffusion plate 109, the lens 110, the lens 111, and the FC connector 112. Incident at the incident end.

Among these, the lens 108 has a function of condensing laser light (at least one of the first laser light and the second laser light) on the rotating diffusion plate 109, and the

lenses

110 and 110 have the function of focusing on the rotating diffusion plate 109. It functions to project an image of the formed condensing point (laser spot) onto the incident end of the optical fiber 11 at an appropriate magnification.

Here, the rotating diffusion plate 109 is a transmission type diffusion plate in which a large number of fine structures (fine lenses, fine particles, etc.) are randomly arranged as shown in FIGS. 2 (A) and 2 (B). Is inserted into the optical path of the laser beam (near the laser spot). The rotation axis 109S of the rotary diffusion plate 109 is parallel to the optical path of the laser beam and is off the condensing point (laser spot) of the laser beam. When the rotating diffusion plate 109 rotates around the rotation axis 109S, the diffusion pattern acting on the laser spot changes, and when the rotation is repeated, the diffusion pattern acting on the laser spot changes periodically. Therefore, the rotary diffusion plate 109 has a function of reducing temporal coherency of the laser light toward the optical fiber 11 and suppressing interference fringe intensity unevenness described later.

As a motor for the rotation mechanism 109A (FIG. 1) for rotating the rotation diffusion plate 109, a brushless motor with little vibration is suitable. The brushless motor can rotate the rotating diffusion plate 109 at a rotational speed of 10,000 rpm to 40000 rpm. At present, the rotational speed of the rotating diffusion plate 109 can be stabilized at about 10,000 rpm. Therefore, in the following, it is assumed that the rotational speed of the rotating diffusion plate 109 is set to 10,000 rpm. By the way, this rotation speed of 10,000 rpm is 6 ms.
Here, the rotation period is the time required for the rotation diffusion plate 109 to make one rotation.

Further, an impermeable mark 109A is formed at a predetermined position on the outer peripheral side of the rotating diffusion plate 109, and the sensor 109B is arranged in a non-contact state at a predetermined position on the locus of the mark 109A. The sensor 109B is composed of a photo interrupter or the like. When the mark 109A is facing the sensor 109B, the value of the detection signal of the sensor 109B is small, and when the mark 190A is not facing the sensor 109B, the sensor 109B is detected. The value of the detection signal becomes larger. Therefore, the detection signal of the sensor 109B can be used as a signal (timing signal) indicating the rotation timing of the rotation diffusion plate 109.

Referring back to FIG. 1, the optical fiber 11 is a multimode optical fiber that guides the laser light, and can guide the laser light while maintaining the coherency reduction effect by the rotating diffusion plate 109. The laser light (at least one of the first laser light and the second laser light) incident on the incident end of the optical fiber 11 propagates through the optical fiber 11 to generate a point light source at the output end of the optical fiber 11, The light enters the illumination optical system 10.

The illumination optical system 10 includes, in order from the point light source side, a collector lens 12, a polarizing plate 23, a diffractive optical element (diffraction grating) 13, a condenser lens 16, a zero-order light cut mask 14, and a lens 25. The field stop 26, the field lens 27, the excitation filter 28, the first dichroic mirror 7, and the objective lens 6 are disposed.

When the laser light (at least one of the first laser light and the second laser light) emitted from the point light source is converted into a parallel light beam by the collector lens 12 and enters the diffraction grating 13 via the polarizing plate 23, the diffraction of each order. Branched into luminous flux. The diffracted light beams of these orders are condensed by the condenser lens 16 at different positions on the pupil conjugate plane 6A ′.

Here, the pupil conjugate plane 6A ′ is the focal position (rear focal position) of the lens 16, and the lens 27 with respect to the pupil 6A of the objective lens 6 described later (position where ± 1st-order diffracted light is condensed). These positions are conjugate positions via the lens 25 (the concept of these positions is determined by a person skilled in the art in consideration of design necessary matters such as the aberration and vignetting of the objective lens 6 and the lenses 27 and 25). The position was also included.)
The diffraction grating 13 is a unidirectional diffraction grating having a periodic structure in a direction perpendicular to the optical axis of the illumination optical system 10, and the polarizing plate 23 has the same polarization direction of the laser light as that of the grating lines of the diffraction grating 13. It is a polarizing plate arranged in the direction.

Among these, the diffraction grating 13 can be translated in the branching direction of the diffracted light beam (= the arrangement direction of the condensing points of ± first-order diffracted light beams on the pupil conjugate plane 6A ′) by a translation mechanism 15A made of a piezo motor or the like. Note that the direction of translational movement may be a direction having a component in the same direction as the branching direction even if it does not coincide with the branching direction. When the diffraction grating 13 is translated in this direction, the phase of interference fringes described later shifts.

Further, both of the diffraction grating 13, the polarizing plate 23, and the translation mechanism 15A can be rotated at a pitch of 120 ° around the optical axis by a rotation mechanism 15B made of an electric motor or the like. When the diffraction grating 13 is rotated, the direction of interference fringes described later is switched between 0 °, 120 °, and 240 °. When the polarizing plate 23 is rotated together with the diffraction grating 13, the direction of the interference fringes and the polarization direction are changed. The relationship is maintained.

The direction of translation by the translation mechanism 15A is set to a predetermined direction so that the phase of the interference fringes can be shifted regardless of the rotation position of the diffraction grating 13 being 0 °, 120 °, or 240 °. Shall be. However, in this case, since the relationship between the translational movement amount and the phase shift amount differs depending on the rotation position of the diffraction grating 13, the translational movement pitch is set so that the phase shift amount becomes equal regardless of the rotation position of the diffraction grating 13. It is assumed that it is set for each rotation position of the diffraction grating 13.

The diffracted light beams of the respective orders emitted from the diffraction grating 13 and directed toward the pupil conjugate plane 6A ′ enter the 0th-order light cut mask 14 disposed in the vicinity of the pupil conjugate plane 6A ′. The 0th-order light cut mask 14 is a mask that selectively passes only the necessary diffracted light beam (here, only ± 1st-order diffracted light beam) among the incident diffracted light beams of each order. The 0th-order light cut mask 14 is formed by forming a plurality of openings or transmission parts on a mask substrate, and the positions where the openings or transmission parts are formed on the substrate are ± on the pupil conjugate plane 6A ′. This corresponds to the position where the first-order diffracted light beam enters.

The ± 1st-order diffracted light beam that has passed through the 0th-order light cut mask 14 is converted into parallel light by the field lens 27 after forming a plane conjugate with the diffraction grating 13 in the vicinity of the field stop 26 by the lens 25, and further, an excitation filter After passing through 28, the light is reflected by the first dichroic mirror 7 and condensed at different positions on the pupil plane 6A of the objective lens 6.

Each of the ± first-order diffracted light beams collected on the pupil plane 6A becomes a parallel light beam when emitted from the tip of the objective lens 6 and overlaps each other on the surface of the sample 5 to form interference fringes. This interference fringe is used as structured illumination light.

When the specimen 5 is illuminated with such interference fringes, the periodic structure of the interference fringes and the periodic structure (of the fluorescent region) of the specimen 5 generate moire fringes. In the moire fringes, the high-frequency structure of the specimen 5 is generated. Is shifted to a lower frequency side than the original frequency, the light generated according to this structure travels toward the objective lens 6 at an angle smaller than the original angle. Therefore, when the specimen 5 is illuminated by the interference fringes, even the high frequency structural information of the specimen 5 is transmitted by the objective lens 6.

Here, the specimen 5 is, for example, a culture solution dropped on a parallel plate-like glass surface, and in the vicinity of the glass interface in the culture solution, there is a fluorescent cell (a cell stained with a fluorescent dye). ) Exists. In this cell, both the first fluorescence region excited by the first laser beam having the wavelength λ1 and the second fluorescence region excited by the second laser beam having the wavelength λ2 are expressed. In the first fluorescence region, the first fluorescence having the center wavelength λ1 'is generated according to the first laser beam, and in the second fluorescence region, the second fluorescence having the center wavelength λ2' is generated according to the second laser beam.

Fluorescence generated in the specimen 5 (at least one of the first fluorescence and the second fluorescence) enters the imaging optical system 30. In the imaging optical system 30, an objective lens 6, a first dichroic mirror 7, a barrier filter 31, a second objective lens 32, and a second dichroic mirror 35 are arranged in this order from the sample 5 side.

When fluorescence (at least one of the first fluorescence and the second fluorescence) generated in the sample 5 is incident on the objective lens 6, it is converted into parallel light by the objective lens 6, and then the first dichroic mirror 7, the barrier filter 31, The light enters the second dichroic mirror 35 through the two objective lens 32. The first fluorescence incident on the second dichroic mirror 35 reflects the second dichroic mirror 35, and the second fluorescence incident on the second dichroic mirror 35 passes through the second dichroic mirror 35.

The first fluorescence reflected from the second dichroic mirror 35 forms a modulated image of the first fluorescence region on the imaging surface 361 of the first imaging element 351, and the second fluorescence transmitted through the second dichroic mirror 35 is second A modulated image of the second fluorescent region is formed on the imaging surface 362 of the imaging element 352.

Each of the first image sensor 351 and the second image sensor 352 is a charge storage type two-dimensional image sensor such as a CCD or a CMOS. The first image sensor 351 captures a modulated image of the first fluorescent region, generates a first modulated image, and sends it to the control device 39. The second image sensor 352 captures a modulated image of the second fluorescent region. As a result, a second modulated image is generated and sent to the control device 39.

The control device 39 includes a first laser light source 101, a second laser light source 102,

shutters

103 and 104, an AOTF 107, a rotation mechanism 109A, a sensor 109B, a translation mechanism 15A, a rotation mechanism 15B, a first image sensor 351, and a second image sensor. When a series of first modulated images and a series of second modulated images are acquired by controlling each of 352, the series of first modulated images and series of second modulated images are provided to the image storage / calculation device 40.

Here, the control device 39 can control the irradiation intensity of the first laser beam on the specimen 5 by controlling the transmittance of the AOTF 107 with respect to the wavelength λ1. As a result, the brightness of the modulated image in the first fluorescent region can be made appropriate, and the dynamic range of the image sensor 351 can be used effectively.

Further, the control device 39 can control the irradiation intensity of the second laser beam on the specimen 5 by controlling the transmittance of the AOTF 107 with respect to the wavelength λ2. Thereby, the brightness of the modulated image in the second fluorescent region can be made appropriate, and the dynamic range of the image sensor 352 can be effectively used.

The control device 39 controls the combination of the on / off timing of the first laser light source 101, the opening / closing timing of the shutter 103, and the on / off timing with respect to the wavelength λ1 of the AOTF 107, whereby the first laser light for the specimen 5 is controlled. The irradiation timing can be controlled.

Further, the control device 39 controls the combination of the on / off timing of the second laser light source 102, the opening / closing timing of the shutter 104, and the on / off timing with respect to the wavelength λ2 of the AOTF 107, whereby the second laser light for the specimen 5 is controlled. The irradiation timing can be controlled.

Further, the control device 39 can control the frame period of the first image sensor 351 by controlling the charge accumulation timing and the charge read timing of the first image sensor 351.

Further, the control device 39 can control the frame period of the second image sensor 352 by controlling the charge accumulation timing and the charge read timing of the second image sensor 352.
Here, the frame period is the time from the start of capturing an image to the start of capturing the next image by the image sensor.

The control device 39 also includes a charge accumulation period within the frame period of the first image sensor 351 and an open period within the frame period of a mechanical shutter (not shown) disposed between the first image sensor 351 and the sample 5. By controlling this combination, the exposure time (fluorescence light reception time) within the frame period of the first image sensor 351 can be controlled.

The control device 39 also includes a charge accumulation period within the frame period of the second image sensor 352, and an open period within the frame period of the mechanical shutter (not shown) disposed between the second image sensor 352 and the sample 5. By controlling the combination, the exposure time (fluorescence light receiving time) within the frame period of the second image sensor 352 can be controlled.

Note that the exposure time of the first imaging element 351 can be controlled by controlling the irradiation period of the first laser light, but the irradiation period of the first laser light and the emission period of the first fluorescence are completely the same. However, it is difficult to maintain control responsiveness. Therefore, in the present embodiment, the exposure time of the first image sensor 351 is controlled by controlling parameters other than the irradiation period of the first laser light as described above. In this case, the irradiation waveform of the first laser light is set to a continuous wave (CW).

Similarly, the exposure time of the second image sensor 352 can be controlled by controlling the irradiation period of the second laser light, but the irradiation period of the second laser light and the emission period of the second fluorescence are completely the same. However, it is difficult to maintain control responsiveness. Therefore, in the present embodiment, the exposure time of the second image sensor 352 is controlled by controlling parameters other than the irradiation period of the second laser light as described above. In this case, the irradiation waveform of the second laser light is set to a continuous wave (CW).

In the present embodiment, the exposure time Te ₁ within the frame period of the first image sensor 351 and the exposure time Te ₂ within the frame period of the second image sensor 352 are individually set, whereas The frame period of the first image sensor 351 and the frame period of the second image sensor 352 are set to a common value Tf. In the present embodiment, for simplicity, it is assumed that the charge reading time of the first image sensor 351 and the charge reading time of the second image sensor 532 are a common value To (for example, 10 ms).

The image storage / arithmetic unit 40 performs a known demodulation calculation on the series of first modulated images given from the control unit 39 to generate a first demodulated image (first super-resolution image), and also the control unit A known demodulation operation is performed on the series of second modulated images given from 39 to generate a second demodulated image (second super-resolution image). As a known demodulation operation, for example, a method disclosed in US Pat. No. 8,115,806 can be used. The image storage / arithmetic apparatus 40 stores the first super-resolution image and the second super-resolution image in an internal memory (not shown) of the image storage / arithmetic apparatus 40 and sends them to the image display apparatus 45. To do.

Next, the operation of the control device 39 will be described in detail. The control device 39 obtains a series of first modulated images necessary for the demodulation operation and a series of second modulated images necessary for the demodulation operation by the following procedures (1) to (7).

(1) The controller 39 sets the rotation position of the diffraction grating 13 (and the polarizing plate 23) to 0 ° (first direction) by driving the rotation mechanism 15B. In addition, the control device 39 starts the rotation of the rotating diffusion plate 109 by driving the rotating mechanism 109A.

(2) The control device 39 starts the translational movement of the diffraction grating 13 by starting to drive the translation mechanism 15A when the rotational speed of the rotary diffusion plate 109 is stabilized.

(3) The control device 39 repeats exposure (imaging) of both the first imaging element 351 and the second imaging element 352 over a plurality of sheets while the diffraction grating 13 is translated.

Here, when the sample 5 is a biological sample, the sample 5 may change during an imaging period of a plurality of images. In this case, a pseudo moiré is generated on the modulated image, and the demodulated image is displayed. There is a risk of artifact noise overlapping. For this reason, when the specimen 5 is a biological specimen, the exposure time Te ₁ of the first image sensor 351 within the frame period and the exposure time Te ₂ of the second image sensor 532 within the frame period are minimized (for example, The first image sensor 351 and the shortest frame period Ts = 11 ms represented by the sum of the longer exposure time (for example, Te ₂ = 1 ms) and the charge readout time To (for example, 10 ms). It is conceivable to set the frame period Tf of the second image sensor 352.

However, if the frame period Tf of the first image sensor 351 and the second image sensor 352 is set to the shortest frame period Ts = 11 ms, uneven luminance between frames of a series of first modulated images and frames of a series of second modulated images. Was found to occur. Therefore, the control device 39 of the present embodiment executes the following procedure (4) during the imaging period for a plurality of images.

(4) The control device 39 detects the rotation period Tr of the rotating diffusion plate 109 (the time required for the rotating diffusion plate 109 to make one rotation) by driving the sensor 109B during the imaging period for a plurality of images. As shown in FIGS. 4A, 4B, and 4C, the frame period Tf of the first image sensor 351 and the second image sensor 352 is set to an integral multiple of the rotation period Tr.

Specifically, the control unit 39, the longer exposure time (e.g., _Te 2 = 1 ms) and a charge readout time To in the exposure time Te ₂ of the exposure time Te ₁ and the second image sensor 352 of the first image sensor 351 = The shortest frame period Ts = 11 ms, which is the sum of 10 ms, and the shortest frame period among the integral multiples of the rotation period Tr = 6 ms (Tr = 6 ms, 2Tr = 12 ms, 3Tr = 18 ms, 4Tr = 24 ms,. The frame period Tf is set to a value (12 ms) that is Ts = 11 ms or more and is closest to the shortest frame period Ts = 11 ms.

However, there is a possibility that the rotation period Tr fluctuates during the imaging period over a plurality of sheets. Therefore, the control device 39 in the procedure (4) performs exposure of each frame using a timing signal generated by the sensor 109B as a trigger. That is, when the frame period Tf is represented by Tf = n × Tr, the control device 39 drives the first image sensor 351 and the second image sensor 352 every time the timing signal is generated n times. In this way, fluctuations in the rotation period Tr can be dealt with.

Incidentally, a slight delay time is generated from the generation of the timing signal to the start of exposure of the first image sensor 352 and the second image sensor 352, but if the delay time does not vary during the imaging period of a plurality of images, the delay There is no problem with the time itself.

As described above, when the frame period Tf of the first image sensor 351 is controlled to be an integral multiple of the rotation period Tr, the laser spot is exposed to the first image sensor 351 on the rotary diffusion plate 109 as shown in FIG. Since a region to be swept during the period (sweep region) can be shared between frames, even if there is unevenness in the transmittance of the rotating diffusion plate 109, unevenness in luminance between frames of a series of first modulated images. It does not occur. This also applies to the second image sensor 352 (that is, the second modulated image) (see FIG. 5B).

6A, 6B, and 6C, when the frame period Tf of the first image sensor 351 is a non-integer multiple of the rotation period Tr, as shown in FIG. Since the region (sweep region) where the laser spot sweeps during the exposure period of the first image sensor 351 on the rotary diffusion plate 109 cannot be shared between frames, the transmittance of the rotary diffusion plate 109 is uneven. Then, luminance unevenness occurs between frames of a series of first modulated images. The same applies to the second image sensor 352 (that is, the second modulated image) (see FIG. 7B).

Further, luminance unevenness between frames of the first modulation image, as when the sweep area shown in FIG. 7 (A) is short, i.e. smaller the exposure time Te ₁ is shorter than the rotation period Tr, so prone, It is considered that the significance of executing this procedure (4) is great. This also applies to the second modulated image (that is, the second image sensor 352).

(5) The control device 39 sets the pitch of the translational movement of the diffraction grating 13 to a pitch corresponding to the above-described frame period Tf during the imaging period of a plurality of frames, thereby adjacent frames of a series of first modulated images. Each of the phase difference of the interference fringes between and the phase difference of the interference fringes between adjacent frames of the series of second modulated images is set to a predetermined value Δφ less than 2π.

(6) The control device 39 drives the rotation mechanism 15B to set the rotation position of the diffraction grating 13 (and the polarizing plate 23) to 120 ° (second direction), and in this state, the procedure (2) Execute (5).

(7) The controller 39 drives the rotating mechanism 15B to set the rotational position of the diffraction grating 13 (and the polarizing plate 23) to 240 ° (third direction), and in this state, the steps (2) to (2) (5) is executed.

Through the above procedures (1) to (7), the control device 39 causes the interference fringe direction to be in the first direction and the phase of the interference fringe is shifted by Δφ, and the interference fringe direction. At least three first modulated images whose directions are the second direction and the phase of the interference fringes are shifted by Δφ, and at least three images whose directions of the interference fringes are the third direction and whose phases are shifted by Δφ Are obtained in order. These at least nine first modulated images are a series of first modulated images necessary for generating the first super-resolution image.

In parallel with this, by the above procedures (1) to (7), the control device 39 causes the direction of the interference fringes to be the first direction and the phase of the interference fringes to be shifted by Δφ by at least three second modulated images. And at least three second modulated images in which the interference fringe direction is the second direction and the interference fringe phase is shifted by Δφ, and the interference fringe direction is the third direction and the interference fringe phase is Δφ increments. At least three shifted second modulated images are sequentially acquired. These at least nine second modulated images are a series of second modulated images necessary for generating the second super-resolution image.

As described above, the structured illumination microscope apparatus 1 of the present embodiment includes the illumination optical system (10) that illuminates the sample (specimen 5) with the laser light from the laser light sources (101, 102), and the illumination optical system (10). The illumination optical system (10) includes a light diffusing element (rotating diffusion plate 109), and includes an imaging unit (imaging elements 351 and 352) that repeatedly images the illuminated sample (specimen 5). The device (rotating diffusion plate 109) further includes a control unit (rotating mechanism 109A, control device 39) that periodically changes the diffusion pattern of the region irradiated with the laser light, and the control unit (rotating mechanism 109A, control device). 39) performs the setting so that the change of the diffusion pattern during the imaging time of the imaging is the same among the plurality of repeated imaging.

Further, the control unit (the rotation mechanism 109A, the control device 39) performs the setting so that the imaging repetition period (Tf) is an integral multiple of the diffusion pattern change period (Tr).

The repetition period (Tf) of imaging is the time from the start of imaging of the image (modulated image) of the sample (sample 5) to the start of imaging of the next sample image (modulated image), as in the frame period. That is.

Also, the diffusion pattern change period (Tr) is the time required to change a series of diffusion patterns (for one set).

In addition, here, the range in which the change of the diffusion pattern is regarded as the same among the plurality of the imaging is as follows. In other words, it is assumed that a plurality of images (image sets) are acquired by repeatedly imaging a sample (specimen 5) having a uniform brightness in the real field of view under common illumination conditions and common imaging conditions. In the image set, an image having the lowest average luminance value of all the pixels constituting the image is set as a reference image. At this time, when the difference between the luminance value of each pixel of the image other than the reference image and the luminance value of the corresponding pixel of the reference image is ± 7% or less (preferably ± 5% or less), a plurality of the repeated It is assumed that the change in the diffusion pattern was the same during the imaging. In order to realize this, the deviation between the position where the light beam passes at the imaging start timing in the light diffusing element (rotating diffusion plate 109) and the rotation reference position of the diffusion pattern is an arc where the light beam sweeps during the imaging time. It may be ± 7% or less (preferably ± 5% or less) with respect to the length. An integer multiple condition is also regarded as an integer multiple within the above range.

Therefore, the structured illumination microscope apparatus 1 of the present embodiment can suppress both luminance unevenness in the frame and luminance unevenness between the frames.

Specifically, in the above-described procedure (4), when the frame period Tf is set to 11 ms (that is, a non-integer multiple of the rotation period Tr), the luminance fluctuation amount between frames is 18% of the average luminance between frames. On the other hand, when the frame period Tf was set to 12 ms (that is, an integer multiple of the rotation period Tr), the luminance fluctuation amount between frames could be suppressed to 0.6% of the average luminance between frames.

In addition, as another method of suppressing the luminance unevenness between the frames, there is a method in which the number of rotating diffusion plates is made plural. In order to obtain the same effect as this embodiment by this method, the number of rotating diffusion plates is reduced. It is necessary to make it 3 or more, and this causes a large loss in the amount of laser light. Therefore, the method of the present embodiment is also effective in that the loss of light quantity of laser light can be suppressed.

Further, the control unit (the rotation mechanism 109A, the control device 39) adjusts the imaging repetition period (Tf) according to the diffusion pattern change period (Tr).

Further, the control unit (the rotation mechanism 109A, the control device 39) adjusts a drive signal to the image pickup unit (image pickup elements 351 and 352) in order to adjust the repetition period (Tf) of the image pickup.

Therefore, even if fluctuations occur in the diffusion pattern change period (Tr), the control unit (the rotation mechanism 109A, the control device 39) can cope with it.

Further, in the structured illumination microscope apparatus 1 of the present embodiment, the light diffusing element (rotating diffusing plate 109) is a rotatable light diffusing plate.

Therefore, the structured illumination microscope apparatus 1 according to the present embodiment can easily stabilize the diffusion pattern change period (Tr). As a result, the imaging repetition period (Tf) is set to the diffusion pattern change period (Tr). It is easy to maintain an integer multiple of.

The structured illumination microscope apparatus 1 of the present embodiment further includes a multimode optical fiber (11) that relays laser light between the laser light source (101, 102) and the illumination optical system (10). The insertion point of the light diffusion element (rotating diffusion plate 109) is between the laser light source (101, 102) and the optical fiber (11).

Therefore, in the structured illumination microscope apparatus 1 of the present embodiment, the insertion position of the light diffusing element (rotating diffusion plate 109) that can be a vibration generation source is isolated from the illumination optical system (10), and the light diffusing element It is possible to both maintain the effect of reducing coherency by the (rotating diffusion plate 109).

In the structured illumination microscope apparatus 1 of the present embodiment, the laser light source (101, 102) includes a first laser light source (101) that emits the first laser light, and the first laser light has a different wavelength. There is a second laser light source (102) that emits a second laser beam, and the imaging unit (531, 532) has a first observation beam (sample 5) emitted from the sample (specimen 5) in response to the first laser beam. A first imaging device (531) that receives the first fluorescence) and a second imaging device (352) that receives the second observation light (second fluorescence) emitted from the sample (specimen 5) in response to the second laser light. ) and there is, the first image sensor (351) one imaging exposure time and (Te _1), and once the imaging exposure time of the second image sensor (352) (Te _2), the control It can be adjusted individually by the unit (control device 39).

Therefore, the structured illumination microscope apparatus 1 of the present embodiment can capture two images having different wavelengths in parallel, and can cope with the difference in brightness between the two images.

Further, the structured illumination microscope apparatus 1 of the present embodiment is structured to illuminate the sample (specimen 5) with interference fringes by laser light, and is a series obtained by the imaging unit (imaging elements 351 and 352). The modulated image is used for a demodulation operation for generating a super-resolution image.

Therefore, if the luminance unevenness between frames is suppressed as described above, the accuracy of the demodulation operation can be improved and the super-resolution effect can be enhanced.

[Modification of First Embodiment]
In addition, although the control apparatus 39 of 1st Embodiment adjusted the repetition period (Tf) of imaging according to the change period (Tr) of the spreading | diffusion pattern, on the contrary, it diffuses according to the repetition period (Tf) of imaging. The pattern change period (Tr) may be adjusted.

Specifically, the shortest controller 39, represented by the sum of the longer exposure time and the charge readout time of the exposure time Te ₂ of the exposure time Te ₁ a second image sensor 352 of the first image sensor 351 The rotation period Tr is set according to the frame period Tf so that the frame period Tf matches the frame period Tf of the first image sensor 351 and the second image sensor 352 and the frame period Tf is an integral multiple of the rotation period Tr. . Therefore, for example, when the frame period Tf is 11 ms, the rotation period Tr is set to 5.5 ms or the like. When this rotation cycle is converted into a rotation speed, it is 10909 rpm.

Thus, adjusting the rotation period Tr according to the frame period Tf makes it possible to make the frame period Tf coincide with the shortest frame period Ts, which is suitable for observing the specimen 5 having a fast movement.

However, in order to adjust the rotation period Tr, it is necessary to adjust the rotation speed of the motor of the rotation mechanism 109A, and it takes time until the rotation speed stabilizes at the set speed. A certain preparation period is required.

In the structured illumination microscope apparatus 1 of the first embodiment, a rotatable light diffusion plate is used as the light diffusion element. However, a light diffusion element having a variable diffusion pattern (for example, a liquid crystal element) is used. The period of the diffusion pattern may be changed by electrical control.

In the structured illumination microscope apparatus 1 of the first embodiment, each of the excitation filter 28, the first dichroic mirror 7, and the barrier filter 31 is a dual-band compatible type (a type that acts on two types of laser light or two types of fluorescence). However, the excitation filter 28 and the barrier filter 31 may not be a dual-band compatible type. The excitation filter 28 can be omitted, and the arrangement place of the barrier filter 31 is not the front stage of the second dichroic mirror 35 but the rear stage (immediately before each of the first image sensor 351 and the second image sensor 352). It does not matter.

Further, in the structured illumination microscope apparatus 1 of the first embodiment, two modulated images having different wavelengths are imaged in parallel by the two image sensors (the first image sensor 351 and the second image sensor 352), but the wavelengths are different. Two modulated images may be sequentially captured by two image sensors (first image sensor 351 and second image sensor 352). In that case, the controller 39 may be a frame period Tf ₁ of the first image sensor 351 and a frame period Tf ₂ of the second imaging element 352 is set individually.

For example, the control device 39 calculates the shortest frame period Ts ₁ composed of the sum of the exposure time Te ₁ of the first image sensor 351 and the charge readout time To, and is an integral multiple of the rotation period Tr (Tr, 2Tr, 3Tr, 4Tr). , ... of), there is the shortest frame period Ts ₁ or more, the value closest to the shortest frame period Ts _1, may be set frame period Tf ₁ of the first imaging element 351. Further, the control device 39 calculates the shortest frame period Ts ₂ that is the sum of the exposure time Te ₂ of the second image sensor 352 and the charge readout time To, and is an integral multiple of the rotation period Tr (Tr, 2Tr, 3Tr, 4Tr). , ... of), there is the shortest frame period Ts ₂ or more, the value closest to the shortest frame period Ts _2, may be set frame period Tf ₂ of the second imaging element 351.

Further, in the structured illumination microscope apparatus 1 of the first embodiment, the number of switching of the number of light source wavelengths is 2, but it may be 3 or more. In that case, the number of image sensors may be three or more.

Further, in the structured illumination microscope apparatus 1 of the first embodiment, the number of imaging elements is plural in order to capture a plurality of modulated images having different wavelengths in parallel, but a plurality of modulated images are sequentially captured. In this case, the number of image sensors may be 1.

In this case, the second dichroic mirror 35 and the image sensor 351 are omitted, and a plurality of units (cubes) including the excitation filter 28, the first dichroic mirror 7, and the barrier filter 31 are prepared, and units arranged in the optical path are prepared. The image sensor 352 may be repeatedly driven while switching between the plurality of units.

Further, in the structured illumination microscope 1 of the first embodiment, a rotatable diffraction grating is used to switch the direction of the interference fringes, but a diffraction grating (which can switch the grating direction in accordance with an electrical signal ( A spatial light modulation element) may be used.

In the first embodiment, in order to switch the direction of the interference fringes at a pitch of 120 °, the one-way diffraction grating 13 that can be rotated at a pitch of 120 °, and the (non-rotated) zero-order light cut mask 14 The three-way diffraction grating 131 ′ (FIG. 3A) having a periodic structure in three directions that differ by 120 ° (FIG. 3A) and a light beam selection that can be rotated at a pitch of 120 ° A combination with the member 18 (FIG. 3B) may be used.

As shown in FIG. 3A, the diffraction grating 131 ′ has a periodic structure in each of the 0 ° direction V ₀ , the 120 ° direction V ₁₂₀ , and the 240 ° direction V ₂₄₀ , and the period in each direction. The period (grating pitch) of the structure (grating line) is common. 0 ° plurality of grid lines arranged toward the direction V ₀ which is a grid line for branching the incident light beam in the direction V ₀ which 0 °, the plurality of grid lines arranged toward the direction V ₁₂₀ of the 120 ° a grid line for branching the incident light beam toward the direction _{V 120} of 120 °, 240 ° plurality of grid lines arranged toward the direction _{V 240} of the for branching the incident light beam toward the direction _{V 240} of the 240 ° It is a grid line. Therefore, the diffraction grating 131 'includes a ± 1-order diffracted light beam branching toward the direction V _0, and ± 1-order diffracted light beam branching toward the direction V _120, and ± 1-order diffracted light beam branching toward the direction V _240, simultaneously generates can do.

As shown in FIG. 3B, the opening pattern of the light beam selecting member 18 includes the first opening 19 and the first opening portion 19 that allow only one pair of ± 1st order diffracted light beams among these 3 pairs of ± 1st order diffracted light beams. 2 openings 20. The light beam selecting member 18 is rotated at a pitch of 120 ° around the optical axis in conjunction with the polarizing plate 23 described above, and the direction of the interference fringe is set between three directions V ₀ , V ₁₂₀ , and V ₂₄₀ . Switch. Incidentally, the length of each of the first opening 19 and the second opening 20 around the optical axis is set such that a linearly polarized ± first-order diffracted light beam can pass through. That is, the shape of each of the first opening 19 and the second opening 20 is a shape close to a partial ring zone.

Further, a plurality of notches 21 (six in the example shown in FIG. 3B) are formed on the outer peripheral portion of the light beam selecting member 18 as shown in FIG. A sensor 22 for detecting the presence or absence of these notches 21 is arranged at a position directly opposite the locus of the notches 21 in the vicinity of. The sensor 22 is composed of a photo interrupter or the like. When the notch 21 is directly facing the sensor 22, the value of the detection signal of the sensor 22 is large, and when the notch 21 is not true to the sensor 22, the sensor 22 The value of the detection signal 22 becomes small. Therefore, the detection signal of the sensor 22 can be used as a signal (angle signal) indicating the rotational position of the light beam selection member 18.

The translation direction of the diffraction grating 131 ′ (FIG. 3A) for shifting the phase of the interference fringe is 0 ° or 120 ° in the direction of the interference fringe (= the branch direction of the selected ± order diffracted light beam). , 240 degrees, it is set in a predetermined direction so that the phase of the interference fringes can be shifted. However, in this case, since the relationship between the translation amount and the phase shift amount differs between the three directions _{_{_{V 0, V 120, V 240}}} , such that the amount of phase shift is equal regardless of the direction of the interference fringes, translation Is set for each direction of the interference fringes.

Moreover, in the structured illumination microscope 1 of the first embodiment, a diffraction grating capable of translational movement is used to shift the phase of the interference fringes, but diffraction that can shift the grating position in accordance with the electrical signal. A grating (spatial light modulation element) or the like may be used.

In the first embodiment, the number of the first modulated images in which the direction of the interference fringes is the same and only the phase of the interference fringes is different is described as “at least three”. However, for example, three or five Is set. In that case, the number of the series of first modulated images used for generating the first super-resolution image is nine or fifteen.

Similarly, in the first embodiment, the number of the second modulated images in the series in which the direction of the interference fringe is the same and only the phase of the interference fringe is different is described as “at least three”. However, for example, three or five And so on. In that case, the number of the series of second modulated images used for generating the second super-resolution image is nine or fifteen.

In the first embodiment, the case where the structured illumination microscope apparatus 1 is used as a two-dimensional structured illumination microscope apparatus has been described. However, the structured illumination microscope apparatus 1 is replaced with a three-dimensional structured illumination microscope apparatus (3D-SIM). : 3D-Structured (Illumination (Microscopy)).

In that case, the 0th-order light cut mask 14 or the light beam selection member 18 is further provided with an opening for passing the 0th-order diffracted light beam. The opening is formed in the vicinity of the optical axis, and the shape of the opening is, for example, a circle. According to the 0th-order light cut mask 14 or the light beam selection member 18 as described above, not only the ± 1st-order diffracted light beam but also the 0th-order diffracted light beam can contribute to the interference fringes.

Thus, the interference fringes generated by the interference of the three diffracted light beams (three-beam interference) are spatially modulated not only in the surface direction of the sample 5 but also in the depth direction of the sample 5. Therefore, according to this interference fringe, a three-dimensional super-resolution image of the sample 5 can be generated.

In the first embodiment, when the structured illumination microscope apparatus 1 is used as a two-dimensional structured illumination microscope apparatus, a combination of a + 1st order diffracted light beam and a −1st order diffracted light beam is used as a diffracted light beam that contributes to interference fringes. Of course, other combinations may be used.

In the first embodiment, when the structured illumination microscope apparatus 1 is used as a three-dimensional structured illumination microscope apparatus, the + 1st order diffracted light beam, the −1st order diffracted light beam, and the 0th order are used as the diffracted light beams contributing to the interference fringes. Although the combination with the folding light beam is used, it goes without saying that other combinations may be used.

In the first embodiment, a biological specimen has been described as an example of an object to be observed (sample). However, the present invention is not limited to this, and the present invention is also applicable when an industrial product part is used as an object to be observed (sample). The invention is applicable.

DESCRIPTION OF SYMBOLS 1 ... Structured illumination microscope apparatus, 100 ... Laser unit, 109 ... Rotating diffuser plate, 109A ... Rotating mechanism, 109B ... Sensor, 11 ... Optical fiber, 10 ... Illumination optical system, 30 ... Imaging optical system, 35 ... Imaging element , 39 ... control device, 40 ... image storage / arithmetic device, 45 ... image display device, 12 ... collector lens, 23 ... polarizing plate, 13 ... one-way diffraction grating, 14 ... zero-order light cut mask, 131 '... three directions Diffraction grating, 18 ... Light flux selection member, 16 ... Condensing lens, 25 ... Lens, 26 ... Field stop, 27 ... Field lens, 28 ... Excitation filter, 7 ... Dichroic mirror, 6 ... Objective lens, 5 ... Sample

Claims

An illumination optical system that illuminates the sample with laser light from a laser light source;
An observation device including an imaging unit that repeatedly images the sample illuminated by the illumination optical system,
The illumination optical system includes a light diffusing element,
A control unit that periodically changes a diffusion pattern of a region irradiated with the laser light in the light diffusing element;
The controller is
An observation apparatus, wherein the setting is performed so that changes in the diffusion pattern during the imaging time of the imaging are the same among the plurality of repeated imaging.
The observation apparatus according to claim 1,
The controller is
The observation apparatus, wherein the setting is performed so that a repetition period of the imaging is an integral multiple of a change period of the diffusion pattern.
The observation apparatus according to claim 2,
The controller is
The observation apparatus characterized by adjusting a repetition period of the imaging according to a change period of the diffusion pattern.
The observation device according to claim 3,
The controller is
An observation apparatus that adjusts a drive signal to the imaging unit in order to adjust a repetition period of the imaging.
The observation apparatus according to claim 2,
The controller is
An observation apparatus, wherein a change cycle of the diffusion pattern is adjusted according to a repetition cycle of the imaging.
In the observation apparatus according to any one of claims 1 to 5,
The light diffusing element is a rotatable light diffusing plate,
The observation device is characterized in that the light diffusion plate is rotationally controlled by the control unit.
In the observation device according to any one of claims 1 to 6,
The light diffusing element is a light diffusing element having a variable diffusion pattern,
The diffusion device is controlled by the control unit.
The observation apparatus according to any one of claims 1 to 7,
A multi-mode optical fiber that relays the laser light between the laser light source and the illumination optical system;
The light diffusion element is disposed between the laser light source and the optical fiber.
The observation apparatus according to any one of claims 1 to 8,
The laser light source is
A first laser light source that emits a first laser light, and a second laser light source that emits a second laser light having a wavelength different from that of the first laser light,
The imaging unit
A first image sensor that receives first observation light emitted from the sample in response to the first laser light, and a second image sensor that receives second observation light emitted from the sample in response to the second laser light. Including
One observation time of the 1st image sensor and one exposure time of the 2nd image sensor can be adjusted individually by the control part. An observation device characterized by things.
In the observation apparatus according to any one of claims 1 to 9,
The illumination optical system includes:
An observation apparatus, characterized by being a structured illumination optical system that illuminates the sample with interference fringes generated by the laser light.