WO2020179039A1 - Microscope, et procédé d'observation - Google Patents

Microscope, et procédé d'observation Download PDF

Info

Publication number
WO2020179039A1
WO2020179039A1 PCT/JP2019/008975 JP2019008975W WO2020179039A1 WO 2020179039 A1 WO2020179039 A1 WO 2020179039A1 JP 2019008975 W JP2019008975 W JP 2019008975W WO 2020179039 A1 WO2020179039 A1 WO 2020179039A1
Authority
WO
WIPO (PCT)
Prior art keywords
detection
image
sample
image processing
processing unit
Prior art date
Application number
PCT/JP2019/008975
Other languages
English (en)
Japanese (ja)
Inventor
陽輔 藤掛
Original Assignee
株式会社ニコン
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社ニコン filed Critical 株式会社ニコン
Priority to PCT/JP2019/008975 priority Critical patent/WO2020179039A1/fr
Publication of WO2020179039A1 publication Critical patent/WO2020179039A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/06—Means for illuminating specimens
    • G02B21/08—Condensers
    • G02B21/14—Condensers affording illumination for phase-contrast observation
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements

Definitions

  • the present invention relates to a microscope and an observation method.
  • Non-Patent Document 1 A scanning microscope that detects fluorescence from a sample has been proposed (for example, see Non-Patent Document 1 below).
  • the microscope according to the first aspect is an illumination including a scanning unit that scans a striped illumination, which is a striped illumination generated by light for multiphoton excitation of a fluorescent substance contained in the sample, in a plurality of directions of the sample.
  • An optical system a detection optical system in which fluorescence from the sample generated by multiphoton excitation is incident, a detection device having a plurality of detection units for detecting fluorescence from the sample via the detection optical system, and the plurality of It is provided with an image processing unit that generates an image by using the detection results of at least two of the detection units.
  • the observation method comprises scanning stripe illumination, which is stripe illumination generated by light for multiphoton exciting a fluorescent substance contained in the sample, in a plurality of directions of the sample, and Detecting fluorescence from the sample generated by photon excitation using a plurality of detection units, and generating an image using detection results of at least two detection units of the plurality of detection units. Including.
  • FIG. 1 is a diagram showing a microscope and an optical path of excitation light according to the first embodiment.
  • the microscope 1 will be described as being a scanning multiphoton excitation fluorescence microscope, but the microscope according to the embodiment is not limited to a scanning microscope or a fluorescence microscope.
  • the microscope 1 includes a stage 2, a light source 3, an illumination optical system 4, a detection optical system 5, a detection device 6, an image processing unit 7, and a control unit 8.
  • the microscope 1 operates as follows.
  • Stage 2 holds the sample S to be observed.
  • the sample S is a cell or the like that has been fluorescently stained in advance.
  • the sample S contains a fluorescent substance such as a fluorescent dye.
  • the light source 3 emits excitation light L1 that multiphoton excites the fluorescent substance contained in the sample S.
  • multiphoton excitation may be simply referred to as excitation.
  • the illumination optical system 4 scans the sample S with the interference fringes L2 of the excitation light L1 in a plurality of directions (for example, the X direction and the Y direction).
  • the illumination optical system 4 scans the sample S two-dimensionally with the interference fringe L2.
  • the detection optical system 5 is arranged at a position where the fluorescence L3 (shown later in FIG.
  • the detection device 6 includes a plurality of detection units 6a (later shown in FIG. 4) that detect the fluorescence L3 from the sample S via the detection optical system 5.
  • the image processing unit 7 uses the detection results of the two or more detection units 6a of the detection device 6 to generate an image (for example, a super-resolution image).
  • the control unit 8 controls the light source 3, the drive unit 22, the image processing unit 7, and the like.
  • a control device 8C is configured to include the image processing unit 7 and the control unit 8.
  • the light source 3 includes a light source such as a laser element.
  • the light source 3 generates coherent light in a predetermined wavelength band.
  • the predetermined wavelength band is set to a wavelength band in which the sample S containing a fluorescent substance can be excited by multiple photons. For example, when the excitation wavelength of the sample S is near ⁇ ex and the sample S is excited by n (n is an integer of 2 or more) photon excitation, the predetermined wavelength band is set to a wavelength band near n ⁇ ex. It is desirable to be done.
  • the excitation light L1 is emitted from the light source 3 as, for example, pulsed light.
  • the excitation light L1 emitted from the light source 3 is, for example, linearly polarized light.
  • a light guide member such as an optical fiber 11 is connected to the emission port of the light source 3.
  • the microscope 1 does not have to include the light source 3, and the light source 3 may be provided separately from the microscope 1.
  • the light source 3 may be replaceably (attachable or detachable) provided to the microscope 1.
  • the light source 3 may be externally attached to the microscope 1 during observation with the microscope 1.
  • the illumination optical system 4 is arranged at a position where the excitation light L1 from the light source 3 is incident. Excitation light L1 is incident on the illumination optical system 4 from the light source 3 through the optical fiber 11.
  • the optical fiber 11 may be a part of the illumination optical system 4 or a part of a light source device including the light source 3.
  • the illumination optical system 4 includes a collimator lens 12, a ⁇ /4 wave plate 13, a polarizer 14, a mask 15 (aperture member), a dichroic mirror 16, a relay optical system 17, and a scan in order from the light source 3 side to the sample S side.
  • the unit 18, the lens 19, the lens 20, and the objective lens 21 are provided.
  • the XYZ Cartesian coordinate system shown in FIG. In this XYZ orthogonal coordinate system, the X direction and the Y direction are directions perpendicular to the optical axis 21a of the objective lens 21, respectively. Further, the Z direction is a direction parallel to the optical axis 21a of the objective lens 21. The optical axis 21a of the objective lens 21 coincides with the optical axis 4a of the illumination optical system 4.
  • the same side as the arrow is referred to as a + side (eg, +X side), and the opposite side is referred to as a ⁇ side (eg, ⁇ X side).
  • the directions corresponding to the X direction, the Y direction, and the Z direction are indicated with subscripts.
  • the Xa direction, the Ya direction, and the Za direction in FIG. 1 are directions corresponding to the X direction, the Y direction, and the Z direction in the optical path from the collimator lens 12 to the dichroic mirror 16, respectively.
  • the collimator lens 12 converts the excitation light L1 emitted from the optical fiber 11 into parallel light.
  • the collimator lens 12 is arranged, for example, so that the focal point on the same side as the light source 3 coincides with the light emission port of the optical fiber 11.
  • the lens included in the illumination optical system 4 the focus on the same side as the light source 3 is referred to as the rear focus, and the focus on the same side as the sample S is referred to as the front focus.
  • the ⁇ / 4 wave plate 13 changes the polarization state of the excitation light L1 to circularly polarized light.
  • the polarizer 14 is, for example, a polarizing plate, and has a characteristic of transmitting linearly polarized light having a predetermined polarization direction.
  • the polarizer 14 is arranged such that the light incident on the sample S is S-polarized (linearly polarized in the Y direction).
  • the polarizer 14 is rotatable around the optical axis 12a of the collimator lens 12.
  • the optical axis 12a of the collimator lens 12 coincides with the optical axis 4a of the illumination optical system 4.
  • the mask 15 is a light beam splitting unit that splits the excitation light that excites the fluorescent substance into a plurality of light beams.
  • the illumination optical system 4 scans the sample S with the interference fringe L2 formed by the interference of two or more light beams among the plurality of light beams divided by the mask 15.
  • the mask 15 is arranged at or near the position of the pupil conjugate surface P1 optically conjugate with the pupil surface P0 of the objective lens 21.
  • the vicinity of the pupil conjugate surface optically conjugate with the pupil surface P0 of the objective lens 21 is a range in which the excitation light L1 can be regarded as a parallel ray in the region including the pupil conjugate surface.
  • the excitation light L1 when the excitation light L1 is a Gaussian beam, it can be sufficiently regarded as a parallel light ray within a range within 1/10 of the Rayleigh length from the beam waist position.
  • Rayleigh length of the wavelength of the excitation light L1 lambda when the beam waist radius was w 0, is given by ⁇ w 0 2 / ⁇ .
  • the Rayleigh length is about 3 m
  • the mask 15 is within 300 mm near the pupil conjugate plane P1 optically conjugate with the pupil plane P0 of the objective lens 21. It may be arranged.
  • the mask 15 may be arranged at or near the pupil plane P0.
  • the mask 15 has an opening 15a and an opening 15b through which the excitation light L1 passes. Interference between the excitation light L1a passing through the opening 15a and the excitation light L1b passing through the opening 15b forms an interference fringe L2.
  • the mask 15 is rotatable around the optical axis 12a of the collimator lens 12.
  • the mask 15 is fixed, for example, relative to the polarizer 14 and rotates integrally with the polarizer 14.
  • the mask 15 and the polarizer 14 are rotated by the torque supplied from the drive unit 22.
  • FIG. 2A is a diagram showing a mask according to the first embodiment.
  • the openings 15a and 15b of the mask 15 are arranged symmetrically with respect to the optical axis 12a of the collimator lens 12 (see FIG. 1).
  • the openings 15a and 15b are arranged in the Xa direction.
  • FIG. 2B is a diagram showing the polarizer according to the first embodiment.
  • the transmission axis 14a of the polarizer 14 is parallel to a direction (Ya direction in FIG. 2B) perpendicular to the direction (Xa direction in FIG. 2A) in which the openings 15a and 15b are aligned in the mask 15. Is set.
  • FIG. 2B is a diagram showing a mask according to the first embodiment.
  • the openings 15a and 15b of the mask 15 are arranged symmetrically with respect to the optical axis 12a of the collimator lens 12 (see FIG. 1).
  • the openings 15a and 15b are arranged in the Xa direction
  • FIG. 2C is a diagram showing the pupil plane P0 of the objective lens 21.
  • Reference numerals P0a and P0b are regions where the excitation light L1 is incident, respectively.
  • the parameters shown in FIG. 2C will be referred to later in the description of the image processing unit 7.
  • the dichroic mirror 16 has a property that the excitation light L1 is reflected and the fluorescence L3 (later shown in FIG. 4) from the sample S is transmitted.
  • the excitation light L1 that has passed through the openings 15a and 15b of the mask 15 is reflected by the dichroic mirror 16, the optical path is bent, and enters the relay optical system 17.
  • the relay optical system 17 guides the excitation light L1 from the dichroic mirror 16 to the scanning unit 18.
  • the relay optical system 17 is represented by one lens in the figure, the number of lenses included in the relay optical system 17 is not limited to one. Further, the relay optical system 17 may not be necessary depending on the distance of the optical system and the like. In addition, in each drawing, two or more lenses may be represented by one lens even in a portion other than the relay optical system 17.
  • the scanning unit 18 scans the sample S with the interference fringes L2 formed by the excitation light L1 in the two directions of the X direction and the Y direction.
  • the scanning unit 18 changes the position where the interference fringe L2 is formed by the excitation light L1 in two directions that intersect the optical axis 21a of the objective lens 21.
  • the scanning unit 18 includes a deflection mirror 18a and a deflection mirror 18b.
  • the tilts of the deflection mirror 18a and the deflection mirror 18b with respect to the optical path of the excitation light L1 are variable.
  • the deflection mirror 18a and the deflection mirror 18b are a galvano mirror, a MEMS mirror, a resonant mirror (resonance type mirror), etc., respectively.
  • the deflection mirror 18a and the deflection mirror 18b may be scanners.
  • the scanning unit 18 is controlled by, for example, the control unit 8.
  • the deflection mirror 18a changes the position where the excitation light L1 is incident on the sample S in the X direction.
  • the deflection mirror 18b changes the position on the sample S where the excitation light L1 is incident in the Y direction.
  • the scanning unit 18 is arranged so that, for example, the position conjugate with the pupil plane P0 of the objective lens 21 is the position of the deflection mirror 18a, the position of the deflection mirror 18b, or the position between the deflection mirror 18a and the deflection mirror 18b. To be done.
  • the scanning unit 18 may be configured such that the deflection mirror 18a changes the position where the excitation light L1 is incident on the sample S in the Y direction and the deflection mirror 18b changes in the X direction.
  • the excitation light L1 from the scanning unit 18 is incident on the lens 19.
  • the lens 19 concentrates the excitation light L1 on the sample conjugate surface Sb optically conjugate with the sample surface Sa of the objective lens 21.
  • the sample surface Sa is a surface that is arranged at a position near the front focal point or the front focal point of the objective lens 21 and is perpendicular to the optical axis 21a of the objective lens 21.
  • An interference fringe is formed on the sample conjugate surface Sb due to the interference between the excitation light L1a passing through the opening 15a of the mask 15 and the excitation light L1b passing through the opening 15b.
  • the excitation light L1 that has passed through the sample conjugate surface Sb is incident on the lens 20.
  • the lens 20 converts the excitation light L1 into parallel light.
  • the excitation light L1 that has passed through the lens 20 passes through the pupil surface P0 of the objective lens 21.
  • the objective lens 21 focuses the excitation light L1 on the sample surface Sa.
  • the lens 20 and the objective lens 21 project the interference fringes formed on the sample conjugate surface Sb onto the sample surface Sa. Local interference fringes L2 are formed on the sample surface Sa.
  • the interference fringe L2 includes a bright portion having a relatively high light intensity and a dark portion having a relatively low light intensity.
  • the direction in which the bright part and the dark part are lined up (X direction in FIG. 1) is appropriately referred to as the periodic direction D1 of the interference fringes L2.
  • the periodic direction D1 of the interference fringe L2 corresponds to the direction in which the openings 15a and 15b of the mask 15 are arranged (Xa direction in FIG. 1).
  • the driving unit 22 is included in the fringe direction changing unit that changes the direction of the interference fringe L2.
  • the drive unit 22 (stripe direction changing unit) is a direction in which two or more light beams are arranged in a plane perpendicular to the optical axis 4a of the illumination optical system 4 (for example, the surface on the light emission side of the mask 15) (hereinafter referred to as a light beam division direction). ) Is changed.
  • the above-described light beam splitting direction is, for example, a direction in which the opening 15a and the opening 15b are arranged, and the drive unit 22 changes the light beam splitting direction by rotating the mask 15.
  • the drive unit 22 rotates the polarizer 14 in conjunction with the mask 15 to change the direction of the transmission axis of the polarizer 14 and adjusts the excitation light L1 to be S-polarized and incident on the sample S. That is, the polarizer 14 and the driving unit 22 are included in the polarization adjusting unit that adjusts the polarization state of the excitation light L1 based on the direction of the interference fringes.
  • FIG. 3 is a diagram showing the mask, the polarizer, the interference fringes, and the polarization state of the excitation light according to the first embodiment.
  • the direction in which the openings 15a and 15b of the mask 15 are arranged is the Xa direction.
  • the transmission axis 14a of the polarizer 14 is in the Ya direction perpendicular to the Xa direction.
  • the excitation light L1 in the excitation light L1 (see FIG. 1), the light flux passing through the opening 15a and the light flux passing through the opening 15b are incident on the sample S to generate the interference fringes L2 in the periodic direction D1.
  • the excitation light L1 incident plane is parallel to the XZ plane.
  • the excitation light L1 when incident on the sample S is in the Y direction whose polarization direction D2 is perpendicular to the incident surface, that is, the excitation light L1 is incident on the sample S with S polarization.
  • the direction in which the openings 15a and 15b of the mask 15 are aligned is the direction obtained by rotating the Xa direction counterclockwise by 120°.
  • the transmission axis 14a of the polarizer 14 is a direction in which the Ya direction is rotated counterclockwise by 120 °.
  • the periodic direction of the interference fringes L2 is a direction forming 120 ° with respect to the X direction.
  • the incident surface of the excitation light L1 is a surface obtained by rotating the XZ plane by 120 ° around the Z direction.
  • the excitation light L1 when entering the sample S has a polarization direction D2 perpendicular to the incident surface, that is, the excitation light L1 enters the sample S as S-polarized light.
  • the direction in which the openings 15a and 15b of the mask 15 are aligned is the direction obtained by rotating the Xa direction counterclockwise by 240°.
  • the transmission axis 14a of the polarizer 14 is a direction in which the Ya direction is rotated counterclockwise by 240 °.
  • the periodic direction D1 of the interference fringe L2 is a direction forming 240 ° with respect to the X direction.
  • the incident surface of the excitation light L1 is a surface obtained by rotating the XZ plane by 240 ° around the Z direction.
  • the excitation light L1 when entering the sample S has a polarization direction D2 perpendicular to the incident surface, that is, the excitation light L1 enters the sample S as S-polarized light.
  • the periodic direction of the interference fringe L2 is changed in three ways in increments of 120 °, but the periodic direction of the interference fringe L2 is not limited to this example.
  • the periodic direction of the interference fringes L2 corresponds to a direction in which the resolution can be improved (a direction in which the super-resolution effect can be obtained) in the image generated by the image processing unit 7 described later.
  • the periodic direction of the interference fringe L2 is appropriately set so that a desired super-resolution effect can be obtained.
  • the periodic directions of the interference fringes L2 may be two or one at an angle of 90 ° to each other.
  • the mask 15 may be replaceable according to the magnification and NA (numerical aperture) of the objective lens 21 and the shape of the illumination pupil.
  • FIG. 4 is a diagram showing the microscope and the optical path of fluorescence according to the first embodiment.
  • the detection optical system 5 forms an image of the fluorescence L3 generated in the sample S.
  • the detection optical system 5 includes an objective lens 21, a lens 20, a lens 19, a scanning unit 18, a relay optical system 17, a dichroic mirror 16, an excitation light cut filter 23, and a lens 24 in order from the sample S toward the detection device 6. ..
  • the fluorescence L3 generated in the sample S passes through the objective lens 21, the lens 20, and the lens 19 in this order and enters the scanning unit 18.
  • the fluorescence L3 is descanned by the scanning unit 18 and enters the dichroic mirror 16 through the relay optical system 17.
  • the dichroic mirror 16 has the property of transmitting the fluorescence L3.
  • the fluorescence L3 transmitted through the dichroic mirror 16 is incident on the excitation light cut filter 23.
  • the excitation light cut filter 23 has a property of blocking the excitation light L1 and transmitting the fluorescence L3.
  • the fluorescence L3 that has passed through the excitation light cut filter 23 enters the lens 24.
  • the lens 24 focuses the fluorescence L3 on the detection device 6.
  • the detection device 6 is an image sensor and includes a plurality of detection units 6a arranged two-dimensionally.
  • the plurality of detection units 6a are arranged in two directions in the detection device 6.
  • the plurality of detection units 6a are arranged in two directions, the Xb direction and the Yb direction.
  • Each of the plurality of detection units 6a is a sensor cell including a photoelectric conversion element such as a photodiode, a pixel, or a photodetector.
  • Each of the plurality of detection units 6a can detect the fluorescence L3.
  • the detection unit 6a corresponds to, for example, one pixel, but a detection region (light receiving region) including a plurality of pixels may be used as one detection unit 6a.
  • the microscope 1 scans the interference fringes L2 on the sample surface Sa by the scanning unit 18, and the detection device 6 detects the fluorescence L3. For example, the microscope 1 illuminates the illumination region selected from the sample surface Sa with the interference fringes L2, and the detection device 6 detects the fluorescence L3 from the illumination region. The microscope changes the illumination area by the scanning unit 18 after the detection by the detection device 6 is completed. The microscope 1 acquires a fluorescence intensity distribution (measured value of the detection device 6) in a desired region by repeating a process of detecting fluorescence and a process of changing the illumination region.
  • the fluorescent substance contained in the sample S is multiphoton excited by the excitation light L1.
  • the wavelength band of the excitation light L1 can be extended to the wavelength band of the infrared light, so that the transmittance of the excitation light L1 to the sample S becomes high.
  • the fluorescence L3 to be detected is generated only in the vicinity of the focal plane of the objective lens 21, it is possible to accurately obtain image data on the sample surface Sa set in an arbitrary portion including the inside of the sample S. it can.
  • the image processing unit 7 generates an image based on the detection result of the detection device 6 obtained as described above.
  • the processing executed by the image processing unit 7 will be described.
  • the coordinate system is described as a vector as appropriate.
  • scanning coordinates coordinates of the scanning destination of the scanning unit 18
  • r s (x s , y s )
  • wave number coordinates coordinates after Fourier transform at r s
  • k s (k xs , k ys )
  • the wave number may be referred to as spatial frequency or frequency.
  • the magnification of the optical system is assumed to be 1 for convenience of explanation, but any magnification may be used.
  • the numerical aperture of the optical system including the objective lens 21 is NA
  • the illumination light wavelength is ⁇ ex
  • the wavelength of the fluorescence L3 is ⁇ em
  • the pupil radius k NA em of the objective lens 21 is expressed by the following formula (1A) and formula (1B).
  • the coordinates of the pupil position may be represented by wave number coordinates.
  • Each of k NA ex and k NA em indicates the value of the pupil radius when the pupil is expressed in wave number coordinates.
  • the pupil plane P0 is represented by a wave number coordinate space (frequency space).
  • the area inside the circle drawn by the dotted line shown in FIG. 2C is the pupil of the objective lens 21, and k NA ex is the pupil radius of the objective lens 21.
  • the region P0a and the region P0b on which the excitation light L1 is incident are assumed to be circular, but are not limited to circular.
  • the radius of each of the region P0a and the region P0b is ⁇ k NA ex .
  • ⁇ is the ratio of the radius of the region P0a or the region P0b to the pupil radius of the objective lens 21.
  • the distance from the optical axis 21a of the objective lens 21 to the center of the region P0a is (1- ⁇ ) k NA ex .
  • the distance between the center of the region P0a and the center of the region P0b is, for example, 2(1- ⁇ )k NA ex , but is not limited to this value.
  • the electric field intensity ill(r) of the excitation light on the sample surface Sa is expressed by the following equation (2).
  • k 0 2(1- ⁇ )k NA ex
  • PSF ill (r) is a point spread function when the numerical aperture of the optical system is ⁇ NA.
  • the interval of the interference fringes will be appropriately referred to as the fringe interval or the period of the interference fringes.
  • the fluorescent substance contained in the sample S is multiphoton excited by the excitation light L1, and the excited fluorescent substance emits the fluorescence L3.
  • the detection device 6 receives the fluorescence L3 and captures an image of the fluorescent substance formed by the detection optical system 5.
  • the detection device 6 captures an image of a fluorescent substance and acquires image data.
  • the size of the detection unit 6a of the detection device 6 is smaller than the size corresponding to the cycle of the interference fringes L2 in the detection device 6 (the length on the detection device 6 corresponding to one cycle). Is small enough.
  • the size of the detection unit 6a is set to about ⁇ em /4NA.
  • the distribution of the fluorescent substance in the sample S is represented by Obj(r)
  • the image data obtained by the detection device 6 when detecting the fluorescence generated by n-photon excitation is represented by I(r,r s ).
  • I(r,r s ) is represented by the following equation (3).
  • R in equation (3) is a convolution for r.
  • the PSF det (r) is a detection PSF determined by the detection optical system 5 including the objective lens 21 and the fluorescence wavelength ⁇ em .
  • the term of the excitation light intensity ill (r) to the nth power appears in the equation (3), reflecting that the fluorescence intensity generated by the n photon excitation is proportional to the nth power of the excitation light intensity.
  • FIG. 5 is a diagram showing effective PSF in each detection unit of the detection device according to the first embodiment.
  • the horizontal axis is the Xb direction of the detection device 6.
  • the effective PSF (solid line) of each detection unit 6a is shown in one graph for the three detection units 6a whose coordinates in the Xb direction are different from each other.
  • the graph in the center of FIG. 5A shows the distribution Q1a (solid line) corresponding to the effective PSF of the detection unit 6a arranged at the position X1a.
  • the graph on the left side of FIG. 5A shows the distribution Q1b (solid line) corresponding to the effective PSF of the detection unit 6a arranged at the position X1b.
  • the graph on the right side of FIG. 5A shows the distribution Q1c (solid line) corresponding to the effective PSF of the detection unit 6a arranged at the position X1c.
  • Reference numeral Q2 corresponding to the dotted line in FIG. 5 is a distribution corresponding to the square of the intensity distribution of the interference fringe L2 shown in FIG.
  • the distribution Q2 corresponds to the electric field intensity ill 2 (r) of the excitation light on the sample surface Sa (the square of the above equation (3)).
  • the case of two-photon excitation is shown as an example, but in the case of n-photon excitation, Q2 has a distribution corresponding to the n-th power of the electric field intensity ill (r) of the interference fringe L2.
  • the position where the intensity of the interference fringe L2 is maximum, that is, the peak positions X2a, X2b, and X2c of the distribution Q2 can be obtained in advance by numerical simulation or the like.
  • Distribution Q2 includes partial distributions Q2a, Q2b, and Q2c.
  • the distribution Q2a is a distribution in the range from the minimum position before the peak position X2a to the next minimum position.
  • the distribution Q2b is a distribution in the range from the minimum position before the peak position X2b to the next minimum position.
  • the distribution Q2c is a distribution in the range from the minimum position before the peak position X2c to the next minimum position.
  • Reference numerals Q3a, Q3b, and Q3c corresponding to the chain double-dashed line in FIG. 5 are distributions corresponding to the detection optical system 5 including the objective lens 21 and the detection PSF determined by the fluorescence wavelength ⁇ em .
  • the detected PSF corresponds to PSF det (r) such as in equation (3).
  • the distribution Q3a shown in the center graph of FIG. 5A is a distribution corresponding to the detection PSF of the detection unit 6a arranged at the position X1a among the plurality of detection units 6a.
  • the distribution Q3a has a maximum (peak) at the position X1a where the detector 6a is arranged (eg, the center position of the light receiving region of the detector 6a).
  • the position X1a is almost the same as the peak position X2a of the distribution Q2a corresponding to the intensity distribution of the interference fringe L2.
  • the distribution Q1a corresponding to the effective PSF is a distribution obtained by multiplying the distribution Q2 corresponding to the intensity distribution of the interference fringes L2 and the distribution Q3a corresponding to the detected PSF of the detection unit 6a arranged at the position X1a.
  • the position X1a of the detection unit 6a that is, the peak position of the detection PSF (the peak position of the distribution Q3a) is deviated from the peak position X2a of the distribution Q2a corresponding to the intensity distribution of the interference fringes L2.
  • the amount is smaller than the predetermined value (for example, the shift amount is almost 0).
  • the effective PSF distribution Q1a has a single maximum (peak).
  • the peak position of the distribution Q1a is almost the same as the position X1a of the detection unit 6a or the peak position X2a of the distribution Q2a corresponding to the intensity distribution of the interference fringes L2.
  • the distribution Q3b shown in the graph on the left side of FIG. 5A is a distribution corresponding to the detection PSF of the detection unit 6a arranged at the position X1b among the plurality of detection units 6a.
  • the distribution Q3b reaches a maximum (peak) at the position X1b where the detection unit 6a is arranged (for example, the center position of the light receiving region of the detection unit 6a).
  • the position X1b is displaced from the peak position X2b of the partial distribution Q2b including the position X1b in the distribution Q2 corresponding to the intensity distribution of the interference fringe L2.
  • the distribution Q1b corresponding to the effective PSF is a distribution obtained by multiplying the distribution Q2 corresponding to the intensity distribution of the interference fringe L2 and the distribution Q3b corresponding to the detected PSF of the detection unit 6a arranged at the position X1b.
  • the position X1b of the detection unit 6a that is, the peak position of the detected PSF (peak position of the distribution Q3b) is deviated from the peak position X2b of the distribution Q2b corresponding to the intensity distribution of the interference fringe L2.
  • the amount is larger than the predetermined value.
  • the distribution Q1b of the effective PSF has two maxima (peaks).
  • the peak of the effective PSF may be divided into two depending on the position of the detector 6a, and such a change in the shape of the effective PSF is called collapse of the shape of the effective PSF.
  • the strongest peak of the effective PSF is called the main lobe, and the other peaks are called the side lobes.
  • the peak position of the main lobe of the effective PSF distribution Q1b deviates from the center position (X2a) of the detection device 6.
  • the position of the main lobe of the effective PSF also shifts due to the relationship between the position (r) of the detection unit 6a of the detection device 6 and the position of the intensity distribution of the interference fringe L2.
  • the displacement of the main lobe of the effective PSF is appropriately referred to as the displacement of the effective PSF.
  • the distribution Q3c shown in the graph on the right side of FIG. 5A is a distribution corresponding to the detection PSF of the detection unit 6a arranged at the position X1c among the plurality of detection units 6a.
  • the distribution Q3c has a maximum (peak) at the position X1c where the detector 6a is arranged (eg, the center position of the light receiving region of the detector 6a).
  • the position X1c deviates from the peak position X2c of the partial distribution Q2c including the position X1c in the distribution Q2 corresponding to the intensity distribution of the interference fringes L2.
  • the distribution Q1c corresponding to the effective PSF is a distribution obtained by multiplying the distribution Q2 corresponding to the intensity distribution of the interference fringes L2 and the distribution Q3c corresponding to the detected PSF of the detection unit 6a arranged at the position X1c.
  • the position X1c of the detection unit 6a that is, the peak position of the detection PSF (the peak position of the distribution Q3c) is deviated from the peak position X2c of the distribution Q2c corresponding to the intensity distribution of the interference fringes L2.
  • the amount is larger than the predetermined value.
  • the distribution Q1b of the effective PSF has two maximums (peaks), the shape of the effective PSF is deformed, and the position of the effective PSF is displaced.
  • the distribution Q3d shown in the graph on the left side of FIG. 5B is a distribution corresponding to the detection unit 6a arranged at the position X1d among the plurality of detection units 6a.
  • the distribution Q3d has a maximum (peak) at the position X1d where the detector 6a is arranged (eg, the center position of the light receiving region of the detector 6a).
  • the position X1d is substantially the same as the peak position X2b of the partial distribution Q2b including the position X1d in the distribution Q2 corresponding to the intensity distribution of the interference fringe L2.
  • the distribution Q1d of the effective PSF has a single maximum (peak), and the peak position of the distribution Q1d is the peak position of the distribution Q2b corresponding to the position X1d of the detection unit 6a or the intensity distribution of the interference fringe L2. It becomes almost the same as X2b. That is, the shape of the effective PSF has not collapsed.
  • the distribution Q3e shown in the graph on the right side of FIG. 5B is a distribution corresponding to the detection unit 6a arranged at the position X1e among the plurality of detection units 6a.
  • the distribution Q3e has a maximum (peak) at the position X1e (for example, the center position of the light receiving region of the detection unit 6a) where the detection unit 6a is arranged.
  • the position X1e is substantially the same as the peak position X2c of the partial distribution Q2c including the position X1e in the distribution Q2 corresponding to the intensity distribution of the interference fringes L2.
  • the distribution Q1e of the effective PSF has a single maximum (peak), and the peak position of the distribution Q1e is the peak position of the distribution Q2c corresponding to the position X1e of the detection unit 6a or the intensity distribution of the interference fringe L2. It becomes almost the same as X2c. That is, the shape of the effective PSF has not collapsed.
  • the image processing unit 7 uses the detection result of the detection unit 6a selected from the plurality of detection units 6a based on the magnification of the detection optical system 5 and the period (strip interval) of the interference fringes L2.
  • the image processing unit 7 selects the detection unit 6a from the plurality of detection units 6a based on the peak position of the interference fringe L2 (eg, peak positions X2a, X2b, X2c in FIG. 5), and the detection result of the selected detection unit 6a.
  • the peak position of the interference fringe L2 corresponds to, for example, the position where the intensity is maximum in the intensity distribution of the interference fringe L2 (eg, the central position of the bright portion).
  • the image processing unit 7 uses, for example, the detection result of the detection unit 6a arranged at the position X1a corresponding to the peak position X2a as the detection result corresponding to the peak position X2a in the center graph of FIG.
  • the peak positions X2a, X2b, and X2c are obtained in advance by numerical simulation or the like, and are stored in the storage unit in advance.
  • the image processing unit 7 selects the detection unit 6a arranged closest to the peak position X2a among the plurality of detection units 6a based on the stored information on the peak position, and displays the detection result of the selected detection unit 6a. To use.
  • the image processing unit 7 may use only the detection result of the one detection unit 6a arranged at the position X1a as the detection result regarding the partial distribution Q2a including one peak in the intensity distribution of the interference fringe L2.
  • the detection result of the detection unit 6a arranged at the position X1a and at least one detection unit 6a around the detection unit 6a may be used.
  • the image processing unit 7 uses, for example, the detection result of the detection unit 6a arranged at the position X1d corresponding to the peak position X2b as the detection result corresponding to the peak position X2b in the graph on the left side of FIG. 5B. .. Based on the magnification of the detection optical system 5 and the cycle of the interference fringes L2, the image processing unit 7 includes a plurality of detectors 6a whose positions match the partial distribution Q2b including one peak in the intensity distribution of the interference fringes L2. Select from the detection unit 6a.
  • the image processing unit 7 may detect the detection unit 6a arranged closest to the peak position X2b (for example, the detection unit arranged at the position X1d) among the plurality of detection units 6a based on the stored information about the peak position. 6a) is selected. The image processing unit 7 uses the detection result of the selected detection unit 6a as the detection result regarding the distribution Q2b.
  • the image processing unit 7 may use only the detection result of the one detection unit 6a arranged at the position X1d as the detection result regarding the partial distribution Q2b including one peak in the intensity distribution of the interference fringe L2.
  • the detection results of the detection unit 6a arranged at the position X1d and at least one detection unit 6a around the detection unit 6a may be used.
  • the effective PSF (PSF eff ) corresponding to the distribution Q1d the collapse of the shape of the effective PSF is reduced by matching the peak position X2b of the distribution Q2b with the position X1d of the detection unit 6a.
  • the image processing unit 7 uses, for example, the detection result of the detection unit 6a arranged at the position X1e corresponding to the peak position X2c as the detection result corresponding to the peak position X2c in the graph on the right side of FIG. 5B. .. Based on the magnification of the detection optical system 5 and the period of the interference fringes L2, the image processing unit 7 includes a plurality of detectors 6a whose positions match the partial distribution Q2c including one peak in the intensity distribution of the interference fringes L2. Select from the detection unit 6a.
  • the image processing unit 7 may detect the detection unit 6a (eg, the detection unit arranged at the position X1e) closest to the peak position X2c among the plurality of detection units 6a based on the stored peak position information. 6a) is selected. The image processing unit 7 uses the detection result of the selected detection unit 6a as the detection result regarding the distribution Q2c.
  • the detection unit 6a eg, the detection unit arranged at the position X1e
  • the image processing unit 7 uses the detection result of the selected detection unit 6a as the detection result regarding the distribution Q2c.
  • the image processing unit 7 may use only the detection result of the one detection unit 6a arranged at the position X1e as the detection result regarding the partial distribution Q2c including one peak in the intensity distribution of the interference fringe L2, You may use the detection result of the detection part 6a arrange
  • the peak position X2c of the distribution Q2c and the position X1e of the detection unit 6a are matched with each other, whereby the collapse of the shape of the effective PSF is reduced.
  • the image processing unit 7 corrects the image position deviation (effective PSF eff peak position or main lobe position deviation) for each detection unit with respect to the detection result of the detection unit 6a selected as described above.
  • the positional deviation of the image for each detection unit can be acquired by theoretical calculation using various design values or from a captured image obtained by photographing a small object such as fluorescent beads with the detection device 6. By correcting such misalignment, the effective PSF of the images obtained by each of the selected detection units 6a can be made substantially the same.
  • the PSF eff of the image obtained in this way can be approximately regarded as equivalent to the PSF eff of the detection unit (detection unit located on the optical axis) at the center position of the detection device 6.
  • the number of fringes (bright part) included in the periodic direction of the interference fringe L2 in the embodiment is larger, the full width at half maximum of PSF eff is narrower and the resolution is better.
  • the number of fringes (bright areas) included in the periodic direction of the interference fringes L2 is preferably 3 or more, for example.
  • the image processing unit 7 generates an image by adding the images having the same PSF eff .
  • the image processing unit 7 can generate the image I SR (r s ) having good resolution and S/N ratio because the PSF eff of the images to be added are almost the same.
  • a wide range of the detection unit 6a for use in the generation of the image I SR (r s) it is possible to increase the signal amount.
  • narrowing the range of the detection unit 6a for use in the generation of the image I SR (r s) it is possible to increase the sectioning capabilities.
  • the effective OTF can be obtained by Fourier-transforming the above equation (6).
  • the microscope according to the embodiment has a cutoff frequency of up to 2k cut conv with the OTF magnified in the direction of the interference fringes L2.
  • the fluorescence wavelength is ⁇
  • the excitation wavelength is n ⁇ ⁇ in the case of n photon excitation.
  • the OTF according to the embodiment is a combination of the OTF of the ordinary fluorescence microscope and the component of the OTF of the ordinary fluorescence microscope shifted in the periodic direction of the interference fringes L2.
  • the microscope 1 improves the resolution of the interference fringes L2 in the periodic direction (X direction in FIG. 1).
  • the microscope 1 can also improve the resolution two-dimensionally by detecting the fluorescence from the sample S by changing the periodic direction of the interference fringes L2.
  • the cycle direction of the interference fringe L2 is changed by 90°.
  • the mask 15 and the polarizer 14 in the state of FIG. 2 are rotated by 90° around the Za direction.
  • the super-resolution image when the periodic direction of the interference fringes L2 is the X direction and I SRx (r s)
  • the super-resolution image when the periodic direction of the interference fringes L2 is the Y direction I SRy (r s)
  • the image processing unit 7 by summing the I SRx (r s) and I SRy (r s), may generate a super-resolution image two-dimensionally resolution is improved.
  • the image processing unit 7 may also generate a super-resolution image by the following processing.
  • the image processing section 7 performs Fourier transform super-resolution image I SRx (r s) and a super-resolution image I SRy the (r s), respectively.
  • the Fourier-transformed super-resolution image I SRx (r s ) is represented by I to SRx (k s ).
  • " ⁇ " In the specification is a tilder in the mathematical formula.
  • the Fourier-transformed super-resolution image I SRy (r s ) is represented by I 1 to SRy (k s ).
  • I ⁇ SRx (k s) as compared to conventional fluorescence microscope, the cutoff frequency is increased with respect to the periodic direction of the interference fringes (X direction).
  • the cutoff frequency is increased with respect to the periodic direction of the interference fringes (Y-direction).
  • the image processing section 7 adds up the I ⁇ SRx (k s) and I ⁇ SRy (k s). This increases the cutoff frequency in the two directions (X direction and Y direction).
  • the added effective OTF shape may be distorted depending on the combination of the directions that change the periodic direction of the interference fringes L2.
  • the image processing unit 7 may apply a frequency filter for correcting the shape of the effective OTF.
  • the illumination optical system 4 changes the periodic direction of the interference fringe L2 into three types of 0°, 120°, and 240°, and the detection device 6 has three periodic directions. May detect the fluorescence L3.
  • the image processing unit 7 may generate a super-resolution image by using three detection results (for example, three images) detected by the detection device 6 in three different cycle directions.
  • the illumination optical system 4 changes the periodic direction of the interference fringe L2 in four or more ways
  • the detection device 6 detects the fluorescence L3 in each of the four or more periodic directions
  • the image processing unit 7 has four ways.
  • a super-resolution image may be generated by using four or more detection results detected by the detection device 6 in the above periodic directions.
  • the method of generating a super-resolution image using a plurality of detection results obtained by changing the cycle direction of the interference fringes a plurality of times described above can also be used in the following embodiments.
  • FIG. 6 is a flowchart showing the observation method according to the embodiment.
  • the control unit 8 of the microscope 1 sets the angles of the scanning mirrors (deflection mirrors 18a and 18b).
  • the illumination optical system 4 of FIG. 1 irradiates the position on the sample determined by the angle of the scanning mirror set in step S1 with the excitation light as interference fringes.
  • the fluorescent substance of the sample is excited by the interference fringes of the excitation light.
  • the detection device 6 of FIG. 4 detects the fluorescence L3 from the sample S via the detection optical system 5.
  • step S4 the control unit 8 determines whether or not to change the angle of the scanning mirror.
  • the control unit 8 determines to execute the angle change of the scanning mirror in step S4 (step S4; Yes). ).
  • step S4; Yes the process returns to step S1 and the control unit 8 sets the angle of the scanning mirror to the next scheduled angle. Then, the processes of steps S2 to S4 are repeated. In this way, the illumination optical system 4 scans the sample S two-dimensionally with the interference fringes of the excitation light L1.
  • step S4 determines not to change the angle of the scanning mirror (step S4; No). ..
  • the image processing unit 7 corrects the positional deviation of the image for each detection unit in step S5.
  • the image processing unit 7 corrects the data obtained from at least one detection unit of the plurality of detection units based on the position of the detection unit. For example, the image processing unit 7 corrects the data obtained from the detection unit selected from the plurality of detection units based on the position of the detection unit.
  • the image processing unit 7 uses the data obtained from the first detection unit (for example, the detection unit arranged at the position X1d in FIG. 5B) of the plurality of detection units as the first detection unit. Correction is performed based on the position (eg, X1d).
  • the image processing unit 7 also generates an image using the detection results of two or more detection units. For example, in step S6, the image processing unit 7 generates an image (for example, a super-resolution image) by adding the images corrected in step S5.
  • the positions of the plurality of detection units 6a of the detection device 6 may be set based on the cycle of the interference fringe L2 so as to match the peak (or maximum or bright point) position of the interference fringe L2.
  • the detection device 6 may be preset so that the spacing between the detection units 6a and the fringe spacing of the interference fringes L2 match.
  • the distance between the detection units 6a is the distance between the center of one detection unit 6a and the center of the detection unit 6a adjacent to the center.
  • the fringe spacing of the interference fringe L2 is the spacing between the centerline of one bright portion and the centerline of the adjacent bright portion in the interference fringe L2.
  • the wave number of the interference fringe L2 is k 0
  • the fringe interval of the interference fringe L2 is 1 / k 0 .
  • the spacing of the detection unit 6a of the detection device 6 is set to be substantially the same as P shown in the following formula (7).
  • the magnification of the detection optical system 5 including the objective lens 21 is set to 1. If the magnification of the detection optical system 5 is M det is changing the spacing between the detectors 6a by a factor fraction, the spacing of the detectors 6a may be set to M det / k 0. Alternatively, by using a part of the detection optical system 5 as a zoom lens, the distance between the detectors 6a can be matched with the period of the interference fringes L2. In this case, it is desirable that the lens 24 capable of changing only the magnification of the detection optical system 5 is a zoom lens. Further, the period of the interference fringes L2 may be adjusted so as to match the intervals of the plurality of detectors 6a of the detection device 6. For example, the period of the interference fringes L2 can be changed by changing the intervals between the openings 15a and 15b of the mask 15.
  • the effective PSFs of the images obtained for each detection unit 6a are aligned by setting the intervals of the detection units 6a and the fringe intervals of the interference fringes L2 to match. Therefore, the image processing unit 7 generates images with high resolution (eg, super-resolution images) while ensuring S/N by adding images based on the detection results of two or more detection units.
  • the fluorescent substance contained in the sample S is multiphoton excited by the excitation light L1.
  • the wavelength band of the excitation light L1 can be extended to the wavelength band of the infrared light, so that the transmittance of the excitation light L1 to the sample S becomes high.
  • the fluorescence L3 to be detected is generated only in the vicinity of the focal plane of the objective lens 21, it is possible to accurately obtain image data on the sample surface Sa set in an arbitrary portion including the inside of the sample S. it can.
  • the microscope 1 scans the interference fringes L2 two-dimensionally by scanning the interference fringes L2 in two directions parallel to the sample surface Sa.
  • the microscope 1 according to the embodiment may scan the interference fringes L2 three-dimensionally by scanning the interference fringes L2 in two directions parallel to the sample surface Sa and one direction perpendicular to the sample surface Sa.
  • the process of scanning the interference fringes L2 in two directions parallel to the sample surface Sa (hereinafter referred to as the two-dimensional process) is the same as the process described in the above embodiment. is there.
  • the microscope 1 can generate, for example, a three-dimensional super-resolution image by changing the position in the Z direction and repeating the two-dimensional processing. Similarly for the embodiments described later, the microscope 1 may scan the interference fringes L2 three-dimensionally.
  • the image processing unit 7 filters the data in the frequency space to generate an image.
  • the image processing unit 7 performs deconvolution on the data obtained from the detection device 6 to generate an image.
  • the image processing unit 7 performs deconvolution and apodization for each detection unit 6a of the detection device 6 as the above-described filtering to generate an image. That is, the image processing unit 7 performs filtering including deconvolution on the data in the frequency space.
  • a series of processes of deconvolution and apodization may be collectively (generally) referred to as deconvolution.
  • FIG. 7 is a diagram showing processing of the image processing unit of the microscope according to the second embodiment.
  • FIG. 7 (A) is the PSF before deconvolution, which is the same as FIG. 5 (A).
  • the distance between the detection units 6a of the detection device 6 does not match the distance between the interference fringes L2.
  • the shape of the effective PSF (solid line) of the image obtained for each detection unit 6a is deformed depending on the position of the detector 6a, as described in the above equation (5).
  • the effective PSF for each detector 6a can be obtained (estimated) by theoretical calculation from the design value or by photographing a small object such as fluorescent beads.
  • the image processing unit 7 uses the thus obtained effective PSF to perform deconvolution so as to correct the shape collapse and the positional deviation of the effective PSF of the image obtained for each detection unit 6a.
  • Figure 7 (B) shows the PSF after deconvolution.
  • reference sign Q4a is an effect obtained by deconvoluting the effective PSF of the detection unit 6a arranged at the distribution Q1a shown in the graph in the center of FIG. 7A, that is, the position X1a.
  • the amount of deviation between the position X1a of the detection unit 6a and the peak position X2a of the distribution Q2a is smaller than a predetermined value, and the distribution Q4a corresponding to the effective PSF after deconvolution corresponds to the effective PSF before deconvolution. It is almost the same as the distribution Q1a.
  • reference sign Q4b is obtained by deconvoluting the distribution Q1b shown in the graph on the left side of FIG. 7A, that is, the effective PSF of the detection unit 6a arranged at the position X1b.
  • reference sign Q4c indicates the distribution Q1c shown in the graph on the right side of FIG. 7(A), that is, the effective PSF of the detection unit 6a arranged at the position X1c is obtained by deconvolution.
  • PSF corresponds to PSF.
  • the effective PSFs of the detectors 6a become substantially the same, as shown in the three graphs of FIG. 7B.
  • the image processing unit 7 uses the result of deconvolution to generate an image.
  • the processing of the image processing unit 7 will be described in more detail.
  • the image processing unit 7 converts at least a part of the detection results of the plurality of detection units 6a into data on the frequency space, and uses the conversion results to generate an image (eg, a super-resolution image).
  • data representing at least a part of the detection results of the plurality of detection units 6a in the frequency space is appropriately referred to as a component of the frequency space.
  • the image processing unit 7 Fourier transforms at least a part of the detection results of the plurality of detection units 6a, and generates an image using the components of the frequency space obtained by the Fourier transform.
  • I 1 to (r, k s ) on the left side of the equation (8) is the Fourier transform of I (r, r s ) with respect to r s .
  • Right side of the OTF eff (r, k s) is, PSF eff (r, r s ) the is obtained by Fourier transform for r s, represents the effective OTF of each detector 6a of the detector 6.
  • the right-hand side of Obj ⁇ (k s) are those Obj a (r s) obtained by Fourier transform on r s.
  • deconvolution methods such as the Wiener filter and Richardson-Lucy method.
  • the process using the Wiener filter will be described as an example, but the image processing unit 7 may execute deconvolution by other processes.
  • the deconvolution of each detection unit by the Wiener filter is represented by the following formula (9).
  • Obj to (r, k s ) are distributions of the fluorescent substance estimated for each detection unit 6a of the detection device 6 (hereinafter, referred to as estimated fluorescent substance distributions).
  • w is a Wiener parameter for suppressing noise.
  • the image processing unit 7 the process shown in the following equation (10), the apodization Obj ⁇ (r, k s), adding the spectrum in the detection section 6a of the detection device 6, the super-resolution image I SR ( to generate a r s).
  • A(k s ) is an apodization function for suppressing the negative value of the image, and multiplying Obj ⁇ (r, k s ) by A(k s ) is called apodization.
  • the functional form of A(k s ) is designed to suppress the negative value of the image by theoretical calculation or simulation.
  • F ks -1 is the inverse Fourier transform relates k s.
  • the image processing unit 7 performs the inverse Fourier transform after adding the spectra of the detection units 6a, but may add the images after performing the inverse Fourier transform. In the processing of Expressions (9) and (10), the image processing unit 7 independently deconvolves each detection unit 6a and then adds the images to each detection unit 6a.
  • the image processing section 7 may collectively deconvolute two or more detection sections 6a as in the following Expression (11).
  • FIG. 8 is a flowchart showing an observation method according to the second embodiment.
  • the processing from step S11 to step S14 is the same as the processing from step S1 to step S4 in FIG. 6, and therefore description thereof will be omitted.
  • the image processing unit 7 Fourier transforms the detection result of each detection unit.
  • the image processing unit 7 executes deconvolution.
  • the image processing unit 7 executes apodization. Apodization may be part of the deconvolution process.
  • the image processing unit 7 adds the images of the detection units 6a using the result of the deconvolution.
  • the image processing unit 7 performs an inverse Fourier transform on the first image (eg, Fourier image) obtained in step S18 to generate a second image (eg, super-resolution image).
  • the image processing unit 7 performs Fourier transform on the detection result of each detection unit and executes deconvolution, thereby aligning the effective PSFs of the images obtained for each detection unit 6a. Therefore, the image processing unit 7 adds the images of the respective detection units 6a using the result of the deconvolution to generate an image with high resolution (eg, super-resolution image) while ensuring S/N. be able to. Further, the fluorescent substance contained in the sample S is multiphoton excited by the excitation light L1. As a result, similar to the first embodiment, the image data on the sample surface Sa set in an arbitrary portion including the inside of the sample S can be acquired with high accuracy.
  • the image processing unit 7 may change the range of the detection unit 6a to be added, as described in the first embodiment. Further, the image processing unit 7 may improve the resolution one-dimensionally or two-dimensionally as described in the first embodiment.
  • FIG. 9 is a diagram showing processing of the image processing unit of the microscope according to the third embodiment. For each part of the microscope, refer to FIG. 1 or FIG. 4 as appropriate.
  • the image processing unit 7 corrects the collapse of the shape of the effective PSF for each detection unit 6a by image processing different from that in the second embodiment.
  • FIG. 9A is a PSF before image processing according to the present embodiment, which is the same as FIG. 5A.
  • the distance between the detection units 6a of the detection device 6 does not match the distance between the interference fringes L2.
  • the shape of the effective PSF (solid line) of the image obtained for each detection unit 6a is deformed depending on the position of the detector 6a, as described in the above equation (5).
  • the image processing unit 7 sets the interference fringes so that the peak position of the partial distribution of intensity distribution of the interference fringes L2 (eg, Q2b shown in the graph on the left side of FIG. 9A) matches the position of the detection unit 6a.
  • the phase of the intensity distribution of L2 is effectively shifted by image processing. This process is appropriately referred to as image processing phase shift processing. Further, the phase shift by the image processing phase shift processing is appropriately referred to as an image processing phase shift.
  • the amount of phase shift by image processing phase shift processing is appropriately referred to as image processing phase shift amount.
  • FIG. 9B shows an effective PSF for each detection unit 6a after the image processing phase shift processing.
  • the distribution Q2f is image processing phase-shifted so that the peak position X2b of the distribution Q2b of FIG. 9A coincides with the position X1b of the detection unit 6a.
  • the peak position X2f of the distribution Q2f substantially coincides with the position X1b of the detection unit 6a.
  • Reference numeral Q1f is a distribution corresponding to the effective PSF obtained from the distribution Q2f in which the phase is subjected to the image processing phase shift and the detected PSF (distribution Q3b) of the detection unit 6a arranged at the position X1b. In the distribution Q1f, the collapse of the shape of the effective PSF is reduced.
  • the distribution Q2g has the phase of the distribution Q2 set to the image processing phase so that the peak position X2c of the distribution Q2c of FIG. 9A matches the position X1c of the detection unit 6a.
  • the peak position X2g of the distribution Q2g substantially coincides with the position X1c of the detection unit 6a.
  • Reference numeral Q1g is a distribution corresponding to the effective PSF obtained from the distribution Q2g in which the phase is subjected to the image processing phase shift and the detected PSF (distribution Q3c) of the detection unit 6a arranged at the position X1c. In the distribution Q1g, the collapse of the shape of the effective PSF is reduced.
  • the shape of the effective PSF (solid line) for each detection unit 6a is corrected so as to be substantially the same.
  • the image processing unit 7 generates an image by using an image for each detection unit 6a having an effective PSF corrected to have substantially the same shape.
  • ⁇ indicates the initial phase of the interference fringe L2.
  • the term of cos(2 ⁇ k 0 ⁇ r s + ⁇ ) in Expression (12) has the same period as the period of the interference fringes formed by the excitation light.
  • the term of cos(4 ⁇ k 0 ⁇ r s +2 ⁇ ) in the equation (12) has a half cycle of the interference fringe cycle. In the case of two-photon excitation, up to the second harmonic component of the period of the interference fringes formed by the excitation light is generated. In the case of n-photon excitation, up to the nth harmonic component of the period of the interference fringes formed by the excitation light is generated.
  • the image processing unit 7 changes the phase of the interference fringe L2 by image processing according to the detector coordinates, and aligns the shapes of the effective PSFs.
  • the microscope 1 acquires the four-dimensional image data I(r,r s ) as described in Expression (3).
  • the image processing unit 7 performs a four-dimensional Fourier transform on I(r,r s ).
  • the four-dimensional data of the frequency space obtained by the Fourier transform is represented by I ⁇ (k, k s ).
  • I 0 to (k, k s ) of the first term on the right side of the equation (13) are referred to as a zero-order component.
  • I 0 to (k, k s ) are defined as the following equation (14).
  • I +1 to (k, k s ) of the second term on the right side of the equation (13) are called +1 order components.
  • I + 1 to (k, k s ) are defined by the following equation (15).
  • I -1 to (k, k s ) of the third term on the right side of the equation (13) are called -1st order components.
  • I -1 to (k, k s ) are defined as the following equation (16).
  • I +2 ⁇ the fourth term on the right-hand side of equation (13) to (k, k s) is called a +2 order component.
  • I +2 to (k, k s ) are defined by the following equation (17).
  • I -2 ⁇ the right side fifth term of formula (13) to (k, k s) is referred to as a minus second-order component.
  • I -2 ⁇ (k, k s ) is defined as the following equation (18).
  • the OTF det (k) is a Fourier transform of the PSF det (r) and represents the OTF of the detection optical system 5. Further, OTF 'ill (k) is obtained by Fourier transform of the PSF 2 ill (r). Obj ⁇ (k s) are those that Obj a (r s) obtained by Fourier transformation.
  • the cutoff frequency of OTF det (k) is given by 2k NA em . Further, the cut-off frequency of OTF 'ill (k) is given by the 4 ⁇ k NA ex. Therefore, I 0 to (k, k s ) have values only within the region satisfying the condition of the following equation (19). A region that satisfies the condition of the following Expression (19) is appropriately referred to as a zero-order component region AR1a.
  • I +1 to (k, k s ) have values only within the region satisfying the condition of the following equation (20).
  • a region that satisfies the condition of the following Expression (20) is appropriately referred to as a +1st order component region AR1b.
  • I -1 to (k, k s ) have values only within the region satisfying the condition of the following equation (21).
  • a region that satisfies the condition of the following expression (21) is appropriately called a -1st order component region AR1c.
  • I +2 ⁇ (k, k s ) has a value only in satisfying the region of Formula (22) below.
  • a region that satisfies the condition of the following Expression (22) is appropriately referred to as a +second-order component region AR1d.
  • I -2 to (k, k s ) have values only within the region satisfying the condition of the following equation (23).
  • a region that satisfies the condition of the following Expression (23) is appropriately referred to as a ⁇ second-order component region AR1e.
  • a process of separating appropriately referred to as a component separation.
  • FIG. 10 is a diagram showing a region of the frequency space used for component separation in the third embodiment.
  • the opening of the mask 15 may have a shape other than a circular shape.
  • the range of the area AR1a of the 0th order component, the area AR1b of the +first order component, the area AR1c of the ⁇ 1st order component, the area AR1d of the +second order component, and the area AR1e of the ⁇ second order component is when the opening of mask 15 is circular.
  • the opening of the mask 15 has a shape other than a circular shape, it can be obtained by numerical simulation, theoretical calculation, or the like.
  • FIG. 10A shows each region on the k xs -kys plane.
  • the 0th-order component area AR1a, the +1st-order component area AR1b, the ⁇ 1st-order component area AR1c, the +second-order component area AR1d, and the ⁇ second-order component area AR1e are circular areas.
  • the 0th-order component area AR1a, the +1st-order component area AR1b, the ⁇ 1st-order component area AR1c, the +second-order component area AR1d, and the ⁇ second-order component area AR1e have the same diameter.
  • the diameter of the 0th-order component region AR1a is 8 ⁇ k NA ex .
  • the region AR1a of the 0th-order component is a region centered on the origin.
  • the area AR1b of the +1st order component, the area AR1c of the ⁇ 1st order component, the area AR1d of the +second order component, and the area AR1e of the ⁇ second order component are areas on the axis of which the center is k xs .
  • the distance A between the center and the origin of the region AR1c of the -1st order component is 2 (1- ⁇ ) k NA ex .
  • the +1st-order component region AR1b is a region symmetric with respect to the -1st-order component region AR1c with respect to the 0th-order component region AR1a.
  • the distance B between the center of the area AR1e of the ⁇ second-order component and the origin is twice the distance A, which is 4(1 ⁇ )k NA ex .
  • the + secondary component region AR1d is a region symmetric with respect to the second-order component region AR1e with respect to the zero-order component region AR1a.
  • FIG. 10B shows each region on the k xs ⁇ k x plane.
  • the 0th-order component area AR1a, the +1st-order component area AR1b, the ⁇ 1st-order component area AR1c, the +second-order component area AR1d, and the ⁇ second-order component area AR1e are parallelogram areas, respectively.
  • the image processing unit 7 sets the region of the frequency space in the component separation based on the light intensity distribution of the excitation light in the sample S. For example, the image processing unit 7 sets a plurality of regions that do not overlap each other based on the square of the electric field intensity ill(r) of the excitation light on the sample surface Sa as the light intensity distribution of the excitation light on the sample S.
  • the plurality of regions described above include five or more regions that do not overlap with each other.
  • the plurality of areas include the area AR1a of FIG. 10 as the first area, the area AR1b of FIG. 10 as the second area, the area AR1c of FIG. 10 as the third area, the area AR1d of FIG. 10 as the fourth area, and the area AR1d of FIG.
  • the region AR1e of FIG. 10 is included as the five regions.
  • the image processing unit 7 selects, from the data in the frequency space, data belonging to the first area (area AR1a), data belonging to the second area (area AR1b), data belonging to the third area (area AR1c), and the fourth area ( The components are separated by extracting the data belonging to the area AR1d) and the data belonging to the fifth area (area AR1e).
  • the image processing unit 7 determines the electric field intensity ill(r) of the excitation light on the sample surface Sa as the light intensity distribution of the excitation light on the sample S. It is possible to set a plurality of regions that do not overlap each other based on the nth power and perform component separation.
  • the image processing unit 7 uses I 0 to (k, k s ), I +1 to (k, k s ), I ⁇ 1 to (k, k s ), I +2 to (k, k s ), I ⁇ 2.
  • image data obtained by performing an inverse Fourier transform on I 0 to (k, k s ) will be represented by I 0 (r, r s ).
  • Image data obtained by inverse Fourier transforming I +1 to (k, k s ) is represented by I +1 (r, r s ).
  • Image data obtained by inverse Fourier transforming I ⁇ 1 to (k, k s ) is represented by I ⁇ 1 (r, r s ).
  • Image data obtained by inverse Fourier transforming I +2 to (k, k s ) is represented by I +2 (r, r s ).
  • Image data obtained by inverse Fourier transforming I ⁇ 2 to (k,k s ) is represented by I ⁇ 2 (r,r s ).
  • the image processing unit 7 uses I 0 (r,r s ), I +1 (r,r s ), I ⁇ 1 (r,r s ), I +2 (r,r s ), and I ⁇ 2 (r,r ). s ) is subjected to the calculations shown in the following formulas (24A) to (24E).
  • ⁇ (r) represents the position of the detection unit 6a of the detection device 6, that is, the image processing phase shift amount for each detector coordinate r.
  • the image processing phase shift amount ⁇ (r) is determined, for example, as follows.
  • the image processing unit 7 calculates the positional deviation amount of the signal detected at the detector coordinate r.
  • the image processing unit 7, for example, by simulation in advance, by obtaining the peak position of the obtained function PSF det and (r + r s) by the product of PSF 2 ill (r s), calculates the position deviation amount of the ..
  • the misalignment of the effective PSF may be considered to be proportional to the detector coordinates r, and if the parameter representing the degree of misalignment is ⁇ , the amount of misalignment of the effective PSF can be represented by r / ⁇ .
  • the value of ⁇ may be calculated from the peak position of the function obtained by the product of the PSF det (r + r s) and PSF ill (r s), may be calculated by other numerical simulations.
  • the image processing phase shift amount ⁇ (r) according to the detector coordinates r is determined.
  • Image processing the phase shift of the interference fringe L2 [psi (r) is the peak position of the function obtained by the product of the PSF det (r + r s) and PSF ill (r s), and the peak position of the interference fringes L2 is Determined to match.
  • the value of the initial phase ⁇ may be a value measured in advance using fluorescent beads or a value estimated from an observed image.
  • the image processing unit 7 determines the amount of phase conversion (image processing phase shift amount) based on the light intensity distribution of the excitation light in the sample S.
  • the image processing unit 7 calculates the sum of the calculation results of the above formulas (24A) to (24E) as shown in the following formula (25).
  • PH(r) is a pinhole function defined by the following Expression (27).
  • the signal amount and the sectioning effect can be adjusted.
  • Increasing the value of r PH increases the amount of signal.
  • a smaller value of r PH improves the sectioning ability.
  • the scan interval and the interval of the detection unit 6a of the detection device 6 may be set based on the cutoff frequency and the Nyquist theorem.
  • the scan interval may be set to ⁇ ex /(8nNA) or less in the periodic direction of the interference fringes.
  • the scan interval may be set to ⁇ ex /(4nNA) or less in the direction perpendicular to the periodic direction of the interference fringes.
  • the interval between the detection units 6a of the detection device 6 may be set to ⁇ em /4NA or less.
  • FIG. 11 is a flowchart showing an observation method according to the third embodiment. Since the processing of steps S21 to S24 is the same as the processing of steps S1 to S4 of FIG. 6, the description thereof will be omitted.
  • the image processing unit 7 Fourier transforms at least a part of the detection results of the plurality of detection units 6a.
  • the image processing section 7 performs Fourier transformation of 4-dimensional relative I (r, r s).
  • the image processing unit 7 separates the components in the frequency space.
  • the image processing unit 7 separates the components of the frequency space obtained by the Fourier transform into each region of the frequency space.
  • the image processing unit 7 performs an inverse Fourier transform on the separated components.
  • step S28 the image processing unit 7 executes the image processing phase shift processing.
  • the image processing unit 7 corrects the positional deviation of the effective PSF in step S29.
  • step S30 the image processing unit 7 generates an image (for example, a super-resolution image) by adding the images obtained by correcting the positional deviation in step S29.
  • the image processing unit 7 converts the phase of at least a part of the data obtained by the component separation to generate an image.
  • the image processing unit 7 executes the phase shift process on the data in the real space. That is, the image processing unit 7 uses, as the data obtained by the component separation, the data (the data on the real space) obtained by converting the data on the component separation (the data on the frequency space) into the data on the real space by the inverse Fourier transform. ..
  • the image processing unit 7 may perform the phase shift process in the frequency space on the data in the frequency space in which the components have been separated.
  • the image processing unit 7 executes the phase shift process to align the effective PSFs of the images obtained by the detection units 6a. Therefore, the image processing unit 7 adds the images of the respective detection units 6a obtained by executing the phase shift process to obtain an image with high resolution (eg, super-resolution image) while ensuring S/N. Can be generated.
  • the fluorescent substance contained in the sample S is multiphoton excited by the excitation light L1.
  • the image data on the sample surface Sa set in an arbitrary portion including the inside of the sample S can be acquired with high accuracy.
  • a fourth embodiment will be described.
  • the same components as those in the above-described embodiment will be designated by the same reference numerals, and the description thereof will be omitted or simplified.
  • the image processing unit 7 (see FIG. 4) performs the component separation described in the third embodiment, and then performs deconvolution on the separated components to generate an image.
  • OTF 0 (k, k s ), OTF +1 (k, k s ), OTF ⁇ 1 (k, k s ), OTF +2 (k, k s ), OTF ⁇ 2 (k, k s ) s ) is represented by the following formulas (29A) to (29E).
  • the image processing unit 7 includes OTF 0 (k, k s ), OTF +1 (k, k s ), OTF ⁇ 1 (k, k s ), OTF +2 (k, k s ), OTF ⁇ 2 (k, k s ).
  • Deconvolution is performed using each measured value or estimated value of s 2 ).
  • processing using a Wiener filter will be described as an example of deconvolution, but deconvolution using another method may be used.
  • deconvolution by the Wiener filter is expressed by the following equation (30A) and equation (30B).
  • w is a Wiener parameter for suppressing noise.
  • a (k s ) is the apodization function for suppressing the negative value of the image.
  • F ks -1 is the inverse Fourier transform relates k s.
  • the image processing section 7 generates the super-resolved image I SR (r s) by using a de-convolution results described above.
  • FIG. 12 is a flowchart showing an observation method according to the fourth embodiment.
  • the processing from step S31 to step S34 is similar to the processing from step S1 to step S4 in FIG. 6, and therefore description thereof will be omitted.
  • step S35 the image processing unit 7 Fourier transforms the detection result.
  • step S36 the image processing unit 7 separates the components in the frequency space.
  • step S37 the image processing unit 7 performs deconvolution using the components separated by the process of step S36.
  • the image processing unit 7 performs apodization.
  • the image processing unit 7 performs an inverse Fourier transform on the data obtained by the deconvolution and the apodization.
  • the image processing unit 7 uses the data obtained by the inverse Fourier transform to generate an image (for example, a super-resolution image).
  • the image processing unit 7 executes component separation, deconvolution, and apodization in the frequency space, converts the data obtained by these processes into the data in the real space, and forms the image. To generate.
  • the image processing unit 7 may generate an image without performing the process of correcting the positional deviation by making the effective PSFs of the detection units 6a of the detection devices 6 substantially match.
  • the image processing unit 7 executes the component separation, deconvolution, and apodization in the frequency space to align the effective PSFs of the images obtained by the detection units 6a. Then, the image processing unit 7 converts the data obtained by these processes into data in the real space to generate an image, thereby ensuring an S/N and an image with high resolution (eg, super-resolution). Image) can be generated.
  • the fluorescent substance contained in the sample S is multiphoton excited by the excitation light L1.
  • the image data on the sample surface Sa set in an arbitrary portion including the inside of the sample S can be acquired with high accuracy.
  • the scan interval and the interval of the detection unit 6a of the detection device 6 may be set based on the cutoff frequency and the Nyquist theorem.
  • the scan interval may be set to ⁇ ex / (8 nNA) or less in the periodic direction of the interference fringes.
  • the scan interval may be set to ⁇ ex /(4nNA) or less in the direction perpendicular to the periodic direction of the interference fringes.
  • the interval between the detection units 6a of the detection device 6 may be set to ⁇ em /4NA or less.
  • the image processing unit 7 may set the range to be integrated with respect to the above k to the range of the entire space or to a part of the range of the entire space. Further, the image processing unit 7 uses I 0 to (k, k s ), I +1 to (k, k s ), I ⁇ 1 to (k, k s ), and I +2 to (k, k s ) by Fourier transform. ), I -2 to (k, k s ) may be limited in the range of r.
  • the image processing unit 7 also includes OTF 0 (k, k s ), OTF +1 (k, k s ), OTF ⁇ 1 (k, k s ), OTF +2 (k, k s ), and OTF ⁇ 2 ( k, k s ) may be data obtained in advance by measurement using fluorescent beads or numerical simulation using design values, or obtained from the result of detection of fluorescence from the sample S by the detection device 6. Data (eg, estimates) may be used.
  • FIG. 13 is a diagram showing a microscope 101 according to the fifth embodiment.
  • the detection device 106 includes a line sensor (line detector) in which a plurality of detection units 6a are arranged one-dimensionally. The plurality of detection units 6a are arranged in one direction in the detection device 106. The detection device 106 is arranged at a position optically conjugate with the sample surface Sa.
  • the direction in which the plurality of detection units 6a are arranged (hereinafter referred to as the arrangement direction) is set to the direction corresponding to the periodic direction of the interference fringes L2.
  • the periodic direction of the interference fringes is the X direction
  • the array direction of the plurality of detection units 6a is set to the Xb direction corresponding to the X direction.
  • the microscope 101 includes a ⁇ /2 wavelength plate 30 and an optical path rotating unit 31 that rotates the optical path around the optical axis.
  • the ⁇ /2 wave plate 30 rotates the polarized light passing through the optical path rotating unit 31 based on the rotation angle of the optical path by the optical path rotating unit 31.
  • the optical path rotating unit 31 is arranged in the optical path between the mask 15 and the sample S in the illumination optical system 4.
  • the optical path rotating unit 31 is arranged, for example, at a position where the excitation light L1 becomes substantially parallel light in the optical path of the illumination optical system 4.
  • the optical path rotating unit 31 is arranged, for example, at a position where the excitation light L1 passes through in the illumination optical system 4 and the fluorescence L3 passes through in the detection optical system 5.
  • the optical path rotating unit 31 is arranged in the optical path between the dichroic mirror 16 and the sample S, for example.
  • the ⁇ /2 wave plate 30 may be arranged on the same side as the sample S with respect to the optical path rotating unit 31, or on the side opposite to the sample S with respect to the optical path rotating unit 31 (eg, on the same side as the excitation light source). ).
  • the optical path rotating unit 31 is, for example, an image rotator such as a Dove prism.
  • the optical path rotating unit 31 is provided rotatably around the optical axis of the illumination optical system 4.
  • the optical path rotating unit 31 is driven by the driving unit 32 and rotates.
  • a Dove prism is used as the optical path rotating unit 31
  • the optical path on the light emission side (the sample S side) from the Dove prism is directed to the Dove prism.
  • the optical path on the light incident side (light source 3 side) is rotated by 2 ⁇ ° around the optical axis of the illumination optical system 4.
  • the incident surface of the excitation light L1 on the sample S rotates 2 ⁇ ° around the Z direction
  • the periodic direction of the interference fringes L2 rotates 2 ⁇ ° around the Z direction.
  • the driving unit 32 rotates the optical path rotating unit 31 by 45 ° around the optical axis of the illumination optical system 4.
  • the optical path rotating portion 31 is included in the fringe direction changing portion that changes the direction of the interference fringes with respect to the sample.
  • the ⁇ /2 wave plate 30 is provided so as to be rotatable around the optical axis of the illumination optical system 4.
  • the ⁇ / 2 wave plate 30 rotates in conjunction with the optical path rotating portion 31.
  • the ⁇ / 2 wave plate 30 rotates by an angle determined based on the rotation angle of the optical path rotating portion 31.
  • the ⁇ /2 wavelength plate 30 is fixed (eg, integrated) with the optical path rotating unit 31 and rotates together with the optical path rotating unit 31. In this case, the ⁇ /2 wave plate 30 rotates by the same angle as the rotation angle of the optical path rotating unit 31.
  • the polarization direction of the excitation light L1 is different from the polarization direction on the light incident side (the light source 3 side). Rotate 2 ⁇ ⁇ ° around the optical axis of. As a result, the polarization state of the excitation light L1 when entering the sample S becomes S-polarized.
  • the optical path rotating unit 31 of FIG. 13 is also included in the image rotating unit.
  • the image rotation unit rotates an image of the sample S (eg, an image of fluorescence from the sample S) with respect to the plurality of detection units 6a around the optical axis of the detection optical system 5. That is, the fringe direction changing portion and the image rotating portion include the optical path rotating portion 31 as the same member (optical member).
  • the optical path rotating unit 31 is arranged at a position on the optical path of the illumination optical system 4 where fluorescence enters.
  • the image rotating unit rotates the fluorescence image by the optical path rotating unit 31.
  • the optical path rotation unit 31 adjusts the cycle direction of the interference fringes L2 with respect to the arrangement direction of the plurality of detection units 6a in the detection device 6.
  • the Dove prism When the Dove prism is used as the optical path rotating unit 31, when the Dove prism is rotated by ⁇ ° around the optical axis of the illumination optical system 4, the cycle direction of the interference fringes L2 is rotated by 2 ⁇ ° around the Z direction. Then, the optical path of the fluorescence L3 from the sample S rotates ⁇ 2 ⁇ ° on the light emitting side (detector 106 side) with respect to the light incident side (sample S side) on the Dove prism.
  • the detection device 106 can detect the fluorescence L3 similarly before and after the change of the periodic direction of the interference fringe L2.
  • the detection device 6 since the detection device 6 includes the plurality of detection units 6a arranged two-dimensionally, the detector coordinates have a two-dimensional coordinate system.
  • the detection device 106 since the detection device 106 includes the plurality of detection units 6a arranged one-dimensionally, the detector coordinates have a one-dimensional coordinate system.
  • the processing relating to detector coordinates is image processing corresponding to a one-dimensional coordinate system (eg, Fourier transform for detector coordinates in the third embodiment is one-dimensional Fourier transform).
  • the image processing unit 7 generates an image (for example, a super-resolution image) by one of the processes described in the first to fourth embodiments based on the detection result of the detection device 106.
  • the driving unit 22 rotates the mask 15 to change the cycle direction of the interference fringes L2.
  • the interference fringes L2 are changed by the optical path rotating unit 31 (for example, the Dove prism).
  • the periodic direction of may be changed.
  • the fringe direction changing unit that changes the cycle direction of the interference fringes L2 may be different from both the drive unit 22 and the optical path rotating unit 31.
  • the stage 2 may be rotatably provided around the Z direction, and the rotation may change the direction of the interference fringes L2 with respect to the sample S.
  • the stage 2 is included in the fringe direction changing portion that changes the direction of the interference fringe L2 with respect to the sample S.
  • the microscope 101' shown in FIG. 14 is a modification of the microscope 101 according to the fifth embodiment, and the position at which the optical path rotating unit 31 is provided is different from that of the microscope 101 shown in FIG.
  • the stripe direction changing unit is the same as in the first embodiment and includes the mask 15 and the driving unit 22.
  • the optical path rotating unit 31 serves as both the stripe direction changing unit and the image rotating unit, but in the microscope 101′ shown in FIG. 14, the optical path rotating unit 31 is provided separately from the stripe direction changing unit. ..
  • the optical path rotating unit 31 is arranged at a position that does not overlap the optical path of the illumination optical system 4 in the optical path of the detection optical system 5.
  • the optical path rotating unit 31 is arranged at a position where the excitation light L1 does not enter and the fluorescence L3 enters.
  • the optical path rotating unit 31 is arranged in the optical path between the dichroic mirror 16 and the detection device 106.
  • the driving unit 22 rotates the mask 15 and the polarizer 14 to change the period direction of the interference fringe L2.
  • the drive unit 32 rotates the optical path rotation unit 31 by an angle determined based on the rotation angle of at least one of the mask 15 and the polarizer 14.
  • the drive unit 32 rotates the optical path rotation unit 31 to match the direction of the image projected on the detection device 106 with the direction in which the plurality of detection units 6 a are arranged.
  • the fifth embodiment it is possible to align the effective PSFs of the images obtained for each detection unit 6a by any of the processes described in the first to fourth embodiments. Therefore, it is possible to generate an image (for example, a super-resolution image) with high resolution while ensuring S/N.
  • the fluorescent substance contained in the sample S is multiphoton excited by the excitation light L1.
  • the image data on the sample surface Sa set in an arbitrary portion including the inside of the sample S can be acquired with high accuracy.
  • FIG. 15 is a diagram showing a microscope 201 according to the sixth embodiment.
  • the microscope 201 according to the sixth embodiment includes a light-shielding member 33.
  • the light shielding member 33 is arranged at a position optically conjugate with the sample surface Sa or in the vicinity thereof.
  • the detection device 106 includes a line sensor in which a plurality of detection units 6a are arranged one-dimensionally, as in the fifth embodiment.
  • the detection device 106 is arranged at a position optically conjugate with the sample surface Sa, and the light shielding member 33 is arranged in the vicinity of the detection device 106.
  • the light blocking member 33 may be arranged at a position conjugate with the sample surface Sa or in the vicinity thereof.
  • the light shielding member 33 has an opening 33a through which the fluorescence L3 passes, and shields the fluorescence L3 around the opening 33a.
  • the opening 33a extends in the arrangement direction (Xb direction) of the plurality of detection units 6a in the detection device 106.
  • the opening 33a is, for example, a rectangular slit.
  • the light shielding member 33 is arranged such that the long side of the opening 33a is substantially parallel to the arrangement direction of the plurality of detection units 6a.
  • One or both of the size and the shape of the opening 33a may be variable in the light blocking member 33.
  • the light blocking member 33 may be a mechanical diaphragm having a variable light blocking region or a spatial light modulator (SLM). ..
  • SLM spatial light modulator
  • One or both of the size and the shape of the opening 33a may be fixed.
  • the detection device 106 detects the fluorescence L3 that has passed through the opening 33a of the light-shielding member 33.
  • the processing regarding the detector coordinates can be the image processing corresponding to the one-dimensional coordinate system. Therefore, the image processing described in the first to fourth embodiments can be applied to the sixth embodiment as well as the fifth embodiment.
  • the image processing unit 7 generates an image (for example, a super-resolution image) by one of the processes described in the first to fourth embodiments based on the detection result of the detection device 106.
  • the sixth embodiment it is possible to align the effective PSFs of the images obtained by the detection units 6a by any of the processes described in the first to fourth embodiments. Therefore, it is possible to generate an image (for example, a super-resolution image) with high resolution while ensuring S/N.
  • the fluorescent substance contained in the sample S is multiphoton excited by the excitation light L1.
  • the image data on the sample surface Sa set in an arbitrary portion including the inside of the sample S can be acquired with high accuracy.
  • FIG. 16 is a diagram showing a microscope 301 according to a seventh embodiment.
  • the microscope 301 according to the seventh embodiment includes a drive unit 22 and a drive unit 34.
  • the drive unit 22 is similar to that of the first embodiment.
  • the drive unit 22 rotates the mask 15 to change the cycle direction of the interference fringe L2.
  • the driving unit 22 is included in the fringe direction changing unit that changes the direction of the interference fringe L2 with respect to the sample S.
  • the detection device 106 includes a line sensor in which a plurality of detection units 6a are arranged one-dimensionally, as in the sixth embodiment.
  • the detection device 106 is arranged at a position optically conjugate with the sample surface Sa, and the light shielding member 33 is arranged near the detection device 106.
  • the detector 106 is rotatable around the Zb direction.
  • the drive unit 34 rotates the detection device 106 around the Zb direction.
  • the drive unit 34 rotates the detection device 106 so that the arrangement direction of the detection units 6a in the detection device 106 corresponds to the cycle direction of the interference fringes L2. For example, when the drive unit 22 rotates the mask 15 by 90 °, the periodic direction of the interference fringe L2 changes by 90 °, so that the drive unit 34 rotates the detection device 106 by 90 °.
  • the drive unit 34 rotates the light blocking member 33 so that the relative position between the detection device 106 and the light blocking member 33 is maintained.
  • the light blocking member 33 and the detection device 106 are integrated, and the drive unit 34 integrally rotates the light blocking member 33 and the detection device 106.
  • the detection device 106 detects the fluorescence L3 that has passed through the opening 33a of the light shielding member 33.
  • the processing regarding the detector coordinates can be the image processing corresponding to the one-dimensional coordinate system. Therefore, the image processing described in the first to fourth embodiments can be applied to the case of the seventh embodiment as well as the fifth embodiment.
  • the image processing unit 7 generates an image (for example, a super-resolution image) by one of the processes described in the first to fourth embodiments based on the detection result of the detection device 106.
  • the effective PSFs of the images obtained for each detection unit 6a can be aligned by any of the processes described in the first to fourth embodiments. Therefore, it is possible to generate an image (for example, a super-resolution image) with high resolution while ensuring S/N.
  • the fluorescent substance contained in the sample S is multiphoton excited by the excitation light L1.
  • the image data on the sample surface Sa set in an arbitrary portion including the inside of the sample S can be acquired with high accuracy.
  • the microscope may include the optical path rotating unit 31 shown in FIG. 14 instead of the drive unit 34 that rotates the detection device 106.
  • the microscope need not include the light shielding member 33 as shown in FIG.
  • FIG. 17 is a diagram showing a microscope 401 according to the eighth embodiment.
  • the example in which the illuminated pupil is divided into two poles (two regions) on the pupil surface P0 (see FIG. 2C) has been described, but the illuminated pupil may be in another form.
  • a form in which the illumination pupil is divided into four poles (four regions) on the pupil plane will be described.
  • the illumination optical system 404 according to the eighth embodiment includes a collimator lens 12, a ⁇ /2 wavelength plate 35, a polarization separation element 36, a mirror 37, a mask 38 (aperture member), a mirror 39, on the light emission side of the optical fiber 11. It includes a mask 40 (opening member) and a polarization separating element 41. Further, the illumination optical system 404 according to the eighth embodiment includes the dichroic mirror 16, the relay optical system 17, the scanning unit 18, the lens 19, the lens 20, and the objective lens 21 as in the first embodiment. The optical axis 21a of the objective lens 21 coincides with the optical axis 404a of the illumination optical system 404.
  • the excitation light L1 emitted from the optical fiber 11 is converted into substantially parallel light by the collimator lens 12 and enters the ⁇ /2 wavelength plate 35.
  • the excitation light L1 that has passed through the ⁇ / 2 wave plate 35 includes the excitation light L1c that is linearly polarized light in the first direction and the excitation light L1d that is linearly polarized light in the second direction.
  • the ⁇ /2 wave plate 35 has its optical axis (fast axis, slow axis) direction set such that the light quantity of the pump light L1c and the light quantity of the pump light L1d have a predetermined ratio.
  • the excitation light L1 (excitation light L1c and excitation light L1d) that has passed through the ⁇ / 2 wave plate 35 is incident on the polarization separation element 36.
  • the polarization separation element 36 has a polarization separation film 36a tilted with respect to the optical axis 12a of the collimator lens 12.
  • the polarization separation film 36a has a characteristic that linearly polarized light in the first direction is reflected and linearly polarized light in the second direction is transmitted.
  • the polarization separating element 36 is, for example, a polarization beam splitter prism (PBS prism).
  • the linearly polarized light in the first direction is S polarized light with respect to the polarization separation film 36a.
  • the linearly polarized light in the second direction is P-polarized light with respect to the polarization separation film 36a.
  • the excitation light L1c which is S-polarized light with respect to the polarization separation membrane 36a, is reflected by the polarization separation membrane 36a and is incident on the mask 38 via the mirror 37.
  • the P-polarized excitation light L1d for the polarization separation film 36a passes through the polarization separation film 36a and enters the mask 40 via the mirror 39.
  • the mask 38 and the mask 40 are luminous flux dividing portions that divide the excitation light that excites the fluorescent substance into a plurality of luminous fluxes. The mask 38 and the mask 40 will be described later with reference to FIG.
  • the excitation light L1c that has passed through the mask 38 and the excitation light L1d that has passed through the mask 40 respectively enter the polarization separation element 41.
  • the polarization separation element 41 has a polarization separation membrane 41a inclined with respect to the optical path of the excitation light L1c and the optical path of the excitation light L1d.
  • the polarization separation film 41a has a characteristic that linearly polarized light in the first direction is reflected and linearly polarized light in the second direction is transmitted.
  • the polarization separating element 41 is, for example, a polarization beam splitter prism (PBS prism).
  • the linearly polarized light in the first direction is S polarized light with respect to the polarization separation film 41a.
  • the linearly polarized light in the second direction is P polarized light with respect to the polarization separation film 41a.
  • the excitation light L1c is S-polarized with respect to the polarization separation film 41a, is reflected by the polarization separation film 41a, and enters the dichroic mirror 16.
  • the excitation light L1d is P-polarized with respect to the polarization separation film 41a, passes through the polarization separation film 41a, and enters the dichroic mirror 16.
  • one or both of the polarization separation element 36 and the polarization separation element 41 may not be the PBS prism.
  • One or both of the polarization separating element 36 and the polarization separating element 41 may be a photonic crystal having different reflection and transmission between TE polarized light and TM polarized light.
  • FIG. 18 is a diagram showing the mask and the polarization state of the excitation light according to the eighth embodiment.
  • the Xc direction, the Yc direction, and the Zc direction are the directions corresponding to the X direction, the Y direction, and the Z direction on the sample surface Sa (see FIG. 17), respectively.
  • the mask 38 has an opening 38a and an opening 38b.
  • the openings 38a and 38b are arranged in the Xc direction.
  • the openings 38a and 38b are, for example, circular, but may have shapes other than circular.
  • the mask 38 is arranged at or near the position of a pupil conjugate plane optically conjugate with the pupil plane P0 of the objective lens 21.
  • the Xd direction, the Yd direction, and the Zd direction are the directions corresponding to the X direction, the Y direction, and the Z direction on the sample surface Sa (see FIG. 17), respectively.
  • the mask 40 has an opening 40a and an opening 40b.
  • the openings 40a and 40b are arranged in the Yd direction.
  • the openings 40a and 40b are, for example, circular, but may have shapes other than circular.
  • the mask 40 is arranged at or near the position of the pupil conjugate surface that is optically conjugate with the pupil surface P0 of the objective lens 21.
  • the vicinity of the pupil conjugate plane optically conjugate with the pupil plane P0 of the objective lens 21 is a range in which the excitation light L1 can be regarded as a parallel light ray in a region including the pupil conjugate plane.
  • the excitation light L1 is a Gaussian beam
  • it can be sufficiently regarded as a parallel light ray within a range within 1/10 of the Rayleigh length from the beam waist position.
  • Rayleigh length of the wavelength of the excitation light L1 lambda when the beam waist radius was w 0, is given by ⁇ w 0 2 / ⁇ .
  • the wavelength of the excitation light L1 is 1 ⁇ m and the beam waist radius is 1 mm
  • the Rayleigh length is about 3 m
  • the mask 38 or the mask 40 is near the pupil conjugate plane optically conjugate with the pupil plane P0 of the objective lens 21. It may be arranged within 300 mm.
  • the mask 38 or the mask 40 may be arranged at or near the pupil plane P0.
  • the reference numeral AR2a is a region where the excitation light L1c passing through the opening 38a of the mask 38 is incident on the pupil surface P0 of the objective lens 21.
  • Reference numeral AR2b is a region on the pupil surface P0 where the excitation light L1c passing through the opening 38b of the mask 38 is incident.
  • the arrows in the regions AR2a and AR2b indicate the polarization directions of the incident excitation light L1c.
  • the area AR2a and the area AR2b are arranged in the X direction.
  • the excitation light L1c incident on the region AR2a and the excitation light L1c incident on the region AR2b are linearly polarized light in the Y direction, respectively.
  • the excitation light L1c incident on the region AR2a and the excitation light L1c incident on the region AR2b have the same polarization direction and interfere with each other on the sample surface Sa (see FIG. 17). Due to this interference, interference fringes whose periodic direction is the X direction are formed on the sample surface Sa.
  • the incident surface of the excitation light L1c with respect to the sample surface Sa is the XZ surface, and the excitation light L1c is incident on the sample S with S polarization.
  • reference numeral AR2c is a region on the pupil plane P0 on which the excitation light L1d passing through the opening 40a of the mask 40 is incident.
  • Reference numeral AR2d is a region on the pupil plane P0 on which the excitation light L1d passing through the opening 40b of the mask 40 is incident.
  • the arrows in the regions AR2c and AR2d indicate the polarization direction of the incident excitation light L1d.
  • the area AR2c and the area AR2d are arranged in the Y direction.
  • the excitation light L1d incident on the area AR2c and the excitation light L1d incident on the area AR2d are each linearly polarized in the X direction.
  • the excitation light L1d entering the area AR2c and the excitation light L1d entering the area AR2d have the same polarization direction and interfere with each other on the sample surface Sa (see FIG. 18). Due to this interference, interference fringes whose periodic direction is the Y direction are formed on the sample surface Sa.
  • the incident surface of the excitation light L1d on the sample surface Sa is the YZ plane, and the excitation light L1c is incident on the sample S as S-polarized light.
  • an interference fringe L2 is formed by synthesizing an interference fringe due to the interference of the excitation light L1c and an interference fringe due to the interference of the excitation light L1d. Since the polarization directions of the excitation light L1c and the excitation light L1d are substantially orthogonal to each other, interference between the excitation light L1c and the excitation light L1d is suppressed.
  • the detection device 6 detects the fluorescence L3 from the sample S via the detection optical system 5.
  • the detection device 6 is an image sensor in which a plurality of detection units 6a are arranged in two directions, an Xb direction and a Yb direction.
  • the image processing unit 7 generates an image (for example, a super-resolution image) by any of the processes described in the first to fourth embodiments based on the detection result of the detection device 6.
  • the eighth embodiment reflecting the fact that the illumination pupil is divided into four poles (four regions) on the pupil plane, interference fringes in the X direction and interference fringes in the Y direction simultaneously occur on the sample surface Sa. ..
  • the five components of the 0th-order component, the ⁇ 1st-order component, and the ⁇ 2nd-order component are used in the above equation (13) and the like.
  • Nine components of the second component, the +secondary component in the Y direction, and the ⁇ secondary component in the Y direction may be used.
  • the above equation (28) and the like use five components of the 0th-order component, the ⁇ 1st-order component, and the ⁇ 2nd-order component.
  • the high-order components in the XY directions may be used not only in the case of two-photon excitation but also in the case of multiphoton excitation.
  • the effective PSFs of the images obtained by the detection units 6a can be aligned by any of the processes described in the first to fourth embodiments. Therefore, it is possible to generate an image (for example, a super-resolution image) with high resolution while ensuring S/N.
  • the fluorescent substance contained in the sample S is multiphoton excited by the excitation light L1.
  • the image data on the sample surface Sa set in an arbitrary portion including the inside of the sample S can be acquired with high accuracy.
  • FIG. 19 is a diagram showing a microscope 501 according to the ninth embodiment.
  • the microscope 501 according to the ninth embodiment has the same configuration as the microscope 401 according to the eighth embodiment.
  • the microscope 501 according to the ninth embodiment includes the ⁇ /2 wave plate 30 and the optical path rotating unit 31 described in FIG. 13.
  • the optical path rotating unit 31 is driven by the driving unit 32 and rotates around the optical axis of the illumination optical system 404.
  • the optical path rotating unit 31 rotates, the optical path of the excitation light L1c and the optical path of the excitation light L1d respectively rotate around the optical axis of the illumination optical system 404.
  • the periodic direction of the interference fringe L2 formed on the sample surface Sa rotates around the Z direction.
  • FIG. 20 is a diagram showing the polarization state of the excitation light according to the ninth embodiment.
  • the regions AR4a on which the excitation light L1c is incident on the pupil surface P0 are aligned in the X direction.
  • the area AR4b and the area AR4b on which the excitation light L1d is incident on the pupil plane P0 are arranged in the Y direction.
  • FIG. 20(B) corresponds to a state in which the Dove prism (optical path rotating unit 31 in FIG. 19) and the ⁇ /2 wave plate 30 are rotated by 22.5° from the state in FIG. 20(A).
  • the regions AR4a on which the excitation light L1c is incident on the pupil surface P0 are arranged in a direction rotated by 45 ° from the X direction.
  • the periodic direction of the interference fringes of the excitation light L1c on the sample surface Sa is a direction rotated by 45° from the X direction.
  • the regions AR4b on the pupil plane P0 on which the excitation light L1d is incident are arranged in the direction rotated by 45° from the Y direction.
  • the periodic direction of the interference fringes of the excitation light L1d on the sample surface Sa is a direction rotated by 45° from the Y direction.
  • the detection device 6 detects the fluorescence L3 from the sample S before and after the cycle direction of the interference fringe L2 is changed.
  • the image processing unit 7 starts from the first embodiment based on the detection result of the detection device 6 before the change of the periodic direction of the interference fringes L2 and the detection result of the detection device 6 after the change of the periodic direction of the interference fringes L2.
  • An image eg, super-resolution image
  • the optical path rotating unit 31 may be arranged in the optical path between the dichroic mirror 16 and the detection device 6 as described with reference to FIG.
  • the effective PSF of the image obtained for each detection unit 6a can be made uniform by any of the processes described in the first to fourth embodiments. Therefore, it is possible to generate an image (for example, a super-resolution image) with high resolution while ensuring S/N.
  • the fluorescent substance contained in the sample S is multiphoton excited by the excitation light L1.
  • the image data on the sample surface Sa set in an arbitrary portion including the inside of the sample S can be acquired with high accuracy.
  • the tenth embodiment will be described.
  • the same components as those in the above-described embodiment will be designated by the same reference numerals, and the description thereof will be omitted or simplified.
  • the microscope 501 according to the ninth embodiment changes the periodic direction of the interference fringes L2 by the optical path rotating unit 31, but the fringe direction changing unit that changes the periodic direction of the interference fringes L2 is also different from the optical path rotating unit 31. Good.
  • FIG. 21 is a diagram showing a microscope 601 according to the tenth embodiment.
  • FIG. 22 is a diagram showing a mask according to the tenth embodiment.
  • the microscope 601 according to the tenth embodiment has the same configuration as the microscope 401 according to the eighth embodiment.
  • the microscope 601 according to the tenth embodiment includes a drive unit 45 and a drive unit 46.
  • the mask 38 is rotatable around the optical axis of the illumination optical system 404.
  • the mask 38 is driven by the drive unit 45 to rotate (see FIG. 22A). In FIG. 22(A), the mask 38 is rotated clockwise by 45°.
  • the mask 40 is rotatable around the optical axis of the illumination optical system 404.
  • the mask 40 is driven by the drive unit 46 to rotate (see FIG. 22B).
  • the drive unit 46 rotates the mask 40 by the same angle as the drive unit 45 rotates the mask 38.
  • the mask 40 is rotated clockwise by 45°.
  • the periodic direction of the interference fringes L2 on the sample surface Sa rotates 45° around the Z direction.
  • a ⁇ /2 wavelength plate 48 is provided in the optical path between the polarization separation element 41 and the dichroic mirror 16.
  • the ⁇ /2 wave plate 48 is driven by the drive unit 49 and rotates around the optical axis of the illumination optical system 404.
  • the ⁇ /2 wave plate 48 and the drive unit 49 adjust the excitation light L1c and the excitation light L1d so that they are incident on the sample S as S-polarized light.
  • the detection device 6 detects the fluorescence L3 from the sample S before and after the period direction of the interference fringes L2 is changed, as in the ninth embodiment.
  • the image processing unit 7 starts from the first embodiment based on the detection result of the detection device 6 before the change of the periodic direction of the interference fringe L2 and the detection result of the detection device 6 after the change of the periodic direction of the interference fringe L2.
  • An image eg, super-resolution image is generated by any of the processes described in the fourth embodiment.
  • the effective PSF of the image obtained for each detection unit 6a can be made uniform by any of the processes described in the first to fourth embodiments. Therefore, it is possible to generate an image (for example, a super-resolution image) with high resolution while ensuring S/N.
  • the fluorescent substance contained in the sample S is multiphoton excited by the excitation light L1.
  • the image data on the sample surface Sa set in an arbitrary portion including the inside of the sample S can be acquired with high accuracy.
  • FIG. 23 is a diagram showing a microscope 701 according to the eleventh embodiment.
  • the microscope 701 according to the eleventh embodiment has the same configuration as the microscope 1 according to the first embodiment.
  • the microscope 701 according to the eleventh embodiment includes a relay optical system 47.
  • the relay optical system 47 is a part of the illumination optical system 4 and a part of the detection optical system 5.
  • the relay optical system 47 is arranged in the scanning unit 18 in the optical path between the deflection mirror 18a and the deflection mirror 18b.
  • the deflection mirror 18b is arranged at substantially the same position as the first pupil conjugate plane optically conjugate with the pupil plane P0 of the objective lens 21.
  • the relay optical system 47 is provided so that a second pupil conjugate surface that is optically conjugate with the first pupil conjugate surface is formed between the relay optical system 47 and the relay optical system 17.
  • the deflection mirror 18a is arranged at substantially the same position as the second pupil conjugate plane described above.
  • the detection device 6 detects the fluorescence L3 from the sample S before and after the period direction of the interference fringe L2 is changed, as in the first embodiment.
  • the image processing unit 7 starts from the first embodiment based on the detection result of the detection device 6 before the change of the periodic direction of the interference fringes L2 and the detection result of the detection device 6 after the change of the periodic direction of the interference fringes L2.
  • An image eg, super-resolution image
  • the configuration in which the relay optical system 47 is provided in the microscope 1 according to the first embodiment is illustrated, but the present invention is not limited to this, and the relay optical system 47 is provided in the microscopes according to the second to tenth embodiments. May be provided.
  • the effective PSF of the image obtained for each detection unit 6a can be made uniform by any of the processes described in the first to fourth embodiments. Therefore, it is possible to generate an image (for example, a super-resolution image) with high resolution while ensuring S/N.
  • the fluorescent substance contained in the sample S is multiphoton excited by the excitation light L1.
  • the image data on the sample surface Sa set in an arbitrary portion including the inside of the sample S can be acquired with high accuracy.
  • the scanning unit 18 is not limited to the above-described form.
  • the stage 2 may include a Y stage that moves in the Y direction with respect to the objective lens 21, and the scanning unit 18 may include a Y stage instead of the deflection mirror 18b.
  • the scanning unit 18 may scan the sample S with the excitation light L1 in the X direction by the deflection mirror 18a, and may scan the sample S with the excitation light L1 in the Y direction by moving the Y stage.
  • the deflection mirror 18a may be arranged at substantially the same position as the pupil conjugate plane optically conjugate with the pupil plane P0 of the objective lens 21.
  • the stage 2 may include an X stage that moves in the X direction with respect to the objective lens 21, and the scanning unit 18 may include an X stage instead of the deflection mirror 18a.
  • the scanning unit 18 may scan the sample S with the excitation light L1 in the X direction by moving the X stage, and may scan the sample S with the excitation light L1 in the Y direction by the deflection mirror 18b.
  • the deflection mirror 18b may be arranged at substantially the same position as the pupil conjugate plane optically conjugate with the pupil plane P0 of the objective lens 21.
  • the stage 2 includes an X stage that moves in the X direction with respect to the objective lens 21 and a Y stage that moves in the Y direction with respect to the objective lens 21, and the scanning unit 18 includes the X stage and the Y stage described above. May be included.
  • the scanning unit 18 may scan the sample S with the excitation light L1 in the X direction by moving the X stage, and may scan the sample S with the excitation light L1 in the Y direction by moving the Y stage. ..
  • the scanning direction for scanning the sample S with the interference fringes is two directions, the X direction and the Y direction, and the illumination optical system scans the sample S two-dimensionally with the interference fringes.
  • the scanning direction in which the sample S is scanned with the interference fringes may be the X direction, the Y direction, and the Z direction.
  • the microscope according to each embodiment performs a 2D process of scanning the sample S with interference fringes in the X and Y directions to acquire a 2D image, for example, moving at least one of the objective lens 21 and the stage 2.
  • the sample S may be three-dimensionally scanned with the interference fringes by changing the Z-direction position where the interference fringes are generated and repeating the 2D processing.
  • the microscope according to each embodiment acquires a plurality of 2D images having different positions in the Z direction by three-dimensionally scanning the sample S with interference fringes, and generates a 3D image (eg, Z-stack). Good.
  • the illumination optical system 4 may scan in the X direction and the Y direction, and may scan in the Z direction by moving at least one of the objective lens 21 and the stage 2.
  • the illumination optical system 4 may scan the sample S three-dimensionally with interference fringes.
  • the image processing unit 7 includes, for example, a computer system.
  • the image processing unit 7 reads out the image processing program stored in the storage unit and executes various processes according to the image processing program.
  • This image processing program causes a computer to generate an image based on the detection results of the detection devices 6 and 106.
  • the detection results of the detection devices 6 and 106 are obtained by dividing the light from the light source into a plurality of light beams, and scanning the sample in a plurality of directions by the interference fringes generated by the interference of at least a part of the light beams. Then, the light from the sample is detected by the detection device including the plurality of detection units via the detection optical system on which the light from the sample is incident.
  • the illumination pupil has two poles in FIG. 2, four poles in FIG. 18, but three poles in FIG. 24 (A).
  • Reference numerals AR5a to AR5c are regions on the pupil plane P0 on which the excitation light enters.
  • the first interference fringe between the excitation light incident on the region AR5a and the excitation light incident on the region AR5b the second interference fringe between the excitation light incident on the region AR5b and the excitation light incident on the region AR5c, and the region.
  • Third interference fringes of the excitation light incident on the AR5c and the excitation light incident on the area AR5a are formed.
  • interference fringes that are a combination of the first interference fringes, the second interference fringes, and the third interference fringes are formed.
  • the periodic direction of the first interference fringes, the periodic direction of the second interference fringes, and the periodic direction of the third interference fringes are the respective periodic directions, and the periodic directions are three directions.
  • a super-resolution effect can be obtained.
  • the illumination pupil may have 5 or more poles.
  • the illumination pupil is circular in FIG. 2 and the like, but may have other shapes.
  • symbol AR6 is an area
  • the area AR6 in FIG. 24B is an area surrounded by a circle AR6a which is a part of a circle centered on the optical axis 21a of the objective lens 21 and a straight line AR6b connecting both ends of the arc AR6a.
  • the area AR6 in FIG. 24C is an area surrounded by an arc that is a part of a circle centered on the optical axis 21a of the objective lens 21 and a curved line AR6c that is symmetrical to the arc AR6a.
  • the resolution in the direction in which the interference fringes are not formed is better and the sectioning is better than that in the case of the circular illumination pupil.
  • the resolution in the direction in which the interference fringes are not formed is better and the sectioning is also better than that in the case of the illumination pupil of the shape of FIG. 24(C). ..
  • the resolution in the direction in which the interference fringes are formed is better than that in the case of the illumination pupil having the shape of FIG.
  • the illuminated pupil has a shape in which the illuminated pupil having the shape shown in FIG. 24 (B) has four poles.
  • the illuminated pupil has a shape in which the illuminated pupil having the shape shown in FIG. 24 (C) has four poles.
  • the number of the plurality of regions (the number of poles) on which the excitation light enters is set to an arbitrary number of 2 or more.
  • the shape of one of the plurality of areas on the pupil plane P0 on which the excitation light is incident may be different from the shape of the other areas.
  • the size of one region may be different from the size of the other regions.
  • the plurality of regions in which the excitation light is incident on the pupil surface P0 may be arranged asymmetrically with respect to the optical axis 21a of the objective lens 21.
  • each pole of the illumination pupil can be realized, for example, by designing the shape, dimensions, and arrangement of the opening of the mask 15 shown in FIG.
  • the mask 15 may be a mechanical diaphragm having a variable light blocking area, a spatial light modulator (SLM), or the like.
  • FIG. 26 is a diagram showing a microscope 711 according to the first modification.
  • the illumination optical system 4 includes a collimator lens 50, a ⁇ /2 wavelength plate 51, a lens 52, a diffraction grating 53, a lens 54, and a mask 15 in the order from the optical fiber 11 to the dichroic mirror 16.
  • the collimator lens 50 converts the excitation light L1 from the optical fiber 11 into substantially parallel light.
  • the ⁇ /2 wave plate 51 adjusts the polarization state of the excitation light L1 when entering the sample S.
  • the lens 52 concentrates the excitation light L1 on the diffraction grating 53.
  • the diffraction grating 53 splits the excitation light L1 into a plurality of light beams by diffraction.
  • the diffraction grating 53 is a light beam splitting unit that splits the excitation light that multi-photon excites the fluorescent material into a plurality of light beams.
  • the diffraction grating 53 is arranged at the focal point or a position near the focal point of the lens 52. That is, the diffraction grating 53 is arranged on the surface conjugate with the sample surface Sa or in the vicinity thereof. The vicinity of the focus is within the range of the depth of focus of the light condensed by the lens 52.
  • the diffraction grating 53 may be arranged within 1 mm near the focus of the lens 52.
  • the plurality of light fluxes include 0th-order diffracted light, +1st-order diffracted light, and ⁇ 1st-order diffracted light.
  • the lens 54 converts the 0th-order diffracted light, the +1st-order diffracted light, and the -1st-order diffracted light into substantially parallel lights, respectively.
  • the mask 15 is provided so as to block the 0th-order diffracted light and pass at least a part of the +1st-order diffracted light and at least a part of the ⁇ 1st-order diffracted light.
  • the diffraction grating 53 may be designed so that the 0th-order diffracted light is not generated. Further, a configuration may be adopted in which the mask 15 is not provided. Further, the diffraction grating 53 may be designed so that diffracted light is generated in a plurality of directions. In that case, the mask 15 may be rotationally driven to use only the diffracted light in a specific direction.
  • FIGS. 27 and 28 are diagrams showing a polarization adjusting unit according to a modification, respectively.
  • the optical path of the illumination optical system 4 is bent by the reflecting member such as the dichroic mirror 16 shown in FIG. 1, in FIGS. 27 and 28, the illumination optical system 4 is expanded so that the optical axis 4a becomes a straight line. Indicated.
  • the Z direction is a direction parallel to the optical axis 4a, and the X direction and the Y direction are directions perpendicular to the optical axis 4a, respectively.
  • the illumination optical system 4 includes a ⁇ /4 wavelength plate 61, a mask 15, and a ⁇ /4 wavelength plate 62.
  • the excitation light L1 emitted from the optical fiber 11 is linearly polarized in the X direction and is incident on the ⁇ /4 wavelength plate 61.
  • a polarizer eg, a polarizing plate having a transmission axis in the X direction may be provided in the optical path between the optical fiber 11 and the ⁇ /4 wavelength plate 61.
  • the phase-advancing axis of the ⁇ / 4 wave plate 61 is set in the direction in which the X direction is rotated counterclockwise by 45 ° when viewed from the + Z side.
  • the excitation light L1 that has passed through the ⁇ /4 wavelength plate 61 becomes circularly polarized light and enters the mask 15.
  • the excitation light L1 that has passed through the openings 15a and 15b of the mask 15 is circularly polarized light and is incident on the ⁇ /4 wavelength plate 62.
  • the fast axis of the ⁇ /4 wave plate 62 is set in a direction obtained by rotating the X direction clockwise by 45° when viewed from the +Z side.
  • the excitation light L1 that has passed through the ⁇ /4 wavelength plate 62 becomes linearly polarized light in the X direction and is applied to the sample surface Sa.
  • the mask 15 is rotatably provided around the optical axis 4a as described in the first embodiment.
  • the periodic direction of the interference fringes changes.
  • the openings 15a and 15b of the mask 15 are aligned in the Y direction, and the periodic direction of the interference fringes is the Y direction.
  • the periodic direction of the interference fringes is rotated 90° and becomes the X direction.
  • the ⁇ /4 wave plate 62 is rotatable around the optical axis 4a.
  • the ⁇ /4 wave plate 62 is provided so as to rotate by the same angle as the mask 15.
  • the ⁇ /4 wave plate 62 is integrated with the mask 15 and rotates integrally with the mask 15.
  • the ⁇ / 4 wave plate 62 is rotated by 90 ° and its phase advance axis becomes parallel to the phase advance axis of the ⁇ / 4 wave plate 61.
  • the excitation light L1 that has passed through the ⁇ /4 wavelength plate 62 becomes linearly polarized light in the Y direction.
  • the incident surface of the excitation light L1 with respect to the sample surface Sa is parallel to the periodic direction of the interference fringes, and the excitation light L1 when incident on the sample surface Sa is linearly polarized light perpendicular to the periodic direction of the interference fringes.
  • the sample surface Sa is irradiated with L1 in the S-polarized state.
  • the ⁇ / 4 wave plate 62 is included in the polarization adjusting unit that adjusts the polarization state of the excitation light L1 when it is incident on the sample S.
  • Such a polarization adjusting unit can reduce the loss of the light amount of the excitation light L1 as compared with the aspect described in FIG.
  • the illumination optical system 4 includes a polarizer 65, a mask 15, and a ⁇ /2 wave plate 66.
  • the excitation light L1 emitted from the optical fiber 11 is linearly polarized light in the X direction and is incident on the polarizer 65.
  • the transmission axis of the polarizer 65 is set in the X direction.
  • the excitation light L1 that has passed through the polarizer 65 is linearly polarized in the X direction and is incident on the mask 15.
  • the excitation light L1 that has passed through the openings 15a and 15b of the mask 15 is linearly polarized light in the X direction and is incident on the ⁇ /2 wave plate 66.
  • the fast axis of the ⁇ /2 wave plate 66 is set in a direction obtained by rotating the X direction clockwise by 45° when viewed from the +Z side.
  • the excitation light L1 that has passed through the ⁇ /2 wavelength plate 66 becomes linearly polarized light in the Y direction and is applied to the sample surface Sa.
  • the mask 15 is rotatably provided around the optical axis 4a as described in the first embodiment.
  • the periodic direction of the interference fringes changes.
  • the openings 15a and 15b of the mask 15 are aligned in the X direction, and the periodic direction of the interference fringes is the X direction.
  • the periodic direction of the interference fringes is rotated by 90° and becomes the Y direction.
  • the ⁇ /2 wave plate 66 is rotatable around the optical axis 4a.
  • the ⁇ /2 wave plate 66 is provided so as to rotate by an angle half the rotation angle of the mask 15. For example, when the mask 15 rotates 90°, the ⁇ /2 wave plate 66 rotates 45°.
  • the excitation light L1 that has passed through the ⁇ /2 wave plate 66 becomes linearly polarized light in the X direction.
  • the incident surface of the excitation light L1 with respect to the sample surface Sa is parallel to the periodic direction of the interference fringes, and the excitation light L1 when incident on the sample surface Sa is linearly polarized light perpendicular to the periodic direction of the interference fringes.
  • the sample surface Sa is irradiated with L1 in the S-polarized state.
  • the ⁇ /2 wave plate 66 is included in the polarization adjusting unit that adjusts the polarization state of the excitation light when entering the sample S.
  • a polarization adjusting unit can reduce the loss of the light amount of the excitation light L1 as compared with the aspect described in FIG.
  • the polarization adjusting unit described in the above-described embodiment and modification is configured such that the illumination light becomes linearly polarized light immediately after passing through the polarization adjusting unit, it may not be perfect linearly polarized light.
  • an additional polarizing element such as a ⁇ /2 wavelength plate or a ⁇ /4 wavelength plate may be added to the polarization adjusting unit to correct the change in the polarization state generated in the optical system on the way.
  • the microscope may include an image rotation unit that rotates the image of the sample S around the optical axis of the detection optical system 5.
  • the periodic direction of the interference fringes is rotated, the periodic direction of the interference fringes and the arrangement direction of the detection unit 6a can be matched by rotating the image of the sample S.
  • the microscope according to each embodiment may include a phase modulation unit that can shift the phase of the interference fringe L2 formed by the excitation light L1 to set the interference fringe L2 in a plurality of phase states. ..
  • shifting the phase of the interference fringe L2 formed by the excitation light L1 may be referred to as changing the phase.
  • FIG. 29 is a diagram showing a microscope 721 according to the second modification.
  • the microscope 721 according to the second modification has the same configuration as the microscope 711 according to the first modification.
  • the microscope 721 according to the second modified example includes a phase modulator 55.
  • the phase modulation unit 55 includes a drive unit capable of moving the diffraction grating 53 in a direction perpendicular to the optical axis 4a of the illumination optical system 4.
  • the phase modulation unit 55 shifts the phase of the fringes of the interference fringes L2 by moving the diffraction grating 53 in parallel in the direction perpendicular to the optical axis 4a of the illumination optical system 4 (the direction corresponding to the periodic direction of the interference fringes L2). Can be made.
  • FIG. 30 is a diagram showing a microscope 731 according to the third modification.
  • FIG. 30 shows a modification applied to the microscope 1 according to the first embodiment as a modification of the microscope, it can also be applied to microscopes of other embodiments.
  • the microscope 731 according to the third modification has the same configuration as the microscope 1 according to the first embodiment.
  • a microscope 731 according to the third modified example includes a phase modulator 70.
  • the phase modulator 70 includes a phase modulator 71 and a driver 78.
  • the phase modulation element 71 is arranged between the mask 15 and the dichroic mirror 16.
  • the phase modulation element 71 is formed into a circular plate shape by performing removal processing such as cutting on a predetermined glass plate.
  • the phase modulation element 71 is arranged so that the central axis of the phase modulation element 71 coincides with the optical axis 4 a of the illumination optical system 4.
  • the drive unit 78 rotates the phase modulation element 71 around the optical axis of the illumination optical system 4.
  • the phase modulation element 71 may be arranged in a portion of the optical path of the parallel light flux in the illumination optical system 4 that does not overlap with the optical path of the detection optical system 5.
  • FIG. 31 is a diagram showing details of the phase modulation element 71.
  • the phase modulation element 71 has a first plate part 72, a second plate part 73, a third plate part 74, a fourth plate part 75, and a fifth plate part 76 having different thicknesses.
  • the first plate portion 72 has the thickest first thickness TH1 among the first to fifth plate portions 72 to 76.
  • the first plate portion 72 has, for example, a fan shape with a central angle of 216° when the phase modulation element 71 is viewed in the thickness direction (optical axis direction).
  • the second plate portion 73 has the second thickest second thickness TH2 of the first to fifth plate portions 72 to 76.
  • the second plate portion 73 has, for example, a fan shape having a central angle of 36° when the phase modulation element 71 is viewed in the thickness direction (optical axis direction).
  • the second plate portion 73 is in the circumferential direction of the first plate portion 72 (counterclockwise direction when viewed from the paper surface side of FIG. 31). Placed next to each other.
  • the third plate portion 74 has a third thickness TH3, which is the third largest among the first to fifth plate portions 72 to 76.
  • the third plate portion 74 has, for example, a fan shape having a central angle of 36° when the phase modulation element 71 is viewed in the thickness direction (optical axis direction).
  • the third plate portion 74 is in the circumferential direction of the second plate portion 73 (counterclockwise direction when viewed from the paper surface side of FIG. 31). Placed next to each other.
  • the fourth plate portion 75 has the fourth thickest fourth thickness TH4 of the first to fifth plate portions 72 to 76.
  • the fourth plate portion 75 has, for example, a fan shape having a central angle of 36° when the phase modulation element 71 is viewed in the thickness direction (optical axis direction).
  • the fourth plate portion 75 is in the circumferential direction of the third plate portion 74 (counterclockwise direction when viewed from the paper surface side of FIG. 31). Placed next to each other.
  • the fifth plate portion 76 has a fifth thickness TH5 that is the fifth largest among the first to fifth plate portions 72 to 76.
  • the fifth plate portion 76 has, for example, a fan shape having a central angle of 36° when the phase modulation element 71 is viewed in the thickness direction (optical axis direction).
  • the fifth plate portion 76 is arranged in the circumferential direction of the fourth plate portion 75 (counterclockwise when viewed from the paper surface side of FIG. 31) when the phase modulation element 71 is viewed in the thickness direction (optical axis direction). Placed next to each other.
  • One of the optical surfaces of the phase modulation element 71 is formed into a flat surface having a step due to the first to fifth plate portions 72 to 76.
  • the other optical surface of the phase modulation element 71 is formed in a flat plane shape as a whole.
  • an optical surface having steps formed by the first to fifth plate portions 72 to 76 is arranged so as to face the mask 15.
  • the phase modulation element 71 may be arranged such that the optical surface having the steps formed by the first to fifth plate portions 72 to 76 faces the dichroic mirror 16.
  • 32(A) to 32(E) are diagrams showing the positional relationship between the phase modulation element 71 and the excitation light L1.
  • the first plate portion 72 has a fan-shaped shape with a central angle of 216° which is larger than 180° when the phase modulation element 71 is viewed in the thickness direction (optical axis direction). .. Therefore, as shown in FIG. 32(A), when the phase modulation element 71 is rotationally moved to the predetermined first rotational position by the rotational driving force of the driving unit 78, the excitation light L1a and the mask 15 that have passed through the opening 15a of the mask 15. Both of the excitation lights L1b that have passed through the opening 15b are transmitted through the first plate portion 72. At this time, there is no difference between the optical path length of the pumping light L1a and the optical path length of the pumping light L1b.
  • the excitation light L1a passing through the opening 15a of the mask 15 becomes the first plate portion.
  • the excitation light L1b that passes through 72 and passes through the opening 15b of the mask 15 passes through the second plate portion 73.
  • the second rotation position is, for example, when the phase modulation element 71 is rotated by 36° around the optical axis of the illumination optical system 4 (clockwise when viewed from the paper surface side of FIG. 32) with respect to the first rotation position. The rotation position.
  • the optical path length of the excitation light L1a since the first plate portion 72 has the first thickness TH1 and the second plate portion 73 has the second thickness TH2 thinner than the first thickness TH1, the optical path length of the excitation light L1a. And the optical path length of the excitation light L1b is different. Therefore, the phase of the fringes of the interference fringes obtained when the phase modulation element 71 is rotationally moved to the second rotation position changes with respect to the interference fringes obtained when the phase modulation element 71 is rotationally moved to the first rotation position. To do.
  • the excitation light L1a that has passed through the opening 15a of the mask 15 is the first plate portion.
  • the excitation light L1b that passes through 72 and passes through the opening 15b of the mask 15 passes through the third plate portion 74.
  • the third rotation position is, for example, when the phase modulation element 71 is rotated by 72° around the optical axis of the illumination optical system 4 (clockwise when viewed from the paper surface side of FIG. 32) with respect to the first rotation position. The rotation position.
  • the optical path length of the excitation light L1a is increased.
  • the optical path length of the excitation light L1b is different. Therefore, with respect to the interference fringes obtained when the phase modulation element 71 rotationally moves to the first rotation position, the phase of the interference fringes obtained when the phase modulation element 71 rotationally moves to the third rotation position changes. To do.
  • the excitation light L1a passing through the opening 15a of the mask 15 is converted into the first plate portion.
  • the excitation light L1b that passes through 72 and passes through the opening 15b of the mask 15 passes through the fourth plate portion 75.
  • the fourth rotation position is, for example, when the phase modulation element 71 is rotated by 108° around the optical axis of the illumination optical system 4 (clockwise when viewed from the paper surface side of FIG. 32) with respect to the first rotation position. The rotation position.
  • the optical path length of the excitation light L1a is reduced.
  • the optical path length of the excitation light L1b is different. Therefore, with respect to the interference fringes obtained when the phase modulation element 71 rotationally moves to the first rotation position, the phase of the interference fringes obtained when the phase modulation element 71 rotationally moves to the fourth rotation position changes. To do.
  • the phase modulation element 71 when the phase modulation element 71 is rotationally moved to the predetermined fifth rotational position by the rotational driving force of the driving section 78, the excitation light L1a passing through the opening 15a of the mask 15 is converted into the first plate section.
  • the excitation light L1b that has passed through 72 and passed through the opening 15b of the mask 15 passes through the fifth plate portion 76.
  • the fifth rotation position is, for example, when the phase modulation element 71 is rotated by 144° around the optical axis of the illumination optical system 4 (clockwise when viewed from the paper surface side of FIG. 32) with respect to the first rotation position. The rotation position.
  • the optical path length of the excitation light L1a is increased.
  • the optical path length of the excitation light L1b is different. Therefore, with respect to the interference fringes obtained when the phase modulation element 71 rotationally moves to the first rotation position, the phase of the interference fringes obtained when the phase modulation element 71 rotationally moves to the fifth rotation position changes. To do.
  • the phase of the fringe of the interference fringe L2 can be changed by the phase modulation element 71.
  • Changing the phase of the stripe is equivalent to changing 2 ⁇ k 0 r, which is the argument of the cosine function, to 2 ⁇ k 0 r+ ⁇ without changing PSF ill (r) on the right side of Expression (2). ..
  • ⁇ is the phase difference between the excitation light L1a and the excitation light L2b.
  • PSF ill (r) does not change means that the envelope (envelope) of the intensity distribution of the interference fringe L2 does not change. Therefore, the phase modulation element 71 changes the bright and dark phases of the intensity distribution of the interference fringe L2 without changing the envelope (envelope) of the intensity distribution of the interference fringe L2.
  • the phase of the fringe of the interference fringe L2 can be changed in five steps by rotating and moving the phase modulation element 71 to the first to fifth rotation positions, respectively.
  • the thicknesses of the first to fifth plate portions 72 to 76 are set so that a necessary phase difference of the interference fringes L2 can be obtained.
  • the thickness of the first to fifth plate portions 72 to 76 (first to fifth thicknesses TH1 to TH5) can be set by numerical simulation or the like.
  • the phase difference between the interference fringes when the phase modulation element 71 is rotationally moved to the first rotation position and the interference fringes when the phase modulation element 71 is rotationally moved to the second rotation position is 2 ⁇ /5.
  • the thickness of the second plate portion 73 (second thickness TH2) can be set.
  • the phase difference between the interference fringes when the phase modulation element 71 is rotationally moved to the first rotation position and the interference fringes when the phase modulation element 71 is rotationally moved to the third rotation position is 4 ⁇ /5. It is possible to set the thickness of the third plate portion 74 (third thickness TH3).
  • the phase difference between the interference fringes when the phase modulation element 71 is rotationally moved to the first rotation position and the interference fringes when the phase modulation element 71 is rotationally moved to the fourth rotation position is 6 ⁇ /5. It is possible to set the thickness of the four plate portion 75 (fourth thickness TH4).
  • the phase difference between the interference fringes when the phase modulation element 71 is rotationally moved to the first rotation position and the interference fringes when the phase modulation element 71 is rotationally moved to the fifth rotation position is 8 ⁇ /5. It is possible to set the thickness (fifth thickness TH5) of the five plate portion 76.
  • phase modulation element 71 rotates and moves not only when changing the phase of the fringes of the interference fringe L2 but also when changing the cycle direction of the interference fringe L2.
  • the relative positional relationship between the openings 15a and 15b of the mask 15 and the portions of the phase modulation element 71 (first to fifth plate portions 72 to 76) facing the openings 15a and 15b of the mask 15 becomes constant.
  • the phase modulation element 71 rotationally moves according to the rotation of the mask 15.
  • the central angle of the fan shape in the first plate portion 72 is not limited to the illustrated angle (216°), but the excitation light L1a passing through the opening 15a of the mask 15 and the excitation light L1b passing through the opening 15b of the mask 15. It suffices that both are set to an angle that allows the first plate portion 72 to pass therethrough.
  • the central angle of the fan shape in the second to fifth plate portions 73 to 76 is not limited to the exemplified angle (36°), and the excitation light L1a or the excitation light L1b is not included in the second to fifth plate portions 73 to 76. It suffices if the angle is set so that can be transmitted.
  • the phase modulation element 71 is not limited to the removal processing, and may be formed into a circular plate shape by, for example, molding processing, processing of laminating thin plate materials using a transparent resin material, or the like. Further, as the phase modulation element, a plurality of plate-shaped phase modulation elements having different thicknesses may be used.
  • the drive unit of the phase modulation unit selects any one of the plurality of phase modulation elements and arranges it between the mask 15 and the dichroic mirror 16.
  • a drive unit for example, a slider or a turret that can move a plurality of phase modulation elements in a direction perpendicular to the optical axis of the illumination optical system 4 may be used.
  • the phase modulation unit described above it is possible to perform component separation in a mode different from the component separation of the image processing described in the third embodiment.
  • the image I(r,r s ) obtained by the detection device 6 is represented by the above formula (12).
  • ⁇ indicates the initial phase of the interference fringe L2.
  • the value of the initial phase ⁇ in equation (12) changes, but the changed value of ⁇ can also be called the phase difference. That is, even if the image data is acquired with a predetermined phase difference ⁇ given, the obtained image data is given by the equation (12).
  • the value of the predetermined phase difference ⁇ includes 0.
  • the +second-order component I′ +2 to (r, k s ) of the equation (31) is defined by the following equation (35).
  • Minus second-order component I '-2 ⁇ of formula (31) (r, k s) is defined as the following equation (36).
  • the microscopes 721 and 731 according to the second and third modifications change the phase of the interference fringe L2 from a predetermined phase by changing the phase of the interference fringe L2 using the phase modulation unit, and change the phase of the interference fringe L2 from a predetermined phase.
  • the phase difference with respect to the phase is the first phase difference ⁇ 1 , the second phase difference ⁇ 2 , the third phase difference ⁇ 3 , the fourth phase difference ⁇ 4 , and the fifth phase difference ⁇ 5.
  • I 1 to (r, k s ; ⁇ 1 ) corresponding to the first phase difference ⁇ 1 , I 1 to (r, k s ; ⁇ 2 ) corresponding to the second phase difference ⁇ 2 , and the third phase difference ⁇ corresponding to 3 I ⁇ (r, k s ; ⁇ 3), fourth I ⁇ corresponding to the phase difference ⁇ 4 (r, k s; ⁇ 4), and a fifth I corresponding to the phase difference phi 5 ⁇ (R, k s ; ⁇ 5 ) are represented by the following expressions (37A) to (37E).
  • the equations (37A) to (37E) represent five unknowns I′ 0 to (r, k s ), I′ +1 to (r, k s ), I′ ⁇ 1 to (r, k s ), I′ +2.
  • ⁇ (R, k s ) and I'- 2 ⁇ (r, k s ) are simultaneous equations.
  • the image processing unit 7 solves the simultaneous equations to obtain the acquired data I 1 to (r, k s ; ⁇ 1 ), I to (r, k s ; ⁇ 2 ), I to (r, k s ; ⁇ 3 ).
  • the image processing unit 7 has I′ 0 to (r, k s ), I′ +1 to (r, k s ), I′ ⁇ 1 to (r, k s ), I′ +2 to (r, k s ).
  • I′ ⁇ 2 to (r, k s ) are inverse Fourier transformed to obtain I 0 (r, r s ), I +1 (r, r s ), I ⁇ 1 (r, r s ), I +2 (R,r s ) and I ⁇ 2 (r,r s ) can be obtained.
  • the image processing unit 7 performs the same processing as that in the third embodiment after the processing for calculating the image data in the real space in the third embodiment.
  • the process corresponding to the component separation can be performed by solving the simultaneous equations of the equations (37A) to (37E), and the position of the detection unit 6a can be performed. It is possible to correct the collapse of the shape of the effective PSF for each (detector coordinate r).
  • FIG. 33 is a flowchart showing an observation method using the microscope 731 according to the third modification.
  • the control unit 8 of the microscope 731 sets the periodic direction of the interference fringes L2.
  • the control unit 8 sets the cycle direction of the interference fringes L2 by rotating the mask 15 by the drive unit 22 and rotating the phase modulation element 71 by the drive unit 78 according to the rotation of the mask 15.
  • the control unit 8 sets the phase of the fringe of the interference fringe L2.
  • the control unit 8 sets the phase of the fringe of the interference fringe L2 by rotating the phase modulation element 71 by the drive unit 78.
  • the control unit 8 sets the angles of the scanning mirrors (deflection mirrors 18a and 18b).
  • the illumination optical system 4 irradiates the excitation light as interference fringes L2 at a position on the sample determined by the angle of the scanning mirror set in step S43.
  • the fluorescent substance of the sample S is excited by the interference fringe L2 of the excitation light.
  • the detection device 6 detects the fluorescence from the sample S via the detection optical system 5 for each of the plurality of detection units 6a.
  • step S46 the control unit 8 determines whether to change the angle of the scanning mirror. When it is determined that the processes of steps S43 to S45 have not been completed for a part of the scheduled observation region, the control unit 8 determines to execute the angle change of the scanning mirror in step S46 (step S46; Yes). ). When the control unit 8 determines to change the angle of the scanning mirror (step S46; Yes), the control unit 8 returns to the process of step S43, and the control unit 8 sets the angle of the scanning mirror to the next scheduled angle. Then, the processing from step S44 to step S46 is repeated.
  • step S46 determines not to change the angle of the scanning mirror (step S46; No). ..
  • step S46; No the control unit 8 determines whether to change the phase of the fringe of the interference fringe L2 in step S47. ..
  • step S47 determines that the processing of steps S42 to S46 has not been completed for a part of the phases of the five interference fringes planned
  • the control unit 8 changes the fringe phase of the interference fringe L2 in step S47. Is determined to be executed (step S47; Yes).
  • step S47; Yes determines to change the phase of the fringe of the interference fringe L2
  • step S47; Yes the control unit 8 returns to the process of step S42, and the control unit 8 schedules the phase of the fringe of the interference fringe L2. Set to the next phase. Then, the processing from step S43 to step S47 is repeated. In this way, the illumination optical system 4 two-dimensionally scans the sample S with the interference fringes L2 of the excitation light for the five interference fringes whose fringe phase is changed.
  • step S47 When it is determined in step S47 that the processes of steps S42 to S46 have been completed for all the phases of the five fringes of the planned interference fringes, the control unit 8 executes the phase change of the fringes of the interference fringes L2. It is determined not to do so (step S47; No). When the control unit 8 determines that the phase of the fringes of the interference fringes L2 is not changed (step S47; No), in step S48, the image processing unit 7 performs image processing to perform an image (eg, super-resolution image). ) Is generated.
  • an image eg, super-resolution image
  • step S49 the control unit 8 determines whether to change the cycle direction of the interference fringe L2.
  • the controller 8 changes the cycle of the interference fringes L2 in step S49.
  • the determination is made (step S49; Yes).
  • step S49; Yes the control unit 8 determines to execute the change of the periodic direction of the interference fringe L2
  • step S49; Yes the control unit 8 returns to the process of step S41, and the control unit 8 determines the next periodic direction of the interference fringe L2. Set in the periodic direction. Then, the processing from step S42 to step S49 is repeated.
  • step S49 determines in step S49 that the processes in steps S41 to S48 have been completed for all the scheduled periodic directions of the interference fringes L2.
  • Step S49; No determines not to change the periodic direction of the interference fringes L2.
  • FIG. 34 is a flowchart showing a sub-flow of image processing by the image processing unit 7.
  • the image processing unit 7 selects at least a part of the plurality of detection units 6a of the detection device 6.
  • the image processing unit 7 Fourier transforms the detection results of at least a part of the plurality of detection units 6a selected in step S51.
  • the image processing unit 7 performs a two-dimensional Fourier transform on I(r, r s ; ⁇ ) given a predetermined phase difference ⁇ .
  • the image processing unit 7 performs a process corresponding to the component separation in the third embodiment by solving the simultaneous equations represented by the above equations (37A) to (37E).
  • step S54 the image processing unit 7 performs an inverse Fourier transform on the separated components.
  • step S55 the image processing unit 7 executes the image processing phase shift processing.
  • the image processing unit 7 corrects the positional deviation of the effective PSF in step S56.
  • step S57 the image processing unit 7 generates an image (for example, a super-resolution image) by adding the images obtained by correcting the positional deviation in step S56.
  • step S58 the control unit 8 determines whether to change the selection of the detection unit 6a.
  • the control unit 8 determines to execute the selection change of the detection unit 6a in step S58 ( Step S58; Yes).
  • step S58; Yes the control unit 8 determines to change the selection of the detection unit 6a
  • step S58; Yes the control unit 8 returns to the process of step S51, and the control unit 8 selects at least another part of the plurality of detection units 6a. .. Then, the processing from step S52 to step S58 is repeated.
  • control unit 8 determines in step S58 that the processing of steps S51 to S57 has been completed for all the scheduled combinations of the detection units 6a, it determines that the selection change of the detection unit 6a is not executed (step). S58; No).
  • the control unit 8 determines that the selection change of the detection unit 6a is not executed (step S58; No)
  • the process returns to the flow of FIG. 33 (step S49). Accordingly, it is possible to generate an image in which the detecting unit 6a to be selected is changed.
  • the microscope 721 according to the second modification may be used instead of the microscope 731 according to the third modification.
  • the image processing unit 7 adds the images of the respective detection units 6a obtained by executing the phase shift process to obtain an image with high resolution (eg, super-resolution image) while ensuring S/N. Can be generated.
  • the fluorescent substance contained in the sample S is multiphoton excited by the excitation light L1.
  • the image data on the sample surface Sa set in an arbitrary portion including the inside of the sample S can be acquired with high accuracy.
  • the interference fringes formed by interference have been described as an example of striped illumination (stripe illumination), but the striped illumination (stripe illumination) formed by interference is not.
  • striped illumination (stripe illumination) formed by a method other than interference may be used.
  • the “stripes” and “stripes” have a bright portion and a dark portion, and have a predetermined interval (predetermined period) between the bright portion and the bright portion or the dark portion and the dark portion.
  • “Striped illumination (stripe illumination)” and “interference fringes” are formed on the sample surface.

Landscapes

  • Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Microscoopes, Condenser (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)

Abstract

Le microscope (1) de l'invention est équipé : d'un système optique d'éclairage (4) qui possède à son tour une unité de balayage (18) balayant un éclairage en bande dans une pluralité de directions d'un échantillon (S), lequel éclairage en bande consiste en un éclairage sous forme de bande généré par une lumière destinée à une excitation multi-photons d'une matière fluorescente contenue dans ledit échantillon (S) ; d'un système optique de détection (5) qui présente une incidence d'une lumière fluorescente (L3) provenant de l'échantillon (S) produite par excitation multi-photons ; d'un dispositif de détection (6) qui possède une pluralité d'unités détection (6a) détectant la lumière fluorescente (L3) provenant de l'échantillon (S) par l'intermédiaire du système optique de détection (5) ; et d'une unité traitement d'image (7) qui génère une image à l'aide de résultats de détection dans au moins deux des unités détection (6a).
PCT/JP2019/008975 2019-03-06 2019-03-06 Microscope, et procédé d'observation WO2020179039A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2019/008975 WO2020179039A1 (fr) 2019-03-06 2019-03-06 Microscope, et procédé d'observation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2019/008975 WO2020179039A1 (fr) 2019-03-06 2019-03-06 Microscope, et procédé d'observation

Publications (1)

Publication Number Publication Date
WO2020179039A1 true WO2020179039A1 (fr) 2020-09-10

Family

ID=72337732

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/008975 WO2020179039A1 (fr) 2019-03-06 2019-03-06 Microscope, et procédé d'observation

Country Status (1)

Country Link
WO (1) WO2020179039A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010088418A1 (fr) * 2009-01-29 2010-08-05 The Regents Of The University Of California Microscopie de haute résolution à éclairage structuré
JP2010271522A (ja) * 2009-05-21 2010-12-02 Nikon Corp 非線形光学顕微鏡
WO2015052936A1 (fr) * 2013-10-09 2015-04-16 株式会社ニコン Dispositif microscope à éclairage structuré
WO2015118634A1 (fr) * 2014-02-05 2015-08-13 株式会社ニコン Dispositif d'éclairage, dispositif d'observation et procédé d'observation
JP2016105126A (ja) * 2014-12-01 2016-06-09 横河電機株式会社 顕微鏡装置
WO2017094184A1 (fr) * 2015-12-04 2017-06-08 オリンパス株式会社 Microscope à balayage et procédé d'acquisition d'image de microscope

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010088418A1 (fr) * 2009-01-29 2010-08-05 The Regents Of The University Of California Microscopie de haute résolution à éclairage structuré
JP2010271522A (ja) * 2009-05-21 2010-12-02 Nikon Corp 非線形光学顕微鏡
WO2015052936A1 (fr) * 2013-10-09 2015-04-16 株式会社ニコン Dispositif microscope à éclairage structuré
WO2015118634A1 (fr) * 2014-02-05 2015-08-13 株式会社ニコン Dispositif d'éclairage, dispositif d'observation et procédé d'observation
JP2016105126A (ja) * 2014-12-01 2016-06-09 横河電機株式会社 顕微鏡装置
WO2017094184A1 (fr) * 2015-12-04 2017-06-08 オリンパス株式会社 Microscope à balayage et procédé d'acquisition d'image de microscope

Similar Documents

Publication Publication Date Title
JP7063337B2 (ja) 顕微鏡および観察方法
JP5888416B2 (ja) 構造化照明顕微鏡装置
US8019136B2 (en) Optical sectioning microscopy
EP2093601B1 (fr) Dispositif de microscope et procédé de traitement d'image
EP2565697B1 (fr) Dispositif de microscope avec éclairement structurel
US9897790B2 (en) Structured illumination device and structured illumination microscope device
JP4835750B2 (ja) 顕微鏡装置
JP2014508969A (ja) 蛍光顕微鏡検査法における照明位相制御のためのシステムおよび方法
US20130342886A1 (en) Variable orientation illumination-pattern rotator
JP6194710B2 (ja) 構造化照明装置及び構造化照明顕微鏡装置
US20210389577A1 (en) Microscope
WO2016199179A1 (fr) Système de microscope à éclairage structuré, procédé et programme
US20150124073A1 (en) Illumination optical system and microscope
TWI414818B (zh) 使用空間光調變器的超解析率廣視野光切片顯微鏡
JP2007279287A (ja) 構造化照明光学系、及びそれを備えた構造化照明顕微鏡
US9507137B2 (en) Microscope with structured illumination using displaceable grid structures
JP6413364B2 (ja) 照明光学系及び顕微鏡装置
WO2020179039A1 (fr) Microscope, et procédé d'observation
JP6269239B2 (ja) 構造化照明装置及び構造化照明顕微鏡装置
WO2020179032A1 (fr) Microscope et procédé d'observation
WO2020179040A1 (fr) Microscope, et procédé d'observation
WO2015052920A1 (fr) Dispositif d'éclairage structuré et dispositif microscope à éclairage structuré
WO2022050221A1 (fr) Microscope
WO2020179068A1 (fr) Microscope

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19918019

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19918019

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP