JP4844137B2 - Microscope equipment - Google Patents

Description

  The present invention relates to a microscope apparatus, and more particularly to a structured illumination microscope apparatus having super resolution.

  In image formation by an imaging optical system such as an optical microscope, a transfer function unique to the imaging optical system exists, and the characteristics of the object image formed by the transfer function are limited. More specifically, of the Fourier components (spatial frequency components) of the optical image to be transmitted, only a specific spatial frequency region determined by the transfer function is transmitted, and the remaining spatial frequency components are cut.

For example, in a general optical microscope, a spatial frequency fcutoff called a cutoff frequency determined by the numerical aperture (NA) of the objective lens,
fcutoff = 2NA / λ (where λ is the wavelength of light)
In the Fourier component of the input optical image, a spatial frequency component higher than the cutoff frequency cannot be imaged.

  The numerical aperture of the objective lens is obtained by multiplying the sine function of the apex angle of the light cone that the objective lens can capture from the observation object O by the refractive index of the medium between the observation object and the front surface of the objective lens, For example, for an object in the air, since the numerical aperture does not become 1 or more, the cutoff frequency does not become 2 / λ or more. Therefore, a fine structure having a period smaller than ½ of the wavelength of light placed in the air cannot be resolved by a general optical microscope.

  However, by inserting means for modulating the spatial frequency between the observation object and the imaging optical system, the spatial frequency component of the observation object in the spatial frequency region that cannot be transmitted by the imaging optical system can be reflected in the imaging. . However, in this case, since the image of the observation object imaged by the imaging optical system is modulated, a correct observation object image can be formed by using the modulation recovery (demodulation) means together. When this is applied to an optical microscope, it becomes possible to resolve the fine structure of an observation object having a spatial frequency higher than the conventional cutoff frequency. This is called super-resolution.

  Such a microscope apparatus using super-resolution is used for super-resolution observation of an object to be observed such as a fixed specimen prepared from a biological sample. (See Patent Documents 1 and 2, Non-Patent Documents 1 and 2). As described above, in this method, the spatial frequency of the structure of the object to be observed is modulated with the spatially modulated illumination light, and high spatial frequency information exceeding the resolution limit is contributed to the imaging of the microscope optical system.

  However, as described above, in order to observe the super-resolution image, it is necessary to demodulate the modulated image of the observed object (modulated image). There are roughly two types of demodulation methods: optical demodulation (see Non-Patent Documents 1 and 2) and demodulation by calculation (see Patent Documents 1 and 2). Optical demodulation is realized by re-modulating the modulated image using a spatial modulation element such as a diffraction grating.

In addition, although not a known technique, the inventors of the present application performed illumination by forming an image of the diffraction grating placed in the illumination optical system on the surface of the object to be observed, and illuminated with the illumination light. A microscope apparatus that forms an image of the object to be observed as an intermediate image in the shape of a diffraction grating by an imaging optical system, and forms the intermediate image that has passed through the diffraction grating on an image sensor by a relay optical system And at least the optical element constituting the illumination optical system from the diffraction grating to the object to be observed and the optical element constituting the imaging optical system from the object to the diffraction grating are shared. Has been filed as Japanese Patent Application No. 2005-299329 (referred to as a prior invention).
Japanese Patent Laid-Open No. 11-242189 US Reissue Patent No. 38307 Specification W. Lukosz, "Optical systems with resolving powers exceeding the clasical limit. II", Journal of the Optical Society of America, Vol. 37, PP. 932, 1967 W.Lukosz and M. Marchand, Opt. Acta. 10,241,1963

  In any of the methods described above, the direction in which super-resolution is obtained is a direction perpendicular to the length direction of the grating of the diffraction grating. Therefore, in order to obtain a two-dimensional super-resolution, it is necessary to change the direction of the diffraction grating, perform imaging in a plurality of directions (usually three directions), and calculate the result.

  As a means for realizing such a method, the most commonly used method is a method of rotating a diffraction grating and picking up a corresponding image. Adopts this method.

  However, the method of accurately stopping by rotating the diffraction grating has a problem that it takes time to position the stop position. In particular, observation of a live cell or the like for observing a cell alive requires high-speed imaging, and there is a problem that the method of rotating and positioning a diffraction grating is not practical.

  The present invention has been made in view of such circumstances, and an object thereof is to provide a microscope apparatus capable of high-speed observation.

  A first means for solving the above-described problem includes an illumination optical system that includes a plurality of optical paths, and that performs illumination by forming an image of a diffraction grating placed in each optical path on the surface of the object to be observed. A microscope apparatus comprising: an imaging optical system that forms an image of the object illuminated by the illumination optical system as an intermediate image on the diffraction grating; and a relay optical system that forms the image of the intermediate image The plurality of diffraction gratings have different grating pattern directions, and the illumination optical system further guides light from a light source by switching to an arbitrary optical path of the plurality of optical paths, and enters the optical path. First illumination light path switching means for guiding the arranged diffraction grating, and second illumination for guiding light from the illuminated diffraction grating to the surface of the object to be observed in synchronization with the first illumination light path switching means. Optical path switching means, the imaging optical system is Furthermore, in synchronization with the first observation light path switching means for guiding light from the object to be observed to the illuminated diffraction grating, the light from the illuminated diffraction grating is synchronized with the first observation light path switching means. A microscope apparatus comprising: a second observation optical path switching unit that leads to a relay optical system.

  The second means for solving the problem is the first means, and at least an optical element constituting the illumination optical system from the first illumination light path switching means to the object to be observed, An optical element constituting the imaging optical system from the object to be observed to the second observation light beam path switching means is shared, and the first illumination light path switching means and the second observation light beam path switching means are the same. And the second illumination light path switching means and the first observation light path switching means are the same.

  A third means for solving the above-described problem includes an illumination optical system that includes a plurality of optical paths and that performs illumination by forming an image of a diffraction grating placed in each optical path on the surface of the object to be observed. A microscope apparatus including an imaging optical system that forms an image of the object illuminated by the illumination optical system, wherein the plurality of diffraction gratings have different grating pattern directions, and the illumination optical system The light from the light source is switched to an arbitrary optical path among the plurality of optical paths and guided to a diffraction grating disposed in the optical path, and the first illumination optical path switching means And a second illumination light path switching means for guiding the light from the illuminated diffraction grating to the surface of the object to be observed.

  A fourth means for solving the above problem is any one of the first to third means, wherein the diffraction grating is a transmissive or reflective spatial light modulator. To do.

  A fifth means for solving the problem is the first means, in which the microscope apparatus captures an image formed by the relay optical system, and the surface of the object to be observed. Phase modulation means for changing the phase of the grating pattern of the image of the diffraction grating to be imaged, and the imaging time of the imaging device is N periods (N is a natural number) of the phase of the grating pattern, To do.

  According to the present invention, a microscope apparatus capable of high-speed observation can be provided.

  Hereinafter, examples of embodiments of the present invention will be described. However, since the main ones of these embodiments are improvements of the embodiments of the prior application, first, examples of the embodiments of the prior application are described. Will be explained. FIG. 1 is a schematic configuration diagram of a microscope apparatus according to an embodiment of the invention of the prior application. As shown in FIG. 1, the microscope apparatus includes a light source 1, a collector lens 2, a lens 3, an excitation filter 4, a dichroic mirror 5, a lens 7, a diffraction grating 8, a second objective lens 9, an objective lens 10, and a fluorescent dye. A specimen (biological specimen) 11, a barrier filter 6, a lens 12, an imaging device (CCD camera etc.) 25, a control / arithmetic device (circuit or computer) 42, an image display device 43, an actuator 40, and a rotary stage 41. Be placed.

  Among these, the light source 1, the collector lens 2, the lens 3, the excitation filter 4, the dichroic mirror 5, the lens 7, the diffraction grating 8, the second objective lens 9, and the objective lens 10 constitute the illumination optical system LS1, and the objective lens 10, the second objective lens 9, the diffraction grating 8, the lens 7, the dichroic mirror 5, the barrier filter 6, and the lens 12 constitute an observation optical system LS2. The objective lens 10 and the second objective lens 9 constitute an imaging optical system LS21, and the lens 7 and the lens 12 constitute a relay optical system LS22. The illumination optical system LS1 and the observation optical system LS2 share an optical path from the objective lens 10 to the dichroic mirror 5.

  Light from the light source 1 of the illumination optical system LS1 is converted into parallel light by the collector lens 2, and a light source image is formed on the pupil conjugate plane 31 by the lens 3. The light from the light source image 31 is wavelength-selected by the excitation filter 4 and then deflected by the dichroic mirror 5 and enters the common optical path of the illumination optical system LS 1 and the observation optical system LS 2, and the conjugate surface 22 of the sample 11 by the lens 7. Concentrate on top. The light emitted from the conjugate surface 22 is incident on the second objective lens 9 via the diffraction grating 8 disposed on the conjugate surface 22, converted into parallel light, and then on the specimen 11 via the objective lens 10. An image 23 of the diffraction grating 8 is formed (at this time, a light source image 32 is formed on the rear focal plane of the objective lens 10). Thus, the specimen 11 is illuminated (structured illumination) with spatially modulated illumination light.

  Here, the diffraction grating 8 is, for example, a phase type or amplitude type diffraction grating having a one-dimensional periodic structure. In particular, an amplitude type diffraction grating is preferable because a white light source can be used as the light source 1 because of its good wavelength characteristics. As the light source 1, a light source having a single wavelength may be used instead of a white light source, or a secondary light source formed on the end face thereof may be used as the light source 1 by guiding light from the laser light source with an optical fiber. .

  Further, in order to make the luminance distribution of the structured illumination (the luminance distribution of the image 23 of the diffraction grating 8) sinusoidal, it is desirable to remove excess diffraction components of order 2 or more generated in the diffraction grating 8. In that case, it is preferable to remove at an appropriate position after the diffraction grating 8 (for example, the pupil plane of the objective lens 10). Alternatively, if the density distribution of the diffraction grating 8 is made sinusoidal in advance, the generation of extra diffraction components can be suppressed and the loss of light quantity can be suppressed.

  Now, fluorescence is generated on the specimen 11 using the structured illumination light as excitation light. At this time, the structure of the specimen 11 viewed from the objective lens 10 side is modulated by structured illumination. Moire fringes are generated in the modulated structure. This moire fringe is a moire fringe formed by the fine structure of the specimen 11 and the pattern of structured illumination, and the fine structure of the specimen 11 is converted to a lower spatial frequency band by the spatial frequency of the structured illumination. . Therefore, even the light having a high spatial frequency structure exceeding the resolution limit is captured by the objective lens 10.

  The fluorescence captured by the objective lens 10 forms a modulated image of the specimen 11 on the conjugate plane 22 by the imaging optical system LS21 including the objective lens 10 and the second objective lens 9. The modulated image is remodulated by the diffraction grating 8 arranged on the conjugate plane 22. In the re-modulated image generated in this way, the structure of the sample 11 whose spatial frequency is changed is returned to the original spatial frequency. The re-modulated image includes the demodulated image of the sample 11.

  However, this re-modulated image contains diffraction components that are unnecessary for the demodulated image. Unnecessary diffraction components are ± first order diffraction components generated at the diffraction grating 8 with respect to the 0th order diffracted light emitted from the sample 11, 0th order diffracted components with respect to the −1st order diffracted light emitted from the sample 11, and emitted from the sample 11. This is the 0th order diffraction component for the + 1st order diffracted light. In order to remove these unnecessary diffraction components from the re-modulated image, the diffraction grating 8 may be moved and averaged by one period or N periods (N is a natural number).

  Fluorescence from the remodulated image passes through the dichroic mirror 5 through the lens 7, enters the single optical path of the observation optical system LS 2, passes through the barrier filter 6, and then is enlarged through the lens 12. 24 is formed. That is, the remodulated image remodulated by the diffraction grating 8 is relayed to the enlarged image 24 by the relay optical system LS22 including the lens 7 and the lens 12. The magnified image 24 is picked up by the image pickup device 25, and image data of a remodulated image is generated. In the case of imaging with the imaging device 25, if the diffraction grating 8 is moved by one period or N periods (N is a natural number) and averaged by accumulating remodulated images, demodulated image data can be obtained. Can do.

This image data includes information for super-resolution observation of the specimen 11 with structured illumination. The image data is captured by the control / arithmetic unit 42, subjected to computation, and then sent to the image display unit 43.
As described above, in the microscope apparatus, the optical path from the conjugate plane 22 of the specimen 11 to the specimen 11 is completely made a common optical path by the illumination optical system LS1 and the observation optical system LS2, and the diffraction grating 8 is arranged on the conjugate plane 22. ing. In this microscope apparatus, the fine structure of the specimen 11 is modulated by the diffraction grating 8. The fine structure of the sample 11 thus modulated is automatically remodulated by the diffraction grating 8 arranged at this position.

  The diffraction grating 8 can be moved by the actuator 40 in a direction Db orthogonal to the grating line. This movement changes the phase of the structured illumination. The control / arithmetic unit 42 controls the actuator 40 and the imaging device 25 and changes the phase by one period or N periods (N is a natural number) while accumulating image data for one frame, From the image data, the structured illumination pattern and unnecessary diffraction components generated during re-modulation are eliminated.

  The diffraction grating 8 can be rotated around the optical axis together with the actuator 40 by the rotary stage 41. This rotation changes the direction of structured illumination. If the control / arithmetic unit 42 controls the rotary stage 41 and the imaging device 25 and acquires image data every time the direction of the structured illumination is changed in a plurality of directions, information for super-resolution observation in a plurality of directions is obtained. Obtainable. Thereby, two-dimensional super-resolution observation of the specimen 11 becomes possible.

  The program necessary for the above operation is preinstalled in the control / arithmetic unit 42 via a recording medium such as a CD-ROM or the Internet.

  Next, conditions necessary for the relay optical system LS22 of the microscope apparatus will be described.

  FIG. 2 is a diagram for explaining the numerical aperture of the imaging light beam from the specimen 11 to the lens 7 of the observation optical system LS2 in this microscope apparatus. In FIG. 2, the image 23 of the diffraction grating 8 and the sample 11 are drawn while being shifted, and the conjugate plane 22 of the sample 11 and the diffraction grating 8 are drawn while being shifted in order to make it easy to understand the change in the spread of the light beam. .

  As shown in FIG. 2, the fluorescence LA generated by the high spatial frequency structure of the specimen 11 has a large emission angle, and therefore normally proceeds as indicated by a broken line and cannot enter the pupil diameter of the objective lens 10. However, since the fluorescence LA is modulated by the image 23 of the diffraction grating 8 (that is, structured illumination), the object whose emission angle has changed to the low frequency side (that is, the small angle) proceeds as indicated by the solid line, and the objective lens It is possible to enter within 10 pupil diameters.

  The fluorescence LA is imaged on the conjugate plane 22 by the imaging optical system LS21 and then remodulated in the diffraction grating 8. The emission angle of the re-modulated fluorescence LA ′ is returned to the high frequency side (that is, a large angle).

  In order to guide such an imaging light beam including the fluorescence LA ′, the object-side numerical aperture NA ′ of the relay optical system LS22 needs to be set large in advance.

  In particular, the higher the spatial frequency of structured illumination (that is, the grating frequency of the diffraction grating 8) is, the larger the opening angle of the imaged light beam is. Therefore, it is necessary to increase the object-side numerical aperture NA ′ of the relay optical system LS22. .

  However, in general, the super-resolution effect by structured illumination is about twice the resolution limit of the objective lens 10. This is because the spatial frequency of structured illumination projected onto the specimen 11 by the illumination optical system LS1 is the upper limit of the resolution of the objective lens 10. Therefore, it is sufficient that the object-side numerical aperture NA ′ of the relay optical system LS22 is large to some extent.

  Next, operations related to the control of the control / arithmetic apparatus 42 will be described. FIG. 3 is a diagram for explaining the numerical aperture of the imaging light beam from the sample 11 to the lens 7 of the observation optical system LS2 in this microscope apparatus. As shown in FIG. 3, when acquiring the image data of the remodulated image, the control / arithmetic unit 42 performs structured illumination during the period from the start of exposure (step S11) to the end of exposure (step S13) of the imaging device 25. The phase is changed by one period (step S12).

  The image data acquired in this way is the time integration of the re-modulated image during the phase change of the structured illumination, and the brightness distribution of the structured illumination is sinusoidal. Is erased. Further, unnecessary diffraction components generated at the time of re-modulation are deleted from the image data. Therefore, this image data represents a demodulated image.

  Further, after changing the direction of the structured illumination (step S15), the control / arithmetic unit 42 performs the processing of steps S11 to S13 again, and image data of another demodulated image from which the structured illumination pattern has been deleted. To get.

  Then, the image data acquisition processing of the demodulated image in the above steps S11 to S13 is repeated until the direction of the structured illumination is set in all predetermined directions (until YES in step S14). The image data of the demodulated image from which the pattern has been deleted is acquired in the number of the set directions.

For example, the control / arithmetic unit 42 repeats the processing of steps S11 to S13 until the structured illumination direction is set to three directions of 0 °, 120 °, and 240 °, and the structured illumination pattern is erased. Image data I1, I2, and I3 of three demodulated images are acquired. The direction of super-resolution differs by 120 ° between the image data I1, I2, and I3 of these demodulated images.

  Next, operations related to the calculation of the control / arithmetic apparatus 42 will be described.

FIG. 4 is an operation flowchart regarding the calculation of the control / arithmetic apparatus 42. Here, a description will be given of calculation when image data I1, I2, and I3 of three demodulated images whose super-resolution directions are different by 120 ° are obtained.

First, the control / arithmetic unit 42 Fourier-transforms each of the three demodulated image data I1, I2, and I3 to obtain three demodulated image data Ik1, Ik2, and Ik3 expressed in wave number space (step). S21). The demodulated image data Ik1, Ik2, and Ik3 are shown in FIGS. 5A, 5B, and 5C.

In FIGS. 5A, 5B, and 5C, symbol Ik ± 1 is a component (± first order modulation component) transmitted by the imaging optical system LS21 in a modulated state (as ± first order diffracted light). The symbol Ik0 indicates a component (0th-order modulation component) transmitted by the imaging optical system LS21 in a non-modulated state (as 0th-order diffracted light). The symbol Db indicates the direction of super-resolution (structured illumination
The symbol K indicates the spatial frequency of the structured illumination.

  Subsequently, the control / arithmetic unit 42 synthesizes the image data Ik1, Ik2, and Ik3 of the three demodulated images on the wave number space as shown in FIG. 6 to obtain one synthesized image data Ik (step S22). Since the image data Ik1, Ik2, and Ik3 of the three demodulated images overlap each other's data ranges (including a common spatial frequency component), the control / arithmetic unit 42 determines the overlapping data in the synthesis. Takes an average value thereof and uses the average value as data of the composite image data Ik. By suppressing the contribution of the low frequency component of the composite image data Ik from becoming too large by this average, it is possible to prevent the relative contribution of the high frequency component from becoming small.

  Subsequently, the control / arithmetic unit 42 performs inverse Fourier transform on the composite image data Ik to obtain image data I expressed in real space. The image data I represents a super-resolution image of the specimen 11 in three directions that differ by 120 ° (step S23). The control / arithmetic unit 42 sends the image data I to the image display unit 43 to display a super-resolution image.

  As described above, in this microscope apparatus, as shown in FIG. 1, the light from the specimen 11 is remodulated by the diffraction grating 8, and further, the diffraction grating 8 is moved and averaged to remove unnecessary diffraction components, thereby obtaining a demodulated image. ing. Therefore, the image data of the demodulated image can be obtained at high speed as much as the demodulation operation is not performed.

  In addition, since the same region of the same diffraction grating 8 is used for modulation and re-modulation, even if there is a shape error or an arrangement error in the diffraction grating 8, the modulation pattern and the re-modulation pattern are the same. Can be. Therefore, the shape error or the arrangement error of the diffraction grating 8 hardly gives noise to the image data of the demodulated image. This is also true when the phase of structured illumination is changed or when the direction of structured illumination is changed. Therefore, this microscope apparatus can obtain a super-resolution image with high accuracy.

  Further, in this microscope apparatus, for the convenience of arranging the diffraction grating 8 on the conjugate plane 22 of the specimen 11, it is necessary to relay the remodulated image formed on the conjugate plane 2 and then capture the image with the imaging apparatus 25. However, the relay optical system LS22 of the microscope apparatus can reliably acquire information necessary for super-resolution.

  In addition, when synthesizing a plurality of pieces of image data (step S22 in FIG. 4), the present microscope apparatus averages the spatial frequency components that the image data has in common, so that a good super frequency with little attenuation of high frequency components can be obtained. A resolution image can be obtained.

  Hereinafter, examples of embodiments of the present invention will be described. Since the basic configuration of these embodiments is almost the same as that of the embodiment of the prior invention shown in FIG. 1, the same as FIG. Parts may be denoted by the same reference numerals and description thereof may be omitted.

  FIG. 7 is a diagram showing an outline of the microscope apparatus according to the first embodiment of the present invention. The microscope apparatus includes a light source 1, a collector lens 2, a lens 3, an excitation filter 4, a dichroic mirror 5, a galvano mirror M1, lenses 7a to 7c, diffraction gratings 8a to 8c, mirrors 31a and 31c, and second objective lenses 9a to 9c. , Galvanometer mirror M2, objective lens 10, specimen (biological specimen) 11 labeled with a fluorescent dye, barrier filter 6, lens 12, imaging device (CCD camera, etc.) 25, control / arithmetic unit (circuit, computer, etc.) 42 An image display device 43 is arranged.

  Among these, the light source 1, collector lens 2, lens 3, excitation filter 4, dichroic mirror 5, galvano mirror M1, lenses 7a to 7c, diffraction gratings 8a to 8c, mirrors 31a and 31c, second objective lenses 9a to 9c, galvano The mirror M2 and the objective lens 10 constitute an illumination optical system. The objective lens 10, the galvanometer mirror M2, the second objective lenses 9a to 9c, the mirrors 31a and 31c, the diffraction gratings 8a to 8c, the lenses 7a to 7c, and the galvanometer mirror. M1, the dichroic mirror 5, the barrier filter 6, and the lens 12 constitute an observation optical system. The objective lens 10, the galvano mirror M2, the second objective lenses 9a to 9c, and the mirrors 31a and 31c constitute an imaging optical system. The lenses 7a to 7c, the galvano mirror M1, the dichroic mirror 5, the barrier filter 6, and the lens 12 are included. Constitutes a relay optical system. The illumination optical system and the observation optical system share an optical path from the objective lens 10 to the dichroic mirror 5.

  Light from the light source 1 of the illumination optical system is converted into parallel light by the collector lens 2, and a light source image is formed on the pupil conjugate plane 31 by the lens 3. On the way, the wavelength is selected by the excitation filter 4 and then deflected by the dichroic mirror 5. Then, the direction is selected by the galvanometer mirror M1, and any one of the diffraction gratings 8a, 8b, and 8c is illuminated through any one of the lenses 7a, 7b, and 7c. The surfaces of the diffraction gratings 8a, 8b, and 8c coincide with the conjugate surfaces 22a, 22b, and 22c of the sample 11, respectively. The light emitted from any one of the conjugate surfaces 22a, 22b, and 22c through any one of the diffraction gratings 8a, 8b, and 8c is incident on any one of the second objective lenses 9a, 9b, and 9c (between them, the conjugate surface 22a). The light emitted from the mirror 31a is reflected by the mirror 31c, and the light emitted from the conjugate surface 22c is reflected by the mirror 31c. After being converted into parallel light, the light is reflected by the galvanometer mirror M2 and is reflected on the specimen 11 via the objective lens 10. Then, an image 23 of any one of the diffraction gratings 8a, 8b, and 8c is formed (at this time, a light source image 32 is formed on the rear focal plane of the objective lens 10). Thus, the specimen 11 is illuminated (structured illumination) with spatially modulated illumination light. Note that the galvanometer mirror M1 and the galvanometer mirror M2 are driven in synchronism so that the light passing through one of the illuminated diffraction gratings 8a, 8b, and 8c reaches the sample 11.

  The grating pattern formed on the specimen 11 by the diffraction gratings 8a, 8b, and 8c is shifted by 60 ° in the direction of the grating lines. Therefore, the optical path is switched by the galvanometer mirrors M1 and M2, and the diffraction gratings 8a, 8b and 8c are switched and used. In the apparatus shown in FIG. You get the same effect as you did. Since the galvanometer mirrors M1 and M2 can be driven at high speed, high-speed measurement is possible.

  Here, the diffraction gratings 8a, 8b, and 8c are, for example, phase type or amplitude type diffraction gratings having a one-dimensional periodic structure. In particular, an amplitude type diffraction grating is preferable because a white light source can be used as the light source 1 because of its good wavelength characteristics. As the light source 1, a light source having a single wavelength may be used instead of a white light source, or a secondary light source formed on the end face thereof may be used as the light source 1 by guiding light from the laser light source with an optical fiber. .

  Further, in order to make the luminance distribution of the structured illumination (the luminance distribution of the image 23 of the diffraction grating 8) sinusoidal, it is desirable to remove excess diffraction components of order 2 or more generated in the diffraction gratings 8a, 8b, and 8c. . For example, when the diffraction gratings 8a, 8b, and 8c are of the amplitude type, an extra diffraction component is likely to be generated. It is good to remove. Alternatively, if the density distributions of the diffraction gratings 8a, 8b, and 8c are made sinusoidal in advance, generation of extra diffraction components can be suppressed and loss of light quantity can be suppressed.

Further, it is more preferable to use the light source 1 while being shifted with respect to the optical axis of the illumination optical system. In that case, the light source 1 is shifted by a predetermined amount in directions perpendicular to the grating patterns of the diffraction gratings 8a, 8b, and 8c, and the 0th order and the next time out of the diffracted lights by the diffraction gratings 8a, 8b, and 8c. It is preferable that the folding light is symmetric with respect to the optical axis of the illumination optical system LS1. The shift amount s is expressed as follows, where fc is the combined focal length of the optical system from the light source 1 to the diffraction gratings 8a, 8b, and 8c, and θg is the primary diffraction angle of the diffraction gratings 8a, 8b, and 8c.
s = fc × sin (θg / 2) (1)
It is good to set so that.

  The reason will be described below. The structured illumination formed on the specimen preferably has a high spatial frequency close to the resolution limit of the imaging optical system to be used. That is, it is desirable that the grating patterns of the diffraction gratings 8a, 8b, and 8c have a high spatial frequency. This is because the higher the spatial frequency to be modulated, the higher the effect of modulation and the higher the super-resolution effect. Further, the luminance distribution of the structured illumination is preferably sinusoidal. This is because unnecessary components are not generated in the diffraction order of the sample diffracted by the structured illumination.

  When the spatial frequency of the grating patterns of the diffraction gratings 8a, 8b, and 8c is high, the diffraction angle is large, so that the diffracted light beam incident on the optical system decreases. However, a structured illumination pattern cannot be formed on the specimen surface unless at least two diffracted light beams are incident. On the other hand, the interference pattern formed by the interference of two light beams has a sinusoidal intensity distribution.

  Therefore, it is convenient to form structured illumination by two-beam interference. Two-beam interference can be realized by generating two beams of zero-order light and first-order light symmetrically with respect to the optical axis and causing interference on the specimen 11. For this purpose, the illumination light must be incident on the diffraction grating surface at an angle. The incident angle is θg / 2 where θg is the first diffraction angle of the diffraction grating. In order to tilt the illumination light, it is only necessary to shift the fiber end by a predetermined amount. Equation (1) is the same regardless of whether the light source 1 is a lamp light source or a laser light source, and the diffraction gratings 8a, 8b, and 8c are either phase type or density type.

  Now, fluorescence is generated on the specimen 11 using the structured illumination light as excitation light. At this time, the structure of the specimen 11 viewed from the objective lens 10 side is modulated by structured illumination. Moire fringes are generated in the modulated structure. This moire fringe is a moire fringe formed by the fine structure of the specimen 11 and the pattern of structured illumination, and the fine structure of the specimen 11 is converted to a lower spatial frequency band by the spatial frequency of the structured illumination. . Therefore, even the light having a high spatial frequency structure exceeding the resolution limit is captured by the objective lens 10.

  The fluorescence captured by the objective lens 10 is converted into a modulated image of the specimen 11 on any one of the conjugate surfaces 22a, 22b, and 22c by the imaging optical system including the objective lens 10, the galvanometer mirror M2, and the second objective lens 9. Although formed, the path through which the fluorescence passes is the same as the path through which the illumination light passes (in the opposite direction). The modulated image is re-modulated by the diffraction gratings 8a, 8b, and 8c disposed on the conjugate surfaces 22a, 22b, and 22c. In the re-modulated image generated in this way, the structure of the sample 11 whose spatial frequency is changed is returned to the original spatial frequency. The re-modulated image includes the demodulated image of the sample 11.

  However, this re-modulated image contains diffraction components that are unnecessary for the demodulated image. Unnecessary diffraction components are ± first order diffraction components generated by the diffraction gratings 8a, 8b, and 8c with respect to the 0th order diffracted light emitted from the sample 11, and 0th order diffraction components with respect to the −1st order diffracted light emitted from the sample 11. This is the 0th-order diffraction component for the + 1st-order diffracted light emitted from the sample 11. In order to remove these unnecessary diffraction components from the re-modulated image, the diffraction grating 8 may be moved and averaged by one period or N periods (N is a natural number).

  Fluorescence from the re-modulated image is reflected by the galvanometer mirror M1 through one of the lenses 7a, 7b, and 7c (through which illumination light passes), passes through the dichroic mirror 5, and then the single optical path of the observation optical system. After entering and passing through the barrier filter 6, an enlarged image 24 of the remodulated image is formed through the lens 12. That is, the remodulated image remodulated by the diffraction gratings 8a, 8b, and 8c is relayed to the enlarged image 24 by the relay optical system having the lenses 7a, 7b, and 7c and the lens 12 as main optical elements. The magnified image 24 is picked up by the image pickup device 25, and image data of a remodulated image is generated. In the case of imaging with the imaging device 25, if the diffraction gratings 8a, 8b, and 8c are moved by one period or N periods (N is a natural number) and averaged by accumulating the remodulated image, the image of the demodulated image Data can be obtained.

This image data includes information for super-resolution observation of the specimen 11 with structured illumination. The image data is captured by the control / arithmetic unit 42, subjected to computation, and then sent to the image display unit 43.
As described above, in the microscope apparatus, the optical path from the conjugate planes 22a, 22b, and 22c of the specimen 11 to the specimen 11 is completely shared by the illumination optical system and the observation optical system, and the conjugate planes 22a, 22b, and 22c are arranged on the conjugate planes 22a, 22b, and 22c. Diffraction gratings 8a, 8b and 8c are arranged. In this microscope apparatus, the fine structure of the specimen 11 is modulated by the diffraction gratings 8a, 8b, and 8c. Then, the microstructure of the modulated specimen 11 is automatically re-modulated by the diffraction gratings 8a, 8b and 8c arranged at this position.

  The diffraction gratings 8a, 8b, and 8c can be moved in a direction perpendicular to the grating lines by an actuator (not shown). This movement changes the phase of the structured illumination. The control / arithmetic unit 42 controls the actuator 40 and the imaging device 25 and changes the phase by one period or N periods (N is a natural number) while accumulating image data for one frame, From the image data, the structured illumination pattern and unnecessary diffraction components generated during re-modulation are eliminated.

  If the diffraction gratings 8a, 8b, and 8c are configured by SLM (Spatial Light Modulator) and a grating pattern is formed by turning on and off (transmission and non-transmission) of the pixels, the diffraction grating a , 8b and 8c are preferable because the phase of structured illumination can be changed without mechanical movement.

  In the above embodiment, the specimen is placed at a position conjugate to the diffraction grating, and an image of the diffraction grating is formed on the specimen by the illumination optical system, and the image of the specimen is formed in the same diffraction grating shape, thereby demodulating the image. It was related with the microscope apparatus of the system which performs. However, the present invention, as described in Patent Document 1, places the sample at a position conjugate with the diffraction grating, creates an image of the diffraction grating on the sample by the illumination optical system, captures the image of the sample, The present invention can also be applied to a microscope apparatus that demodulates an image by calculation.

  An example is shown in FIG. In FIG. 8, the same components as those shown in FIG. 7 are denoted by the same reference numerals, and the description thereof is omitted. In FIG. 8, light emitted from the end face of the optical fiber LG is used as a light source. Other illumination optical systems are substantially the same as those shown in FIG. 9 except that the excitation filter 4 and the dichroic mirror are provided in the subsequent stage of the galvano mirror M2. However, it is unnecessary to explain that the operation of the illumination optical system is the same as that shown in FIG.

  The fluorescence generated from the specimen 11 is deflected by the dichroic mirror 5 through the objective lens 10, and an enlarged image 24 of the specimen 11 is formed on the imaging device 25 by the lens 12 through the barrier filter 6. The control / arithmetic apparatus 42 takes in the magnified image 24 and performs a calculation as described in Patent Document 1 to obtain a super-resolution image of the sample 11.

  Also in this case, the grating pattern formed on the specimen 11 by the diffraction gratings 8a, 8b, and 8c is shifted by 60 ° in the direction of the grating line. Therefore, by switching the optical path by the galvanometer mirrors M1 and M2 and switching and using the diffraction gratings 8a, 8b, and 8c, the same effect as that obtained by rotating the diffraction grating 8 by 120 ° and performing the measurement can be obtained. Since the galvanometer mirrors M1 and M2 can be driven at high speed, high-speed measurement is possible.

It is a schematic block diagram of the microscope apparatus which is embodiment of prior invention. It is a figure explaining the numerical aperture of the imaging light beam from the sample 11 of the observation optical system LS2 in the microscope apparatus which is embodiment of prior invention invention to the lens 7. FIG. It is a figure explaining the numerical aperture of the imaging light beam from the sample 11 of the observation optical system LS2 in the microscope apparatus which is embodiment of prior invention invention to the lens 7. FIG. It is an operation | movement flowchart regarding the calculation of the control and arithmetic unit 42 in embodiment of prior invention. It is a figure explaining step S21 in FIG. It is a figure explaining step S22 in FIG. It is a figure which shows the outline | summary of the microscope apparatus which is the 1st Embodiment of this invention. It is a figure which shows the outline | summary of the microscope apparatus which is the 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Light source, 2 ... Collector lens, 3 ... Lens, 4 ... Excitation filter, 5 ... Dichroic mirror, 6 ... Barrier filter, 7, 7a-7c ... Lens, 8, 8a-8c ... Diffraction grating, 9, 9a-9c DESCRIPTION OF SYMBOLS 2nd objective lens, 10 ... Objective lens, 11 ... Sample, 12 ... Lens, 22a-22c ... Conjugate surface, 23 ... Image of diffraction grating 8, 24 ... Enlarged image, 25 ... Imaging device, 31 ... Light source image (pupil) Conjugate plane), 31a, 31b ... mirror, 32 ... light source image (pupil conjugate plane), 33 ... pupil conjugate plane, 42 ... control / calculation device, 43 ... image display device, M1, M2 ... galvanometer mirror, LG ... optical fiber

Claims (5)

  An illumination optical system that includes a plurality of optical paths and that illuminates by forming an image of a diffraction grating placed in each optical path on the surface of the observation object, and the observation object illuminated by the illumination optical system A microscope apparatus including an imaging optical system that forms an image as an intermediate image on the diffraction grating and a relay optical system that forms the image of the intermediate image, wherein the plurality of diffraction gratings The illumination optical system further switches and directs light from the light source to an arbitrary optical path of the plurality of optical paths, and guides the light to a diffraction grating disposed in the optical path. And second illumination light path switching means for guiding light from the illuminated diffraction grating to the surface of the object to be observed in synchronization with the first illumination light path switching means, and the imaging optical system Further illuminates the light from the object to be observed In synchronization with the first observation optical path switching means, the first observation optical path switching means for guiding the light from the illuminated diffraction grating to the relay optical system in synchronization with the first diffraction optical path. And a microscope apparatus.   An optical element constituting at least the illumination optical system from the first illumination optical path switching means to the object to be observed, and the imaging optical system from the observation object to the second observation light beam path switching means are constituted. An optical element is shared, the first illumination light path switching means and the second observation light path switching means are the same, and the second illumination light path switching means and the first observation light path switching The microscope apparatus according to claim 1, wherein the means is the same.   An illumination optical system that includes a plurality of optical paths and illuminates by forming an image of a diffraction grating placed in each optical path on the surface of the object to be observed, and the image of the object illuminated by the illumination optical system A plurality of diffraction gratings having different grating pattern directions, and the illumination optical system transmits light from a light source among the plurality of optical paths. The light from the illuminated diffraction grating is synchronized with the first illumination light path switching means and the first illumination light path switching means for guiding the light to the diffraction grating disposed in the optical path. And a second illumination light path switching means for guiding the light to the surface of the object to be observed.   The microscope apparatus according to any one of claims 1 to 3, wherein the diffraction grating is a transmission or reflection type spatial light modulation element.   The microscope apparatus includes: an imaging device that captures an image formed by the relay optical system; and a phase modulation unit that changes a phase of a grating pattern of the image of the diffraction grating formed on the surface of the object to be observed. The microscope apparatus according to claim 1, wherein an imaging time of the imaging apparatus is a phase N period (N is a natural number) of the lattice pattern.

Images