JP6690424B2 - Microscope device, observation method, and control program - Google Patents

Description

  The present invention relates to a microscope device, an observation method, and a control program.

  As one form of a super-resolution microscope, a structured illumination microscope that generates a super-resolution image of a sample by illuminating a sample with spatially modulated illumination light to obtain a modulated image and demodulating the modulated image ( SIM: Structured Illumination Microscopy) is known (for example, refer to Patent Document 1).

US Reissued Patent Invention No. 38307

  Super-resolution microscopes are expected to accelerate image acquisition.

  It is an object of the present invention to provide a microscope apparatus, an observation method, and a control program that can speed up image acquisition.

  According to the first aspect of the present invention, an acoustic wave standing wave is generated, and an optical modulator that splits incident light into a plurality of light beams is caused to interfere with at least a part of the plurality of light beams, and the sample is formed with interference fringes. An illumination optical system for illuminating the sample, an imaging optical system for forming an image of the sample irradiated with the interference fringes, an imaging unit for capturing the image of the sample formed by the imaging optical system, and at least one of the plurality of light beams. The control unit includes a phase adjustment unit that changes the phases of the two light beams, and the control unit controls the phase adjustment unit when the first drive signal is applied to the optical modulator to generate the sonic standing wave. The phase is changed according to the first pattern to cause the image capturing unit to capture the first image, the phase adjusting unit is configured to change the phase with the second pattern to cause the image capturing unit to capture the second image, and the optical modulator When a second drive signal is given to generate a sound wave standing wave, the phase adjustment unit sets the phase to the third pattern. The imaging unit captures the third image by changing the phase, the phase adjustment unit changes the phase in the fourth pattern, the imaging unit captures the fourth image, and the optical modulator receives the third drive signal. When the sound wave standing wave is generated by changing the phase in the fifth pattern by the phase adjustment unit, the imaging unit captures the fifth image, and the phase adjustment unit changes the phase by the sixth pattern. There is provided a microscope device that causes an imaging unit to capture a sixth image.

  According to the second aspect of the present invention, the acoustic wave standing wave is generated, the incident light is branched into a plurality of light fluxes by the optical modulator, and at least a part of the plurality of light fluxes is caused to interfere with each other to form an interference fringe. Illuminating the sample with, forming an image of the sample irradiated with the interference fringes, capturing an image of the formed sample, and changing the phase of at least one of the plurality of light beams. Controlling the generation of the sonic standing wave by the optical modulator, the change of the phase, and the imaging, the control providing the first drive signal to the optical modulator to cause the sonic standing wave to be generated. At this time, the phase is changed in the first pattern to capture the first image, the phase is changed in the second pattern to capture the second image, and the second drive signal is supplied to the optical modulator. When a standing wave is generated by applying it, the phase is changed in the third pattern. When a third image is captured, the phase is changed in a fourth pattern to capture the fourth image, and a third drive signal is given to the optical modulator to generate a sound wave standing wave, Changing the phase in a fifth pattern to capture a fifth image and changing the phase in a sixth pattern to generate a sixth image.

  According to the third aspect of the present invention, an acoustic wave standing wave is generated, and at least a part of the plurality of light fluxes is caused to interfere with the optical modulator that splits the incident light into the plurality of light fluxes, and the sample is formed with interference fringes. An illumination optical system for illuminating the sample, an imaging optical system for forming an image of the sample irradiated with the interference fringes, an imaging unit for capturing the image of the sample formed by the imaging optical system, and at least one of the plurality of light beams. A control program that causes a computer to control a microscope apparatus having a phase adjustment unit that changes the phases of two light fluxes, the control being such that a first drive signal is applied to the optical modulator to generate a sound wave standing wave. When this is done, the phase adjusting unit changes the phase in the first pattern to cause the image capturing unit to capture the first image, and the phase adjusting unit changes the phase in the second pattern to cause the image capturing unit to capture the second image. And give a second drive signal to the optical modulator. When a sound wave standing wave is generated, the phase adjusting unit changes the phase in the third pattern, causes the image capturing unit to capture the third image, and then causes the phase adjusting unit to change the phase in the fourth pattern. When the imaging unit is caused to capture a fourth image and the optical modulator is given a third drive signal to generate a sonic standing wave, the phase adjustment unit is caused to change the phase in the fifth pattern. A control program is provided that includes causing the imaging unit to capture the fifth image and causing the phase adjustment unit to change the phase in the sixth pattern to cause the imaging unit to capture the sixth image.

  According to the present invention, it is possible to provide a microscope apparatus, an observation method, and a control program that can speed up image acquisition.

It is a figure which shows the microscope apparatus of this embodiment. It is a figure which shows the branch part which concerns on this embodiment. It is a figure which shows the optical path from a branch part to a field stop. It is a figure which shows the example of the mask seen from the direction of the optical axis of an illumination optical system. It is a figure which shows the time change of distribution of the refractive index in the sound wave propagation path of an optical modulator. It is a conceptual diagram which shows the correspondence of the pattern of a standing wave and the interference fringe of a 1st-order diffracted light. FIG. 7A is a diagram showing a light incident region in a sound wave propagation path, and FIG. 8B is a diagram showing a shift of interference fringes due to a change in the number of waves. It is a figure which shows each standing wave and interference fringe of the first half and the latter half. It is a flow chart which shows operation in 2D-SIM mode. It is a figure which shows the optical path from a light source part to a sample in 3D-SIM mode. It is a figure which shows the optical path from a branch part to a field stop, and a drive signal. It is a figure which shows a standing wave and the interference fringe before and behind phase adjustment. It is a figure which shows the drive signal in several frames, a standing wave, and an interference fringe. It is a flow chart which shows operation in 3D-SIM mode. It is a figure which shows the setting conditions regarding one direction of an interference fringe. It is a figure which shows the other example of the drive signal of a phase modulator. It is a figure which shows the example of the amount of phase adjustments by a phase modulator. It is a figure which shows the time change of the refractive index of the antinode of a standing wave, and the phase modulation amount of 0th-order diffracted light. It is a figure which shows the other example of the drive signal of a phase modulator. It is a figure which shows the other example of an optical modulator.

  Next, embodiments will be described with reference to the drawings. FIG. 1 is a diagram showing a microscope apparatus 1 of this embodiment. The microscope device 1 includes a structured lighting device 2, an imaging device 3, a control device 4, and a display device 5. The microscope apparatus 1 is, for example, a fluorescence microscope, and is used for observing a sample X containing cells and the like that have been fluorescently stained in advance. The sample X is held on, for example, a stage (not shown).

  The microscope device 1 operates as follows in brief. The structured illumination device 2 generates interference fringes and illuminates the sample X with the interference fringes. The image pickup device 3 picks up an image (sample image, moire image) of the sample X modulated by the interference fringes. The moire image contains the information of the sample X and has a lower spatial frequency than the sample X, and thus can be imaged through the optical system of the imaging device 3. The control device 4 controls each part of the microscope device 1. The control device 4 controls the structured lighting device 2 to switch the phase and direction of the interference fringes into a plurality of states. The control device 4 controls the imaging device 3 to capture an image of the sample X in each of a plurality of states of interference fringes and acquires a plurality of images. The control device 4 can generate a super-resolution image that exceeds the resolution limit of the optical system of the imaging device 3 by performing demodulation processing using a plurality of images.

  The microscope apparatus 1 includes a 2D-SIM mode for generating a two-dimensional super-resolution image of the surface of the sample X to be observed (hereinafter referred to as a sample surface) and a three-dimensional image including information in a direction perpendicular to the sample surface. 3D-SIM mode for generating a typical super-resolution image. First, the 2D-SIM mode will be described, and then the 3D-SIM mode will be described.

  The structured lighting device 2 includes a light source unit 10 and an illumination optical system 11. The light source unit 10 includes a light source such as a laser element, and emits coherent light in a predetermined wavelength band. In the case of a fluorescence microscope, the predetermined wavelength band is set to the wavelength band including the excitation wavelength of the sample X. In addition, at least a part of the light source unit 10 may not be included in the structured lighting device 2. For example, the light source unit 10 may be replaceably (attachable or detachable) provided in the structured lighting device 2. For example, the light source unit 10 may be attached to the structured lighting device 2 during observation with the microscope device 1.

The illumination optical system 11 includes a collector lens 12 and a branching unit 13 on the light emitting side of the light source unit 10 . The light emitted from the light source unit 10 is converted into parallel light by the collector lens 12 and enters the branching unit 13. The branching unit 13 branches the incident light into a plurality of light fluxes. The branching unit 13 branches the light from the light source unit 10 into a plurality of light beams by diffraction. In addition, in FIG. 1, the 0th-order diffracted light (shown by a solid line), the + 1st-order diffracted light (shown by a broken line), and the -1st-order diffracted light (shown by a two-dot chain line) are representatively shown among a plurality of light fluxes. In the following description, when referring to one or both of the + 1st-order diffracted light and the −third-order diffracted light of the diffracted light, the term “first-order diffracted light” is simply used.

    FIG. 2A is a diagram showing the branching unit 13 according to this embodiment, and FIG. 2B is a diagram showing an operation of the branching unit 13. The branching unit 13 is an optical modulator (eg, spatial light modulator) including, for example, an acousto-optic modulator (hereinafter, abbreviated as AOM). The branch unit 13 includes an acousto-optic medium 22, a transducer 23a, a transducer 23b, and a transducer 23c.

  The acousto-optic medium 22 is a prismatic member made of, for example, quartz glass, tellurite glass, heavy flint glass, flint glass, or the like. The acousto-optic medium 22 has a pair of side surfaces 22a, a pair of side surfaces 22b, and a pair of side surfaces 22c around the optical axis 11a of the illumination optical system 11. The pair of side surfaces 22a, the pair of side surfaces 22b, and the pair of side surfaces 22c each include two surfaces facing each other and parallel to each other, and are arranged, for example, in parallel with the optical axis of the illumination optical system 11. The angle formed by the side surface 22a and the side surface 22b is 120 °, for example, and the angle formed by the side surface 22a and the side surface 22c is 120 °, for example.

  The acousto-optic medium 22 has a sound wave propagation path R therein. The sound wave propagation path R includes a sound wave propagation path Ra between the pair of side surfaces 22a, a sound wave propagation path Rb between the pair of side surfaces 22b, and a sound wave propagation path Rc between the pair of side surfaces 22c. The sound wave propagation paths Ra to Rc are partially overlapped when viewed from the direction of the optical axis of the illumination optical system 11. Hereinafter, a portion where the sound wave propagation path Ra, the sound wave propagation path Rb, and the sound wave propagation path Rc all overlap is arranged around the optical axis 11a of the illumination optical system 11 and its periphery. At least a part of this overlapping portion is an incident area Sa on which the light from the light source unit 10 is incident.

  The transducer 23a is provided on the side surface 22a. The transducer 23a is, for example, an ultrasonic transducer, and includes a piezoelectric body and two electrodes that sandwich the piezoelectric body. The transducer 23a is joined to the side surface 22a of the acousto-optic medium 22 via one electrode. When a high-frequency sinusoidal AC voltage (drive signal) is applied between the two electrodes of the transducer 23a, the piezoelectric body vibrates (expands and contracts) in the thickness direction (the normal direction of the side surface 22a). As a result, a plane ultrasonic wave that reciprocates in the sound wave propagation path Ra is generated, and this plane ultrasonic wave is a sound wave standing wave (hereinafter abbreviated as a standing wave) when the frequency of the AC voltage is set to a specific frequency. ). Due to this standing wave, a coarse and dense periodic distribution (for example, a sinusoidal distribution) is generated in the sound wave propagation path Ra, and the part of the acoustic wave propagation path Ra in the acoustooptic medium 22 is perpendicular to the propagation direction of the ultrasonic waves. It becomes a phase type diffraction grating with a phase grating. Hereinafter, the propagation direction of ultrasonic waves in the sound wave propagation path Ra is referred to as a first direction.

  The transducer 23b is provided on the side surface 22b. The transducer 23b has the same configuration as the transducer 23a, and includes a piezoelectric body and two electrodes. The transducer 23b generates a standing wave in the sound wave propagation path Rb by applying a sinusoidal AC voltage (drive signal) having a predetermined frequency between the two electrodes. As a result, the sound wave propagation path Rb becomes a phase type diffraction grating having a phase grating perpendicular to the propagation direction of ultrasonic waves. Hereinafter, the propagation direction of ultrasonic waves in the sound wave propagation path Rb is referred to as a second direction. The second direction is a direction that makes an angle of 60 ° with the first direction.

  The transducer 23c is provided on the side surface 22c. The transducer 23c has the same configuration as the transducer 23a and includes a piezoelectric body and two electrodes. The transducer 23c generates a standing wave in the sound wave propagation path Rc by applying a sinusoidal AC voltage (drive signal) having a predetermined frequency between the two electrodes. As a result, the sound wave propagation path Rc becomes a phase type diffraction grating having a phase grating perpendicular to the propagation direction of ultrasonic waves. Hereinafter, the propagation direction of ultrasonic waves in the sound wave propagation path Rc is referred to as a third direction. The third direction is a direction that forms an angle of 60 ° with the first direction in the opposite direction to the second direction.

  The branch unit 13 is driven by the drive unit 21. The drive unit 21 includes a power supply circuit 25 and a switch 26. The power supply circuit 25 is connected to the AC power supply 27. The power supply circuit 25 generates a drive signal by the power supplied from the AC power supply 27. The power supply circuit 25 is controlled by the control device 4 and generates a drive signal having a frequency designated by the control device 4. The power supply circuit 25 outputs the generated drive signal to the switch 26.

  The switch 26 has an input-side terminal 28 and output-side terminals 29a to 29c. The switch 26 is controlled by the control device 4 and connects (sets into conduction) a terminal designated by the control device 4 among the output side terminals 29a to 29c and the input side terminal. The switch 26 includes, for example, a transistor, and can electrically switch between a conductive state and an insulated state between the input-side terminal 28 and the output-side terminals 29a to 29c. The terminal 29a is connected to one electrode of the transducer 23a. The terminal 29b is connected to one electrode of the transducer 23b. The terminal 29c is connected to one electrode of the transducer 23c. The other electrodes of the transducers 23a to 23c are connected to the reference potential (ground) of the drive signal.

  When the drive signal is supplied from the power supply circuit 25 to the terminal 28 of the switch 26 and the terminal 28 is connected to the terminal 29a, the transducer 23a is driven by this drive signal and the sound wave propagation path Ra becomes a phase type diffraction grating. . In this state, as shown in FIG. 2B, the light incident on the sound wave propagation path Ra is branched in the first direction by diffraction. First-order and higher-order diffracted light (eg, ± first-order diffracted light) is deflected in the first direction with respect to the 0th-order diffracted light. In the following description, the direction in which the 1st or higher order diffracted light is deflected with respect to the 0th order diffracted light is referred to as the diffraction direction.

  Further, when the drive signal is supplied from the power supply circuit 25 to the terminal 28 of the switch 26 and the terminal 28 is connected to the terminal 29b, the sound wave propagation path Rb becomes a phase type diffraction grating, and the light incident on the incident area Sa is incident. Is branched in the second direction by diffraction. Similarly, when the drive signal is supplied from the power supply circuit 25 to the terminal 28 of the switch 26 and the terminal 28 is connected to the terminal 29c, the sound wave propagation path Rc becomes a phase type diffraction grating and is incident on the incident area Sa. The light splits in the third direction due to diffraction.

  Such a branch 13 can change the pitch of the periodic structure (the period of the density of the acoustooptic medium 22) (the interval between the maximum positions of the density) according to the frequency of the standing wave. Further, the branching unit 13 can electrically select (switch) which of the sound wave propagation paths Ra to Rc is to be made effective (whether to have a periodic structure by standing waves) by the switch 26. The branching unit 13 can electrically switch the deflection direction (diffraction direction) of the ± first-order diffracted light with respect to the 0th-order diffracted light.

  The branch unit 13 may include at least a part of the drive unit 21. Further, the branching portion 13 may include a diffraction grating having a fixed periodic structure. Examples of such a diffraction grating include a quartz grating in which a plurality of grooves are cut in a quartz plate. In order to change the diffraction direction with such a diffraction grating, for example, the diffraction grating may be rotated in a plane intersecting with the light from the light source unit 10. Further, the branching section 13 may be a spatial light modulator using a liquid crystal element or the like instead of the AOM.

  Returning to the description of FIG. 1, the illumination optical system 11 includes a lens 30, a phase adjustment unit 14, a lens 31, a field stop 32, a lens 33, a filter 34, a dichroic mirror 35, and an objective lens on the light exit side of the branch unit 13. 36 is provided.

  FIG. 3 is a diagram showing an optical path from the branch unit 13 to the field stop 32. The diffracted light generated in the branch portion 13 enters the lens 30. The lens 30 is arranged, for example, such that the front focal position of the lens 30 substantially coincides with the position of the branch portion 13. Diffracted light of the same order among the plurality of light beams branched by the branching unit 13 is condensed at the same position to form a so-called pupil conjugate plane P1. For example, the lens 30 focuses the 0th-order diffracted light (shown by the solid line in FIG. 3) generated at the branching portion 13 on the optical axis 11a of the illumination optical system 11 (front focus position). The lens 30 focuses the + 1st order diffracted light (shown by the broken line in FIG. 3) generated at the branching portion 13 at a position away from the optical axis 11a of the illumination optical system 11. The lens 30 collects the −1st-order diffracted light (shown by a chain double-dashed line in FIG. 3) generated at the branching portion 13 at a position symmetrical with the + 1st-order diffracted light with respect to the optical axis 11 a of the illumination optical system 11.

  The phase adjuster 14 allows the diffracted light used to generate the interference fringes and blocks the diffracted light not used to generate the interference fringes. In the 2D-SIM mode, for example, the illumination optical system 11 generates interference fringes of the + 1st-order diffracted light and the −1st-order diffracted light, and does not use the 0th-order diffracted light and the diffracted light of the 2nd-order or higher. In the 2D-SIM mode, the phase adjustment unit 14 allows the 1st-order diffracted light to pass and blocks the 0th-order diffracted light and the 2nd-order and higher-order diffracted lights. In the 3D-SIM mode, the illumination optical system 11 generates, for example, a combined fringe of the interference fringes of the + 1st-order diffracted light and the −1st-order diffracted light and the interference fringes of the 0th-order diffracted light and the 1st-order diffracted light. In the 3D-SIM mode, the phase adjustment unit 14 allows the 0th-order diffracted light and the 1st-order diffracted light to pass through and blocks the 2nd-order and higher-order diffracted lights.

  Further, the phase adjusting unit 14 changes the phase of at least one light beam of the plurality of light beams branched by the branching unit 13. The phase adjustment unit 14 appropriately adjusts the phase of the diffracted light used to generate the interference fringes. For example, the phase adjustment unit 14 adjusts (changes) the relative phases of the 0th-order diffracted light and the 1st-order diffracted light in the 3D-SIM mode. The phase adjusting unit 14 may or may not adjust the relative phases of the + 1st order diffracted light and the −1st order diffracted light in the 2D-SIM mode.

  The phase adjusting unit 14 is arranged at a position where the optical path of the 0th-order diffracted light and the optical path of the 1st-order diffracted light are separated. The phase adjuster 14 is arranged, for example, on the pupil conjugate plane P1. The pupil conjugate plane P1 is a plane optically conjugate with the pupil plane P0 (the rear focal plane of the objective lens 36). The phase adjustment unit 14 is arranged at the back focal position of the lens 30, for example. The phase adjustment unit 14 according to this embodiment includes a mask 40, a phase modulator 41, a correction block 42, and a drive unit 43. The mask 40 and the phase modulator 41 block light not used for interference fringes. The phase modulator 41 and the correction block 42 can change the relative phases of the 0th-order diffracted light and the 1st-order diffracted light.

  The phase modulator 41 is arranged on the optical axis 11 a of the illumination optical system 11. The phase modulator 41 is arranged at a position where the 0th-order diffracted light is condensed. The phase modulator 41 modulates the phase of incident light (here, the 0th-order diffracted light) with a modulation width π in the time direction. The phase modulator 41 includes, for example, an electro-optic modulator that uses the Kerr effect (hereinafter abbreviated as EOM), an EO modulator that uses the Pockels effect, or an AOM.

  By the way, the portion of the phase modulator 41 through which the 0th-order diffracted light passes has, for example, a refractive index different from that of the atmospheric gas, and has a different optical distance from the optical path in the atmospheric gas. Therefore, the 0th-order diffracted light that has passed through the phase modulator 41 has a phase offset as compared with the case where the 0th-order diffracted light passes through the atmosphere gas instead of the phase modulator 41. The correction block 42 cancels at least a part of the phase difference between the 0th-order diffracted light and the 1st-order diffracted light caused by this offset. The correction block 42 is provided at each of the position where the −1st-order diffracted light is condensed and the position where the + 1st-order diffracted light is condensed. The correction block 42 is a member made of a material that transmits the first-order diffracted light. The optical distance of the optical path in the correction block 42 is substantially the same as the optical distance of the optical path in the phase modulator 41 when the phase modulation amount by the phase modulator 41 is a reference value (eg, lower limit value), for example. Is set to.

  The phase modulator 41 is driven by the drive unit 43. The drive unit 43 includes a power supply circuit 25 and a phase adjustment circuit 44. The power supply circuit 25 is shared with, for example, the drive unit 21 of the branch unit 13, but may be provided separately from the drive unit 21. The drive unit 43 is controlled by the control device 4 and controls the phase modulator 41. The adjustment of the phase using the phase modulator 41 will be described later together with the description of the 3D-SIM mode.

  4A to 4C are diagrams showing examples of the mask 40 viewed from the direction of the optical axis of the illumination optical system 11. Each of the masks 40 shown in FIGS. 4A to 4C is a so-called aperture stop. The mask 40 defines the angle of the light beam incident on the sample X. The mask 40 is arranged, for example, on the pupil conjugate plane P1 (the position of the aperture stop).

  The mask 40 in FIG. 4A has openings 40a to 40c. The opening 40a is arranged at a position where the 0th-order diffracted light enters. A phase modulator 41 is provided in the opening 40a. For example, the phase modulator 41 uses the mask 40 as a support substrate and is arranged inside the opening 40a. The phase modulator 41 also serves as, for example, a shutter unit that switches between a state of passing the 0th-order diffracted light and a state of blocking the 0th-order diffracted light. The opening 40b is arranged at a position where the + 1st order diffracted light enters, and the opening 40c is arranged at a position where the −1st order diffracted light enters. The correction block 42 is provided in the opening 40b. For example, the correction block 42 uses the mask 40 as a support substrate and is arranged inside the opening 40b and inside the opening 40c, respectively.

  Note that, in FIG. 4A, the diffraction direction by the branch portion 13 is set to the first direction, and when the diffraction direction is switched, the position where the first-order diffracted light is incident on the mask 40 is the illumination optical system. The system 11 is switched to a position rotated around the optical axis 11a. When switching the diffraction direction, the mask 40 is rotatably provided around the optical axis 11a of the illumination optical system 11, and is driven so that the opening 40b and the opening 40c are arranged at the position where the first-order diffracted light enters. It

  The mask 40 in FIG. 4B has openings 40a to 40g. The opening 40d and the opening 40e are arranged at positions where the first-order diffracted light enters, for example, when the diffraction direction is set to the second direction. The opening 40d and the opening 40e are arranged, for example, at positions where the opening 40b and the opening 40c are rotated by 60 ° around the optical axis 11a of the illumination optical system 11. The opening 40f and the opening 40g are arranged at positions where the first-order diffracted light enters, for example, when the diffraction direction is set to the third direction. The opening 40f and the opening 40g are arranged, for example, at positions where the opening 40b and the opening 40c are rotated by −60 ° around the optical axis 11a of the illumination optical system 11. When this mask 40 is used, the correction block 42 is provided in each of the openings 40a to 40g, for example. The mask 40 can pass the first-order diffracted light when the diffraction direction is set to any of the first direction, the second direction, and the third direction. Therefore, the mechanism for rotating the mask 40 according to the diffraction direction can be omitted.

  The mask 40 in FIG. 4C has an annular frame shape and has an opening 40h. A phase modulator 41 and a correction block 42 are provided in the opening 40h. For example, the phase modulator 41 is provided in a region including the optical axis, and the correction block 42 is in contact with the phase modulator 41 and the edge defining the opening 40h (the inner side surface of the opening 40h). It is provided. That is, the correction block 42 may be supported by the mask 40, and the phase modulator 41 may be supported by the correction block 42. The mask 40 can pass the first-order diffracted light when the diffraction direction is set to any of the first direction, the second direction, and the third direction. Therefore, the mechanism for rotating the mask 40 according to the diffraction direction can be omitted. The mask 40 may include a stay that extends inward from the annular frame-shaped portion and supports the phase modulator 41. This stay is arranged at a position where the first-order diffracted light does not enter, for example, when the diffraction direction is set to any of the first direction, the second direction, and the third direction.

  At least one of the phase modulator 41 and the correction block 42 may be supported by a member different from the mask 40, or may not be supported by the mask 40. Wirings, cables, and the like that supply electric power and signals to the phase modulator 41 may be supported by the mask 40, or may be supported by a member other than the mask 40. Further, the phase adjusting unit 14 may not include the mask 40. For example, the mask 40 may be included in the illumination optical system 11. The phase modulator 41 does not have to serve as a shutter unit that can switch between passing and blocking the 0th-order diffracted light. Such a shutter unit may be provided separately from the phase modulator 41, or may be included in the illumination optical system 11.

  Returning to the description of FIG. 1, the light that has passed through the phase adjustment unit 14 enters the lens 31. The lens 30 and the lens 31 form a surface that is optically conjugate with the branch portion 13 (hereinafter, referred to as an intermediate image surface 31a). The field stop 32 is arranged on the intermediate image plane 31a, for example. The field stop 32 defines a range (illumination field, illumination area) in which light is emitted from the illumination optical system 11 to the sample X in a plane perpendicular to the optical axis 11a of the illumination optical system 11. The light that has passed through the field stop 32 enters the lens 33. The lens 33 condenses the + 1st-order diffracted light emitted from the branching unit 13 at a position away from the optical axis 11a on the pupil plane P0 (the rear focal plane of the objective lens 36). Further, the lens 33 is symmetric to the + 1st-order diffracted light with respect to the optical axis of the illumination optical system 11 on the pupil plane P0 (the rear focal plane of the objective lens 36) for the -1st-order diffracted light emitted from the branching portion 13. Focus on the position.

  The light passing through the lens 33 enters the filter 34. The filter 34 is, for example, an excitation filter. The filter 34 has a characteristic that light in a wavelength band including the excitation wavelength of the fluorescent substance contained in the sample X selectively passes therethrough. The filter 34 blocks, for example, at least a part of wavelengths other than the excitation wavelength in the light from the light source unit 10, stray light, external light, and the like. The light passing through the filter 34 enters the dichroic mirror 35. The dichroic mirror 35 has a characteristic that light in a wavelength band including the excitation wavelength of the fluorescent substance contained in the sample X is reflected and light in a predetermined wavelength band (for example, fluorescence) of the light from the sample X passes through. The light from the filter 34 is reflected by the dichroic mirror 35 and enters the objective lens 36.

  The objective lens 36 and the lens 33 form a surface optically conjugate with the intermediate image surface 31a, that is, a surface optically conjugate with the branching portion 13. As described above, the + 1st-order diffracted light is condensed on the pupil plane P0 of the objective lens 36 at a position away from the optical axis 11a. Further, the −1st order diffracted light is condensed on the pupil plane P0 at a position symmetrical with the + 1st order diffracted light with respect to the optical axis 11a.

  At the time of observation, the portion of the sample X to be observed is arranged on the front focal plane of the objective lens 36. Interference fringes are formed on the sample X due to the interference between the + 1st-order diffracted light and the -1st-order diffracted light. This interference fringe has, for example, a periodic distribution of light intensity in a direction perpendicular to the optical axis 11a of the illumination optical system 11. This interference fringe includes a bright portion and a dark portion which are periodically arranged in a direction corresponding to the diffraction direction of the branch portion 13. In the portion of the sample X arranged in the bright part of the interference fringe, the fluorescent substance is excited to emit fluorescence. This fluorescence image is a moire image (modulation image) of the interference fringes formed by the illumination optical system 11 and the pattern of the distribution of the fluorescent substance in the sample X (hereinafter referred to as the sample pattern). The imaging device 3 acquires an image of this moire image (modulation image).

  The imaging device 3 includes an imaging optical system 50 (imaging optical system) and an imaging element 51. The imaging optical system 50 includes an objective lens 36, a dichroic mirror 35, a filter 52, and a lens 53. The imaging optical system 50 shares the objective lens 36 and the dichroic mirror 35 with the illumination optical system 11. The light (observation light) from the sample X enters the objective lens 36 to be collimated, and then enters the filter 52 through the dichroic mirror 35.

  The filter 52 is, for example, a fluorescent filter. The filter 52 has a characteristic that light (eg, fluorescence) in a predetermined wavelength band of the light from the sample X selectively passes therethrough. The filter 52 blocks, for example, the light emitted from the illumination optical system 11 to the sample X and reflected by the sample X, external light, stray light, and the like. The light passing through the filter 52 enters the lens 53. The lens 53 and the objective lens 36 form a plane (image plane) optically conjugate with the front focal plane (object plane) of the objective lens 36. An image (moiré image) due to fluorescence from the sample X is formed on this image plane.

  The image pickup element 51 (image pickup unit) picks up an image of the sample X formed by the image pickup optical system 50. The image sensor 51 includes a two-dimensional image sensor such as a CCD image sensor or a CMOS image sensor. The imaging element 51 has, for example, a plurality of pixels arranged two-dimensionally, and a photoelectric conversion element such as a photodiode is arranged in each pixel. The image pickup device 51 converts the charges accumulated in the photoelectric conversion device by light from the sample X into digital data, and outputs digital format image data, for example. The charge accumulation time per frame by the image sensor 51 is, for example, about 1/30 second, about 1/60 second, or the like.

  By the way, the spatial frequency of the moire image corresponds to the difference between the spatial frequency of the sample pattern and the spatial frequency of the interference fringes formed by the illumination optical system 11, and is lower than the spatial frequency of the sample pattern. Therefore, when the moire image acquired within the resolution limit of the imaging optical system 50 is demodulated, an image (super-resolution image) of the sample X having a resolution exceeding the resolution limit of the imaging optical system 50 is obtained. The control device 4 displays the super-resolution image on the display device 5, for example.

The microscope device 1 includes a demodulation unit 55 that performs a demodulation process using the image acquired by the imaging device 3. The demodulation unit 55 is provided, for example, in the control device 4, but may be provided separately from the control device 4. The demodulation unit 55 can generate a restored image by using a plurality of modulation images captured with the directions and directions of the interference fringes different from each other. The demodulation unit 55 may perform the demodulation process by, for example, the method disclosed in US Pat. No. 8,115,806, but is not limited to this method.

  Further, the control device 4 includes a control unit 56 that controls each unit of the microscope device 1. The control unit 56 controls the generation of the sound wave standing wave by the branching unit 13, the change (adjustment) of the phase by the phase adjusting unit 14, and the imaging by the image sensor 51. The control unit 56 changes the period direction of the interference fringes by controlling the switch 26 of the branch unit 13 shown in FIG. 2A to switch the diffraction direction. Further, the control device 4 controls the power supply circuit 25 to change the phase of the interference fringe. The mechanism for controlling the phase of the interference fringes will be described below.

  FIG. 5 is a diagram showing the change over time in the distribution of the refractive index in the sound wave propagation path R of the branch section 13. In FIG. 5, the time t at which the refractive index of the sound wave propagation path is uniform is set to t = 0, and one cycle of the vibration of the antinode of the standing wave is set to T, and at each time from time t = 0 to T / 8. The refractive index distribution is shown. The cycle T corresponds to the reciprocal of the frequency of the AC voltage supplied to the branch unit 13. At each time point, the horizontal axis represents the position in the propagation direction of the sound wave propagation path, and the vertical axis represents the refractive index. In the propagation direction of the sound wave propagation path, the position of the node whose refractive index does not change with time is indicated by a circle, the position of one antinode between the two nodes is indicated by the symbol a, and the position of the antinode adjacent to the antinode a is indicated by the symbol. Indicated by b.

  The refractive index of the antinode a increases from time t = 0 and reaches a maximum at time t = 2T / 8. The refractive index of the abdomen a decreases from the time t = 2T / 8, becomes the same value as the time t = 0 at the time t = 4T / 8, and becomes the minimum at the time t = 6T / 8. The refractive index of the abdomen a increases from time t = 6T / 8, and becomes the same value as time t = 0 at time t = 8T / 8. In the following description, the period from time t = 0 to time t = 4T / 8 is called the first half period, and the period from time t = 4T / 8 to time t = 8T / 8 is called the second half period. The refractive index of the abdomen a is equal to or higher than the refractive index of the node in the first half, and is equal to or lower than the refractive index of the node in the second half.

  The refractive index of the antinode b exhibits the opposite behavior to the refractive index of the antinode a. The refractive index of the abdomen b decreases from the time t = 0, reaches a minimum value at the time t = 2T / 8, then starts to increase, and becomes the same value as the time t = 0 at the time t = 4T / 8. Further, the refractive index of the abdomen b increases from time t = 4T / 8, reaches a maximum at time t = 6T / 8, and then starts to decrease, and has the same value as time t = 0 at time t = 8T / 8. become. The refractive index of the abdomen b is equal to or lower than the refractive index of the node in the first half period, and is equal to or higher than the refractive index of the node in the second half period.

  Next, the relationship between the standing wave pattern and the interference fringes will be described. FIG. 6 is a conceptual diagram showing the correspondence between the standing wave pattern at a certain time and the interference fringes of the first-order diffracted light. In the upper part of FIG. 6, the portion where the refractive index is less than or equal to the refractive index of the node due to the standing wave is shown by the bright portion, and the portion where the refractive index is greater than or equal to the refractive index of the node due to the standing wave is shown at the dark portion. One wave from the one dark part peak to the next dark part peak is defined as one wave, and the number is called the number of standing waves. In addition, in the interference fringes, a set of one dark portion and one bright portion is defined as one fringe, and the number is called the number of fringes of the interference fringe. In the case of the interference fringes of the + 1st order diffracted light and the −1st order diffracted light, the number of fringes of the interference fringes is twice the number of standing waves.

In the lower part of FIG. _6, f _-1, showing a standing wave when driven at _{f 0,} f _+1, the frequency _f -1 respectively wave branching part 13 of _{the, f 0,} f _+1. The symbol L indicates the length of the sound wave propagation path R. When the number of standing waves is changed by 1/2 such as 2, (2 + 1/2) and 3, the number of interference fringes changes by 1, such as 4, 5, and 6. Here, the phase of the interference fringe corresponding to a portion (shown by a white arrow in FIG. 6) displaced from the one end of the sound wave propagation path R by L / 2 is _{− in} the case of the frequency f ₋₁ as compared with the case of the frequency f _0. It is deviated by π, and is deviated by π in the case of the frequency f ₊₁ compared with the case of the frequency f ₀ . Further, the phase of the interference fringe corresponding to a portion (shown by a black arrow in FIG. 6) shifted from one end of the sound wave propagation path R by L / 3 is −2π in the case of the frequency f ₋₁ compared with the case of the frequency f _0. It is shifted by / 3, and is shifted by 2π / 3 in the case of the frequency f ₊₁ compared with the case of the frequency f ₀ . Therefore, when the position of the incident area Sa shown in FIG. 2A is set at a position displaced from the end of the sound wave propagation path R by L / 3, the frequencies of the drive signals are f ₋₁ , f ₀ , and f ₊₁ . By switching with, the phase of the interference fringe can be set in three ways.

For example, when the number of stripes on the sample X is 200, the number of waves in the branching portion 13 is 100, and when the length (L) of the sound wave propagation path R is 30 mm, the frequency of the drive signal (eg, f ₀ ) is , 80 MHz. When the number of stripes is reduced by 1 with respect to such a reference state, the number of waves in the branching section 13 becomes 99.5, and the frequency (eg, f ₋₁ ) of the drive signal becomes 79.946 MHz. When the branching section 13 is driven by the driving signal having such a frequency, the phase of the interference fringes on the sample X is shifted by -2/3. Further, when the number of stripes is increased by 1 with respect to the reference state, the number of waves in the branching unit 13 becomes 100.5, and the frequency (eg, f ₊₁ ) of the drive signal becomes 80.054 MHz. When the branching section 13 is driven by the drive signal having such a frequency, the phase of the interference fringes on the sample X is shifted by + 2π / 3.

  FIG. 7A is a diagram showing a light incident region Sa on the sound wave propagation path R, and FIG. 7B is a diagram showing a shift of interference fringes due to a change in the number of waves. The incident area Sa corresponds to a spot formed on the sound wave propagation path R by the light from the light source unit 10. The center of the incident area Sa is set at a position separated from the end of the sound wave propagation path R by a distance D (eg, D = L / 3), for example. As shown in FIG. 7B, when the number of standing waves changes by 1/2, the number of interference fringes Sb formed on the sample X changes by 1, and the positions of the bright portion and the dark portion are displaced. In FIG. 7B, the pattern of the interference fringes Sb before the change of the number of waves (the deviation of the number of fringes is 0) is shown by a broken line, and the pattern of the interference fringes Sb after the change of the number of waves is shown by a solid line. The difference is highlighted.

  The length L of the sound wave propagation path R is set to be sufficiently larger than the diameter φ of the incident area Sa so that the deviation of the number of fringes of the interference fringe Sb is equal to or less than the allowable value. For example, the length L of the sound wave propagation path R and the diameter φ of the incident region Sa are set so as to satisfy the relationship of φ / L <δ with respect to the allowable amount δ of the fringe number deviation of the interference fringes Sb. It

  In the image restoration processing, a plurality of frequency components are superposed as an image, and the phases of the respective frequency components at that time need to match as much as possible. The allowable value of the error is, for example, 0.5 radian. This is 0.079 when converted to the number of stripes. When the sound wave propagation path R is installed so that the position of the center of the visual field is the distance D (D = L / 3) from the end of the sound wave propagation path R as described above, it is shown in FIG. Although it can be considered that the value dp0 obtained by converting the shift of the interference fringe Sb in the central portion into the number of fringes can be ignored, the shift dp1 at the field edge in that case is substantially equal at both ends, and is in opposite directions. Since it is necessary for each to be within the above-mentioned 0.079 lines, the allowable value δ of the deviation of the number of fringes of the interference fringes Sb in the entire field of view is doubled to 0.15. Rounded down to the second decimal place.

On the other hand, since the number of sound waves changes by 0.5 in the entire sound wave propagation path R, the change in the number of stripes of the interference fringe Sb changes by 1 in the range corresponding to the entire sound wave propagation path R. Therefore, within the range of the diameter φ of the incident area Sa, the change 2dp1 in the number of fringes Sb becomes equal to φ / L. Therefore, it is necessary to satisfy φ / L ≦ 0.15. For example, when the diameter φ of the incident area Sa is 4 mm and the length L of the sound wave propagation path R is 30 mm, φ / L = 0.133, and the above condition can be satisfied.

  By the way, when the acousto-optic module is used for the branching section 13, the refractive index at each position of the standing wave changes between the first half and the second half as shown in FIG. FIG. 8 is a diagram showing standing waves and interference fringes in the first half and the second half, respectively. As shown in FIG. 8, the pattern of the standing wave in the latter half of the period is in a state in which the phase is shifted by π from the pattern of the standing wave in the first half of the period (see FIG. 5). In the case of interference fringes of the −1st order diffracted light and the + 1st order diffracted light, the number of fringes of the interference fringes is twice the number of standing waves. , The phase is shifted by 2π. Therefore, the light part in the first half becomes a light part having a phase shift of 2π in the second half, and the dark part in the first half becomes a dark part having a phase shift of 2π in the second half. As described above, in the 2D-SIM mode, the phase of the interference fringe does not substantially change between the first half and the second half.

  Next, an observation method in the 2D-SIM mode will be described based on the operation of the microscope apparatus 1. FIG. 9 is a flowchart showing the operation of the microscope apparatus 1 in the 2D-SIM mode (see FIGS. 1 to 3 for each part of the microscope apparatus 1). In step S11, the control unit 56 of the control device 4 controls the switch 26 and sets the connection destination of the switch 26 to the transducer 23a. As a result, the periodic direction of the interference fringes Sb is set to the direction corresponding to the first direction.

In step S12, the control unit 56 controls the power supply circuit 25 to set the frequency of the drive signal to f ₋₁ . As a result, the phase shift amount of the interference fringe Sb is set to −2π / 3. In step S13, the control unit 56 controls the imaging device 3 to image the sample X and acquires the image data I ₋₁ in the state where the phase shift amount is set to −2π / 3.

In step S14, the control unit 56 controls the power supply circuit 25 to set the frequency of the drive signal to f ₀ . As a result, the phase shift amount of the interference fringe Sb is set to zero. In step S15, the control unit 56 controls the imaging device 3 to image the sample X and acquires the image data I ₀ in the state where the phase shift amount is set to ₀ .

In step S16, the control unit 56 controls the power supply circuit 25 to set the frequency of the drive signal to f _{+ 1} . As a result, the phase shift amount of the interference fringe Sb is set to + 2π / 3. In step S17, the control unit 56 controls the imaging device 3 to image the sample X and acquires the image data I ₊₁ in the state where the phase shift amount is set to + 2π / 3. As described above, the microscope apparatus 1 sets the periodic direction of the interference fringes to the first direction, switches the phase to three types, and acquires three images.

  In step S18, the control unit 56 determines whether or not the periodic directions of the interference fringes Sb have been set in all three directions. When the control unit 56 determines that the imaging according to the three cycle directions of the interference fringes is not completed (step S18; No), the connection destination of the switch 26 is switched to the next transducer in step S19. Further, after the processing of step 19 is finished, the control unit 56 repeats the processing of steps S12 to S17 in a state where the cycle direction of the interference fringes is changed (eg, the second direction), and an image corresponding to this cycle direction is obtained. Get the data. In addition, when the control unit 56 determines that the imaging according to the cycle direction of the interference fringes is completed (step S18; Yes), the series of processes ends.

  In this way, the microscope apparatus 1 acquires three images in each of the three directions, and acquires a total of nine images. In addition, the demodulation unit 55 of the control device 4 executes demodulation processing using the acquired image data to generate super-resolution image data. The control device 4 stores the data of the super-resolution image in a storage device (not shown), for example. Further, the control device 4 stores the super-resolution image in the display device 5, for example.

  Next, the 3D-SIM mode will be described. FIG. 10 is a diagram showing an optical path from the light source unit 10 to the sample X in the 3D-SIM mode. In FIG. 10, the optical path from the sample X to the image pickup element 51 is omitted in order to make the diagram easy to see, but the image pickup apparatus 3 picks up the image of the sample X similarly to FIG. 1. In the 3D-SIM mode, the structured illumination device 2 causes the interference fringes of the + 1st-order diffracted light and the -1st diffracted light, the interference fringes of the + 1st-order diffracted light and the 0th-order diffracted light, and the interference between the -1st-order diffracted light and the 0th-order diffracted light. A combined interference fringe is formed by combining the fringes. This synthetic interference fringe has a periodic structure in a total of three directions, two directions perpendicular to the optical axis 11a of the illumination optical system 11 and one direction parallel to the optical axis 11a.

  FIG. 11 is a diagram showing an optical path from the branching section 13 to the field stop 32 and a drive signal. In the 3D-SIM mode, the control device 4 (control unit 56) controls the phase adjustment unit 14 to adjust the relative phases of the 0th-order diffracted light and the 1st-order diffracted light. First, the adjustment of the phase during one frame period of image pickup (exposure period within one frame) will be described.

  FIG. 12 is a diagram showing interference fringes of the 0th-order diffracted light and the 1st-order diffracted light before and after the standing wave and the phase adjustment in each of the first half period and the second half period. As described above, the standing wave has a relationship in which the phases are shifted by π between the first half and the second half. In the case of the interference fringes of the 0th-order diffracted light and the 1st-order diffracted light, since the order difference is 1, the number of fringes of the interference fringes is 1 times the wavenumber. Therefore, like the pre-adjustment interference fringes (first half period, second half period) in FIG. 12, the phase of the interference fringes is deviated by π between the first half period and the second half period. When the sample X is illuminated during the first half and the second half without adjusting the phase, the contrast of the obtained image decreases. For example, when the total exposure amount in the first half period and the total exposure amount in the second half period included in one frame of imaging are the same, as long as the structured illumination is based on the interference fringes of the 0th-order diffracted light and the 1st-order diffracted light, the obtained image is The image is obtained when the sample is illuminated with uniform illumination.

Therefore, the phase adjustment unit 14 adjusts the relative phase (phase difference) between the 0th-order diffracted light and the 1st-order diffracted light to different values in the first half period and the second half period. When the relative phase is not adjusted, the intensity distribution of the interference fringes in the first half period is represented by the following formula (1), and the intensity distribution of the interference fringes in the latter half period is represented by the following formula (2).
Ii (r) = I0 + 2I + 4 ・ √I0 ・ √I ・ cos (kxX) ・ cos (kzZ) +2 ・ I ・ cos (2kxX) ・ ・ ・ (1)
Ii (r) = I0 + 2I + 4 ・ √I0 ・ √I ・ cos (kxX + π) ・ cos (kzZ) +2 ・ I ・ cos (2kxX + 2π) ・ ・ ・ (2)

  In Expressions (1) and (2), X represents a position in a direction corresponding to the pitch direction (period direction) of the phase type diffraction grating, and Z represents a direction parallel to the optical axis 11a of the illumination optical system 11. Indicates the position. I0 represents the intensity of the 0th-order diffracted light, I represents the intensity of the ± 1st-order diffracted light, and kx and kz represent the wave numbers in the X direction and the Z direction, respectively. The first term in the equations (1) and (2) is the intensity of the 0th-order diffracted light, the second term is the sum of the intensities of the + 1st-order diffracted light and the −1st-order diffracted light, and the 3rd term is It is the interference intensity distribution between the ± 1st order diffracted light and the 0th order diffracted light, and the fourth term is the interference intensity distribution between the ± 1st order diffracted lights. Only the third term is different between Expression (1) and Expression (2).

Here, for convenience of explanation, the phase adjustment amount of the 0th-order diffracted light in the first half period is π, the phase adjustment amount of the 0th-order diffracted light in the latter half period is 0, and the + 1st-order diffracted light and the −1st-order diffracted light are the first-half period and the second-half period. The phase adjustment amount of is set to 0. In this case, the intensity distribution of the interference fringes in the first half period is expressed by the following equation (3). This equation (3) is equivalent to the above equation (2), and the pattern of the interference fringes in the first half period after the phase adjustment becomes the same as the pattern of the interference fringes in the latter half period (the adjusted interference fringes in FIG. 12). reference). The phase adjustment amount according to the embodiment may be 0 as described above.
Ii (r) = I0 + 2I + 4 ・ √I0 ・ √I ・ cos (kxX + π) ・ cos (kzZ) +2 ・ I ・ cos (2kxX) ・ ・ ・ (3)

  Returning to the description of FIG. 11, the control device 4 controls the power supply circuit 25 to supply the switch 26 with the drive signal DW1 and the phase adjustment circuit 44 with the drive signal DW2. The power supply circuit 25 makes the frequencies of the drive signal DW1 and the drive signal DW2 the same, for example. The drive signal DW1 is supplied to the branching unit 13 via the switch 26. The drive signal DW1 has, for example, a sinusoidal voltage waveform.

  The drive signal DW2 is supplied to the phase modulator 41 of the phase adjusting unit 14 via the phase adjusting circuit 44. The phase adjustment circuit 44 is controlled by the control device 4 (control unit 56) and adjusts the phase of the drive signal DW2. The drive signal DW2 has, for example, a rectangular wave shape. The low level (L) of the drive signal DW2 is a reference potential such as a ground potential, for example, a potential that does not apply an electric field to the electro-optical material of the phase modulator 41. Here, it is assumed that the phase adjustment amount when the electric field is not applied to the electro-optical material of the phase modulator 41 is 0, and the phase adjustment amount by the refractive index of the electro-optical material (offset offset by the correction block 42). Ignore. The high level (H) of the drive signal DW2 is a level at which an electric field is applied to the electro-optical material so as to add a phase of π to the 0th-order diffracted light as compared with the low level. The drive signal DW2 rises to H when the level of the drive signal DW1 changes to a positive potential with respect to a reference potential (eg, ground potential), and rises to L when the level of the drive signal DW1 changes from the reference potential to a negative potential. It is a waveform that falls to. The drive signal DW2 is maintained at H while the level of the drive signal DW1 is a positive potential, and is maintained at L while the level of the drive signal DW1 is a negative potential.

  The drive signal DW1 includes a pulse having a plurality of cycles in one frame of imaging, for example, one pulse having a period from when the signal level rises to H to when the signal level next rises to H. When the cycle of the drive signal DW1 is T, the cycle of the drive signal DW2 is T, which is H in the first half and L in the second half. When the relative phase of the 0th-order diffracted light and the 1st-order diffracted light is changed by π between the first half and the second half, the drive signal DW2 has a waveform that becomes L in the first half and becomes H in the second half. Good.

  Next, the adjustment of the phase in a plurality of frames of imaging will be described. In order to generate one super-resolution image in the 3D-SIM mode, generally, the directions of the interference fringes are changed in three ways, and the phase of the interference fringes is changed in five ways for each direction of the interference fringes. Then, imaging is performed using each of the 15 states of the interference fringes. In order to change the phase in five ways, for example, there is a method of setting the incident area Sa at the position of L / 5 from the end of the sound wave propagation path R in the branching section 13 and changing the wave number in five ways. This method requires an optical modulator for 3D-SIM mode, and the sound wave propagation path R becomes large with respect to the incident area Sa. Further, it becomes necessary to exchange the optical modulator in the 2D-SIM mode and the 3D-SIM mode.

  The structured illumination device 2 according to the present embodiment changes the relative phase of the 0th-order diffracted light and the 1st-order diffracted light between the first frame and the second frame of the plurality of frames of the image pickup, thereby generating the interference fringes. The phase can be shifted. Thereby, for example, the branching unit 13 can be shared between the 2D-SIM mode and the 3D-SIM mode.

  FIG. 13 is an explanatory diagram showing drive signals, standing waves, and interference fringes in a plurality of frames. The control device 4 (control unit 56) controls the power supply circuit 25 to supply the drive signal DW1 to the branching unit 13 in the period of the first frame of imaging. Further, the control unit 56 causes the branching unit 13 to supply a drive signal (for example, drive signal DW1) having the same frequency and phase as those of the first frame during the period of the second frame of imaging. That is, the drive signal in the first frame changes from the negative potential to the positive potential at a predetermined timing with reference to the start time of the first frame, and the drive signal in the second frame has a predetermined reference with respect to the start time of the second frame. At the timing of, the negative potential changes to the positive potential. Note that in FIG. 13, the predetermined timing is the same as the start time of each frame, but may be deviated from the start time of each frame.

  The control unit 56 controls the power supply circuit 25 and the phase adjustment circuit 44 to supply the drive signal DW2 to the phase modulator 41 of the phase adjustment unit 14 in the period of the first frame of imaging. Further, the control unit 56 controls the power supply circuit 25 and the phase adjustment circuit 44 to supply the drive signal DW3 different from that in the first frame to the phase modulator 41 in the period of the second frame of imaging.

  In FIG. 13, the drive signal DW3 has the same phase as the drive signal DW2. The drive signal DW2 rises at a predetermined timing based on the start time of the first frame, and the drive signal DW3 rises at a predetermined timing based on the start time of the second frame. Further, compared with the drive signal DW1, in the first frame, the drive signal DW1 changes from the negative potential to the positive potential at a predetermined timing from the start time of the first frame, and the drive signal DW2 shows the start time of the first frame. It rises from the predetermined timing. In the second frame, the drive signal DW1 changes from a negative potential to a positive potential at a predetermined timing from the start time of the second frame, and the drive signal DW3 rises at a predetermined timing from the start time of the second frame. Note that in FIG. 13, the predetermined timing is the same as the start time of each frame, but may be deviated from the start time of each frame.

  In FIG. 13, the drive signal DW3 has a voltage value different from that of the drive signal DW2. The drive signal DW2 in the first frame has a waveform in which the signal level switches between a low level (L) and a first high level (H1). H1 is a level at which the amount of phase adjustment by the phase modulator 41 changes by π compared to L. The drive signal DW3 in the second frame has a waveform whose level switches between H1 and the second high level (H2). H2 is a level at which the phase adjustment amount by the phase modulator 41 changes by π as compared with H1, and is a level at which the phase adjustment amount by the phase modulator 41 changes by 2π as compared with L. . That is, the upper limit value, the center value, and the lower limit value of the modulation width of the phase modulator 41 change depending on the drive signal DW2 and the drive signal DW3. For example, the power supply circuit 25 is controlled by the control device 4 to generate the drive signal DW2. The drive signal DW3 is generated by adding an offset (for example, H1).

The signal level (voltage value) of the drive signal DW2 has a correspondence relationship with the phase adjustment amount. In the first frame, the voltage value of the drive signal DW2 is switched between L and H1, and here, the phase adjustment amount when the voltage value is L is the first phase adjustment amount, and the voltage value is H1. In this case, the phase adjustment amount is the second phase adjustment amount. In the first frame, the patterns (first pattern, third pattern, fifth pattern) in which the phase adjustment unit 14 changes the phase are the first phase adjustment amount corresponding to the voltage value L and the voltage value H1. And a phase adjustment amount corresponding to (2nd phase adjustment amount) is applied according to time to change the phase. The frequencies of the first pattern, the third pattern, and the fifth pattern are different from each other. For example, the first pattern, the third pattern, the fifth pattern, respectively, the frequency of the drive signal DW2 _{f -1,} is realized by setting the f _{0, f +1.}

Further, the signal level (voltage value) of the drive signal DW3 has a correspondence relationship with the phase adjustment amount. In the second frame, the voltage value of the drive signal DW3 is switched between H1 and H2. Here, the phase adjustment amount when the voltage value is H1 is the phase adjustment amount obtained by adding π to the first phase adjustment amount. Is the amount. The phase adjustment amount when the voltage is H2 is the phase adjustment amount obtained by adding π to the second phase adjustment amount. In the second frame, the patterns (second pattern, fourth pattern, sixth pattern) in which the phase adjusting unit 14 changes the phase are the phase adjustment amount obtained by adding π to the first phase adjustment amount, This is a pattern for changing the phase by adding the phase adjustment amount obtained by adding π of the phase adjustment amount of 2 according to time. The frequencies of the second pattern, the fourth pattern, and the sixth pattern are different from each other. For example, the second pattern, the fourth pattern, the pattern of the sixth, respectively, the frequency of the drive signal DW3 _{f -1,} is realized by setting the f _{0, f +1.}

  The standing wave pattern is the same in the first frame and the second frame. Further, the interference fringes in the second frame are out of phase with the interference fringes in the first frame because the relative phases of the 0th-order diffracted light and the 1st-order diffracted light are different from those in the 1st frame. For example, the central position of the bright portion of the first frame is periodically arranged with a pitch of ΔB. Further, the center position of the bright portion of the second frame is periodically arranged at a pitch of ΔB from a position shifted by about ΔB / 2 from the center position of the bright portion of the first frame. The control unit 56 controls the phase adjustment unit 14 so that, for example, the center of the bright portion of the interference fringes in the first frame matches the center of the dark portion of the interference fringes in the second frame.

  Here, the phase modulator 41 gives the phase π in the first half of one cycle of the drive signal DW2 and does not give the phase in the second half. Further, the phase modulator 41 imparts a phase of 2π in the first half of one cycle of the drive signal DW3 and imparts a phase of π in the latter half of the period. For example, the relative phase between the 0th-order diffracted light and the 1st-order diffracted light in the first frame is switched between 0 and π, and the relative phase between the 0th-order diffracted light and the 1st-order diffracted light in the second frame is π It can be switched with 2π. The phase modulator 41 gives (β + π) a phase in the first half of one cycle of the drive signal DW2 and β in the second half of the cycle, according to the contrast of the interference fringes, and gives β in the first half of one cycle of the drive signal DW3. (Β + 2π), and (β + π) in the latter half.

As described above, when the relative phases of the 0th-order diffracted light and the 1st-order diffracted light are deviated between the first frame and the second frame, the dark portion in the second frame is arranged at the position of the bright portion in the first frame. It That is, the interference fringes of the second frame are in a relationship in which the phase is deviated from the interference fringes of the first frame by π. For example, when the phase shift amount of the interference fringes in the first frame is −2π / 3, the phase shift amount of the interference fringes in the second frame is π / 3. When the phase shift amount of the interference fringes in the first frame is 0, the phase shift amount of the interference fringes in the second frame is π. When the phase shift amount of the interference fringes in the first frame is 2π / 3, the phase shift amount of the interference fringes in the second frame is 5π / 3. That is, when the frequency of the drive signal of the branching unit 13 is switched to f ₋₁ , f ₀ , and f ₊₁ and the phase is adjusted by the phase modulator 41 for each frequency, the phase shift amount is 0, π / It is possible to switch in 6 ways of 3, 2π / 3, π, 4π / 3, and 5π / 3.

  Next, an observation method in the 3D-SIM mode will be described. FIG. 14 is a flowchart showing the operation of the microscope apparatus 1 in the 3D-SIM mode (see FIGS. 10 and 11 for each part of the microscope apparatus 1). In step S21, the control unit 56 of the control device 4 controls the switch 26 and sets the connection destination of the switch 26 to the transducer 23a. As a result, the periodic direction of the interference fringes Sb is set to the direction corresponding to the first direction.

In step S22, the control unit 56 gives the branching unit 13 a first drive signal to generate a sound wave standing wave. The first drive signal is a drive signal having a frequency of f ₋₁ (see FIG. 6), and the control unit 56 controls the power supply circuit 25 to set the frequency of the drive signal to f ₋₁ . In step S23, the control unit 56 changes the relative phase of the 0th-order diffracted light and the 1st-order diffracted light, for example, by changing the phase of the 0th-order diffracted light in the first pattern. Here, the first pattern is a pattern that changes the phase by applying two types of phase adjustment amounts according to time, and the difference between the two types of phase adjustment amounts is π. The control unit 56 controls the power supply circuit 25 to set the drive signal of the phase modulator 41 to the drive signal DW2 (see FIG. 13) in order to change the phase of the 0th-order diffracted light in the first pattern. As a result, the phase shift amount of the interference fringe Sb is set to −2π / 3. In step S24, the control unit 56 causes the image sensor 51 to capture the first image of the sample X and acquires the image data I ₋₁ a corresponding to the first image. In step S25, the control unit 56 changes the phase of the 0th-order diffracted light to the 0th-order diffracted light and the 1st-order diffracted light, for example, by changing the phase of the 0th-order diffracted light in the state where the first drive signal is applied to the branching unit 13. Change the relative phase of. Here, the second pattern is a pattern that changes the phase by applying two types of phase adjustment amounts according to time, and the difference between the two types of phase adjustment amounts is π. The control unit 56 controls the power supply circuit 25 to set the drive signal of the phase modulator 41 to the drive signal DW3 (see FIG. 13) in order to change the phase of the 0th-order diffracted light in the second pattern. As a result, the phase shift amount of the interference fringe Sb is set to π / 3. In step S26, the control unit 56 causes the image sensor 51 to capture the second image of the sample X, and acquires the image data I ₋₁ b corresponding to the second image.

In step S27, the control unit 56 gives the branching unit 13 a second drive signal to generate a sound wave standing wave. The second drive signal is a drive signal whose frequency is f ₀ (see FIG. 6), and the control unit 56 controls the power supply circuit 25 to set the frequency of the drive signal to f ₀ . In step S28, the control unit 56 changes the relative phase of the 0th-order diffracted light and the 1st-order diffracted light, for example, by changing the phase of the 0th-order diffracted light in the third pattern. Here, the third pattern is a pattern that changes the phase by applying two types of phase adjustment amounts according to time, and the difference between the two types of phase adjustment amounts is π. The control unit 56 controls the power supply circuit 25 to convert the drive signal of the phase modulator 41 into the drive signal obtained by converting the frequency of the drive signal DW2 into f ₀ in order to change the phase of the 0th-order diffracted light in the third pattern. Switch to. The drive signal is a drive signal that switches the phase adjustment amount for 0th-order diffracted light between 0 and π. As a result, the phase shift amount of the interference fringe Sb is set to zero. In step S29, the control unit 56 causes the image sensor 51 to capture the third image of the sample X, and acquires the image data I ₀ a corresponding to the third image. In step S30, the control unit 56 changes the phase of the 0th-order diffracted light to the 0th-order diffracted light and the 1st-order diffracted light by changing the phase of the 0th-order diffracted light in the fourth pattern while the second drive signal is applied to the branching unit 13. Change the relative phase of. Here, the fourth pattern is a pattern that changes the phase by applying two types of phase adjustment amounts according to time, and the difference between the two types of phase adjustment amounts is π. In order to change the phase of the 0th-order diffracted light in the fourth pattern, the control unit 56 controls the power supply circuit 25 to output the drive signal and the drive signal corresponding to the third pattern in step S28 to offset the voltage value. Switch to the added drive signal. For example, the control unit 56 controls the power supply circuit 25 to switch the drive signal of the phase modulator 41 to the drive signal obtained by converting the frequency of the drive signal DW3 into f ₀ . It should be noted that this drive signal is a drive signal that switches the phase adjustment amount for the 0th-order diffracted light between π and 2π. As a result, the phase shift amount of the interference fringe Sb is set to π. In step S31, the control unit 56 controls the imaging device 3 to image the sample X and acquires the image data I ₀ b.

In step S32, the control unit 56 gives the branching unit 13 a third drive signal to generate a sound wave standing wave. The third drive signal is a drive signal having a frequency of f ₊₁ (see FIG. 6), and the control unit 56 controls the power supply circuit 25 to set the frequency of the drive signal to f ₊₁ . In step S33, the control unit 56 changes the relative phase of the 0th-order diffracted light and the 1st-order diffracted light, for example, by changing the phase of the 0th-order diffracted light in the fifth pattern. Here, the fifth pattern is a pattern that changes the phase by applying two types of phase adjustment amounts according to time, and the difference between the two types of phase adjustment amounts is π. The control unit 56 controls the power supply circuit 25 to change the drive signal of the phase modulator 41 into the drive signal obtained by converting the frequency of the drive signal DW2 into f ₊₁ in order to change the phase of the 0th-order diffracted light in the fifth pattern. Switch to. The drive signal is a drive signal that switches the phase adjustment amount for 0th-order diffracted light between 0 and π. As a result, the phase shift amount of the interference fringe Sb is set to 2π / 3. In step S34, the control unit 56 causes the image sensor 51 to capture the fifth image of the sample X, and acquires the image data I ₊₁ a corresponding to the fifth image. In step S35, the relative phase between the 0th-order diffracted light and the 1st-order diffracted light is changed by changing the phase of the 0th-order diffracted light in the sixth pattern while the third drive signal is applied to the branching unit 13, for example. Change. Here, the sixth pattern is a pattern for changing the phase by applying two types of phase adjustment amounts according to time, and the difference between the two types of phase adjustment amounts is π. The control unit 56 controls the power supply circuit 25 to switch the drive signal to the drive signal obtained by adding the voltage value offset to the drive signal corresponding to the fifth pattern in step S33. For example, the control unit 56 controls the power supply circuit 25 to switch the drive signal of the phase modulator 41 to the drive signal obtained by converting the frequency of the drive signal DW3 into f ₊₁ . It should be noted that this drive signal is a drive signal that switches the phase adjustment amount for the 0th-order diffracted light between π and 2π. As a result, the phase shift amount of the interference fringe Sb is set to 5π / 3. In step S36, the control unit 56 causes the imaging element 51 to capture the sixth image of the sample X and acquires the image data I ₊₁ b corresponding to the sixth image. As described above, the microscope apparatus 1 sets the periodic direction of the interference fringes to the first direction, switches the phase to six ways, and acquires six images.

  In step S37, the control unit 56 determines whether or not the periodic directions of the interference fringes Sb have been set in all three directions. When the control unit 56 determines that the imaging according to the cycle direction of the interference fringes is not completed (step S37; No), the connection destination of the switch 26 is switched to the next transducer in step S38. After the processing of step S38 is completed, the control unit 56 repeats the processing of steps S22 to S36 in a state in which the cycle direction of the interference fringes is changed (for example, the second direction), and an image corresponding to this cycle direction is obtained. Get the data. In addition, when the control unit 56 determines that the imaging according to the periodic direction of the interference fringes is completed (step S37; Yes), the series of processes is ended.

FIG. 15 is a diagram showing setting conditions for one direction of interference fringes. The structured lighting device 2 switches the frequency of the drive signal of the branch unit 13 to three of f ₋₁ , f ₀ , and f ₊₁ and performs phase modulation by the phase modulator for each frequency of the drive signal of the branch unit 13. By switching the amount between 0 and π, the phases of the interference fringes can be switched between 0, π / 3, 2π / 3, π, 4π / 3, and 5π / 3. The microscope apparatus 1 switches the phase in six ways in each of the three directions and acquires a total of 18 images.

  In addition, the demodulation unit 55 of the control device 4 executes demodulation processing using the acquired image data to generate super-resolution image data. The demodulation unit 55 performs demodulation processing using, for example, 5 images (15 in 3 directions) out of 6 images having different phases in each direction of interference fringes, and the remaining 1 image in each direction (3 directions). The three images) may be used for interpolation or the like, or may not be used for demodulation processing. Note that the control unit 56 does not have to cause the image pickup device 3 to pick up an image that is not used in the demodulation process. For example, the phases of the interference fringes are 0, π / 3, 2π / 3, π, 4π / 3. It is also possible to acquire 5 images by changing it in 5 ways out of 5π / 3.

  FIG. 16 is a diagram showing another example of the drive signal of the phase modulator 41. In FIG. 16, the drive signal DW4 of the second frame has a waveform in which the signal level switches between L and H1. The drive signal DW4 is different in phase from the drive signal DW2 of the first frame. The time from the start time of the second frame to the rising edge of the drive signal DW4 is different from the time from the start time of the first frame to the rising edge of the drive signal DW2. The drive signal DW4 is displaced from the drive signal DW2 by T / 2 (π in phase) in time from the start time of the frame to the rising edge. For example, the phase adjustment circuit 44 is supplied with the drive signal DW2 from the power supply circuit 25 and adjusts the phase of the drive signal DW by π to generate the drive signal DW4.

  The signal level (voltage value) of the drive signal DW2 has a correspondence relationship with the phase adjustment amount. In the first frame, the voltage value of the drive signal DW2 is switched between L and H1, and here, the phase adjustment amount when the voltage value is L is the first phase adjustment amount, and the voltage value is H1. The phase adjustment amount of is the second phase adjustment amount. In the first frame, the patterns (first pattern, third pattern, fifth pattern) in which the phase adjustment unit 14 changes the phase are the first phase adjustment amount corresponding to the voltage value L and the voltage value H1. Is a pattern in which the second phase adjustment amount corresponding to is alternately applied according to time to change the phase.

  Further, the signal level (voltage value) of the drive signal DW4 has a correspondence relationship with the phase adjustment amount. The drive signal DW4 is a drive signal whose signal level is inverted between L and H1 as compared with the drive signal DW2. The pattern (the second pattern, the fourth pattern, the sixth pattern) in which the phase adjustment unit 14 changes the phase in the second frame is the pattern (the first pattern in which the phase adjustment unit 14 changes the phase in the first frame (first pattern). Pattern, the third pattern, and the fifth pattern).

  The phase modulator 41 is driven by the drive signal DW2 to adjust the phase of the 0th-order diffracted light by π in the first half and by 0 in the second half, and is driven by the drive signal DW4 to generate the 0th-order diffracted light. The phase of is adjusted by 0 in the first half and adjusted by π in the second half. In this way, the drive signal of the phase modulator 41 in the second frame may be out of phase with the drive signal of the phase modulator in the first frame within each frame.

  In the structured illumination device 2 according to the present embodiment as described above, the phase adjusting unit 14 adjusts the relative phase of the 0th-order diffracted light and the 1st-order diffracted light, and thus the phase of the interference fringes can be changed. Image acquisition can be speeded up. Further, for example, when the branch unit 13 has a configuration using an acousto-optic module, the branch unit 13 can be shared by the 2D-SIM mode and the 3D-SIM mode. Further, for example, as compared with the configuration in which the phase of the interference fringes is changed in six ways by the branching section 13 without using the phase adjusting section 14, the branching section 13 becomes larger with respect to the incident area Sa of the light from the light source. You can avoid that.

  The branching section 13 may be configured to use a diffraction grating formed on a quartz substrate or the like, or may be configured to use a spatial light modulation element such as a liquid crystal element. For example, when the direction or phase of the interference fringes is changed by a diffraction grating formed on a quartz substrate or the like, the diffraction grating is moved in five phases, but when the phase adjusting unit 14 is used together, movement in three phases is sufficient. As a result, it is possible to reduce the waiting time until the vibration accompanying the movement of the diffraction grating subsides, for example. Further, when the liquid crystal element is used for the branching section 13, when shifting 1 phase by 5 phases, it requires 5 pixels in total for the bright section and the dark section. You only need 3 pixels per minute. Therefore, when liquid crystal elements having the same number of pixels are used, the number of cycles of patterns that can be formed can be increased.

  In the microscope device 1 described above, the control device 4 is composed of, for example, a computer having a CPU and a memory. The computer reads a control program from an external or internal storage device and causes the above-described processing to be executed according to the control program. This control program is, for example, an illumination that illuminates a sample with an interference fringe by causing at least a part of the plurality of light beams to interfere with an optical modulator that generates an acoustic standing wave and splits incident light into a plurality of light beams. An optical system, an imaging optical system that forms an image of the sample irradiated with the interference fringes, an imaging unit that captures the image of the sample formed by the imaging optical system, and a phase of at least one of the plurality of light beams. Is a control program that causes a computer to execute control of a microscope apparatus having a phase adjustment unit that changes the, and the control is such that when a first driving signal is applied to the optical modulator to generate a sound wave standing wave, Changing the phase in the first pattern by the phase adjustment unit to cause the imaging unit to capture the first image, and causing the phase adjustment unit to change the phase in the second pattern to cause the imaging unit to capture the second image. And a second drive signal to the optical modulator When a sound wave standing wave is generated by changing the phase in the phase adjusting unit according to the third pattern and causing the image capturing unit to capture the third image, the phase adjusting unit changes the phase in the fourth pattern. When the image pickup section is caused to pick up a fourth image and the optical modulator is given a third drive signal to generate a sound wave standing wave, the phase adjustment section changes the phase in the fifth pattern. Causing the imaging unit to capture the fifth image and the phase adjustment unit to change the phase in the sixth pattern to cause the imaging unit to capture the sixth image.

  The phase adjustment amount of the first half period and the phase adjustment amount of the second half period by the phase modulator 41 can be changed appropriately. For example, in the above-described embodiment, the phase modulator 41 sets the phase adjustment amount of the 0th-order diffracted light to 0, π, 2π, or the like, but it may be set to a phase adjustment amount other than this. For example, the control device 4 changes the phase adjustment amount of the phase modulator 41 while referring to the contrast of the image output from the imaging device 3, using a sample whose characteristics are known in advance, such as a calibration sample. The phase adjustment amount may be set to an initial value when the contrast becomes maximum (or when the contrast becomes equal to or higher than a threshold).

  FIG. 17 is a diagram showing an example of the amount of phase adjustment by the phase modulator 41. In the example of FIG. 17A, the phase adjustment amount for the first-order diffracted light is 0 and the phase adjustment amount for the 0th-order diffracted light is π in the first half period, as in the above-described embodiment. In the first half, the relative phase adjustment amount of the 0th-order diffracted light and the 1st-order diffracted light is π. In the latter half of the period, the phase adjustment amount for the 1st-order diffracted light is 0, and the phase adjustment amount for the 0th-order diffracted light is 0. In the latter half of the period, the relative phase adjustment amount of the 0th-order diffracted light and the 1st-order diffracted light is zero. As described above, the relative phase adjustment amounts of the 0th-order diffracted light and the 1st-order diffracted light differ between the first half and the second half.

  In the example of FIG. 17B, the phase adjustment amount for the first-order diffracted light is π and the phase adjustment amount for the 0th-order diffracted light is 0 in the first half period. In the first half, the relative phase adjustment amount of the 0th-order diffracted light and the 1st-order diffracted light is π. In the latter half of the period, the phase adjustment amount for the 1st-order diffracted light is 0, and the phase adjustment amount for the 0th-order diffracted light is 0. In the latter half of the period, the relative phase adjustment amount of the 0th-order diffracted light and the 1st-order diffracted light is zero. In the description of FIGS. 13 to 16, by changing the 0th-order diffracted light in the first to sixth patterns, the relative phase of the 0th-order diffracted light and the 1st-order diffracted light is changed. The phase adjustment amount of the 1st-order diffracted light instead of the 0th-order diffracted light may be changed between the first half and the second half. For example, instead of changing the 0th-order diffracted light in the first to sixth patterns, the 1st-order diffracted light is changed in the first to sixth patterns. The relative phase between the folded light and the first-order diffracted light can be changed. As a method of changing the 1st-order diffracted light into the 0th-order diffracted light in the first to sixth patterns, for example, a phase modulator is provided in place of the correction block 42 of FIG. 11, and the drive signal DW2 or There is a method of supplying a drive signal such as the drive signal DW3 by switching between the first half period and the second half period.

  In addition, in order to adjust the relative phases of the ± 1st order diffracted light and the 0th order diffracted light, the target of phase modulation may be both the ± 1st order diffracted light and the 0th order diffracted light. In this case, the phase modulator 41 may be arranged in each of the region where the ± 1st order diffracted light enters and the region where the 0th order diffracted light enters. For example, the same phase modulator 41 may be arranged over the region where the 0th-order diffracted light is incident and the region where the 1st-order diffracted light is incident, or in the region where the 0th-order diffracted light is incident and the region where the 1st-order diffracted light is incident. Alternatively, the phase modulators 41 may be arranged respectively.

  In the example of FIG. 17C, in the first half, the phase adjustment amount for the + 1st order diffracted light is α + π, the phase adjustment amount for the −1st order diffracted light is −α + π, and the phase adjustment amount for the 0th order diffracted light is 0. . In the latter half of the period, the phase adjustment amount for the + 1st order diffracted light is α, the phase adjustment amount for the −1st order diffracted light is −α, and the phase adjustment amount for the 0th order diffracted light is 0. In this way, the + 1st-order diffracted light and the -1st-order diffracted light may have different phase adjustment amounts. Such α (offset) is set as follows, for example. First, in the 2D-SIM mode, the 0th-order diffracted light is blocked, the + 1st-order diffracted light is given a phase offset α, and the −1st-order diffracted light is given a phase offset −α. That is, the phase offset of the + 1st-order diffracted light with respect to the 0th-order diffracted light and the phase offset of the -1st-order diffracted light with respect to the 0th-order diffracted light are set to equal signs opposite to each other. Even in this state, proper two-beam structured illumination can be generated. Then, when switching the phase of the interference fringe, the phase offset amount α is switched in a predetermined pattern instead of switching the wavelength of the standing wave in the predetermined pattern. For example, if the phase offset amount α is switched in three ways at a pitch of 2π / 3, the phase of the interference fringes can be switched in three ways at a pitch of 2π / 3. Note that adjusting the relative phase of the 0th-order diffracted light and the 1st-order diffracted light by changing the phase of the 1st-order diffracted light instead of the 0th-order diffracted light is the same as the description of FIG. 17B. .

  In the above-described embodiment, a combination of ± 1st-order diffracted light and 0th-order diffracted light is used as the diffracted light for forming the interference fringe (interference fringe of 2 light fluxes, interference fringe of 3 light fluxes), but other combinations May be used.

  It should be noted that the drive signal DW2 supplied to the phase modulator 41 may be out of phase with the drive signal DW1 supplied to the branching unit 13 within one frame of imaging. FIG. 18 is a diagram showing a comparison between the time change of the refractive index of the antinode a of the standing wave and the time change of the phase modulation amount of the 0th-order diffracted light. In FIG. 18, the time change of the refractive index of the antinode a is shown by a solid line, and the time change of the phase modulation amount of the 0th-order diffracted light is shown by a broken line.

  In FIG. 18A, the phase difference ΔΨ between the waveform showing the time change of the refractive index of the antinode a and the waveform showing the time change of the phase modulation amount of the 0th-order diffracted light is equal to an even multiple of π / 2. There is. In this case, the period in which the phase of the 0th-order diffracted light is displaced coincides with the first half or the second half of the change in the refractive index, so that the contrast of the obtained image becomes maximum.

  In FIG. 18B, the phase difference ΔΨ between the waveform showing the time change of the refractive index of the antinode a and the waveform showing the time change of the phase modulation amount of the 0th-order diffracted light coincides with an even multiple of π / 2. Absent. In this case, the phases of the interference fringes Sb are inverted in the first half and the second half, respectively, so that they cancel each other out and the contrast of the interference fringes decreases. Therefore, it is desirable that ΔΨ be equal to an even multiple of π / 2, but ΔΨ is deviated from an even multiple of π / 2 within a range in which the contrast between the bright portion and the dark portion required for demodulation processing can be secured. May be.

  In FIG. 18C, the phase difference ΔΨ between the waveform showing the time change of the refractive index of the antinode a and the waveform showing the time change of the phase modulation amount of the 0th-order diffracted light coincides with an odd multiple of π / 2. ing. In this case, the total exposure amount in the first half period and the total exposure amount in the second half period included in one frame of imaging are the same, and the contrast of the obtained image is almost zero. Therefore, as shown in FIG. 18B, even if ΔΨ is deviated from an even multiple of π / 2, it is set so as not to be an odd multiple of π / 2.

  In the 3D-SIM mode, the controller 4 may automatically adjust the phase difference ΔΨ between the time-varying waveform of the refractive index and the time-varying waveform of the phase modulation amount of the 0th-order diffracted light. May be done manually. For example, the control device 4 may display the image output from the imaging device 3 on the display device 5 in real time so that the user can check the contrast of the image being adjusted.

  In the 3D-SIM mode, the frequency of the drive signal of the phase modulator 41 may not match the frequency of the drive signal of the branch unit 13 with the frequency of the sine signal. FIG. 19 is a diagram showing another example of the drive signal of the phase modulator 41. For example, the frequency of the drive signal of the phase modulator 41 is set to 1 / N times (N is an integer of 1 or more) the frequency of the drive signal of the branch unit 13, and the duty ratio (signal May be set to 1 / (2N). FIG. 19 is an example when N = 3.

  In this case, the phase modulator 41 adjusts the phases of the first half period and the second half period in one of the three cycles of the standing wave by the branch unit 13. Even in such a case, the contrast between the bright portion and the dark portion can be obtained in one cycle in which the phase is adjusted. Further, in such a case, the imaging device 3 does not have to perform charge accumulation in one cycle in which the phase modulator 41 adjusts the phase, and does not perform charge accumulation in two cycles in which the phase modulator 41 does not adjust the phase. . In this case, the charge accumulation period is, for example, a period (T / 2) in which the phase of the 0th-order diffracted light is displaced and an equivalent period (T / 2) before or after the phase modulation period (N × T). The time period (T) may be set as The duty ratio may deviate from 1 / (2N) within a range where the contrast can be secured. Further, the drive signal of the phase modulator 41 does not have to have a rectangular wave shape, and may have, for example, a sine wave shape or another waveform.

  The phase adjusting unit 14 includes the phase modulator 41 that modulates the phase of the diffracted light. However, instead of the phase modulator 41 or in combination with the phase modulator 41, the phase adjusting unit 14 includes refraction members having different thicknesses. Good. The refraction members having different thicknesses may be, for example, wedge-shaped or step-shaped glass members. For example, by disposing a wedge-shaped or step-shaped glass member in a region where the 0th-order diffracted light enters and moving the glass member in a predetermined direction, or by rotating the glass member about a predetermined axis, the 0th-order diffracted light is generated. It is also possible to periodically change the thickness of the glass member that passes through to modulate the phase of the diffracted light that passes through the glass member.

  In the branching section 13, the directions of the sound waves propagating through the sound wave propagation path R do not have to be three directions, and may be, for example, one direction, two directions, or four or more directions. Further, the frequency change pattern of the AC voltage applied to the transducer may be a pattern in which the number of standing waves changes by M / 2. In this case, in order to set the pitch of the phase shift amount of the interference fringes to an arbitrary value Δψ, the distance D from one end of the sound wave propagation path to the center of the incident area Sa and the total length L of the sound wave propagation path are It suffices to satisfy the relationship of D: L = Δψ / M: 2π. When M = 1, the number of standing waves changes only by 1/2, so that the deviation of the number of stripes can be suppressed. Further, when the number of image data used for the separation calculation is k, if Δψ = 2π / k, the image data can be acquired. However, k is preferably an integer of 3 or more.

  The light passage area (incident area Sa) of the branch 13 in the sound wave propagation path R does not have to be limited to a partial area distant from both ends of the sound wave propagation path R. For example, when the light passing through the sound wave propagation path R is narrowed down by the field stop 32, the area of the sound wave propagation path R where the light contributing to the interference fringes formed in the illumination area of the sample X is D: L = Δψ / It is only necessary to satisfy the relationship of M: 2π.

  FIG. 20 is a diagram showing another example of the branch unit 13. In this branch portion 13, the three sound wave propagation paths Ra, Rb, and Rc are arranged in a rotationally symmetric relationship with respect to the center of the incident area Sa. In the above-described embodiment, the sound wave propagation paths Ra to Rc have the same length and the drive signals given to the transducers 23a to 23c are common, but the sound wave propagation paths Ra, Rb, and Rc have the same length. Differently, the drive signals given to the transducers 23a-23c may be different from each other.

  Note that the technical scope of the present invention is not limited to the modes described in the above embodiments and the like. One or more of the requirements described in the above embodiments and the like may be omitted. Further, the requirements described in the above-described embodiments and the like can be combined as appropriate. In addition, as long as it is permitted by law, the disclosures of all the documents cited in the above-mentioned embodiments are incorporated into the description of the text.

1 ... Microscope device, 2 ... Structured illumination device, 3 ... Imaging device, 4 ... Control device, 11 ... Illumination optical system, 13 ... Optical modulator, 14 ... Phase adjuster, 56 ... Control unit, DW1 ... Drive signal, DW2 ... Drive signal, DW3 ... Drive signal, DW4 ... Drive signal, R ... Sound wave propagation path, X ... ·sample

Claims (10)

An optical modulator that generates a sound wave standing wave and splits incident light into a plurality of light fluxes,
An illumination optical system that interferes at least a part of the plurality of light fluxes and illuminates a sample with interference fringes,
An imaging optical system that forms an image of the sample irradiated with the interference fringes,
An image capturing section configured to capture an image of the sample formed by the image forming optical system;
A phase adjustment unit that changes the phase of at least one light beam of the plurality of light beams;
A control unit,
Have
The control unit is
When a first drive signal is applied to the optical modulator to generate the sonic standing wave,
The phase adjusting unit changes the phase in a first pattern to cause the image capturing unit to capture a first image, and the phase adjusting unit changes the phase in a second pattern to cause the image capturing unit to generate a second image. Image of
When a second driving signal is applied to the optical modulator to generate the sonic standing wave,
The phase adjusting unit changes the phase in a third pattern to cause the image capturing unit to capture a third image, and the phase adjusting unit changes the phase in a fourth pattern to cause the image capturing unit to acquire a fourth image. Image of
When a third driving signal is applied to the optical modulator to generate the sonic standing wave,
The phase adjusting unit changes the phase in a fifth pattern to cause the image capturing unit to capture a fifth image, and the phase adjusting unit changes the phase in a sixth pattern to cause the image capturing unit to perform a sixth image. A microscope device that captures images of. The microscope apparatus according to claim 1, wherein the control unit generates an image of the sample using at least five images out of a plurality of images captured by the imaging unit. The first pattern, the second pattern, the third pattern, the fourth pattern, the fifth pattern, and the sixth pattern each have two types of phase adjustment amounts according to time. The microscope apparatus according to claim 1 or 2, wherein the microscope is a pattern that is applied to change the phase, and a difference between the two types of phase adjustment amounts is π. The first pattern, the third pattern, and the fifth pattern are patterns that change the phase by applying a first phase adjustment amount and a second phase adjustment amount according to time. Yes,
In each of the second pattern, the fourth pattern, and the sixth pattern, a phase adjustment amount obtained by adding π to the first phase adjustment amount and π is added to the second phase adjustment amount. The microscope device according to claim 3, which is a pattern for changing the phase by applying a phase adjustment amount according to time. In the first pattern, the third pattern, and the fifth pattern, a first phase adjustment amount and a second phase adjustment amount are alternately applied according to time to change the phase. Pattern,
The said 2nd pattern, the said 4th pattern, and the said 6th pattern are the patterns which reversed the said 1st pattern, the said 3rd pattern, and the said 5th pattern, respectively. Microscope device. The plurality of light fluxes are diffracted light diffracted by the light modulator,
The microscope apparatus according to any one of claims 1 to 5, wherein the phase adjustment unit changes a phase of 0th-order diffracted light in the diffracted light. The plurality of light fluxes are diffracted light diffracted by the light modulator,
The microscope apparatus according to claim 1, wherein the phase adjustment unit changes a phase of first-order diffracted light of the diffracted light. The said optical modulator switches the direction which generate | occur | produces the said acoustic wave standing wave in several directions within the surface which intersects with the direction which the said light injects, The any one of Claim 1 to Claim 7 characterized by the above-mentioned. Microscope device . A sound wave standing wave is generated, and the incident light is split into a plurality of light fluxes by an optical modulator,
Illuminating the sample with interference fringes by interfering at least a portion of the plurality of light fluxes;
Forming an image of the sample illuminated with the interference fringes;
Capturing an image of the formed sample,
Changing the phase of at least one light beam of the plurality of light beams;
Controlling the generation of the sonic standing wave by the light modulator, the change of the phase, and the imaging,
The control is
When a first drive signal is applied to the optical modulator to generate the acoustic wave standing wave, the phase is changed in a first pattern to capture a first image, and the phase is changed to a second pattern. To capture the second image by changing the
When a second drive signal is applied to the optical modulator to generate the sonic standing wave, the phase is changed in a third pattern to capture a third image, and the phase is changed to a fourth pattern. To change the image to capture the fourth image,
When a third driving signal is applied to the optical modulator to generate the sonic standing wave, the phase is changed in a fifth pattern to capture a fifth image, and the phase is changed to a sixth pattern. And making the sixth image by changing the observation method. An optical modulator that generates a sound wave standing wave and splits incident light into a plurality of light fluxes,
An illumination optical system that interferes at least a part of the plurality of light fluxes and illuminates a sample with interference fringes,
An imaging optical system that forms an image of the sample irradiated with the interference fringes,
An image capturing section configured to capture an image of the sample formed by the image forming optical system;
A phase adjustment unit that changes the phase of at least one light beam of the plurality of light beams;
A control program for causing a computer to execute control of a microscope apparatus having:
The control is
When a first drive signal is given to the optical modulator to generate the sonic standing wave, the phase adjusting unit changes the phase in a first pattern and the image capturing unit captures a first image. And causing the phase adjusting unit to change the phase in a second pattern to cause the image capturing unit to capture a second image,
When a second driving signal is applied to the optical modulator to generate the sonic standing wave, the phase adjusting unit changes the phase in a third pattern and the image capturing unit captures a third image. And then causing the phase adjusting unit to change the phase in a fourth pattern to cause the image capturing unit to capture a fourth image,
When a third drive signal is applied to the optical modulator to generate the sonic standing wave, the phase adjusting unit changes the phase in a fifth pattern to capture a fifth image in the image capturing unit. And causing the phase adjusting unit to change the phase in a sixth pattern to cause the image capturing unit to capture a sixth image.

Images