JP2024063351A - Observation device and observation method - Google Patents

Abstract

[Problem] To provide an apparatus and method for easily observing a moving observation object with an inexpensive configuration. [Solution] The observation apparatus 1 includes a light source 11, a branching section 12, a combining section 15, a first modulation section 21, a second modulation section 22, an image capturing section 23, a processing section 24, and the like. The first modulation section 21 inputs a first branched light L1 arriving from a mirror 13, modulates the first branched light L1 with each of a plurality of modulation frequencies different from each other, and outputs the first branched light L1 after the frequency modulation. The lenses 31 to 33 irradiate the first branched light L1 modulated by the first modulation section 21 to the observation object S along an irradiation direction according to the modulation frequency Ωn. The image capturing section 23 has a plurality of pixels that are one-dimensionally arranged in a direction intersecting with the moving direction of the image of the observation object S formed by the first branched light L1 on an image capturing plane (P3 plane) on which an image of the observation object S is formed. The image capturing section 23 receives the combined light and repeatedly outputs a detection signal representing a one-dimensional interference image. [Selected Figure] FIG.

Description

The present invention relates to an apparatus and method for observing a moving object.

Patent Document 1 discloses an invention of an apparatus and method for observing a moving observation object. The observation apparatus described in this document splits light output from a light source into a first branched light and a second branched light, combines the first branched light, which has been Doppler shifted by passing through a moving observation object, with the second branched light, whose optical frequency has been shifted by the heterodyne frequency, and causes heterodyne interference between the first branched light and the second branched light. This observation apparatus can obtain time series data of a complex amplitude image of the first branched light on the imaging plane based on time series data of an intensity image (two-dimensional interference image) of the interference light that has reached the imaging plane of the camera. This observation apparatus can image a moving observation object non-stained and non-invasively, and can be used, for example, to observe moving cells with a flow cytometer.

International Publication No. 2013/065796

W. Choi, C. Fang-Yen, K.Badizadegan, S. Oh, N. Lue, R. R. Dasari, and M. S. Feld, "Tomographic phase microscopy," Nat Meth 4(9), 717-719 (2007). Y. Sung, W. Choi, C. Fang-Yen, K.Badizadegan, R. R. Dasari, and M. S. Feld, "Optical diffraction tomography for high resolution live cell imaging," Optics Express 17(1), 266 (2009).

When considering the application of the invention disclosed in Patent Document 1 to a flow cytometer, the camera that captures the intensity image of the interference light must be an area camera with multiple pixels arranged two-dimensionally (e.g., 1280 x 800) on the imaging surface and with a high frame rate (e.g., 25 kfps). Such cameras are very expensive, and therefore the observation device is also expensive.

In addition, in the invention disclosed in Patent Document 1, the optical system from the object to be observed to the camera is special, so it is not easy to adjust this optical system, and it is not easy to observe the object to be observed.

The present invention was made to solve the above problems, and aims to provide an apparatus and method that can easily observe a moving object to be observed with an inexpensive configuration.

The observation apparatus of the present invention is an apparatus for observing a moving observation object.
A first aspect of the observation device of the present invention includes: (1) a light source that outputs light; (2) a branching unit that branches the light into a first branched light and a second branched light; (3) a first modulation unit that modulates the first branched light with each of a plurality of modulation frequencies different from each other; (4) an irradiation unit that irradiates an observation object with the first branched light modulated by the first modulation unit along an irradiation direction corresponding to the modulation frequency; (5) a combining unit that combines the first branched light and the second branched light after passing through the observation object and outputs the combined light; (6) an imaging unit having a plurality of pixels arranged in a direction intersecting the movement direction of the image of the observation object on an imaging plane on which an image of the observation object is formed, the plurality of pixels receiving the combined light, and repeatedly outputting detection signals representing a one-dimensional interference image; and (7) a processing unit that generates a complex amplitude image for each of a plurality of irradiation directions of the observation object corresponding to the plurality of modulation frequencies based on time series data of the one-dimensional interference image generated from the detection signals repeatedly output from the imaging unit, and generates a three-dimensional refractive index distribution image of the observation object based on the complex amplitude images for each of the plurality of irradiation directions.

The second aspect of the observation device of the present invention, in addition to the first aspect, further comprises a second modulation section that is provided on the optical path of the first branched light between the branching section and the first modulation section, or on the optical path of the second branched light between the branching section and the multiplexing section, and that provides a frequency difference between the first branched light and the second branched light.

In a third aspect of the observation device of the present invention, in addition to the first or second aspect, the first modulation unit includes an acousto-optical element that frequency-modulates light by Bragg diffraction, and provides an RF signal having a plurality of different frequency components to the acousto-optical element, and modulates the first branched light at a modulation frequency corresponding to each frequency component of the RF signal by inputting the first branched light to the acousto-optical element.

In a fourth aspect of the observation device of the present invention, in addition to the first or second aspect, the first modulation unit includes an acousto-optical element that frequency-modulates light by Raman-Nath diffraction, and an RF signal of a single frequency is provided to the acousto-optical element, and the first branched light is incident on the acousto-optical element, thereby modulating the first branched light with each of a plurality of modulation frequencies that are different from each other.

In a fifth aspect of the observation device of the present invention, in addition to the first or second aspect, the first modulation unit includes a plate-like object on which diffraction gratings having a plurality of mutually different spatial frequency components are superimposed and recorded, and the first branched light is modulated with each of a plurality of mutually different modulation frequencies by moving the plate-like object and passing the first branched light through the plate-like object.

In a sixth aspect of the observation device of the present invention, in addition to the first or second aspect, the first modulation section includes a phase modulation type spatial light modulator, and changes the amount of phase modulation at different speeds depending on the region of the modulation surface of the spatial light modulator, and modulates the first branched light at a modulation frequency according to the speed of change of the amount of phase modulation in each region of the modulation surface of the spatial light modulator by making the first branched light incident on the modulation surface of the spatial light modulator.

In a seventh aspect of the observation device of the present invention, in addition to the first or second aspect, the first modulation unit includes an intensity modulation type spatial light modulator, and the intensity modulation amount distribution along a predetermined direction of the modulation surface of the spatial light modulator has a plurality of spatial frequency components that are different from each other. The intensity modulation amount distribution is moved in the predetermined direction on the modulation surface of the spatial light modulator, and the first branched light is incident on the modulation surface of the spatial light modulator, thereby modulating the first branched light with each of the plurality of modulation frequencies that are different from each other.

In addition to any of the first to seventh aspects, the seventh aspect of the observation device of the present invention further includes (1) an irradiation position detection unit that detects the irradiation position of the first branched light on the movement path of the observation object, and (2) a control unit that controls the position of the optical system from the observation object to the imaging unit or the imaging unit based on the irradiation position detected by the irradiation position detection unit so that the combined light can be received by multiple pixels of the imaging unit.

The observation method of the present invention is a method for observing a moving observation object.
A first aspect of the observation method of the present invention includes: (1) a branching step of branching light output from a light source into a first branched light and a second branched light using a branching unit; (2) a first modulation step of modulating the first branched light with a plurality of modulation frequencies different from each other using a first modulation unit; (3) an irradiation step of irradiating an observation object with the first branched light modulated by the first modulation unit along an irradiation direction according to the modulation frequency; (4) a combining step of combining the first branched light and the second branched light after passing through the observation object using a combining unit to output combined light; (5) an imaging step of receiving the combined light with an imaging unit having a plurality of pixels arranged in a direction intersecting a moving direction of an image of the observation object on an imaging plane on which an image of the observation object is formed, using the plurality of pixels, and repeatedly outputting a detection signal representing a one-dimensional interference image; and (6) and a processing step of generating a complex amplitude image for each of a plurality of irradiation directions onto the object to be observed corresponding to the plurality of modulation frequencies based on time series data of a one-dimensional interference image generated from detection signals repeatedly output from the imaging unit, and generating a three-dimensional refractive index distribution image of the object to be observed based on the complex amplitude images for each of the plurality of irradiation directions.

A second aspect of the observation method of the present invention includes, in addition to the first aspect, a second modulation step of imparting a frequency difference between the first branched light and the second branched light using a second modulation section provided on the optical path of the first branched light between the branching section and the first modulation section, or on the optical path of the second branched light between the branching section and the multiplexing section.

In a third aspect of the observation method of the present invention, in addition to the first or second aspect, in the first modulation step, a first modulation unit including an acousto-optical element that frequency-modulates light by Bragg diffraction is used to provide an RF signal having multiple different frequency components to the acousto-optical element, and the first branched light is incident on the acousto-optical element to modulate the first branched light at a modulation frequency corresponding to each frequency component of the RF signal.

In a fourth aspect of the observation method of the present invention, in addition to the first or second aspect, in the first modulation step, a first modulation unit including an acousto-optical element that frequency-modulates light by Raman-Nath diffraction is used to provide an RF signal of a single frequency to the acousto-optical element, and the first branched light is incident on the acousto-optical element, thereby modulating the first branched light with each of a plurality of modulation frequencies different from each other.

In a fifth aspect of the observation method of the present invention, in addition to the first or second aspect, in the first modulation step, a first modulation unit including a plate-like object on which diffraction gratings having a plurality of mutually different spatial frequency components are superimposed and recorded is used to move the plate-like object and pass the first branched light through the plate-like object, thereby modulating the first branched light with each of a plurality of mutually different modulation frequencies.

In a sixth aspect of the observation method of the present invention, in addition to the first or second aspect, in the first modulation step, a first modulation unit including a phase modulation type spatial light modulator is used to change the phase modulation amount at different speeds depending on the region of the modulation surface of the spatial light modulator, and the first branched light is incident on the modulation surface of the spatial light modulator to modulate the first branched light at a modulation frequency according to the speed of change of the phase modulation amount in each region of the modulation surface of the spatial light modulator.

In a seventh aspect of the observation method of the present invention, in addition to the first or second aspect, in the first modulation step, a first modulation unit including an intensity modulation type spatial light modulator is used to make the intensity modulation amount distribution along a predetermined direction of the modulation surface of the spatial light modulator have a plurality of spatial frequency components different from each other, and the intensity modulation amount distribution is moved in the predetermined direction on the modulation surface of the spatial light modulator, and the first branched light is made incident on the modulation surface of the spatial light modulator, thereby modulating the first branched light with each of the plurality of modulation frequencies different from each other.

The eighth aspect of the observation method of the present invention is, in addition to any of the first to seventh aspects, to detect the irradiation position of the first branched light on the movement path of the observation object, and based on this detected irradiation position, control the position of the optical system from the observation object to the imaging unit or the imaging unit so that the combined light can be received by multiple pixels of the imaging unit.

The present invention makes it possible to easily observe a moving object using an inexpensive configuration.

FIG. 1 is a diagram showing the configuration of an observation device 1. FIG. 2 is a diagram showing point images of the first branched light for each modulation frequency _Ωn formed on the front focal plane (P1 plane) of the lens 33. As shown in FIG. FIG. 3 is a diagram for explaining irradiation of the first branched light onto the observation object S by the lens 33. As shown in FIG. FIG. 4 is a diagram showing the configuration of the observation device 1A. FIG. 5 is a diagram showing the configuration of the observation device 1B. FIG. 6 is a diagram showing the configuration of an observation apparatus 1C. FIG. 7 is a diagram showing the configuration of the first modulation section 21A and the surrounding optical system. FIG. 8 is a diagram showing the configuration of the first modulation section 21A and the surrounding optical system. 9A and 9B are diagrams for explaining the diffraction gratings recorded on the plate-like object of the first modulation section 21A. Fig. 9A shows the modulation distribution of each diffraction grating having a plurality of spatial frequency components _kn ( _kmin to _kmax ). Fig. 9B shows the modulation distribution after the diffraction gratings having a plurality of spatial frequency components are superimposed. Fig. 10A is a diagram showing the first modulation section 21B and the surrounding optical system, and Fig. 10B is a diagram showing the modulation frequency of each region on the modulation surface of the spatial light modulator of the first modulation section 21B by shading. Fig. 11A is a diagram showing the first modulation section 21C and the surrounding optical system, and Fig. 11B is a diagram showing the modulation frequency of each region on the modulation surface of the spatial light modulator of the first modulation section 21C by shading. Fig. 12(a) is a diagram showing the first modulation section 21D and the surrounding optical system. Fig. 12(b) is a diagram showing the intensity modulation amount of each region on the modulation surface of the spatial light modulator of the first modulation section 21D by shading. Fig. 12(c) is a graph showing the intensity modulation amount of each region along the y direction on the modulation surface of the spatial light modulator of the first modulation section 21D. FIG. 13 is a diagram showing a modified example of the peripheral configuration of the imaging unit 23. In FIG.

Below, the embodiments for carrying out the present invention will be described in detail with reference to the attached drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated explanations will be omitted. The present invention is not limited to these examples, but is indicated by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

Figure 1 is a diagram showing the configuration of the observation device 1. The observation device 1 is a device for observing a moving observation target S, and includes a light source 11, a branching section 12, a combining section 15, a first modulation section 21, a second modulation section 22, an imaging section 23, and a processing section 24. The observation method using this observation device 1 includes a branching step, a combining step, a first modulation step, a second modulation step, an imaging step, and a processing step.

The light source 11 outputs light. Preferably, the light source 11 is a laser light source that outputs laser light having a single optical frequency _ω0 . The light source 11 is, for example, a He-Ne laser light source. The splitter 12 is an optical system optically connected to the light source 11. The splitter 12 receives the light output from the light source 11 and splits the light into two beams, a first split beam L1 and a second split beam L2 (splitting step). The splitter 12 outputs the first split beam L1 to the mirror 13 and outputs the second split beam L2 to the second modulator 22. The splitter 12 is a beam splitter and may be a half mirror.

The mirror 13 is optically connected to the branching unit 12. The mirror 13 receives the first branched light L1 output from the branching unit 12, reflects the first branched light L1, and outputs it to the first modulation unit 21. The mirror 14 is optically connected to the second modulation unit 22. The mirror 14 receives the second branched light L2 output from the branching unit 12 and passed through the second modulation unit 22, and reflects the second branched light L2 and outputs it to the lens 36.

The combining unit 15 is an optical system that inputs the first branched light L1 that is reflected by the mirror 13 and passes through the first modulation unit 21 and the observation object S, etc., and inputs the second branched light L2 that is reflected by the mirror 14 and passes through the lens 36, etc. The combining unit 15 combines the input first branched light L1 and second branched light L2, and outputs the combined light to the lens 35 (combining step). The combining unit 15 is a beam splitter, and may be a half mirror.

The imaging unit 23 receives the combined light output from the combining unit 15 and passing through the lens 35, and outputs a detection signal representing an interference image between the first branched light L1 and the second branched light L2 to the processing unit 24 (imaging step). The processing unit 24 generates a three-dimensional refractive index distribution image of the observation object S based on the detection signal repeatedly output from the imaging unit 23 (processing step).

The optical system from the branching section 12 through the mirrors 13 and 14 to the multiplexing section 15 constitutes a Mach-Zehnder interferometer. The observation object S moves so as to intersect with the optical path of the first branched light L1 from the mirror 13 to the multiplexing section 15. The observation object S is, for example, a cell moving in a line at an average speed of 20 mm/s through a flow path of a flow cytometer having a cross-sectional size of 250× ²⁵⁰ μm2. In the following, an xyz orthogonal coordinate system is set, the moving direction of the observation object S is the y direction, the direction parallel to the optical axis of the lens 34 at the rear of the observation object S is the z direction, and the direction perpendicular to both the y direction and the z direction is the x direction.

The first modulation section 21, lens 31, lens 32, and lens 33 are arranged in this order on the optical path of the first branched light L1 from the mirror 13 to the observation object S. The lens 34 is arranged on the optical path of the first branched light L1 from the observation object S to the multiplexing section 15. The second modulation section 22, lens 36, lens 37, and lens 38 are arranged in this order on the optical path of the second branched light L2 from the branching section 12 to the multiplexing section 15 via the mirror 14. The lens 35 is arranged on the optical path of the multiplexed light from the multiplexing section 15 to the imaging section 23.

The first modulation unit 21 inputs the first branched light L1 arriving from the mirror 13, modulates the first branched light L1 with each of a plurality of different modulation frequencies, and outputs the first branched light L1 after the frequency modulation (first modulation step). The first modulation unit 21 can be configured to include an acousto-optical element that frequency-modulates light by Bragg diffraction. In this case, the first modulation unit 21 provides an RF signal having a plurality of different frequency components to the acousto-optical element, and by making the first branched light L1 incident on the acousto-optical element, it is possible to modulate the first branched light L1 with a modulation frequency corresponding to each frequency component of the RF signal.

If the wavelength of the first branched light L1 is λ, the speed of the acoustic wave in the acousto-optical element is Va, and the n-th frequency component of the N frequency components Ω ₁ to Ω _N of the RF signal is Ω _n , then the diffraction angle θ _n of the first branched light L1 modulated with the frequency Ω _n is expressed by the following formula (1). The first modulating unit 21 can output the first branched light L1 of optical frequency ω ₀ +Ω _n modulated with each frequency Ω _n in the direction of the diffraction angle θ _n . That is, the first modulating unit 21 can output the first branched light L1 modulated with each of the N frequency components Ω ₁ to Ω _N in directions different from each other.

For example, suppose that diffracted light having a positive diffraction order (diffracted light output in the same direction as the propagation direction of the acoustic wave) is used. The moving speed Vs of the observation object S is 20 mm/s, the optical resolution Δy is 0.2 μm, and the bandwidth BW is 100 kHz (= Vs/Δy). The interval of the modulation frequencies is 100 kHz, N = 100, and Ω _n = (195 + 0.1n) MHz. That is, the minimum modulation frequency Ω ₁ is 195.1 MHz, and the maximum modulation frequency Ω _N is 205.0 MHz.

The lenses 31 to 33 are an optical system constituting an irradiation unit, and irradiate the first branched light L1 modulated by the first modulation unit 21 to the observation target S along an irradiation direction according to the modulation frequency Ω _n (irradiation step). The lenses 31 and 32 are cylindrical lenses. The lens 33 is a spherical lens. In this figure, the light flux of the first branched light L1 in the yz plane (within the paper surface) is shown by a solid line, and the light flux of the first branched light L1 in the xz plane (within the plane perpendicular to the paper surface) is shown by a dashed line (the same applies to the second branched light L2, and the same applies to the following figures). In addition, the lens shape of the cylindrical lens shown by the dashed line indicates the lens shape in the xz plane (within the plane perpendicular to the paper surface).

The lens 32 forms a point image of the first branched light L1 for each modulation frequency Ω _n at discrete positions along the y direction on the front focal plane (P1 plane) of the lens 33. The back focal plane (P2 plane) of the lens 33 is a plane including the moving path of the observation target S. FIG. 2 is a diagram showing the point images of the first branched light for each modulation frequency Ω _n formed on the front focal plane (P1 plane) of the lens 33. Each point image is indicated by a black circle. Since the P1 plane is a Fourier plane of the P2 plane, the horizontal axis (x axis) of the P1 plane corresponds to the spatial frequency k _x in the x direction of the P2 plane, and the vertical axis (y axis) of the P1 plane corresponds to the spatial frequency k _y in the y direction of the P2 plane. The lens 33 can input light from the point images of the first branched light L1 for each modulation frequency Ω _n shown in FIG. 2 and irradiate the observation target S with the first branched light L1 for each modulation frequency Ω _n as a plane wave along an irradiation direction according to the modulation frequency Ω _n .

3 is a diagram for explaining irradiation of the first branched light by the lens 33 onto the observation object S. This diagram shows that the y component of the wave vector when the first branched light L1 having an optical frequency ω _min (=ω ₀ +Ω ₁ ) modulated with a frequency Ω ₁ is incident on the P2 surface as a plane wave is k _min , and that the y component of the wave vector when the first branched light L1 having an optical frequency ω _max (=ω ₀ +Ω _N ) modulated with a frequency Ω _N is incident on the P2 surface as a plane wave is k _max .

Lenses 34 and 35 constitute an imaging optical system that inputs the first branched light L1 that has passed through the observation object S and forms an image of the observation object S on the imaging plane (P3 plane) of the imaging unit 23. The imaging plane (P3 plane) is optically conjugate with a plane (P2 plane) that includes the movement path of the observation object S. For example, lens 34 is an objective lens with a magnification of 60x, and lens 35 is an imaging lens with a focal length of 200 mm. A combining unit 15 is disposed on the optical path between lenses 34 and 35.

The second modulation unit 22 provides a frequency difference between the first branched light L1 and the second branched light L2 (second modulation step). The second modulation unit 22 is disposed on the optical path of the second branched light L2. The second modulation unit 22 inputs the second branched light L2, and frequency-modulates and outputs the second branched light L2. The second modulation unit 22 may include, for example, an acousto-optic frequency shifter (AOFS, for example, model number 1250C by ISOMET), a function generator (for example, AFG3252C by Tektronics), and a high-speed amplifier. In this case, the sine wave signal output from the function generator is amplified by the high-speed amplifier with an amplification factor of, for example, +25 dB, and the sine wave signal with an amplitude of, for example, 25 dBm due to the amplification is provided to the AOFS. By inputting the second branched light L2 to this AOFS, the second branched light L2 can be modulated at a frequency Ω _LO . If diffracted light having a positive diffraction order (diffracted light output in the same direction as the propagation direction of the acoustic wave) is used, the optical frequency ω _LO of the modulated second split light L2 becomes ω ₀ +Ω _LO , where Ω _LO =195 MHz, for example.

The second modulation unit 22 is provided to adjust the optical heterodyne frequency ΔΩ _n (the optical frequency difference between the first branched light L1 and the second branched light L2 when multiplexed by the multiplexing unit 15) to be equal to or less than half the sampling frequency Ω _S of the imaging unit 23. Therefore, if the modulation frequency Ω _n of the first branched light L1 is sufficiently small, and even if Ω _LO =0, the optical heterodyne frequency ΔΩ _n is equal to or less than half the sampling frequency Ω _S of the imaging unit 23, the second modulation unit 22 does not need to be provided.

Lenses 36 to 38 form an optical system that causes the second branched light L2 modulated by the second modulation section 22 to enter the multiplexing section 15. Lenses 36 and 37 are each spherical lenses and form a beam expander that expands the beam diameter of the second branched light L2. Lens 38 is a cylindrical lens. Lenses 36 to 38, together with lens 35, focus the second branched light L2 in a line shape on the imaging plane of the imaging section 23.

The imaging unit 23 has a plurality of pixels that are one-dimensionally arranged in a direction intersecting the movement direction of the image of the observation object S by the first branched light L1 on the imaging plane (P3 plane) where the image of the observation object S is formed. A sensor such as a line sensor can be used as the imaging unit 23. The second branched light L2 is focused in a line shape along these one-dimensionally arranged pixels. The imaging unit 23 receives the combined light by the one-dimensionally arranged pixels and repeatedly outputs a detection signal representing a one-dimensional interference image.

It is preferable that the relationship of the following formula (2) be satisfied among the lowest optical frequency ω _min (=ω ₀ +Ω ₁ ) of the first branched light L1 frequency-modulated by the first modulator 21, the highest optical frequency ω _max (=ω ₀ +Ω _N ) of the first branched light L1 frequency-modulated by the first modulator 21, and the optical frequency ω _LO (=ω ₀ +Ω _LO ) of the second branched light L2 frequency-modulated by the second modulator 22. ω ₀ is the optical frequency of the light output from the light source 11. In addition, it is preferable that the sampling frequency Ω _S of the imaging unit 23 satisfy the condition of the following formula (3) or (4).

If the maximum modulation frequency _ΩN of the first branched light L1 is 205 MHz and the modulation frequency ΩLO of the second branched light _L2 is 195 MHz, the maximum optical heterodyne frequency is 10 MHz, so that the sampling frequency _ΩS of the imaging unit 23 should be set to 20 MHz.

The processing unit 24 generates a complex amplitude image for each of a plurality of irradiation directions of the observation object S corresponding to a plurality of modulation frequencies based on the time series data of the one-dimensional interference image generated from the detection signal repeatedly output from the imaging unit 23. The processing unit 24 is, for example, a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), a storage medium such as a RAM (Random Access Memory) or a ROM (Read Only Memory), a communication module, and an input/output module (computer, etc.). For example, the processing unit 24 may be configured with an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). Then, the processing unit 24 generates a three-dimensional refractive index distribution image of the observation object S based on the complex amplitude images for each of the plurality of irradiation directions. Specifically, it is as follows.

The time series data of the one-dimensional interference image generated from the detection signals repeatedly output from the imaging unit 23 is represented as I(x,t). x is a variable representing the position (x-direction position) of each of the multiple pixels arranged one-dimensionally on the imaging surface of the imaging unit 23. t is a variable representing the time when the imaging unit 23 captures the one-dimensional interference image. The processing unit 24 performs a Fourier transform on this time series data I(x,t) with respect to time t. The result of this Fourier transform is represented as J(x,f). f is a variable representing frequency.

The processing unit 24 extracts data in a frequency range (ΔΩ _n -BW/2<f<ΔΩ _n +BW/2) with a bandwidth BW centered on the optical heterodyne frequency ΔΩ _n from the data J(x,f), and performs a Fourier transform on the extracted frequency range data with respect to frequency f. The result of this Fourier transform is represented as I _n (x,t). Here, since the observation target S is moving in the y direction, the variable t representing the time can be converted to a variable y representing the position in the y direction. Therefore, I _n (x,t) can be expressed as I _n (x,y).

I _n (x, y) represents a _two- dimensional complex amplitude image of the observation object S by the first branched light L1 that reaches the imaging surface of the imaging unit 23 when the observation object S is irradiated with the first branched light L1 along an irradiation direction corresponding to the optical heterodyne frequency ΔΩ _n (i.e., an irradiation direction corresponding to the modulation frequency Ω n). The processing unit 24 generates a complex amplitude image I _n (x, y) for the irradiation direction of the observation object S corresponding to each modulation frequency Ω _n .

Then, the processing unit 24 generates a three-dimensional refractive index distribution image of the observation object S based on the complex amplitude images I ₁ (x, y) to I _N (x, y) in each of the multiple irradiation directions. When generating a three-dimensional refractive index distribution image based on the complex amplitude images I ₁ (x, y) to I _N (x, y), the method described in Non-Patent Document 1 or Non-Patent Document 2 can be used. The method described in Non-Patent Document 1 uses the Fourier Slice theorem and is applicable when light scattering in the observation object S can be ignored. The method described in Non-Patent Document 2 uses the Fourier Diffraction theorem and generates a three-dimensional refractive index distribution image of the observation object S taking into account the light scattering in the observation object S.

As described above, the observation device 1 can have a high line rate (sampling frequency Ω _S ) and an inexpensive configuration because it can use a line sensor instead of an area sensor as the imaging unit 23. Furthermore, because the optical system from the observation object S to the imaging unit 23 can be a normal imaging optical system, this optical system can be easily adjusted and the observation object S can be easily observed.

Next, modified examples of the overall configuration of the observation device 1 will be described with reference to FIGS.
The observation device 1A of the first modified example shown in Fig. 4 differs from the observation device 1 (Fig. 1) in the configuration in terms of the position where the second modulation section 22 is arranged. In the observation device 1 (Fig. 1), the second modulation section 22 is arranged on the optical path of the second branched light L2. In contrast, in the observation device 1A (Fig. 4), the second modulation section 22 is arranged on the optical path of the first branched light L1 between the branching section 12 and the first modulation section 21. Even in this configuration, the second modulation section 22 can be adjusted so that the optical heterodyne frequency ΔΩ _n (the optical frequency difference between the first branched light L1 and the second branched light L2 when multiplexed by the multiplexing section 15) is equal to or less than half the sampling frequency Ω _S of the imaging section 23.

The observation device 1B of the second modified example shown in Fig. 5 differs from the respective configurations of the observation device 1 (Fig. 1) and the observation device 1A (Fig. 4) in that it does not include the second modulation unit 22. As described above, if the modulation frequency _Ωn of the first branched light is sufficiently small, and even if _ΩLO = 0, the optical heterodyne frequency _ΔΩn is equal to or less than half the sampling frequency _ΩS of the imaging unit 23, the second modulation unit 22 does not need to be provided.

The third modified observation device 1C shown in FIG. 6 differs from the configuration of the observation device 1 (FIG. 1) in the arrangement of the beam combining unit 15, lens 34, and lens 35. In the observation device 1 (FIG. 1), the beam combining unit 15 was placed on the optical path between lens 34 and lens 35. In contrast, in the observation device 1C (FIG. 6), the beam combining unit 15 is placed after lens 35. Accordingly, in the observation device 1C (FIG. 6), lens 36 is a spherical lens, and lenses 37 and 38 are cylindrical lenses.

In the observation device 1C of the third modified example shown in Fig. 6, the second modulator 22 may be disposed not on the optical path of the second branched light L2 but on the optical path of the first branched light L1 between the branch 12 and the first modulator 21. In the observation device 1C of the third modified example shown in Fig. 6, if the modulation frequency _Ωn of the first branched light is sufficiently small, and even if _ΩLO = 0, the optical heterodyne frequency _ΔΩn is equal to or less than half the sampling frequency _ΩS of the imaging unit 23, the second modulator 22 may not be provided.

Next, examples of the configuration of the first modulation section 21 will be described with reference to FIGS.
The first modulation section 21, which has already been described, is configured to include an acousto-optic element that frequency-modulates light by Bragg diffraction. This first modulation section 21 provides an RF signal having a plurality of different frequency components to the acousto-optic element, and by making the first branched light L1 incident on the acousto-optic element, it is possible to modulate the first branched light L1 at a modulation frequency corresponding to each frequency component of the RF signal. The first modulation section 21 is able to output the first branched light L1 modulated with each of the N frequency components Ω ₁ to Ω _N in different directions.

The first modulation section of the first modification may be configured to include an acousto-optical element that frequency-modulates light by Raman-Nath diffraction. In this case, the first modulation section can modulate the first branched light L1 with a plurality of modulation frequencies that differ from each other according to the diffraction order by providing an RF signal of a single frequency (e.g., 3 MHz) to the acousto-optical element and making the first branched light L1 incident on the acousto-optical element. The first modulation section can output the first branched light L1 modulated with each of the N frequency components Ω ₁ to Ω _N in directions that differ from each other according to the diffraction order. The optical system of the irradiation section between the first modulation section and the observation object S can be configured similarly to that shown in FIG. 1.

The first modulation section 21A of the second modified example shown in FIG. 7 and FIG. 8 includes a plate-like object on which diffraction gratings having a plurality of different spatial frequency components are superimposed and recorded. The plate-like object of the first modulation section 21A shown in these figures is a disk-like object parallel to the xy plane, can rotate around a central axis O parallel to the z direction, and has diffraction gratings having a plurality of spatial frequency components superimposed and recorded along the circumferential direction. The first modulation section 21A rotates the plate-like object at a constant speed around the central axis and passes the first branched light L1 through the plate-like object. At this time, the first branched light L1 is incident on a position approximately in the xz plane including the central axis O of the rotation of the plate-like object, and the plate-like object is moved in the y direction at the incident position. This allows the first branched light L1 to be modulated with each of a plurality of different modulation frequencies.

In the configuration shown in Fig. 7, the optical system of the irradiation unit between the first modulation unit 21A and the observation object S includes two cylindrical lenses 41, 42. In the configuration shown in Fig. 8, the optical system of the irradiation unit between the first modulation unit 21A and the observation object S includes one cylindrical lens 43. In either case, the irradiation unit irradiates the observation object S with the first branched light L1 modulated by the first modulation unit 21A along an irradiation direction according to the modulation frequency _Ωn .

9 is a diagram for explaining the diffraction grating recorded on the plate-like object of the first modulation section 21A of the second modified example. In this diagram, the horizontal axis represents the circumferential position, and the vertical axis represents the modulation amount. Fig. 9(a) shows the modulation distribution of each diffraction grating having a plurality of spatial frequency components _kn ( _kmin to _kmax ). Fig. 9(b) shows the modulation distribution after the diffraction gratings having a plurality of spatial frequency components are superimposed.

The plate-like object of the first modulation unit 21A shown in Figures 7 and 8 is disk-shaped, but the plate-like object on which diffraction gratings having multiple different spatial frequency components are superimposed and recorded may be strip-shaped. In this case, the strip-shaped plate-like object of the first modulation unit 21A can be moved in the y direction, and diffraction gratings having multiple spatial frequency components are superimposed and recorded along the moving direction. The first modulation unit 21A moves the strip-shaped plate-like object in the y direction and passes the first branched light L1 through the plate-like object, thereby modulating the first branched light L1 with each of multiple different modulation frequencies. The two ends of the strip-shaped plate-like object are connected to each other to form a ring shape, and the ring-shaped plate-like object is rotated, so that the plate-like object can be moved in the y direction for a long period of time.

The first modulation section 21B of the third modified example shown in FIG. 10 includes a phase-modulation type spatial light modulator (SLM). In this figure, a reflective spatial light modulator is shown. FIG. 10(a) shows the first modulation section 21B and the surrounding optical system. FIG. 10(b) shows the modulation frequency of each region on the modulation surface of the spatial light modulator of the first modulation section 21B by shading.

The spatial light modulator of the first modulation unit 21B changes the phase modulation amount at different speeds in each of a plurality of regions divided along the y direction in a region of the modulation surface (P1 surface) having a limited width in the x direction. That is, the phase modulation amount φ of each region is changed continuously in time with respect to time t in a relationship of φ=Ω _n t. The first branched light L1 passes through cylindrical lenses 52 and 53, is reflected by the beam splitter 51, and is incident on the modulation surface of the spatial light modulator. The cylindrical lens 52 is a concave lens, and the cylindrical lens 53 is a convex lens. By making the first branched light L1 incident on a region of the modulation surface of the spatial light modulator extending in the y direction (a region having a limited width in the x direction), the first branched light L1 can be modulated at the modulation frequency Ω _n in each region along the y direction of the modulation surface of the spatial light modulator. The modulated first split light L1 is transmitted through the beam splitter 51, and then irradiated onto the observation object S along an irradiation direction according to the modulation frequency _Ωn by the cylindrical lens 54. Note that the light reflected from the modulation surface, whose phase modulation amount does not change continuously over time, is prevented from being irradiated onto the observation object S by a mask or the like (not shown).

The first modulation section 21C of the fourth modified example shown in FIG. 11 also includes a phase-modulation type spatial light modulator (SLM). In this figure, a reflective spatial light modulator is also shown. FIG. 11(a) shows the first modulation section 21C and the surrounding optical system. FIG. 11(b) shows the modulation frequency of each region on the modulation surface of the spatial light modulator of the first modulation section 21C by shading.

The spatial light modulator of the first modulation unit 21C changes the phase modulation amount at different speeds in each of a plurality of regions divided along the y direction in the region covering the entire x direction of the modulation surface (P1 surface). That is, the phase modulation amount φ of each region is changed continuously in time with respect to time t in a relationship of φ=Ω _n t. The first branched light is reflected by the beam splitter 61 after the beam diameter is expanded by the lenses 62 and 63, and is incident on substantially the entire modulation surface of the spatial light modulator. By making the first branched light L1 incident on substantially the entire modulation surface of the spatial light modulator, the first branched light L1 can be modulated at the modulation frequency Ω _n in each region along the y direction of the modulation surface of the spatial light modulator. The modulated first branched light L1 is transmitted through the beam splitter 61, and then irradiated by the lenses 64 and 65 to the observation object S along the irradiation direction according to the modulation frequency Ω _n . Note that the light reflected by the modulation surface in which the phase modulation amount does not change continuously in time is prevented from being irradiated to the observation object S by a mask or the like (not shown).

The first modulation section 21D of the fifth modified example shown in FIG. 12 includes an intensity modulation type spatial light modulator (SLM). A reflective spatial light modulator is also shown in this figure. FIG. 12(a) shows the first modulation section 21D and the surrounding optical system. FIG. 12(b) shows the intensity modulation amount of each region on the modulation surface of the spatial light modulator of the first modulation section 21D using shading. FIG. 12(c) shows the intensity modulation amount of each region along the y direction on the modulation surface of the spatial light modulator of the first modulation section 21D in a graph.

The spatial light modulator of the first modulation section 21D sets an intensity modulation amount in each of a plurality of regions divided along the y direction in a region of the modulation plane (P0 plane) having a limited width in the x direction, and moves the y direction distribution of the intensity modulation amount in the y direction. If the intensity modulation amount distribution is a sine wave shape with a period k _i and the y direction movement speed is V, a phase shift of exp(ik _i Vt) occurs in the Fourier plane (P1 plane) relative to the modulation plane (P0 plane), and a frequency modulation of Ω=k _i V occurs. The fifth modified example utilizes this fact.

The intensity modulation amount distribution along the y direction on the modulation surface of the spatial light modulator is assumed to have a plurality of spatial frequency components different from each other, and the intensity modulation amount distribution is moved at a constant speed in the y direction on the modulation surface of the spatial light modulator. The first branched light L1 passes through cylindrical lenses 72 and 73, is reflected by the beam splitter 71, and is incident on the modulation surface of the spatial light modulator. The cylindrical lens 72 is a concave lens, and the cylindrical lens 73 is a convex lens. The front focal plane of the cylindrical lens 74 is the modulation surface (P0 surface) of the spatial light modulator, and the back focal plane of the cylindrical lens 74 is the P1 surface. The front focal plane of the cylindrical lens 75 is the P1 surface, and the back focal plane of the cylindrical lens 75 is the surface (P2 surface) that includes the movement path of the observation object S.

When the first branched light L1 is incident on a region extending in the y direction (region with a limited width in the x direction) of the modulation surface (P0 surface) of the spatial light modulator, a point image of the first branched light L1 modulated at the modulation frequency Ω _n can be formed in each region along the y direction on the P1 surface, as in Fig. 2. The lens 75 inputs light from the point image of the first branched light L1 for each modulation frequency Ω _n on the P1 surface, and can irradiate the first branched light L1 for each modulation frequency Ω _n as a plane wave onto the observation object S along an irradiation direction according to the modulation frequency Ω _n .

In addition, the first modulation units 21B to 21D in the third to fifth modified examples are configured to include a reflective spatial light modulator, but may also be configured to include a transmissive spatial light modulator. Also, a DMD (Digital Mirror Device) may be used as a reflective intensity modulation spatial light modulator, in which case multi-stage intensity modulation can be applied to the light by PWM (Pulse Width Modulation) modulation.

Next, a modified example of the configuration around the imaging unit 23 will be described with reference to FIG. 13. In the modified configuration shown in this figure, a lens 81, an irradiation position detection unit 82, a control unit 83, and a movement unit 84 are further provided. The multiplexing unit 15, which is disposed on the optical path between the lens 34 and the lens 35, multiplexes the first branched light L1 and the second branched light L2, and outputs the multiplexed light to the lens 35 and also to the lens 81.

Lens 81 is disposed on the optical path of the combined light between combining unit 15 and irradiation position detection unit 82. Lens 34 and lens 81 constitute an imaging optical system that inputs first branched light L1 that has passed through the observation object S and forms an image of the observation object S on the imaging plane (P4 plane) of irradiation position detection unit 82. Irradiation position detection unit 82 detects the irradiation position of first branched light L1 on the movement path of observation object S, and is, for example, an area camera that can capture a two-dimensional image. This area camera does not need to be capable of high-speed imaging.

The control unit 83 drives the moving unit 84 based on the irradiation position detected by the irradiation position detection unit 82, thereby controlling the y-direction position of the imaging unit 23 so that the multiple pixels arranged one-dimensionally in the imaging unit 23 can receive the combined light. The moving unit 84 moves the imaging unit 23 in the movement direction (y direction) of the image of the observation object S on the imaging plane (P3 plane) of the imaging unit 23 in accordance with the drive by the control unit 83, and is, for example, a uniaxial stage with an electric actuator.

By adopting such a configuration, even if the irradiation position of the first branched light L1 on the movement path of the observed object S drifts over time in the y direction, the incident position of the first branched light L1 on the imaging plane (P3 plane) of the imaging unit 23 can be fixed, so that the imaging unit 23 can always stably acquire a one-dimensional interference image.

In order to enable the multiple pixels arranged one-dimensionally in the imaging unit 23 to stably receive the combined light, in addition to or instead of controlling the y-direction position of the imaging unit 23, the optical system from the observation object S to the imaging unit 23 may be controlled. In the latter case, for example, a mirror with a variable orientation of the reflecting surface may be disposed midway in the optical system from the observation object S to the imaging unit 23, and the orientation of the reflecting surface of this mirror may be controlled by the control unit 83.

In addition, the irradiation position detection unit 82 only needs to be able to detect the irradiation position of the first branched light L1 on the movement path of the observation object S, and therefore only needs to receive the first branched light L1, not the combined light. For example, in the configuration shown in FIG. 6, a beam splitter may be disposed between the lens 34 and the lens 35, and an image of the observation object S by the first branched light L1 branched by this beam splitter may be formed on the imaging plane (P4 plane) of the irradiation position detection unit 82 by the lens 81.

The irradiation position detection unit 82 may be a profile sensor with a center of gravity calculation circuit for detecting the spot light incidence position (e.g., S15366-512 manufactured by Hamamatsu Photonics K.K.), or a PSD (Position Sensitive Detector, e.g., S2044 manufactured by Hamamatsu Photonics K.K.).

1, 1A to 1C...observation device, 11...light source, 12...branching section, 13, 14...mirror, 15...combining section, 21, 21A to 21D...first modulation section, 22...second modulation section, 23...imaging section, 24...processing section, 31 to 38...lenses.

Claims (16)

An apparatus for observing a moving observation object, comprising:
A light source that outputs light;
a branching unit that branches the light into a first branched light and a second branched light;
a first modulation unit that modulates the first branched light with each of a plurality of modulation frequencies different from one another;
an illumination unit that illuminates the first branched light modulated by the first modulation unit onto the observation object along an illumination direction corresponding to a modulation frequency;
a multiplexing unit that multiplexes the first split light and the second split light that have passed through the observation object and outputs multiplexed light;
an imaging section having a plurality of pixels arranged in a direction intersecting a moving direction of the image of the observed object on an imaging surface on which the image of the observed object is formed, the imaging section receiving the combined light with the plurality of pixels and repeatedly outputting a detection signal representing a one-dimensional interference image;
a processing unit that generates a complex amplitude image for each of a plurality of irradiation directions of the observation object corresponding to the plurality of modulation frequencies based on time series data of the one-dimensional interference image generated from detection signals repeatedly output from the imaging unit, and generates a three-dimensional refractive index distribution image of the observation object based on the complex amplitude images for each of the plurality of irradiation directions;
An observation device comprising: a second modulation section provided on an optical path of the first branched light between the branching section and the first modulation section, or on an optical path of the second branched light between the branching section and the multiplexing section, and configured to provide a frequency difference between the first branched light and the second branched light;
The observation device according to claim 1. the first modulation unit includes an acousto-optical element that frequency-modulates light by Bragg diffraction, and provides an RF signal having a plurality of frequency components different from each other to the acousto-optical element, and modulates the first branched light at a modulation frequency corresponding to each frequency component of the RF signal by inputting the first branched light to the acousto-optical element;
3. The observation device according to claim 1 or 2. the first modulation unit includes an acousto-optical element that frequency-modulates light by Raman-Nath diffraction, and provides an RF signal of a single frequency to the acousto-optical element, and modulates the first branched light with each of a plurality of modulation frequencies different from each other by making the first branched light incident on the acousto-optical element;
3. The observation device according to claim 1 or 2. the first modulation unit includes a plate-like object on which diffraction gratings having a plurality of spatial frequency components different from each other are superimposed and recorded, and the first branched light is modulated with each of a plurality of modulation frequencies different from each other by moving the plate-like object and passing the first branched light through the plate-like object.
3. The observation device according to claim 1 or 2. the first modulation unit includes a phase modulation type spatial light modulator, and changes a phase modulation amount at different speeds depending on regions of a modulation surface of the spatial light modulator, and modulates the first branched light at a modulation frequency according to a speed of change of the phase modulation amount in each region of the modulation surface of the spatial light modulator by making the first branched light incident on the modulation surface of the spatial light modulator;
3. The observation device according to claim 1 or 2. the first modulation unit includes an intensity modulation type spatial light modulator, and an intensity modulation amount distribution along a predetermined direction on a modulation surface of the spatial light modulator has a plurality of spatial frequency components different from each other, and the intensity modulation amount distribution is moved in the predetermined direction on the modulation surface of the spatial light modulator, and the first branched light is incident on the modulation surface of the spatial light modulator, thereby modulating the first branched light with each of a plurality of modulation frequencies different from each other.
3. The observation device according to claim 1 or 2. an irradiation position detection unit that detects an irradiation position of the first split light on a moving path of the observation object;
a control unit that controls a position of an optical system from the observation object to the imaging unit or a position of the imaging unit based on the irradiation position detected by the irradiation position detection unit so that the combined light can be received by the plurality of pixels of the imaging unit; and
The observation device according to claim 1 or 2, further comprising: A method for observing a moving observation object, comprising the steps of:
a branching step of branching light output from the light source into a first branched light and a second branched light using a branching unit;
a first modulation step of modulating the first branched light with each of a plurality of modulation frequencies different from each other using a first modulation unit;
an irradiation step of irradiating the first branched light modulated by the first modulation unit onto the observation object along an irradiation direction according to a modulation frequency;
a combining step of combining the first split light and the second split light having passed through the observation object using a combining unit and outputting a combined light;
an imaging step of receiving the combined light with a plurality of pixels using an imaging unit having a plurality of pixels arranged in a direction intersecting a moving direction of the image of the observed object on an imaging surface on which the image of the observed object is formed, and repeatedly outputting a detection signal representing a one-dimensional interference image;
a processing step of generating a complex amplitude image for each of a plurality of irradiation directions of the observation object corresponding to the plurality of modulation frequencies based on time series data of the one-dimensional interference image generated from the detection signals repeatedly output from the imaging unit, and generating a three-dimensional refractive index distribution image of the observation object based on the complex amplitude images for each of the plurality of irradiation directions;
An observation method comprising: a second modulation step of providing a frequency difference between the first branched light and the second branched light by using a second modulation unit provided on an optical path of the first branched light between the branching unit and the first modulation unit, or on an optical path of the second branched light between the branching unit and the multiplexing unit,
The observation method according to claim 9. In the first modulation step, an RF signal having a plurality of different frequency components is provided to the acousto-optical element by using the first modulation unit including an acousto-optical element that frequency-modulates light by Bragg diffraction, and the first branched light is input to the acousto-optical element, thereby modulating the first branched light with a modulation frequency corresponding to each frequency component of the RF signal.
The observation method according to claim 9 or 10. In the first modulation step, an RF signal having a single frequency is provided to the acousto-optical element, which includes an acousto-optical element that frequency-modulates light by Raman-Nath diffraction, and the first branched light is input to the acousto-optical element, thereby modulating the first branched light with each of a plurality of modulation frequencies different from each other.
The observation method according to claim 9 or 10. In the first modulation step, the first modulation unit includes a plate-like object on which diffraction gratings having a plurality of spatial frequency components different from each other are superimposed and recorded, and the plate-like object is moved while the first branched light passes through the plate-like object, thereby modulating the first branched light with each of a plurality of modulation frequencies different from each other.
The observation method according to claim 9 or 10. In the first modulation step, the first modulation unit including a phase modulation type spatial light modulator is used to change the phase modulation amount at different speeds depending on the region of the modulation surface of the spatial light modulator, and the first branched light is incident on the modulation surface of the spatial light modulator to modulate the first branched light at a modulation frequency according to the speed of change of the phase modulation amount in each region of the modulation surface of the spatial light modulator.
The observation method according to claim 9 or 10. In the first modulation step, the first modulation unit includes an intensity modulation type spatial light modulator, and an intensity modulation amount distribution along a predetermined direction of a modulation surface of the spatial light modulator is made to have a plurality of spatial frequency components different from each other, and the intensity modulation amount distribution is moved in the predetermined direction on the modulation surface of the spatial light modulator, and the first branched light is made incident on the modulation surface of the spatial light modulator, thereby modulating the first branched light with each of a plurality of modulation frequencies different from each other.
The observation method according to claim 9 or 10. detecting an irradiation position of the first branched light on a moving path of the observation object, and controlling a position of the imaging unit or an optical system from the observation object to the imaging unit based on the detected irradiation position so that the combined light can be received by the multiple pixels of the imaging unit;
The observation method according to claim 9 or 10.

Images