JP7014681B2 - Phase image acquisition device and phase image acquisition method - Google Patents

Description

The present invention relates to an apparatus and a method for acquiring a phase image of an object.

Patent Document 1 discloses an invention of an apparatus and a method capable of acquiring an interference image and a phase image of an object. The apparatus described in this document includes a light source, a two-luminous flux interferometer that splits the light output from this light source into two luminous fluxes, and then combines the two luminous fluxes to output interference light, and the two-luminous flux interferometer. It is equipped with an imager that receives the interference light output from. In this device, the optical path length difference between the two light beams in the two light beam interferometer is stabilized at each of a plurality of different set values, and the interference image of the object is acquired by the imager in each state, and these acquired plurality. The phase image is obtained based on the interference image of.

The conventional phase image acquisition device detects this optical path length difference in order to stabilize the optical path length difference between the two light fluxes in the two light flux interferometer, and also makes the two light flux length difference a set value. The position of the mirror arranged on any of the optical paths is feedback-controlled.

However, for example, when a phase image of a cell as an object contained in a culture solution is obtained, the optical path length of the light transmitted through the culture solution fluctuates with time due to the fluctuation of the liquid surface of the culture solution. The width of the fluctuation of the liquid level is one wavelength or more, and the frequency of the fluctuation is several tens of Hz. Due to this fluctuation in the liquid level, it was difficult to stabilize the optical path length difference between the two luminous fluxes in the two luminous flux interferometer.

Conventionally, in order to deal with such a problem, the culture solution and the gas (generally air) are separated by covering the culture solution with a lid or a cover glass or using a submerged submerged objective lens. It was devised so that light would not pass through the interface (see Non-Patent Document 1). By this device, it is possible to prevent the liquid level from fluctuating on the optical path, and it is possible to stabilize the optical path length difference between the two luminous fluxes in the two luminous flux interferometer.

International Publication No. 2016/12248

Toyohiko Yamauchi, et al., "Miniaturization of Quantitative Phase Contrast Microscope and Various Applied Measurements = Desktop Double Luminous Flux Interference Microscope without Air Surface Plate =", Optical Alliance, February 2017, pp.26-29. Daniel Malacara, et al, "Interferogram Analysis for Optical Testing," Second Edition, Taylor & Francis, (2005). Mitsuo Takeda, et al., "Three-dimensional measurement and interference measurement by fast Fourier transform method", Optics, Vol. 10, No. 6, pp.476-480, (1981).

Conventionally, a lid, a cover glass, or a submerged submerged objective lens provided for suppressing fluctuations in the liquid level come into contact with the culture solution, which may cause a problem that cells are contaminated.

The present invention has been made to solve the above-mentioned problems, and obtains a phase image of an object even when the optical path length difference between the two light fluxes fluctuates with time in the two-luminous flux interferometer. It is an object of the present invention to provide a device and a method which can be used.

The phase image acquisition device of the present invention comprises (1) a light source, a two-beam interference meter that splits the light output from the light source into two light beams, and then combines the two light beams to output interference light, and two-beam interference. Interference image acquisition unit that includes an imager that receives the interference light output from the meter and acquires an interference image of an object placed on the optical path of any of the light beams in the two-beam interference meter by capturing the interference image with the imager. And (2) an arithmetic unit that generates a phase image of an object based on the interference image acquired by the interference image acquisition unit. The interference image acquisition unit acquires M interference images of an object in a state where the optical path length difference between the two light fluxes fluctuates with time in the two-luminous flux interferometer. The calculation unit estimates the offset phase shift amount of the entire image for each of the M interference images, sets a set including N interference images having different offset phase shift amounts from the M interference images, and sets the set. A phase image is generated based on the N interference images included in. However, 3 ≦ N ≦ M.

In the phase image acquisition device according to one aspect of the present invention, the arithmetic unit sets a plurality of sets and generates a phase image based on N interference images included in each set for each of the plurality of sets, and a plurality of sets are generated. After correcting the offset phase shift amount for the phase images generated in each set, the summing is performed. The arithmetic unit may sort the M interference images in ascending or descending order with respect to the offset phase shift amount, and then distribute them into a plurality of sets in order. Alternatively, the arithmetic unit may adjust the coefficients multiplied by the interfering image to generate the phase image by iterative optimization calculations.

Further, the phase image acquisition device of the present invention may include an interference image acquisition unit similar to the above and the following calculation unit. That is, the calculation unit estimates the distribution of the offset phase shift amount for each of the M interference images, and for each pixel position, N pieces having different offset phase shift amounts from the corresponding pixels of the M interference images. A set including pixels may be set, a phase value may be obtained for each pixel position based on N pixels included in the set, and a phase image may be generated based on the phase value of each pixel position. However, 3 ≦ N ≦ M. At this time, on one aspect, the calculation unit sets a plurality of sets, obtains a phase value based on N pixels included in each set for each of the plurality of sets, and obtains a phase value for each of the plurality of sets. The offset phase shift amount is corrected and then integrated, and a phase image is generated based on the integrated value of the phase values at each pixel position. Further, the arithmetic unit may sort the pixels of each of the M interference images in ascending or descending order with respect to the offset phase shift amount for each pixel position, and then distribute them into a plurality of sets in order.

In the phase image acquisition device according to the other aspect of the present invention, the two-luminous flux interferometer uses the optical path length difference between the two luminous fluxes due to the fluctuation of the liquid level of the liquid arranged on the optical paths of both or one of the two luminous fluxes. Causes temporal fluctuations in.

In the phase image acquisition device according to still another aspect of the present invention, the light source outputs incoherent light.

The phase image acquisition method of the present invention comprises (1) a light source, a double light beam interferometer that branches the light output from the light source into two light sources, and then combines the two light sources to output interference light, and two light beam interferences. Using an interference image acquisition unit including an imager that receives the interference light output from the meter, in the two-beam interferometer, the difference in optical path length between the two light beams fluctuates with time. The interference image acquisition step of capturing and acquiring M interference images with an imager for an object placed on the optical path of any light beam, and (2) offset phase shift of the entire image for each of the M interference images. An offset phase shift amount estimation step for estimating the amount, (3) a set setting step for setting a set including N interference images having different offset phase shift amounts from among M interference images, and (4) a set. The present invention includes a phase image generation step of generating a phase image based on the N interference images included in the above. However, 3 ≦ N ≦ M.

In the phase image acquisition method according to one aspect of the present invention, a plurality of sets are set in the set setting step, and a phase image is generated based on N interference images included in each set for each of the plurality of sets in the phase image generation step. Further, an integration step of correcting the offset phase shift amount and then integrating the phase images generated by each of the plurality of sets is provided. In the set setting step, the M interference images may be sorted in ascending or descending order with respect to the offset phase shift amount, and then sorted into a plurality of sets in order. Alternatively, in the phase image generation step, the coefficients multiplied by the interfering image to generate the phase image may be adjusted by iterative optimization calculations.

Further, the phase image acquisition method of the present invention includes (1) an interference image acquisition step similar to the above, and (2) an offset phase shift amount estimation step for estimating the distribution of the offset phase shift amount for each of the M interference images. , (3) For each pixel position, a set setting step for setting a set including N pixels having different offset phase shift amounts from the corresponding pixels of each of the M interference images, and (4) for each pixel position. , A phase value calculation step for obtaining a phase value based on N pixels included in the set, and (5) a phase image generation step for generating a phase image based on the phase value at each pixel position may be provided. .. However, 3 ≦ N ≦ M. At this time, in one aspect, in the phase image acquisition method of the present invention, a plurality of sets are set in the set setting step, and the phase value is set based on the N pixels included in each set for each of the plurality of sets in the phase value calculation step. A phase image is generated based on the integrated value of the phase value of each pixel position in the phase image generation step. do. In the set setting step, for each pixel position, the pixels of each of the M interference images may be rearranged in ascending or descending order with respect to the offset phase shift amount, and then sorted into a plurality of sets in order.

In the phase image acquisition method according to the other aspect of the present invention, in the interference image acquisition step, due to the fluctuation of the liquid level of the liquid arranged on the optical path of both or one of the two luminous fluxes in the two luminous flux interferometer, the two light beams are obtained. It causes a temporal variation in the optical path length difference between the moments.

In the phase image acquisition method according to still another aspect of the present invention, the light source outputs incoherent light.

According to the present invention, it is possible to acquire a phase image of an object even in a state where the optical path length difference between the two luminous fluxes fluctuates with time in the two luminous flux interferometer.

FIG. 1 is a diagram showing a configuration example of the phase image acquisition device 1. FIG. 2 is a diagram showing a configuration example of the sample. FIG. 3 is a diagram showing three interference images out of the 100 interference images acquired in the embodiment. FIG. 4 is a graph showing the offset phase shift amount of each of the 100 interference images acquired in the embodiment. FIG. 5 is a phase image before the phase unwrapping process acquired in the embodiment. FIG. 6 is a phase image after the phase unwrapping process and background distortion correction acquired in the embodiment. FIG. 7 is a diagram showing an example in which the offset phase shift amount α of each of the M interference images is phasor-displayed on the unit circle. FIG. 8 is a diagram illustrating a method of selecting three interference images from among the M interference images in a diagram in which the offset phase shift amount α of each of the M interference images is phasor-displayed on a unit circle. .. FIG. 9 is a phase image (correct image) obtained when α ₂ = −120 ° and α ₃ = + 120 ° and no noise is applied. FIG. 10 shows interference images I ₁ to I ₃ when noise is applied with α ₂ = −120 ° and α ₃ = + 120 °. FIG. 11 is a phase image obtained based on the interference images I ₁ to I ₃ of FIG. FIG. 12 shows interference images I ₁ to I ₃ when noise is applied with α ₂ = −20 ° and α ₃ = + 30 °. FIG. 13 is a phase image obtained based on the interference images I ₁ to I ₃ of FIG. FIG. 14 is an image showing an error of the phase image of FIG. 13 with respect to the correct image of FIG. FIG. 15 is a diagram in which the offset phase shift amount α of each of the three interference images is phasor-displayed on the unit circle. FIG. 16 is a diagram showing the relationship between the offset phase shift quantities α ₂ and α ₃ and the standard deviation std in shades of light. FIG. 17 is a diagram showing the relationship between the offset phase shift quantities α ₂ and α ₃ and the standard deviation std by contour lines. FIG. 18 is a graph plotting the area area of the triangle and the standard deviation std obtained by setting the offset phase shift quantities α ₂ and α ₃ to various values. FIG. 19 is a graph showing the relationship between the minimum value of the standard deviation std and the area of the triangle. FIG. 20 is a table showing the relationship between the minimum value of the standard deviation std and the area of the triangle. FIG. 21 is a table summarizing the offset phase shift amounts, groupings, and complex coefficient sequences of the 12 interference images I ₁ to I ₁₂ prepared in the simulation. FIG. 22 is a phase image obtained by the first method. FIG. 23 is an image showing an error of the phase image of FIG. 22 with respect to the correct image of FIG. FIG. 24 is a phase image obtained by the second method. FIG. 25 is an image showing an error of the phase image of FIG. 24 with respect to the correct image of FIG. FIG. 26 is a phase image obtained by the third method. FIG. 27 is an image showing an error of the phase image of FIG. 26 with respect to the correct image of FIG. FIG. 28 is a diagram showing a case where the offset phase shift amount of each interference image increases substantially monotonously over time in a certain period. FIG. 29 is a phase image obtained by the fourth method. FIG. 30 is an image showing an error of the phase image of FIG. 29 with respect to the correct image of FIG. FIG. 31 is a table summarizing the offset phase shift amounts, groupings, and complex coefficient sequences of the 12 interference images I ₁ to I ₁₂ prepared in the simulations by the second to fourth methods. FIG. 32 is a graph showing a comparison of standard deviations of errors obtained in simulations by the first to fourth methods. FIG. 33 is a diagram in which the complex coefficient sequence according to the second method and the complex coefficient sequence according to the fourth method are plotted on the complex plane. FIG. 34 is a diagram in which the offset phase shift amount α of each of the 12 interference images is phasor-displayed on a unit circle, and four sets of three interference images out of the 12 interference images are displayed based on the second method. It is a figure explaining the selected method. FIG. 35 is a diagram in which the offset phase shift amount α of each of the 12 interference images is phasor-displayed on a unit circle, and four sets of three interference images out of the 12 interference images are displayed based on the third method. It is a figure explaining the selected method. FIG. 36 is a diagram showing a phase image used in the simulation. FIG. 37 is a diagram showing a phase image used in the simulation. FIG. 38 (a) is a diagram showing a phase distribution function when the phase value is 1 regardless of the position (x, y). FIG. 38 (b) is a diagram showing a phase division function cloth in the case of having a phase value proportional to the x-coordinate value. FIG. 38 (c) is a diagram showing a phase distribution function when the phase value is proportional to the y coordinate value. FIG. 38 (d) is a diagram showing a phase distribution function when the phase value is proportional to the value of x ² + y ^2-1 . 39 (a) to 39 (d) are diagrams showing an example of an offset phase shift amount distribution Δφ _n (x, y). 40 shows the coefficients p1 (n), p2 (n), p3 (n), p4 () set in each example of the offset phase shift amount distribution Δφ _n (x, y) in FIGS. 39 (a) to 39 (d). It is a table summarizing each value of n). FIG. 41 is a diagram showing three interference images out of the 36 interference images used in the simulation. FIG. 42 is a diagram showing a Fourier transform image obtained by performing a two-dimensional _Fourier transform on a certain interference image In (x, y). FIG. 43 is an enlarged view showing the vicinity of the origin of the Fourier transform image of FIG. 42. FIG. 44 is a diagram showing a two-dimensional phase distribution image obtained by performing a two-dimensional inverse Fourier transform using only the spatial frequency information of a certain region including the position of one of the spectral peaks in the Fourier transform image of FIG. 42. .. FIG. 45 is a diagram showing a central region of a phase image obtained by a conventional Fourier transform method. FIG. 46 is a diagram showing an example of a reference phase distribution image φLPF0 (x, y). FIG. 47 is a diagram showing an example of the phase distribution image ΔφLPF _n (x, y) relative to the reference phase distribution image φLPF 0 (x, y). FIG. 48 (a) is a diagram showing the relative phase distribution in the central region of the relative phase distribution image ΔφLPF _n (x, y) of FIG. 47. FIG. 48 (b) is a diagram showing a relative phase distribution obtained by approximating the relative phase distribution of FIG. 48 (a). FIG. 49 is a diagram showing a phase image generated in the simulation. FIG. 50 is a diagram showing a phase image generated in the simulation. FIG. 51 is a diagram showing an image of the difference between the phase image generated in the simulation and the correct answer image (FIG. 36).

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are designated by the same reference numerals, and duplicate description will be omitted. The present invention is not limited to these examples, and is indicated by the scope of claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

FIG. 1 is a diagram showing a configuration example of the phase image acquisition device 1. The optical system of the phase image acquisition device may have various configurations, and the configuration of the optical system shown in this figure is only an example. The phase image acquisition device 1 shown in this figure includes an interference image acquisition unit 2 and a calculation unit 3. The interference image acquisition unit 2 includes a light source 10, a lens 11, a lens 12, a branching beam splitter 21, a wave splitter 22, a beam splitter 31, a first reflection mirror 32, a beam splitter 41, a second reflection mirror 42, and a piezo. It includes an element 43, a stage 44, a lens 51, a lens 52, a tube lens 53, an imager 61, and a control unit 72. The two-luminous flux interferometer 20 from the branching beam splitter 21 to the combining beam splitter 22 constitutes a Mach-Zehnder interferometer.

The phase image acquisition device 1 acquires an interference image based on the transmitted light of the object placed in the container 90. The object is not limited to a specific cell or biological sample. For example, as objects, cultured cells, immortalized cells, primary cultured cells, cancer cells, fat cells, liver cells, myocardial cells, nerve cells, glia cells, somatic stem cells, embryonic stem cells, pluripotent stem cells, iPS Examples include cells and cell clusters (spheroids) formed from at least one of these cells. Further, the object is not limited to a living body, and examples thereof include industrial samples that can be measured in a transmissive configuration, such as the inside of glass, the inside of a semiconductor element, a resin material, a liquid crystal crystal, a polymer compound, and an optical element.

The light source 10 outputs light. In order to reduce speckle noise, it is preferable that the light source 10 outputs incoherent light. The light source 10 is, for example, a lamp-based light source such as a halogen lamp, an LED (Light emitting emission) light source, an SLD (Superluminescent diode) light source, an ASE (Amplified spontaneous emission) light source, or the like. The lenses 11 and 12 collect the light output from the light source 10 on the object in the container 90.

The branching beam splitter 21 is optically coupled to the light source 10, inputs the light output from the light source 10 and passed through the lenses 11 and 12, and splits the light into two to obtain the first luminous flux and the second luminous flux. The branching beam splitter 21 may be, for example, a half mirror. The branching beam splitter 21 outputs the first luminous flux to the beam splitter 31 of the measurement side optical system, and outputs the second luminous flux to the beam splitter 41 of the reference side optical system.

The measurement side optical system is provided with a beam splitter 31 and a first reflection mirror 32. The beam splitter 31 inputs the first light flux output from the branching beam splitter 21 and reflects it to the first reflection mirror 32, and inputs the first light beam reflected by the first reflection mirror 32 to combine waves. It is transmitted to the beam splitter 22. The beam splitter 31 may be, for example, a half mirror.

The reference side optical system is provided with a beam splitter 41, a second reflection mirror 42, a piezo element 43, and a stage 44. The beam splitter 41 inputs the second light beam output from the branching beam splitter 21 and reflects it to the second reflection mirror 42, and inputs the second light beam reflected by the second reflection mirror 42 to combine waves. It is transmitted to the beam splitter 22. The beam splitter 41 may be, for example, a half mirror.

The piezo element 43 can move the second reflection mirror 42 in a direction perpendicular to the reflection surface of the second reflection mirror 42. The stage 44 can move the second reflection mirror 42 and the piezo element 43 in a direction perpendicular to the reflection surface of the second reflection mirror 42. The piezo element 43 and the stage 44 can adjust the optical path length of the reference side optical system, and can adjust the optical path length difference between the first luminous flux and the second luminous flux. The stage 44 can roughly adjust the optical path length difference, and the piezo element 43 can finely adjust the optical path length difference.

The beam splitter 22 for the combined wave inputs the first light flux output from the beam splitter 31 and passed through the object in the container 90 and the lens 51, and inputs the second light flux output from the beam splitter 41 and passed through the lens 52. , The first luminous flux and the second luminous flux are combined to output combined light (interference light). The beam splitter 22 for the combined wave may be, for example, a half mirror.

The tube lens 53 guides the interference light output from the beam splitter 22 for combining waves to the imager 61, and forms an image of the interference light on the image pickup surface of the imager 61. The imager 61 receives the interference light and outputs a detection signal, and particularly outputs a detection signal representing the intensity distribution of the interference light on the image pickup surface. The imager 61 is an image sensor such as a CCD area image sensor or a CMOS area image sensor.

The calculation unit 3 inputs the detection signal output from the imager 61, acquires an interference image of the object in the container 90 based on the detection signal, and further acquires a phase image. The arithmetic unit 3 is configured to include an image processing processor such as an FPGA (field-programmable gate array) or GPU (Graphics Processing Unit), or may be a computer such as a personal computer or a tablet terminal. Further, the calculation unit 3 may include a display unit for displaying an interference image or the like.

The control unit (controller) 72 moves the second reflection mirror 42 by driving both or one of the piezo element 43 and the stage 44, and adjusts the optical path length of the reference side optical system. As a result, the control unit 72 can adjust the phase difference between the first luminous flux and the second luminous flux at the time of the wave combination by the wave splitting beam splitter 22.

The arithmetic unit 3 and the control unit 72 are computers including a processor, a memory, and the like. Further, the arithmetic unit 3 and the control unit 72 may be individual computers or one computer. The computer may be, for example, a smart device such as a personal computer or a tablet terminal. Further, the calculation unit 3 or the control unit 72 has an input unit (keyboard, mouse, tablet terminal, etc.) that receives input from the user, and a display unit (display, tablet terminal, speaker, vibrator) that displays the interference strength and the like. You may be prepared. If the display unit is a device capable of displaying the screen, such as a display or a tablet terminal, an interference image or the like may be displayed together with the interference intensity.

The interference image of the object in the container 90 can be acquired by using the phase image acquisition device 1 as follows. The light output from the light source 10 passes through the lenses 11 and 12 and is split into two by the branching beam splitter 21 to be the first luminous flux and the second luminous flux. The first light flux output from the branching beam splitter 21 passes through the beam splitter 31 and is reflected by the first reflection mirror 32. The first light flux reflected by the first reflection mirror 32 is reflected by the beam splitter 31, is focused on the object in the container 90, and passes through the object. The first light flux transmitted through the object is input to the beam splitter 22 for combining waves via the lens 51. This first luminous flux has an optical delay in transmission in the object. The second light flux output from the branching beam splitter 21 is reflected by the beam splitter 41 and reflected by the second reflection mirror 42. The second luminous flux reflected by the second reflection mirror 42 passes through the beam splitter 41, passes through the lens 52, and is input to the beam splitter 22 for combining waves.

The first luminous flux input from the lens 51 to the combine beam splitter 22 and the second light flux input from the lens 52 to the combine beam splitter 22 are combined by the combine beam splitter 22. The interference light due to the combined wave is received by the imager 61 via the tube lens 53. The calculation unit 3 acquires an interference image based on the detection signal output from the imager 61 that has received the interference light, and further acquires a phase image based on the plurality of interference images. Further, by controlling the position of the second reflection mirror 42 by the piezo element 43 or the stage 44 driven by the control unit 72, the optical path length difference between the first luminous flux and the second luminous flux is adjusted, and the wave combination occurs. The phase difference between the first luminous flux and the second luminous flux at the time of the combined wave by the beam splitter 22 is adjusted.

In the prior art, when an interference image is acquired, the optical path length difference between the two light beams in the two light beam interferometer 20 is stabilized at each of a plurality of different set values, and the interference image of the object is captured in each state. It is acquired by the device 61, and a phase image is obtained based on these acquired plurality of interference images. Further, in order to stabilize the optical path length difference between the two luminous fluxes in the two luminous flux interferometer 20, this optical path length difference is detected, and one of the two luminous fluxes is set so that the optical path length difference becomes a set value. The position of the mirror placed on the optical path of is feedback-controlled. However, even with such feedback control, for example, when cells as an object are placed in a culture solution and the liquid level of the culture solution is fluctuating, the optical path length fluctuates due to the fluctuation of the liquid level. Since the amount is larger than the fluctuation amount of the optical path length that can be stabilized by the feedback control, it is difficult to stabilize the optical path length difference between the two light fluxes in the two light flux interferometer 20. In order to deal with this problem, in the prior art, in order to suppress the fluctuation of the liquid level of the culture solution, a lid or a cover glass is covered so as to be in contact with the culture solution, or a submerged submerged objective lens is used. Although it was devised to prevent light from passing through the interface between the culture solution and the gas (for example, air), in that case, there was a problem that the cells were contaminated.

On the other hand, in the embodiment described below, for example, when a cell as an object is placed in a culture solution and the liquid level of the culture solution is fluctuating, the two light flux interferometers are used for dimming. Even when the optical path length difference between the bundles fluctuates with time, it is possible to acquire a phase image of the object. The temporal fluctuation of the optical path length difference may be due to the fluctuation of the liquid level of the liquid arranged on the optical path of the first luminous flux or the second luminous flux separately from the culture solution, or the piezo element. It may be due to the vibration of the second reflection mirror 42 driven by the 43, or it may be due to the vibration applied to the device from the outside. The temporal variation of the optical path length difference may be aperiodic and random, periodic, and may be unidirectionally increasing or decreasing. .. Vibration may be applied to the device from the outside in order to obtain a sufficiently large phase difference, or the interference image may be continuously acquired until a sufficiently large phase difference is obtained.

FIG. 2 is a diagram showing a configuration example of the sample. The object is cells 93 in the culture medium 92 placed in the container 90. The upper part of the container 90 is covered with a lid 91. There is a space between the culture solution 92 and the lid 91, and the liquid surface of the culture solution 92 is in contact with gas (generally air). Therefore, due to the fluctuation of the liquid level of the culture solution 92, the optical path length of the light transmitted through the culture solution 92 fluctuates with time, and the optical path length difference between the two light fluxes in the two-luminous flux interferometer 20 fluctuates with time.

In the present embodiment, the interference image acquisition unit 2 acquires M interference images of the object in a state where the optical path length difference between the two light fluxes fluctuates with time in the two-luminous flux interferometer 20 (interference image acquisition). Step). The acquisition timings of the M interference images are different from each other. Then, the calculation unit 3 estimates the offset phase shift amount of the entire image for each of these M interference images (offset phase shift amount estimation step), and the offset phase shift amount is different from each other among the M interference images. A set including the interference images is set (set setting step), and a phase image is generated based on the N interference images included in the set (phase image generation step).

Preferably, the calculation unit 3 sets a plurality of sets as described above (set setting step), and generates a phase image based on the N interference images included in each set for each of the plurality of sets (set setting step). (Phase image generation step), the phase images generated by each of these plurality of sets are integrated after the offset phase shift amount is corrected (integration step). In the integration step, the average value may be obtained by dividing the integrated value by the number of pairs, and there is no substantial difference between the integrated value and the average value. Of the M interference images, there may be interference images that are not included in any of the sets. Further, among the M interference images, there may be interference images included in two or more sets.

N is an integer of 3 or more. If only one set containing N interference images is set, M is an integer greater than or equal to N. If a plurality of sets including N interference images are set, M is an integer of (N + 1) or more. In order to obtain a good phase image for SN, it is preferable that the number of pairs is large, and it is preferable that M is large.

For each interference image, the estimation of the offset phase shift amount of the entire image (offset phase shift amount estimation step) is performed, for example, as follows. Focusing on the region where the interference fringes are linear and appear at equal intervals in the interference image, the interference intensity distribution in this region is spatially complex Fourier transformed. The phase value of the complex spatial frequency that becomes the peak in the spatial frequency distribution obtained by this complex Fourier transform is obtained, and this phase value is estimated as the offset phase shift amount of the interference image. The region to be the target of the complex Fourier transform may be a two-dimensionally cut out interference image or a one-dimensionally extracted part of the line profile of the interference image.

The generation of the phase image (phase image generation step) based on the N interference images included in each set is performed as follows. Hereinafter, the case where N = 3 will be described. The three interference images I ₁ to I ₃ included in a certain set are represented by the following equation (1). I ₁ to I ₃ , a, b, and φ are functions of the position (x, y). φ is a phase value to be obtained as a pixel value of the phase image. α ₁ to α ₃ are offset phase shift amounts due to the difference in optical path length in the two-luminous flux interferometer at the acquisition timing of each interference image. From this equation (1), the following equation (2) can be obtained (see Non-Patent Document 2). The right side of the equation (2) is represented by the offset phase shift amounts α ₁ to α ₃ and the phase value φ, and does not depend on a and b. Since the offset phase shift amounts α ₁ to α ₃ of each interference image have already been estimated, the phase value φ can be obtained from this equation (2). That is, a phase image can be obtained.

In the integration (integration step) of the phase images generated by each of the plurality of sets, after correcting the offset phase shift amount of each phase image so that the pixel value is not attenuated by the integration, each phase image has a complex amplitude. Accumulate as an image. By integrating in this way, it is possible to reduce the error of each pixel value of the phase image.

Next, an embodiment will be described. As a container, a glass bottom dish having a diameter of 35 mm was used. Immersion oil was adhered to the inside of the bottom of the container as an object, water was poured into the immersion oil to a depth of about 7 mm, and a lid was put on the container. A laser diode having a center wavelength of 650 nm was used as a light source. By modulating the injection current into the laser diode at a frequency of 100 kHz, the line width of the light output from the laser diode was widened. By expanding the line width in this way, the output light is made incoherent, and speckle noise and instability of the laser light are reduced. The exposure time for acquiring each interference image was 3 milliseconds, and the imaging frame rate was 30 fps. 100 interference images were acquired.

FIG. 3 is a diagram showing three interference images out of the 100 interference images acquired in the embodiment. In the two-luminous flux interferometer, the difference in optical path length between the two light fluxes fluctuates with time, so that the offset phase shift amount also fluctuates with time. Therefore, these three interference images are different from each other.

FIG. 4 is a graph showing the offset phase shift amount of each of the 100 interference images acquired in the embodiment. In this figure, the phase unwrapping process in the time direction is not performed. The horizontal axis is the number of the interference image. The vertical axis is the offset phase shift amount (radians) estimated for each interference image in the offset phase shift amount estimation step. In this way, it can be seen that the offset phase shift amount fluctuates with time.

In the set setting step, 20 sets were set by setting the number N of the interference images included in each set to 3. The kth set out of the 20 sets has a difference in the amount of offset phase shift between the (5k-4) th interference image out of 100 interference images and the (5k-4) th interference image. It was set to include an interference image closest to + 120 ° and an interference image having an offset phase shift amount difference closest to −120 ° with respect to this (5k-4) interference image. k is an integer from 1 to 20.

In the phase image generation step, a phase image was generated for each of the 20 sets based on the three interference images included in each set. Then, in the integration step, the phase images generated by each of the 20 sets were integrated after correcting the offset phase shift amount to obtain a final phase image.

5 and 6 are diagrams showing the phase images obtained in the examples. FIG. 5 is a phase image before the phase unwrapping process. FIG. 6 is a phase image after phase unwrapping processing and background distortion correction. This phase image well represents the shape of the immersion oil as an object attached to the inside of the bottom of the container. The maximum value of the thickness of the object corresponds to a phase difference of 40 radians, but even with such a large phase value, there is no problem in the phase unwrapping process.

In this way, for example, when cells as an object are placed in a culture solution and the liquid level of the culture solution is fluctuating, the optical path length difference between the two light fluxes in the two-luminous flux interferometer is temporally different. It is possible to acquire a phase image of an object even in a fluctuating state. Since it is not necessary to provide a cover glass or the like in contact with the culture solution in order to suppress fluctuations in the liquid level of the culture solution as in the conventional case, the problem of cell contamination by the cover glass or the like does not occur. Further, since the phase image can be acquired even if there is no mechanical moving part, the problem of deterioration of mechanical parts can be suppressed, and a simple configuration can be obtained.

Next, the setting of the set including the N interference images (set setting step) will be described in detail. When setting a set including N interference images, it is preferable to select N interference images so that the phase value φ can be calculated accurately based on the above equation (2). For that purpose, N interference images can be selected as follows. First, the offset phase shift amount α of each of the M interference images is phasor-displayed on the circumference of a unit circle having a radius of 1 centered on the origin in a Cartesian coordinate system having cosα on the horizontal axis and sinα on the vertical axis. FIG. 7 is a diagram showing an example in which the offset phase shift amount α of each of the M interference images is phasor-displayed on the unit circle.

In this figure, the black circle on the unit circle represents the position of the offset phase shift amount α displayed by the phasor. From the M offset phase shift amount α displayed on the unit circle as a phaser, the area of the N-square formed by connecting the points of N offset phase shift amount α in order with a line segment becomes as large as possible. It is preferable to select N offset phase shift amounts α and select N interference images corresponding to these to form a set. ..

FIG. 8 is a diagram illustrating a method of selecting three interference images from among the M interference images in a diagram in which the offset phase shift amount α of each of the M interference images is phasor-displayed on a unit circle. .. When N = 3, the area of the triangle formed by connecting the points of the three offset phase shift amounts α in order from the M offset phase shift amount α displayed on the unit circle by a line segment. Select three offset phase shift amounts α so that do. In this figure, the points of the offset phase shift amount α of the three interference images included in each set are connected by a solid line, a broken line, or an alternate long and short dash line. Three sets including three interference images are set.

When N = 3, the area of the triangle formed by connecting the three points of the offset phase shift amount α displayed on the unit circle with a line segment in order, and the three interference images corresponding to these points. A simulation was performed on the relationship between the error of each pixel value in the phase image generated based on. The _nth interference image In of the three interference images was expressed by the following equation. a = 1.5 and b = 1.0. _Noise was added to each pixel value of the interference image In. The added noise was a uniform random number in the range of −n _f / 2 to + n _f / 2 with _{n f} = 0.15 sqrt (In). That is, the shot noise proportional to the square root of the number of photons was simulated. Then, the offset phase shift amount α ₁ of the first interference image I ₁ of the three interference images is set to 0, and the offset phase shift amount α ₂ of the second interference image I ₂ and the third interference image I ₃ are set. The offset phase shift amount α ₃ of was set to various values.

FIG. 9 is a phase image (correct image) obtained when α ₂ = −120 ° and α ₃ = + 120 ° and no noise is applied. FIG. 10 shows interference images I ₁ to I ₃ when noise is applied with α ₂ = −120 ° and α ₃ = + 120 °. FIG. 11 is a phase image obtained based on the interference images I ₁ to I ₃ of FIG. FIG. 12 shows interference images I ₁ to I ₃ when noise is applied with α ₂ = −20 ° and α ₃ = + 30 °. FIG. 13 is a phase image obtained based on the interference images I ₁ to I ₃ of FIG. FIG. 14 is an image showing an error of the phase image of FIG. 13 with respect to the correct image of FIG. From these figures, in order to reduce the error of the phase image when the interference image contains noise, the area of the triangle formed by connecting the three points of the offset phase shift amount α with a line segment in order. It can be seen that it is preferable that the value is large.

FIG. 15 is a diagram in which the offset phase shift amount α of each of the three interference images is phasor-displayed on the unit circle. The offset phase shift amount α ₁ of the first interference image I ₁ of the three interference images is set to 0, the offset phase shift amount of the second interference image I ₂ is set to + α ₂ , and the third interference image I ₃ is set. The offset phase shift amount of is set to _−α3 . For each value of α ₂ and each value of α ₃ , it is possible to obtain the area area of the triangle formed by connecting the points of the three offset phase shift amounts in order with a line segment, and it corresponds to these. The standard deviation std of the error of each pixel value in the phase image generated based on the three interference images can be obtained.

FIG. 16 is a diagram showing the relationship between the offset phase shift quantities α ₂ and α ₃ and the standard deviation std in shades of light. FIG. 17 is a diagram showing the relationship between the offset phase shift quantities α ₂ and α ₃ and the standard deviation std by contour lines. FIG. 18 is a graph plotting the area area of the triangle and the standard deviation std obtained by setting the offset phase shift quantities α ₂ and α ₃ to various values. FIG. 19 is a graph showing the relationship between the minimum value of the standard deviation std and the area of the triangle. FIG. 20 is a table showing the relationship between the minimum value of the standard deviation std and the area of the triangle. As can be seen from these figures, the error is the smallest when α ₂ = α ₃ = 120 °, that is, when the triangle connecting the three points is an equilateral triangle. The larger the area of the triangle, the smaller the error.

Next, an example of setting a set including N interference images (set setting step) will be described. First, it is assumed that M interference images Im ( _m = 1 to M) have been acquired, and the offset phase shift amount α _m of each interference image _Im is known. As described above, any three interference images can be selected from the _M interference images I ₁ to IM to generate a phase image. Here, by modifying the above equation (2), it is possible to formulate a complex amplitude image Comp (x, y) having an argument corresponding to the generated phase image. The process of formula transformation is described below.

Complex numbers A and B are expressed by the following equation (4) for the coefficients appearing on the right side of equation (2). By multiplying the denominator and numerator on the right side of equation (2) by cosφ, the following equation (5) can be obtained. The unknowns are cosφ and sinφ.

Using the matrix M of the following equation (6), the equation (5) is expressed by the following equation (7). If the matrix M is a regular matrix, the equation (7) is expressed by the following equation (8).

The complex amplitude image Comp (x, y) is expressed by the following equation (9) in the form of the following equation (10). That is, the complex amplitude image Comp is represented by the inner product of the interference image and the complex number sequence.

The calculation of generating a complex amplitude image from three interference images in each of a plurality of sets, integrating these multiple complex amplitude images, and then taking the argument is to generate a complex amplitude image by the following equation (11). Then, it is equivalent to obtaining the phase image by taking the argument of this complex amplitude image.

Therefore, to generate a phase image from three interference images in each of a plurality of sets, the complex coefficient _{coef_m1} , when the three interference images selected in each set are Im1, _Im2 , and I _m3 , It corresponds to the operation of sequentially determining coef_m2 and coef_m3. From these, it can be generalized to the operation of generating a phase image from a plurality of interference images in the present embodiment and determining the complex coefficient vector (coef_1 ... coef_N) of the above equation (11).

In the following, four methods are given as methods for determining this coefficient, and the simulation results of each are shown. In the first method, three interfering images are selected completely randomly. In the second method, three interference images are appropriately selected. In the third method, the method of selecting the three interference images is inappropriate. In the fourth method, iterative optimization is performed for each element of the coefficient vector when all of the 12 interference images are used.

In this simulation, 12 interference images I ₁ to I ₁₂ were prepared. Each interference image _Im is represented by the above equation (3), and a = 1.5 and b = 1.0. The offset phase shift amount α _m of each interference image _Im was set as a random number sequence uniformly distributed in the range of −π to + π. Noise was added to each pixel value of the interference image _Im . The added noise was a uniform random number in the range of −n _f / 2 to + n _f / 2 with n _f = 0.15 sqrt (I _m ). That is, the shot noise proportional to the square root of the number of photons was simulated. Further, the phase image (correct image) when there is no noise is the same as in FIG.

The contents and results of the simulation by the first method are as follows. A phase image can be generated by arbitrarily selecting 3 interference images from 12 interference images I ₁ to I ₁₂ each given a random offset phase shift amount α _m , but the simplest. As a method, the interference images I ₁ to I ₃ are set as the first set, the interference images I ₄ to I ₆ are set as the second set, the interference images I ₇ to I ₉ are set as the third set, and the interference images I ₁₀ are used. ~ I ₁₂ was set as the fourth set. As a result, a complex coefficient sequence as shown in FIG. 21 was obtained. FIG. 21 is a table summarizing the offset phase shift amounts, groupings, and complex coefficient sequences of the 12 interference images I ₁ to I ₁₂ prepared in the simulation. Based on these complex coefficient sequences, a complex amplitude image was generated by the above equation (11), and a phase image was further generated. FIG. 22 is a phase image obtained by the first method. FIG. 23 is an image showing an error of the phase image of FIG. 22 with respect to the correct image of FIG. The unit in these images is radians. The standard deviation of the error was 33.4 milliradians.

The contents and results of the simulation by the second method are as follows. The 12 interference images I ₁ to I ₁₂ were rearranged in ascending or descending order with respect to the offset phase shift amount, and the 12 interference images after the rearrangement were designated as J ₁ to J ₁₂ in order. Then, the 12 interference images J ₁ to J ₁₂ were sequentially distributed into four sets. That is, the interference images J ₁ , J ₅ , and J ₉ are the first set, the interference images J ₂ , J ₅ , and J ₁₀ are the second set, and the interference images J ₃ , J ₆ , and J ₁₁ are the third set. The set was set, and the interference images J ₄ , J ₇ , and J ₁₂ were set as the fourth set (see FIG. 34). FIG. 24 is a phase image obtained by the second method. FIG. 25 is an image showing an error of the phase image of FIG. 24 with respect to the correct image of FIG. The standard deviation of the error was 23.6 milliradians.

The second method will be described in a more general manner. For _M interference images J1 to _JM , when the maximum integer that does not exceed M / ₃ is MG, these interference images can be distributed to _MG sets. Before sorting, the interference images J1 to _JM are sorted in ascending or descending order with respect to the offset _phase shift amount. Then, the interference images J ₁ , J _{1 + MG} , J _{1 + 2 × MG} are set as the first set, and the interference images J ₂ , J _{2 + MG} , J _{2 + 2 × MG} are set as the second set, and the interference images J ₃ , J _{3 + MG} and J _{3 + 2 × MG are} set as the third set, and the following can be distributed to the third set. That is, the kth set is composed of the interference images J _k , J _{k + MG} , and J _{k + 2 × MG} . By sorting in this way, the relative phase differences of the three interference images included in each set can be grouped so as to be approximately 120 ° each.

The contents and results of the simulation by the third method are as follows. The 12 interference images I ₁ to I ₁₂ were rearranged in ascending or descending order with respect to the offset phase shift amount, and the 12 interference images after the rearrangement were designated as J ₁ to J ₁₂ in order. Then, the interference images J ₁ to J ₃ are set as the first set, the interference images J ₄ to J ₆ are set as the second set, the interference images J ₇ to J ₉ are set as the third set, and the interference images J ₁₀ to J are set. ₁₂ was used as the fourth set (see FIG. 35). FIG. 26 is a phase image obtained by the third method. FIG. 27 is an image showing an error of the phase image of FIG. 26 with respect to the correct image of FIG. The standard deviation of the error was 235 milliradians. It should be noted that such grouping of the interference images is continuous when the offset phase shift amount of each interference image increases (or decreases) monotonously over time as shown in FIG. 28. It corresponds to including the three interference images obtained in each set in each set.

The contents and results of the simulation by the fourth method are as follows. In the fourth method, the 12 interference images are not grouped, and the number N of the interference images included in each group is N = M = 12, and the image reconstruction is performed. As shown by the first to third methods, various values can be taken as the complex coefficient sequence for reconstructing the phase distribution from the M interference images, and the original interference image has noise. If there is no, there are innumerable complex coefficient sequences that give a phase distribution, but by devising how to take this complex coefficient sequence, it is possible to derive a complex coefficient sequence that is more robust to noise.

First, for the complex coefficient sequence (coef_1 ... coef_M) of _M interference images I ₁ to IM, the coefficient sequence as an initial condition is set. For example, the complex coefficient sequence obtained by the second method can be used as the initial condition. After that, consider changing some elements of the complex coefficient sequence on the complex plane by a small amount. For example, for the L-th coefficient coef_L, the real part is increased by 0.1 or the imaginary part is decreased by 0.1. At this time, the newly created complex coefficient sequence (coef_1… coef_M) cannot reconstruct the image of the phase image, but for another one or more coefficients that are not the Lth, both the real part and the imaginary part, or either of them. By increasing or decreasing one of them by a minute amount, the complex coefficient sequence can be slightly corrected to create a second coefficient sequence. In this calculation, an interference image in which the offset phase amount is α ₁ to α _M and the phase is evenly distributed over −π to + π is created on the computer under noise-free conditions, and the phase image is imaged for them. It may be realized by adjusting the complex coefficient sequence so that it can be reconstructed.

Next, the robustness of the newly obtained complex coefficient sequence with respect to noise is evaluated. To evaluate the robustness to noise, for example, on a computer, an interference image having an offset phase amount of α ₁ to α _M , including noise, and the phase being evenly distributed over −π to + π is created, and an interference image is created for them. It may be realized by applying the newly obtained complex coefficient sequence to obtain the phase image and obtaining the squared average error from the phase image which is the ideal correct answer. Here, the squared average error in the phase image using the coefficient sequence as the initial condition is compared with the squared average error in the phase image using the coefficient sequence newly obtained by slightly changing from the initial condition. However, if the newly obtained complex coefficient sequence gives better reconstruction results for noise, the newly obtained complex coefficient sequence is adopted as the provisional best complex coefficient sequence.

On the contrary, if the newly obtained complex coefficient sequence gives a poor reconstruction result for noise, the complex coefficient sequence in the previous stage is adopted as the provisional best complex coefficient sequence. There are many trials as to which element in the complex coefficient sequence is to be changed minutely, whether it is the real part or the imaginary part, and whether the direction of the minute change is positive or negative. There can be. Therefore, we should try many evaluations of the robustness against minute changes and noise of the complex coefficient sequence, and gradually adopt the complex coefficient sequence that gives the best result as the provisional best complex coefficient sequence. Can be done.

For example, in the case of a complex coefficient sequence having M elements, when the real part or the imaginary part of the Lth coefficient is changed by a small amount in the positive or negative direction, there are 4 × M ways to take it. Further, when a certain L-th coefficient is changed, for another coefficient that is not the L-th, the complex coefficient sequence is formed by increasing or decreasing both the real part and / or the imaginary part by a minute amount. A new coefficient sequence can be created by making minor corrections. At this time, there is (M-1) how to select which coefficient compensates for the minute change of the Lth coefficient. Therefore, when changing the real part or the imaginary part of a certain coefficient by a small amount in the positive or negative direction and changing another coefficient by a small amount in order to compensate for the change, there are 4 × M × (M-1) ways. There is a combination of. For all of these combinations, the robustness to noise may be evaluated, and the complex coefficient sequence that gives the best result in the combination may be gradually adopted as the provisional best complex coefficient sequence.

After the provisional best complex coefficient sequence can be obtained, the complex coefficient more robust to noise can be similarly obtained starting from the provisional best complex coefficient sequence. By iteratively repeating this trial on a computer, the complex coefficient sequence expected to give the best results can be obtained.

It should be noted that the information required for obtaining this complex coefficient sequence is the numerical value of the offset phase amount α ₁ to α _M , not the actual interference image itself. Therefore, there is no problem even if an interference image containing noise simulated on a computer is used in the evaluation of the robustness to noise, and the actually acquired interference image is used in the final image reconstruction.

For the complex coefficient sequences (coef_1 ... coef_12) of the 12 interference images I ₁ to I ₁₂ , the complex coefficient sequence obtained by the second method is used as the initial value, and the complex coefficient sequence more robust to noise is iteratively used. The complex number of each element was determined by the optimization calculation. FIG. 29 is a phase image obtained by the fourth method. FIG. 30 is an image showing an error of the phase image of FIG. 29 with respect to the correct image of FIG. The standard deviation of the error was 22.0 milliradians.

FIG. 31 is a table summarizing the offset phase shift amounts, groupings, and complex coefficient sequences of the 12 interference images I ₁ to I ₁₂ prepared in the simulations by the second to fourth methods. From the above simulation results, the following can be understood.

Under ideal conditions where no noise is present in the interference image, a phase image can be generated based on the above equation (11) by any of the first to fourth methods. However, noise exists in the actual interference image, and among them, shot noise of light is physically unavoidable. As a result of the simulation using the interference image containing noise, the accuracy of the generated phase image was the best in the fourth method, followed by the second method and the first method. In the third method, the accuracy of the phase image was very poor. FIG. 32 is a graph showing a comparison of standard deviations of errors obtained in simulations by the first to fourth methods.

As shown in FIG. 28, when the rate of change of the offset phase shift amount is slower than the image pickup frame rate of the imager, the interference images arranged in the acquisition order are as in J ₁ to J ₁₂ of the third method. It will be lined up in order. In such a case, creating a set containing three interference images in the order of simply obtained images to generate a phase image will give unfavorable results. That is, in the method of obtaining the complex coefficient sequence from the three interference images, it is preferable to appropriately rearrange the obtained interference images.

The complex coefficient sequence of the above equation (11) can also be obtained by optimization using iterative calculation on a computer. In this case, it is possible to generate a phase image having less noise than the best result when the complex coefficient sequence is obtained from the three interference images. FIG. 33 is a diagram in which the complex coefficient sequence according to the second method and the complex coefficient sequence according to the fourth method are plotted on the complex plane.

In the embodiments described so far, the offset phase shift amount of the entire image is estimated for each of the M interference images (offset phase shift amount estimation step), and the offset phase shift amounts are different from each other among the M interference images. A set including N interference images is set (set setting step), and a phase image is generated based on the N interference images included in the set (phase image generation step). Further, preferably, a plurality of sets as described above are set (set setting step), and a phase image is generated based on N interference images included in each set for each of the plurality of sets (phase image generation). Step), the phase images generated by each of these plurality of sets are integrated after correcting the offset phase shift amount (integration step).

Each interference image to be processed may be the whole image acquired by the phase image acquisition device, or may be a partial interference image which is a part of the images acquired by the phase image acquisition device. .. The interference image acquired by the phase image acquisition device may be divided into several partial interference images, and each of the above steps may be performed on each partial interference image to generate a phase image. At this time, the grouping may be different depending on the partial interference image.

The embodiments described so far are suitable when it can be considered that the offset phase shift amount changes with time in the interference image (or partial interference image) to be processed but is constant regardless of the pixel position. It is a thing. For example, if the range of the field of view to be imaged by the phase image acquisition device is sufficiently smaller than the area of the container, it can be considered that the offset phase amount shift amount is constant regardless of the pixel position in the acquired interference image. can.

On the other hand, in the interference image (or partial interference image) to be processed, the offset phase shift amount cannot be regarded as constant regardless of the pixel position, and the non-uniformity of the distribution of the offset phase shift amount becomes uneven. It may not be ignored. For example, when the range of the field of view to be imaged by the phase image acquisition device is about the same as the area of the container, the non-uniformity of the distribution of the offset phase amount shift amount cannot be ignored in the acquired interference image due to the fluctuation of the water surface. be. In such a case, it is preferable to generate a phase image from the interference image as in the embodiment described below.

In the following, a method of dividing the interference image into pixel units and performing each step to generate a phase image will be described together with simulation results. 36 and 37 are diagrams showing the phase images used in the simulation. FIG. 36 shows a phase image of a phase object. FIG. 37 shows a phase image when a tilt is present at the time of acquiring an interference image. Let φ _true (x, y) be the phase image shown in FIG. 37. In this simulation, it is assumed that 36 interference images I ₁ to I ₃₆ are acquired by the phase image acquisition device for the phase object represented by such a phase image. Further, it is assumed that the optical path length difference between the two luminous fluxes in the two luminous flux interferometer at the time of acquiring the interference image fluctuates with time due to, for example, fluctuation of the water surface.

It is assumed that the _nth interference image In (x, y) of the 36 interference images I ₁ to I ₃₆ is represented by the following equation (12). The interference image In (x, y) of the equation (12) _{is the interference obtained when the offset phase shift amount distribution α n (} x, y) is added to the true phase image φ _true (x, y). Random noise is added to the image. A uniform random number in the range of 0 to 0.05 was used as the random noise. The offset phase shift amount distribution α _n (x, y) is expressed by the following equation (13) by simulating the fluctuation of the liquid surface. Random numbers were used as the coefficients p1 (n), p2 (n), p3 (n) and p4 (n) in Eq. (13).

The first term on the right side of equation (13) results in translation of the interference fringes, the second and third terms result in changes in the slope of the interference fringes, and the fourth term results in two-dimensional distortion of the interference fringes. .. FIG. 38 (a) is a diagram showing a phase distribution function when the phase value is 1 regardless of the position (x, y). FIG. 38 (b) is a diagram showing a phase distribution function when the phase value is proportional to the x-coordinate value. FIG. 38 (c) is a diagram showing a phase distribution function when the phase value is proportional to the y coordinate value. FIG. 38 (d) is a diagram showing a phase distribution function when the phase value is proportional to the value of x ² + y ^2-1 . The range of x and y is standardized in the range of -1 to +1.

Equation (13) above corresponds to the sum of these phase distribution functions multiplied by the coefficients p1 (n), p2 (n), p3 (n), and p4 (n). 39 (a) to 39 (d) are diagrams showing an example of an offset phase shift amount distribution α _n (x, y). 40 shows the coefficients p1 (n), p2 (n), p3 (n), p4 () set in each example of the offset phase shift amount distribution α _n (x, y) in FIGS. 39 (a) to 39 (d). It is a table summarizing each value of n).

The offset phase shift amount distribution α _n (x, y) includes a constant term, a term proportional to x or y, and a term proportional to the square of x or y in the above equation (13). Further, a higher-order term, an exponential function term, and a triangular function (sin, cos) term may be included. In general, the offset phase shift quantity distribution α _n (x, y) can be represented by the linear sum of a finite number of spatial distribution functions.

FIG. 41 is a diagram showing three interference images out of the 36 interference images used in the simulation. These interference images are created based on the above equations (12) and (13) using the _true phase image φtrue (x, y).

The calculation unit 3 can generate a phase image from an interference image by performing the following offset phase shift amount estimation step, set setting step, phase value calculation step, and phase image generation step.

In the offset phase shift amount estimation step, the distribution of the offset phase shift amount is estimated for each interference image. In the offset phase shift amount estimation step, for example, it is preferable to perform phase distribution calculation processing by the spatial fringe analysis method by Fourier transform, difference processing from the reference phase distribution, and smoothing processing in order as follows. ..

As described in Non-Patent Document 3, the spatial fringe analysis method by Fourier transform (hereinafter referred to as "Fourier transform method") creates a Fourier transform image by two-dimensional Fourier transforming an interference image, and this Fourier transform. A two-dimensional phase distribution image is obtained by performing a two-dimensional inverse Fourier transform using only the spatial frequency information of a certain region including the spectral peak position in the image. The resulting two-dimensional phase distribution image has a low spatial frequency and is gentle.

In general, the Fourier transform method requires that the spatial frequency of the interference fringes be sufficiently higher than the spatial frequency distribution of the object, and the use of laser light is indispensable. However, when laser light is used, the problem of speckle noise arises. When the spatial frequency of the interference fringes is increased and the incoherent light is used, the interference fringes can be obtained only in a part of the image. This embodiment is particularly effective when the spatial frequency of the interference fringes is lower than the spatial frequency distribution of the object. Such a limitation occurs especially in the case of interference measurement by incoherent light having a wide spectral width of a light source.

FIG. 42 is a diagram showing a Fourier transform image obtained by performing a two-dimensional _Fourier transform on a certain interference image In (x, y). FIG. 43 is an enlarged view showing the vicinity of the origin of the Fourier transform image of FIG. 42. Since the spatial frequency of the interference fringes in the interference image is low, the spectral peak is near the origin in the Fourier transform image. Further, in the Fourier transform image, two spectral peaks exist at positions symmetrical with each other across the origin.

FIG. 44 is obtained by performing a two-dimensional inverse Fourier transform using only the spatial frequency information of a fixed region (the region indicated by the broken line circle in FIG. 43) including the position of one spectral peak in the Fourier transform image of FIG. 42. It is a figure which shows the 2D phase distribution image which was made. The phase distribution image (FIG. 44) obtained here does not show a structure having a frequency component higher than the spatial frequency of the interference fringes, and the spatial frequency is low and gentle. This phase distribution image (FIG. 44) represents only the low frequency components of the correct image (FIG. 37).

After obtaining the phase distribution image from each of the 36 interference images I ₁ to I ₃₆ by the Fourier transform method, the slope correction is performed by taking the average of these 36 phase distribution images after two-dimensional phase unwrapping. The obtained image is the reproduction result of the phase distribution by the conventional method. FIG. 45 is a diagram showing a central region of a phase image obtained by a conventional Fourier transform method. The image quality of this phase image (FIG. 45) is significantly deteriorated as compared with the correct image (FIG. 36), and the phase distribution cannot be reproduced.

In the present embodiment, the phase distribution image φLPF _n (x, y) obtained by the Fourier transform method from each interference image In (x, y) _is subjected to the offset phase shift amount of the interference image In ₍ x, y). Used to estimate the distribution. The 36 phase distribution images φLPF ₁ (x, y) to φLPF ₃₆ (x, y) are generally different from each other. Therefore, the phase distribution images ΔφLPF ₁ (x, y) to ΔφLPF ₃₆ (x, y) relative to the reference phase distribution image φLPF 0 (x, y) are obtained. The reference phase distribution image φLPF 0 (x, y) may be any one of 36 phase distribution images φLPF ₁ (x, y) to φLPF ₃₆ (x, y), or 36. The phase distribution image may be an image obtained by averaging φLPF ₁ (x, y) to φLPF ₃₆ (x, y). The relative phase distribution image ΔφLPF _n (x, y) is expressed by the following equation (14). Here, arg is an operation for taking the argument of a complex number, i is an imaginary unit, and exp is an exponential function.

FIG. 46 is a diagram showing an example of a reference phase distribution image φLPF0 (x, y). FIG. 47 is a diagram showing an example of the phase distribution image ΔφLPF _n (x, y) relative to the reference phase distribution image φLPF 0 (x, y). The relative phase distribution image ΔφLPF _n (x, y) can be approximated by the above equation (13). The values of the coefficients p1, p2, p3, and p4 when the relative phase distribution image ΔφLPF _n (x, y) is approximated by the above equation (13) can be obtained by the least squares method. From this approximation, the offset phase shift quantity distribution α _n (x, y) can be estimated for each interference image In ₍ x, y).

FIG. 48 (a) is a diagram showing the relative phase distribution in the central region of the relative phase distribution image ΔφLPF _n (x, y) of FIG. 47. FIG. 48 (b) is a diagram showing a relative phase distribution obtained by approximating the relative phase distribution of FIG. 48 (a). In this example, p1 = 3.11, p2 = -0.21, p3 = -0.31, p4 = -0.06. The relative phase distribution image ΔφLPF _n (x, y) is approximated by the above equation (13) in order to smooth the relative phase distribution image ΔφLPF _n (x, y). A smoothing process may be performed by another method, for example, a process using a smoothing filter may be performed.

In this way, in the offset phase shift amount estimation step, the offset phase shift amount distribution α _n (x, y) can be estimated for each interference image In ₍ x, y). After that, after performing the set setting step and the phase value calculation step for each pixel position, the phase image generation step is performed.

Luminance value In (x1, _y1 ) and offset phase shift amount α of pixels at arbitrary positions (x1, y1) in each of the 36 interference images I ₁ (x, y) to I ₃₆ (x, y) _n (x1, y1) can be made to correspond to each other. Therefore, for each pixel position (x1, y1), the offset phase shift amount _{is different from the 36 pixels (brightness value In (x1, y1), offset phase shift amount α n (} x1, y1)). A set including a set of pixels can be set (set setting step), and a phase value can be obtained by the above equation (2) based on the N pixels included in the set (phase value calculation step). The setting of the set and the calculation of the phase value are the same as those described above. Then, a phase image can be generated based on the phase value of each pixel position (phase image generation step).

Further, a plurality of such sets are set for each pixel position (x1, y1) (set setting step), and the phase value is obtained for each of the plurality of sets based on the N pixels included in each set (phase value). Calculation step), the phase values obtained in each of the plurality of sets may be corrected for the offset phase shift amount and then integrated (integration step) to generate a phase image based on the integrated value of the phase values at each pixel position. (Phase image generation step). Further, for each pixel position (x1, y1), 36 pixels (brightness value In (x1, y1), offset phase shift amount α _{n (} x1, y1)) are arranged in ascending or descending order with respect to the offset phase shift amount. After the change, it may be distributed to a plurality of groups in order (group setting step).

In the following, 36 pixels (brightness value In (x1, y1), offset phase shift amount α _n (x1, y1)) are rearranged in ascending _order for each pixel position (x1, y1). Later, the simulation results will be described in the case of allocating to a plurality of sets in order. The number of pixels included in each set was set to 3, and the number of sets was set to 12. After sorting in ascending order, the first pixel, the thirteenth pixel, and the 25th pixel are distributed to the first group, the second pixel, the 14th pixel, and the 26th pixel are distributed to the second group, and so on. The pixels, the 24th pixel, and the 36th pixel were distributed to the 12th set (set setting step).

Then, the phase value is obtained for each of the 12 sets based on the three pixels included in each set (phase value calculation step), and the average value is obtained after correcting the offset phase shift amount for the phase value obtained for each of the 12 sets. (Integration step). In this way, the phase value was obtained for each pixel position (x1, y1), and a phase image was generated based on the phase value of each pixel position (phase image generation step).

The grouping of 36 pixels may differ depending on the pixel position (x1, y1). However, since the grouping can be performed so that a highly accurate phase value can be obtained for each pixel position (x1, y1), the finally obtained phase image is also highly accurate.

49 and 50 are diagrams showing the phase images generated in the simulation. FIG. 49 shows a phase image before phase unwrapping and tilt correction. FIG. 50 shows the central region of the phase image after phase unwrapping and tilt correction. It can be seen that the phase images shown in these figures are accurately reproduced up to high frequency components as compared with the phase images obtained by the conventional Fourier transform method shown in FIG. 45.

FIG. 51 is a diagram showing an image of the difference between the phase image generated in the simulation and the correct answer image (FIG. 36). Although the accuracy is somewhat low in the peripheral region of the image mainly due to the estimation error of the offset phase shift amount, the phase distribution is reproduced with high accuracy in the central region of the image. The standard deviation of the error in the central region (200 x 200 pixels) of the phase image (400 x 400 pixels) was 53 milliradians. For example, when the wavelength of the light source is 550 nanometers, this amount of error corresponds to the optical path length difference of 4.6 nanometers.

When the first term on the right side of the above equation (13) is dominant in the offset phase shift amount distribution α _n (x1, y1) estimated for each interference image In (v, y) ₍ that is, the offset phase). (When the spatial change in the shift amount is small), the interference image is divided into several partial interference images, an approximate offset phase shift amount is set for each partial interference image, and the phase from each partial interference image is set. An image may be generated. At this time, the grouping may be different depending on the partial interference image. By doing so, the processing load can be reduced.

1 ... Phase image acquisition device, 2 ... Interference image acquisition unit, 3 ... Arithmetic unit, 10 ... Light source, 11, 12 ... Lens, 20 ... Two light beam interferometer, 21 ... Branching beam splitter, 22 ... Combined beam splitter , 31 ... Beam Splitter, 32 ... First Reflection Mirror, 41 ... Beam Splitter, 42 ... Second Reflection Mirror, 43 ... Piezo Element, 44 ... Stage, 51, 52 ... Lens, 53 ... Tube Lens, 61 ... Imager, 72 ... Control unit.

Claims (18)

A light source, a two-beam interferometer that branches the light output from the light source into two light sources and then combines the two light sources to output interference light, and receives interference light output from the two-beam interference meter. An interference image acquisition unit that includes an imager and acquires an interference image of an object arranged on the optical path of any light source in the two light source interferometer by the imager.
An arithmetic unit that generates a phase image of the object based on the interference image acquired by the interference image acquisition unit, and a calculation unit.
Equipped with
The interference image acquisition unit acquires M interference images of the object in a state where the optical path length difference between the two light fluxes fluctuates with time in the two-luminous flux interferometer.
The arithmetic unit
The offset phase shift amount of the entire image is estimated for each of the M interference images, and the offset phase shift amount is estimated.
A set including N interference images having different offset phase shift amounts from the M interference images is set.
A phase image is generated based on the N interference images included in the set.
Phase image acquisition device (however, 3 ≦ N ≦ M). The arithmetic unit
A plurality of the sets are set, and a phase image is generated for each of the plurality of sets based on the N interference images included in each set.
After correcting the offset phase shift amount for the phase images generated by each of the plurality of sets, integration is performed.
The phase image acquisition device according to claim 1. The calculation unit sorts the M interference images in ascending or descending order with respect to the offset phase shift amount, and then sorts the M interference images into the plurality of sets in order.
The phase image acquisition device according to claim 2. The calculator adjusts the coefficients multiplied by the interfering image to generate the phase image by iterative optimization calculations.
The phase image acquisition device according to any one of claims 1 to 3. A light source, a two-beam interferometer that branches the light output from the light source into two light sources and then combines the two light sources to output interference light, and receives interference light output from the two-beam interference meter. An interference image acquisition unit that includes an imager and acquires an interference image of an object arranged on the optical path of any light source in the two light source interferometer by the imager.
An arithmetic unit that generates a phase image of the object based on the interference image acquired by the interference image acquisition unit, and a calculation unit.
Equipped with
The interference image acquisition unit acquires M interference images of the object in a state where the optical path length difference between the two light fluxes fluctuates with time in the two-luminous flux interferometer.
The arithmetic unit
The distribution of the offset phase shift amount is estimated for each of the M interference images, and the distribution is estimated.
For each pixel position, a set including N pixels having different offset phase shift amounts from the pixels of each of the M interference images is set.
For each pixel position, the phase value was obtained based on the N pixels included in the set.
Generate a phase image based on the phase value of each pixel position,
Phase image acquisition device (however, 3 ≦ N ≦ M). The arithmetic unit
A plurality of the sets are set, and the phase value is obtained for each of the plurality of sets based on the N pixels included in each set.
After correcting the offset phase shift amount for the phase values obtained for each of the plurality of sets, they are integrated.
Generates a phase image based on the integrated value of the phase values at each pixel position.
The phase image acquisition device according to claim 5. The calculation unit sorts the pixels of each of the M interference images in ascending or descending order with respect to the offset phase shift amount for each pixel position, and then sorts them into the plurality of sets in order.
The phase image acquisition device according to claim 6. The two-luminous flux interferometer causes a temporal variation in the optical path length difference between the two light fluxes due to fluctuations in the liquid level of the liquid arranged on both or one of the two light fluxes.
The phase image acquisition device according to any one of claims 1 to 7. The light source outputs incoherent light,
The phase image acquisition device according to any one of claims 1 to 8. It receives the light source, the two-beam interferometer that branches the light output from the light source into two light beams and then combines the two light beams to output the interference light, and the interference light output from the two-beam interference meter. Using an interference image acquisition unit including an imager, in a state where the difference in optical path length between the two light beams in the two light beam interferometers fluctuates with time, on the optical path of any light beam in the two light beam interferometers. The interference image acquisition step of capturing and acquiring M interference images with the imager for the object arranged in
An offset phase shift amount estimation step for estimating the offset phase shift amount of the entire image for each of the M interference images, and an offset phase shift amount estimation step.
A set setting step for setting a set including N interference images having different offset phase shift amounts from the M interference images, and a set setting step.
A phase image generation step of generating a phase image based on the N interference images included in the set, and a phase image generation step.
(However, 3 ≦ N ≦ M). In the set setting step, a plurality of the sets are set, and the set is set.
In the phase image generation step, a phase image is generated for each of the plurality of sets based on the N interference images included in each set.
Further, an integration step of correcting the offset phase shift amount and then integrating the phase images generated by each of the plurality of sets is provided.
The phase image acquisition method according to claim 10. In the set setting step, the M interference images are sorted in ascending or descending order with respect to the offset phase shift amount, and then are sequentially distributed to the plurality of sets.
The phase image acquisition method according to claim 11. In the phase image generation step, the coefficients multiplied by the interfering image to generate the phase image are adjusted by iterative optimization calculations.
The phase image acquisition method according to any one of claims 10 to 12. It receives the light source, the two-beam interferometer that branches the light output from the light source into two light beams and then combines the two light beams to output the interference light, and the interference light output from the two-beam interference meter. Using an interference image acquisition unit including an imager, in a state where the difference in optical path length between the two light beams in the two light beam interferometers fluctuates with time, on the optical path of any light beam in the two light beam interferometers. The interference image acquisition step of capturing and acquiring M interference images with the imager for the object arranged in
An offset phase shift amount estimation step for estimating the distribution of the offset phase shift amount for each of the M interference images, and an offset phase shift amount estimation step.
A set setting step for setting a set including N pixels having different offset phase shift amounts from the pixels of each of the M interference images for each pixel position.
For each pixel position, a phase value calculation step of obtaining a phase value based on the N pixels included in the set, and
A phase image generation step that generates a phase image based on the phase value of each pixel position,
(However, 3 ≦ N ≦ M). In the set setting step, a plurality of the sets are set, and the set is set.
In the phase value calculation step, the phase value is obtained for each of the plurality of sets based on the N pixels included in each set.
Further provided with an integration step of correcting the offset phase shift amount for the phase values obtained in each of the plurality of sets and then integrating them.
In the phase image generation step, a phase image is generated based on the integrated value of the phase values of each pixel position.
The phase image acquisition method according to claim 14. In the set setting step, for each pixel position, the pixels of each of the M interference images are rearranged in ascending or descending order with respect to the offset phase shift amount, and then are sequentially distributed to the plurality of sets.
The phase image acquisition method according to claim 15. In the interference image acquisition step, the fluctuation of the liquid level of the liquid arranged on the optical path of both or one of the two luminous fluxes in the two luminous flux interferometer causes the temporal variation of the optical path length difference between the two luminous fluxes. Causes,
The phase image acquisition method according to any one of claims 10 to 16. The light source outputs incoherent light,
The phase image acquisition method according to any one of claims 10 to 17.

Images