KR102628967B1 - Tunable Fourier ptychographic microscopy - Google Patents

Abstract

The present invention relates to a tunable FPM, and more specifically, to the FPM, a first illuminator for emitting an LED beam; It consists of a tilting mirror array in which a plurality of tilting mirrors are arranged, and the LED first beam is irradiated from the first illuminator, reflected, transmitted through the objective lens, and sequentially directed to the measurement object at different angles. Dynamic Mirror Device (DMD) for irradiation; and a plurality of LED light sources, and is composed of a second LED array that sequentially irradiates a plurality of second LED beams to the measurement object at different angles after irradiation by the first illuminator, and the objective lens is located in the central hole. a second illuminator having a second panel; A condenser lens configured to collect beams coming from a measurement object to which the first and second LED beams are irradiated; and a photodetector that receives light from the condenser lens and acquires images for each of the plurality of first beams and second beams.

Description

Tunable Fourier ptychographic microscopy}

The present invention relates to a tunable FPM and its operating method. More specifically, it relates to a tunable FPM that can change specifications such as resolution and FOV using a dynamic mirror device (DMD).

Fourier Ptychographic Microscopy (FPM) is a phase retrieval method developed by Guoan Zheng in 2013, and the phase can be calculated without using a reference beam like existing digital holographic microscopy.

This FPM can calculate the phase without a reference beam, so the system can be compact, and it has the advantage of being resistant to vibration because the signal beam and reference beam are not differentiated.

In addition, while existing digital holographic microscopy has a small FOV (field of view) and DOF (depth of focus) when using a high NA objective lens to obtain high resolution, FPM replaces the condenser lens with an LED array to achieve high resolution. By enabling the illumination angle, high resolution can be obtained even with a low NA objective lens, which has the advantage of having a wide FOV and DOF. Additionally, since a high illumination angle is achieved with an LED array, there is no need for a mechanical driver.

Figure 1 shows the configuration of the transmission type FPM system 1, and Figure 2 shows the spectrum of the measurement object. As shown in Figure 1, the transmission type FPM system (1) consists of an LED array (10) consisting of a plurality of LED light sources (11) that irradiate the measurement object (2) at different angles, and the measurement object (2). It is composed of an objective lens 20 that forms an image of the LED beam that has passed through, a condenser lens 30, and a photodetector 40 that acquires an image of the LED beam irradiated to the measurement object 2.

Using the LED array 10 composed of N LED light sources 11, N beams of different irradiation angles are sequentially irradiated to the measurement object 2 and N images are stored in the photodetector 40.

And the analysis means FFTs the N images, as shown in FIG. 2 (○ in FIG. 2 is the spectrum of the signal focused by the objective lens through the beam coming from one LED), and the angles of θx and θy in the spectral domain The phase is calculated by stitching while positioning the corresponding position.

However, while transmissive FPM is widely used in measuring transmissive samples such as biological samples, much research has been conducted, while reflective FPM is in a state of slow research due to difficulties in setting up despite the high demand for measuring opaque industrial samples.

Figure 3 shows the configuration of a conventional reflective FPM system. Figure 4a shows an enlarged view of the dark field LED array panel portion in Figure 3, and Figure 4b shows a photograph of the dark field LED array panel.

As shown in FIGS. 3, 4A, and 4B, there is a problem in that the signal-to-noise ratio (SNR) is reduced due to the use of a flat LED illuminator. In other words, the farther the LED light source is from the measurement object S, the weaker the intensity of light reaching the measurement object is. This causes a low SNR, resulting in a problem of low resolution due to the inability to obtain a high spatial frequency signal. .

Japanese published patent 2016-530567 Japanese published patent 2018-504627

Therefore, the present invention was developed to solve the above-described conventional problems. According to an embodiment of the present invention, the position of the tilting mirror at the on angle is changed according to the NA of the objective lens to be used using the DMD, ultimately It can perform the same function as changing the position of the LED, saving the time and cost of manufacturing the LED array on the PCB each time depending on the objective lens to be used, and specifications such as resolution and FOV are not fixed. The purpose is to provide a tunable FPM that can be changed.

Meanwhile, the technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly apparent to those skilled in the art from the description below. It will be understandable.

The first object of the present invention is to provide, in FPM, a first illuminator for emitting an LED beam; It consists of a tilting mirror array in which a plurality of tilting mirrors are arranged, and the LED first beam is irradiated from the first illuminator, reflected, transmitted through the objective lens, and sequentially projected to the measurement object at different angles. Dynamic Mirror Device (DMD) for irradiation; and a plurality of LED light sources, and is composed of a second LED array that sequentially radiates a plurality of second LED beams to the measurement object at different angles after irradiation by the first illuminator, and the objective lens is located in the central hole. a second illuminator having a second panel; A condenser lens configured to collect beams coming from a measurement object to which the first and second LED beams are irradiated; And a photodetector that receives light from the condenser lens and acquires images for each of the plurality of first beams and second beams. It can be achieved as a tunable FPM, including a.

And it may be characterized by including a mirror driving unit that drives each of the plurality of tilting mirrors provided in the DMD to be at an on or off angle.

In addition, the first LED beam reflected and irradiated from the DMD passes through the optical system, is reflected by the beam splitter, and then enters the measurement object through the objective lens, and the on angle is an angle that causes it to be incident on the optical system side, The off angle may be characterized as an angle that causes incident to occur outside the optical system.

It may further include a control unit that controls the driving unit to change the position of the tilting mirror at the on angle to change the NA of the illuminator to adjust the specifications of the FPM.

In addition, when the objective lens NA is changed due to replacement of the objective lens, the control unit controls the driving unit to change the position of the tilting mirror at the on angle so that the set specifications of the FPM are maintained. .

It may further include a parabolic mirror that causes each second beam generated by the second illuminator to be reflected and incident on the measurement object.

In addition, the second panel has a ring shape with a central hole, the second LED array has a ring shape arranged at a predetermined distance from each other in the circumferential direction with respect to the center point of the second panel, and the ring-shaped second LED array has a radial direction. It may be characterized as being arranged in plural numbers spaced apart at a specific interval.

A second object of the present invention is a transmissive FPM, which includes an illuminator that irradiates an LED beam; A dynamic mirror device (dynamic mirror device) that consists of a tilting mirror array in which a plurality of tilting mirrors are arranged, receives the LED beam from the illuminator, reflects it in the form of an LED array, and sequentially radiates a plurality of LED beams to the measurement object at different angles. DMD); and an objective lens through which the LED beam passing through the measurement object is imaged; A condenser lens configured to collect the beam coming from the measurement object irradiated with the LED beam; And a photodetector that receives light from the condenser lens and acquires an image for each LED beam. It can be achieved as a tunable FPM, including a.

And for the second purpose, it may be characterized by including a mirror driving unit that drives each of the plurality of tilting mirrors provided in the DMD to be at an on or off angle.

In addition, for the second purpose, the on angle may be an angle that causes incident to occur toward the measurement object, and the off angle may be an angle that causes incident to occur outside the measurement object.

And for the second purpose, it may further include a control unit that controls the mirror driving unit to change the position of the tilting mirror at the on angle to change the illuminator NA to adjust the specifications of the FPM.

In addition, for the second purpose, when the objective lens NA is changed due to replacement of the objective lens, the control unit controls the mirror driving unit to change the position of the tilting mirror at the on angle so that the set specifications of the FPM are maintained. It can be characterized as:

For the first and second purposes, the method may further include analysis means for calculating a composite image having phase information of the measurement object by locating and stitching each of the images in the spectral domain.

According to the tunable FPM according to the present invention, the same function as changing the position of the LED can be performed by changing the position of the tilting mirror whose on angle is turned on according to the NA of the objective lens to be used using DMD. Depending on the objective lens used, the time and cost of manufacturing an LED array on a PCB can be saved each time, and specifications such as resolution and FOV are not fixed but can be changed.

Meanwhile, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

The following drawings attached to this specification illustrate preferred embodiments of the present invention, and serve to further understand the technical idea of the present invention along with the detailed description of the invention, so the present invention is limited only to the matters described in such drawings. It should not be interpreted as such.
Figure 1 is a configuration diagram of a transmission type FPM system;
Figure 2 shows the spectrum of the measurement object,
Figure 3 is a configuration diagram of a conventional transmissive FPM system;
Figure 4a is an enlarged view of the dark field LED array panel portion in Figure 3;
Figure 4b is a photo of a dark field LED array panel,
Figure 5 is a configuration diagram of a reflective FPM using a parabolic mirror, which is an earlier patent of the present invention;
Figure 6 is a configuration diagram of a tunable reflective FPM according to an embodiment of the present invention;
Figure 7 shows a configuration diagram of a tunable transmissive FPM according to an embodiment of the present invention.

The above objects, other objects, features and advantages of the present invention will be easily understood through the following preferred embodiments related to the attached drawings. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided so that the disclosed content will be thorough and complete and so that the spirit of the present invention can be sufficiently conveyed to those skilled in the art.

In this specification, when an element is referred to as being on another element, it means that it may be formed directly on the other element or that a third element may be interposed between them. Also, in the drawings, the thickness of components is exaggerated for effective explanation of technical content.

Embodiments described herein will be explained with reference to cross-sectional views and/or plan views, which are ideal illustrations of the present invention. In the drawings, the thicknesses of films and regions are exaggerated for effective explanation of technical content. Therefore, the shape of the illustration may be changed depending on manufacturing technology and/or tolerance. Accordingly, embodiments of the present invention are not limited to the specific form shown, but also include changes in form produced according to the manufacturing process. For example, an area shown as a right angle may be rounded or have a shape with a predetermined curvature. Accordingly, the regions illustrated in the drawings have properties, and the shapes of the regions illustrated in the drawings are intended to illustrate a specific shape of the region of the device and are not intended to limit the scope of the invention. In various embodiments of the present specification, terms such as first and second are used to describe various components, but these components should not be limited by these terms. These terms are merely used to distinguish one component from another. Embodiments described and illustrated herein also include complementary embodiments thereof.

The terminology used herein is for describing embodiments and is not intended to limit the invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, 'comprises' and/or 'comprising' does not exclude the presence or addition of one or more other elements.

In describing specific embodiments below, various specific details have been written to explain the invention in more detail and to aid understanding. However, a reader with sufficient knowledge in the field to understand the present invention can recognize that it can be used without these various specific details. In some cases, it is mentioned in advance that when describing the invention, parts that are commonly known but are not significantly related to the invention are not described in order to prevent confusion without any reason in explaining the invention.

Before explaining the tunable FPM according to an embodiment of the present invention, a reflective FPM using a parabolic mirror, which is a previously filed patent of the inventor of the present invention (10-2020-0020510, filed on February 19, 2020, unpublished at the time of filing of the present invention) Let me explain briefly.

Figure 5 shows the configuration of a reflective FPM using a parabolic mirror, which is a previously filed patent of the present invention.

As shown in FIG. 5, the reflective FPM 100 using a parabolic mirror includes a first illuminator 110 corresponding to a bright field illuminator, an objective lens 20, and a second illuminator corresponding to a dark field illuminator. It can be seen that it is comprised of 150, a condenser lens 30, and a photodetector 40.

The first illuminator 110 includes a first panel 120, an optical system 130, and a beam splitter 140, and is disposed on the first panel 120 and includes a plurality of first LEDs. It is configured to include a plurality of first LED arrays 121 having a light source 122.

The second illuminator 150 is a dark field illuminator and is configured to be reflected by the parabolic mirror 180 and incident on the measurement object 2 without passing through the objective lens 20.

By applying the parabolic mirror 180, the intensity of light reaching the measurement object 2 is made uniform even if it corresponds to the second LED light source 162, which is far from the measurement object 2, and thus the conventional reflective FPM (3) Compared to , it is possible to increase SNR and obtain high resolution.

The second illuminator 150 includes a second panel 160 and a parabolic mirror 180. The second panel 160 is provided with a plurality of second LED light sources 162, and after irradiation by the first illuminator 110, a plurality of second LED beams are sequentially irradiated to the measurement object 2 at different angles. It consists of a plurality of second LED arrays 161.

As shown in Figure 5, the second panel 160 is configured in the shape of a ring plate with a central hole 170, and the measurement object 2 is located in this central hole 170.

In the case of this conventional FPM (1, 3, 100), resolution has a relationship with NA _effective as shown in Equation 1 below.

[Equation 1]

In Equation 1, Λ is the minimum width that can be resolved, and λ is the wavelength of light used.

Additionally, NA _effective is the sum of NA _obj of the objective lens 20 and NA _illu of the illumination beam. And in the case of the first illuminator 110, which is a bright field illuminator, the NA of the illumination beam is less than or equal to NA _obj .

Once the NA _illu of the illumination beam is determined, the positions of the LEDs in the LED array 121 of the bright field illuminator 110 are determined and manufactured on the PCB and cannot be modified.

Therefore, even if it is desired to use various objective lenses 20, a problem arises in that each LED array must be manufactured accordingly.

In other words, the conventional FPM (1, 3, 100) has fixed specifications such as resolution, FOV, etc., and if the specifications are to be changed, the LED array must be manufactured each time, resulting in excessive manufacturing time and cost.

Therefore, the development of an FPM that can tune and change specifications such as resolution without redesigning and manufacturing the LED array was required.

Figure 6 shows a configuration diagram of a tunable reflective FPM according to an embodiment of the present invention.

The configuration of the tunable reflective FPM 200 according to the embodiment of the present invention shown in FIG. 6 is basically the first lens 131 and the field stop 132 in the configuration of the reflective FPM in FIG. 5 mentioned above. , an optical system 130 consisting of a second lens 133, a beam splitter 140, a condenser lens 30, an objective lens 20, a second illuminator 150, a photodetector 40, and a parabolic mirror 180. ) can be configured to include as is.

In the case of the first illuminator, in the tunable reflective FPM according to an embodiment of the present invention, it may not be configured in the form of an LED array but may be configured to emit a general LED beam.

The LED beam generated from this LED illuminator 210 passes through the lens 220 and is irradiated to the dynamic mirror device (DMD 230).

The DMD 230 is composed of a tilting mirror array in which a plurality of tilting mirrors are arranged, and receives the LED beam from the LED illuminator 210 and reflects it in the form of an LED array. That is, the irradiated LED flat beam is reflected on the tilting mirror located at the on angle and is incident on the optical system 130. In other words, it is possible to change the NA of the illuminator by changing the position and number of tilting mirrors corresponding to the on angle.

The first LED beam reflected from the DMD (230) passes through the first lens (131), field stop (132), and second lens (133) of the optical system (130) and is reflected from the beam splitter (140) to the objective lens (20). ) is transmitted and sequentially irradiated to the measurement object (2) at different angles.

The second illuminator 150 is configured in the same way as the reflective FPM according to FIG. 5 mentioned above. A second LED array 161 is equipped with a plurality of LED light sources and sequentially irradiates a plurality of second LED beams to the measurement object 2 at different angles after irradiation by the LED illuminator 210 and the DMD 230. It is configured to have a second panel 160 on which the objective lens 20 is located in the central hole 170.

In addition, it may include a parabolic mirror 180 that allows each second beam generated by the second illuminator 150 to be reflected and incident on the measurement object 2, and at this time, the second panel 160 has a central hole. (170) is formed in the form of a ring plate, and the second LED array 161 is a ring shape arranged at a predetermined distance from each other in the circumferential direction with respect to the center point of the second panel 160, and the ring-shaped second LED array 161 is They are arranged in plural numbers spaced apart at specific intervals in the radial direction.

And the condenser lens 30 is configured to collect the beam coming from the measurement object 2 to which the first and second LED beams are irradiated, and the photodetector 40 receives the light from the condenser lens 30 and collects a plurality of beams. Images for each of the first beam and the second beam are acquired.

Additionally, the embodiment of the present invention includes a mirror driving unit that drives each of the plurality of tilting mirrors provided in the DMD 230 to be turned on or off.

The first LED beam reflected and irradiated from the DMD (230) passes through the optical system (130), is reflected by the beam splitter (140), then passes through the objective lens (20) and enters the measurement object (2), and is then transmitted to the tilting mirror. The on angle corresponds to an angle that causes the reflected LED first beam to enter the optical system 130, and the off angle corresponds to an angle that causes the reflected LED first beam to incident out of the optical system 130.

The control unit controls the mirror driving unit to change the position of the tilting mirror at the on angle, thereby changing the NA of the illuminator to adjust the specifications of the FPM.

In addition, when the objective lens NA is changed due to replacement of the objective lens 20, the control unit controls the mirror driving unit to change the position of the tilting mirror at the on angle so that the set specifications of the FPM are maintained.

That is, according to an embodiment of the present invention, the position of the tilting mirror whose on angle is changed depending on the NA of the objective lens 20 to be used using the DMD, ultimately performing the same function as changing the position of the LED. do.

Therefore, it is possible to save the time and cost of manufacturing the LED array on the PCB each time depending on the objective lens 20 to be used, and to provide a tunable FPM in which specifications such as resolution and FOV are not fixed and can be changed. There will be.

Figure 7 shows a configuration diagram of a tunable transmissive FPM according to an embodiment of the present invention. This tunable FPM using DMD can be applied not only to the reflective FPM, but also to the transmissive FPM as shown in Figure 7. In the transmissive FPM, the bright field illuminator and the dark field illuminator are not divided, so the NA _effective can be changed more effectively to adjust the FPM. It becomes possible to adjust the specifications.

The tunable transmissive FPM 200 according to an embodiment of the present invention includes an illuminator 210, a DMD 230, an objective lens 20, a condenser lens 30, and a photodetector 40 that irradiate an LED beam. It can be configured to include.

In the LED illuminator 210, a general LED beam, not in an array form, is irradiated to the DMD side through the lens 220. As mentioned above, the DMD (230) consists of a tilting mirror array in which a plurality of tilting mirrors are arranged, and receives the LED beam from the LED illuminator (210) and reflects it in the form of an LED array to measure the measurement object (2) at different angles. ) to irradiate multiple LED beams sequentially.

Then, the LED beam that has passed through the measurement object (2) is formed into an image in the objective lens (20), and the condenser lens (30) is configured to collect the beam coming from the measurement object (2) irradiated with the LED beam. The photodetector 40 receives light from the condenser lens 30 and acquires images for each LED beam. Then, the analysis means positions each image in the spectral domain and stitches it to produce a composite image with phase information of the measurement object 2.

As mentioned earlier, the mirror driving unit drives each of the plurality of tilting mirrors provided in the DMD 230 to be at an on or off angle, where the on angle is the angle that causes incident to occur toward the measurement object 2, and the off angle is It corresponds to the angle that causes the incident to occur outside the measurement object (2).

The control unit controls the mirror driving unit to change the position of the tilting mirror at the on angle and changes the NA of the illuminator to adjust the specifications of the FPM.

And when the objective lens NA is changed due to replacement of the objective lens 20, the control unit controls the mirror driving unit to change the position of the tilting mirror at the on angle so that the set specifications of the FPM are maintained.

In addition, the apparatus and method described above are not limited to the configuration and method of the above-described embodiments, but all or part of each embodiment can be selectively combined so that various modifications can be made. It may be composed.

1: Conventional transmissive FPM
2: Measurement object
3: Conventional reflective FPM
4:LED illuminator
5:Lens
10:LED array
11:LED light source
20: Objective lens
30: Condenser lens
40: Photodetector
100: Reflective FPM using parabolic mirror
110: First illuminator (bright field illuminator)
120: First panel (bright field panel)
121: 1st LED array (bright field LED array)
122: 1st LED light source (bright filter LED light source)
130:Optics
131: First lens
132:Field stop
133:Second lens
140: Beam splitter
150: Second illuminator (dark field illuminator)
160: Second panel (dark field panel)
161: Second LED array (dark field LED array)
162: Second LED light source (dark field LED light source)
170:Central hall
180: Parabolic mirror
200: Tunable FPM
210:LED illuminator
220:Lens
230:DMD

Claims (13)

In reflective FPM,
A first illuminator that emits an LED beam;
It consists of a tilting mirror array in which a plurality of tilting mirrors are arranged, and the LED first beam is irradiated from the first illuminator, reflected, transmitted through the objective lens, and sequentially projected to the measurement object at different angles. Dynamic Mirror Device (DMD) for irradiation; and
It is equipped with a plurality of LED light sources and consists of a second LED array that sequentially radiates a plurality of second LED beams to the measurement object at different angles after irradiation by the first illuminator, and the objective lens is provided in the central hole. a second illuminator having a second panel;
A condenser lens configured to collect beams coming from a measurement object to which the first and second LED beams are irradiated;
a photodetector that receives light from the condenser lens and acquires images for each of the plurality of first beams and second beams; and
a mirror driving unit that drives each of the plurality of tilting mirrors provided in the DMD to be at an on or off angle;
A control unit that controls the mirror driving unit to change the position of the tilting mirror at the on angle to change the NA of the illuminator to adjust the specifications of the FPM,
The first LED beam reflected and irradiated from the DMD passes through the optical system, is reflected by the beam splitter, and then enters the measurement object through the objective lens,
The on angle is an angle that causes incident to occur toward the optical system, and the off angle is an angle that causes incident to occur outside the optical system,
When the objective lens NA is changed due to replacement of the objective lens, the control unit controls the mirror driving unit to change the position of the tilting mirror at the on angle so that the set specifications of the FPM are maintained. This is possible with FPM.
delete delete delete delete According to clause 1,
A tunable FPM further comprising a parabolic mirror that causes each second beam generated by the second illuminator to be reflected and incident on the measurement object.
According to clause 6,
The second panel is in the shape of a ring plate with a central hole formed,
The second LED array is a ring-shaped second LED array arranged at a predetermined distance from each other in the circumferential direction with respect to the center point of the second panel, and the ring-shaped second LED array is arranged in a plurality at a specific interval in the radial direction. Tunable FPM.
In transmissive FPM,
An illuminator that emits an LED beam;
A dynamic mirror device (dynamic mirror device) that consists of a tilting mirror array in which a plurality of tilting mirrors are arranged, receives the LED beam from the illuminator, reflects it in the form of an LED array, and sequentially radiates a plurality of LED beams to the measurement object at different angles. DMD); and
An objective lens in which the LED beam that passes through the measurement object is imaged;
A condenser lens configured to collect the beam coming from the measurement object irradiated with the LED beam; and
a photodetector that receives light from the condenser lens and acquires an image for each LED beam;
a mirror driving unit that drives each of the plurality of tilting mirrors provided in the DMD to be at an on or off angle; and
A control unit that controls the mirror driving unit to change the position of the tilting mirror at the on angle to change the illuminator NA to adjust the specifications of the FPM,
The on angle is an angle that causes incident to occur toward the measurement object, and the off angle is an angle that causes incident to occur outside the measurement object,
When the objective lens NA is changed due to replacement of the objective lens, the control unit controls the mirror driving unit to change the position of the tilting mirror at the on angle so that the set specifications of the FPM are maintained. This is possible with FPM.
delete delete delete delete According to claim 1 or 8,
A tunable FPM further comprising analysis means for calculating a composite image having phase information of the measurement object by locating and stitching each of the images in the spectral domain.

Images