KR102640598B1 - Fourier ptychographic microscopy - Google Patents

Abstract

This specification relates to a Fourier tychographic microscope, and more specifically, to a Fourier tychographic microscope capable of acquiring a high-resolution final image at a wide angle. A Fourier tychographic microscope according to an embodiment of the present specification includes a sample plane on which a sample to be observed is provided, an LED array including a plurality of LEDs that sequentially emit light and illuminate the sample at various angles, an aperture function (Pupil function), and An objective lens that projects the light scattered from the sample onto a Fourier plane using Fourier transform and generates a plurality of observation images, and the plurality of observation images and an objective function It includes an image processing unit that acquires a high-resolution reconstructed image.

Description

FOURIER PTYCHOGRAPHIC MICROSCOPY}

This specification relates to a Fourier tychographic microscope, and more specifically, to a Fourier tychographic microscope capable of acquiring a high-resolution final image at a wide angle.

3D microscopy technology for acquiring 3D information largely includes a method for restoring phase information and a light field method. Methods for restoring phase information include Fourier ptychographic, digital holography, or multi-heights measuring methods.

Fourier tychographic microscopy collects multiple low-resolution images at various illumination angles in Fourier space and reconstructs them into a high-resolution composite image. Fourier tychographic microscopes are similar to general microscopes in terms of optical systems, but use multiple LEDs rather than halogen lamps as the light source.

A conventional Fourier tychographic microscope requires tens to hundreds of observation images to restore one image, and each of these individually captured observation images is restored into a restored image through a restoration algorithm.

However, such a conventional Fourier tychographic microscope restores hundreds of observation images into one reconstructed image, so a lot of noise occurs and the difference between bright and dark areas is not noticeable (i.e., boundary contrast). ) is a small) problem.

In addition, the conventional Fourier tychographic microscope acquired a reconstructed image by assuming an ideal aperture function without considering noise for the aperture function. Therefore, since it is difficult to obtain an accurate reconstructed image, the need for an aperture function that takes noise into account is required.

The purpose of this specification is to provide a Fourier tychographic microscope that can remove noise and improve contrast through a third loss function and a fourth loss function.

Additionally, the purpose of the present specification is to provide a Fourier tychographic microscope that can have high data fidelity by using an aperture function that reflects noise corresponding to each of a plurality of LEDs without assuming an ideal aperture function.

The purposes of the present specification are not limited to the purposes mentioned above, and other purposes and advantages of the present specification that are not mentioned can be understood through the following description and will be more clearly understood by the examples of the present specification. Additionally, it will be readily apparent that the objects and advantages of the present specification can be realized by the means and combinations thereof indicated in the patent claims.

A Fourier tychographic microscope according to an embodiment of the present specification includes a sample plane on which a sample to be observed is provided, an LED array including a plurality of LEDs that sequentially emit light and illuminate the sample at various angles, an aperture function (Pupil function), and An objective lens that projects the light scattered from the sample onto a Fourier plane using Fourier transform and generates a plurality of observation images, and the plurality of observation images and an objective function It includes an image processing unit that acquires a high-resolution reconstructed image.

In addition, the Fourier tychographic microscope according to an embodiment of the present specification further includes an image restoration unit that obtains a visualized reconstructed image through inverse Fourier transform of the reconstructed image.

Additionally, in one embodiment of the present specification, the image processing unit acquires the reconstructed image in a way that minimizes loss through an objective function.

Additionally, in one embodiment of the present specification, the objective function is expressed by Equation 1 below.

[Equation 1]

here, is the objective function, is the reconstructed image, is the aperture function, is a plurality of observation images, is the first loss function, is the second loss function, is the third loss function, is the fourth loss function, , , Each represents a coefficient that determines the weight.

Additionally, in one embodiment of the present specification, the first loss function is calculated through Equation 2 below.

[Equation 2]

here, is the first loss function, is the reconstructed image, is the aperture function, is the observation image, means restored image.

Additionally, in one embodiment of the present specification, the second loss function is calculated through Equation 3 below.

[Equation 3]

here, is the aperture function including noise, means an ideal aperture function that does not consider noise.

Additionally, in one embodiment of the present specification, the third loss function is calculated through Equation 6 below.

[Equation 6]

here, refers to the reconstructed image, and refers to components that exist in the spatial domain of the reconstructed image.

Additionally, in one embodiment of the present specification, the fourth loss function is calculated through Equation 7 below.

[Equation 7]

here, refers to the reconstructed image, a and b are When is a complex number in the form a+bi, it means a=Re(U) and b=Im(U), respectively.

The Fourier tychographic microscope according to an embodiment of the present specification can remove noise and improve contrast through a third loss function and a fourth loss function.

Additionally, the Fourier tychographic microscope according to an embodiment of the present specification can have high data fidelity by using an aperture function that reflects noise corresponding to each of a plurality of LEDs without assuming an ideal aperture function.

1 is a block diagram of a Fourier tychographic microscope according to an embodiment of the present specification.
Figure 2 is a diagram showing the structure of a Fourier tychographic microscope according to an embodiment of the present specification.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention. While describing each drawing, similar reference numerals are used for similar components.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term and/or includes any of a plurality of related stated items or a combination of a plurality of related stated items.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

FIG. 1 is a block diagram of a Fourier tychographic microscope according to an embodiment of the present specification, and FIG. 2 is a diagram showing the structure of a Fourier tychographic microscope according to an embodiment of the present specification.

1 and 2, the Fourier tychographic microscope 100 includes a sample plane 110, an LED array 120, an objective lens 130, an image processing unit 140, and an image restoration unit 150. .

The sample plane 110 is provided with a sample that is an object of observation. The sample may be a microscopic object that is difficult to identify with the average human eye.

LED array 120 includes a plurality of LEDs that illuminate the sample at various angles. A plurality of LEDs may be arranged at predetermined intervals and emit light sequentially to irradiate light to the sample.

The objective lens 130 magnifies the image of the sample and serves to Fourier transform the incident light using scattering information irradiated to the sample from the LED array. That is, the light scattered from the sample enters the objective lens 130, is Fourier transformed, and is projected onto the Fourier plane 131 to generate an observation image.

At this time, the location and type of the generated observation image may be different depending on the emission position of the LED or the angle of light incidence, and one observation image is generated corresponding to the emission of one LED. Accordingly, a plurality of observation images can be generated according to the emission of a plurality of LEDs.

Additionally, the spectrum of observed images that can be obtained at one time is limited in proportion to the objective lens magnification. In other words, the higher the magnification of the objective lens, the larger the sample is magnified and the observable area is reduced, and the lower the objective lens magnification, the smaller the sample is magnified and the observable area increases. Therefore, when the objective lens has a high magnification, the spectrum is limited, but high-resolution observation images can be obtained.

Additionally, each observed image may appear in a circular shape depending on the aperture function. The aperture function is a function that defines the imageable area according to microscope photography, and the inside of the area can be expressed as 1 and the outside of the area as 0. A detailed description of the aperture function will be described later with reference to Equation 4 and Equation 5.

The image processing unit 140 acquires a high-resolution reconstructed image through a plurality of observation images and an objective function.

The objective function is a loss function used through an image model to obtain a reconstructed image, and the image processing unit 140 acquires the reconstructed image in a direction that minimizes the objective function.

Specifically, the objective function can be expressed as Equation 1 below.

[Equation 1]

here, is the objective function, is the reconstructed image, is the aperture function, is a plurality of observation images, is the first loss function, is the second loss function, is the third loss function, is the fourth loss function, , , Each represents a coefficient that determines the weight.

When a plurality of observation images are input to the objective function, the image processing unit 140 outputs a reconstructed image and an aperture function value that minimize the objective function. The objective function is composed of the sum of the first to fourth loss functions, and when each loss is minimized, the optimal reconstructed image and aperture function can be obtained.

The first loss function is calculated through Equation 2 below.

[Equation 2]

here, is the first loss function, is the reconstructed image, is the aperture function, is the observation image, means restored image. According to Equation 2, the smaller the difference between the restored image and the observed image, the smaller the first loss function has a loss value.

The second loss function is calculated through Equation 3 below.

[Equation 3]

here, is the aperture function including noise, means an ideal aperture function that does not consider noise. The second loss function is a loss function for the aperture function, and the aperture function considering noise in this specification can be calculated through Equation 4 below.

[Equation 4]

here, means noise.

Additionally, the ideal aperture function that does not consider noise Can be expressed as Equation 5.

[Equation 5]

Here, r means the radius of the aperture, that is, the radius of the observation area, and p means the center point of the observation area. Therefore, the aperture function does not take noise into account. The distance from the center point of the observation area is compared with the radius of the aperture. If the distance from the center point of the observation area is equal to or smaller than the radius of the aperture, the inner area has a value of 1, and the distance from the center point of the observation area is the radius of the aperture. If it is larger than the radius, it is the outer area and has a value of 0.

Meanwhile, in this specification, noise may have different values depending on the LED emission position. Accordingly, the aperture function may also have different values depending on the LED emission position.

As such, the Fourier tychographic microscope of the present specification can have high data fidelity by using an aperture function that reflects noise corresponding to each of a plurality of LEDs without assuming an ideal aperture function.

The third loss function and the fourth loss function are regulation terms, which are items for removing noise by maintaining the changing variables around a specific value at a constant value through total variation.

The third loss function is calculated through Equation 6 below.

[Equation 6]

Here, U means reconstructed image, and refers to components that exist in the spatial domain of the reconstructed image.

Additionally, the fourth loss function is calculated through Equation 7 below.

[Equation 7]

Here, U means the reconstructed image, and a and b mean a=Re(U) and b=Im(U), respectively, when U is a complex number in the form a+bi.

As such, the Fourier tychographic microscope of the present specification includes a third loss function and a fourth loss function, which are adjustment terms, in the objective function, so that the image processor 140 can obtain a high-resolution reconstructed image through noise removal.

The image restoration unit 150 obtains a visualized reconstructed image through inverse Fourier transform on the reconstructed image. The image restoration unit 150 may include a tube lens and a camera, and the user can identify the corresponding image through the visualized restored image.

The restored image can be calculated using Equation 8 below.

[Equation 8]

here, is the restored image, is the reconstructed image, means the aperture function.

As above, the Fourier tychographic microscope according to an embodiment of the present specification can remove noise and improve contrast through the third loss function and the fourth loss function.

Additionally, the Fourier tychographic microscope according to an embodiment of the present specification can have high data fidelity by using an aperture function that reflects noise corresponding to each of a plurality of LEDs without assuming an ideal aperture function.

As described above, the present invention has been described with reference to the illustrative drawings, but the present invention is not limited to the embodiments and drawings disclosed herein, and various modifications may be made by those skilled in the art within the scope of the technical idea of the present invention. It is obvious that transformation can occur. In addition, although the operational effects according to the configuration of the present invention were not explicitly described and explained while explaining the embodiments of the present invention above, it is natural that the predictable effects due to the configuration should also be recognized.

Claims (8)

a sample plane where a sample to be observed is provided;
An LED array including a plurality of LEDs that sequentially emit light and illuminate the sample at various angles;
An objective lens that projects light scattered from the sample onto a Fourier plane using an aperture function and Fourier transform and generates a plurality of observation images;
an image processor that acquires a high-resolution reconstructed image through the plurality of observation images and an objective function;
It includes an image restoration unit that obtains a visualized restored image through inverse fourier transform of the reconstructed image;
The objective function is expressed as Equation 1 below:
Fourier tychographic microscopy

[Equation 1]

here, is the objective function, is the reconstructed image, is the aperture function, is a plurality of observation images, is the first loss function, is the second loss function, is the third loss function, is the fourth loss function, , , Each represents a coefficient that determines the weight.
delete According to paragraph 1,
The image processing unit
Obtaining the reconstructed image in a direction that minimizes loss through the objective function
Fourier tychographic microscopy
delete According to paragraph 1,
The first loss function is
Fourier tychographic microscope calculated through Equation 2 below.

[Equation 2]


here, is the first loss function, is the reconstructed image, is the aperture function, is the observation image, means restored image.
According to paragraph 1,
The second loss function is
Fourier tychographic microscope calculated through Equation 3 below.

[Equation 3]


here, is the aperture function including noise, means an ideal aperture function that does not consider noise.
According to paragraph 1,
The third loss function is
Fourier tychographic microscope calculated through Equation 6 below.

[Equation 6]


here, refers to the reconstructed image, and refers to components that exist in the spatial domain of the reconstructed image.
According to paragraph 1,
The fourth loss function is
Fourier tychographic microscope calculated through Equation 7 below.

[Equation 7]


here, refers to the reconstructed image, a and b are When is a complex number in the form a+bi, it means a=Re(U) and b=Im(U), respectively.