CN112697751B - Multi-angle illumination lensless imaging method, system and device - Google Patents

Abstract

The invention discloses a multi-angle illumination lens-free imaging method, a system and a device, wherein the method comprises the following steps: s100, acquiring a plurality of diffraction patterns, wherein the diffraction patterns are images acquired by an image sensor when light waves with different illumination angles irradiate a sample surface through a diaphragm; s200, performing iterative reconstruction based on the diffraction pattern to obtain an imaging result, specifically, calculating relative displacement corresponding to each diffraction image; performing analog calculation on the imaging process based on the relative displacement to obtain light intensity data transmitted to a sample surface; calculating a current target estimation error, and updating a target sample function corresponding to a sample surface based on light intensity data and the current target estimation error; and repeating the steps until a preset iteration condition is reached, and outputting the updated target sample function as a corresponding imaging result. The invention carries out phase recovery and laminated imaging through the process of simulated imaging, and iterates, thereby improving the reconstruction resolution and optimizing the imaging result.

Description

Multi-angle illumination lensless imaging method, system and device

technical field

The invention relates to the field of lensless imaging, in particular to a multi-angle illumination lensless imaging method, system and device.

Background technique

Lensless on-chip microscopy is a new type of computational imaging technology that does not require focusing of the imaging lens, but directly attaches the target sample slide to the sensor to record the image, and combines the corresponding image restoration algorithm to achieve clear image reconstruction . Lensless microscopy has been widely accepted as an alternative and cost-effective solution for many applications, such as pathology, cell counting, etc. The main advantages of lensless microscopy are its overall compactness, simplicity, and space Scalability in terms of resolution and field of view, however, the effectiveness of lensless imaging relies on the high signal-to-noise ratio of the captured image, and the achievable spatial resolution is closely related to the measured signal-to-noise ratio, but in lensless microscopy It is difficult to guarantee the high signal-to-noise ratio condition in the optical system of .

Noise sources include photon noise, electronic noise from the image sensor, quantization noise from factor-to-analog conversion, and speckle noise from coherent light sources; in addition, some stained biological samples with high absorption and low transmittance will cause the resulting images to have Low signal-to-noise ratio; if the recorded original image is corrupted by noise, the restored image results will be degraded by the creation of artifacts. In addition, in the lensless imaging structure, because the sensor itself does not have the ability to magnify, the imaging resolution is often low. Conventional lensless holography tends to use coherent light for illumination and place the target slide as close as possible to the light source to obtain sufficient magnification. But this method has a fundamental contradiction, that is, the contradiction between the magnification and the size of the field of view. When the magnification becomes larger, the field of view becomes smaller, and vice versa. The former limits the ultimate resolution that the system can achieve, while the latter determines the range of its imaging.

SUMMARY OF THE INVENTION

Aiming at the shortcomings in the prior art, the present invention provides a multi-angle illumination lensless imaging method, system and device capable of optimizing imaging results.

In order to solve the above-mentioned technical problems, the present invention is solved by the following technical solutions:

A multi-angle illumination lensless imaging method, comprising the following steps:

S100. Acquire a number of diffraction patterns, where the diffraction patterns are images collected by an image sensor when light waves with different illumination angles are irradiated on the sample surface through a diaphragm;

S200, performing iterative reconstruction based on the diffraction pattern to obtain an imaging result, and the specific steps are:

Extracting the diffraction pattern corresponding to the light wave located at the optical axis as a reference pattern, and cross-correlating the reference pattern and each diffraction image to obtain the relative displacement;

Perform a simulation calculation on the imaging process based on the relative displacement to obtain light intensity data propagating to the sample surface;

Calculate the current target estimation error, and update the target sample function corresponding to the sample surface based on the light intensity data and the current target estimation error;

The above steps are repeated until the preset iterative condition is reached, and the updated target sample function is obtained as the corresponding imaging result and output.

As an implementation:

A corresponding plane wave is obtained based on the relative displacement simulation, and the plane wave corresponds to the light wave one-to-one;

The illumination function is obtained based on the aperture function and the plane wave calculation;

Calculate the corresponding exit wave estimate based on the illumination function and the target sample function, and obtain the first exit wave estimate;

Simulate the process of the estimated propagation of the first exit wave to the image sensor to obtain the corresponding diffraction pattern estimate;

Based on preset constraints, simulate and calculate the process of back-propagating the diffraction pattern estimate back to the sample surface to obtain a second exit wave estimate;

Calculate the current target estimation error, update the target sample function and the illumination function based on the first exit wave estimate, the second exit wave estimate and the current target estimation error; The aperture function is updated.

As an implementation:

Calculate and obtain the corresponding current target estimation error based on the second exit wave estimation, the target sample function and the illumination function;

Calculate and obtain a corresponding loss value based on the gradient of the current target estimation error relative to the target sample function;

Calculate and obtain a first update weight and a second update weight based on the loss value;

Calculate and obtain the first adaptive adjustment step size based on the first update weight and the target sample function;

Calculate and obtain a second adaptive adjustment step size based on the second update weight and the illumination function;

Update the target sample function based on the first adaptive adjustment step size, the first exit wave estimate, the second exit wave estimate and the illumination function, and obtain the updated target sample function;

The illumination function is updated based on the second adaptive adjustment step size, the first exit wave estimation, the second exit wave estimation and the target sample function to obtain an updated illumination function; and the aperture function is updated based on the updated illumination function.

As an implementation:

The objective sample function O ⁿ⁺¹ (r) obtained after the nth iteration update is:

表示第n次迭代中第j个光波对应的光照函数，

表示第n次迭代中第j个光波对应的第一出口波估计，

表示第n次迭代中第j个光波对应的第二出口波估计，*代表复共轭，a_O表示第一自适应调节步长；Among them, On (r) represents the target sample function of the ^nth iteration,

represents the illumination function corresponding to the jth light wave in the nth iteration,

represents the estimation of the first exit wave corresponding to the jth light wave in the nth iteration,

represents the estimation of the second exit wave corresponding to the jth light wave in the nth iteration, * represents the complex conjugate, and a _O represents the first adaptive adjustment step;

为：The illumination function obtained after the nth iteration update

for:

Among them, a _p represents the second adaptive adjustment step size.

As an implementation:

The first adaptive adjustment step size is:

表示第n次迭代的第一更新权重，其计算公式为：in,

Represents the first update weight of the nth iteration, and its calculation formula is:

The second adaptive adjustment step size is:

表示第n次迭代的第二更新权重，其计算公式为：in,

Represents the second update weight of the nth iteration, and its calculation formula is:

In the above formula, e ⁿ represents the loss value corresponding to the nth iteration, and η is a constant.

As an implementation:

The formula for calculating the loss value e ⁿ at the nth iteration is:

make:

The above Δε _j (On ( ^r )) is the gradient of the current target estimation error with respect to the target sample function.

As an embodiment, before the imaging process is simulated and calculated based on the relative displacement, a relative displacement correction step is also included, and the specific steps are:

Calculate the corresponding correction value based on the corresponding diffraction pattern and the diffraction pattern estimation;

The relative displacement is corrected based on the correction value, and an imaging process is simulated based on the corrected relative displacement.

As an implementation:

When the number of iterations reaches a preset number of times threshold, or when it is determined that the target sample function converges based on the loss value, outputting the obtained updated target sample function as an imaging result;

The present invention also provides a multi-angle illumination lensless imaging system, comprising:

an acquisition module, configured to acquire several diffraction patterns, the diffraction patterns are images collected by the image sensor when light waves of different illumination angles are irradiated on the sample surface through the diaphragm;

a reconstruction module, configured to perform iterative reconstruction based on the diffraction pattern to obtain an imaging result, which includes a phase recovery unit, a stacked imaging unit and an output unit;

The phase recovery unit is used to extract the diffraction pattern corresponding to the light wave located at the optical axis as a reference pattern, and to cross-correlate the reference pattern with each diffraction image to obtain a relative displacement; The imaging process is simulated and calculated to obtain the light intensity data propagating to the sample surface;

The stacked imaging unit is used to calculate the current target estimation error, and update the target sample function corresponding to the sample surface based on the light intensity data and the current target estimation error;

The reconstruction module is used for obtaining and outputting the updated target sample function as the corresponding imaging result when the preset iterative condition is reached.

The present invention also provides a multi-angle illumination lensless imaging device, comprising a multi-angle light source, a diaphragm, a target glass slide and an image sensor arranged in sequence along the optical axis; the light beam generated by the multi-angle light source passes through the light After the diaphragm is irradiated onto the target glass slide, the image sensor performs imaging acquisition to obtain several corresponding diffraction patterns.

The present invention has significant technical effects due to the adoption of the above technical solutions:

Through the design of multi-angle illumination, the present invention can obtain multiple low-resolution diffraction images at various angles, and by simulating the imaging process, phase recovery and stack imaging are performed on the obtained diffraction images to obtain corresponding imaging results. Compared with the existing technology, the imaging can be restored and optimized without adding a mechanical displacement structure;

The present invention updates the target sample function based on the current estimation error by means of an adaptive step size, which can effectively avoid the problem that the iterative process falls into a local optimal solution.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a schematic structural diagram of a multi-angle illumination lensless imaging device of the present invention;

Figure 2 is a schematic diagram of the phase of the imaging result, wherein the left picture is a schematic phase diagram of the imaging result obtained by the first iteration, and the right picture is a schematic phase diagram of the imaging result obtained by the 15th iteration;

Figure 3 is a schematic diagram of the amplitude of the imaging results, wherein the left picture is a schematic diagram of the amplitude of the imaging results obtained in the first iteration, and the right picture is a schematic diagram of the amplitude of the imaging results obtained in the 15th iteration.

In the picture:

1 is a multi-angle light source, 2 is a diffuser, 3 is a diaphragm, 4 is a target slide, and 5 is an image sensor.

Detailed ways

The present invention will be further described in detail below in conjunction with the examples. The following examples are to explain the present invention and the present invention is not limited to the following examples.

Due to the wave-particle duality of light, when the light emitted by the light source with weak coherence passes through the diaphragm 3, the phenomenon that the light deviates from the straight line will appear, which is called light diffraction. According to the different propagation distances of diffracted light waves, it can be divided into three regions from near to original, namely, the Rayleigh-Sommerfield diffraction region, the Fresnel diffraction region and the Fraunhofer diffraction region. The diffraction formula of the corresponding area is selected by yourself to express the complex amplitude of the corresponding light wave.

Embodiment 1. A multi-angle illumination lensless imaging device, comprising a multi-angle light source 1, a diaphragm 3, a target glass slide 4 and an image sensor 5 arranged in sequence along the optical axis;

The distance d0 between the multi-angle light-emitting source 1 and the diaphragm 3 is 100-600 mm, and the distance d1 between the diaphragm 3 and the target glass slide 4 and the distance d2 between the target glass slide 4 and the image sensor 5 can be much smaller than d0.

The multi-angle light-emitting source 1 is used to generate a low-coherence light beam, so that the image sensor 5 can obtain multiple low-resolution diffraction patterns, and the angles corresponding to each diffraction pattern are different, so as to facilitate subsequent reconstruction based on the collected diffraction patterns. Optimized imaging results;

In this embodiment, the multi-angle light source 1 uses an LED matrix, and the number of diffraction patterns obtained is consistent with the number of LEDs in the LED matrix. In theory, the more diffraction patterns used in the reconstruction process, the better, but with the increase of the number of LEDs , the measurement angle of the light beam generated by the peripheral LED to the target sample is relatively large, which will lead to poor imaging effect. Therefore, those skilled in the art can set up a suitable LED matrix according to actual needs, such as 5×5～10×10. For the LED matrix, the spacing of the LEDs may be, for example, 2 cm to 5 cm.

The diaphragm 3 is used to filter out stray light and generate a circular light wave, which is irradiated to the target glass slide 4 containing the target sample, and imaged and collected by the image sensor 5, and the light is used in the reconstruction process. The function represents this light wave.

In this embodiment, the radius of the micro-holes of the diaphragm 3 is 100-300um.

The image sensor 5 may be, for example, a charge-coupled device (CCD, CCD) or a complementary metal semiconductor oxide (Complementary Metal Oxide Semiconductor, CMOS).

Further, a diffuser 2 is provided between the multi-angle light source 1 and the diaphragm 3;

When the light beam emitted by the multi-angle light source 1 passes through the diffuser 2, a series of light waves at random angles are introduced into the light beam by the diffuser 2. By adding the diffuser 2 in this embodiment, the image quality can be further improved. Imaging effect.

Case:

A multi-angle illumination lensless imaging device, comprising a multi-angle light source 1, a diffuser 2, a diaphragm 3, a target glass slide 4 and an image sensor 5 arranged in sequence along the optical axis;

The distance d0 between the multi-angle light source 1 and the diaphragm 3 is 300 mm, the distance d1 between the diaphragm 3 and the target glass slide 4 is 5 mm, and the distance d2 between the target glass slide 4 and the image sensor 5 is 0.5 mm.

The multi-angle light source 1 is a 10×10 LED matrix, wherein the spacing between the LEDs is 3.5 cm.

The radius of the aperture of aperture 3 is 150um;

The image sensor 5 uses a charge-coupled element;

The distance d0 between the multi-angle light-emitting source 1 and the diaphragm 3 is d0, the distance d1 between the diaphragm 3 and the target glass slide 4 is , and the distance d2 between the target glass slide 4 and the image sensor 5 is .

After the light beam emitted by each LED passes through the diffuser 2, the stray light is filtered by the diaphragm 3 to form a circular light wave; the light wave is irradiated on the target glass slide 4 containing the target sample, and is transmitted to the image sensor 5. Image acquisition, 100 diffraction patterns were obtained, and the diffraction patterns were in one-to-one correspondence with the LEDs.

Aiming at the contradiction between the field of view and imaging resolution, Feng et al. used Ronchi grating to realize Talbot grating illumination, and the dark fringe light intensity of the grating on the object surface was 0 as the support domain of the object surface. The reconstructed light intensity is restored, but this method has high requirements on the sample and limited reconstruction effect. The Ozcan laboratory proposed a phase recovery algorithm with multiple sample-sensor distances, but it has high hardware requirements, requires a mechanical displacement structure, and its iterative algorithm is easy to fall into a local optimal solution.

In this embodiment, through the design of a multi-angle illumination lensless imaging device, a set of low-resolution diffraction patterns can be obtained through multi-angle illumination, which is convenient for subsequent layered imaging based on the diffraction patterns, improves the quality of imaging results, and also improves the quality of imaging results. The contradiction between the field of view range and the resolution in the existing lensless microscopic imaging technology can be solved without the need of a mechanical displacement structure.

Embodiment 2, a multi-angle illumination lensless imaging method, comprising the following steps:

S100, acquiring several diffraction patterns, the diffraction patterns are images collected by the image sensor 5 when light waves of different illumination angles are irradiated on the sample surface through the diaphragm 3;

The diffraction pattern in this embodiment is the image captured by the lensless imaging device described in Embodiment 1.

S200. Perform iterative reconstruction based on the diffraction pattern to obtain an imaging result.

The specific steps are:

S210, extracting the diffraction pattern corresponding to the light wave located at the optical axis as a reference pattern, and cross-correlating the reference pattern and each diffraction image to obtain a relative displacement;

The optical axis is the optical axis of the lensless imaging device.

S220. Perform a simulation calculation on the imaging process based on the relative displacement to obtain light intensity data propagated to the sample surface;

S230, calculating the current target estimation error, and updating the target sample function corresponding to the sample surface based on the light intensity data and the current target estimation error;

The target sample function is a pixel matrix representing the imaging result, and this step is to perform stacked imaging based on the light intensity data to update the imaging result.

S240. Repeat the above steps S220 and S230 until the preset iteration condition is reached, and obtain and output the updated target sample function as a corresponding imaging result.

The reconstruction method based on the diffraction image in this embodiment includes two parts: phase recovery and stack imaging, wherein the phase recovery is the process of simulating the imaging through the relative displacement obtained by calculation, and based on preset constraints, the propagation to the sample surface is analyzed. The light intensity information is updated. Stacked imaging superimposes each updated diffraction pattern. Because the LEDs are in different positions, each diffraction pattern is partially overlapped and superimposed to form a complete image.

In this embodiment, multiple low-resolution diffraction patterns are obtained at various angles, and the target sample is reconstructed by simulating illumination light from different angles to illuminate the target sample in combination with overlapping apertures, thereby effectively improving the resolution of reconstruction. In this embodiment, based on the current target The estimation error automatically adjusts the step size, which can improve the resolution of reconstruction and further improve the convergence speed. In this embodiment, the current target estimation error is a global error, which can effectively avoid falling into a local optimal solution in the iterative process.

In step S210, the diffraction pattern measured by the LED corresponding to the central optical axis of the lensless imaging device is selected as the reference pattern, and it is cross-correlated with other diffraction patterns, and the relative displacement of the corresponding diffraction pattern, this embodiment The relative displacement Δd _j between the diffraction pattern produced by the jth LED and the reference pattern is:

Among them, d represents the coordinates of the diffraction plane, j is the coordinate position of the LED array (representing the position of the diffraction pattern generated by the j-th LED), I _ref (d) represents the reference pattern, and I _j (d+Δd) represents the j-th Diffraction patterns produced by LEDs.

Step S220 is used to realize phase recovery, and the specific steps are:

S221. Obtain a corresponding plane wave based on the relative displacement simulation, and the plane wave corresponds to the light wave one-to-one;

S222, calculating and obtaining an illumination function based on the aperture function and the plane wave;

S223, calculating and obtaining a corresponding exit wave estimate based on the illumination function and the target sample function, and obtaining a first exit wave estimate;

S224, performing simulation calculation on the estimated propagation process of the first exit wave to the image sensor 5 to obtain a corresponding diffraction pattern estimate;

S225. Perform a simulation calculation on the process of back-propagating the diffraction pattern estimate back to the sample surface based on a preset constraint condition, to obtain a second exit wave estimate;

The light intensity data includes the first exit wave estimate and the second exit estimate obtained by the above calculation. In this embodiment, the imaging process is simulated and calculated by relative displacement to obtain the first exit wave estimate, and then based on preset constraints. The condition updates the first exit wave estimate to obtain a second exit wave estimate.

In step S221, the plane wave corresponding to the LED is simulated based on the relative displacement, and the plane wave Γ _j (r) corresponding to the jth LED is:

Among them, r is the coordinate position of the LED in the real space, M is the pixel value of the entire window, and Δd _j is the relative displacement corresponding to the jth LED.

In step S222, the process of propagating the plane wave to the diaphragm 3 and then irradiating it to the sample surface is simulated to obtain the corresponding illumination function; the illumination function corresponding to the jth LED at the nth iteration is:

表示光阑3到样本面的传播函数，即，光阑3中微孔到目标载玻片4的传播函数；Aⁿ(r)表示第n次迭代的光阑函数；Among them, n represents the number of iterations;

Represents the propagation function from diaphragm 3 to the sample surface, that is, the propagation function from the aperture in diaphragm 3 to the target glass slide 4; An (r) represents the diaphragm function of the ^nth iteration;

The aperture function in this embodiment represents a rough estimation of the diffuser 2 and aperture 3. In the first iteration, the aperture function adopts a preset initial aperture function, and in this embodiment, the initial aperture function adopts the same diffraction pattern as the diffraction pattern. All zero values with the same size, otherwise the diaphragm function updated in the last iteration is used, and the update method is as described in step S238.

采用角谱传播，具体过程为：The propagation function in this embodiment

Using angular spectrum propagation, the specific process is:

Input=F(Γ _j (r)A ⁿ (r))

Output=Input*CTF

Among them, F is the Fourier change, F ^-1 is the inverse Fourier change, Input is the function of the plane wave to the frequency domain through Fourier transform, and the frequency is obtained by multiplying the coherent transfer function (CTF) with the coherent transfer function (CTF). Lighting function for sample faces within the domain.

为：In step S223, through the product of the illumination function and the target sample function, the exit wave estimate when leaving the target sample is calculated and obtained, the corresponding first exit wave estimate is obtained, and the first exit wave estimate corresponding to the jth LED during the nth iteration is estimated.

for:

Wherein, On ( ^r ) represents the target sample function, which is the target sample function after the last iterative reconstruction is updated, and in the first iteration, the target sample function is the preset initial target sample function.

In step S224, the process of propagating the first exit wave estimate to the image sensor 5 for imaging is simulated to obtain a corresponding diffraction image estimate;

为：In this embodiment, the diffraction image estimation is calculated based on the angular spectrum propagation, and the diffraction image estimation corresponding to the jth LED in the nth iteration

for:

Among them, F is the Fourier transform, h is the distance from the sample surface to the image sensor 5, that is, the distance (d2) from the target glass slide 4 to the image sensor 5, λ is the wavelength of light, and r is the coordinates of the LED in the real space Location.

为：In step S225, the obtained diffraction image estimate is back-propagated back to the sample surface using preset constraints, and the first exit wave estimate is updated to obtain a corresponding second exit wave estimate, and the jth LED in the nth iteration The corresponding second exit wave estimate

for:

The preset constraint conditions in this embodiment are the constraint of angular spectrum propagation and the constraint of the initially acquired diffraction pattern.

It can be seen from the above that in this embodiment, the phase is recovered by switching multiple times in the spatial domain and the frequency domain. This method can not only reconstruct the complex amplitude distribution of the object on the focal plane, but also further improve the reconstruction resolution of the object.

Further, before performing the simulation calculation on the imaging process based on the relative displacement, the step further includes a relative displacement correction step, and the specific steps are:

为：Based on the corresponding diffraction pattern and diffraction pattern estimation, the corresponding correction value is calculated and obtained; the correction value corresponding to the jth LED at the nth iteration

for:

第n次迭代时d位置的衍射图样估计。Among them, I _j (d) represents the diffraction image corresponding to the jth LED,

Diffraction pattern estimation at position d at the nth iteration.

为。The relative displacement is corrected based on the correction value, the imaging process is simulated based on the corrected relative displacement, and the relative displacement of the simulated imaging is performed at the n+1th iteration

for.

That is, a corresponding correction value is generated based on the diffraction pattern estimation in the current iteration, and the relative displacement is updated based on the obtained correction value, so that the updated relative displacement is used for imaging simulation in the next iteration, and the initial relative displacement is calculated in step S210. The relative displacement Δd _j .

Because the correlation degree of the relative displacement Δd _j obtained by the previous calculation is low, and because of the low coherence condition of the LED, the diffraction pattern is smoothed, resulting in a certain error. Therefore, in this embodiment, a mutual correlation is established by designing the correction value. to improve accuracy.

Step S230 is the step of stack imaging. In this embodiment, the current target error is based on the global error, and the step size is automatically adjusted by the current target estimation error, which can effectively avoid the problem that the iterative process falls into the global optimal solution.

Specific steps are as follows:

S231, calculating and obtaining the corresponding current target estimation error ε based on the second exit wave estimation, the target sample function and the illumination function;

The error function is:

表示当前迭代中第j个LED所对应的第二出口波估计，

表示当前迭代中第j个LED所对应的光照函数，Oⁿ(r)表示当前迭代中的目标样本函数(即，上一次迭代中更新后的目标样本函数)。in,

represents the second exit wave estimate corresponding to the jth LED in the current iteration,

represents the illumination function corresponding to the jth LED in the current iteration, and On ( ^r ) represents the target sample function in the current iteration (ie, the updated target sample function in the previous iteration).

S232, calculating and obtaining a corresponding loss value based on the gradient of the current target estimation error relative to the target sample function;

The formula for calculating the loss value ^en at the current iteration is:

Among them, Δε _j (On ( ^r )) is the gradient of the current target estimation error relative to the target sample function, which can be expressed as:

In the above formula, * represents complex conjugate.

S233, calculating and obtaining a first update weight and a second update weight based on the loss value;

的计算公式为：The first update weights at the current iteration

The calculation formula is:

Wherein, n is a constant, and those skilled in the art can set the value of n according to actual needs, and the value of n in this embodiment is set to 0.0001;

That is, when the calculated loss value is close to convergence, the step size is adaptively attenuated.

的计算公式为：Same as above, the first update weight in the current iteration

The calculation formula is:

During the first iteration, the first update weight and the second update weight are both preset initial values, and those skilled in the art can set the initial values according to actual needs. In this embodiment, the first update weight and the second update weight are The initial value of is 0.001.

S234, calculating and obtaining the first adaptive adjustment step size based on the first update weight and the target sample function;

The first adaptive adjustment step size a _O in the current iteration is:

S235, calculating and obtaining a second adaptive adjustment step size based on the second update weight and the illumination function;

The second adaptive adjustment step size a _p in the current iteration is:

S236, update the target sample function based on the first adaptive adjustment step size, the first exit wave estimation, the second exit wave estimation and the illumination function, and obtain the updated target sample function;

The target sample function O ⁿ⁺¹ (r) obtained after the current iteration (ie the nth iteration) is updated is:

表示第n次迭代中第j条光波(即，第j个LED)所对应的光照函数，

表示第n次迭代中第j条光波(即，第j个LED)对应的第一出口波估计，

表示第n次迭代中第j个光波(即，第j个LED)对应的第二出口波估计，*代表复共轭，a_o表示第一自适应调节步长。Among them, On (r) represents the target sample function of the ^nth iteration,

represents the illumination function corresponding to the jth light wave (ie, the jth LED) in the nth iteration,

represents the first exit wave estimate corresponding to the jth light wave (ie, the jth LED) in the nth iteration,

represents the second exit wave estimate corresponding to the jth light wave (ie, the jth LED) in the nth iteration, * represents the complex conjugate, and a _o represents the first adaptive adjustment step size.

It can be seen from the above that the process of updating the target sample function is the process of stacking imaging based on the updated diffraction Yuyang.

S237, update the illumination function based on the second adaptive adjustment step size, the first exit wave estimation, the second exit wave estimation and the target sample function, and obtain the updated illumination function;

为：The lighting function obtained after the current iteration (ie the nth iteration) is updated

for:

S238. Update the aperture function based on the updated illumination function.

In this embodiment, using the updated illumination function, after removing the inclined plane wave, the aperture function is updated, so that the corresponding illumination function is obtained by calculating based on the updated aperture function in the next iteration;

The updated aperture function A ⁿ⁺¹ (r) of the current iteration (ie the nth iteration) is:

Those skilled in the art can set the iteration conditions described in step S240 by themselves according to actual needs, for example:

When the number of iterations reaches a preset number of times threshold, or when it is determined that the target sample function converges based on the loss value, it is determined that the iteration is terminated, and the obtained updated target sample function is output as an imaging result.

For the above-mentioned calculation method of the loss value, those skilled in the art can set the conditions for determining the convergence of the target sample function based on the loss value by themselves according to actual needs, so it is not limited.

Based on the method proposed in this embodiment, the corresponding diffraction pattern is iteratively reconstructed. The reconstruction results are shown in Figures 2 and 3. From the comparison of Figures 2 and 3, it can be seen that the resolution of the output imaging results after 15 iterations Much higher than the imaging results output after one iteration, it can be proved that the imaging method proposed in this embodiment can effectively improve the resolution and solve the problems of poor imaging results and visual problems in the existing lensless microscopic imaging technology. Conflict between field range and imaging resolution.

Embodiment 3, a multi-angle illumination lensless imaging system, comprising:

an acquisition module, configured to acquire several diffraction patterns, the diffraction patterns are images collected by the image sensor 5 when light waves of different illumination angles are irradiated on the sample surface through the diaphragm 3;

a reconstruction module, configured to perform iterative reconstruction based on the diffraction pattern to obtain an imaging result, which includes a phase recovery unit, a stacked imaging unit and an output unit;

The phase recovery unit is used to extract the diffraction pattern corresponding to the light wave located at the optical axis as a reference pattern, and to cross-correlate the reference pattern with each diffraction image to obtain a relative displacement; The imaging process is simulated and calculated to obtain the light intensity data propagating to the sample surface;

The stacked imaging unit is used to calculate the current target estimation error, and update the target sample function corresponding to the sample surface based on the light intensity data and the current target estimation error;

The output unit is configured to acquire and output the updated target sample function as a corresponding imaging result when a preset iteration condition is reached.

This embodiment is a product embodiment corresponding to Embodiment 2. Since it is basically similar to Method Embodiment 2, the description is relatively simple. For relevant parts, please refer to the partial description of Method Embodiment 2.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments may be referred to each other.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to the invention. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing terminal equipment to produce a machine that causes the instructions to be executed by the processor of the computer or other programmable data processing terminal equipment Means are created for implementing the functions specified in a flow or flows of the flowcharts and/or a block or blocks of the block diagrams.

These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal equipment to operate in a particular manner, such that the instructions stored in the computer readable memory result in an article of manufacture comprising instruction means, the The instruction means implement the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing terminal equipment, so that a series of operational steps are performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby executing on the computer or other programmable terminal equipment The instructions executed on the above provide steps for implementing the functions specified in the flowchart or blocks and/or the block or blocks of the block diagrams.

It should be noted:

Reference in the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases "one embodiment" or "an embodiment" in various places throughout the specification are not necessarily all referring to the same embodiment.

Although preferred embodiments of the present invention have been described, additional changes and modifications to these embodiments may occur to those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be construed to include the preferred embodiment and all changes and modifications that fall within the scope of the present invention.

In addition, it should be noted that, in the specific embodiments described in this specification, the shapes and names of parts and components thereof may be different. All equivalent or simple changes made according to the structures, features and principles described in the patent concept of the present invention are included in the protection scope of the patent of the present invention. Those skilled in the art to which the present invention pertains can make various modifications or additions to the described specific embodiments or substitute in similar manners, as long as they do not deviate from the structure of the present invention or go beyond the scope defined by the claims, All should belong to the protection scope of the present invention.

Claims (9)

1. a multi-angle illumination lensless imaging method, is characterized in that, comprises the following steps: S100. Acquire a number of diffraction patterns, where the diffraction patterns are images collected by an image sensor when light waves with different illumination angles are irradiated on the sample surface through a diaphragm; S200, performing iterative reconstruction based on the diffraction pattern to obtain an imaging result, and the specific steps are: S210, extracting the diffraction pattern corresponding to the light wave located at the optical axis as a reference pattern, and cross-correlating the reference pattern and each diffraction image to obtain a relative displacement; S220. Perform a simulation calculation on the imaging process based on the relative displacement to obtain light intensity data propagated to the sample surface; S230, calculating the current target estimation error, and updating the target sample function corresponding to the sample surface based on the light intensity data and the current target estimation error; S230: Repeat the above steps S220 and S230 until the preset iteration condition is reached, and obtain and output the updated target sample function as a corresponding imaging result. 2. The multi-angle illumination lensless imaging method according to claim 1, wherein: A corresponding plane wave is obtained based on the relative displacement simulation, and the plane wave corresponds to the light wave one-to-one; The illumination function is obtained based on the aperture function and the plane wave calculation; Calculate the corresponding exit wave estimate based on the illumination function and the target sample function, and obtain the first exit wave estimate; Simulate the process of the estimated propagation of the first exit wave to the image sensor to obtain the corresponding diffraction pattern estimate; Based on preset constraints, simulate and calculate the process of back-propagating the diffraction pattern estimate back to the sample surface to obtain a second exit wave estimate; Calculate the current target estimation error, update the target sample function and the illumination function based on the first exit wave estimate, the second exit wave estimate and the current target estimation error; The aperture function is updated. 3. The multi-angle illumination lensless imaging method according to claim 2, wherein: Calculate and obtain the corresponding current target estimation error based on the second exit wave estimation, the target sample function and the illumination function; Calculate and obtain a corresponding loss value based on the gradient of the current target estimation error relative to the target sample function; Calculate and obtain a first update weight and a second update weight based on the loss value; Calculate and obtain the first adaptive adjustment step size based on the first update weight and the target sample function; Calculate and obtain a second adaptive adjustment step size based on the second update weight and the illumination function; Update the target sample function based on the first adaptive adjustment step size, the first exit wave estimate, the second exit wave estimate and the illumination function, and obtain the updated target sample function; The illumination function is updated based on the second adaptive adjustment step size, the first exit wave estimation, the second exit wave estimation and the target sample function to obtain an updated illumination function; and the aperture function is updated based on the updated illumination function. 4. The multi-angle illumination lensless imaging method according to claim 3, wherein: The objective sample function O ⁿ⁺¹ (r) obtained after the nth iteration update is:

Among them, On (r) represents the target sample function of the ^nth iteration,

represents the illumination function corresponding to the jth light wave in the nth iteration,

represents the estimation of the first exit wave corresponding to the jth light wave in the nth iteration,

represents the estimation of the second exit wave corresponding to the jth light wave in the nth iteration, * represents the complex conjugate, and a _O represents the first adaptive adjustment step; The illumination function obtained after the nth iteration update

for:

Among them, a _p represents the second adaptive adjustment step size. 5. The multi-angle illumination lensless imaging method according to claim 4, wherein: The first adaptive adjustment step size is:

in,

Represents the first update weight of the nth iteration, and its calculation formula is:

The second adaptive adjustment step size is:

in,

Represents the second update weight of the nth iteration, and its calculation formula is:

In the above formula, e ⁿ represents the loss value corresponding to the nth iteration, and η is a constant. 6. The multi-angle illumination lensless imaging method according to claim 5, wherein: The formula for calculating the loss value e ⁿ at the nth iteration is:

make:

The above Δε _j (On ( ^r )) is the gradient of the current target estimation error with respect to the target sample function. 7. The multi-angle illumination lensless imaging method according to any one of claims 2 to 6, characterized in that, before the imaging process is simulated and calculated based on the relative displacement, a relative displacement correction step is also included, and the specific steps are: Calculate the corresponding correction value based on the corresponding diffraction pattern and the diffraction pattern estimation; The relative displacement is corrected based on the correction value, and an imaging process is simulated based on the corrected relative displacement. 8. The multi-angle illumination lensless imaging method according to any one of claims 3 to 6, wherein: When the number of iterations reaches a preset number threshold, or when the target sample function is determined to converge based on the loss value, the obtained updated target sample function is output as an imaging result. 9. A multi-angle illumination lensless imaging system, comprising: an acquisition module, configured to acquire several diffraction patterns, the diffraction patterns are images collected by the image sensor when light waves with different illumination angles are irradiated on the sample surface through the diaphragm; a reconstruction module, configured to perform iterative reconstruction based on the diffraction pattern to obtain an imaging result, which includes a phase recovery unit, a stacked imaging unit and an output unit; The phase recovery unit is used to extract the diffraction pattern corresponding to the light wave located at the optical axis as a reference pattern, and to cross-correlate the reference pattern with each diffraction image to obtain a relative displacement; The imaging process is simulated and calculated to obtain the light intensity data propagating to the sample surface; The stacked imaging unit is used to calculate the current target estimation error, and update the target sample function corresponding to the sample surface based on the light intensity data and the current target estimation error; The output unit is configured to acquire and output the updated target sample function as a corresponding imaging result when a preset iteration condition is reached.

Images