CN118071866A - Sparse digital holographic image reconstruction method - Google Patents
Sparse digital holographic image reconstruction method Download PDFInfo
- Publication number
- CN118071866A CN118071866A CN202410464769.2A CN202410464769A CN118071866A CN 118071866 A CN118071866 A CN 118071866A CN 202410464769 A CN202410464769 A CN 202410464769A CN 118071866 A CN118071866 A CN 118071866A
- Authority
- CN
- China
- Prior art keywords
- matrix
- image
- amplitude
- phase
- noise
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 78
- 239000011159 matrix material Substances 0.000 claims abstract description 129
- 238000010521 absorption reaction Methods 0.000 claims abstract description 33
- 238000003062 neural network model Methods 0.000 claims abstract description 33
- 238000005070 sampling Methods 0.000 claims abstract description 33
- 230000008569 process Effects 0.000 claims abstract description 22
- 238000012549 training Methods 0.000 claims abstract description 15
- 238000009792 diffusion process Methods 0.000 claims description 33
- 230000006870 function Effects 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 3
- 238000004590 computer program Methods 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims description 2
- 230000000644 propagated effect Effects 0.000 claims 1
- 238000004422 calculation algorithm Methods 0.000 abstract description 5
- 230000006872 improvement Effects 0.000 abstract description 3
- 230000000295 complement effect Effects 0.000 abstract 1
- 238000003384 imaging method Methods 0.000 description 7
- 230000007423 decrease Effects 0.000 description 5
- 238000003491 array Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000009977 dual effect Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001093 holography Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Holo Graphy (AREA)
Abstract
The invention discloses a sparse digital holographic image reconstruction method, which specifically comprises the following steps: training the U-net neural network model to obtain a trained U-net neural network model; acquiring an undersampled hologram by using a sparse sensor array as a sparse sampling hologram; the sparse sampling hologram is transmitted backwards to obtain an object image of an amplitude and phase dual-channel matrix, and absorption constraint and support constraint are respectively implemented on the amplitude matrix and absorption constraint is implemented on the phase matrix; the rotation iteration between the spatial domain and the holographic domain of the present invention can mutually complement each other's information, and constraints based on the target energy distribution and absorption characteristics are imposed on the spatial domain image to obtain a high quality image. In addition, the hologram after fidelity is sent to the next iteration after each iteration, the amplitude and the phase of the reconstructed image are accurately recovered through a back propagation algorithm, and the gradual improvement of the image quality in the continuous iteration process is ensured.
Description
Technical Field
The invention relates to the technical field of image reconstruction, in particular to a sparse digital holographic image reconstruction method.
Background
Digital holographic imaging (DH), a technique that uses a digital camera to capture holograms instead of conventional optical recording materials and uses numerical methods to reconstruct the amplitude and phase information of the light field emitted by an object. Digital holographic imaging has received considerable attention due to its importance in three-dimensional identification, microscopic imaging, surface feature extraction, and other applications.
Existing digital holographic imaging techniques face a number of drawbacks in handling image reconstruction under extremely sparse sensor arrays:
1. imaging accuracy and quality are limited: in the case of using a single small aperture sensor or long distance imaging scene, the prior art cannot adequately capture the complete information of the holographic field, resulting in limited accuracy and quality of the reconstructed image.
2. Sensitive to initial conditions: such as a self-recovery method (SRSAA) of a sparse aperture array, is aimed at incrementally recovering the information lost in the sensor gap. Although SRSAA provides acceptable image reconstruction quality, the method is sensitive to the selection of initial conditions, is easy to fall into local optimum due to lack of full extraction and utilization of prior information of target light field distribution, and meanwhile, the quality of a reconstructed image is limited by the performance of a sensor, and although the method provides certain image reconstruction quality, the method is sensitive to the selection of initial conditions, is easy to fall into local optimum solutions, and limits the reliability of image reconstruction.
3. The prior information is underutilized: the prior method can not fully extract and utilize prior information of the target light field distribution, so that the information is underutilized in the reconstruction process, and the effect and efficiency of image reconstruction are further affected.
Disclosure of Invention
The invention aims to solve the problem of imaging precision of a digital holographic technology under the condition of an extremely sparse sensor array. In the prior art, the quality of the reconstructed image is limited, especially in the case of a limited number of sensors, because the acquisition range of the detector cannot cover the entire holographic field. The invention provides a solution for high-quality image reconstruction by efficiently utilizing sparse sensor data, and overcomes the quality limitation of the prior art when reconstructing images under a highly sparse sensor array.
According to a first aspect of the present invention, there is provided a sparse digital holographic image reconstruction method, comprising the steps of:
Training the U-net neural network model to obtain a trained U-net neural network model;
acquiring an undersampled hologram by using a sparse sensor array as a sparse sampling hologram;
The sparse sampling hologram is transmitted backwards to obtain an object image of an amplitude and phase dual-channel matrix, and absorption constraint and support constraint are respectively implemented on the amplitude matrix and absorption constraint is implemented on the phase matrix;
Loading an object image which implements absorption constraint and support constraint on an amplitude matrix and implements absorption constraint on a phase matrix into a diffusion model, adding random noise of a preset time step to the constrained amplitude matrix by the diffusion model to obtain a pure noise amplitude matrix, adding random noise of the preset time step to the constrained phase matrix to obtain a pure noise phase matrix, and denoising the pure noise amplitude matrix and the pure noise phase matrix by using a trained U-net neural network model to reconstruct to obtain an object image with two channels of amplitude and phase;
The object image with the two channels of amplitude and phase obtained by reconstruction is transmitted forward to obtain a hologram, and the hologram is subjected to fidelity operation based on the sparse sampling hologram so as to obtain a fidelity image;
Returning the image after fidelity to the step of carrying out backward propagation on the sparse sampling hologram to obtain an object image of an amplitude and phase dual-channel matrix, and respectively implementing absorption constraint and support constraint on the amplitude matrix and implementing absorption constraint on the phase matrix, and iterating until the number of iterations reaches the preset iteration number;
and (5) carrying out backward propagation on the image after the fidelity, recovering to obtain a reconstructed image and outputting the reconstructed image.
Further, the step of respectively implementing absorption constraint and support constraint on the amplitude matrix and implementing absorption constraint on the phase matrix specifically includes:
Firstly implementing absorption constraint on the amplitude matrix, and then implementing support constraint, wherein the support constraint is used for assigning 0 to the pixel value outside the support area, and the pixel value of the support area is not changed; the absorption constraint is implemented on the phase matrix, and the specific constraint formula is as follows:
(x,y)=/>
In the method, in the process of the invention, (X, y) represents the pixel value of the coordinates (x, y) in the amplitude matrix; p represents outside the support region; /(I)(X, y) represents the pixel value of the coordinates (x, y) in the phase matrix, and the supporting area is the area where the foreground is located.
Further, the step of training the U-net neural network model to obtain a trained U-net neural network model specifically includes:
Acquiring an image dataset;
randomly adding noise into an original image corresponding to the image dataset to obtain a noise image;
Inputting the amplitude and phase dual-channel matrix based on the noise image into a U-net neural network model to obtain a reconstructed image;
constructing a fractional function based on the deviation of the data distribution of the pixel values of the amplitude and phase matrix of the reconstructed image and the data distribution of the pixel values of the amplitude and phase matrix of the original image;
And continuously training the U-net neural network model until the U-net neural network model converges to obtain a target score function.
The further scheme is that the step of denoising the pure noise amplitude matrix and the pure noise phase matrix by using the trained U-net neural network model to reconstruct an object image with two channels of amplitude and phase specifically comprises the following steps:
denoising the pure noise amplitude matrix and the pure noise phase matrix based on the target scoring function;
Sequentially calculating a noise matrix corresponding to the previous moment from a pure Gaussian amplitude matrix and a pure noise phase matrix respectively until an amplitude matrix and a phase matrix corresponding to the moment 0 are calculated;
reconstructing an object image based on the amplitude matrix and the phase matrix corresponding to the moment 0;
wherein, (/>)/>；
(/>)/>；
Where t=t-1, T-2,..1, 0For training a target score function obtained by a U-net neural network model, when a noise matrix corresponding to a t-th time step is calculated, the noise amplitude matrix/>, corresponding to the t+1 time step, which is obtained by calculation is calculatedAnd noise phase matrix/>Respectively input into the objective score function,/>And/>Respectively carrying out noise scheduling corresponding to a t time step and a t+1 time step in the forward process; /(I)Representing standard normalization.
The further scheme is that the calculation formula of the step of obtaining the image after fidelity is as follows:
；
Wherein the method comprises the steps of Representing a Fidelity image, W representing a sparse sampling matrix associated with sensor array arrangement,/>Representing fourier transform forward propagation, O represents reconstructed amplitude and phase dual-channel object images, and M represents a sparse sampled hologram.
The method comprises the steps of performing backward propagation on a sparse sampling hologram to obtain a sparse sampling hologram, and performing inverse Fourier transform on the sparse sampling hologram;
The hologram obtained by forward propagation of the reconstructed amplitude and phase dual-channel object image is obtained by forward propagation of the reconstructed amplitude and phase dual-channel object image through Fourier transformation.
According to a second aspect of the present invention, there is provided an electronic device comprising: a memory and a processor;
The memory is used for storing programs;
The processor is configured to invoke a program stored in the memory to perform a sparse digital holographic image reconstruction method as set forth in any preceding claim.
According to a third aspect of the present invention there is provided a readable storage medium having stored thereon a computer program which, when executed by a processor, implements a sparse digital holographic image reconstruction method as defined in any of the above.
Compared with the prior art, the invention has the beneficial effects that: the invention provides a sparse digital holographic image reconstruction method. The method mainly learns complex amplitude priori information of a target light field from an amplitude and phase dual-channel image dataset, and supports an iterative reconstruction process of an image by utilizing the complex amplitude priori information. The space domain is obtained by backward propagation of the hologram after fidelity, the holographic domain is obtained by forward propagation of the amplitude and the phase after reconstruction of the diffusion model, and the alternate iteration between the space domain and the holographic domain can mutually supplement the mutual information. The absorption constraint and the support constraint are applied to the amplitude matrix and the absorption constraint is applied to the phase matrix to obtain a high-quality image, so that the influence of noise outside a foreground area on the image quality can be removed, and pixel values which do not accord with a physical principle are removed. In addition, the hologram after fidelity is sent to the object image after each iteration and is subjected to backward propagation, the next iteration is carried out in the step of obtaining the object image of the amplitude and phase dual-channel matrix, the amplitude and phase of the reconstructed image are accurately recovered through a backward propagation algorithm, gradual improvement of the image quality in the continuous iteration process is ensured, in the fidelity process, the pixel value obtained by sparse sampling of the sensor is replaced to the pixel value of the corresponding position of the object image obtained by reconstruction, and the image after fidelity is close to the sparse sampling hologram sampled by the sensor array, so that the fidelity is ensured.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a sparse digital holographic image reconstruction method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a sensor array with different distance gaps according to an embodiment of the present invention;
FIG. 3 is a graph showing the comparison of the reconstruction results of the method of the present invention and SRSAA methods under different gaps provided by the examples of the present invention;
FIG. 4 is a schematic diagram of a distribution of different numbers of sensor arrays according to an embodiment of the present invention;
FIG. 5 is a graph comparing the reconstruction results of the method of the present invention and SRSAA methods under different numbers of sensor arrays provided in the examples of the present invention.
Detailed Description
In order that the objects, features and advantages of the invention will be readily understood, a more particular description of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings.
It will be understood that when an element is referred to as being "fixed to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.
Referring to fig. 1, the present invention provides a sparse digital holographic image reconstruction method, which specifically includes the following steps:
S1, training a U-net neural network model to obtain a trained U-net neural network model;
specifically, an MNIST image data set is obtained, noise is randomly added to an original image corresponding to the image data set, and a noise image is obtained; and training a U-net neural network model based on the two-channel matrix of the amplitude and the phase of the noise image by taking the noise image as input, continuously iterating to predict and obtain a reconstructed image for eliminating noise in the training process of the U-net neural network model, constructing a fractional function based on the data distribution of the pixel values of the amplitude and the phase matrix of the reconstructed image and the deviation of the data distribution of the pixel values of the amplitude and the phase matrix of the original image, and continuously training the U-net neural network model until the U-net neural network model converges, wherein the obtained U-net neural network model is a trained U-net neural network model, and the obtained fractional function is a target fractional function.
The convergence of the U-net neural network model may be that the iteration number reaches a preset number or the deviation is smaller than a preset value.
S2, acquiring an undersampled hologram through a sparse sensor array to serve as a sparse sampling hologram;
In the field of digital holography, the diffracted beam from each point on the object can be considered as a cone diffraction cone, taking into account the constraints of the sensor pixel pitch. The maximum spatial frequency that the sensor can capture is limited. Since full field sensors cannot effectively capture the entire image map, acquisition with a sparse sensor array, however, loss of information in the sensor gap will result in loss of frequency components corresponding to the entire scene, and thus an undersampled hologram will be acquired as a sparse sampling hologram . In this embodiment, the sparse sample hologram/>Sparse sampling holograms/>, holograms that are handwritten charactersThe pixel size of (1) is 1200 x 1200, and the region where the handwritten character is located is a supporting region, i.e. the supporting region is the region where the foreground is located; the support region is at the very center region of the sparse sampled hologram M, with a pixel size of 28 x 28.
It should be noted that loss of frequency components corresponding to the entire scene due to information loss of gaps between sensors in the sensor array will result in limited reconstructed image quality, especially in the case of a limited number of sensors, especially in the case of an extremely sparse number of sensors.
It should be further noted that, since the sparse sampling hologram M of the present application is a hologram of a handwritten character, the reconstructed image is essentially a reconstruction of pixel values corresponding to the region where the handwritten character is located.
S3, carrying out backward propagation on the sparse sampling hologram to obtain an object image of an amplitude and phase dual-channel matrix, and respectively implementing absorption constraint and support constraint on the amplitude matrix and implementing absorption constraint on the phase matrix;
Specifically, the undersampled hologram is transmitted backwards through inverse Fourier transform to obtain an object image of an amplitude and phase dual-channel matrix; in this embodiment, the pixel size of the amplitude and phase matrices are 1200 x 1200.
As described above, the reconstructed image is substantially a reconstruction of the pixel values corresponding to the region in which the handwritten character is located, and therefore, in order to remove the influence of noise on the quality of the reconstructed image, the amplitude matrix is subjected to the absorption constraint first and then to the support constraint for assigning 0 to the pixel values outside the support region without changing the pixel values of the support region; meanwhile, supporting constraint is implemented on the phase matrix, and pixel values which do not accord with a physical principle are removed; the specific constraint formula is as follows:
(x,y)=/>
In the method, in the process of the invention, (X, y) represents the pixel value of the coordinates (x, y) in the amplitude matrix; p represents outside the support region; /(I)(X, y) represents the pixel value of the coordinates (x, y) in the phase matrix.
S4, loading an object image which implements absorption constraint and support constraint on the amplitude matrix and implements absorption constraint on the phase matrix into a diffusion model, adding random noise of a preset time step to the constrained amplitude matrix by the diffusion model to obtain a pure noise amplitude matrix, adding random noise of the preset time step to the constrained phase matrix to obtain a pure noise phase matrix, and denoising the pure noise amplitude matrix and the pure noise phase matrix by using a trained U-net neural network model to reconstruct an object image with dual channels of amplitude and phase;
Specifically, the diffusion model includes a forward process and a backward process, and the forward process of the diffusion model includes a total of T time steps, in this embodiment, T is 2000, and the amplitude matrix corresponds to the loaded object image Continuously adding random noise after T time steps to obtain a pure Gaussian noise amplitude matrix/>The matrix corresponding to the t-th time step is/>; Phase matrix/>, corresponding to a loaded object imageContinuously adding random noise after T time steps to obtain a pure Gaussian noise phase matrix/>The matrix corresponding to the t-th time step is/>. The forward diffusion process can be understood as a markov chain, i.e. by gradually adding gaussian noise to the constraint matrix until it finally becomes a pure gaussian noise matrix. The reverse process of the diffusion model is opposite to the forward process, and the diffusion model is gradually sampled from the pure noise matrix to obtain the amplitude and phase matrix after constraint, so that the object image with the two channels of amplitude and phase is reconstructed.
Wherein, in the reverse process of the diffusion model,
(/>)/>；
(/>)/>；
Where t=t-1, T-2,..1, 0, i.e. the noise matrix at the previous moment T-1 is calculated in sequence starting from the pure gaussian noise matrixFor training a target score function obtained by a U-net neural network model, when a noise matrix corresponding to a t-th time step is calculated, the noise amplitude matrix/>, corresponding to the t+1 time step, which is obtained by calculation is calculatedAnd noise phase matrix/>Respectively input into the objective score function,/>And/>Respectively carrying out noise scheduling (noise scheduling) corresponding to a t time step and a t+1 time step in the forward process; /(I)Representing standard normalization.
The object image is gradually converted into random noise through a continuously evolving diffusion process, and then the process is reversed, so that the object image with two channels of amplitude and phase is reconstructed from the generated sample noise, and the information loss of gaps between the sensors can be reconstructed and recovered.
S5, carrying out forward propagation on the reconstructed object image with the two channels of amplitude and phase to obtain a hologram, and carrying out fidelity operation on the hologram based on the sparse sampling hologram to obtain a fidelity image;
specifically, the object image with two channels of amplitude and phase obtained by reconstruction is transmitted forward through Fourier transform to obtain a hologram, and fidelity operation is carried out based on the sparse sampling hologram to obtain a fidelity image;
the calculation formula of the fidelity operation is as follows:
；
Wherein the method comprises the steps of Representing a Fidelity image, W representing a sparse sampling matrix associated with sensor array arrangement,/>Representing fourier transform forward propagation, O represents reconstructed amplitude and phase dual channel object images.
It can be understood that the fidelity operation is performed, the pixel value of the corresponding position of the object image obtained by reconstruction is replaced by the pixel value obtained by sparse sampling of the sensor, so that the image after fidelity is close to the sparse sampling hologram M, and the fidelity is ensured.
Step S6, returning the image after fidelity to the step of carrying out backward propagation on the sparse sampling hologram to obtain an object image of the amplitude and phase dual-channel matrix, and respectively implementing absorption constraint and support constraint on the amplitude matrix and implementing absorption constraint on the phase matrix for iteration until the number of iterations reaches the preset number of iterations;
Specifically, the image after fidelity is returned to the object image of the dual-channel matrix of amplitude and phase after inverse Fourier back propagation, the steps of absorbing constraint and supporting constraint are respectively implemented on the amplitude matrix and absorbing constraint is implemented on the phase matrix, the iteration times are added by 1, then the steps S3-S5 are repeated, the image after fidelity is obtained again until the iteration times reach the preset iteration times, and then the step S7 is executed. In this embodiment, the preset number of iterations is 500.
S7, carrying out backward propagation on the image after fidelity, recovering to obtain a reconstructed image and outputting the reconstructed image;
Specifically, the image after the iteration times reach the fidelity of the preset iteration times is transmitted to the opposite direction after the inverse Fourier transform, and the final reconstructed amplitude and phase matrix is obtained; and generating a reconstructed image based on the final reconstructed amplitude matrix and phase matrix and outputting the reconstructed image.
In summary, the invention provides a sparse digital holographic image reconstruction method. The method mainly learns complex amplitude priori information of a target light field from an amplitude and phase dual-channel image dataset, and supports an iterative reconstruction process of an image by utilizing the complex amplitude priori information. The space domain is obtained by backward propagation of the hologram after fidelity, the holographic domain is obtained by forward propagation of the amplitude and the phase after reconstruction of the diffusion model, and the alternate iteration between the space domain and the holographic domain can mutually supplement the mutual information. The absorption constraint and the support constraint are applied to the amplitude matrix and the support constraint is applied to the phase matrix so as to obtain a high-quality image, the influence of noise outside a foreground area on the image quality can be removed, and pixel values which do not accord with a physical principle are removed. In addition, the hologram after fidelity is sent to the object image after each iteration and is subjected to backward propagation, the next iteration is carried out in the step of obtaining the object image of the amplitude and phase dual-channel matrix, the amplitude and phase of the reconstructed image are accurately recovered through a backward propagation algorithm, gradual improvement of the image quality in the continuous iteration process is ensured, in the fidelity process, the pixel value obtained by sparse sampling of the sensor is replaced to the pixel value of the corresponding position of the object image obtained by reconstruction, and the image after fidelity is close to the sparse sampling hologram sampled by the sensor array, so that the fidelity is ensured.
The invention also provides a contrast test of the sparse digital holographic image reconstruction method and other algorithms:
1. data set preparation
The dataset contained 60000 images, each image having a resolution of 1200 x 1200 pixels. For the amplitude, the digital portion is assigned a pixel value of 0.1, and the background portion is designated as a pixel value of 1. In phase, the pixel values of the numbers and the background are 1 and 0.
2. Model training and parameter selection
The parameters were chosen such that the wavelength was 500nm, the side length of the object region was 0.001, the propagation distance was 0.0024, and the hologram side length was 0.001. The amount of noise added to the model was 2000, the maximum noise pixel value was 10, and the minimum was 0.01. The random number seed used is set to 42. The model is trained by Adam algorithm, and the learning rate is 0.0002. The method is implemented using a computer equipped with NVIDIATITANGPU. In the reconstruction phase, the number of iterations is set to n=500.
3. Quantitative index
To quantitatively evaluate the quality of the reconstructed data, mean Square Error (MSE), peak signal to noise ratio (PSNR), and Structural Similarity Index (SSIM) are employed. MSE quantifies the error between pairs of observations representing the same phenomenon. It is defined as:
Wherein the method comprises the steps of Is the number of pixels in the reconstruction result. As the MSE approaches zero, this indicates that the reconstructed image is closer to the reference image.
PSNR describes the relationship between the maximum possible power of a signal and the noise-corrupted power. A higher PSNR means a better reconstruction quality. PSNR is expressed as:
SSIM is used to measure the similarity between a real image and a reconstructed image. It is expressed as:
Wherein the method comprises the steps of And/>Is/>Mean and variance of (c). /(I)Is/>And/>Is a covariance of (c). /(I)And/>Is used to maintain a constant.
4. Reconstruction of different gap sizes
In order to evaluate the effectiveness of holographic diffusion in holographic image reconstruction at different sensor spacings, experimental verification was performed. And the reconstruction result is qualitatively and quantitatively compared with SRSAA method.
In the experiment, the number of sensors was 4, and the sensor size was designated as 450. The number of pixels of the sensor used for the simulation was 512×512. Under the above-described fixed conditions, the effects of the image reconstructed using the holographic diffusion and SRSAA method under various gap conditions were confirmed, as shown in fig. 2.
As shown in fig. 3, the image quality of the two-channel phase and amplitude reconstructed by the two methods gradually worsens with increasing gap, where (a) is a real image, (b) is a SRSAA-method reconstructed image, (c) is a residual image between (a) and (b), (d) is a method reconstructed image of the present invention, and (e) is a residual image between (a) and (d) in fig. 3. As compared to SRSAA, holographic diffusion shows clearer results with increasing gap size, indicating improved performance when processing a more sparse sensor configuration. SRSAA can achieve a satisfactory reconstruction at small gap sizes, with more artifacts in SRSAA becoming apparent as the gap size gets wider. The reconstructed amplitude and phase produced using holographic diffusion exhibits better image quality than SRSAA. For example, when the gap size is 45, an image reconstructed using holographic diffusion appears clearer and shows significantly reduced artifacts. In the bottom row of fig. 3, the image reconstructed using the SRSAA method shows a large portion of the loss of target pixel detail, and the linked regions are incoherent. In contrast, the image reconstructed by the holographic diffusion method is very similar to the real case, while keeping the details and structure unchanged.
As shown in table 1, the average PSNR, SSIM and MSE values of 100 images reconstructed from the MNIST dataset are recorded. Holographic diffusion can achieve significant average PSNR gains of 6.16dB, 6.38dB, 10.44dB, 12.28dB, and 11.10dB at various gaps. It is exciting that the phase and amplitude of the PSNR reconstruction can reach 35.51dB and 41.62dB, respectively, when the gap is 90 °. At the same time, the reconstruction results of the holographic diffusion show higher SSIM values and smaller mean square error values compared to the SRSAA method. Thus, holographic diffusion represents a significant advance in suppressing noise and artifacts at larger gap sizes.
Table 1 results of quantitative reconstruction at various gaps SRSAA and methods of the invention.
5. Reconstruction of different number of sensors
To demonstrate the effectiveness and robustness of the holographic diffusion method in reconstructing images with different numbers of Sensors (SN), a comparison was made between the holographic diffusion method and the SRSAA method.
In the experiments of this section, the gap was 60 and the sensor size was 500. The number of pixels of the sensor for simulation was set to 512×512. Under the fixed conditions, the effects of holographic diffusion and SRSAA method reconstruction images under different sensor numbers are verified.
As shown in fig. 4, the square block in the figure represents the sensor array, and (a), (b) and (c) in the figure represent sampling conditions when the number of sensors is 2,3 and 4, respectively. As the number of sensors decreases, a dramatic decrease in the quality of the reconstructed amplitude and phase is observed for both methods. Although SRSAA is capable of reconstructing a clear image using a large number of sensors, as shown in fig. 5, its performance may significantly decrease as the number of sensors decreases, deviating from ground reality in terms of basic outline and detail. As the number of sensors decreases, the holographically diffusion reconstructed image is clearer than the SRSAA reconstructed image. Furthermore, the reconstructed image may be more nearly ground-truth when the basic structure and contours of the image remain unchanged. Experiments with different numbers of sensors indicate that holographic diffusion not only reconstructs image details more effectively, but also suppresses the generation of artifacts and twinning images.
Table 2 quantitative reconstruction results for SRSAA method and the method of the invention under different numbers of sensors.
The average PSNR, SSIM and MSE values for 100 images reconstructed from the MNIST dataset with different numbers of sensor arrays are recorded in table 2. Overall, holographic diffusion is always excellent in various sensor counts. When the number of the sensors is 3, PSNR values of the phase and the amplitude of the holographic diffusion reconstruction are respectively improved by 8.11dB and 6.79dB. Furthermore, the reconstruction results obtained using the holographic diffusion method show higher SSIM values and smaller MSE values. Holographic diffusion is effective to suppress noise and twin images to a large extent even with fewer sensors.
In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the device or element being referred to must have a specific orientation, be configured and operated in a specific orientation, and therefore should not be construed as limiting the invention.
In the description of the present specification, reference to the terms "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples.
It will be apparent that the described embodiments are only some, but not all, embodiments of the application. Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application for the embodiment. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly understand that the embodiments described herein may be combined with other embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
While embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the spirit and principles of the invention, the scope of which is defined by the claims and their equivalents.
Claims (8)
1. The sparse digital holographic image reconstruction method is characterized by comprising the following steps of:
Training the U-net neural network model to obtain a trained U-net neural network model;
acquiring an undersampled hologram by using a sparse sensor array as a sparse sampling hologram;
The sparse sampling hologram is transmitted backwards to obtain an object image of an amplitude and phase dual-channel matrix, and absorption constraint and support constraint are respectively implemented on the amplitude matrix and absorption constraint is implemented on the phase matrix;
Loading an object image which implements absorption constraint and support constraint on an amplitude matrix and implements absorption constraint on a phase matrix into a diffusion model, adding random noise of a preset time step to the constrained amplitude matrix by the diffusion model to obtain a pure noise amplitude matrix, adding random noise of the preset time step to the constrained phase matrix to obtain a pure noise phase matrix, and denoising the pure noise amplitude matrix and the pure noise phase matrix by using a trained U-net neural network model to reconstruct to obtain an object image with two channels of amplitude and phase;
The object image with the two channels of amplitude and phase obtained by reconstruction is transmitted forward to obtain a hologram, and the hologram is subjected to fidelity operation based on the sparse sampling hologram so as to obtain a fidelity image;
Returning the image after fidelity to the step of carrying out backward propagation on the sparse sampling hologram to obtain an object image of an amplitude and phase dual-channel matrix, and respectively implementing absorption constraint and support constraint on the amplitude matrix and implementing absorption constraint on the phase matrix, and iterating until the number of iterations reaches the preset iteration number;
and (5) carrying out backward propagation on the image after the fidelity, recovering to obtain a reconstructed image and outputting the reconstructed image.
2. The method of reconstructing a sparse digital holographic image of claim 1, wherein said steps of respectively implementing absorption constraints and support constraints on the amplitude matrix and absorption constraints on the phase matrix comprise:
Firstly implementing absorption constraint on the amplitude matrix, and then implementing support constraint, wherein the support constraint is used for assigning 0 to the pixel value outside the support area, and the pixel value of the support area is not changed; the absorption constraint is implemented on the phase matrix, and the specific constraint formula is as follows:
(x,y)=/>
In the method, in the process of the invention, (X, y) represents the pixel value of the coordinates (x, y) in the amplitude matrix; p represents outside the support region; /(I)(X, y) represents the pixel value of the coordinates (x, y) in the phase matrix, and the supporting area is the area where the foreground is located.
3. The method for reconstructing a sparse digital holographic image of claim 1, wherein said step of training a U-net neural network model to obtain a trained U-net neural network model comprises:
Acquiring an image dataset;
randomly adding noise into an original image corresponding to the image dataset to obtain a noise image;
Inputting the amplitude and phase dual-channel matrix based on the noise image into a U-net neural network model to obtain a reconstructed image;
constructing a fractional function based on the deviation of the data distribution of the pixel values of the amplitude and phase matrix of the reconstructed image and the data distribution of the pixel values of the amplitude and phase matrix of the original image;
And continuously training the U-net neural network model until the U-net neural network model converges to obtain a target score function.
4. The method for reconstructing a sparse digital holographic image of claim 3, wherein said step of denoising said pure noise amplitude matrix and said pure noise phase matrix using said trained U-net neural network model to reconstruct an object image having two channels of amplitude and phase comprises:
denoising the pure noise amplitude matrix and the pure noise phase matrix based on the target scoring function;
Sequentially calculating a noise matrix corresponding to the previous moment from a pure Gaussian amplitude matrix and a pure noise phase matrix respectively until an amplitude matrix and a phase matrix corresponding to the moment 0 are calculated;
reconstructing an object image based on the amplitude matrix and the phase matrix corresponding to the moment 0;
wherein, (/>)/>；
(/>)/>；
Where t=t-1, T-2,..1, 0For training a target score function obtained by a U-net neural network model, when a noise matrix corresponding to a t-th time step is calculated, the noise amplitude matrix/>, corresponding to the t+1 time step, which is obtained by calculation is calculatedAnd noise phase matrix/>Respectively input into the objective score function,/>And/>Respectively carrying out noise scheduling corresponding to a t time step and a t+1 time step in the forward process; /(I)Representing standard normalization.
5. The method of claim 1, wherein the step of obtaining the fidelity-derived image is calculated as follows:
；
Wherein the method comprises the steps of Representing a Fidelity image, W representing a sparse sampling matrix associated with sensor array arrangement,/>Representing fourier transform forward propagation, O represents reconstructed amplitude and phase dual-channel object images, and M represents a sparse sampled hologram.
6. A sparse digital holographic image reconstruction method as claimed in claim 1, wherein:
The step of backward propagation of the sparse sampling hologram is that the sparse sampling hologram is backward propagated through inverse Fourier transform;
The hologram obtained by forward propagation of the reconstructed amplitude and phase dual-channel object image is obtained by forward propagation of the reconstructed amplitude and phase dual-channel object image through Fourier transformation.
7. An electronic device, comprising: a memory and a processor;
The memory is used for storing programs;
The processor for invoking a program stored in the memory to perform a sparse digital holographic image reconstruction method as defined in any one of claims 1-6.
8. A readable storage medium, characterized in that the readable storage medium has stored thereon a computer program which, when executed by a processor, implements a sparse digital holographic image reconstruction method as defined in any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410464769.2A CN118071866A (en) | 2024-04-18 | 2024-04-18 | Sparse digital holographic image reconstruction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410464769.2A CN118071866A (en) | 2024-04-18 | 2024-04-18 | Sparse digital holographic image reconstruction method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118071866A true CN118071866A (en) | 2024-05-24 |
Family
ID=91102338
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410464769.2A Pending CN118071866A (en) | 2024-04-18 | 2024-04-18 | Sparse digital holographic image reconstruction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118071866A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101458393A (en) * | 2007-12-13 | 2009-06-17 | 崔庆君 | Multiple angular field and multi-angle image rebuilding apparatus |
| US20190294108A1 (en) * | 2018-03-21 | 2019-09-26 | The Regents Of The University Of California | Method and system for phase recovery and holographic image reconstruction using a neural network |
| CN110728728A (en) * | 2019-09-17 | 2020-01-24 | 南京信息工程大学 | Compressed sensing network image reconstruction method based on non-local regularization |
| CN114066735A (en) * | 2021-11-30 | 2022-02-18 | 重庆邮电大学 | Sparse sampling Fourier laminated imaging artificial neural network reconstruction method based on least one-time multiplication |
| CN114067015A (en) * | 2021-11-11 | 2022-02-18 | 中国人民解放军陆军装甲兵学院 | Pure phase hologram generation method and system combining DNN |
| CN115797231A (en) * | 2022-12-05 | 2023-03-14 | 上海交通大学 | Real-time hologram generation method based on neural network of Fourier inspiration |
| CN116645432A (en) * | 2023-06-07 | 2023-08-25 | 上海交通大学 | High-quality hologram generating method based on improved ViT network |
| CN117237473A (en) * | 2023-10-16 | 2023-12-15 | 南昌大学 | Photoacoustic tomography sparse reconstruction method based on diffusion model |
| US20240103439A1 (en) * | 2022-09-20 | 2024-03-28 | Army Academy of Armored Forces of PLA | Method and system for optimizing first-diffraction-order reconstruction of holograms, device and medium |
| CN117876591A (en) * | 2024-01-09 | 2024-04-12 | 浙江工业大学 | Real fuzzy three-dimensional hologram reconstruction method for combined training of multiple neural networks |
-
2024
- 2024-04-18 CN CN202410464769.2A patent/CN118071866A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101458393A (en) * | 2007-12-13 | 2009-06-17 | 崔庆君 | Multiple angular field and multi-angle image rebuilding apparatus |
| US20190294108A1 (en) * | 2018-03-21 | 2019-09-26 | The Regents Of The University Of California | Method and system for phase recovery and holographic image reconstruction using a neural network |
| CN110728728A (en) * | 2019-09-17 | 2020-01-24 | 南京信息工程大学 | Compressed sensing network image reconstruction method based on non-local regularization |
| CN114067015A (en) * | 2021-11-11 | 2022-02-18 | 中国人民解放军陆军装甲兵学院 | Pure phase hologram generation method and system combining DNN |
| CN114066735A (en) * | 2021-11-30 | 2022-02-18 | 重庆邮电大学 | Sparse sampling Fourier laminated imaging artificial neural network reconstruction method based on least one-time multiplication |
| US20240103439A1 (en) * | 2022-09-20 | 2024-03-28 | Army Academy of Armored Forces of PLA | Method and system for optimizing first-diffraction-order reconstruction of holograms, device and medium |
| CN115797231A (en) * | 2022-12-05 | 2023-03-14 | 上海交通大学 | Real-time hologram generation method based on neural network of Fourier inspiration |
| CN116645432A (en) * | 2023-06-07 | 2023-08-25 | 上海交通大学 | High-quality hologram generating method based on improved ViT network |
| CN117237473A (en) * | 2023-10-16 | 2023-12-15 | 南昌大学 | Photoacoustic tomography sparse reconstruction method based on diffusion model |
| CN117876591A (en) * | 2024-01-09 | 2024-04-12 | 浙江工业大学 | Real fuzzy three-dimensional hologram reconstruction method for combined training of multiple neural networks |
Non-Patent Citations (4)
| Title |
|---|
| YANG J等: "Study on extending the depth of field in reconstructed image for a micro digital hologram", 《物理学报》, 31 December 2015 (2015-12-31) * |
| 周桂文;: "联合稀疏约束的截断傅里叶变换数字全息重构", 信息通信, no. 04, 15 April 2016 (2016-04-15) * |
| 张华;曹良才;金国藩;白瑞迪;: "基于压缩感知算法的无透镜数字全息成像研究", 激光与光电子学进展, no. 08, 25 April 2020 (2020-04-25) * |
| 张志刚;李涛;张玉霞;刘艳辉;: "计算成像系统中的模糊图像重建研究", 激光杂志, no. 07, 25 July 2020 (2020-07-25) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Meraner et al. | Cloud removal in Sentinel-2 imagery using a deep residual neural network and SAR-optical data fusion | |
| CN111369487B (en) | Hyperspectral and multispectral image fusion method, system and medium | |
| CN107507135B (en) | Image reconstruction method based on coding aperture and target | |
| Wang et al. | High-fidelity view synthesis for light field imaging with extended pseudo 4DCNN | |
| CN110490937B (en) | Method and device for accelerating hyperspectral video reconstruction | |
| CN114677300A (en) | Hyperspectral image depth noise reduction method and system based on two-stage learning framework | |
| CN116228912B (en) | Image compressed sensing reconstruction method based on U-Net multi-scale neural network | |
| Chung et al. | Missing cone artifact removal in ODT using unsupervised deep learning in the projection domain | |
| CN116757955A (en) | Multi-fusion comparison network based on full-dimensional dynamic convolution | |
| CN114119417A (en) | Image processing method, system and storage medium | |
| CN118071866A (en) | Sparse digital holographic image reconstruction method | |
| US20180188686A1 (en) | Method and apparatus for hologram resolution transformation | |
| CN117522761A (en) | Underwater dynamic polarization imaging method, system, equipment and storage medium | |
| Danielyan et al. | Deblurring of Poissonian images using BM3D frames | |
| CN117057119A (en) | PnP-ADMM inversion method and inversion equipment based on multi-constraint coupling | |
| CN113448233B (en) | Under-sampling hologram compression holographic multi-scale self-focusing reconstruction method and system | |
| CN112288633B (en) | Novel sub-pixel resolution diffraction imaging method | |
| CN112486003A (en) | Phase hologram generation method based on self-adaptive weight feedback GS algorithm | |
| Chang et al. | An Efficient Way for Active None-Line-of-Sight: End-to-End Learned Compressed NLOS Imaging | |
| Hou et al. | Fast Monte Carlo rendering via multi-resolution sampling | |
| CN117288325B (en) | High-light-efficiency snapshot type multispectral imaging method and system | |
| CN118197293A (en) | Method and equipment for improving accuracy of voice emotion classification model | |
| Wang et al. | Robust extension of light fields with probable 3D distribution based on iterative scene estimation from multi-focus images | |
| CN114397700B (en) | Method, device, equipment and storage medium for interpolating pre-stack seismic data of node seismograph based on graph signal constraint | |
| Murugan | Facial information recovery from heavily damaged images using generative adversarial network-part 1 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |