CN114387395A - Phase-double resolution ratio network-based quick hologram generation method - Google Patents

Phase-double resolution ratio network-based quick hologram generation method Download PDF

Info

Publication number
CN114387395A
CN114387395A CN202210029238.1A CN202210029238A CN114387395A CN 114387395 A CN114387395 A CN 114387395A CN 202210029238 A CN202210029238 A CN 202210029238A CN 114387395 A CN114387395 A CN 114387395A
Authority
CN
China
Prior art keywords
phase
hologram
network
convolution
calculating
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210029238.1A
Other languages
Chinese (zh)
Inventor
王帅
于挺
田子建
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China University of Mining and Technology Beijing CUMTB
Original Assignee
China University of Mining and Technology Beijing CUMTB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China University of Mining and Technology Beijing CUMTB filed Critical China University of Mining and Technology Beijing CUMTB
Priority to CN202210029238.1A priority Critical patent/CN114387395A/en
Publication of CN114387395A publication Critical patent/CN114387395A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Holo Graphy (AREA)

Abstract

The invention discloses a phase-double resolution ratio network-based hologram rapid generation method, which comprises the following steps: the method has the advantages that a real pure phase hologram mask does not need to be manufactured, the micromation of an angular spectrum method is utilized, and the unsupervised training of a convolutional neural network is realized by using a natural image; the convolutional layer learns feature mapping in the same space, rather than learning feature mapping across distances; reducing the network model calculation amount and the memory occupancy rate of the GPU by using the void convolution and the group convolution; a combination of MS-SSIM and MSE losses is used as a consistency loss function to generate a reconstructed map that is more consistent with the human visual system. The calculation of the method for generating the 1080P resolution hologram only needs 57 milliseconds, and the peak signal-to-noise ratio of the optimal numerical reconstruction image and the target intensity image reaches 31.17 dB.

Description

Phase-double resolution ratio network-based quick hologram generation method
Technical Field
The invention relates to the field of computer generated holography and the field of computer vision, in particular to a method for quickly generating a hologram based on a phase-double-resolution network.
Background
Holographic display is considered to be an advanced display technology, and is gradually applied to three-dimensional scene reconstruction, virtual reality and augmented display systems, so that a three-dimensional display mode without glasses is possible. When the real-time holographic display is realized, how to ensure the high fidelity of the holographic display image is still a difficult problem. The phase type spatial light modulator has high optical efficiency, and has no interference of conjugate images in the reconstruction process, and a pure phase hologram becomes a main encoding method for generating the hologram by calculation. Therefore, developing an algorithm capable of calculating and generating a high-quality phase type hologram in real time is of great significance to the development of holographic display technology.
For more than half a century, many algorithms for computationally generating holograms have been introduced, including iterative and non-iterative optimization algorithms. The Gercheberg-Saxton algorithm proposed in 1972 is a typical representation of iterative optimization computationally generated holograms. Then, some improved algorithms based on the Gercheberg-Saxton iterative optimization algorithm exist, for example, random noise optimization is added in the Gercheberg-Saxton iterative optimization algorithm to generate the hologram; generating a hologram by a bidirectional error diffusion algorithm; and (3) solving a non-convex optimization problem through gradient descent or Wirtinger derivative to indirectly calculate and generate a hologram and the like. Since the iterative optimization algorithm needs to take a long calculation time to generate the hologram and is not suitable for the real-time calculation of large-scale and high-resolution holograms, researchers have proposed non-iterative optimization algorithms, such as bi-phase amplitude encoding and one-step phase extraction algorithms. Although the calculation speed of the non-iterative optimization algorithm is greatly improved and real-time calculation can be met, the reconstructed image of the hologram contains more speckle noises. Therefore, the non-iterative optimization algorithm cannot guarantee the quality of the generated hologram.
In recent years, with the continuous and deep algorithm research, deep learning technology is gradually introduced into the optical field, and the emergence of deep learning related algorithms also provides a new method for generating holograms through calculation. Convolutional neural networks, as a generalized approximation function, can learn the mapping between inputs and outputs. In 2021, Shi et al introduced large-scale fresnel holographic datasets, trained convolutional neural networks to generate realistic three-dimensional holograms, and were able to shift focal lengths. Researchers also use the fresnel method to generate a large number of holograms as data sets to train confrontation generation networks. The training set used in this type of method is a hologram generated by calculation using a conventional iterative optimization algorithm. Therefore, the performance of the network model is directly limited by the quality of the training set and indirectly limited by the traditional iterative optimization algorithm. How to break these limitations becomes a key challenge for deep learning applications for computationally generated holograms. More importantly, the convolutional layer acts on the spatial dimensions of the input, being best suited to model and compute the spatial relationship mapping between the input and output. The convolution neural network maps the image defined in the object plane to the pure phase hologram mask defined in the hologram plane in a cross-domain mode, and the problems that the space corresponding relation cannot be reserved and the strong characteristic mapping capability of the convolution neural network cannot be exerted exist.
Disclosure of Invention
In order to solve the above problems in the background art, the present application provides a method for rapidly generating a hologram based on a phase-dual resolution network, which mainly adopts the following technical scheme:
construction of phase-Dual resolution network model fnet1And fnet2；
Trained phase-dual resolution network fnet1Calculating an initial phase phi according to the input target intensity map I0；
According to the initial phase phi0Calculating a complex value wave field U with the target intensity graph Iz(ii) a From the angle spectrum method, a complex-valued wavefield U is calculatedzComplex wave field U obtained after propagation of-z in free space0；
Trained phase-dual resolution network fnet2From the input complex-valued wavefield U0A phase-only hologram is calculated.
Preferably, the constructing the phase-dual resolution network model includes:
the phase-dual resolution network is mainly divided into an encoder and a decoder;
in the encoder of the phase-dual resolution network, the first two layers of networks use group convolution to carry out feature extraction; and (3) performing feature extraction on the rest network layers in the encoder by using a hole convolution, wherein a hole factor in the hole convolution follows a maximum distance calculation formula between two non-zero values in the convolution:
Mi＝max[Mi+1-2ri,Mi+1-2(Mi+1-ri),ri],Mn＝rn；
in a decoder, performing up-sampling on the features extracted by the encoder through sub-pixel convolution until the resolution is the same as that of an input image; the third layer network output of the encoder uses the first layer of the jump connection to the decoder for feature splicing; taking the 1 × 1 convolution as the last layer of the decoder, so that the output is a single-channel image;
the gray values of the single channel images are normalized to [ -pi, pi ].
Preferably, the training process of the trained phase-dual resolution network includes:
constructed phase-dual resolution network fnet2Calculating to generate phase-only hologram, and calculating the reconstructed image of the hologram according to angular spectrum method
Figure BDA0003464893540000028
Wherein the reconstruction distance is z;
calculating an input target intensity map I and a reconstructed map calculated by an angular spectrum method
Figure BDA0003464893540000029
MS-SSIM loss and MSE loss in between;
summing MS-SSIM loss and MSE loss according to a certain weight ratio to form an integral consistency loss function, and realizing a phase-double resolution network model fnet1And fnet2While training at the same time.
Preferably, said complex valued wavefield UzThe calculation formula (2) includes:
Figure BDA0003464893540000021
preferably, the summing of the MS-SSIM loss and the MSE loss according to a certain weight ratio as an overall loss function includes:
Figure BDA0003464893540000022
Figure BDA0003464893540000023
Figure BDA0003464893540000024
wherein, I represents the target intensity map,
Figure BDA0003464893540000025
a reconstructed image is shown which is,
Figure BDA0003464893540000026
for calculating I and
Figure BDA0003464893540000027
with p representing pixels, for calculating the error pixel by pixel, a is set to 0.84 empirically.
Drawings
FIG. 1 shows an overall block diagram of the generation of a phase-only hologram based on a phase-dual resolution network according to the present application;
FIG. 2 shows a block diagram of a phase-dual resolution network of the present application;
FIG. 3 shows a holographic display structure schematic diagram and a simulation real object diagram constructed by the application
Detailed Description
The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which preferred embodiments of the invention are shown.
Holographic display technology is gradually applied to three-dimensional scene reconstruction, virtual reality and augmented display systems, and a three-dimensional display mode without glasses becomes possible. In the holographic display technology, a hologram is used as an information carrier and contains depth and intensity information of an object, so that the calculation of generating the hologram is a key step in the holographic display. The current method for generating the hologram by calculation mainly comprises an iterative optimization algorithm and a non-iterative optimization algorithm. However both algorithms have been a trade-off between the quality of the optimization of the hologram and the computation time for many years. With the gradual introduction of deep learning techniques into the field of optical research, many satisfactory results are obtained. The invention provides a neural network algorithm capable of calculating and generating a high-quality hologram in real time, and has important significance for the development of real-time holographic display technology.
Therefore, the application provides a method for rapidly generating the hologram based on the phase-dual resolution network. Firstly, introducing an angular spectrum method for a light field to propagate in a free space, wherein a calculation formula is as follows:
uz(x1,y1)＝fAS{φ(x,y)}＝IFFT{FFT{usrc(x,y)·eiφ(x,y)}·H(fx,fy)} (1)
wherein u issrcRepresenting the complex wave field, u, produced by a coherent light sourcezRepresenting the complex-valued wave field at the diffraction distance z, f (x, y) representing the phase-only hologram loaded into the spatial light modulator, FFT and IFFT being the Fourier transform and inverse Fourier transform operators, respectively, H (f)x, fy) Is an angular spectrum transformation function. H (f)x,fy) Is expressed as:
Figure BDA0003464893540000031
wherein f isxAnd fyIs the spatial frequency; λ is the wavelength of the light, z is the diffraction distance, H (f)x,fy) Is the transfer function of angular spectroscopy. From the formula (1), usrc(x, y) is a coherent light source that is content independent, so the angular spectroscopy conductivity is only that of H (f)x,fy) It is related. According to equation (2), the transfer function of angular spectroscopy is related to wavelength, propagation distance, pixel pitch, and spatial resolution. When the optical system is fixed, the four parameters are fixed, and the transfer function can be regarded as a constant in the back propagation of the convolutional neural network. Therefore, the calculation formula of the angular spectrum analysis method satisfies the gradient propagation. According to general approximate theorem of neural network, utilizing forward propagation of convolution neural network to produce pure phase hologram and using angular spectrum method to calculate holomorphismAnd (4) numerical reconstruction of the information map. This process can be expressed in the following form:
Figure BDA0003464893540000032
wherein I represents the target amplitude, fnetAn approximation function, phi, representing a convolutional neural networkholoA phase-only hologram is represented,
Figure BDA00034648935400000310
representing a reproduced image. The process of calculating the gradient by the loss function and the angular spectrum method can be expressed as:
Figure BDA0003464893540000033
wherein wkAnd bkK-th layer trainable parameters, phi, representing a convolutional neural networkholoA hologram is shown.
Fig. 1 shows the overall structure of a phase-dual resolution network-based hologram fast generation method. The input is any one of intensity maps, the phase-dual resolution network f after trainingnet1Mapping it into a phase map; the phase map and the input target intensity map are calculated as a complex-valued wavefield U by equation (5)z(ii) a According to the angle spectrum method, the complex value wave field is propagated in free space by-z distance to obtain a complex value wave field U0(ii) a Due to the phase-dual resolution network fnet2Cannot perform complex operations, so the complex-valued wavefield U is transformed according to the Euler equation0Decomposing into a real part and an imaginary part, and performing matrix splicing operation; matrix-spliced feature matrix input phase-dual resolution network fnet2And calculating to obtain the hologram.
Figure BDA0003464893540000034
The above process is to use a trained phase-dual resolution network fnet1And fnet2Generation ofThe process of hologram. Since high quality holographic masks are difficult to obtain, the training process is accomplished by convolutional neural network training without a holographic mask. Phase-dual resolution network f in training phasenet2Calculating the hologram by angular spectrum method
Figure BDA0003464893540000035
Equation (3) can be further expressed here as:
Figure BDA0003464893540000036
in the formula (I), the compound is shown in the specification,
Figure BDA0003464893540000037
represents a reproduced image phiholoRepresenting phase-only holograms, fASRepresenting angular spectroscopy, I representing the target amplitude. Respectively calculating an input target intensity map I and a reconstructed map calculated by an angular spectrum method
Figure BDA0003464893540000038
The MS-SSIM loss and the MSE loss are added according to a certain proportion to be used as an integral consistency loss function, and the phase-double resolution ratio network model f is realizednet1And fnet2While training at the same time. The consistency loss function can be expressed as:
Figure BDA0003464893540000039
wherein I represents the amplitude of the target,
Figure BDA0003464893540000041
a reconstructed image is shown which is,
Figure BDA0003464893540000042
for calculating I and
Figure BDA0003464893540000043
multiple ruler betweenStructural similarity, p denotes pixel, for calculating the error pixel by pixel, and α is set to 0.84 empirically.
Fig. 2 shows a diagram of a phase-dual resolution network architecture. Wherein CBP represents a convolution module comprising a convolution layer, a batch normalization layer and a PReLU activation function. RB denotes a residual block and RBB denotes a residual bottleneck block. Dapm stands for deep polymerization pyramid pooling module. The phase-dual resolution network is mainly composed of an encoder and a decoder, wherein the encoder is composed of a high resolution branch and a low resolution branch.
The high resolution branch of the encoder may retain more high frequency features while the low resolution branch may extract high level abstract features of the target amplitude. The first two layers of the network of the encoder use group convolution for feature extraction. And (3) performing feature extraction on the rest network layers in the encoder by using a hole convolution, wherein a hole factor in the hole convolution follows a maximum distance calculation formula between two non-zero values in the convolution:
Mi＝max[Mi+1-2ri,Mi+1-2(Mi+1-ri),ri],Mn＝rn (8)
where i denotes the ith layer of convolution, MiRepresenting the maximum distance between non-zero values in the ith layer of the convolution output. r isiIndicating the hole factor of the i-th layer of the convolution. The calculation amount and GPU memory occupation when the phase-dual resolution ratio network generates the high-resolution hologram can be reduced by using the group convolution, and the receptive field enhanced feature extraction capability of convolution operation can be enlarged by using the hole convolution.
In the decoder, the features extracted by the encoder are upsampled by sub-pixel convolution until the resolution is the same as the input image resolution. The third layer network output of the encoder uses the first layer which is connected to the decoder through jumping to carry out feature splicing, and the feature decoding capability is enhanced. The 1 × 1 convolution is taken as the last layer of the decoder, so that the output is a single-channel image. The grey values of the single channel image are normalized to-pi, pi so that the final output of the network is a phase map.
FIG. 3 shows a holographic display structure schematic diagram and a simulation real object diagram constructed by the application. The model of the spatial light modulator is Holoeye Photonics AG, the pixel distance is 8 μm, the laser wavelength is 532nm, and the propagation distance is 40 cm.

Claims (5)

1. A phase-double resolution ratio network-based hologram rapid generation method is characterized in that:
(1) construction of phase-Dual resolution network model fnet1And fnet2；
(2) Trained phase-dual resolution network fnet1Calculating an initial phase phi according to the input target intensity map I0；
(3) According to the initial phase phi0Calculating a complex value wave field U with the target intensity graph Iz(ii) a From the angle spectrum method, a complex-valued wavefield U is calculatedzComplex wave field U obtained after propagation of-z in free space0；
(4) Trained phase-dual resolution network fnet2From the input complex-valued wavefield U0A phase-only hologram is calculated.
2. The method of claim 1, wherein constructing the phase-dual resolution network model comprises:
(1) in the encoder of the phase-dual resolution network, the first two layers of networks use group convolution to carry out feature extraction; and (3) performing feature extraction on the rest network layers in the encoder by using a hole convolution, wherein a hole factor in the hole convolution follows a maximum distance calculation formula between two non-zero values in the convolution:
Mi＝max[Mi+1-2ri,Mi+1-2(Mi+1-ri),ri],Mn＝rn；
(2) in a decoder, performing up-sampling on the features extracted by the encoder through sub-pixel convolution until the resolution is the same as that of an input image; the third layer network output of the encoder uses the first layer of the jump connection to the decoder for feature splicing; taking the 1 × 1 convolution as the last layer of the decoder, so that the output is a single-channel image;
(3) the gray values of the single channel images are normalized to [ -pi, pi ].
3. The method of claim 1, wherein the training of the trained phase-dual resolution network comprises:
(1) constructed phase-dual resolution network fnet2Calculating to generate phase-only hologram, and calculating the reconstructed image of the hologram according to angular spectrum method
Figure FDA0003464893530000011
Wherein the reconstruction distance is z;
(2) respectively calculating an input target intensity map I and a reconstructed map calculated by an angular spectrum method
Figure FDA0003464893530000012
MS-SSIM loss and MSE loss in between;
(3) summing MS-SSIM loss and MSE loss according to a certain weight ratio to form an integral consistency loss function, and realizing a phase-double resolution network model fnet1And fnet2While training at the same time.
4. The method of claim 1, wherein the complex-valued wavefield UzThe calculation formula comprises:
Figure FDA0003464893530000013
5. the method of claim 3, wherein the overall consistency loss function, the calculation formula comprises:
Figure FDA0003464893530000014
Figure FDA0003464893530000015
Figure FDA0003464893530000016
wherein, I represents the target amplitude,
Figure FDA0003464893530000017
the amplitude of the reconstruction is represented by,
Figure FDA0003464893530000018
for calculating I and
Figure FDA0003464893530000019
with p representing pixels, for calculating the error pixel by pixel, a is set to 0.84 empirically.
CN202210029238.1A 2022-01-11 2022-01-11 Phase-double resolution ratio network-based quick hologram generation method Pending CN114387395A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210029238.1A CN114387395A (en) 2022-01-11 2022-01-11 Phase-double resolution ratio network-based quick hologram generation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210029238.1A CN114387395A (en) 2022-01-11 2022-01-11 Phase-double resolution ratio network-based quick hologram generation method

Publications (1)

Publication Number Publication Date
CN114387395A true CN114387395A (en) 2022-04-22

Family

ID=81201867

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210029238.1A Pending CN114387395A (en) 2022-01-11 2022-01-11 Phase-double resolution ratio network-based quick hologram generation method

Country Status (1)

Country Link
CN (1) CN114387395A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114967398A (en) * 2022-05-13 2022-08-30 安徽大学 Large-size two-dimensional calculation hologram real-time generation method based on deep learning
CN115097709A (en) * 2022-07-05 2022-09-23 东南大学 Holographic encoding method based on complex optimizer or complex solver
CN115294412A (en) * 2022-10-10 2022-11-04 临沂大学 Real-time coal rock segmentation network generation method based on deep learning

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114967398A (en) * 2022-05-13 2022-08-30 安徽大学 Large-size two-dimensional calculation hologram real-time generation method based on deep learning
CN114967398B (en) * 2022-05-13 2024-05-31 安徽大学 Large-size two-dimensional calculation hologram real-time generation method based on deep learning
CN115097709A (en) * 2022-07-05 2022-09-23 东南大学 Holographic encoding method based on complex optimizer or complex solver
CN115097709B (en) * 2022-07-05 2023-11-17 东南大学 Holographic coding method based on complex optimizer or complex solver
CN115294412A (en) * 2022-10-10 2022-11-04 临沂大学 Real-time coal rock segmentation network generation method based on deep learning

Similar Documents

Publication Publication Date Title
CN114387395A (en) Phase-double resolution ratio network-based quick hologram generation method
Liu et al. 4K-DMDNet: diffraction model-driven network for 4K computer-generated holography
CN111915660B (en) Binocular disparity matching method and system based on shared features and attention up-sampling
CN113467211B (en) Holographic encoding method based on gradient reduction of spectral loss function
CN105976408A (en) Digital holographic compression transmission method of quantum backward propagation nerve network
KR102605678B1 (en) Method and apparatus for processing holographic image
CN114067015A (en) Pure phase hologram generation method and system combining DNN
CN109447897B (en) Real scene image synthesis method and system
Yu et al. Phase dual-resolution networks for a computer-generated hologram
CN115797231A (en) Real-time hologram generation method based on neural network of Fourier inspiration
CN115100300B (en) Computer hologram compression method based on attention-based heuristic neural network
CN114841859A (en) Single-image super-resolution reconstruction method based on lightweight neural network and Transformer
CN117876591A (en) Real fuzzy three-dimensional hologram reconstruction method for combined training of multiple neural networks
Li et al. Speckle noise suppression algorithm of holographic display based on spatial light modulator
US20240103439A1 (en) Method and system for optimizing first-diffraction-order reconstruction of holograms, device and medium
CN113627487A (en) Super-resolution reconstruction method based on deep attention mechanism
Rong et al. Reconstruction efficiency enhancement of amplitude-type holograms by using Single-Scale Retinex algorithm
CN115690252A (en) Hologram reconstruction method and system based on convolutional neural network
KR20210095433A (en) Deep learning model for generating phase hologram at high speed and learning method thereof
Lee et al. Complex hologram generation of multi-depth images using deep neural network
Liu et al. Calculating real-time computer-generated holograms for holographic 3D displays through deep learning
CN113658330B (en) Holographic encoding method based on neural network
CN115797181A (en) Image super-resolution reconstruction method for mine fuzzy environment
Chen et al. Real-time hologram generation using a non-iterative modified Gerchberg-Saxton algorithm
CN113538236A (en) Image super-resolution reconstruction method based on generation countermeasure network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination