CN114387395A - Phase-double resolution ratio network-based quick hologram generation method - Google Patents
Phase-double resolution ratio network-based quick hologram generation method Download PDFInfo
- Publication number
- CN114387395A CN114387395A CN202210029238.1A CN202210029238A CN114387395A CN 114387395 A CN114387395 A CN 114387395A CN 202210029238 A CN202210029238 A CN 202210029238A CN 114387395 A CN114387395 A CN 114387395A
- Authority
- CN
- China
- Prior art keywords
- phase
- hologram
- network
- convolution
- calculating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Holo Graphy (AREA)
Abstract
The invention discloses a phase-double resolution ratio network-based hologram rapid generation method, which comprises the following steps: the method has the advantages that a real pure phase hologram mask does not need to be manufactured, the micromation of an angular spectrum method is utilized, and the unsupervised training of a convolutional neural network is realized by using a natural image; the convolutional layer learns feature mapping in the same space, rather than learning feature mapping across distances; reducing the network model calculation amount and the memory occupancy rate of the GPU by using the void convolution and the group convolution; a combination of MS-SSIM and MSE losses is used as a consistency loss function to generate a reconstructed map that is more consistent with the human visual system. The calculation of the method for generating the 1080P resolution hologram only needs 57 milliseconds, and the peak signal-to-noise ratio of the optimal numerical reconstruction image and the target intensity image reaches 31.17 dB.
Description
Technical Field
The invention relates to the field of computer generated holography and the field of computer vision, in particular to a method for quickly generating a hologram based on a phase-double-resolution network.
Background
Holographic display is considered to be an advanced display technology, and is gradually applied to three-dimensional scene reconstruction, virtual reality and augmented display systems, so that a three-dimensional display mode without glasses is possible. When the real-time holographic display is realized, how to ensure the high fidelity of the holographic display image is still a difficult problem. The phase type spatial light modulator has high optical efficiency, and has no interference of conjugate images in the reconstruction process, and a pure phase hologram becomes a main encoding method for generating the hologram by calculation. Therefore, developing an algorithm capable of calculating and generating a high-quality phase type hologram in real time is of great significance to the development of holographic display technology.
For more than half a century, many algorithms for computationally generating holograms have been introduced, including iterative and non-iterative optimization algorithms. The Gercheberg-Saxton algorithm proposed in 1972 is a typical representation of iterative optimization computationally generated holograms. Then, some improved algorithms based on the Gercheberg-Saxton iterative optimization algorithm exist, for example, random noise optimization is added in the Gercheberg-Saxton iterative optimization algorithm to generate the hologram; generating a hologram by a bidirectional error diffusion algorithm; and (3) solving a non-convex optimization problem through gradient descent or Wirtinger derivative to indirectly calculate and generate a hologram and the like. Since the iterative optimization algorithm needs to take a long calculation time to generate the hologram and is not suitable for the real-time calculation of large-scale and high-resolution holograms, researchers have proposed non-iterative optimization algorithms, such as bi-phase amplitude encoding and one-step phase extraction algorithms. Although the calculation speed of the non-iterative optimization algorithm is greatly improved and real-time calculation can be met, the reconstructed image of the hologram contains more speckle noises. Therefore, the non-iterative optimization algorithm cannot guarantee the quality of the generated hologram.
In recent years, with the continuous and deep algorithm research, deep learning technology is gradually introduced into the optical field, and the emergence of deep learning related algorithms also provides a new method for generating holograms through calculation. Convolutional neural networks, as a generalized approximation function, can learn the mapping between inputs and outputs. In 2021, Shi et al introduced large-scale fresnel holographic datasets, trained convolutional neural networks to generate realistic three-dimensional holograms, and were able to shift focal lengths. Researchers also use the fresnel method to generate a large number of holograms as data sets to train confrontation generation networks. The training set used in this type of method is a hologram generated by calculation using a conventional iterative optimization algorithm. Therefore, the performance of the network model is directly limited by the quality of the training set and indirectly limited by the traditional iterative optimization algorithm. How to break these limitations becomes a key challenge for deep learning applications for computationally generated holograms. More importantly, the convolutional layer acts on the spatial dimensions of the input, being best suited to model and compute the spatial relationship mapping between the input and output. The convolution neural network maps the image defined in the object plane to the pure phase hologram mask defined in the hologram plane in a cross-domain mode, and the problems that the space corresponding relation cannot be reserved and the strong characteristic mapping capability of the convolution neural network cannot be exerted exist.
Disclosure of Invention
In order to solve the above problems in the background art, the present application provides a method for rapidly generating a hologram based on a phase-dual resolution network, which mainly adopts the following technical scheme:
construction of phase-Dual resolution network model fnet1And fnet2;
Trained phase-dual resolution network fnet1Calculating an initial phase phi according to the input target intensity map I0;
According to the initial phase phi0Calculating a complex value wave field U with the target intensity graph Iz(ii) a From the angle spectrum method, a complex-valued wavefield U is calculatedzComplex wave field U obtained after propagation of-z in free space0;
Trained phase-dual resolution network fnet2From the input complex-valued wavefield U0A phase-only hologram is calculated.
Preferably, the constructing the phase-dual resolution network model includes:
the phase-dual resolution network is mainly divided into an encoder and a decoder;
in the encoder of the phase-dual resolution network, the first two layers of networks use group convolution to carry out feature extraction; and (3) performing feature extraction on the rest network layers in the encoder by using a hole convolution, wherein a hole factor in the hole convolution follows a maximum distance calculation formula between two non-zero values in the convolution:
Mi=max[Mi+1-2ri,Mi+1-2(Mi+1-ri),ri],Mn=rn;
in a decoder, performing up-sampling on the features extracted by the encoder through sub-pixel convolution until the resolution is the same as that of an input image; the third layer network output of the encoder uses the first layer of the jump connection to the decoder for feature splicing; taking the 1 × 1 convolution as the last layer of the decoder, so that the output is a single-channel image;
the gray values of the single channel images are normalized to [ -pi, pi ].
Preferably, the training process of the trained phase-dual resolution network includes:
constructed phase-dual resolution network fnet2Calculating to generate phase-only hologram, and calculating the reconstructed image of the hologram according to angular spectrum methodWherein the reconstruction distance is z;
calculating an input target intensity map I and a reconstructed map calculated by an angular spectrum methodMS-SSIM loss and MSE loss in between;
summing MS-SSIM loss and MSE loss according to a certain weight ratio to form an integral consistency loss function, and realizing a phase-double resolution network model fnet1And fnet2While training at the same time.
Preferably, said complex valued wavefield UzThe calculation formula (2) includes:
preferably, the summing of the MS-SSIM loss and the MSE loss according to a certain weight ratio as an overall loss function includes:
Drawings
FIG. 1 shows an overall block diagram of the generation of a phase-only hologram based on a phase-dual resolution network according to the present application;
FIG. 2 shows a block diagram of a phase-dual resolution network of the present application;
FIG. 3 shows a holographic display structure schematic diagram and a simulation real object diagram constructed by the application
Detailed Description
The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which preferred embodiments of the invention are shown.
Holographic display technology is gradually applied to three-dimensional scene reconstruction, virtual reality and augmented display systems, and a three-dimensional display mode without glasses becomes possible. In the holographic display technology, a hologram is used as an information carrier and contains depth and intensity information of an object, so that the calculation of generating the hologram is a key step in the holographic display. The current method for generating the hologram by calculation mainly comprises an iterative optimization algorithm and a non-iterative optimization algorithm. However both algorithms have been a trade-off between the quality of the optimization of the hologram and the computation time for many years. With the gradual introduction of deep learning techniques into the field of optical research, many satisfactory results are obtained. The invention provides a neural network algorithm capable of calculating and generating a high-quality hologram in real time, and has important significance for the development of real-time holographic display technology.
Therefore, the application provides a method for rapidly generating the hologram based on the phase-dual resolution network. Firstly, introducing an angular spectrum method for a light field to propagate in a free space, wherein a calculation formula is as follows:
uz(x1,y1)=fAS{φ(x,y)}=IFFT{FFT{usrc(x,y)·eiφ(x,y)}·H(fx,fy)} (1)
wherein u issrcRepresenting the complex wave field, u, produced by a coherent light sourcezRepresenting the complex-valued wave field at the diffraction distance z, f (x, y) representing the phase-only hologram loaded into the spatial light modulator, FFT and IFFT being the Fourier transform and inverse Fourier transform operators, respectively, H (f)x, fy) Is an angular spectrum transformation function. H (f)x,fy) Is expressed as:
wherein f isxAnd fyIs the spatial frequency; λ is the wavelength of the light, z is the diffraction distance, H (f)x,fy) Is the transfer function of angular spectroscopy. From the formula (1), usrc(x, y) is a coherent light source that is content independent, so the angular spectroscopy conductivity is only that of H (f)x,fy) It is related. According to equation (2), the transfer function of angular spectroscopy is related to wavelength, propagation distance, pixel pitch, and spatial resolution. When the optical system is fixed, the four parameters are fixed, and the transfer function can be regarded as a constant in the back propagation of the convolutional neural network. Therefore, the calculation formula of the angular spectrum analysis method satisfies the gradient propagation. According to general approximate theorem of neural network, utilizing forward propagation of convolution neural network to produce pure phase hologram and using angular spectrum method to calculate holomorphismAnd (4) numerical reconstruction of the information map. This process can be expressed in the following form:
wherein I represents the target amplitude, fnetAn approximation function, phi, representing a convolutional neural networkholoA phase-only hologram is represented,representing a reproduced image. The process of calculating the gradient by the loss function and the angular spectrum method can be expressed as:
wherein wkAnd bkK-th layer trainable parameters, phi, representing a convolutional neural networkholoA hologram is shown.
Fig. 1 shows the overall structure of a phase-dual resolution network-based hologram fast generation method. The input is any one of intensity maps, the phase-dual resolution network f after trainingnet1Mapping it into a phase map; the phase map and the input target intensity map are calculated as a complex-valued wavefield U by equation (5)z(ii) a According to the angle spectrum method, the complex value wave field is propagated in free space by-z distance to obtain a complex value wave field U0(ii) a Due to the phase-dual resolution network fnet2Cannot perform complex operations, so the complex-valued wavefield U is transformed according to the Euler equation0Decomposing into a real part and an imaginary part, and performing matrix splicing operation; matrix-spliced feature matrix input phase-dual resolution network fnet2And calculating to obtain the hologram.
The above process is to use a trained phase-dual resolution network fnet1And fnet2Generation ofThe process of hologram. Since high quality holographic masks are difficult to obtain, the training process is accomplished by convolutional neural network training without a holographic mask. Phase-dual resolution network f in training phasenet2Calculating the hologram by angular spectrum methodEquation (3) can be further expressed here as:
in the formula (I), the compound is shown in the specification,represents a reproduced image phiholoRepresenting phase-only holograms, fASRepresenting angular spectroscopy, I representing the target amplitude. Respectively calculating an input target intensity map I and a reconstructed map calculated by an angular spectrum methodThe MS-SSIM loss and the MSE loss are added according to a certain proportion to be used as an integral consistency loss function, and the phase-double resolution ratio network model f is realizednet1And fnet2While training at the same time. The consistency loss function can be expressed as:
wherein I represents the amplitude of the target,a reconstructed image is shown which is,for calculating I andmultiple ruler betweenStructural similarity, p denotes pixel, for calculating the error pixel by pixel, and α is set to 0.84 empirically.
Fig. 2 shows a diagram of a phase-dual resolution network architecture. Wherein CBP represents a convolution module comprising a convolution layer, a batch normalization layer and a PReLU activation function. RB denotes a residual block and RBB denotes a residual bottleneck block. Dapm stands for deep polymerization pyramid pooling module. The phase-dual resolution network is mainly composed of an encoder and a decoder, wherein the encoder is composed of a high resolution branch and a low resolution branch.
The high resolution branch of the encoder may retain more high frequency features while the low resolution branch may extract high level abstract features of the target amplitude. The first two layers of the network of the encoder use group convolution for feature extraction. And (3) performing feature extraction on the rest network layers in the encoder by using a hole convolution, wherein a hole factor in the hole convolution follows a maximum distance calculation formula between two non-zero values in the convolution:
Mi=max[Mi+1-2ri,Mi+1-2(Mi+1-ri),ri],Mn=rn (8)
where i denotes the ith layer of convolution, MiRepresenting the maximum distance between non-zero values in the ith layer of the convolution output. r isiIndicating the hole factor of the i-th layer of the convolution. The calculation amount and GPU memory occupation when the phase-dual resolution ratio network generates the high-resolution hologram can be reduced by using the group convolution, and the receptive field enhanced feature extraction capability of convolution operation can be enlarged by using the hole convolution.
In the decoder, the features extracted by the encoder are upsampled by sub-pixel convolution until the resolution is the same as the input image resolution. The third layer network output of the encoder uses the first layer which is connected to the decoder through jumping to carry out feature splicing, and the feature decoding capability is enhanced. The 1 × 1 convolution is taken as the last layer of the decoder, so that the output is a single-channel image. The grey values of the single channel image are normalized to-pi, pi so that the final output of the network is a phase map.
FIG. 3 shows a holographic display structure schematic diagram and a simulation real object diagram constructed by the application. The model of the spatial light modulator is Holoeye Photonics AG, the pixel distance is 8 μm, the laser wavelength is 532nm, and the propagation distance is 40 cm.
Claims (5)
1. A phase-double resolution ratio network-based hologram rapid generation method is characterized in that:
(1) construction of phase-Dual resolution network model fnet1And fnet2;
(2) Trained phase-dual resolution network fnet1Calculating an initial phase phi according to the input target intensity map I0;
(3) According to the initial phase phi0Calculating a complex value wave field U with the target intensity graph Iz(ii) a From the angle spectrum method, a complex-valued wavefield U is calculatedzComplex wave field U obtained after propagation of-z in free space0;
(4) Trained phase-dual resolution network fnet2From the input complex-valued wavefield U0A phase-only hologram is calculated.
2. The method of claim 1, wherein constructing the phase-dual resolution network model comprises:
(1) in the encoder of the phase-dual resolution network, the first two layers of networks use group convolution to carry out feature extraction; and (3) performing feature extraction on the rest network layers in the encoder by using a hole convolution, wherein a hole factor in the hole convolution follows a maximum distance calculation formula between two non-zero values in the convolution:
Mi=max[Mi+1-2ri,Mi+1-2(Mi+1-ri),ri],Mn=rn;
(2) in a decoder, performing up-sampling on the features extracted by the encoder through sub-pixel convolution until the resolution is the same as that of an input image; the third layer network output of the encoder uses the first layer of the jump connection to the decoder for feature splicing; taking the 1 × 1 convolution as the last layer of the decoder, so that the output is a single-channel image;
(3) the gray values of the single channel images are normalized to [ -pi, pi ].
3. The method of claim 1, wherein the training of the trained phase-dual resolution network comprises:
(1) constructed phase-dual resolution network fnet2Calculating to generate phase-only hologram, and calculating the reconstructed image of the hologram according to angular spectrum methodWherein the reconstruction distance is z;
(2) respectively calculating an input target intensity map I and a reconstructed map calculated by an angular spectrum methodMS-SSIM loss and MSE loss in between;
(3) summing MS-SSIM loss and MSE loss according to a certain weight ratio to form an integral consistency loss function, and realizing a phase-double resolution network model fnet1And fnet2While training at the same time.
5. the method of claim 3, wherein the overall consistency loss function, the calculation formula comprises:
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210029238.1A CN114387395A (en) | 2022-01-11 | 2022-01-11 | Phase-double resolution ratio network-based quick hologram generation method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210029238.1A CN114387395A (en) | 2022-01-11 | 2022-01-11 | Phase-double resolution ratio network-based quick hologram generation method |
Publications (1)
Publication Number | Publication Date |
---|---|
CN114387395A true CN114387395A (en) | 2022-04-22 |
Family
ID=81201867
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210029238.1A Pending CN114387395A (en) | 2022-01-11 | 2022-01-11 | Phase-double resolution ratio network-based quick hologram generation method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114387395A (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114967398A (en) * | 2022-05-13 | 2022-08-30 | 安徽大学 | Large-size two-dimensional calculation hologram real-time generation method based on deep learning |
CN115097709A (en) * | 2022-07-05 | 2022-09-23 | 东南大学 | Holographic encoding method based on complex optimizer or complex solver |
CN115294412A (en) * | 2022-10-10 | 2022-11-04 | 临沂大学 | Real-time coal rock segmentation network generation method based on deep learning |
-
2022
- 2022-01-11 CN CN202210029238.1A patent/CN114387395A/en active Pending
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114967398A (en) * | 2022-05-13 | 2022-08-30 | 安徽大学 | Large-size two-dimensional calculation hologram real-time generation method based on deep learning |
CN114967398B (en) * | 2022-05-13 | 2024-05-31 | 安徽大学 | Large-size two-dimensional calculation hologram real-time generation method based on deep learning |
CN115097709A (en) * | 2022-07-05 | 2022-09-23 | 东南大学 | Holographic encoding method based on complex optimizer or complex solver |
CN115097709B (en) * | 2022-07-05 | 2023-11-17 | 东南大学 | Holographic coding method based on complex optimizer or complex solver |
CN115294412A (en) * | 2022-10-10 | 2022-11-04 | 临沂大学 | Real-time coal rock segmentation network generation method based on deep learning |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN114387395A (en) | Phase-double resolution ratio network-based quick hologram generation method | |
Liu et al. | 4K-DMDNet: diffraction model-driven network for 4K computer-generated holography | |
CN111915660B (en) | Binocular disparity matching method and system based on shared features and attention up-sampling | |
CN113467211B (en) | Holographic encoding method based on gradient reduction of spectral loss function | |
CN105976408A (en) | Digital holographic compression transmission method of quantum backward propagation nerve network | |
KR102605678B1 (en) | Method and apparatus for processing holographic image | |
CN114067015A (en) | Pure phase hologram generation method and system combining DNN | |
CN109447897B (en) | Real scene image synthesis method and system | |
Yu et al. | Phase dual-resolution networks for a computer-generated hologram | |
CN115797231A (en) | Real-time hologram generation method based on neural network of Fourier inspiration | |
CN115100300B (en) | Computer hologram compression method based on attention-based heuristic neural network | |
CN114841859A (en) | Single-image super-resolution reconstruction method based on lightweight neural network and Transformer | |
CN117876591A (en) | Real fuzzy three-dimensional hologram reconstruction method for combined training of multiple neural networks | |
Li et al. | Speckle noise suppression algorithm of holographic display based on spatial light modulator | |
US20240103439A1 (en) | Method and system for optimizing first-diffraction-order reconstruction of holograms, device and medium | |
CN113627487A (en) | Super-resolution reconstruction method based on deep attention mechanism | |
Rong et al. | Reconstruction efficiency enhancement of amplitude-type holograms by using Single-Scale Retinex algorithm | |
CN115690252A (en) | Hologram reconstruction method and system based on convolutional neural network | |
KR20210095433A (en) | Deep learning model for generating phase hologram at high speed and learning method thereof | |
Lee et al. | Complex hologram generation of multi-depth images using deep neural network | |
Liu et al. | Calculating real-time computer-generated holograms for holographic 3D displays through deep learning | |
CN113658330B (en) | Holographic encoding method based on neural network | |
CN115797181A (en) | Image super-resolution reconstruction method for mine fuzzy environment | |
Chen et al. | Real-time hologram generation using a non-iterative modified Gerchberg-Saxton algorithm | |
CN113538236A (en) | Image super-resolution reconstruction method based on generation countermeasure network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |