CN112528869B

CN112528869B - Phase-free data imaging method based on complex neural network

Info

Publication number: CN112528869B
Application number: CN202011470944.7A
Authority: CN
Inventors: 曾杰; 罗喜伶; 蒋淑园
Original assignee: Hangzhou Innovation Research Institute of Beihang University
Current assignee: Hangzhou Innovation Research Institute of Beihang University
Priority date: 2020-12-14
Filing date: 2020-12-14
Publication date: 2023-04-25
Anticipated expiration: 2040-12-14
Also published as: CN112528869A

Abstract

The invention discloses a phase-free data imaging method based on a complex neural network, and belongs to the technical field of imaging of electromagnetic inverse problems. Firstly, a real sample image is acquired, and phase-free total field data and ideal scattered field data are generated by the real sample image to form a training set. The method comprises the steps of establishing a complex domain Unet network, wherein the complex domain Unet network comprises a first branch and a second branch, the first branch comprises an imaginary part generating module and a first complex network, the second branch is a second complex network, and the complex network comprises a complex convolution module, a complex batch processing module and a complex activation function module. Training the first branch and the second branch respectively, cascading the first branch and the second branch after the training is finished, and further fine tuning. And generating a corresponding image by utilizing the trained complex domain Unet network aiming at the non-phase data to be imaged. The invention utilizes two branches in the complex domain Unet network to process the complex domain problem of electromagnetic backscattering, and has high precision and quick response.

Description

Phase-free data imaging method based on complex neural network

Technical Field

The invention belongs to the technical field of imaging of electromagnetic inversion problems, and particularly relates to a phase-free data imaging method based on a complex neural network.

Background

Electromagnetic backscattering is widely applied in the fields of microwave remote sensing, medical imaging, geological investigation and the like. The physical and geometric characteristic information of the unknown object is reconstructed through the reflected electromagnetic wave data, and the physical and geometric characteristic information mainly comprises the position, the shape, the dielectric constant and the like of the detection body. The greatest challenges of electromagnetic backscatter imaging problems arise from its discomfort and high degree of nonlinearity. The electromagnetic wave information obtained is a result of multiple scattering, refraction and diffraction and is not normally propagated in a simple path in the imaging region. The quantized imaging results require solving an overall system of nonlinear equations that are ill-conditioned, with non-unique and unstable solutions, i.e., small variations in data can lead to large deviations, which cause electromagnetic backscatter problems to be difficult to solve.

In practical situations, it is difficult to accurately measure the phase information of the scattered field data in the high frequency range, and in order to obtain the phase information, the cost of hardware is greatly increased, and new noise is inevitably introduced. Therefore, the inverse problem imaging research based on the phase-free data has important engineering practical application significance.

The imaging method of the inverse problem mainly includes a learning-based method and a model-based method. Conventional model-based iterative algorithms include, among others, iterative threshold contraction algorithms (Iterative Shrinkage/Thresholding Algorithm, ISTA), alternate direction multiplier method Alternating Direction Method of Multipliers, ADMM), and original dual hybrid gradient algorithms (Primal Dual Hybrid Gradient, PDHG). With the continuous progress in the field of machine learning, particularly the application of deep learning, neural networks are increasingly being used to solve the problem of nonlinear fitting in inverse imaging. The inverse imaging solution method using deep learning mainly has a method based on complete learning, that is, the mapping of measurement data to image data is learned only by fitting a large amount of training data, such end-to-end learning network solves inverse imaging in the form of a "black box", which requires that the network model must learn all the physical rules of inverse imaging, so that the learned network has poor portability; the second is an image preprocessing-based method. The network model is mainly input by an initial image processed by an iterative algorithm, so that the imaging time is prolonged, but the interpretability and the operability of the model are improved; third is a depth network based in part on a model. The neural network not only replaces the imaging optimization stage in the second method, but also directly replaces the initial imaging stage of the iterative algorithm. I.e. another neural network is used instead of the traditional iterative algorithm, as is the case with the learnable iterative threshold contraction algorithm.

When the prior art scheme faces to objects with larger detection target areas, the traditional high-precision calculation electromagnetic method (such as a moment method, a finite element method, a time domain finite difference method and the like) needs to consume a large amount of calculation resources, and is extremely long in time consumption. Although various approximation numerical methods (e.g., high frequency approximation, born approximation, etc.) improve the calculation speed to some extent, these methods tend to ignore electromagnetic interactions between targets, resulting in lower calculation accuracy. On the other hand, for electromagnetic backscatter problems, such as radar imaging, many reconstruction algorithms have been applied over the past decades. Such as backprojection algorithm, newton's iterative algorithm, genetic algorithm, etc. These algorithms are generally limited by processes such as point-by-point scanning imaging or iterative imaging, and when applied to a large-scale imaging scene, the calculation amount is huge, and the real-time requirement cannot be met.

In recent years, deep learning has been widely used in various fields to complete classification and regression tasks and achieve good results. At present, most mainstream networks are built based on real number domains, and electromagnetic scattering and backscattering problems are in complex number domains; secondly, due to the "black box" nature of the deep learning network, the lack of accurate physical support makes it difficult for existing deep learning frameworks to be directly applied to electromagnetic scattering and backscatter problems, even though training results are good, it is difficult to adapt to other different data sets.

Disclosure of Invention

In order to solve the problems that the high-precision electromagnetic calculation method in the prior art is large in calculated amount and cannot meet the application and real-time requirements in a large-scale imaging scene, and the existing deep learning network extremely depends on a training set and is poor in interpretability and adaptability, the invention provides a complex neural network-based phase-free data imaging method, which utilizes two branches in a complex domain Unet network to process the complex domain problem of electromagnetic backscatter, and is high in precision and quick in response.

In order to achieve the above purpose, the present invention adopts the following technical scheme:

a phase-free data imaging method based on a complex neural network comprises the following steps:

1) Collecting a real sample image, generating phase-free total field data and ideal scattered field data by the real sample image, and forming a training set by the phase-free total field data, the ideal scattered field data and the real sample image;

2) Establishing a double-branch complex domain Unet network, wherein a branch I comprises an imaginary part generating module and a first complex network, and a branch II is a second complex network, and the first complex network and the second complex network are composed of a complex convolution module, a complex batch processing module and a complex activation function module;

3) Training branch one:

inputting the non-phase total field data as a real part into a branch I of a complex domain Unet network, firstly generating an imaginary part corresponding to the real part through an imaginary part generating module, and combining the real part and the generated imaginary part as the input of a first complex network to obtain predicted scattered field data; calculating loss according to the predicted scattered field data and the ideal scattered field data, and training the branch I;

4) Training the second branch:

inputting ideal scattered field data into a branch II of a complex domain Unet network, processing by a second complex network, outputting complex data, and then taking a model to obtain a predicted image; calculating loss according to the predicted image and the real sample image, and training the second branch;

5) Performing fine tuning training on a complex domain Unet network:

the trained branch I and the branch II are cascaded, the predicted scattered field data which is predicted and output after the phase-free total field data passes through the branch I is used as the input of the branch II, and the predicted image data is output; calculating loss according to the predicted image and the real sample image, and performing fine adjustment on the first branch and the second branch which are cascaded in the complex domain Unet network to obtain a trained complex domain Unet network;

6) And generating a corresponding image by utilizing the trained complex domain Unet network aiming at the non-phase data to be imaged.

Compared with the prior art, the invention has the advantages that:

1) The problem of backscattering is characterized in that an iterative optimization algorithm, such as a FISTA algorithm, is usually adopted due to inadaptability and nonlinearity, but the iterative optimization algorithm mainly has the defects of long time consumption, difficulty in real-time construction and the like; although the non-iterative algorithm can complete the reconstruction work in a short time, the accuracy is not high, especially for objects with high dielectric constants. Algorithms based on artificial neural networks tend to be only inadaptable for objects of specific shape, position and size.

The total field data phase recovery algorithm based on deep learning mainly adopts a real network, and has difficulty in achieving a better fitting effect on scattered field data output in a complex value form; processing the complex value data of the scattered field by a sub-channel, and neglecting the internal relation between the real part and the imaginary part of the complex value data; therefore, the invention provides an imaging algorithm based on a Unet model of a complex neural network, wherein the Unet core of the complex neural network is to replace a real convolutional neural network by a complex convolution neural network based on a real part and a real part, a real part and an imaginary part, an imaginary part and a real part and an imaginary part of a complex basic algorithm, and in addition, the complex convolution neural network uses real data to generate imaginary data, and complex output results are subjected to modulo processing and the like to improve the imaging quality.

2) According to the method, step-by-step training is adopted, scattered field data is introduced in the training process of the network to guide the optimal descending direction, and the method has more controllability and interpretability for the end-to-end training mode of directly recovering the total field data to the image; and finally, achieving better phase recovery and data imaging in a cascade fine tuning mode.

3) Finally, the Unet network based on the complex neural network can well complete recovery from total field data to scattered field data and mapping from the scattered field data to image data, meanwhile, good contour information and prediction of dielectric constants are reserved, and stronger robustness is achieved.

Drawings

FIG. 1 is a flow chart of a method for phase-free data imaging based on a complex neural network according to the present invention;

FIG. 2 is a schematic diagram of a branching process in a complex-domain Unet network according to the present invention;

FIG. 3 is a schematic diagram of a branching flow in a complex-domain Unet network according to the present invention;

FIG. 4 is a schematic diagram of a cascaded complex domain Unet network according to the present invention;

FIG. 5 is a graph of test results with Emnist as the test set;

fig. 6 is a graph of test results with Austria as the test set;

FIG. 7 is a graph of the results of testing Austria for different dielectric constants;

FIG. 8 is a schematic diagram of a branch structure in the present embodiment;

fig. 9 is a schematic diagram of a branch two structure in the present embodiment.

Detailed Description

The invention is further illustrated and described below in connection with specific embodiments. The technical features of the embodiments of the invention can be combined correspondingly on the premise of no mutual conflict.

The invention provides a phase-free data imaging method based on a complex neural network, which is provided with an arrangement flow chart shown in figure 1 and mainly comprises the following steps:

step one: collecting a real sample image, generating phase-free total field data and ideal scattered field data by the real sample image, and forming a training set by the phase-free total field data, the ideal scattered field data and the real sample image;

step two: establishing a double-branch complex domain Unet network, wherein a branch I comprises an imaginary part generating module and a first complex network, and a branch II is a second complex network, and the first complex network and the second complex network are composed of a complex convolution module, a complex batch processing module and a complex activation function module;

step three: training branch one:

step four: training the second branch:

step five: performing fine tuning training on a complex domain Unet network:

step six: and generating a corresponding image by utilizing the trained complex domain Unet network aiming at the non-phase data to be imaged.

The following describes the specific steps of the present invention in detail.

The dimensions of the real sample images in the training set are uniform, and the dimensions of the phase-free total field data and the scattered field data generated by the real sample images are uniform. In this embodiment, step one acquires a total of 5000 real sample images, wherein the non-phase total field data and the ideal scattered field data are generated from the real sample images under ideal conditions, and the dimensions are (32,16) and (2,32,16), respectively; the dielectric constant of the real image data ranges from 1.0 to 2.0.

In this embodiment, the generation process of the non-phase total field data and the ideal scattered field data adopts the prior art means in the field, and the specific operation includes firstly, establishing a model, namely, defining a solving equation, establishing model geometry, defining material properties, establishing a metal boundary and a radiation boundary, determining the position relationship between the model and the model, and determining the number of transmitting antennas and receiving antennas; wherein the material properties include relative permeability, relative permittivity and conductivity; then subdividing grids, discretizing a model space by using a finite element method, wherein when solving an electromagnetic wave problem, the grid resolution is critical when dealing with the related problem of the wave type, and only the grids are fine enough to analyze the wavelengths in all media; finally, a set of linear equations describing the electric field are solved in the analog domain, and useful information is extracted through the calculated electric field result. The non-phase total field data, the ideal scattered field data and the real sample image together form a training set.

In this embodiment, the complex domain Unet network established in the second step includes a first branch and a second branch; firstly, generating imaginary part data through an imaginary part generating module, combining the imaginary part data with the real part data to obtain complex data, inputting the complex data into a complex network for processing to obtain predicted data; the second branch also adopts the same complex network as the first branch. The complex network comprises a complex convolution module, a complex batch processing module and a complex activation function module. The complex convolution module carries out convolution operation on the real number part and the imaginary number part of the input complex data respectively, the convolution process takes complex basic operation as a criterion, and the final result is output in a complex form.

The first and second branches of this embodiment are schematically shown in fig. 2 and 3.

The branch I is mainly used for phase recovery, inputs are non-phase data with the dimension of (1, 32, 16), and data with the dimension of (2, 32,16) are obtained by combining the non-phase data with the dimension of (1, 32, 16) with the real input through the imaginary part generating module and serve as the input of a complex Unet network, and output is scattered field data with the dimension of (2, 32, 16) predicted.

And the second branch is mainly used for scattered field imaging, inputs the scattered field data under the ideal condition with the dimension of (2,32,16), outputs complex data with the dimension of (1,32,16) after passing through a complex UNet network, and obtains image data with the dimension of (1,32,32) by taking the complex modulus. The cascade network is to cascade the trained first and second branch networks, i.e. the input of the second branch is the output of the first branch.

In one implementation of the invention, both the real and complex activation function modules employ a Relu activation function, wherein the complex activation function module applies an activation function on the real and generated imaginary parts, respectively.

The complex convolution module adopts a Unet network and comprises a downsampling coding structure formed by a plurality of downsampling layers, an upsampling decoding structure formed by a plurality of upsampling layers and jump connection; the number of the downsampling layers is equal to that of the upsampling layers; the number of the downsampling layers and the upsampling layers in the complex convolution module of the first branch is recorded as n, and the number of the downsampling layers and the upsampling layers in the complex convolution module of the second branch is recorded as m;

where downsampling is a compressed process, i.e., encoder. The image size is reduced by convolution and downsampling to extract some shallow features. Upsampling is the process of decoding, i.e., decoder. Some deep features are obtained by convolution and upsampling. The complex domain Unet network is realized by constructing a complex convolution module, a complex batch processing module and a complex activation function to replace a real Unet network.

In a complex convolution module of the first branch, combining a real part corresponding to the phase-free total field data with a generated imaginary part to serve as an input of a first-layer downsampling layer, taking an output of a last downsampling layer as an input of a next downsampling layer, and taking an output of a last downsampling layer as an input of a first-layer upsampling; the output of the i-th layer lower sampling layer and the output of the n-i-th layer upper sampling layer are fused in a jump connection mode, and the fusion result is used as the input of the next layer upper sampling layer until the output of the last layer upper sampling layer is obtained as the output of the complex convolution module; in this embodiment, as shown in fig. 8, n is 3.

In the complex convolution module of the second branch, firstly taking scattered field data as input of a first layer of downsampling layer, then taking output of a last layer of downsampling layer as input of a next layer of downsampling layer, and finally taking output of the last layer of downsampling layer as input of the first layer of upsampling; and the output of the i-th layer down-sampling layer and the output of the m-i-th layer up-sampling layer are fused in a jump connection mode, and the fusion result is used as the input of the next layer up-sampling layer until the output of the last layer up-sampling layer is obtained as the output of the complex convolution module. In this embodiment, as shown in fig. 9, m is 3.

In the complex convolution module, a complex two-dimensional convolution network is adopted for the downsampling layer and the upsampling layer. The two-dimensional complex convolution operation is mainly that a complex weight matrix W=A+iB is defined firstly, and the input h=x+iy of a complex convolution layer is different from a real number domain convolution network, and the complex convolution network requires the input to be complex; the complex weight matrix is then convolved with the input matrix, as shown in the following equation, where x represents the convolution operation.

w*h＝(A+iB)*(x+iy)＝(A*x-B*y)+i(B*x+A*y)

The written matrix form is shown in the following formula, wherein R (W.times.h), I (W.times.h) represents the real part and the imaginary part of the output:

wherein, R (w×h) represents the real part of the complex two-dimensional convolution network output, I (w×h) represents the imaginary part of the complex two-dimensional convolution network output, W is a complex weight matrix, a is the real part in the complex weight, B is the imaginary part in the complex weight, h represents the input of the complex two-dimensional convolution network, x and y represent the real part and the imaginary part of h respectively, and x represents the convolution operation.

And training the first branch and the second branch in the network structure respectively, and fine-tuning the whole structure. The specific implementation process is as follows:

1) The first stage of network training is phase recovery.

The input is phase-free total field data, and the output is predicted scattered field data, specifically:

the method comprises the steps of inputting the non-phase total field data as a real part to an imaginary part generation module, generating a corresponding imaginary part, combining the real part with the generated imaginary part, and obtaining predicted scattered field data after a complex convolution module, a complex batch processing module and a complex activation function module respectively as input of a complex network.

Branch one trained Loss function is the square of the difference between the true and predicted values (MSE Loss); the training iteration number is 200, the input sample number of each round is 20, and the optimizer is Adam.

2) The second phase is a fringe field imaging phase.

And inputting the data as the scattered field data under ideal conditions, and performing modulo processing on complex output of the complex Unet to obtain final predicted sample imaging data. The method comprises the following steps: and inputting the ideal scattered field data into a branch II in the complex domain Unet network, processing the ideal scattered field data under the complex domain by a complex convolution module, a complex batch processing module and a complex activation function module respectively, and taking the modulus of the complex data output by the branch II to obtain a predicted image.

The Loss function of the branch two training consists of the square of the difference between the true and predicted values (MSE) and the structural similarity Loss function (SSIM Loss). When SSIM of the sample true value and the predicted value is calculated, respectively carrying out normalization processing on the sample true value and the predicted value to ensure that the pixel value is in the range of 0-1. The training super parameters are the same as above.

3) The third stage is cascade fine tuning training.

And cascading the models trained in the first stage and the second stage, namely predicting output scattered field data after no-phase data pass through the first branch as the input of the second branch, and outputting the predicted output scattered field data as image data.

In the training process, when the error of the test set is smaller than the threshold value 0.001, the stage three training is considered to achieve the purpose of fine tuning, and the training is ended, so that a trained complex domain Unet network is obtained. The training iteration number is 20, the input sample number of each round is 20, the optimizer is SGD, the learning rate is 0.01, and the momentum coefficient is 0.9.

4) The model after the cascade training is the final prediction model, and for the non-phase data to be imaged, as shown in fig. 4, the non-phase data is used as the input of the first branch in the trained complex domain Unet network, the predicted scattered field data is obtained and used as the input of the second branch, and the complex data output by the second branch is subjected to modulo obtaining the generated image.

In one embodiment of the invention, the test set Emnist, austraia, is used to perform generalization performance detection on the network in sequence; emnist, austria test sets were 100 and 10, respectively, with dielectric constants in the range of 1-2. The test results in this embodiment are compared and analyzed with the real network, the BPS algorithm, and the bps+unet network, respectively, as shown in fig. 5, 6, and 7. Wherein fig. 5 is a test set Emnist result, fig. 6 is a test set Austria result, and fig. 7 is a result of testing Austria for different dielectric constants.

Wherein the first behavior of fig. 5 and 6 is a real network output image; the second behavior BPS algorithm outputs an image; the third line BPS+UNet outputs an image, and the BPS and the scattered field of the second line are taken as inputs; a fourth behavior complex network image; the fifth act as a true sample image;

the numbers above the image represent the structural similarity SSIM loss, mean square error MSE loss, mean absolute error MAE loss, respectively.

The foregoing examples illustrate only a few embodiments of the invention and are described in detail herein without thereby limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.

Claims

1. The phase-free data imaging method based on the complex neural network is characterized by comprising the following steps of:

in a complex convolution module of the first branch, combining a real part corresponding to the phase-free total field data with a generated imaginary part to serve as an input of a first-layer downsampling layer, taking an output of a last downsampling layer as an input of a next downsampling layer, and taking an output of a last downsampling layer as an input of a first-layer upsampling; the output of the i-th layer lower sampling layer and the output of the n-i-th layer upper sampling layer are fused in a jump connection mode, and the fusion result is used as the input of the next layer upper sampling layer until the output of the last layer upper sampling layer is obtained as the output of the complex convolution module;

in the complex convolution module of the second branch, firstly taking scattered field data as input of a first layer of downsampling layer, then taking output of a last layer of downsampling layer as input of a next layer of downsampling layer, and finally taking output of the last layer of downsampling layer as input of the first layer of upsampling; the output of the ith layer of lower sampling layer and the output of the mth-ith layer of upper sampling layer are fused in a jump connection mode, and the fusion result is used as the input of the next layer of upper sampling layer until the output of the last layer of upper sampling layer is obtained as the output of the complex convolution module;

in the complex convolution module, a complex two-dimensional convolution network is adopted by a downsampling layer and an upsampling layer, and a calculation formula is as follows:

wherein R (W.times.h) represents the real part of the complex two-dimensional convolution network output, I (W.times.h) represents the imaginary part of the complex two-dimensional convolution network output, W is a complex weight matrix, A is the real part in complex weight, B is the imaginary part in complex weight, h represents the input of the complex two-dimensional convolution network, x and y represent the real part and the imaginary part of h respectively, and x represents convolution operation;

3) Training branch one:

4) Training the second branch:

5) Performing fine tuning training on a complex domain Unet network:

6) Aiming at the non-phase data to be imaged, taking the non-phase data as the input of a first branch in a trained complex domain Unet network, obtaining predicted scattered field data, taking the predicted scattered field data as the input of a second branch, and taking the modulus of the complex data output by the second branch to obtain a generated image.

2. The method of phase-free data imaging based on a complex neural network of claim 1, wherein the real sample image of step 1) has a dielectric constant in the range of 1.0-2.0.

3. The method of claim 1, wherein the complex activation function module applies the activation function in real and imaginary parts, respectively.

4. The method of claim 1, wherein the loss function uses a mean square error loss when training the pair of branches.

5. The method of claim 1, wherein the sum of the structural similarity loss and the mean square error loss is taken as the total loss when training the two branches.

6. The method for phase-free data imaging based on a complex neural network according to claim 5, wherein when the structural similarity loss is calculated, respectively performing normalization processing on the predicted image and the real sample image to enable the pixel value to be in a range of 0-1.

7. The complex neural network-based non-phase data imaging method of claim 1, wherein the dimensions of the real sample images in the training set are uniform, and the non-phase total field data and the fringe field data generated from the real sample images are uniform.