CN112528869A

CN112528869A - Phase-free data imaging method based on complex neural network

Info

Publication number: CN112528869A
Application number: CN202011470944.7A
Authority: CN
Inventors: 曾杰; 罗喜伶; 蒋淑园
Original assignee: Hangzhou Innovation Research Institute of Beihang University
Current assignee: Hangzhou Innovation Research Institute of Beihang University
Priority date: 2020-12-14
Filing date: 2020-12-14
Publication date: 2021-03-19
Anticipated expiration: 2040-12-14
Also published as: CN112528869B

Abstract

The invention discloses a phase-free data imaging method based on a complex neural network, and belongs to the technical field of imaging of electromagnetic inverse problems. Firstly, a real sample image is collected, and phase-free total field data and ideal scattered field data are generated from the real sample image to form a training set. The method comprises the steps of establishing a complex field Unet network, wherein the complex field Unet network comprises a first branch and a second branch, the first branch comprises an imaginary part generating module and a first complex network, the second branch is a second complex network, and the complex network comprises a complex convolution module, a complex batch processing module and a complex activation function module. And training the first branch and the second branch respectively, and after the training is finished, cascading the first branch and the second branch and further finely adjusting. And generating a corresponding image by using the trained complex field Unet network aiming at the phase-free data to be imaged. The invention processes the complex field problem of electromagnetic backscattering by using two branches in the complex field Unet network, and has high precision and quick response.

Description

Phase-free data imaging method based on complex neural network

Technical Field

The invention belongs to the technical field of imaging of electromagnetic inverse problem, and particularly relates to a phase-free data imaging method based on a complex neural network.

Background

Electromagnetic backscattering is widely applied to the fields of microwave remote sensing, medical imaging, geological exploration and the like. Specifically, the physical and geometric characteristic information of the unknown object is reconstructed through the reflected electromagnetic wave data, and the physical and geometric characteristic information mainly comprises the position, the shape, the dielectric constant and the like of a probe. The greatest challenge of the electromagnetic backscatter imaging problem arises from its unsuitability and high degree of non-linearity. The resulting electromagnetic information is the result of multiple scattering, refraction, and diffraction and generally does not travel a simple path within the imaging region. The quantified imaging results require solving all the non-linear equations that are ill-conditioned with non-unique and unstable solutions, i.e., small changes in data can lead to large deviations that make the electromagnetic backscattering problem difficult to solve.

In practical situations, it is difficult to accurately measure the phase information of the scattered field data in the high frequency range, and the overhead of hardware cost is greatly increased to obtain the phase information, and new noise is inevitably introduced. Therefore, the inverse problem imaging research based on the phase-free data has important engineering practical application significance.

Imaging methods for inverse problems mainly include learning-based methods and model-based methods. The conventional model-based Iterative Algorithm includes an Iterative threshold Shrinkage Algorithm (ISTA), an Alternating Direction Method of Multipliers (ADMM), and a Primal Dual Hybrid Gradient Algorithm (PDHG). With the continuous progress of the field of machine learning, especially the application of deep learning, neural networks are increasingly used to solve the problem of nonlinear fitting in inverse imaging. The inverse imaging solution method utilizing deep learning mainly includes a complete learning-based method, namely, the mapping from the measured data to the image data is learned only through the fitting of a large amount of training data, and the end-to-end learning network solves inverse imaging in a black box form, which requires that a network model must learn all inverse imaging physical rules, so that the portability of the learned network is poor; the second is a method based on image pre-processing. The input of the network model is an initial image processed by an iterative algorithm, so that although the imaging time is prolonged, the interpretability and operability of the model are increased; and the third is a deep network based in part on the model. The neural network not only replaces the imaging optimization stage in the second method, but also directly replaces the initial imaging stage of the iterative algorithm. I.e. replacing the conventional iterative algorithm with another neural network, as is the case with the learnable iterative threshold-shrinking algorithm.

When the prior art scheme faces to detect an object with a large target area, the traditional high-precision electromagnetic calculation method (such as a moment method, a finite element method, a time-domain finite difference method and the like) needs to consume a large amount of calculation resources, and the time consumption is extremely long. Although various approximation numerical methods (e.g., high frequency approximation method, Born approximation method, etc.) increase the calculation speed to some extent, these methods often ignore electromagnetic mutual coupling between objects, resulting in lower calculation accuracy. On the other hand, for electromagnetic backscattering problems, such as radar imaging, etc., many reconstruction algorithms have been applied during the last decades. Such as a backprojection algorithm, a newton's iteration algorithm, a genetic algorithm, etc. These algorithms are usually limited to processes such as point-by-point scanning imaging or iterative imaging, and when applied to large-scale imaging scenes, the calculation amount is often huge, and the real-time requirement cannot be met.

In recent years, deep learning has been widely used in a plurality of fields to complete classification and regression tasks and achieve good results. At present, most mainstream networks are built based on a real number domain, and the problems of electromagnetic scattering and backscattering are in a complex number domain; secondly, due to the 'black box' attribute of the deep learning network, accurate physical support is lacked, so that the existing deep learning framework is difficult to be directly applied to the problems of electromagnetic scattering and inverse scattering, and even if the training result is better, the deep learning framework is difficult to adapt to other different data sets.

Disclosure of Invention

In order to solve the problems that in the prior art, a high-precision electromagnetic calculation method is large in calculation amount and cannot meet the application and real-time requirements in a large-scale imaging scene, and an existing deep learning network is extremely dependent on a training set and is poor in interpretability and adaptability, the invention provides a phase-free data imaging method based on a complex neural network.

In order to achieve the purpose, the invention adopts the following technical scheme:

a phase-free data imaging method based on a complex neural network comprises the following steps:

1) acquiring a real sample image, generating phase-free total field data and ideal scattered field data by the real sample image, and forming a training set by the phase-free total field data, the ideal scattered field data and the real sample image;

2) establishing a dual-branch complex domain Unet network, wherein a branch I comprises an imaginary part generating module and a first complex network, a branch II is a second complex network, and the first complex network and the second complex network are respectively composed of a complex convolution module, a complex batch processing module and a complex activation function module;

3) training branch one:

inputting the phase-free total field data into a first branch of a complex-domain Unet network as a real part, firstly generating an imaginary part corresponding to the real part through an imaginary part generating module, and combining the real part and the generated imaginary part to be used as the input of a first complex network to obtain predicted scattered field data; calculating loss according to the predicted scattered field data and the ideal scattered field data, and training the branch I;

4) and training branch two:

inputting ideal scattered field data into a second branch of the Unet network in the complex field, processing the ideal scattered field data by a second complex network, outputting complex data, and then performing modulus acquisition to obtain a predicted image; calculating loss according to the predicted image and the real sample image, and training the branch two;

5) fine tuning training is carried out on the Unet network in the complex field:

cascading the trained branch I and the trained branch II, and outputting predicted scattered field data which is predicted and output after the phase-free total field data passes through the branch I as the input of the branch II to output predicted image data; calculating loss according to the predicted images and the real sample images, and finely adjusting the first branch and the second branch after cascading in the complex field Unet network to obtain a trained complex field Unet network;

6) and generating a corresponding image by using the trained complex field Unet network aiming at the phase-free data to be imaged.

Compared with the prior art, the invention has the advantages that:

1) due to the fact that the inverse scattering problem has inadaptability and nonlinearity, an iterative optimization algorithm such as a FISTA algorithm is generally adopted, but the iterative optimization algorithm mainly has the defects of long time consumption, difficulty in application of real-time construction and the like; the non-iterative algorithm can complete the reconstruction in a short time, but has low accuracy, especially for objects with high dielectric constants. The algorithm based on the artificial neural network is usually only used for objects with specific shapes, positions and sizes, and is not strong in adaptability.

The total field data phase recovery algorithm based on deep learning mainly adopts a real number network, and a good fitting effect on scattered field data which is output in a complex value form is difficult to achieve; for the scattered field complex value data, processing is carried out by channels, and the internal relation between the real part and the imaginary part of the complex value data is ignored; therefore, the invention provides an imaging algorithm of a Unet model based on a complex neural network, wherein the Unet core of the complex neural network adopts a rewinding product neural network based on real parts and real parts, real parts and imaginary parts, imaginary parts and real parts, and imaginary parts of a complex basic algorithm to replace a real convolution neural network, and in addition, real data is used for generating imaginary data, and the imaging quality is improved in modes of complex output result modulus processing and the like.

2) The method adopts step-by-step training, and introduces scattered field data to guide the optimal descending direction in the training process of the network, so that the method has higher controllability and interpretability for the end-to-end training mode that the total field data is directly restored to the image; and finally, better phase recovery and data imaging are achieved in a cascading fine adjustment mode.

3) Finally, the Unet network based on the complex neural network can well complete the recovery from the total field data to the scattered field data and the mapping from the scattered field data to the image data, and meanwhile, better prediction of profile information and dielectric constant is kept, so that the method has stronger robustness.

Drawings

FIG. 1 is a flow chart of a complex neural network-based phase-free data imaging method of the present invention;

FIG. 2 is a schematic diagram illustrating a branch flow in a multiple domain Unet network according to the present invention;

FIG. 3 is a schematic diagram of a second branch flow in a Unet network of plural domains according to the present invention;

fig. 4 is a schematic diagram of a cascaded plural-domain Unet network according to the present invention;

FIG. 5 is a test result chart with Emnist as the test set;

FIG. 6 is a graph of the test results using Austria as the test set;

FIG. 7 is a graph showing the results of testing Austria for different dielectric constants;

FIG. 8 is a schematic diagram of a branching structure in the present embodiment;

fig. 9 is a schematic diagram of the second branch structure in this embodiment.

Detailed Description

The invention will be further illustrated and described with reference to specific embodiments. The technical features of the embodiments of the present invention can be combined correspondingly without mutual conflict.

The invention provides a phase-free data imaging method based on a complex neural network, which is characterized in that a finishing flow chart is shown in figure 1, and the method mainly comprises the following steps:

the method comprises the following steps: acquiring a real sample image, generating phase-free total field data and ideal scattered field data by the real sample image, and forming a training set by the phase-free total field data, the ideal scattered field data and the real sample image;

step two: establishing a dual-branch complex domain Unet network, wherein a branch I comprises an imaginary part generating module and a first complex network, a branch II is a second complex network, and the first complex network and the second complex network are respectively composed of a complex convolution module, a complex batch processing module and a complex activation function module;

step three: training branch one:

step four: and training branch two:

step five: fine tuning training is carried out on the Unet network in the complex field:

step six: and generating a corresponding image by using the trained complex field Unet network aiming at the phase-free data to be imaged.

The following is a detailed description of specific embodiments of the present invention.

The sizes of real sample images in the training set are uniform, and the dimensions of phase-free total field data and scattered field data generated by the real sample images are uniform. In this embodiment, in the first step, 5000 real sample images in total are acquired, where the phase-free total field data and the ideal scattered field data are generated from the real sample images under ideal conditions, and the dimensions are (32,16) and (2,32,16), respectively; the dielectric constant of the real image data ranges from 1.0 to 2.0.

In this embodiment, the generation process of the phase-free total field data and the ideal scattered field data adopts the means of the prior art in the field, and the specific operations are firstly to establish a model, namely, to define a solution equation, to establish a model geometry, to define material properties, to establish a metal boundary and a radiation boundary, to determine the position relationship between the model and the model, and to determine the number of transmitting and receiving antennas; wherein the material properties include relative permeability, relative permittivity and electrical conductivity; then, subdividing the grids, discretizing the model space by using a finite element method, and solving the electromagnetic wave problem, wherein the grid resolution is crucial in processing the relevant problems of the wave types, and the wavelengths in all the media can be analyzed only if the grids are fine enough; and finally, solving a group of linear equations describing the electric field in a simulation domain, and finally extracting useful information through the electric field result obtained through calculation. And (3) forming a training set by the phase-free total field data, the ideal scattered field data and the real sample image together.

In this embodiment, the plural domain Unet network established in step two includes a branch one and a branch two; the first branch generates imaginary part data through an imaginary part generating module, the imaginary part data and the real part data are combined to obtain complex data, and the complex data are input into a complex network to be processed to obtain predicted data; branch two also employs the same complex network as branch one. The complex networks comprise complex convolution modules, complex batch processing modules and complex activation function modules. The complex convolution module carries out convolution operation on real number and imaginary number parts of input complex data respectively, the convolution process takes complex basic operation as a criterion, and the final result is output in a complex form.

The first branch and the second branch in the present embodiment are schematically shown in fig. 2 and 3.

The first branch is mainly used for phase recovery, phase-free data with the dimensionality of (1,32,16) is input, data with the dimensionality of (2,32,16) is obtained through combination of an imaginary part generation module and a real part input and serves as input of a complex Unet network, and scattered field data with the dimensionality of (2,32,16) is output.

And the second branch is mainly used for scattered field imaging, scatterfield data under the ideal condition with the dimension of (2,32,16) is input, the scattered field data passes through a complex UNet network, the complex data with the dimension of (1,32,16) is output, and the image data with the dimension of (1,32,32) is obtained by performing modulus on the complex data. The cascade network cascades the trained branch one network and branch two network, namely the input of the branch two is the output of the branch one.

In one embodiment of the present invention, the real activation function module and the complex activation function module each employ a Relu activation function, wherein the complex activation function module applies the activation function in the real part and the generated imaginary part, respectively.

The complex convolution module adopts a Unet network and comprises a down-sampling coding structure consisting of a plurality of down-sampling layers, an up-sampling decoding structure consisting of a plurality of up-sampling layers and jump connection; the number of the down-sampling layers and the up-sampling layers is equal; recording the number of down-sampling layers and up-sampling layers in the complex convolution module of the branch I as n, and recording the number of down-sampling layers and up-sampling layers in the complex convolution module of the branch II as m;

wherein the downsampling is a process of compression, i.e., Encoder. The image size is reduced by convolution and downsampling, extracting some of the light-appearing features. Upsampling is the process of decoding, i.e., Decoder. Some deep level features are obtained by convolution and upsampling. The complex-domain Unet network replaces a real Unet network by constructing a complex convolution module, a complex batch processing module and a complex activation function.

In a complex convolution module of a branch I, combining a real number part corresponding to phase-free total field data with a generated imaginary number part to serve as the input of a first layer of down-sampling layer, then taking the output of an upper layer of down-sampling layer as the input of a lower layer of down-sampling layer, and taking the output of a last layer of down-sampling layer as the input of the first layer of up-sampling; the output of the i-th down-sampling layer and the output of the n-i-th up-sampling layer are fused in a jump connection mode, and the fused result is used as the input of the next up-sampling layer until the output of the last up-sampling layer is obtained and used as the output of the complex convolution module; in this embodiment, as shown in fig. 8, n is 3.

In a complex convolution module of a branch two, firstly, scattered field data is used as the input of a first layer of down-sampling layer, then the output of an upper layer of down-sampling layer is used as the input of a lower layer of down-sampling layer, and finally the output of a lower layer of down-sampling layer is used as the input of the first layer of up-sampling; and the output of the i-th down-sampling layer and the output of the m-i-th up-sampling layer are fused in a jump connection mode, and the fused result is used as the input of the next up-sampling layer until the output of the last up-sampling layer is obtained and used as the output of the complex convolution module. In this embodiment, as shown in fig. 9, m is 3.

In the complex convolution module, a complex two-dimensional convolution network is adopted by the down-sampling layer and the up-sampling layer. The operation of two-dimensional complex convolution is mainly characterized in that firstly, a complex weight matrix W is defined as A + iB, and the input h of a complex convolution layer is defined as x + iy, which is different from a real number domain convolution network, wherein the complex convolution network requires the input of complex numbers; then, the complex weight matrix is convolved with the input matrix, as shown in the following formula, wherein x represents the convolution operation.

w*h＝(A+iB)*(x+iy)＝(A*x-B*y)+i(B*x+A*y)

Written in matrix form is shown below, where R (W h), I (W h) represent the real and imaginary parts of the output:

wherein, R (W h) represents the real part of the complex two-dimensional convolution network output, I (W h) represents the imaginary part of the complex two-dimensional convolution network output, W is a complex weight matrix, A is the real part of the complex weight, B is the imaginary part of the complex weight, h represents the input of the complex two-dimensional convolution network, x and y respectively represent the real part and the imaginary part of h, and x represents convolution operation.

And respectively training the first branch and the second branch in the network structure, and finely adjusting the whole structure. The specific implementation process is as follows:

1) the first phase of network training is phase recovery.

The input is phase-free total field data, and the output is predicted scattered field data, specifically:

the method comprises the steps of inputting phase-free total field data serving as a real part into an imaginary part generating module to generate a corresponding imaginary part, combining the real part with the generated imaginary part to serve as the input of a complex network, and obtaining predicted scattered field data after the real part and the generated imaginary part are respectively processed by a complex convolution module, a complex batch processing module and a complex activation function module.

The Loss function of the branch one training is the square of the difference value (MSE Loss) of the real value and the predicted value; the number of training iterations is 200, the number of input samples per round is 20, and the optimizer is Adam.

2) The second stage is a fringe field imaging stage.

And (3) inputting scattered field data under an ideal condition, and performing modulus processing on the complex output of the complex Unet to serve as final prediction sample imaging data. The method specifically comprises the following steps: and inputting the ideal scattered field data into a second branch of the Unet network in the complex field, processing the ideal scattered field data in the complex field by a complex convolution module, a complex batch processing module and a complex activation function module respectively, and performing modulus extraction on the complex data output by the second branch to obtain a predicted image.

The Loss function of the branch two training consists of the square (MSE) of the difference value between the real value and the predicted value and a structure similarity Loss function (SSIM Loss). When the SSIM of the real value and the predicted value of the sample is calculated, normalization processing is respectively carried out on the SSIM, and the pixel value is enabled to be in the range of 0-1. The training hyper-parameters are as above.

3) The third stage is cascaded fine tuning training.

And (4) cascading the models trained in the first stage and the second stage, namely predicting and outputting scattered field data as the input of the second branch after the phase-free data passes through the first branch, and outputting the scattered field data as image data.

And in the training process, when the error of the test set is smaller than the threshold value of 0.001, the purpose of fine adjustment is considered to be achieved in the stage three training, and the training is finished to obtain the trained complex field Unet network. The number of training iterations is 20, the number of input samples per round is 20, the optimizer is SGD, the learning rate is 0.01, and the momentum coefficient is 0.9.

4) The model after training after the cascade connection is the final prediction model, and for the non-phase data to be imaged, as shown in fig. 4, the non-phase data is used as the input of a first branch in the trained complex field Unet network to obtain predicted scatter field data and is used as the input of a second branch, and the complex data output by the second branch is subjected to modulus operation to obtain a generated image.

In one embodiment of the invention, the test sets Emnist and Austria are used in sequence to carry out generalization performance detection on the network; the Emnist and Austria test sets are 100 and 10 pieces respectively, and the dielectric constant ranges from 1 to 2. The test result of this embodiment is compared and analyzed with the real number network, the BPS algorithm, and the BPS + uet network, respectively, as shown in fig. 5, 6, and 7. Wherein fig. 5 is the test set Emnist results, fig. 6 is the test set Austria results, and fig. 7 is the results of testing Austria for different dielectric constants.

Wherein, the first behavior in fig. 5 and 6 is a real number network output image; outputting an image by a second behavior BPS algorithm; the third line is BPS + UNet output image, and the BPS and the scattered field of the second line are used as input; the fourth line is a plurality of network images; the fifth element is a real sample image;

the numbers above the image represent the structural similarity SSIM loss, mean square error MSE loss and mean absolute error MAE loss respectively.

The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A phase-free data imaging method based on a complex neural network is characterized by comprising the following steps:

3) training branch one:

4) and training branch two:

2. The complex neural network-based phase-free data imaging method according to claim 1, wherein the dielectric constant of the real sample image in step 1) is in a range of 1.0-2.0.

3. The complex neural network-based phase-free data imaging method of claim 1, wherein the complex activation function module applies activation functions in real and imaginary parts, respectively.

4. The complex neural network-based phase-free data imaging method of claim 1, wherein the complex convolution module adopts a Unet network, and comprises a downsampling coding structure composed of a plurality of downsampling layers, an upsampling decoding structure composed of a plurality of upsampling layers and a jump connection; the number of the down-sampling layers and the up-sampling layers is equal; recording the number of down-sampling layers and up-sampling layers in the complex convolution module of the branch I as n, and recording the number of down-sampling layers and up-sampling layers in the complex convolution module of the branch II as m;

in a complex convolution module of a branch I, combining a real number part corresponding to phase-free total field data with a generated imaginary number part to serve as the input of a first layer of down-sampling layer, then taking the output of an upper layer of down-sampling layer as the input of a lower layer of down-sampling layer, and taking the output of a last layer of down-sampling layer as the input of the first layer of up-sampling; the output of the i-th down-sampling layer and the output of the n-i-th up-sampling layer are fused in a jump connection mode, and the fused result is used as the input of the next up-sampling layer until the output of the last up-sampling layer is obtained and used as the output of the complex convolution module;

in a complex convolution module of a branch two, firstly, scattered field data is used as the input of a first layer of down-sampling layer, then the output of an upper layer of down-sampling layer is used as the input of a lower layer of down-sampling layer, and finally the output of a lower layer of down-sampling layer is used as the input of the first layer of up-sampling; and the output of the i-th down-sampling layer and the output of the m-i-th up-sampling layer are fused in a jump connection mode, and the fused result is used as the input of the next up-sampling layer until the output of the last up-sampling layer is obtained and used as the output of the complex convolution module.

5. The complex neural network-based phase-free data imaging method of claim 4, wherein in the complex convolution module, the down-sampling layer and the up-sampling layer adopt a complex two-dimensional convolution network, and the calculation formula is as follows:

6. The complex neural network-based phase-free data imaging method of claim 1, wherein the loss function employs a mean square error loss when training branch one.

7. The complex neural network-based phase-free data imaging method of claim 1, wherein a sum of a structural similarity loss and a mean square error loss is used as a total loss when training branch two.

8. The method according to claim 7, wherein the loss of structural similarity is calculated by normalizing the predicted image and the real sample image to have pixel values in the range of 0 to 1.

9. The complex neural network-based phase-free data imaging method of claim 1, wherein the sizes of real sample images in the training set are uniform, and the dimensions of phase-free total field data and scattered field data generated from the real sample images are uniform.

10. The phase-free data imaging method based on the complex neural network as claimed in claim 1, wherein the step 6) is specifically as follows:

and regarding the phase-free data to be imaged, taking the phase-free data as the input of a first branch in the trained complex domain Unet network to obtain predicted scatter field data and taking the predicted scatter field data as the input of a second branch, and performing modulus extraction on the complex data output by the second branch to obtain a generated image.