CN113240610B - Double-channel ghost imaging reconstruction method and system based on human eye imitation mechanism - Google Patents
Double-channel ghost imaging reconstruction method and system based on human eye imitation mechanism Download PDFInfo
- Publication number
- CN113240610B CN113240610B CN202110587552.7A CN202110587552A CN113240610B CN 113240610 B CN113240610 B CN 113240610B CN 202110587552 A CN202110587552 A CN 202110587552A CN 113240610 B CN113240610 B CN 113240610B
- Authority
- CN
- China
- Prior art keywords
- channel
- light intensity
- image
- human eye
- ghost imaging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 69
- 238000000034 method Methods 0.000 title claims abstract description 56
- 230000007246 mechanism Effects 0.000 title claims abstract description 23
- 238000005259 measurement Methods 0.000 claims abstract description 31
- 238000012549 training Methods 0.000 claims abstract description 25
- 239000011159 matrix material Substances 0.000 claims abstract description 22
- 238000012937 correction Methods 0.000 claims abstract description 16
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims abstract description 7
- 230000015556 catabolic process Effects 0.000 claims abstract description 6
- 238000013135 deep learning Methods 0.000 claims abstract description 6
- 238000006731 degradation reaction Methods 0.000 claims abstract description 6
- 230000008569 process Effects 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 13
- 238000013528 artificial neural network Methods 0.000 claims description 7
- 238000000342 Monte Carlo simulation Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 5
- 230000003287 optical effect Effects 0.000 claims description 5
- 230000005540 biological transmission Effects 0.000 claims description 4
- 238000004088 simulation Methods 0.000 claims description 4
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000003860 storage Methods 0.000 claims description 2
- 230000000739 chaotic effect Effects 0.000 abstract description 4
- 238000013527 convolutional neural network Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 10
- 230000006835 compression Effects 0.000 description 7
- 238000007906 compression Methods 0.000 description 7
- 238000002474 experimental method Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000000052 comparative effect Effects 0.000 description 4
- 238000009826 distribution Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 239000002245 particle Substances 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000033001 locomotion Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000012634 optical imaging Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000035772 mutation Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- NCGICGYLBXGBGN-UHFFFAOYSA-N 3-morpholin-4-yl-1-oxa-3-azonia-2-azanidacyclopent-3-en-5-imine;hydrochloride Chemical compound Cl.[N-]1OC(=N)C=[N+]1N1CCOCC1 NCGICGYLBXGBGN-UHFFFAOYSA-N 0.000 description 1
- 108010074864 Factor XI Proteins 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 239000000443 aerosol Substances 0.000 description 1
- 230000008033 biological extinction Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000004907 flux Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000001556 precipitation Methods 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20076—Probabilistic image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a two-channel ghost imaging reconstruction method and a two-channel ghost imaging reconstruction system based on a human eye imitation mechanism, wherein a two-channel image reconstruction model based on a residual block convolutional neural network is adopted, when a training data set is manufactured, a human eye imitation strategy is adopted to simulate light intensity generation of a specific image set, a degradation data set with an emission path scattering characteristic is introduced, the training data set comprises human eye imitation interested region characteristics and scattering characteristics, after the network is trained to be converged, a measurement matrix is manufactured by utilizing the human eye imitation strategy, the measurement matrix is loaded into a DMD (digital device) to acquire light intensity data, the input data is corrected by a nonlinear light intensity correction method, the corrected data is input into the two-channel image reconstruction model, and a deep learning ghost imaging clear image is obtained through operation. The invention eliminates the influence of scattering medium in the transmitting path on imaging and the nonlinear influence of chaotic medium.
Description
Technical Field
The invention relates to the field of image processing, in particular to a two-channel ghost imaging reconstruction method and system based on a human eye imitation mechanism.
Background
The traditional optical imaging observation technology acquires an image by establishing a direct mapping relation between a sensor pixel and an observed scene, so that the observation result is seriously dependent on the sensor performance, and therefore, the most direct method for acquiring a high-resolution image reduces the pixel size by improving the sensor performance, but reduces the pixel size of the sensor and the luminous flux at the same time, and shot noise is generated to interfere with the image quality. In addition, the traditional optical imaging observation process in a complex environment is affected by the environment, and does not have the capability of all-day, all-weather and high-robustness imaging, for example: light is influenced by atmospheric turbulence, air molecules, suspended particles (aerosol particles, precipitation particles) and the like in the propagation process, phenomena such as attenuation, flicker, offset, intensity, phase fluctuation and the like occur, so that the originally ordered wave front phase is seriously distorted, and when a target is observed and imaged, a speckle pattern is formed on an observation surface due to receiving disordered light field signals; meanwhile, in practical application, motion degradation is caused by transient imaging of a moving target with complex Doppler characteristics, and in computational optical imaging, unknown moving targets can disturb the relevance between sampling frames, and the imaging quality can be seriously reduced under the conditions.
The existing imaging technology applied to the scattering medium comprises a ghost imaging technology, which modulates a target by means of a measurement mode under a preset time sequence, reconstructs an image by acquiring a light intensity sequence, and can effectively remove the influence of scattering when a static scattering medium exists in an imaging receiving light path, namely between a detector and the target. However, this de-scattering imaging effect will fail to some extent when the imaging environment is in any of the following states: 1. scattering media, such as atmospheric haze, are in dynamic variation; 2. the scattering medium is positioned in an emission light path of the imaging system, namely between the light source and the target, and the two states break the light intensity distribution rule between measurement frames in an ideal state, so that the basis of the image reconstruction process is lost.
Disclosure of Invention
In order to solve the technical problem that a scattering medium affects image reconstruction, the invention provides a double-channel ghost imaging reconstruction method and system based on a human eye imitation mechanism.
Therefore, the double-channel ghost imaging reconstruction method based on the human eye imitation mechanism provided by the invention specifically comprises the following steps of:
s1, constructing a two-channel image reconstruction model;
s2, a training data set is manufactured, the two-channel image reconstruction model is trained, and the two-channel image reconstruction model is trained to be converged based on the training data set;
s3, generating a measurement matrix based on a human-eye-simulated strategy, loading the measurement matrix to a micro-mirror array, synchronously controlling the micro-mirror array and a photoelectric detector, and acquiring a light intensity sequence by the photoelectric detector to finish data acquisition of a target sample;
s4, correcting the acquired data by adopting a nonlinear light intensity correction method;
s5, inputting the corrected data into the two-channel image reconstruction model, and returning the corrected data into a two-dimensional image after passing through two channels to obtain a deep learning ghost imaging clear image.
Further, the two-channel image reconstruction model adopts a two-channel enhanced neural network, two branches of the two-channel enhanced neural network are a ghost imaging image branch and a light intensity residual branch, and data sources of the two-channel enhanced neural network are ghost imaging reconstructed images and light intensity characteristic sequences respectively.
Further, the ghost imaging reconstructed image is calculated by the following formula:
I(x)＝<(S i -<S i >)(P i -<P i >)>
wherein the method comprises the steps of<·>Representing the arithmetic mean of all measured values, S i Representing the sequence of light intensities input by the network, P i Representing a random measurement pattern employed to acquire the intensity data.
Further, the ghost image branches acquire preliminary feature images from six convolution layers having feature image dimensions of 128×128×1, 128×128×32, 128×128×16, 128×128×32, 128×128×1, respectively.
Further, the light intensity residual branch comprises a full connection layer and a two-dimensional residual block and two convolution layers, the full connection layer being located between the measurement value sequence and the convolution layers.
Further, in the step S2, a specific image set is generated by adopting a human eye simulation strategy to simulate the light intensity, a monte carlo simulation mode is adopted to simulate the scattering process, a degradation data set with the scattering characteristic of the transmission path is introduced, and a data set to be trained is created.
Further, in the step S4, the nonlinear light intensity correction method specifically includes that when the optical system receives interference of the scattering medium, the measured value is disturbed, assuming that the attenuation coefficient of the previous L frames is ζ, and after L measurements, the attenuation coefficient of the scattering medium is mutated, the following formula may be obtained:
wherein y is a light intensity measurement value, M represents the total number of measurements, N represents the length of a one-dimensional signal, and dividing the measurement average values before and after dynamic change according to the characteristics of ghost imaging and compressed sensing algorithms can obtain the following formula:
multiplying data with a sequence number greater than L by a correction factor ζ according to the above formula L And/ζ, a new light intensity sequence is formed.
Therefore, the two-channel ghost imaging reconstruction system based on the human-simulated eye mechanism comprises a central processing unit, a memory and a data acquisition part, wherein a two-channel image reconstruction model and a program which can be run by the central processing unit are stored in the memory, and the program can realize the two-channel ghost imaging reconstruction method based on the human-simulated eye mechanism in the process of being run by the central processing unit.
Further, the data acquisition portion includes a micromirror array, a photodetector, an imaging lens, and a laser.
Therefore, the computer readable storage medium provided by the invention stores a two-channel image reconstruction model and a program which can be run by a central processing unit, and the program can realize the two-channel ghost imaging reconstruction method based on the human eye imitation mechanism in the process of being run by the central processing unit.
Compared with the prior art, the invention has the following beneficial effects:
the light intensity data is acquired through the human eye imitation mechanism and used for model training and input, the parameterization of the imaging process of the imaging optical system is facilitated, a nonlinear light intensity correction method is provided, and nonlinear influence of chaotic media is eliminated.
In some embodiments of the invention, there are also the following benefits:
1) The dual-channel image reconstruction model based on the residual block convolutional neural network is provided, and the comprehensive extraction capacity and regression capacity of one-dimensional light intensity and two-dimensional image characteristics are improved from the model design level;
2) In order to eliminate the effect of the scattering medium in the transmit path on the imaging, the scattering process is simulated in a Monte Carlo simulation and a data set to be trained is created.
Drawings
FIG. 1 is a block diagram of a data acquisition portion;
FIG. 2 is a flow chart of a dual channel ghost imaging reconstruction method;
FIG. 3 is a block diagram of a two-channel image reconstruction model;
FIG. 4 is a schematic representation of a multiple scattering geometry model based on a large number of photons;
FIG. 5A is a schematic diagram of experimental results of image reconstruction results with a compression ratio of 2 and a contrast ratio of 285;
FIG. 5B is a schematic diagram of experimental results of image reconstruction results with a compression ratio of 4 and a contrast ratio of 206;
FIG. 5C is a diagram showing experimental results of image reconstruction results with a compression ratio of 10 and a contrast ratio of 273;
FIG. 5D is a graph showing experimental results of image reconstruction results with a compression ratio of 20 and a contrast ratio of 266;
FIG. 5E is a graph showing experimental results of image reconstruction results with a compression ratio of 40 and a contrast ratio of 62;
FIG. 5F is a schematic diagram of experimental results of image reconstruction results with a contrast of 2 using a CMOS artwork;
FIG. 6A is a schematic diagram of experimental results in the case of strong scattering degradation in a comparative experiment;
fig. 6B is a schematic diagram of experimental results of a CSGI-based reconstruction in a comparative experiment;
FIG. 6C is a schematic diagram of experimental results of PC-GI based reconstruction in a comparative experiment;
FIG. 6D is a schematic diagram of the results of the experiments in which the reconstruction was performed based on MC-PC-GI in the comparative experiments.
Detailed Description
For a clearer understanding of technical features, objects, and effects of the present invention, a specific embodiment of the present invention will be described with reference to the accompanying drawings.
The dual-channel ghost imaging reconstruction system based on the human eye imitation mechanism comprises a central processing unit, a memory and a data acquisition part. As shown in fig. 1, the data acquisition section includes a micromirror array (DMD) 2, a Photodetector (PD) 6, an imaging lens 5, and a laser 1, 3 in the drawing representing a scattering medium, 4 representing an object, a laser light source is modulated using the micromirror array (DMD), wherein the DMD uses a digital voltage signal to control the micromirrors to perform mechanical movement, intensity modulation of the light source is achieved by controlling deflection angles of the micromirrors, and a state of each of the micromirrors is controlled by 0 or 1 in the loaded measurement matrix data. The memory stores a two-channel image reconstruction model and a program related to the two-channel ghost imaging reconstruction. The central processing unit mainly realizes two functions: 1) Manufacturing a training data set and training a two-channel image reconstruction model; 2) In the actual imaging process, the nonlinear light intensity correction method is used for correcting input data, the corrected data is input into the two-channel image reconstruction model, and the deep learning ghost imaging clear image is obtained through operation.
As shown in fig. 2, the two-channel ghost imaging reconstruction method based on the human eye imitation mechanism in the embodiment of the invention specifically includes the following steps:
s1, constructing a two-channel image reconstruction model, wherein the two-channel image reconstruction model adopts a two-channel enhanced neural network, as shown in FIG. 3, two branches of the neural network are a ghost imaging image branch and a light intensity residual branch, and data sources of the two branches are a ghost imaging reconstruction image (GI reconstruction image) and a light intensity characteristic sequence respectively. Wherein the GI reconstructed image is calculated by the following formula:
I(x)＝<(S i -<S i >)(P i -<P i >)> (1)
wherein the method comprises the steps of<·>Representing the arithmetic mean of all measured values, S i Representing the sequence of light intensities input by the network, P i Representing a random measurement pattern employed to acquire the intensity data.
The ghost imaging image branches obtain preliminary feature images by six convolution layers, wherein the feature image dimensions are 128×128×1, 128×128×32, 128×128×16, 128×128×32 and 128×128×1 respectively, and 32 feature images with different 128×128 sizes can be obtained after the 128×128×32 convolution layers, namely the convolution layers contain 32 channels.
The light intensity residual branch comprises a full connection layer, a two-dimensional residual Block (Res 2Net Block) and two convolution layers, wherein the full connection layer is positioned between the measured value sequence and the convolution layers and is used for extracting features of one-dimensional data and arranging the features into a two-dimensional image. Each residual block comprises a residual structure, the head and the tail of which are respectively connected by a convolution layer, the convolution kernel of the front convolution layer is 5×5×32, namely 128×128×64 is output after a feature map with 128×128×1 dimensions is input, and the convolution kernel of the rear convolution layer is 5×5×1, namely 128×128×1 is output after a feature map with 128×128×32 dimensions is input. In RM, with ReLU as activation function, each residual structure is followed by a batch normalization layer (Batch Normalization, BN). In the residual structure, the 128×128×32 feature map is equally divided into four parts, each part corresponds to a convolution kernel of 5×5×4 except for the first part, and the result after the convolution of the ith layer is added to the (i+1) th layer as a combined input.
The two characteristic branches are combined in an addition mode, namely, the two characteristic graphs with 128 multiplied by 1 dimensions are added and then sent to a second residual error module for characteristic induction, and finally, the product of the output of the second residual error and the GI reconstructed image is used as a predicted image, and the cost function is defined by the mean square error between the product and the truth image for training. Because the GI reconstructed image contains noise characteristics of ghost images, the channel can realize image denoising, and fluctuation in the light intensity sequence contains the most original signal acquisition characteristics of ghost images, and the introduction of the fluctuation in the light intensity sequence can enhance the detail reconstruction of the image.
S2, a training data set is manufactured, a two-channel image reconstruction model is trained, the key of the two-channel image reconstruction model training is the consistency of the training set and the light intensity data input by an actual imaging scene, for example, the light intensity data to be input during imaging are acquired under a strong scattering environment, and the network training data set also has strong scattering characteristics, namely, the input light intensity signal is required to be scattered and degraded. In order to realize key sampling, a human-simulated eye acquisition mode is adopted for data acquisition during imaging, so that a human-simulated eye strategy is adopted during training set manufacturing. The data set theory of deep learning ghost imaging is to be collected and manufactured in the field, but the time cost is high, in order to manufacture the data set more economically and conveniently, a human eye imitation strategy is adopted to simulate the generation of light intensity of a specific image set, a Monte Carlo simulation mode is adopted to simulate the scattering process, a degradation data set with the scattering characteristic of a transmission path is introduced into the self-manufactured data set, a data set to be trained is created, the training data set comprises the characteristics of an area of interest of the human eye imitation and the scattering characteristics, and a double-channel image reconstruction model is trained to be converged (mean square error is smaller than 0.5) based on the training data set.
As shown in fig. 4, the static scattering medium was simulated, and θ was calculated using the following equation, respectively 0 ，
θ 0 ＝sin -1 {1-ξ 1 [1-cos(β E /2)]} (2)
Wherein, xi 1 And xi 2 Are all random numbers with values between 0 and 1, beta E The value is 0.
The motion of the photons can be simulated using the random numbers generated by the program, and the linear distance between each two consecutive collision sites of a single photon is calculated using the following formula:
wherein, xi 3 To take on a random number between 0 and 1, sigma a And sigma (sigma) s The sum represents the linear extinction coefficient of the particles.
By pairs of
Representing the relative transmission direction of photons after undergoing the ith scattering, the probability distribution density function is:
wherein v is an atmospheric model parameter, and only the parameter is needed to pass through in the specific calculation process
Solving for θ from probability density distribution i And set to a random number between 0 and 1 to determine θ in reverse i Is a value of (a). The direction vector of photons in the global cartesian coordinate system has been updated as:
after the nth scattering, the energy remainder of the photon is:
setting the energy threshold of photons to epsilon=10 -5 I.e. when the current energy value of a photon is below a threshold, it is considered a non-viable photon and cannot be received. The scattered photons and the energy carried by the scattered photons comprehensively represent the intensity distribution of the degenerated light field, and the light field and the target gray level are subjected to dot multiplication operation, so that simulated light intensity data can be finally obtained.
S3, generating a measurement matrix based on a human eye imitation strategy, loading the measurement matrix into the DMD for light intensity data acquisition, taking a reconstruction target with the resolution of 64 multiplied by 64 as an example, setting the size of the basic measurement matrix to 1024 rows and 4096 columns under the condition that the sampling rate is 25%, wherein the acquisition mode is as follows: firstly generating a Hadamard matrix with the size of 4096×4096, then taking the first 1024 rows of the matrix as a basic measurement matrix, respectively taking out each row of the basic measurement matrix, recombining each 4096-number-containing sequence into a 64×64 image according to the rows to form a matrix group M 1 The Hadamard matrix with 1024×1024 size is reorganized into 1024 matrices with 32×32 size according to rows in the above way, and then the matrix is fully expanded into matrix group M with 64×64 size by bilinear interpolation method 2 A region of interest R is artificially selected in a 64×64 matrix, the unselected region is NR, a region control matrix C is set, wherein the pixel value of the R-containing region in C is 1, otherwise, 0, and thus the human eye simulated measurement pattern group is obtained: m is M F ＝M 1· *C+M 2· * And (I-C), wherein I is an identity matrix, a matrix group containing 1024 measurement patterns is loaded into an upper computer of the DMD, the DMD and a Photoelectric Detector (PD) are synchronously controlled, the PD can acquire 1024 frames of light intensity sequences at the same frame rate, and the process completes the data acquisition of one target sample.
S4, because the scattering medium in the training set is uniform, the change caused by the mutation or the non-uniformity of the scattering medium is needed to be considered in the actual collection, so that the influence of the non-uniformity characteristic on the data is required to be weakened, the nonlinear light intensity correction method is used for correcting the input data, when the optical system receives the interference of the scattering medium, the measured value is disturbed, the attenuation coefficient of the former L frames is assumed to be xi, and after L times of measurement, the attenuation coefficient of the scattering medium is mutated, and the following formula can be obtained:
where y is the intensity measurement, M represents the total number of measurements and N represents the length of the one-dimensional signal. The nonlinear correction method for the light intensity value is shown as a formula (10), wherein xi is L And/ζ is an accurate correction factor. According to the characteristics of ghost imaging and compressed sensing algorithms, the average value measured before and after dynamic change can be divided to obtain the following expression:
the method corrects the light intensity data once and finds out the mutation point with the most serious influence on the light intensity by the dynamic change of the scattering medium. Calculated from (10)
Where l=1, 2,3 …, M. Find to make->
The value of L with the largest value is taken as the abrupt point L of light intensity correction 0 ζ at this time L And/ζ is a correction factor. Correcting the light intensity sequence according to equations (9) and (10), i.e. multiplying the data with sequence number greater than L by correction factor xi L And/ζ, a new light intensity sequence is formed.
S5, inputting the corrected data into a two-channel image reconstruction model, and returning the corrected data into a two-dimensional image after passing through two channels to obtain a deep learning ghost imaging clear image.
The laser range finders are adopted to measure the distance in Shenzhen urban areas to ensure that the scattering distance is about 50m for imaging, the experiment is carried out in a thick fog scene at certain night, and a prototype can be verified in an actual more extreme haze environment. The target uses standard test boards printed with black and white stripes and a three-dimensional shaped ornament.
As can be seen from the reconstruction results and the contrast quantification values, the method has a significant effect on the contrast improvement of black and white stripes, as shown in fig. 5A-5F. When the compression ratio is not more than 20, the reconstruction contrast ratio is kept above 200, and a reconstruction result with the contrast ratio of 62 can still be obtained under the compression ratio of 40.
In order to verify the advantages of the PC-GI network (named MC-PC-GI) of Monte Carlo simulation data, as shown in FIGS. 6A-6D, the PC-GI network based on the compressed sensing reconstruction method CSGI and the PC-GI network based on non-scattering simulation data are adopted for experimental comparison, the result of training the PC-GI network by adopting the non-scattering simulation data is slightly better than that of the CSGI method, the detail information is more sufficient, but the noise is more, in contrast, the MC-PC-GI method drives the network training under more ideal data, the regression result is more accurate, and the robustness is stronger under the strong scattering imaging environment.
The double-channel ghost imaging reconstruction method and system based on the human eye imitation mechanism provided by the invention have the following beneficial effects:
1) The method comprises the steps of providing a dual-channel image reconstruction model based on a residual block convolutional neural network, considering data training modes of different dimensions, improving comprehensive extraction capacity and regression capacity of one-dimensional light intensity and two-dimensional image characteristics from a model design level, acquiring light intensity data for model training and input through a human eye imitation mechanism, enhancing the light intensity data characteristics and overall image quality through key acquisition of a region of interest, facilitating realization of parameterization of an imaging optical system imaging process, and further improving image accuracy in a deep neural network;
2) In order to recover the inter-frame light intensity correlation under the dynamic chaotic scattering medium, a nonlinear light intensity correction method is provided, and a compensation factor is designed to eliminate nonlinear influence of the chaotic medium;
3) Because the actual light intensity data is collected in a strong scattering environment, the training set data also has strong scattering characteristics, namely the input light intensity signal is scattered and degraded, and in order to eliminate the influence of scattering medium in a transmitting path on imaging, a Monte Carlo simulation mode is adopted to simulate the scattering process and create a data set to be trained.
The foregoing disclosure is illustrative of the present invention and is not to be construed as limiting the scope of the invention, which is defined by the appended claims.
Claims (8)
1. The double-channel ghost imaging reconstruction method based on the human eye imitation mechanism is characterized by comprising the following steps of:
s1, constructing a two-channel image reconstruction model with ghost imaging image branches and light intensity residual branches, comprehensively extracting and regressing one-dimensional light intensity and two-dimensional image characteristics of an image, removing noise of the image, and enhancing image reconstruction details; the ghost imaging image branch acquires a preliminary characteristic image by six convolution layers, the light intensity residual branch comprises a full-connection layer, a two-dimensional residual block and two convolution layers, wherein the full-connection layer is used for arranging one-dimensional data into a two-dimensional image, and the full-connection layer is positioned between a measured value sequence and the convolution layers;
s2, simulating light intensity generation on a specific image set based on a human eye simulating strategy, simulating a scattering process by adopting a Monte Carlo simulation mode, introducing a degradation data set with a transmission path scattering characteristic, creating a data set to be trained, enabling a training data set to contain human eye region of interest characteristics and scattering characteristics, obtaining simulated light intensity data, training the two-channel image reconstruction model, and training the two-channel image reconstruction model to be converged based on the training data set;
s3, generating a measurement matrix based on a human-simulated eye strategy, loading the measurement matrix to a micro-mirror array, selecting an interested region, synchronously controlling the micro-mirror array and a photoelectric detector, acquiring a light intensity sequence by the photoelectric detector, completing data acquisition of a target sample by adopting a human-simulated eye acquisition mode, performing key acquisition on the interested region, and enhancing light intensity data characteristics;
s4, correcting the acquired data by adopting a nonlinear light intensity correction method;
s5, inputting the corrected data into the two-channel image reconstruction model, and returning the corrected data into a two-dimensional image after passing through two channels to obtain a deep learning ghost imaging clear image.
2. The two-channel ghost imaging reconstruction method based on the human eye simulation mechanism according to claim 1, wherein the two-channel image reconstruction model adopts a two-channel enhanced neural network, and the data sources of the ghost imaging image branch and the light intensity residual branch are ghost imaging reconstructed images and light intensity characteristic sequences respectively.
3. The two-channel ghost imaging reconstruction method based on the human eye imitation mechanism as claimed in claim 2, wherein the ghost imaging reconstruction image is calculated by the following formula:
I(x)＝<(S i -<S i >)(P i -<P i >)>
wherein the method comprises the steps of<·>Representing the arithmetic mean of all measured values, S i Representing the sequence of light intensities input by the network, P i Representing a random measurement pattern employed to acquire the intensity data.
4. A two-channel ghost imaging reconstruction method based on a human eye-like mechanism as claimed in claim 2, wherein, the feature map dimensions of the six convolutional layers are 128×128×1, 128×128×32, 128×128×16, 128×128×32, 128×128×1, respectively.
5. The two-channel ghost imaging reconstruction method based on the human eye-like mechanism according to claim 1, wherein in the step S4, the nonlinear light intensity correction method specifically includes that when the optical system receives interference of the scattering medium, the measured value is disturbed, assuming that the attenuation coefficient of the previous L frames is ζ, and after L measurements, the attenuation coefficient of the scattering medium is mutated, the following formula can be obtained:
wherein y is a light intensity measurement value, M represents the total number of measurements, N represents the length of a one-dimensional signal, and dividing the measurement average values before and after dynamic change according to the characteristics of ghost imaging and compressed sensing algorithms can obtain the following formula:
multiplying data with a sequence number greater than L by a correction factor ζ according to the above formula L And/ζ, a new light intensity sequence is formed.
6. A two-channel ghost imaging reconstruction system based on a human eye imitation mechanism, comprising a central processing unit, a memory and a data acquisition part, wherein a two-channel image reconstruction model and a program which can be run by the central processing unit are stored in the memory, and the program can realize the two-channel ghost imaging reconstruction method based on the human eye imitation mechanism according to any one of claims 1-5 in the process of being run by the central processing unit.
7. A two-channel ghost imaging reconstruction system based on a human eye-like mechanism as in claim 6 wherein the data acquisition portion comprises a micromirror array, a photodetector, an imaging lens and a laser.
8. A computer readable storage medium, storing a two-channel image reconstruction model and a program executable by a central processing unit, the program being capable of implementing the two-channel ghost imaging reconstruction method based on a human eye imitation mechanism as claimed in any one of claims 1-5 in the process of being executed by the central processing unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110587552.7A CN113240610B (en) | 2021-05-27 | 2021-05-27 | Double-channel ghost imaging reconstruction method and system based on human eye imitation mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110587552.7A CN113240610B (en) | 2021-05-27 | 2021-05-27 | Double-channel ghost imaging reconstruction method and system based on human eye imitation mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113240610A CN113240610A (en) | 2021-08-10 |
| CN113240610B true CN113240610B (en) | 2023-05-12 |
Family
ID=77139278
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110587552.7A Active CN113240610B (en) | 2021-05-27 | 2021-05-27 | Double-channel ghost imaging reconstruction method and system based on human eye imitation mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113240610B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113763540A (en) * | 2021-09-08 | 2021-12-07 | 四川川大智胜软件股份有限公司 | Three-dimensional reconstruction method and equipment based on speckle fringe hybrid modulation |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106646512A (en) * | 2016-12-29 | 2017-05-10 | 北京理工大学 | Ghost imaging method and ghost imaging system based on bionic vision mechanism |
| CN109752844A (en) * | 2019-03-14 | 2019-05-14 | 中国科学院微电子研究所 | A kind of imaging method and system based on random Intensity Fluctuation |
| CN112802145A (en) * | 2021-01-27 | 2021-05-14 | 四川大学 | Color calculation ghost imaging method based on deep learning |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107121709B (en) * | 2017-06-01 | 2023-07-25 | 华南师范大学 | Object imaging system based on compressed sensing and imaging method thereof |
| JP6984010B2 (en) * | 2017-09-28 | 2021-12-17 | コーニンクレッカ フィリップス エヌ ヴェKoninklijke Philips N.V. | Deep learning-based scattering correction |
| US10937206B2 (en) * | 2019-01-18 | 2021-03-02 | Canon Medical Systems Corporation | Deep-learning-based scatter estimation and correction for X-ray projection data and computer tomography (CT) |
| US11895405B2 (en) * | 2019-07-12 | 2024-02-06 | University College Cork—National University of Ireland, Cork | Method and system for performing high speed optical image detection |
| CN110675326B (en) * | 2019-07-24 | 2022-04-22 | 西安理工大学 | Method for calculating ghost imaging reconstruction recovery based on U-Net network |
| US10802066B1 (en) * | 2019-12-17 | 2020-10-13 | Quantum Valley Ideas Laboratories | Single-pixel imaging of electromagnetic fields |
| CN111652059B (en) * | 2020-04-27 | 2023-03-24 | 西北大学 | Target identification model construction and identification method and device based on computational ghost imaging |
| CN111551955B (en) * | 2020-06-22 | 2022-07-08 | 北京理工大学 | Bionic blocking ghost imaging method and system |
-
2021
- 2021-05-27 CN CN202110587552.7A patent/CN113240610B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106646512A (en) * | 2016-12-29 | 2017-05-10 | 北京理工大学 | Ghost imaging method and ghost imaging system based on bionic vision mechanism |
| CN109752844A (en) * | 2019-03-14 | 2019-05-14 | 中国科学院微电子研究所 | A kind of imaging method and system based on random Intensity Fluctuation |
| CN112802145A (en) * | 2021-01-27 | 2021-05-14 | 四川大学 | Color calculation ghost imaging method based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113240610A (en) | 2021-08-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11200456B2 (en) | Systems and methods for generating augmented training data for machine learning models | |
| DE102016107959B4 (en) | Structured light-based multipath erasure in ToF imaging | |
| Güven et al. | An augmented Lagrangian method for complex-valued compressed SAR imaging | |
| Yonel et al. | Deep learning for passive synthetic aperture radar | |
| Guo et al. | Tackling 3d tof artifacts through learning and the flat dataset | |
| CN110675326B (en) | Method for calculating ghost imaging reconstruction recovery based on U-Net network | |
| Wang et al. | TPSSI-Net: Fast and enhanced two-path iterative network for 3D SAR sparse imaging | |
| CN110187143B (en) | Chromatography PIV reconstruction method and device based on deep neural network | |
| CN111489301B (en) | Image defogging method based on image depth information guide for migration learning | |
| CN111047681B (en) | Single-pixel three-dimensional end-to-end reconstruction method and device based on deep learning | |
| CN105931196A (en) | Fourier optical modeling-based coded aperture camera image restoration method | |
| CN113238189B (en) | Sound source identification method and system based on array measurement and sparse prior information | |
| CN109884625B (en) | Radar correlation imaging method based on convolutional neural network | |
| CN113240610B (en) | Double-channel ghost imaging reconstruction method and system based on human eye imitation mechanism | |
| CN106686281B (en) | Fuse circuit board noise suppression ability test system | |
| CN109991602A (en) | ISAR image resolution enhancement method based on depth residual error network | |
| Nakashima et al. | Learning to drop points for lidar scan synthesis | |
| CN111861926A (en) | Image rain removing method based on airspace group enhancement mechanism and long-time and short-time memory network | |
| Wang et al. | Underwater self-supervised monocular depth estimation and its application in image enhancement | |
| CN111352126B (en) | Single-pixel imaging method based on atmospheric scattering medium modulation | |
| CN110926611A (en) | Noise suppression method applied to compressed sensing spectral imaging system | |
| Marcus et al. | A lightweight machine learning pipeline for LiDAR-simulation | |
| WO2023023961A1 (en) | Piv image calibration apparatus and method based on laser linear array | |
| CN112468791B (en) | Light intensity measurement iterative imaging method based on single-pixel detection | |
| CN113156434A (en) | Reconstruction of elevation information from radar data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20221123 Address after: Second floor, building a, Tsinghua campus, Shenzhen University Town, Xili street, Nanshan District, Shenzhen City, Guangdong Province Applicant after: Shenzhen International Graduate School of Tsinghua University Applicant after: BEIJING INSTITUTE OF TECHNOLOGY Address before: Second floor, building a, Tsinghua campus, Shenzhen University Town, Xili street, Nanshan District, Shenzhen City, Guangdong Province Applicant before: Shenzhen International Graduate School of Tsinghua University |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |