CN113240610A - Double-channel ghost imaging reconstruction method and system based on human eye simulation mechanism - Google Patents

Double-channel ghost imaging reconstruction method and system based on human eye simulation mechanism Download PDF

Info

Publication number
CN113240610A
CN113240610A CN202110587552.7A CN202110587552A CN113240610A CN 113240610 A CN113240610 A CN 113240610A CN 202110587552 A CN202110587552 A CN 202110587552A CN 113240610 A CN113240610 A CN 113240610A
Authority
CN
China
Prior art keywords
channel
light intensity
human eye
image
ghost
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110587552.7A
Other languages
Chinese (zh)
Other versions
CN113240610B (en
Inventor
程雪岷
高子琪
郝群
叶恒志
陈棵
王安琪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Institute of Technology BIT
Shenzhen International Graduate School of Tsinghua University
Original Assignee
Shenzhen International Graduate School of Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen International Graduate School of Tsinghua University filed Critical Shenzhen International Graduate School of Tsinghua University
Priority to CN202110587552.7A priority Critical patent/CN113240610B/en
Publication of CN113240610A publication Critical patent/CN113240610A/en
Application granted granted Critical
Publication of CN113240610B publication Critical patent/CN113240610B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a double-channel ghost imaging reconstruction method and a double-channel ghost imaging reconstruction system based on a human eye simulating mechanism, wherein a double-channel image reconstruction model based on a residual block convolutional neural network is adopted, when a training data set is manufactured, a human eye simulating strategy is adopted to carry out simulated light intensity generation on a specific image set, a degradation data set with a transmission path scattering characteristic is introduced, the training data set comprises the human eye simulating interest region characteristic and the scattering characteristic, after the network is trained to be converged, a measurement matrix is manufactured by the human eye simulating strategy and loaded into a DMD to carry out light intensity data acquisition, a nonlinear light intensity correction method is used to correct input data, the corrected data is input into the double-channel image reconstruction model, and a deep learning ghost imaging clear image is obtained through calculation. The invention eliminates the influence of scattering media in the transmitting path on imaging and the nonlinear influence of chaotic media.

Description

Double-channel ghost imaging reconstruction method and system based on human eye simulation mechanism
Technical Field
The invention relates to the field of image processing, in particular to a double-channel ghost imaging reconstruction method and a double-channel ghost imaging reconstruction system based on a human eye simulating mechanism.
Background
In the traditional optical imaging observation technology, a direct mapping relation is established between a sensor pixel and an observed scene to obtain an image, so the observation result of the image depends on the performance of a sensor seriously, and the most direct method for obtaining a high-resolution image reduces the pixel size by improving the performance of the sensor, but reduces the pixel size of the sensor and reduces the luminous flux at the same time, and generates shot noise to interfere the image quality. Besides, the traditional optical imaging observation process in a complex environment is influenced by the environment, and the all-time and all-weather high-robustness imaging capability is not provided, for example: when light is influenced by atmospheric turbulence, air molecules, suspended particles (aerosol particles, precipitation particles) and the like in the transmission process, phenomena such as attenuation, flicker, offset, intensity, phase fluctuation and the like occur, the original ordered wavefront phase is seriously distorted, and when a target is observed and imaged, a speckle pattern can be formed on an observation surface due to the fact that disordered light field signals are received; meanwhile, most of moving targets with complex Doppler characteristics bring motion degradation to instantaneous imaging in practical application, and in computational optical imaging, unknown moving targets can disturb the correlation among sampling frames, and the conditions can seriously reduce the imaging quality.
The existing imaging technology applied to scattering media comprises a computational ghost imaging technology, which modulates a target by means of a measurement mode under a preset time sequence, reconstructs an image by acquiring a light intensity sequence, and can effectively remove the influence of scattering when a static scattering medium appears in a receiving light path of imaging, namely between a detector and the target. However, this de-scatter imaging effect will be somewhat ineffective when the imaging environment is in any of the following states: firstly, scattering media, such as atmospheric haze, are in dynamic change; and secondly, the scattering medium is positioned in an emission light path of the imaging system, namely between the light source and the target, and the two states break the light intensity distribution rule between measurement frames in an ideal state, so that the basis is lost in the image reconstruction process.
Disclosure of Invention
In order to solve the technical problem that a scattering medium influences image reconstruction, the invention provides a dual-channel ghost imaging reconstruction method and system based on a human eye simulating mechanism.
Therefore, the double-channel ghost imaging reconstruction method based on the human eye simulating mechanism provided by the invention specifically comprises the following steps:
s1, constructing a dual-channel image reconstruction model;
s2, making a training data set, training the dual-channel image reconstruction model, and training the dual-channel image reconstruction model to be convergent based on the training data set;
s3, generating a measurement matrix based on a human eye simulation strategy, loading the measurement matrix to a micro mirror array, synchronously controlling the micro mirror array and a photoelectric detector, and acquiring a light intensity sequence by the photoelectric detector to finish data acquisition of a target sample;
s4, correcting the acquired data by adopting a nonlinear light intensity correction method;
and S5, inputting the corrected data into the dual-channel image reconstruction model, and regressing the data into a two-dimensional image after passing through two channels to obtain a clear deep learning ghost imaging image.
Furthermore, the dual-channel image reconstruction model adopts a dual-channel enhanced neural network, two branches of the dual-channel enhanced neural network are a ghost image branch and a light intensity residual branch, and data sources of the two branches are a ghost image reconstruction image and a light intensity characteristic sequence respectively.
Further, the ghost image reconstructed image is calculated by the following formula:
I(x)＝<(Si-<Si>)(Pi-<Pi>)>
wherein<·>Representing the arithmetic mean of all measurements, SiLight intensity sequence, P, representing network inputiRepresenting a random measurement pattern used to acquire light intensity data.
Further, the ghost image map branches to acquire preliminary feature images from six convolution layers, the feature map dimensions of the six convolution layers are 128 × 128 × 1, 128 × 128 × 32, 128 × 128 × 16, 128 × 128 × 16, 128 × 128 × 32, and 128 × 128 × 1, respectively.
Further, the light intensity residual error branch comprises a fully connected layer and a two-dimensional residual error block and two convolutional layers, wherein the fully connected layer is positioned between the measurement value sequence and the convolutional layers.
Further, in step S2, a human eye-simulated strategy is used to generate simulated light intensity for a specific image set, a monte carlo simulation method is used to simulate a scattering process, a degraded data set with a scattering characteristic of an emission path is introduced, and a data set to be trained is created.
Further, in step S4, the method for correcting nonlinear light intensity specifically includes that when the optical system receives scattering medium interference, the measured value will be disturbed, assuming that the attenuation coefficient of the previous L frames is ξ, and after L measurements, the attenuation coefficient of the scattering medium changes abruptly, so as to obtain the following formula:
Figure BDA0003088239130000021
wherein y is the light intensity measurement value, M represents the total number of measurements, N represents the length of the one-dimensional signal, and the following formula can be obtained by dividing the measurement average values before and after the dynamic change according to the characteristics of the ghost imaging and the compressed sensing algorithm:
Figure BDA0003088239130000031
multiplying the data with the serial number greater than L by the correction factor xi according to the formulaLAnd/xi, forming a new light intensity sequence.
Therefore, the two-channel ghost imaging reconstruction system based on the human eye simulation mechanism comprises a central processing unit, a memory and a data acquisition part, wherein a two-channel image reconstruction model and a program capable of being operated by the central processing unit are stored in the memory, and the program can realize the above two-channel ghost imaging reconstruction method based on the human eye simulation mechanism in the process of being operated by the central processing unit.
Further, the data acquisition part comprises a micromirror array, a photodetector, an imaging lens and a laser.
To this end, the computer-readable storage medium provided by the present invention stores a dual-channel image reconstruction model and a program executable by a central processing unit, wherein the program is capable of implementing the above-mentioned dual-channel ghost imaging reconstruction method based on the human eye-imitated mechanism during the execution of the program by the central processing unit.
Compared with the prior art, the invention has the following beneficial effects:
the light intensity data is obtained by a human eye simulating mechanism and is used for model training and input, the parameterization of the imaging process of the imaging optical system is favorably realized, a nonlinear light intensity correction method is provided, and the nonlinear influence of a chaotic medium is eliminated.
In some embodiments of the invention, the following advantages are also provided:
1) a dual-channel image reconstruction model based on a residual block convolutional neural network is provided, and the comprehensive extraction capability and regression capability of one-dimensional light intensity and two-dimensional image features are improved from the model design level;
2) in order to eliminate the influence of the scattering medium in the emission path on imaging, a Monte Carlo simulation mode is adopted to simulate the scattering process and create a data set to be trained.
Drawings
FIG. 1 is a block diagram of a data acquisition portion;
FIG. 2 is a flow chart of a two-channel ghost imaging reconstruction method;
FIG. 3 is a block diagram of a two-channel image reconstruction model;
FIG. 4 is a schematic diagram of a multiple scattering geometry model based on a large number of photons;
FIG. 5A is a graph showing the results of an experiment of image reconstruction for a compression ratio of 2 and a contrast ratio of 285;
FIG. 5B is a graph showing the results of an experiment for the image reconstruction for a compression ratio of 4 and a contrast ratio of 206;
FIG. 5C is a graph showing the results of an experiment for the image reconstruction in the case where the compression ratio is 10 and the contrast ratio is 273;
FIG. 5D is a graph showing the results of an experiment for the image reconstruction for a compression ratio of 20 and a contrast ratio of 266;
FIG. 5E is a graph showing the results of an experiment in which the image reconstruction is performed at a compression ratio of 40 and a contrast ratio of 62;
FIG. 5F is a diagram illustrating experimental results of image reconstruction results with a contrast of 2 using CMOS original images;
FIG. 6A is a graph showing the results of a comparison experiment in the case of strong scattering degradation;
FIG. 6B is a graph showing the results of a CSGI-based reconstitution experiment in a comparative experiment;
FIG. 6C is a graph showing the results of a PC-GI based reconstruction in a comparative experiment;
FIG. 6D is a graph showing the results of a comparative experiment in which the MC-PC-GI-based reconstitution was performed.
Detailed Description
In order to more clearly understand the technical features, objects, and effects of the present invention, embodiments of the present invention will now be described with reference to the accompanying drawings.
The two-channel ghost imaging reconstruction system based on the human eye simulating mechanism comprises a central processing unit, a memory and a data acquisition part. As shown in fig. 1, the data acquisition part includes a micromirror array (DMD)2, a Photodetector (PD)6, an imaging lens 5 and a laser 1, 3 in the drawing represents a scattering medium, 4 represents an object, and a laser light source is modulated using the micromirror array (DMD), wherein the DMD uses a digital voltage signal to control a micromirror to perform a mechanical motion, intensity modulation of the light source is achieved by controlling a deflection angle of the mirror, and a state of each mirror is controlled by 0 or 1 in the loaded measurement matrix data. The memory stores a two-channel image reconstruction model and a program related to two-channel ghost imaging reconstruction. The central processing unit mainly realizes two functions: 1) making a training data set, and training a dual-channel image reconstruction model; 2) in the actual imaging process, input data are corrected by a nonlinear light intensity correction method, the corrected data are input into a dual-channel image reconstruction model, and a depth learning ghost imaging clear image is obtained through calculation.
As shown in fig. 2, the dual-channel ghost imaging reconstruction method based on the human eye-imitated mechanism in the embodiment of the present invention specifically includes the following steps:
and S1, constructing a dual-channel image reconstruction model, wherein the dual-channel image reconstruction model adopts a dual-channel enhanced neural network, as shown in FIG. 3, two branches of the neural network are a ghost image branch and a light intensity residual error branch, and data sources of the neural network are a ghost image reconstruction image (GI reconstruction image) and a light intensity characteristic sequence. Wherein the GI reconstructed image is calculated by the following formula:
I(x)＝<(Si-<Si>)(Pi-<Pi>)> (1)
wherein<·>Representing the arithmetic mean of all measurements, SiLight intensity sequence, P, representing network inputiRepresenting a random measurement pattern used to acquire light intensity data.
The ghost image branch obtains a preliminary feature image from six convolutional layers, the dimensions of the feature image are 128 × 128 × 1, 128 × 128 × 32, 128 × 128 × 16, 128 × 128 × 16, 128 × 128 × 32 and 128 × 128 × 1, wherein 32 different feature images with 128 × 128 sizes, namely the convolutional layers containing 32 channels, can be obtained after the feature images are processed by the 128 × 128 × 32 convolutional layers.
The light intensity residual error branch comprises a full connection layer, a two-dimensional residual error Block (Res2Net Block) and two convolution layers, wherein the full connection layer is positioned between the measured value sequence and the convolution layers and is used for performing feature extraction on one-dimensional data and arranging the one-dimensional data as a two-dimensional image. Each residual block comprises a residual structure, the head and the tail of the residual structure are respectively connected by a convolution layer, the convolution kernel of the front convolution layer is 5 multiplied by 32, namely the output is 128 multiplied by 64 after a characteristic diagram with dimensions of 128 multiplied by 1 is input, and the convolution kernel of the rear convolution layer is 5 multiplied by 1, namely the output is 128 multiplied by 1 after the characteristic diagram with dimensions of 128 multiplied by 32 is input. In RM, with ReLU as the activation function, a Batch Normalization layer (BN) is placed after each residual structure. In the residual structure, the 128 × 128 × 32 feature map is divided into four parts, each part except the first part corresponds to a convolution kernel of 5 × 5 × 4, and the result after the convolution of the ith layer is added to the (i + 1) th layer as the combined input.
The two characteristic branches are combined in an addition mode, namely two characteristic graphs with dimensions of 128 multiplied by 1 are added and then sent to a second residual error module for characteristic induction, finally, the product of the output of the second residual error and the GI reconstructed image is used as a predicted image, and a cost function is defined by the mean square error between the predicted image and a truth value graph for training. Since the GI reconstructed image contains the noise characteristics of the ghost imaging, the channel can implement image denoising, and the fluctuation in the light intensity sequence contains the most primitive signal acquisition characteristics of the ghost imaging, which is introduced to enhance the image detail reconstruction.
S2, making a training data set, and training a dual-channel image reconstruction model, wherein the key of the training of the dual-channel image reconstruction model lies in the consistency of light intensity data input by the training set and an actual imaging scene, if the light intensity data to be input is acquired in a strong scattering environment during imaging, the network training data set also has strong scattering characteristics, namely, an input light intensity signal is degraded by scattering. In order to realize key sampling, a human-eye-simulated acquisition mode is adopted to acquire data during imaging, so that a human-eye-simulated strategy is also adopted during training set production. The data set of the deep learning ghost imaging is collected and manufactured in a real-world manner, however, the time cost is high, in order to manufacture the data set more economically and conveniently, a human eye simulation strategy is adopted to generate simulated light intensity for a specific image set, a Monte Carlo simulation mode is adopted to simulate a scattering process, a degraded data set with a scattering characteristic of an emission path is introduced into the self-manufactured data set, a data set to be trained is created, the training data set comprises a human eye-simulated interest region characteristic and a scattering characteristic, and a dual-channel image reconstruction model is trained to be convergent (the mean square error is less than 0.5) based on the training data set.
As shown in FIG. 4, the static scattering medium was simulated, and θ was calculated by the following equation0，
Figure BDA0003088239130000051
θ0＝sin-1{1-ξ1[1-cos(βE/2)]} (2)
Figure BDA0003088239130000053
Wherein ξ1And xi2Are all random numbers with values between 0 and 1, betaEThe value is 0.
The motion of the photon can be simulated by using random numbers generated by a program, and the linear distance between every two continuous collision positions of a single photon is calculated by using the following formula:
Figure BDA0003088239130000052
wherein ξ3To take random numbers between 0 and 1, σaAnd σsThe sum represents the linear extinction coefficient of the particle.
Using pairs of
Figure BDA0003088239130000061
The probability distribution density function of the relative transmission direction of the photon after the ith scattering is represented as:
Figure BDA0003088239130000062
Figure BDA0003088239130000063
for a scattering phase function under a particular scattering type, which specifies the probability that a photon will experience a back-scattered deviation angle, the phase function is expressed as a rayleigh scattering phase function:
Figure BDA0003088239130000064
wherein, v is atmospheric model parameter, and only in the specific calculation processNeed to pass through
Figure BDA0003088239130000065
Solving for theta by probability density distributioniAnd set to a random number between 0 and 1, theta can be determined inverselyiThe value of (a). The direction vector of the photon in the global cartesian coordinate system has been updated as:
Figure BDA0003088239130000066
after the nth scattering, the energy remaining for the photon is:
Figure BDA0003088239130000067
setting the energy threshold of a photon to be 10 ∈-5That is, when the current energy value of a photon is lower than the threshold, the photon is regarded as a non-survival photon and cannot be received. And comprehensively representing the intensity distribution of the degraded light field by the scattered photons and the energy carried by the photons, and performing dot multiplication operation with the target gray scale to finally obtain simulated light intensity data.
S3, generating a measurement matrix based on the human eye simulation strategy, loading the measurement matrix into a DMD for light intensity data acquisition, taking a reconstruction target with a resolution of 64 x 64 as an example, the size of a basic measurement matrix needs to be set to 1024 rows and 4096 columns under the condition that the sampling rate is 25%, and the acquisition mode is as follows: firstly, generating a Hadamard matrix with the size of 4096 x 4096, then taking the first 1024 rows of the matrix as a basic measurement matrix, respectively taking out each row of the basic measurement matrix, recombining each sequence containing 4096 numbers into 64 x 64 images according to the rows to form a matrix group M1The Hadamard matrix with 1024 × 1024 sizes is re-organized into 1024 matrices with 32 × 32 sizes by rows in the above manner, and then all the Hadamard matrices are expanded into matrix groups M with 64 × 64 sizes by the bilinear interpolation method2Artificially selecting an interested region R in a 64 x 64 matrix, wherein the unselected region is NR, and setting a region control matrix C, wherein the C contains the pixel value of the R region1, otherwise 0, thereby obtaining a human eye-simulated measurement pattern set: mF＝M1·*C+M2·And I-C, wherein I is a unit matrix, a matrix group containing 1024 measurement patterns is loaded into an upper computer of the DMD, the DMD and a Photoelectric Detector (PD) are synchronously controlled, the PD can acquire 1024 frames of light intensity sequences at the same frame rate, and the process finishes the data acquisition of a target sample.
S4, because the scattering medium in the training set is uniform, and the change brought by the sudden change or the unevenness of the scattering medium to the input data needs to be considered in the actual acquisition, the influence of the uneven characteristic to the data needs to be weakened, at this time, a nonlinear light intensity correction method is used for correcting the input data, when the optical system receives the interference of the scattering medium, the measured value is disturbed, the attenuation coefficient of the previous L frames is assumed to be xi, and after L times of measurement, the attenuation coefficient of the scattering medium is suddenly changed, and the following formula can be obtained:
Figure BDA0003088239130000071
where y is the light intensity measurement, M represents the total number of measurements, and N represents the length of the one-dimensional signal. The method for non-linear correction of light intensity value is shown in formula (10), wherein xiLAnd/ξ is the exact correction factor. According to the characteristics of the ghost imaging and compressed sensing algorithm, the measured average values before and after dynamic change are divided to obtain the following expression:
Figure BDA0003088239130000072
the method corrects the light intensity data once and finds out the catastrophe points with the most serious influence of the dynamic change of the scattering medium on the light intensity. Calculated by equation (10)
Figure BDA0003088239130000073
Wherein L is 1,2,3 …, M. Look for so that
Figure BDA0003088239130000074
The value of L with the maximum value is used as the mutation point L of the light intensity correction0Xi at this timeLAnd/xi is a correction factor. Correcting the light intensity sequence according to the equations (9) and (10), namely multiplying the data with the sequence number larger than L by a correction factor xiLAnd/xi, forming a new light intensity sequence.
And S5, inputting the corrected data into a dual-channel image reconstruction model, and regressing the data into a two-dimensional image after passing through two channels to obtain a clear deep learning ghost imaging image.
Adopt laser range finder range finding to guarantee that scattering distance images about 50m in Shenzhen urban area, this experiment is gone on under the dense fog scene at certain night, and the model machine can be verified under the comparatively extreme haze environment of reality. The target used a standard test plate printed with black and white stripes and a three-dimensional shaped ornament.
As shown in fig. 5A to 5F, it can be seen from the reconstruction result and the contrast quantization value that the method has a significant effect on improving the contrast of the black and white stripes. When the compression ratio is not more than 20, the reconstructed contrast ratio is maintained above 200, and at a compression ratio of 40, a reconstructed result with a contrast ratio of 62 can still be obtained.
In order to verify the advantages of the PC-GI network (denoted as MC-PC-GI) of the monte carlo simulation data, as shown in fig. 6A to 6D, the PC-GI network of the non-scattering simulation data and the CSGI network of the reconstruction method based on the compressed sensing are used for experimental comparison, the result of training the PC-GI network by using the non-scattering simulation data is slightly better than that of the CSGI method, the detailed information is sufficient, but the noise points are too much, in comparison, the MC-PC-GI method drives the network training under more ideal data, the regression result is more accurate, and the robustness is stronger under the strong-scattering imaging environment.
The double-channel ghost imaging reconstruction method and the system based on the human eye-imitating mechanism have the following beneficial effects:
1) the method comprises the steps of providing a dual-channel image reconstruction model based on a residual block convolutional neural network, considering data training modes of different dimensions, improving comprehensive extraction capacity and regression capacity of one-dimensional light intensity and two-dimensional image characteristics from a model design level, acquiring light intensity data for model training and inputting through a human eye simulation mechanism, enhancing the light intensity data characteristics and the overall image quality through key acquisition of an interested region, facilitating realization of imaging process parameterization of an imaging optical system, and further improving image accuracy in a deep neural network;
2) in order to restore the interframe light intensity correlation under the dynamic chaotic scattering medium, a nonlinear light intensity correction method is provided, and a compensation factor is designed to eliminate the nonlinear influence of the chaotic medium;
3) because the actual light intensity data is collected in a strong scattering environment, the training set data also has strong scattering characteristics, namely, the input light intensity signal should be degraded by scattering, and in order to eliminate the influence of a scattering medium positioned in an emission path on imaging, a Monte Carlo simulation mode is adopted to simulate the scattering process and create a data set to be trained.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.

Claims (10)

1. A double-channel ghost imaging reconstruction method based on a human eye simulation mechanism is characterized by comprising the following steps:
s1, constructing a dual-channel image reconstruction model;
s2, making a training data set, training the dual-channel image reconstruction model, and training the dual-channel image reconstruction model to be convergent based on the training data set;
s3, generating a measurement matrix based on a human eye simulation strategy, loading the measurement matrix to a micro mirror array, synchronously controlling the micro mirror array and a photoelectric detector, and acquiring a light intensity sequence by the photoelectric detector to finish data acquisition of a target sample;
s4, correcting the acquired data by adopting a nonlinear light intensity correction method;
and S5, inputting the corrected data into the dual-channel image reconstruction model, and regressing the data into a two-dimensional image after passing through two channels to obtain a clear deep learning ghost imaging image.
2. The human eye mechanism-like two-channel ghost imaging reconstruction method according to claim 1, wherein the two-channel image reconstruction model adopts a two-channel enhanced neural network, two branches of the two-channel enhanced neural network are a ghost image branch and a light intensity residual branch, and data sources thereof are a ghost image reconstruction image and a light intensity characteristic sequence respectively.
3. The two-channel ghost-imaging reconstruction method based on human-eye-like mechanism according to claim 2, wherein the ghost-imaging reconstructed image is calculated by the following formula:
I(x)＝<(Si-<Si>)(Pi-<Pi>)>
wherein<·>Representing the arithmetic mean of all measurements, SiLight intensity sequence, P, representing network inputiRepresenting a random measurement pattern used to acquire light intensity data.
4. The two-channel ghost imaging reconstruction method based on human eye-mimicking mechanism of claim 2, wherein the ghost image branch obtains the preliminary feature image from six convolutional layers, and the feature map dimensions of the six convolutional layers are 128 × 128 × 1, 128 × 128 × 32, 128 × 128 × 16, 128 × 128 × 16, 128 × 128 × 32, and 128 × 128 × 1, respectively.
5. The eye-mimicking mechanism two-channel ghost imaging reconstruction method of claim 2, wherein the intensity residual branch comprises a fully-connected layer and a two-dimensional residual block and two convolutional layers, the fully-connected layer being located between the sequence of measurement values and the convolutional layers.
6. The method for reconstructing two-channel ghost imaging based on human eye simulation mechanism according to claim 1, wherein in the step S2, a human eye simulation strategy is adopted to generate simulated light intensity for a specific image set, a monte carlo simulation mode is adopted to simulate a scattering process, a degraded data set with a scattering characteristic of an emission path is introduced, and a data set to be trained is created.
7. The method for reconstructing two-channel ghost imaging based on human eye-imitated mechanism according to claim 1, wherein in the step S4, the nonlinear optical intensity correction method specifically includes that when the optical system receives the scattering medium interference, the measured value is disturbed, assuming that the attenuation coefficient of the previous L frames is ξ, and after L measurements, the attenuation coefficient of the scattering medium is abruptly changed, the following formula can be obtained:
Figure FDA0003088239120000021
wherein y is the light intensity measurement value, M represents the total number of measurements, N represents the length of the one-dimensional signal, and the following formula can be obtained by dividing the measurement average values before and after the dynamic change according to the characteristics of the ghost imaging and the compressed sensing algorithm:
Figure FDA0003088239120000022
multiplying the data with the serial number greater than L by the correction factor xi according to the formulaLAnd/xi, forming a new light intensity sequence.
8. A two-channel ghost imaging reconstruction system based on a human eye simulating mechanism, which comprises a central processing unit, a memory and a data acquisition part, wherein the memory stores a two-channel image reconstruction model and a program which can be executed by the central processing unit, and the program can realize the two-channel ghost imaging reconstruction method based on the human eye simulating mechanism according to any one of claims 1-7 in the process of being executed by the central processing unit.
9. The human-eye-like mechanism-based two-channel ghost imaging reconstruction system of claim 8, wherein the data acquisition portion comprises a micromirror array, a photodetector, an imaging lens, and a laser.
10. A computer-readable storage medium, in which a dual-channel image reconstruction model and a program executable by a central processor are stored, the program being capable of implementing the dual-channel ghost imaging reconstruction method based on an eye-mimicking mechanism as claimed in any one of claims 1-7 during execution by the central processor.
CN202110587552.7A 2021-05-27 2021-05-27 Double-channel ghost imaging reconstruction method and system based on human eye imitation mechanism Active CN113240610B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110587552.7A CN113240610B (en) 2021-05-27 2021-05-27 Double-channel ghost imaging reconstruction method and system based on human eye imitation mechanism

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110587552.7A CN113240610B (en) 2021-05-27 2021-05-27 Double-channel ghost imaging reconstruction method and system based on human eye imitation mechanism

Publications (2)

Publication Number Publication Date
CN113240610A true CN113240610A (en) 2021-08-10
CN113240610B CN113240610B (en) 2023-05-12

Family

ID=77139278

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110587552.7A Active CN113240610B (en) 2021-05-27 2021-05-27 Double-channel ghost imaging reconstruction method and system based on human eye imitation mechanism

Country Status (1)

Country Link
CN (1) CN113240610B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113763540A (en) * 2021-09-08 2021-12-07 四川川大智胜软件股份有限公司 Three-dimensional reconstruction method and equipment based on speckle fringe hybrid modulation

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106646512A (en) * 2016-12-29 2017-05-10 北京理工大学 Ghost imaging method and ghost imaging system based on bionic vision mechanism
CN109752844A (en) * 2019-03-14 2019-05-14 中国科学院微电子研究所 A kind of imaging method and system based on random Intensity Fluctuation
CN110675326A (en) * 2019-07-24 2020-01-10 西安理工大学 Method for calculating ghost imaging reconstruction recovery based on U-Net network
US20200234471A1 (en) * 2019-01-18 2020-07-23 Canon Medical Systems Corporation Deep-learning-based scatter estimation and correction for x-ray projection data and computer tomography (ct)
CN111448590A (en) * 2017-09-28 2020-07-24 皇家飞利浦有限公司 Scatter correction based on deep learning
CN111551955A (en) * 2020-06-22 2020-08-18 北京理工大学 Bionic blocking ghost imaging method and system
CN111652059A (en) * 2020-04-27 2020-09-11 西北大学 Target identification model construction and identification method and device based on computational ghost imaging
US10802066B1 (en) * 2019-12-17 2020-10-13 Quantum Valley Ideas Laboratories Single-pixel imaging of electromagnetic fields
WO2021009017A1 (en) * 2019-07-12 2021-01-21 University College Cork - National University Of Ireland, Cork A method and system for performing high speed optical image detection
US20210144278A1 (en) * 2017-06-01 2021-05-13 South China Normal University Compressed sensing based object imaging system and imaging method therefor
CN112802145A (en) * 2021-01-27 2021-05-14 四川大学 Color calculation ghost imaging method based on deep learning

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106646512A (en) * 2016-12-29 2017-05-10 北京理工大学 Ghost imaging method and ghost imaging system based on bionic vision mechanism
US20210144278A1 (en) * 2017-06-01 2021-05-13 South China Normal University Compressed sensing based object imaging system and imaging method therefor
CN111448590A (en) * 2017-09-28 2020-07-24 皇家飞利浦有限公司 Scatter correction based on deep learning
US20200273214A1 (en) * 2017-09-28 2020-08-27 Koninklijke Philips N.V. Deep learning based scatter correction
US20200234471A1 (en) * 2019-01-18 2020-07-23 Canon Medical Systems Corporation Deep-learning-based scatter estimation and correction for x-ray projection data and computer tomography (ct)
CN109752844A (en) * 2019-03-14 2019-05-14 中国科学院微电子研究所 A kind of imaging method and system based on random Intensity Fluctuation
WO2021009017A1 (en) * 2019-07-12 2021-01-21 University College Cork - National University Of Ireland, Cork A method and system for performing high speed optical image detection
CN110675326A (en) * 2019-07-24 2020-01-10 西安理工大学 Method for calculating ghost imaging reconstruction recovery based on U-Net network
US10802066B1 (en) * 2019-12-17 2020-10-13 Quantum Valley Ideas Laboratories Single-pixel imaging of electromagnetic fields
CN111652059A (en) * 2020-04-27 2020-09-11 西北大学 Target identification model construction and identification method and device based on computational ghost imaging
CN111551955A (en) * 2020-06-22 2020-08-18 北京理工大学 Bionic blocking ghost imaging method and system
CN112802145A (en) * 2021-01-27 2021-05-14 四川大学 Color calculation ghost imaging method based on deep learning

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
TOMOYOSHI SHIMOBABA ET AL: "Computational ghost imaging using deep learning", 《ARXIV》 *
冯维 等: "基于卷积神经网络的计算鬼成像方法研究", 《光子学报》 *
庄佳衍 等: "基于压缩感知的动态散射成像", 《物理学报》 *
杜永成 等: "辐射传输介质散射相函数的蒙特卡洛算法", 《光谱学与光谱分析》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113763540A (en) * 2021-09-08 2021-12-07 四川川大智胜软件股份有限公司 Three-dimensional reconstruction method and equipment based on speckle fringe hybrid modulation

Also Published As

Publication number Publication date
CN113240610B (en) 2023-05-12

Similar Documents

Publication Publication Date Title
US11200456B2 (en) Systems and methods for generating augmented training data for machine learning models
US20200057831A1 (en) Real-time generation of synthetic data from multi-shot structured light sensors for three-dimensional object pose estimation
CN110675326B (en) Method for calculating ghost imaging reconstruction recovery based on U-Net network
CN110187143B (en) Chromatography PIV reconstruction method and device based on deep neural network
Wang et al. TPSSI-Net: Fast and enhanced two-path iterative network for 3D SAR sparse imaging
CN111833248B (en) Super-resolution ghost imaging method and system based on partial Hadamard matrix
CN110889426A (en) Three-dimensional expression generation system
CN106686281B (en) Fuse circuit board noise suppression ability test system
US20240095999A1 (en) Neural radiance field rig for human 3d shape and appearance modelling
CN109991602A (en) ISAR image resolution enhancement method based on depth residual error network
Nakashima et al. Learning to drop points for lidar scan synthesis
CN113240610B (en) Double-channel ghost imaging reconstruction method and system based on human eye imitation mechanism
EP4191539A1 (en) Method for performing volumetric reconstruction
Plack et al. Fast differentiable transient rendering for non-line-of-sight reconstruction
CN113724146A (en) Single-pixel imaging method based on plug-and-play prior
CN111352126B (en) Single-pixel imaging method based on atmospheric scattering medium modulation
Shen et al. U 2 PNet: An Unsupervised Underwater Image-Restoration Network Using Polarization
CN115760603A (en) Interference array broadband imaging method based on big data technology
WO2023023961A1 (en) Piv image calibration apparatus and method based on laser linear array
CN104777329B (en) A kind of linear programming algorithm for the reconstruct of particle image velocimetry three dimensional particles field
CN116977535B (en) Real-time ray tracing method and device, storage medium and electronic equipment
Li et al. Autoencoder for Single-pixel imaging
CN110533706B (en) Three-dimensional target reconstruction method based on computational ghost imaging
WO2024117090A1 (en) Information processing device, information processing method, and program
KR102648938B1 (en) Method and apparatus for 3D image reconstruction based on few-shot neural radiance fields using geometric consistency

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20221123

Address after: Second floor, building a, Tsinghua campus, Shenzhen University Town, Xili street, Nanshan District, Shenzhen City, Guangdong Province

Applicant after: Tsinghua Shenzhen International Graduate School

Applicant after: BEIJING INSTITUTE OF TECHNOLOGY

Address before: Second floor, building a, Tsinghua campus, Shenzhen University Town, Xili street, Nanshan District, Shenzhen City, Guangdong Province

Applicant before: Tsinghua Shenzhen International Graduate School

GR01 Patent grant
GR01 Patent grant