CN115220061A - Deep learning polarization ghost imaging method and system based on orthogonal normalization - Google Patents
Deep learning polarization ghost imaging method and system based on orthogonal normalization Download PDFInfo
- Publication number
- CN115220061A CN115220061A CN202210836280.4A CN202210836280A CN115220061A CN 115220061 A CN115220061 A CN 115220061A CN 202210836280 A CN202210836280 A CN 202210836280A CN 115220061 A CN115220061 A CN 115220061A
- Authority
- CN
- China
- Prior art keywords
- neural network
- ghost imaging
- barrel detector
- network model
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 86
- 238000010606 normalization Methods 0.000 title claims abstract description 36
- 238000013135 deep learning Methods 0.000 title claims abstract description 34
- 230000010287 polarization Effects 0.000 title claims abstract description 31
- 238000005070 sampling Methods 0.000 claims abstract description 83
- 238000000034 method Methods 0.000 claims abstract description 29
- 238000012545 processing Methods 0.000 claims abstract description 27
- 238000012549 training Methods 0.000 claims abstract description 26
- 239000013598 vector Substances 0.000 claims abstract description 15
- 238000003062 neural network model Methods 0.000 claims description 61
- 238000012360 testing method Methods 0.000 claims description 22
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000012795 verification Methods 0.000 claims description 9
- 238000010276 construction Methods 0.000 claims description 7
- 230000001678 irradiating effect Effects 0.000 claims description 6
- 238000002310 reflectometry Methods 0.000 claims description 5
- 239000000463 material Substances 0.000 claims description 3
- 229910000831 Steel Inorganic materials 0.000 claims description 2
- 239000010959 steel Substances 0.000 claims description 2
- 238000013528 artificial neural network Methods 0.000 abstract description 5
- 239000000284 extract Substances 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 18
- 238000004590 computer program Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 11
- 230000000694 effects Effects 0.000 description 11
- 238000000605 extraction Methods 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000013515 script Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- CWYNVVGOOAEACU-UHFFFAOYSA-N Fe2+ Chemical compound [Fe+2] CWYNVVGOOAEACU-UHFFFAOYSA-N 0.000 description 1
- 238000012952 Resampling Methods 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000001010 compromised effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- NAIXASFEPQPICN-UHFFFAOYSA-O p-nitrophenylphosphocholine Chemical compound C[N+](C)(C)CCOP(O)(=O)OC1=CC=C([N+]([O-])=O)C=C1 NAIXASFEPQPICN-UHFFFAOYSA-O 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
- 239000013077 target material Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/481—Constructional features, e.g. arrangements of optical elements
- G01S7/4811—Constructional features, e.g. arrangements of optical elements common to transmitter and receiver
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Networks & Wireless Communication (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Electromagnetism (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a deep learning polarization ghost imaging method and system based on orthogonal normalization.A target is illuminated by using a random light field, and a one-dimensional vector formed by orthogonal normalized barrel detector values is input into a neural network so as to recover a clear target image; under the condition of extremely low sampling rate, the method can still recover a very clear image and has good generalization. The invention effectively and uniformly extracts the target information by using the random light field, so that the barrel detector always has rich target information, and then the orthogonal normalization processing is carried out, thereby being more efficient for training the neural network. The efficiency and the imaging speed of the deep learning ghost imaging are greatly improved.
Description
[ technical field ] A method for producing a semiconductor device
The invention belongs to the technical field of imaging, and particularly relates to a deep learning polarization ghost imaging method and system based on orthogonal normalization.
[ background of the invention ]
In an imaging system, since there are many environments where the reflectivity of an object is close to that of a background, the contrast of an image is low when imaging. Under the addition of polarized light, the information of the target can be effectively extracted from the background by the characteristic that the polarization characteristics of the target and the background are greatly different.
In recent years, many researchers provide ghost imaging based on deep learning, the structure of a network is continuously optimized, the sampling rate of the ghost imaging is greatly reduced, the imaging efficiency is improved, and deep learning brings a new mode for development of ghost imaging. In most deep learning ghost imaging, the light field used for sampling is the Hadamard light field, and due to the special orthogonality, the imaging effect can be greatly improved. However, the special mathematical characteristics of the Hadamard light field also determine the limitation of the sampling region, many Hadamard light fields are blocky, the region where the target is located is unknown in the sampling process, and the bright part of the light field does not irradiate on the target so that the information of the target is likely to be lost easily during sampling. In addition, aiming at different target types and possible environmental changes, the reconstruction effect brought by the same image reconstruction method can be changed under the condition of the same sampling rate, and the existing reconstruction method is unknown to the reconstruction effect and does not fully utilize prior information; under the condition, how to effectively extract target information in the sampling process is a problem to be solved, so that the imaging efficiency and the imaging quality of ghost imaging can be improved;
the invention effectively and uniformly extracts the target information by using the orthogonal random light field, so that the barrel detector always has rich target information, and the training of the neural network is more efficient. The imaging efficiency and the imaging speed of the deep learning ghost imaging are greatly improved. By setting the comparison table, historical prior information of the neural network model and the environment can be utilized, so that a reconstructed image with stable effect can be obtained if the sampling time is visible for a user, and the user experience is greatly improved.
[ summary of the invention ]
In order to solve the above problems in the prior art, the present invention provides a deep learning polarization ghost imaging method and system based on orthogonal normalization, wherein the method includes:
step S1: building a reconstruction environment, and sampling a target by using horizontal polarized light; the method specifically comprises the following steps: the transmitting end utilizes a laser light source to emit light onto the polarizing plate after passing through the prism group to obtain horizontal polarized light, and then the horizontal polarized light is modulated into m different random light fields R by the digital micromirror device DMD 1 ,R 2 ,R 3 ,···,R m Irradiating the target, dividing the reflected light beam into horizontal polarized light and vertical polarized light through a polarization beam splitter after the reflected light beam passes through a third prism, receiving and combining the horizontal polarized light and the vertical polarized light by a first barrel detector and a second barrel detector respectively to form a barrel detector value S, and transmitting the barrel detector value S to a calculation unit; wherein: the first barrel detector and the second barrel detector are arranged in a 90-degree intersection manner; each bucket detector value S is a one-dimensional vector;
step S2: orthogonalizing and normalizing the random light field to form a new barrel detector value; normalizing the corresponding barrel detector values and putting the normalized barrel detector values into a sample set as samples;
and step S3: constructing a neural network model based on deep learning;
and step S4: training a neural network model;
step S5: testing and verifying the neural network model by using the sample set;
step S6: testing and verifying the neural network model by using a network sample set;
step S7: acquiring a ghost imaging request, judging whether the ghost imaging request is similar to a historical reconstruction environment, if so, acquiring a barrel detector value based on the ghost imaging request, performing orthogonal normalization processing to obtain a new barrel detector value, and inputting the new barrel detector value into a neural network model to obtain a reconstruction result; otherwise, acquiring a target sampling rate based on the ghost imaging request, returning to the step S1 to rebuild the reconstruction environment based on the target sampling rate, so that the rebuilt reconstruction environment is similar to the ghost imaging request; executing the step S2 to obtain an updated sample set, and executing the step S4 to train a neural network model based on the updated sample set; acquiring a barrel detector value based on the ghost imaging request, performing orthogonal normalization processing to obtain a new barrel detector value, and inputting the new barrel detector value into the retrained neural network model to obtain a reconstruction result.
Further, the neural network model is an improved U-net neural network model.
Further, the target and background reflectivities are close.
Further, the object is composed of ferrous materials.
Further, the transmitting end is a laser.
A deep learning polarization ghost imaging system based on orthogonal normalization, the system comprising:
an environment reconstruction module: constructing a reconstruction environment, and performing ghost imaging by using horizontal polarized light through line sampling; the method specifically comprises the following steps: the transmitting end utilizes a laser light source to emit light onto the polarizing plate after passing through the prism group to obtain horizontal polarized light, and then the horizontal polarized light is modulated into m different random light fields R by the digital micromirror device DMD 1 ,R 2 ,R 3 ,···,R m Irradiating the target, dividing the reflected light beam into horizontal polarized light and vertical polarized light through a polarization beam splitter after the reflected light beam passes through a third prism, receiving and combining the horizontal polarized light and the vertical polarized light by a first barrel detector and a second barrel detector respectively to form a barrel detector value S, and transmitting the barrel detector value S to a calculation unit; wherein: the first barrel detector and the second barrel detector are arranged in a 90-degree intersection manner; each bucket detector value S is a one-dimensional vector;
a sample construction module: orthogonalizing and normalizing the random light field to form a new barrel detector value; normalizing the corresponding barrel detector values and putting the normalized barrel detector values into a sample set as samples;
a model construction module: constructing a neural network model based on deep learning;
a model training module: training a neural network model;
a verification module: testing and verifying the neural network model by using the sample set;
a generalization module: testing and verifying the neural network model by using a network sample set;
a request processing module: acquiring a ghost imaging request, judging whether the ghost imaging request is similar to a historical reconstruction environment, if so, acquiring a barrel detector value based on the ghost imaging request, performing orthogonal normalization processing to obtain a new barrel detector value, and inputting the new barrel detector value into a neural network model to obtain a reconstruction result; otherwise, acquiring a target sampling rate based on the ghost imaging request, and rebuilding a rebuilding environment by an environment rebuilding module based on the target sampling rate so as to enable the rebuilt rebuilding environment to be similar to the ghost imaging request; acquiring an updated sample set, and training a neural network model based on the updated sample set; acquiring a barrel detector value based on the ghost imaging request, performing orthogonal normalization processing to obtain a new barrel detector value, and inputting the new barrel detector value into the retrained neural network model to obtain a reconstruction result.
Further, the neural network model is an improved U-net neural network model.
A processor configured to execute a program, wherein the program is configured to perform the method for deep-learning polarization ghost imaging based on orthogonal normalization when executed.
A computer-readable storage medium containing a program which, when run on a computer, causes the computer to execute the method for deep learning polarization ghost imaging based on orthogonal normalization.
An execution device comprising a processor coupled to a memory, the memory storing program instructions that, when executed by the processor, implement the method for orthonormal based deep learning polarization ghost imaging.
The beneficial effects of the invention include:
(1) The method effectively and uniformly extracts the target information by utilizing the difference between the orthogonal random light field and the Hardman light field, so that the barrel detector always has rich target information, the training of a neural network is more efficient, the polarized light is used for extracting the polarization characteristic of the target, and in an imaging system of ghost imaging, an image with high contrast can be recovered under the condition that the reflectivity of the target is close to that of the background; (2) Calculating a dynamic adjustment reconstruction environment based on the sampling rate, performing model training under the condition that the sampling time is met, realizing a model construction mode combining static + dynamic + generalization training, adjusting a field reconstruction environment under the condition of the least sampling time, and improving the reconstruction effect; (3) By setting the first and second comparison tables, the adjustment of the m value is combined with the requirement of the reconstruction request, the historical prior information of the field environment can be obtained, and on the basis of meeting the requirement of the reconstruction request, the most abundant information can be obtained by the least acquisition environment adjustment;
[ description of the drawings ]
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, and are not to be considered limiting of the invention, in which:
FIG. 1 is a schematic diagram of a deep learning polarization ghost imaging method based on orthogonal normalization according to the present invention.
FIG. 2 is a schematic diagram of a ghost imaging method using horizontal polarized light line sampling according to the present invention.
FIG. 3 is a graphical representation of the results of the imaging test of the present invention.
FIG. 4 is a diagram illustrating the results of the generalization test performed in accordance with the present invention.
[ detailed description ] embodiments
The present invention will now be described in detail with reference to the drawings and specific embodiments, wherein the exemplary embodiments and descriptions are only intended to illustrate the present invention, but not to limit the present invention.
As shown in fig. 1, in the present invention, a random light field is formed by modulating emitted polarized light to illuminate a target, then a barrel detector receives target information after passing through a polarization beam splitter, a large number of targets are illuminated to obtain a vector formed by barrel detector values related to the target, and a data set is formed based on the vector to train a constructed neural network model, wherein a target original image is used as a label during training, and a part of the data set is used for testing, so that a high-quality image is recovered at a high rate and high efficiency;
as shown in fig. 1, the present invention provides a deep learning polarization ghost imaging method based on orthogonal normalization, which includes the following steps:
step S1: building a reconstruction environment, and sampling a target by using horizontal polarized light; (ii) a As shown in fig. 2, the step S1 specifically includes: the transmitting end utilizes a laser light source to emit light onto the polarizing plate after passing through the prism group to obtain horizontal polarized light, and then the horizontal polarized light is modulated into m different random light fields R by the digital micromirror device DMD 1 ,R 2 ,R 3 ,···,R m Irradiating the target, dividing the reflected light beam into horizontal polarized light and vertical polarized light through a polarization beam splitter after the reflected light beam passes through a third prism, receiving and combining the horizontal polarized light and the vertical polarized light by a first barrel detector and a second barrel detector respectively to form a barrel detector value S, and transmitting the barrel detector value S to a calculation unit; wherein: the first barrel detector and the second barrel detector are arranged in a 90-degree intersection manner; each bucket detector value S is a one-dimensional vector;
preferably: the m is dynamically set according to the reconstruction environment adjustment based on the sampling rate;
preferably: the reflectivity of the target and the background is close, and the difference of the polarization characteristics is large;
preferably: the target is composed of steel materials; the transmitting end is a laser;
preferably: sampling 5000 digital targets in the MINIST data set to obtain vectors consisting of 5000 bucket detector values corresponding to the targets;
step S2: orthogonalizing and normalizing the random light field to form a new barrel detector value; normalizing the corresponding barrel detector values and putting the normalized barrel detector values into a sample set as samples; specifically, the method comprises the following steps: the computing unit utilizes the formulas (1) and (2) to convert the random light field R into a random light field R 1 ,R 2 ,R 3 ,···,R m Performing Schmidt orthogonalization to obtain an orthogonalized random light field, and performing normalization by using a formula (3); the barrel detector values S corresponding to each light field are subjected to orthogonal normalization using formula (4) to obtain a new set of barrel detector values
Wherein: if the target has N pixels, the sampling frequency is m times, namely m random light fields are used for sampling the target, m barrel detector values of each target form m one-dimensional vectors S, and m new barrel detector values are obtained after orthogonal normalization processing
New bucket detector value
Putting samples serving as neural network models into a sample set;
wherein: wherein c is mn Is a projection coefficient calculated from the illumination field; n is an element of [1 to N ∈]
The number of the bucket detector values obtained for the same target can be adjusted by adjusting the DMD, so that the sampling rate and the subsequent reconstruction effect are further adjusted; the expansion of the m value is combined with the requirement of the reconstruction request, and on the basis of meeting the requirement of the reconstruction request, the richest information is obtained through the least adjustment of the acquisition environment; then when the number of targets is 5000, through the above process, 5000 one-dimensional vectors are obtained, each one-dimensional vector corresponding to a corresponding target; the 5000 vectors form a training set containing 5000 elements, and are used for training a neural network model;
and step S3: constructing a neural network model based on deep learning;
preferably: the neural network model is an improved U-net neural network model; the improved U-net neural network model comprises a feature extraction part, a jump link part and an up-sampling part; wherein: the feature extraction part and the up-sampling part are used for extracting vector features to obtain feature information; the jump link part is used for fusing characteristic information;
the improved U-Net neural network model takes a DenseNet network as a U-Net feature extraction network, wherein the DenseNet is a composite layer consisting of four dense blocks, and each layer in the dense blocks is connected with the next layer through connection operation, so that the connection method enables the transfer of features and gradients to be more effective and the training process of the network to be easier;
preferably: inputting part of samples in the sample set into a neural network model to carry out model training;
and step S4: training a neural network model; specifically, the method comprises the following steps: setting the following formula (5) as an objective function NPCC of the model of the neural network; taking a first part in the sample set as training, and using a second part for verification and testing; when the target function meets a preset target value, the training is terminated;
where w is the image width and h is the image height; g1 and Y1 represent the mean of the original target and the reconstructed image, respectively, Y (i, j) represents the reconstructed image data, and G (i, j) is the original image data;
preferably: the first portion is 80% of the training set, the second portion is 20%;
preferably: the activation function is a corrected linear unit ReLU, and the network can be trained quickly and effectively;
preferably: training of the neural network model is carried out in an image processing unit (NVIDIARTX 3090) by using a Pythroch frame with Python 3.6;
step S5: testing and verifying the neural network model by using the sample set, and setting a first comparison table of sampling rate and sampling time based on the verification and test result; setting a second comparison table between the sampling rate and the objective function value; wherein, a sampling rate alpha = m/N is defined;
the first comparison table for setting the sampling rate and the sampling time based on the verification and test results specifically comprises the following steps: the first comparison table is set according to different sampling time ranges, one sampling time range corresponds to one or more sampling rate ranges, after one verification and test is completed each time, the sampling rate is calculated, the current sampling time is recorded, a record of the time range hit by the current sampling time is found, and the record corresponding to the corresponding sampling time is covered after comprehensive calculation is performed on the basis of the sampling rate and the historical sampling rate; the comprehensive calculation can be weighting, averaging and the like;
the method comprises the following steps of performing comprehensive calculation based on the current sampling rate and the historical sampling rate, and then covering a record corresponding to the corresponding sampling time, specifically: let the record hit at the current sample time be (area (TL, TU), SPR (SPL, SPU)); judging whether the current sampling rate CR _ SPR falls into SPR (SPL, SPU), and if so, keeping the record unchanged; if not, adjusting SPR (SPL, SPU) according to the current sampling rate CR _ SPR;
the SPR (SPL, SPU, specifically: when) is adjusted according to the current sampling rate CR _ SPR
And (ω 1 × SPL + ω 2 × CR _ SPR)/2 is less than SP, replacing the SPL with (ω 1 × SPL + ω 2 × CR _ SPR)/2; epsilon is the adjustment step, omega 1+ omega 2=1; for example: ε =0.01;
when (ω 1 × SPL + ω 2 × CR _ SPR)/2 < (1 + ∈) × SPU and (ω 1 × SPU + ω 2 × CR _ SPR)/2 is greater than SPU, replacing SPU with (ω 1 × SPU + ω 2 × CR _ SPR)/2;
by the mode, the record can be dynamically adjusted in a small step, and meanwhile, the calculation cost is relatively low; of course, the number of times of adjustment may be calculated, and when the number of times of adjustment or the frequency is high, all the historical data may be clustered again to initialize the first lookup table;
preferably: the sampling time ranges in the first comparison table are overlapped, and the sampling rate ranges are not overlapped;
preferably: setting an initial first comparison table and a second comparison table in a mode of clustering historical data;
the second comparison table between the set sampling rate and the objective function value specifically comprises the following steps: the second comparison table is set according to different target function value ranges, one target function value range corresponds to one sampling rate range, after verification and test are completed once each time, the current sampling rate is calculated, the current target function value is recorded, the record of the target function value range hit by the current target function value is found, and the record of the target function value range corresponding to the target function value is covered after comprehensive calculation is carried out by the current sampling rate and the historical sampling rate; the processing mode is similar to that of the first comparison table;
under the condition of different target types and possible environmental changes such as illumination and the like, the reconstruction effect brought by the same image reconstruction method can be changed under the condition of the same sampling rate, and the historical prior information of the neural network model and the environment can be utilized by setting a comparison table, so that the reconstructed image with stable effect can be obtained when the sampling time is visible for a user, and the user experience is greatly improved;
step S6: testing and verifying the neural network model by using a network sample set; the steps further include: adjusting the first comparison table and the second comparison table according to the verification and test results obtained by adopting the network sample set; the network sample set is different from the samples obtained by the reconstruction environment, the network sample set can be regarded as a static sample, the generalization processing capability of the model can be improved, the samples obtained in the reconstruction environment can be regarded as dynamic samples, the model can be adjusted to the field reconstruction environment under the condition of the least sampling time, and the reconstruction effect is improved;
adjusting the first comparison table and the second comparison table by using the network generalization test result, wherein the adjustment mode is the same as the setting mode of the first comparison table;
step S7: acquiring a ghost imaging request, judging whether the ghost imaging request is similar to a historical reconstruction environment or not, if so, acquiring a barrel detector value based on the ghost imaging request, performing orthogonal normalization processing to obtain a new barrel detector value, and inputting the new barrel detector value into a neural network model to obtain a reconstruction result; otherwise, inquiring the first comparison table and the second comparison table based on the ghost imaging request to obtain a target sampling rate, and returning to the step S1 to rebuild the reconstruction environment based on the target sampling rate, so that the rebuilt reconstruction environment is similar to the ghost imaging request; executing the step S2 to obtain an updated sample set, and executing the step S4 to train a neural network model based on the updated sample set; acquiring a barrel detector value based on a ghost imaging request, performing orthogonal normalization processing to obtain a new barrel detector value, and inputting the new barrel detector value into a retrained neural network model to obtain a reconstruction result;
when the ghost imaging request is acquired again, the step S7 is executed again, and the times of executing the steps S1, S2 and S4 again are gradually reduced along with the stabilization and generalization of the model;
preferably: the number of the historical reconstruction environments is one or more; the historical reconstruction environment is the reconstruction environment set up in the step S1 each time;
the judging whether the ghost imaging request is similar to the historical reconstruction environment specifically comprises the following steps: judging whether a target type (such as a target material) and a light field involved in the ghost imaging request are similar to one or more historical reconstruction environments or not, if so, determining that the target type and the light field are similar, otherwise, determining that the target type and the light field are not similar;
the querying the first comparison table and the second comparison table based on the request to obtain the sampling rate specifically comprises: acquiring request parameters such as sampling time and the like in a request, and converting the request parameters into corresponding reconstruction request parameters; wherein: the reconstruction requirement parameters comprise an objective function value, the number of target pixels, a sampling time target type, light field attributes and the like; inquiring the first comparison table and the second comparison table based on the sampling time and the objective function value to obtain two sampling rates respectively corresponding to the two tables, and selecting the higher one of the two sampling rates as an objective sampling rate; when the reconstruction requirement parameters comprise the number of target pixels, calculating the number of random light fields according to the target sampling rate and the number of the target pixels, and adjusting a reconstruction environment according to the number of the random light fields; of course, when the sampling time is a limit, the number of retrains must be compromised;
as shown in fig. 3-4, the reconstruction conditions of the target image at different sampling rates are respectively tested, and when the sampling rate is 1.5%, the target image can still be recovered with high accuracy, the reconstruction effect is higher than that of the hadamard-light-field-based deep learning ghost imaging, and when the sampling rate is 1.5% (total pixels are 4096, and the resampling times are 60), the sampling time required under the same reconstruction effect is greatly reduced, and compared with the hadamard-light-field-based deep learning ghost imaging, the sampling rate is greatly reduced again;
based on the same inventive concept, the invention provides a deep learning polarization ghost imaging system based on orthogonal normalization;
an environment reconstruction module: constructing a reconstruction environment, and performing ghost imaging by using horizontal polarized light through line sampling; the method specifically comprises the following steps: the transmitting end utilizes a laser light source to emit light onto the polarizing plate after passing through the prism group to obtain horizontal polarized light, and then the horizontal polarized light is modulated into m different random light fields R by the digital micromirror device DMD 1 ,R 2 ,R 3 ,···,R m Irradiating the target, dividing the reflected light beam into horizontal polarized light and vertical polarized light through a polarization beam splitter after the reflected light beam passes through a third prism, receiving and combining the horizontal polarized light and the vertical polarized light by a first barrel detector and a second barrel detector respectively to form a barrel detector value S, and transmitting the barrel detector value S to a calculation unit; wherein: the first barrel detector and the second barrel detector are arranged in a 90-degree intersection manner; each bucket detector value S is a one-dimensional vector;
a sample construction module: orthogonalizing and normalizing the random light field to form a new barrel detector value; normalizing the corresponding barrel detector values and putting the normalized barrel detector values into a sample set as samples;
a model construction module: constructing a neural network model based on deep learning;
a model training module: training a neural network model;
a verification module: testing and verifying the neural network model by using the sample set;
a generalization module: testing and verifying the neural network model by using a network sample set;
a request processing module: acquiring a ghost imaging request, judging whether the ghost imaging request is similar to a historical reconstruction environment or not, if so, acquiring a barrel detector value based on the ghost imaging request, performing orthogonal normalization processing to obtain a new barrel detector value, and inputting the new barrel detector value into a neural network model to obtain a reconstruction result; otherwise, acquiring a target sampling rate based on the ghost imaging request, and rebuilding a rebuilding environment by an environment rebuilding module based on the target sampling rate so as to enable the rebuilt rebuilding environment to be similar to the ghost imaging request; acquiring an updated sample set, and training a neural network model based on the updated sample set; and acquiring a barrel detector value based on the ghost imaging request, performing orthogonal normalization processing to obtain a new barrel detector value, and inputting the new barrel detector value into the retrained neural network model to obtain a reconstruction result.
The terms "data processing apparatus", "data processing system", "user equipment" or "computing device" encompass all kinds of apparatus, devices and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or a plurality or combination of the above. The apparatus can comprise special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform execution environment, a virtual machine, or a combination of one or more of the above. The apparatus and execution environment may implement a variety of different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.
A computer program (also known as a program, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. The computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting the same, and although the present invention is described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications and equivalents may be made to the embodiments of the invention without departing from the spirit and scope of the invention, which is to be covered by the claims.
Claims (10)
1. A deep learning polarization ghost imaging method based on orthogonal normalization is characterized by comprising the following steps:
step S1: building a reconstruction environment, and sampling a target by using horizontal polarized light; the method specifically comprises the following steps: the transmitting end utilizes a laser light source to emit light onto the polarizing plate after passing through the prism group to obtain horizontal polarized light, and then the horizontal polarized light is modulated into m different random light fields R by the digital micromirror device DMD 1 ,R 2 ,R 3 ,···,R m Irradiating the target, dividing the reflected light beam into horizontal polarized light and vertical polarized light through a polarization beam splitter after the reflected light beam passes through a third prism, receiving and combining the horizontal polarized light and the vertical polarized light by a first barrel detector and a second barrel detector respectively to form a barrel detector value S, and transmitting the barrel detector value S to a calculation unit; wherein: the first barrel detector and the second barrel detector are arranged in a 90-degree intersection manner; each bucket detector value S is a one-dimensional vector;
step S2: orthogonalizing and normalizing the random light field to form a new barrel detector value; normalizing the corresponding barrel detector values and putting the normalized barrel detector values into a sample set as samples;
and step S3: constructing a neural network model based on deep learning;
and step S4: training a neural network model;
step S5: testing and verifying the neural network model by using the sample set;
step S6: testing and verifying the neural network model by using a network sample set;
step S7: acquiring a ghost imaging request, judging whether the ghost imaging request is similar to a historical reconstruction environment or not, if so, acquiring a barrel detector value based on the ghost imaging request, performing orthogonal normalization processing to obtain a new barrel detector value, and inputting the new barrel detector value into a neural network model to obtain a reconstruction result; otherwise, acquiring a target sampling rate based on the ghost imaging request, returning to the step S1 to rebuild the reconstruction environment based on the target sampling rate, so that the rebuilt reconstruction environment is similar to the ghost imaging request; executing the step S2 to obtain an updated sample set, and executing the step S4 to train a neural network model based on the updated sample set; acquiring a barrel detector value based on the ghost imaging request, performing orthogonal normalization processing to obtain a new barrel detector value, and inputting the new barrel detector value into the retrained neural network model to obtain a reconstruction result.
2. The deep learning polarization ghost imaging method based on orthogonal normalization of claim 1, wherein the neural network model is a modified U-net neural network model.
3. The method of claim 2, wherein the target and background reflectivities are close.
4. The deep learning polarization ghost imaging method based on orthogonal normalization according to claim 3, wherein the target is composed of steel material.
5. The deep learning polarization ghost imaging method based on orthogonal normalization of claim 4, wherein the transmitting end is a laser.
6. A deep learning polarization ghost imaging system based on orthogonal normalization, the system comprising:
an environment reconstruction module: constructing a reconstruction environment, and performing ghost imaging by using horizontal polarized light through line sampling; the method comprises the following specific steps: the transmitting end utilizes a laser light source to emit light onto the polarizing plate after passing through the prism group to obtain horizontal polarized light, and then the horizontal polarized light is modulated into m different random light fields R by the digital micromirror device DMD 1 ,R 2 ,R 3 ,···,R m Irradiating the target, dividing the reflected light beam into horizontal polarized light and vertical polarized light through a polarization beam splitter after the reflected light beam passes through a third prism, receiving and combining the horizontal polarized light and the vertical polarized light by a first barrel detector and a second barrel detector respectively to form a barrel detector value S, and transmitting the barrel detector value S to a calculation unit; wherein: the first barrel detector and the second barrel detector are arranged in a 90-degree intersection manner; each bucket detector value S is a one-dimensional vector;
a sample construction module: orthogonalizing and normalizing the random light field to form a new barrel detector value; normalizing the values of the barrel detectors correspondingly, and putting the normalized values serving as samples into a sample set;
a model construction module: constructing a neural network model based on deep learning;
a model training module: training a neural network model;
a verification module: testing and verifying the neural network model by using the sample set;
a generalization module: testing and verifying the neural network model by using a network sample set;
a request processing module: acquiring a ghost imaging request, judging whether the ghost imaging request is similar to a historical reconstruction environment, if so, acquiring a barrel detector value based on the ghost imaging request, performing orthogonal normalization processing to obtain a new barrel detector value, and inputting the new barrel detector value into a neural network model to obtain a reconstruction result; otherwise, acquiring a target sampling rate based on the ghost imaging request, and rebuilding a rebuilding environment by an environment rebuilding module based on the target sampling rate so as to enable the rebuilt rebuilding environment to be similar to the ghost imaging request; acquiring an updated sample set, and training a neural network model based on the updated sample set; acquiring a barrel detector value based on the ghost imaging request, performing orthogonal normalization processing to obtain a new barrel detector value, and inputting the new barrel detector value into the retrained neural network model to obtain a reconstruction result.
7. The deep learning polarization ghost imaging system based on orthogonal normalization of claim 6, wherein the neural network model is a modified U-net neural network model.
8. A processor, wherein the processor is configured to execute a program, wherein the program is executed to perform the method for deep learning polarization ghost imaging based on orthogonal normalization according to any of claims 1-5.
9. A computer-readable storage medium characterized by comprising a program which, when run on a computer, causes the computer to execute the method for deep learning polarization ghost imaging based on orthogonal normalization according to any one of claims 1 to 5.
10. An execution device comprising a processor coupled with a memory, the memory storing program instructions that, when executed by the processor, implement the method for deep learning polarization ghost imaging based on orthogonal normalization according to any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210836280.4A CN115220061B (en) | 2022-07-15 | Orthogonal normalization-based deep learning polarization ghost imaging method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210836280.4A CN115220061B (en) | 2022-07-15 | Orthogonal normalization-based deep learning polarization ghost imaging method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115220061A true CN115220061A (en) | 2022-10-21 |
| CN115220061B CN115220061B (en) | 2024-05-10 |
Family
ID=
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117201691A (en) * | 2023-11-02 | 2023-12-08 | 湘江实验室 | Panoramic scanning associated imaging method based on deep learning |
| CN117462073A (en) * | 2023-12-25 | 2024-01-30 | 西北工业大学宁波研究院 | Hand-held polarization imaging intraocular pressure detection device and method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019000659A1 (en) * | 2017-06-29 | 2019-01-03 | 清华大学深圳研究生院 | Imaging device and method under strong atmospheric scattering condition |
| CN110646810A (en) * | 2019-09-27 | 2020-01-03 | 北京理工大学 | Speckle optimization compressed sensing ghost imaging method and system |
| CN110675326A (en) * | 2019-07-24 | 2020-01-10 | 西安理工大学 | Method for calculating ghost imaging reconstruction recovery based on U-Net network |
| CN113219490A (en) * | 2021-05-14 | 2021-08-06 | 合肥工业大学 | Reflection type scanning calculation polarization ghost imaging system under haze dispersion system |
| WO2023045251A1 (en) * | 2021-09-24 | 2023-03-30 | 浪潮(北京)电子信息产业有限公司 | Noise cancellation method and system in computational ghost imaging sampling process, and related assembly |
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019000659A1 (en) * | 2017-06-29 | 2019-01-03 | 清华大学深圳研究生院 | Imaging device and method under strong atmospheric scattering condition |
| CN110675326A (en) * | 2019-07-24 | 2020-01-10 | 西安理工大学 | Method for calculating ghost imaging reconstruction recovery based on U-Net network |
| CN110646810A (en) * | 2019-09-27 | 2020-01-03 | 北京理工大学 | Speckle optimization compressed sensing ghost imaging method and system |
| CN113219490A (en) * | 2021-05-14 | 2021-08-06 | 合肥工业大学 | Reflection type scanning calculation polarization ghost imaging system under haze dispersion system |
| WO2023045251A1 (en) * | 2021-09-24 | 2023-03-30 | 浪潮(北京)电子信息产业有限公司 | Noise cancellation method and system in computational ghost imaging sampling process, and related assembly |
Non-Patent Citations (3)
| Title |
|---|
| 徐陈祥等: "High-performance deep-learning based polarization computational ghost imaging with random patterns and orthonormalization", 《PHYSICA SCRIPTA》, 9 May 2023 (2023-05-09) * |
| 曹非;郑素赢;赵生妹;郑宝玉;: "正交散斑鬼成像", 信号处理, no. 05, 25 May 2019 (2019-05-25) * |
| 郭忠义等: "影响计算鬼成像质量的两种关键技术", 《激光与光电子学进展》, 25 September 2021 (2021-09-25) * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117201691A (en) * | 2023-11-02 | 2023-12-08 | 湘江实验室 | Panoramic scanning associated imaging method based on deep learning |
| CN117201691B (en) * | 2023-11-02 | 2024-01-09 | 湘江实验室 | Panoramic scanning associated imaging method based on deep learning |
| CN117462073A (en) * | 2023-12-25 | 2024-01-30 | 西北工业大学宁波研究院 | Hand-held polarization imaging intraocular pressure detection device and method |
| CN117462073B (en) * | 2023-12-25 | 2024-04-19 | 西北工业大学宁波研究院 | Hand-held polarization imaging intraocular pressure detection device and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10984517B2 (en) | Utilizing a machine learning model to automatically visually validate a user interface for multiple platforms | |
| CN111833237B (en) | Image registration method based on convolutional neural network and local homography transformation | |
| Bonaldi et al. | Square kilometre array science data challenge 1: analysis and results | |
| US20210224590A1 (en) | Region extraction model learning apparatus, region extraction model learning method, and program | |
| CN114842019B (en) | Battery plate surface defect detection method, system, storage medium and equipment | |
| JP6612486B1 (en) | Learning device, classification device, learning method, classification method, learning program, and classification program | |
| BR112021015324A2 (en) | SHADOW AND CLOUD MASKING FOR AGRICULTURAL APPLICATIONS USING CONVOLUTIONAL NEURAL NETWORKS | |
| US11908183B2 (en) | Image analysis and processing pipeline with real-time feedback and autocapture capabilities, and visualization and configuration system | |
| Henry et al. | Aerial road segmentation in the presence of topological label noise | |
| WO2020081125A1 (en) | Analyzing complex single molecule emission patterns with deep learning | |
| CN111579097A (en) | High-precision optical scattering compensation method based on neural network | |
| Siavelis et al. | An improved GAN semantic image inpainting | |
| Rajan et al. | Bayesian multiple target localization | |
| Schaaf et al. | Towards measuring bias in image classification | |
| CN115220061A (en) | Deep learning polarization ghost imaging method and system based on orthogonal normalization | |
| CN115220061B (en) | Orthogonal normalization-based deep learning polarization ghost imaging method and system | |
| CN112633501A (en) | Development method and system of machine learning model framework based on containerization technology | |
| Isogawa et al. | Which is the better inpainted image? Training data generation without any manual operations | |
| Lin et al. | Pixel-level self-paced learning for super-resolution | |
| CN113723515B (en) | Moire pattern recognition method, device, equipment and medium based on image recognition | |
| CN113191359B (en) | Small sample target detection method and system based on support and query samples | |
| Ma et al. | Reduced-reference stereoscopic image quality assessment based on entropy of gradient primitives | |
| US20220198258A1 (en) | Saliency Prioritization for Image Processing | |
| Meng et al. | Siamese CNN-based rank learning for quality assessment of inpainted images | |
| Bald et al. | spatialMaxent: Adapting species distribution modeling to spatial data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |