CN116337010A - Associated imaging reconstruction recovery method for intelligent optimization of speckle - Google Patents
Associated imaging reconstruction recovery method for intelligent optimization of speckle Download PDFInfo
- Publication number
- CN116337010A CN116337010A CN202310211373.2A CN202310211373A CN116337010A CN 116337010 A CN116337010 A CN 116337010A CN 202310211373 A CN202310211373 A CN 202310211373A CN 116337010 A CN116337010 A CN 116337010A
- Authority
- CN
- China
- Prior art keywords
- speckle
- patterns
- aver
- imaging
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 40
- 238000000034 method Methods 0.000 title claims abstract description 18
- 238000005457 optimization Methods 0.000 title claims abstract description 14
- 238000011084 recovery Methods 0.000 title claims abstract description 8
- 230000002349 favourable effect Effects 0.000 claims abstract description 7
- 238000005259 measurement Methods 0.000 claims description 15
- 238000013528 artificial neural network Methods 0.000 claims description 13
- 238000012549 training Methods 0.000 claims description 7
- 230000000149 penetrating effect Effects 0.000 claims description 2
- 238000001514 detection method Methods 0.000 abstract description 4
- 238000004364 calculation method Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 12
- 238000011176 pooling Methods 0.000 description 8
- 238000012935 Averaging Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/48—Laser speckle optics
-
- G—PHYSICS
- G02—OPTICS
- G02F—OPTICAL DEVICES OR ARRANGEMENTS FOR THE CONTROL OF LIGHT BY MODIFICATION OF THE OPTICAL PROPERTIES OF THE MEDIA OF THE ELEMENTS INVOLVED THEREIN; NON-LINEAR OPTICS; FREQUENCY-CHANGING OF LIGHT; OPTICAL LOGIC ELEMENTS; OPTICAL ANALOGUE/DIGITAL CONVERTERS
- G02F1/00—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics
- G02F1/01—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics for the control of the intensity, phase, polarisation or colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Optics & Photonics (AREA)
- Nonlinear Science (AREA)
- Multimedia (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of stable detection in a temporary security and protection technical system, and particularly relates to an intelligent speckle optimization associated imaging reconstruction recovery method. The method comprises the following steps: light emitted by the light source is projected onto a Digital Micromirror Device (DMD), a series of speckle patterns are generated by the DMD, projected onto a target, and intensity information projected or reflected by the target using a single pixel detector. The convolution operation is used for extracting information which is favorable for imaging in the speckle, the information and the light intensity collected by the single pixel collector are subjected to association calculation to obtain a target image, and the difference between the speckle pattern light intensity and the light intensity collected by the single pixel collector is used as a parameter for restricting the network, so that the network can better extract the information which is favorable for imaging in the speckle.
Description
Technical Field
The invention belongs to the technical field of stable detection in a temporary security and protection technical system. In particular to an associated imaging reconstruction recovery method for intelligently optimizing speckle.
Background
The optical imaging technology is an important means for human to observe and perceive the objective world, and the imaging system and the optical observation technology are utilized to collect the fundamental characteristic quantity in the process of freely transmitting light beams and interacting with a medium, so that the information of the processable objective scene world is obtained. The associated imaging can acquire images under the condition of interference of cloud, smoke and the like, has new capability exceeding that of the traditional imaging method, and provides a new scheme for solving the problem for high-resolution imaging of dynamic scenes. The correlation imaging is a classical indirect light field correlation imaging technology, acquires image information based on the second-order or higher-order correlation operation of the random light field and the detected light intensity of the light intensity detection sensor, is favorable for keeping the fluctuation trend of the acquired light intensity information, and has unique advantages for realizing high-resolution imaging of large-range dynamic scenes such as scattering medium imaging, long-distance detection and the like. The stronger the algorithm capability, the higher the target image reconstruction efficiency. With the development of deep learning technology, convolutional neural networks are widely applied to the field of image processing, and particularly play a great role in understanding, recovering, enhancing and the like of images, so that the convolutional neural networks are successfully applied to the field of image processing.
The traditional correlation imaging method lacks of condensing useful speckle information, searches second-order correlation operation of all speckle information and light intensity, introduces unnecessary noise into a generated image, and influences the quality of the generated image. The associated imaging method combined with the deep learning mode needs a large amount of training samples to ensure the efficiency of the network, so that the cost of data acquisition is greatly increased.
Disclosure of Invention
In order to solve the problems of poor imaging quality and difficult acquisition of neural network training samples in the existing associated imaging method, the invention discloses an associated imaging reconstruction recovery method for intelligently optimizing speckle. The method fully combines the advantages of the traditional association imaging method and the deep learning, and uses the neural network to extract the speckle information favorable for imaging. And performing associated imaging calculation by using the extracted speckle information and the light intensity. By means of the characteristic that the neural network can efficiently explore information, useful information in speckle is fully mined, and imaging quality of associated imaging is improved. Meanwhile, the method optimizes the network parameters by using the difference between the intensity value of the generated image and the intensity value received by the detector as a reference standard, and does not need an additional training data set, thereby greatly saving the data acquisition cost.
The technical scheme adopted for solving the technical problems is as follows: an intelligent speckle optimization associated imaging reconstruction recovery method is characterized by comprising the following steps:
and step 1, importing speckle information patterns.
Step 1.1, sequentially modulating a light source by using a spatial light modulator to generate a light pattern consistent with speckle information patterns;
step 1.2, collecting and recording the total light intensity measurents of the modulated light pattern penetrating through the object to be imaged by using a barrel detector without spatial resolution capability;
where patterns are three-dimensional data, consisting of (n, w, h). Where n represents the number of speckle patterns and w and h represent the width and height, respectively, of a single speckle pattern. Measurements are one-dimensional data consisting of n measured intensity values.
Step 2, constructing a neural network f for optimizing speckle cnn (·)。
Step 3, using the constructed neural network f cnn (. About.) the speckle information patterns introduced in the first step is optimized, and speckle which is more favorable for imaging is extracted. Speckle optimization function:
patterns＝f cnn (patterns) (1)
and step 4, performing differential correlation imaging by using the speckle patterns optimized in the step three and the intensity information measurements measured in the step one to form an image DGI. The differential correlation imaging formula is:
SI_aver＝mean(patterns*measurements) (2)
B_aver＝Mean(measurements) (3)
pattern＝sum(sum(patterns)) (4)
R_aver＝mean(patterns) (5)
RI_aver＝mean(pattern*patterns) (6)
DGI＝SI_aver-B_aver/R_aver*RI_aver (7)
step 5, calculating an intensity value out_y of the image DGI obtained in the step four, designing a loss function L according to the difference between the intensity value out_y of the generated image and the intensity value measured detected in the step one, completing the parameter training process of the neural network, outputting the DGI used in the last optimization as a final image, and minimizing an objective function L:
out_y＝DGI*patterns (8)
L＝min(mean(out_y-measurements) 2 ) (9)
the beneficial effects of the invention are as follows: the method uses deep learning to extract information which is helpful for imaging in the speckle, omits information which has influence on imaging in the speckle, can greatly improve imaging efficiency and enhance the quality of generated images. Meanwhile, the difference between the light intensity of the generated image and the light intensity received by the detector is used as a loss function, so that the generated image is in wireless approximation with the real image. The use of such a loss function does not require the special adoption of an additional data set as a training set, and reduces the data acquisition cost.
The invention is described in detail below with reference to the drawings and the detailed description.
Drawings
FIG. 1 is a flow chart of an associated imaging reconstruction restoration method for intelligent optimization of speckle in accordance with the present invention.
Detailed Description
Referring to fig. 1, the specific steps of the associated imaging reconstruction recovery method for intelligent speckle optimization in this embodiment are as follows:
and step 1, importing speckle information patterns.
Step 1.1, modulating a light source by using a spatial light modulator to generate a light pattern similar to speckle information patterns;
step 1.2, collecting and recording the total light intensity measurents of the modulated light pattern transmitted through an object to be imaged by using a barrel detector without spatial resolution;
where patterns are three-dimensional data, consisting of (w, h, n). Where n represents the number of speckle patterns and w and h represent the width and height, respectively, of a single speckle pattern. Measurements are one-dimensional data consisting of n measured intensity values.
Step 2, constructing a speckle optimization neural network f cnn (. Cndot.) and is given an initial weight value. Wherein f cnn (. Cndot.) a neural network comprising 6 convolutional layers and 3 pooling layers, wherein the first 3 convolutional layers raise the channels of the image to 16, 32 and 64, respectively, and the last 3 convolutional layers lower the number of channels of the image to 32, 16 and 1. The first 3 convolution layers respectively carry out downsampling operation on the image along with 1 pooling layer, and the last 3 convolution layers carry out upsampling operation on the image while reducing channels. The convolution kernels of all convolution layers are set to 5*5.
Step 3, using the constructed neural network f cnn (.) the speckle information patterns measured in the first step is optimized, and speckle which is more favorable for imaging is extracted. Speckle optimization function:
patterns＝f cnn (patterns)
the specific process is as follows:
step 3.1, taking the speckle information in the step 1 as input, performing one-time convolution operation, wherein the convolution kernel size 5*5 is 1, and the output characteristic layer is 16;
step 3.2, carrying out pooling on the output result of the step 3.1 by using an average pooling operation;
step 3.3, performing one-time convolution operation on the output result in the step 3.2, wherein the convolution kernel size is 5*5, the step length is 1, and the output characteristic layer is 32;
step 3.4, carrying out pooling on the output result in the step 3.3 by using an average pooling operation;
step 3.5, performing one-time convolution operation on the output result in the step 3.4, wherein the convolution kernel size is 5*5, the step length is 1, and the output characteristic layer is 64;
step 3.6, carrying out pooling on the output result in the step 3.5 by using an average pooling operation;
step 3.7, performing a convolution operation on the output result in the step 3.6, wherein the convolution kernel is 5*5, the step length is 1, the output characteristic layer is 32, and up-sampling is performed on the result by using a deconvolution operation;
step 3.8, performing a convolution operation on the output result in the step 3.7, wherein the convolution kernel is 5*5, the step length is 1, the output characteristic layer is 16, and up-sampling is performed on the result by using a deconvolution operation;
and 3.9, performing one convolution operation on the output result in the step 3.8, wherein the convolution kernel is 5*5, the step length is 1, the output characteristic layer is 1, and up-sampling the result by using a deconvolution operation.
And step 4, performing differential correlation imaging by using the speckle patterns optimized in the step three and the intensity information measured in the step one to form an image DGI.
Multiplying all speckle information patterns by corresponding intensity values measurements, and calculating their average value SI_aver
SI_aver＝mean(patterns*measurements)
Where mean (·) represents the averaging function, here acting in the n-dimension, for averaging the multiple speckle patterns, si_aver is two-dimensional data consisting of width w and height h.
An average value B _ averr of the total light intensity collected a plurality of times is calculated.
B_aver＝Mean(measurements)
Where mean (·) represents the averaging function, here acting in the n-dimension, for averaging multiple times, si_aver is two-dimensional data consisting of width w and height h.
Calculate the sum pattern_s of the pixels in each speckle pattern:
patterns_s＝sum(sum(patterns))
the sum (·) represents a summation function, where the sum (·) acts on the w dimension and the h dimension respectively, to calculate a sum of pixels in each speckle pattern, and patterns_s is one-dimensional data, and a sum of pixel points of each of the n speckle patterns is recorded.
Calculating an average value R_averr of the pixel points and patterns_s of each speckle pattern:
R_aver＝mean(patterns_s)
where mean (-) represents the averaging function, here acting in the n-dimension, that is used to average R_aver for the individual images of all speckle patterns.
Calculating all pixel values pattern_s of each speckle pattern and the product of the pixel values pattern_s and the speckle pattern, and calculating the overall average value RI_averr of the speckle patterns:
RI_aver＝mean(patterns_s*patterns)
differential correlation imaging is performed by using the SI_aver, B_ave, R_aver and RI_aver obtained above, and an image DGI is obtained:
DGI＝SI_aver-B_aver/R_aver*RI_aver
step 5, calculating an intensity value out_y of the image DGI obtained in the step four, designing a loss function L according to the difference between the intensity value out_y of the generated image and the intensity value measured detected in the step one, completing the parameter training process of the neural network, outputting the DGI used in the last optimization as a final image, and minimizing an objective function L:
out_y＝DGI*patterns
L＝min(mean(out_y-measurements) 2 )
the effects of the present invention are further described by the following simulation experiments.
Simulation conditions: the invention is a simulation performed by MATLAB software on an operating system with a central processing unit of Intel (R) Core (TM) i7-6800K CPU@3.40GHz and a memory 4-G, ubuntu.
Claims (1)
1. The associated imaging reconstruction recovery method for intelligent optimization of speckle is characterized by comprising the following steps of:
step 1, importing speckle information patterns;
step 1.1, sequentially modulating a light source by using a spatial light modulator to generate a light pattern consistent with speckle information patterns;
step 1.2, collecting and recording the total light intensity measurents of the modulated light pattern penetrating through the object to be imaged by using a barrel detector without spatial resolution capability;
wherein patterns are three-dimensional data, and are composed of (n, w, h), wherein n represents the number of speckle patterns, and w and h represent the width and height of a single speckle respectively; measurements are one-dimensional data consisting of n measured intensity values;
step 2, constructing a neural network f for optimizing speckle cnn (·)；
Step 3, using the constructed neural network f cnn (. About.) optimizing the speckle information patterns imported in the first step, and extracting speckle more favorable for imaging, wherein the speckle optimization function is as follows:
patterns＝f cnn (patterns) (1)
step 4, performing differential correlation imaging by using the speckle patterns optimized in the step three and the intensity information measurements measured in the step one to form an image DGI; the differential correlation imaging formula is:
SI_aver＝mean(patterns*measurements) (2)
B_aver＝Mean(measurements) (3)
pattern＝sum(sum(patterns)) (4)
R_aver＝mean(patterns) (5)
RI_aver＝mean(pattern*patterns) (6)
DGI＝SI_aver-B_aver/R_aver*RI_aver (7)
step 5, calculating an intensity value out_y of the image DGI obtained in the step four, designing a loss function L according to the difference between the intensity value out_y of the generated image and the intensity value measured detected in the step one, completing the parameter training process of the neural network, outputting the DGI used in the last optimization as a final image, and minimizing an objective function L:
out_y＝DGI*patterns (8)
L＝min(mean(out_y-measurements) 2 ) (9)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310211373.2A CN116337010A (en) | 2023-03-07 | 2023-03-07 | Associated imaging reconstruction recovery method for intelligent optimization of speckle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310211373.2A CN116337010A (en) | 2023-03-07 | 2023-03-07 | Associated imaging reconstruction recovery method for intelligent optimization of speckle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116337010A true CN116337010A (en) | 2023-06-27 |
Family
ID=86886973
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310211373.2A Pending CN116337010A (en) | 2023-03-07 | 2023-03-07 | Associated imaging reconstruction recovery method for intelligent optimization of speckle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116337010A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117201691A (en) * | 2023-11-02 | 2023-12-08 | 湘江实验室 | Panoramic scanning associated imaging method based on deep learning |
-
2023
- 2023-03-07 CN CN202310211373.2A patent/CN116337010A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117201691A (en) * | 2023-11-02 | 2023-12-08 | 湘江实验室 | Panoramic scanning associated imaging method based on deep learning |
| CN117201691B (en) * | 2023-11-02 | 2024-01-09 | 湘江实验室 | Panoramic scanning associated imaging method based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102812709B (en) | Compressed color image sampling and the method and system rebuild | |
| US9900562B2 (en) | System and method for light-field imaging | |
| CN112733950A (en) | Power equipment fault diagnosis method based on combination of image fusion and target detection | |
| CN110675326B (en) | Method for calculating ghost imaging reconstruction recovery based on U-Net network | |
| CN111047681B (en) | Single-pixel three-dimensional end-to-end reconstruction method and device based on deep learning | |
| CN110097634B (en) | Self-adaptive multi-scale three-dimensional ghost imaging method | |
| Peng et al. | Residual pixel attention network for spectral reconstruction from RGB images | |
| CN109447891A (en) | A kind of high quality imaging method of the spectrum imaging system based on convolutional neural networks | |
| CN111563562A (en) | Color target reconstruction method of single-frame scattering image based on convolutional neural network | |
| CN116337010A (en) | Associated imaging reconstruction recovery method for intelligent optimization of speckle | |
| CN115484410B (en) | Event camera video reconstruction method based on deep learning | |
| CN112818920B (en) | Double-temporal hyperspectral image space spectrum joint change detection method | |
| CN111833261A (en) | Image super-resolution restoration method for generating countermeasure network based on attention | |
| CN114841856A (en) | Image super-pixel reconstruction method of dense connection network based on depth residual channel space attention | |
| CN112802145A (en) | Color calculation ghost imaging method based on deep learning | |
| CN116245726A (en) | Compressed sensing polarization super-resolution imaging method based on deep learning framework | |
| Cao et al. | Learning adaptive warping for real-world rolling shutter correction | |
| CN109447890A (en) | A kind of Encoding Optimization of the spectrum imaging system based on convolutional neural networks | |
| CN110766609B (en) | Depth-of-field map super-resolution reconstruction method for ToF camera | |
| CN116128743A (en) | Depth convolution hybrid neural network-based calculation associated imaging reconstruction algorithm | |
| CN110044474B (en) | Correlation imaging optimization method based on interpolation algorithm | |
| CN112950507B (en) | Method for improving single-pixel color imaging performance under scattering environment based on deep learning | |
| CN117474764B (en) | High-resolution reconstruction method for remote sensing image under complex degradation model | |
| Zhao et al. | Redundant compressed single-pixel hyperspectral imaging system | |
| Kong et al. | Block Image Compressive Sensing with Local and Global Information Interaction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |