CN114659634A - Miniature snapshot type compressed spectrum imaging detection device and detection method - Google Patents
Miniature snapshot type compressed spectrum imaging detection device and detection method Download PDFInfo
- Publication number
- CN114659634A CN114659634A CN202210335549.0A CN202210335549A CN114659634A CN 114659634 A CN114659634 A CN 114659634A CN 202210335549 A CN202210335549 A CN 202210335549A CN 114659634 A CN114659634 A CN 114659634A
- Authority
- CN
- China
- Prior art keywords
- spectrum
- array
- detector
- image
- beam splitting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001228 spectrum Methods 0.000 title claims abstract description 125
- 238000001514 detection method Methods 0.000 title claims abstract description 44
- 238000003384 imaging method Methods 0.000 title claims abstract description 43
- 238000013528 artificial neural network Methods 0.000 claims abstract description 40
- 238000012545 processing Methods 0.000 claims abstract description 34
- 238000000701 chemical imaging Methods 0.000 claims description 50
- 230000003595 spectral effect Effects 0.000 claims description 37
- 230000003287 optical effect Effects 0.000 claims description 24
- 238000009826 distribution Methods 0.000 claims description 14
- 230000000903 blocking effect Effects 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 6
- 230000010355 oscillation Effects 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 5
- 238000002310 reflectometry Methods 0.000 claims description 4
- 238000012549 training Methods 0.000 claims description 4
- OAICVXFJPJFONN-UHFFFAOYSA-N Phosphorus Chemical compound [P] OAICVXFJPJFONN-UHFFFAOYSA-N 0.000 claims description 3
- 239000000203 mixture Substances 0.000 claims description 2
- 239000004973 liquid crystal related substance Substances 0.000 claims 1
- 238000005070 sampling Methods 0.000 abstract description 15
- 230000005540 biological transmission Effects 0.000 abstract description 8
- 230000006872 improvement Effects 0.000 abstract description 5
- 230000001360 synchronised effect Effects 0.000 abstract description 4
- 238000003491 array Methods 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 description 22
- 238000000034 method Methods 0.000 description 8
- 230000008859 change Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 4
- 230000004907 flux Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000010354 integration Effects 0.000 description 3
- 230000010287 polarization Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000002591 computed tomography Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 239000006185 dispersion Substances 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000009615 fourier-transform spectroscopy Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 235000001968 nicotinic acid Nutrition 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 229910052704 radon Inorganic materials 0.000 description 1
- SYUHGPGVQRZVTB-UHFFFAOYSA-N radon atom Chemical compound [Rn] SYUHGPGVQRZVTB-UHFFFAOYSA-N 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000000411 transmission spectrum Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2823—Imaging spectrometer
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/10—Beam splitting or combining systems
Abstract
A miniature snapshot type compressed spectrum imaging detection device and a detection method are disclosed, wherein the device comprises a spectrum mixed coding array, a lens array, a detector and a data acquisition processing display system; each subunit of the spectrum mixed coding array has different pseudorandom spectrum transmission characteristics to form an unrelated random modulation sampling unit, so that high-compression-ratio spectrum sampling is realized by using the minimum unit number, the resolution and quality of a reconstructed spectrum and the convergence speed of a reconstruction algorithm are ensured, and a compact and small spectrum imaging system is constructed by using fewer arrays. The detector is connected with the data acquisition, processing and display system; the data acquisition, processing and display system is used for controlling the detector to snapshot a frame of image array carrying spectrum mixed codes in one exposure period, and the spectrum image can be reconstructed by utilizing a compressed sensing reconstruction algorithm or a deep neural network. The invention has the advantages of simple and ultra-compact structure, economy, high timeliness, high resolution, synchronous performance improvement and the like.
Description
Technical Field
The invention belongs to the technical field of optical remote sensing detection, and particularly relates to a miniature snapshot type compressed spectrum imaging detection device and a detection method.
Background
The electromagnetic wave radiated by the object contains spectral information which changes along with the space position, and can be used for inverting the shape, the physical chemistry and other characteristics of the target. The spectral imaging technology is a leading-edge remote sensing technology for simultaneously acquiring two-dimensional space target spectral information, has certain potential for improving the efficiency and accuracy of target detection, identification and classification, and has important application value and prospect in various fields such as military reconnaissance, earth resource general investigation, environmental sanitation monitoring, natural disaster prediction, atmospheric detection, astronomical observation, machine vision bionics, biomedical diagnosis and the like.
The spectral imaging technology is classified according to the time resolution of acquiring two-dimensional spectral information, and can be divided into a time sequence type and a snapshot type. Currently, most spectral imaging technologies employ a time-series scanning mode (such as a frame mode, a push-broom mode, or a window-broom mode) to acquire a spectral image of a two-dimensional scene, and a polarization spectral image of a two-dimensional space target needs to be extracted and recombined from multi-frame image data acquired at different times. The time sequence acquisition technology is not suitable for a dynamic or fast changing target, the instability of the atmosphere or the surrounding environment can also influence the imaging quality, the imaging quality is difficult to carry on a platform with larger jitter or larger maneuverability, and the phenomena of multi-dimensional information mismatch and confusion are easy to occur, so that the later application problems such as inconsistent maps are caused.
Compared with the prior art, the snapshot type spectral imaging technology can acquire the spectral image of the two-dimensional space target within a single exposure time, has the advantages of rapid real-time detection, can improve the working efficiency, and can effectively avoid the influence caused by environmental change during sequence measurement, so the snapshot type spectral imaging technology is the main direction of current and future development and has important application potential.
The snapshot type spectral imaging technology can be divided into: direct spectral imaging and computational spectral imaging. Direct spectral imaging mainly refers to the fact that data acquired by an optical system is what is seen or what is obtained, a spectral image can be directly provided, or only a simple data reconstruction process is required. The technology mainly comprises the following steps:
the integrated field spectrum imaging technology introduced in U.S. Pat. No. 8174694B 2, chinese patent CN 103592030B, etc.;
filter array aperture division imaging techniques described in r.shogenji, y.kitamura, k.yamada, s.miyatake, j.tanida, "Multispectral imaging using compounds," opt.express 12(8),1643(2004), and b.gel, n.tack, a.lambreads, "a Snapshot Multispectral Imager with Integrated, finished Filters and Optical duplexing," SPIE vol.8613,861314(2013), etc.;
the filter array focal plane imaging techniques introduced in US 8081244B 2, US 8109634B 2, etc.
The integral field spectrum imaging technology usually needs dispersive optics, the system volume is large, and compact miniaturization is difficult to realize. In comparison, the snapshot type spectral imaging technology based on the optical filter array is simple in structure and can realize compact miniaturization. However, the number of spectral channels of the filter array aperture-splitting spectral imaging technology is limited by the processing and integration technology of small-aperture narrow-band filters; to obtain high spectral resolution, hundreds of filters are integrated, which is very difficult to realize (refer to P.Lapray, X.Wang, J.Thomas, and P.Gouton, "Multi spectral Filter Arrays: Recent Advances and Practical Implementation," Sensors 14,21626-21659 (2014)). The filter array focal plane imaging technology generally adopts a Bayesian arrangement method to periodically arrange several narrow-band filters with different wave bands on a focal plane, also needs precise processing and integration technology, and can only acquire spectral images of several wave bands simultaneously. The linear gradient filter can continuously modulate the spectrum, and has the advantages of mature technology, complete process, high quality and low price in manufacturing. However, it is often used in slit-based push-broom hyperspectral imaging systems to acquire hyperspectral information of a two-dimensional object by relative movement of the system and the scene. Recently, a linear gradient filter is combined with a lens array which is specially arranged to realize high spectral imaging capability, for example, patent CN107271039A, but spectral channels and spatial resolutions are mutually restricted; the spatial resolution is reduced by increasing the number of spectral channels and vice versa.
In summary, the direct spectral imaging type always has the bottleneck problem of mutual restriction among multiple parameters such as imaging spatial range, spatial resolution, spectral range and spectral resolution, all performance parameters are difficult to be synchronously improved, parameter indexes need to be designed according to application scenes, and the application flexibility is limited. In addition, the direct spectral imaging type often obeys the nyquist sampling theorem during data acquisition, resulting in large data acquisition amount, limited transmission rate and the like.
The computational spectral imaging mainly means that data acquired by an optical system is not what is obtained when the data is seen, and a final spectral image can be obtained through later-stage complex and heavy reconstruction algorithm processing. The technology mainly relates to computed tomography spectral imaging (such as the patent US 6522403B 2), compressed sensing spectral imaging (such as the patent US8553222B2), interference spectral imaging (such as the M.W. Kudenov and E.L. Dereniak, "Compact-time bifurcating imaging spectrometer" Opt.express 20,17973 (2012)), and the like. The interference spectrum imaging is mainly based on a Fourier transform spectroscopy reconstruction algorithm, and has the advantages of multiple channels, high flux and high signal-to-noise ratio; however, since it is necessary to generate an interferogram that is symmetrical about zero optical path difference and sampling of optical path difference needs to satisfy the nyquist sampling theorem, the number of samples is large, and the resolution of the restored spectrum depends on the number of optical path difference samples and the maximum optical path difference, which results in a complicated interferometer apparatus being required. The polarization interferometer has a relatively compact structural characteristic and is increasingly used in an interference imaging spectrum system; however, the birefringent element in the current polarization interferometer requires a combination of at least two or more pieces, resulting in a complicated structure, difficult processing and uneconomical performance. Computed tomography spectral imaging is mainly based on a Radon transform algorithm, and due to limited angle projection sampling, a transform domain has a cone loss problem, and spatial resolution improvement is limited.
Compressed spectrum imaging based on a compressed sensing and sparse sampling theory framework can utilize a simple optical imaging system, increases the number of spectrum channels on the premise of keeping large spatial resolution, mainly depends on spectrum coding hardware and decoding algorithm software, and becomes a development key point and a hotspot in the field of hyperspectral imaging. The coded aperture spectral imaging is a typical representation of compressed spectral imaging, the coded aperture multi-complex technology is utilized, the spatial dimension and the spectral dimension are simultaneously mixed and coded, the Nyquist rule (namely underdetermined sampling) is not required to be met during information acquisition, the data acquisition amount is small, and the hyperspectral image can be reconstructed subsequently according to an imaging physical model and a compressed sensing principle. However, the technology needs precise imaging, collimation and re-imaging multiple relay optical configuration, has a complex structure and a large volume, and is difficult to compact and miniaturize.
Therefore, the existing single way is difficult to realize the spectral imaging technology which has the advantages of simple and compact structure, rapidness, convenience, economy, high resolution and the like, and a novel snapshot-type spectral imaging device and a detection method thereof are urgently needed.
Disclosure of Invention
In order to overcome the above-mentioned drawbacks of the prior art, an object of the present invention is to provide a miniature snapshot-type compressed spectral imaging detection apparatus and a detection method, so as to solve one or more of the above-mentioned technical problems. The invention has the advantages of simple and ultra-compact structure, rapidness, convenience, economy, high resolution, synchronous performance improvement and the like.
In order to achieve the purpose, the invention adopts the technical scheme that:
the miniature snapshot type compressed spectrum imaging detection device comprises a spectrum mixed coding array 12, a lens array 13, a detector 14 and a data acquisition processing display system 15 connected with the detector 14;
the spectrum mixing coding array 12 is used for modulating light beams emitted by a target, and along the light beam direction, the subunits of the spectrum mixing coding array 12 are provided with two surfaces with intensity reflectivity of 20% -80%, the distance between the two surfaces is larger than the wavelength magnitude, and the distances between the two surfaces in each subunit are different;
the lens array 13 has sub-lenses corresponding to the sub-units of the spectrum mixed coding array 12 one by one, and each sub-lens focuses the light beam output by the corresponding sub-unit on the photosensitive surface of the detector 14 for imaging;
the data acquisition processing display system 15 is used for acquiring the coded image array acquired by the detector 14.
In one embodiment, each subunit of the spectrum mixing and encoding array 12 is a planar beam splitting plate one 121, the two surfaces are a front surface one 1211 and a back surface two 1212 of the planar beam splitting plate one 121, and the distance between the two surfaces is the thickness of the planar beam splitting plate one 121;
alternatively, the first and second electrodes may be,
the subunits of the spectrally mixed coding array 12 are a second planar beam splitting plate 122 and a third planar beam splitting plate 123 which are sequentially arranged along the incident light direction, a first outer surface 1221 of the second planar beam splitting plate 122 and a second outer surface 1231 of the third planar beam splitting plate 123 are parallel to each other, a first inner surface 1222 of the second planar beam splitting plate 122 and a second inner surface 1232 of the second planar beam splitting plate 122 are parallel to each other, the two surfaces are a first inner surface 1222 and a second inner surface 1232, and the distance between the first inner surface 1222 and the second inner surface 1232 is the distance between the first inner surface 1222 and the second inner surface 1232.
In one embodiment, the first outer surface 1221 and the first inner surface 1222 of the second plane beam splitter plate 122 are not parallel, forming a wedge angle to prevent multiple reflection oscillation of light rays inside;
and/or the presence of a gas in the gas,
the second outer surface 1231 and the second inner surface 1232 of the third planar beam splitter plate 123 are not parallel to each other, forming a wedge angle to prevent multiple reflection oscillation of light rays inside.
In one embodiment, the miniature snapshot type compressed spectrum imaging detection device further comprises: a light-blocking aperture array 16;
the lens array 13 is a lens array;
the detector 14 is a single area array detector;
the light blocking hole array 16 is disposed between the lens array 13 and the detector 14, and is used for limiting a field range and preventing adjacent sub-images from overlapping.
In one embodiment, the miniature snapshot type compressed spectrum imaging detection device further comprises: a collimating optical system 11;
the collimating optical system 11 includes: an objective lens 111, a field stop 112 and a collimator mirror 113 arranged in this order along the incident light direction; wherein the field diaphragm 112 is disposed on the image plane of the objective 111, and the image plane of the objective 111 coincides with the front focal plane of the collimator 113;
the collimating optical system 11 is located in front of the spectrum mixing coding array 12 and the lens array 13, and is used for limiting the field range and preventing adjacent sub-images from overlapping.
In one embodiment, the lens array 13 is an objective lens array; the detector 14 is an area array detector array.
The invention also provides a detection method for the miniature snapshot type compressed spectral imaging detection device by utilizing energy, which comprises the following steps:
after being modulated by the spectrum mixed coding array 12, light beams emitted by a target are focused on a photosensitive surface of the detector 14 through each sub-lens in the lens array 13 to be imaged, the data acquisition processing display system 15 controls the detector 14 to snapshot a frame of coding image array carrying spatial spectrum information, the intensity g distribution of each spatial position in a coding subimage is extracted, a spectrum modulation matrix H of each spatial position in the subimage is calibrated by using a standard light source with known spectrum distribution, and the incident spectrum distribution f, the intensity g and the spectrum modulation matrix H of a two-dimensional space are set up to be in the following linear relation:
g＝Hf，
directly estimating the incident spectral distribution f by using a compressed sensing algorithm:
or
Wherein gamma is a regularization parameter, | · |. non-woven phosphor1Is represented by1The norm, Φ, is the regularization function.
In one embodiment, the data acquisition, processing and display system 15 includes: a physical layer neural network and a reconstruction neural network;
the physical layer neural network takes the interval between the two surfaces as a variable and is used for simulating the function of acquiring the sub-image array by the miniature snapshot type compressed spectral imaging detection device;
the reconstruction neural network takes the network weight and the offset parameter as variables and is used for reconstructing a hyperspectral image cube;
the data acquisition, processing and display system 15 trains an optimized physical layer neural network and a reconstructed neural network simultaneously;
in the training optimization stage, the number of the subunits of the minimized spectrum mixed coding array 12 is respectively used as a target and a reconstructed high-quality spectrum image cube is used as a target;
when a spectrum image cube meeting the preset requirements is trained, the optimal number and thickness of the subunits of the spectrum mixed coding array 12 are determined, and meanwhile, the optimal network weight and the bias parameters of the reconstructed neural network are obtained.
In one embodiment, during the inference phase, the data acquisition processing display system 15 utilizes the reconstructed neural network 152 and its optimized network weights and bias parameters to infer a spectral image cube from the sub-image array.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a miniature snapshot type compressed spectrum imaging detection device which has the characteristics of simple and ultra-compact structure, economy, high timeliness, high resolution, synchronous improvement of performance and the like. Specifically, the method integrates the multi-complex modulation characteristics of spectrum mixed coding, compressed sensing sparse sampling and reconstruction characteristics, and strong learning capacity of a deep neural network, simplifies the system structure, compresses the system size, retains space dimension information, only performs multi-complex compressed coding sampling in the spectrum dimension, and only needs to enhance spectrum dimension learning in the subsequent reconstruction model, so that the requirement on model complexity is reduced, and the reconstruction quality and timeliness are improved; the sampling number is minimized in the spectral dimension, and high compression ratio is realized, so that the imaging space range and the spatial resolution performance are maximized; and a sampling channel is optimized in a spatial dimension, the spectrum multi-complex coding capability is improved, and the spectrum range and the spectrum resolution performance are maximized.
In the invention, the spectrum imaging device adopts the combination of the spectrum mixed coding array and the lens array, and has the advantages of simplicity, ultra-compactness and miniaturization in structure.
Compared with a snapshot type spectrum imaging device and method based on a narrow-band filtering array or a dispersion element, the spectrum mixed coding array has the advantages of multiple channels, multiple elements, high flux and high signal-to-noise ratio.
In the invention, the deep neural network is utilized to train the physical neural network of the simulated spectral imaging system and the reconstruction neural network of the restored spectral image simultaneously, so that an adaptive imaging hardware system and a reconstruction software system can be obtained, and the deep neural network of 'black box operation' is endowed with physical significance.
In the invention, in the spectral image reconstruction stage, the compressed sensing algorithm and the deep neural network algorithm can be supervised mutually.
In the detection method, the sub-image array carrying the spectrum mixed code can be obtained within one exposure period of the detector.
In the invention, the optimal spectrum mixed coding array is obtained by using the simulated spectrum imaging system to aim at minimizing the number of subunits and maximizing the quality of the reconstructed spectrum image, and the spatial resolution is improved.
In the invention, relative to a time sequence type spectral imaging system, a snapshot system can acquire two-dimensional spectral image information of a space target by single exposure, is suitable for detecting dynamic or rapidly-changed targets, can effectively avoid negative effects caused by target change, shaking noise or environmental change and other factors, and has potential application value in the fields of astronomical observation, space detection, earth remote sensing, machine vision, biomedical diagnosis and the like.
Drawings
Fig. 1 is a schematic structural diagram of a miniature snapshot type compressed spectral imaging detection device of the present invention.
FIG. 2 is a schematic diagram of a single plane beam splitting plate in the spectral mixture coding array subunit of the miniature snapshot type compressed spectral imaging detection apparatus in FIG. 1.
Fig. 3 is a schematic diagram of two plane beam splitting plates in a spectrum mixing coding array subunit in the miniature snapshot type compressed spectrum imaging detection device in fig. 1.
Fig. 4 is a side view of the light blocking array inserted in the miniature snapshot type compressed spectrum imaging detection device in fig. 1.
Fig. 5 is a schematic structural diagram of a collimating optical system added in front of a lens array in the miniature snapshot type compressed spectral imaging detection apparatus in fig. 1.
In the figure, 11 is a collimating optical system, 111 is an objective lens, 112 is a field stop, 113 is a collimating mirror, 12 is a spectrum mixing coding array, 121 is a plane beam splitting plate, 1211 is the front surface of 121, 1211 is the back surface of 121, 122 is the plane beam splitting plate, 1221 is the outer surface of 122, 1222 is the inner surface of 122, 123 is the plane beam splitting plate, 1231 is the outer surface of 123, 1232 is the inner surface of 123, 13 is a lens array, 14 is a detector, and 15 is a data acquisition processing display system.
Detailed Description
The embodiments of the present invention will be described in detail below with reference to the drawings and examples.
As described above, in the existing snapshot-type spectral imaging technology, the structure required for direct spectral imaging is complex, the volume is large, the requirement for processing precision is extremely high, the data acquisition amount is large, the transmission efficiency is limited, and multiple parameters are restricted with each other, which limits the application flexibility. The computational spectral imaging also requires a very large number of samples, and the high requirements for the interferometer also result in a complex structure and difficult processing.
Compressed sensing spectral imaging based on a spectrum mixed coding array is expected to improve the number of spectral channels and the spatial resolution by a simple and compact optical structure; the method mainly utilizes the broadband property and diversity of the transmission spectrum, carries out different spectrum mixed coding on each unit and improves luminous flux, synthesizes the mixed coding data of all the units, and utilizes a de-duplication algorithm to reconstruct a spectrum channel image with the number far more than that of array units; the key point is the diversity of the spectrum mixed coding array, and the minimum of the correlation degree of the spectrum mixed coding of each unit is ensured so as to ensure the resolution and quality of the reconstructed spectrum; at present, the direction is still in an exploration stage, the manufacturing and integration processes of the spectrum mixed coding optical filter array are not mature, and the challenge of obtaining the optical filter array with wide spectrum and irrelevant spectrum transmission is still achieved.
The reconstruction algorithm of the compressed spectrum comprises constrained convex optimization, non-convex optimization, iterative threshold shrinkage and the like, and the complexity of the reconstruction algorithm depends on the characteristics of the spectrum mixed coding array. In recent years, machine learning based on deep neural networks, such as deep learning, has remarkable efficacy in the fields of computational imaging, remote sensing and the like. The deep neural network trained by the prior data learning can finally become a direct reconstruction type feedforward neural network structure, not only can improve the reconstruction quality, but also can reduce the reconstruction time. The deep learning based on the convolutional neural network is applied to various spectral imaging schemes of compression sampling, has the potential of solving the problem of multi-parameter restriction such as imaging time resolution, spatial range, spatial resolution, spectral range and spectral resolution, is expected to synchronously improve various parameter indexes, and is very worthy of deep exploration.
Accordingly, the invention provides a miniature snapshot type compressed spectrum imaging detection device, which is shown in fig. 1 and comprises a spectrum mixed coding array 12, a lens array 13, a detector 14 and a data acquisition processing display system 15 which are sequentially arranged along incident light.
The spectrum mixing coding array 12 is disposed in front of the lens array 13, or may be disposed between the lens array 13 and the detector 14.
The spectrum mixing coding array 12 is used for modulating light beams emitted by a target, and the spectrum mixing coding array 12 is composed of a plurality of subunits arranged in an array form. The array form can be rectangular, annular, and the like. Along the beam direction, each subunit should have two surfaces with intensity reflectivity between 20% and 80%, and the distance between the two surfaces is larger than the wavelength order, and the distance between the two surfaces in each subunit is different.
The lens array 13 mainly includes a plurality of sub-lenses, and the sub-lenses are in a one-to-one correspondence with the sub-units of the spectrum hybrid coding array 12, and each sub-lens focuses the light beam output by the corresponding sub-unit on the photosensitive surface of the detector 14 for imaging. It will be readily appreciated that the photosensitive surface of the detector 14 is preferably located at the image plane of the lens array 13.
The data acquisition processing display system 15 is connected to the detector 14, and is mainly used for acquiring the coded image array acquired by the detector 14, and also for controlling the snapshot operation of the detector 14 to obtain a frame of coded image array carrying spatial spectrum information, and further performing calculation processing to obtain the incident spectrum distribution.
Based on the multi-beam interference principle, the distance between the two surfaces is larger than the wavelength order to generate a plurality of spectrum transmission peaks, the intensity reflectivity is between 20% and 80% to enlarge the width of each spectrum transmission peak, and the distance between the two surfaces in each subunit is different to generate different spectrum transmission units for spectrum coding.
In one embodiment of the present invention, the subunits of the spectrally mixed encoded array 12, which may be a single entity, is a planar beam splitting plate 121, for example, as described with reference to FIG. 2. At this time, the aforementioned "two surfaces" are the first front surface 1211 and the second rear surface 1212 of the first planar beam splitting plate 121, and the "distance between the two surfaces" is the thickness of the first planar beam splitting plate 121, wherein "front" and "rear" are defined according to the incident light. The structure adopts a single entity, and has the characteristics of compact structure and high robustness.
Another form of subunits of the spectrally mixed encoded array 12 may consist of two entities with a separation along the incident light direction, e.g., referring to FIG. 3, which is a planar beam splitting plate two 122 and a planar beam splitting plate three 123. Along the incident light direction, the second planar beam splitting plate 122 is located in front of the third planar beam splitting plate 123, the front surface of the second planar beam splitting plate 122 is defined as a first outer surface 1221, and the rear surface is defined as a first inner surface 1222; the front surface of the third planar beam splitter plate 123 is defined as the second inner surface 1232, and the rear surface is defined as the second outer surface 1231. That is, inner surface one 1222 is opposite inner surface two 1232. Wherein the two outer surfaces are parallel to each other and the two inner surfaces are also parallel to each other. In this case, the "two surfaces" are the first inner surface 1222 and the second inner surface 1232, and the "two surfaces distance" is the distance between the first inner surface 1222 and the second inner surface 1232. The structure adopts two entities, and has the characteristics of flexible tuning and reconfigurability.
In this version, outer surface one 1221 and inner surface one 1222 may be non-parallel to form a slight wedge angle to prevent multiple reflection oscillations of light within it. Likewise, the outer surface two 1231 and the inner surface two 1232 may not be parallel to form a slight wedge angle to prevent multiple reflection oscillations of light inside. It will be readily appreciated that these two non-parallel structures may be present or may be chosen alternatively.
Further, referring to fig. 4, in an embodiment of the present invention, the light shielding device further includes a light shielding hole array 16; at this time, the lens array 13 is a lens array; the detector 14 is a single area array detector, and the number of light blocking holes of the light blocking hole array 16 is the same as the number of sub-lenses of the lens array 13 and corresponds to one another. An array of light blocking apertures 16 is interposed between the lens array 13 and the detector 14 to limit the field of view and prevent adjacent sub-images from overlapping.
Referring to fig. 5, in the embodiment of the present invention, a collimating optical system 11 is further included; the collimating optical system 11 includes: an objective lens 111, a field stop 112 and a collimator mirror 113 arranged in this order along the incident light direction; the field diaphragm 112 is arranged on the image surface of the objective 111, and the image surface of the objective 111 is superposed with the front focal surface of the collimator 113; the collimating optical system 11 is located in front of the spectrum mixing coding array 12 and the lens array 13, and is used for limiting the field range and preventing adjacent sub-images from overlapping.
Referring to fig. 1, in an embodiment of the present invention, the lens array 13 is an objective lens array, the detector 14 is an area array detector array, and each unit corresponds to one unit. Each objective lens unit projects a target onto a corresponding area array detector unit, the target image units modulated by the corresponding spectrum coding units are independently collected, all the units can be controlled to collect in parallel and simultaneously, and the area array detectors of all the units have the capability of collecting images at high resolution and high speed, so that the whole system also has the characteristic of real-time, quick and high-resolution image collection.
Based on this, the detection method of the miniature snapshot type compressed spectral imaging detection device comprises the following steps:
after being modulated by the spectrum mixing coding array 12, light beams from a two-dimensional space target are focused on a photosensitive surface of the detector 14 through each sub-lens in the lens array 13 to form an image, and the data acquisition, processing and display system 15 controls the detector 14 to snapshot a frame of coding image array carrying spatial spectrum information.
In the embodiment of the present invention, the data acquisition, processing and display system 15 extracts the intensity g distribution of each spatial position in the subimage, and calibrates the spectrum modulation matrix H of each spatial position in the subimage by using a standard light source with known spectrum distribution, and establishes the following linear relationship between the incident spectrum distribution f of the two-dimensional space and the intensity g and the spectrum modulation matrix H:
g＝Hf，
in the embodiment of the invention, the incident spectrum distribution f can be directly estimated by using a compressed sensing algorithm:
or
Wherein gamma is a regularization parameter, | · |. non-woven phosphor1Is represented by1Norm, Φ is the regularization function;
or, in the embodiment of the present invention, the incident spectral distribution f is indirectly estimated by using the sparse basis W and the sparse representation Θ of the signal:
or
The solution calculator may be a GPSR algorithm or a TwIST algorithm or others.
In this embodiment of the present invention, the data acquisition, processing and display system 15 may include: a physical layer neural network and a reconstruction neural network;
in the embodiment of the invention, the physical layer neural network takes the interval of two plane beam splitting plates in the subunits of the spectrum mixed coding array 12 as a variable and is used for simulating the function of a miniature snapshot type compressed spectrum imaging detection device for acquiring a sub-image array; the reconstruction neural network takes the network weight and the offset parameter as variables and is used for reconstructing the hyperspectral image cube;
in the embodiment of the invention, in the training optimization stage, the data acquisition, processing and display system 15 simultaneously trains an optimized physical layer neural network and a reconstructed neural network, and determines the optimal number and thickness of the subunits of the spectrum hybrid coding array 12 when a spectrum image cube meeting the preset requirement is trained by respectively taking the number of the subunits of the minimized spectrum hybrid coding array 12 as a target and the reconstructed high-quality spectrum image cube as a target, and simultaneously obtains the optimized network weight and the offset parameter of the reconstructed neural network.
In the embodiment of the invention, in the inference stage, the data acquisition, processing and display system 15 utilizes the reconstructed neural network and the optimized network weight and the bias parameters thereof to infer and obtain the spectral image cube from the subimage array.
In the embodiment of the invention, the spectral imaging system adopts the combination of the spectral mixed coding array and the lens array, and has the advantages of simplicity, ultra-compactness and miniaturization in structure. Compared with a snapshot type spectral imaging device and method based on a narrow-band filtering array or a dispersion element, the spectral hybrid coding array has the advantages of multiple channels, multiple elements, high flux and high signal-to-noise ratio. The deep neural network is used for simultaneously training the physical neural network of the simulated spectral imaging system and the reconstructed neural network of the restored spectral image, so that an adaptive imaging hardware system and a reconstructed software system can be obtained, and the deep neural network of black box operation is endowed with physical significance. In the spectral image reconstruction stage, the compressed sensing algorithm and the deep neural network department can supervise each other. Compared with a time sequence type spectral imaging system, the snapshot type system can acquire two-dimensional spectral image information of a space target through single exposure, is suitable for detecting dynamic or rapidly-changed targets, and can effectively avoid negative effects caused by factors such as target change, shaking noise or environmental change.
In summary, the invention discloses a miniature snapshot type compressed spectrum imaging device and a detection method, which comprises a spectrum mixed coding array, a lens array, a detector and a data acquisition processing display system, wherein the spectrum mixed coding array, the lens array, the detector and the data acquisition processing display system are sequentially arranged along incident light; each subunit of the spectrum mixed coding array has different pseudorandom spectrum transmission characteristics to form an unrelated random modulation sampling unit, so that high-compression-ratio spectrum sampling is realized by using the minimum unit number, the resolution and quality of a reconstructed spectrum and the convergence speed of a reconstruction algorithm are ensured, and a compact and small spectrum imaging system is constructed by using fewer arrays. The detector is connected with the data acquisition, processing and display system; the data acquisition, processing and display system is used for controlling the detector to snapshot a frame of image array carrying spectrum mixed codes in one exposure period, and the spectrum image can be reconstructed by utilizing a compressed sensing reconstruction algorithm or a deep neural network. The invention has the advantages of simple and ultra-compact structure, economy, high timeliness, high resolution, synchronous performance improvement and the like.
Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art can make modifications and equivalents to the embodiments of the present invention without departing from the spirit and scope of the present invention, which is set forth in the claims of the present application.
Claims (9)
1. The miniature snapshot type compressed spectrum imaging detection device is characterized by comprising a spectrum mixed coding array (12), a lens array (13), a detector (14) and a data acquisition processing display system (15) connected with the detector (14);
the spectrum mixing coding array (12) is used for modulating light beams emitted by a target, and along the direction of the light beams, the subunits of the spectrum mixing coding array (12) are provided with two surfaces with intensity reflectivity of 20% -80%, the distance between the two surfaces is larger than the wavelength magnitude, and the distances between the two surfaces in each subunit are different;
the lens array (13) is provided with sub-lenses which correspond to the sub-units of the spectrum mixed coding array (12) one by one, and each sub-lens focuses the light beams output by the corresponding sub-unit on the photosensitive surface of the detector (14) for imaging;
the data acquisition processing display system (15) is used for acquiring the coded image array acquired by the detector (14).
2. The miniature snapshot-type compressed spectral imaging detection device according to claim 1, wherein each subunit of the spectral mixture coding array (12) is a planar beam splitting plate one (121), the two surfaces are a front surface one (1211) and a back surface two (1212) of the planar beam splitting plate one (121), and the distance between the two surfaces is the thickness of the planar beam splitting plate one (121);
alternatively, the first and second liquid crystal display panels may be,
each subunit of the spectrum mixing coding array (12) is a second planar beam splitting plate (122) and a third planar beam splitting plate (123) which are sequentially arranged along incident light, a first outer surface (1221) of the second planar beam splitting plate (122) and a second outer surface (1231) of the third planar beam splitting plate (123) are parallel to each other, a first inner surface (1222) of the second planar beam splitting plate (122) and a second inner surface (1232) of the second planar beam splitting plate (122) are parallel to each other, the two surfaces are a first inner surface (1222) and a second inner surface (1232), and the distance between the first inner surface (1222) and the second inner surface (1232) is the distance between the first inner surface (1222) and the second inner surface (1232).
3. The miniature snapshot-type compressed spectral imaging detection device of claim 2, wherein the first outer surface (1221) and the first inner surface (1222) of the second planar beam splitter plate (122) are not parallel, forming a wedge angle to prevent multiple reflection oscillations of light rays inside the wedge angle;
and/or the presence of a gas in the gas,
the second outer surface (1231) and the second inner surface (1232) of the third plane beam splitting plate (123) are not parallel, and a wedge angle is formed to prevent multiple reflection oscillation of light rays in the plane beam splitting plate.
4. The miniature snapshot-based compressed spectral imaging detection device of claim 1, further comprising: an array of light blocking apertures (16);
the lens array (13) is a lens array;
the detector (14) is a single area array detector;
the light blocking hole array (16) is arranged between the lens array (13) and the detector (14) and used for limiting the field range and preventing adjacent sub-images from overlapping.
5. The miniature snapshot-type compressed spectral imaging detection device of claim 1 or 4, further comprising a collimating optical system (11);
the collimating optical system (11) comprises: an objective lens (111), a field diaphragm (112) and a collimator lens (113) which are arranged in sequence along incident light; wherein the field diaphragm (112) is arranged on the image surface of the objective lens (111), and the image surface of the objective lens (111) is coincident with the front focal surface of the collimator lens (113);
the collimating optical system (11) is positioned in front of the spectrum mixing coding array (12) and the lens array (13) and used for limiting the field range and preventing adjacent sub-images from overlapping.
6. The miniature snapshot compressed spectral imaging detection device of claim 1, wherein the lens array (13) is an objective lens array; the detector (14) is an area array detector array.
7. The detection method using the miniature snapshot type compressed spectrum imaging detection device according to any one of claims 1 to 6, characterized by comprising the following steps:
after light beams emitted by a target are modulated by a spectrum mixed coding array (12), the light beams are focused on a photosensitive surface of a detector (14) through each sub-lens in a lens array (13) to be imaged, a data acquisition processing display system (15) controls the detector (14) to snapshot a frame of coding image array carrying spatial spectrum information, intensity g distribution of each spatial position in a coding sub-image is extracted, a spectrum modulation matrix H of each spatial position in the sub-image is calibrated by using a standard light source with known spectrum distribution, and the following linear relation is established between incident spectrum distribution f of a two-dimensional space and the intensity g and the spectrum modulation matrix H:
g＝Hf，
directly estimating the incident spectral distribution f by using a compressed sensing algorithm:
or
Wherein gamma is a regularization parameter, | · |. non-woven phosphor1Is represented by1The norm, Φ, is the regularization function.
8. The detection method according to claim 7, characterized in that the data acquisition processing display system (15) comprises: a physical layer neural network and a reconstruction neural network;
the physical layer neural network takes the interval between the two surfaces as a variable and is used for simulating the function of acquiring the sub-image array by the miniature snapshot type compressed spectral imaging detection device;
the reconstruction neural network takes the network weight and the offset parameter as variables and is used for reconstructing a hyperspectral image cube;
the data acquisition, processing and display system (15) trains an optimized physical layer neural network and a reconstructed neural network simultaneously;
in the training optimization stage, the number of subunits of the minimized spectrum mixed coding array (12) is respectively used as a target and a reconstructed high-quality spectrum image cube is used as a target;
when a spectrum image cube meeting the preset requirements is trained, the optimal number and thickness of the subunits of the spectrum mixed coding array (12) are determined, and meanwhile, the optimal network weight and the bias parameters of the reconstructed neural network are obtained.
9. The detection method according to claim 8, characterized in that in the inference phase, the data acquisition processing display system (15) infers from the sub-image array a spectral image cube using the reconstructed neural network (152) and its optimized network weights and bias parameters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210335549.0A CN114659634A (en) | 2022-03-31 | 2022-03-31 | Miniature snapshot type compressed spectrum imaging detection device and detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210335549.0A CN114659634A (en) | 2022-03-31 | 2022-03-31 | Miniature snapshot type compressed spectrum imaging detection device and detection method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114659634A true CN114659634A (en) | 2022-06-24 |
Family
ID=82033885
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210335549.0A Pending CN114659634A (en) | 2022-03-31 | 2022-03-31 | Miniature snapshot type compressed spectrum imaging detection device and detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114659634A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115014522A (en) * | 2022-06-30 | 2022-09-06 | 北京理工大学 | Integrated calculation spectral imaging method and device |
| CN116183024A (en) * | 2023-04-24 | 2023-05-30 | 中国科学院长春光学精密机械与物理研究所 | Spectrum imager based on fusion imaging is cut apart with spectral band to visual field |
-
2022
- 2022-03-31 CN CN202210335549.0A patent/CN114659634A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115014522A (en) * | 2022-06-30 | 2022-09-06 | 北京理工大学 | Integrated calculation spectral imaging method and device |
| CN115014522B (en) * | 2022-06-30 | 2023-08-11 | 北京理工大学 | Integrated calculation spectrum imaging method and device |
| CN116183024A (en) * | 2023-04-24 | 2023-05-30 | 中国科学院长春光学精密机械与物理研究所 | Spectrum imager based on fusion imaging is cut apart with spectral band to visual field |
| CN116183024B (en) * | 2023-04-24 | 2023-07-04 | 中国科学院长春光学精密机械与物理研究所 | Spectrum imager based on fusion imaging is cut apart with spectral band to visual field |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7283231B2 (en) | Compressive sampling and signal inference | |
| US7532772B2 (en) | Coding for compressive imaging | |
| CN103592026B (en) | Time flight imaging spectrum system and method based on compressed sensing and coding transformation | |
| CN111426381B (en) | Ultra-compact spectrum light field camera system based on super-structure lens array | |
| CN105589210B (en) | A kind of digitlization synthetic aperture imaging method based on pupil modulation | |
| CN114659634A (en) | Miniature snapshot type compressed spectrum imaging detection device and detection method | |
| CN102818631B (en) | Compression spectral imaging system based on random wavefront phase modulation | |
| CN103471718B (en) | Hyperspectral imaging system and method based on sparse aperture compressing calculation correlation | |
| CN107271039A (en) | Compact miniature fast illuminated spectral imaging detecting device and detection method | |
| CN104121990A (en) | Random grating based compressed sensing broadband hyperspectral imaging system | |
| JP2013535931A (en) | Reduced image acquisition time for compression imaging devices | |
| CN103913229A (en) | Coding template multi-target active imaging spectrum system and method | |
| CN104019898A (en) | Ultrasensitive spectral imaging astronomical telescope and astronomical spectral imaging method | |
| CN115307733B (en) | Space dimension coding spectrum polarization integrated imaging system and design method thereof | |
| CN112113661B (en) | Deep learning type snapshot spectrum imaging device and detection method thereof | |
| CN109781260B (en) | Ultra-compact snapshot type polarization spectrum imaging detection device and detection method | |
| CN113155284B (en) | Refraction-diffraction mixed spectral coding imaging system and method | |
| CN208270074U (en) | Space-time combined modulation light field spectrum imaging system | |
| CN103916600A (en) | Coding template multi-target super-resolution imaging system and method | |
| CN109682476A (en) | A method of compression high light spectrum image-forming is carried out using adaptive coding aperture | |
| CN103743484A (en) | Frequency spectrum coding imaging apparatus | |
| CN110631703A (en) | Single-pixel spectral imaging system based on tunable optical filter | |
| CN109798978A (en) | A kind of spectrum imaging method using special-shaped blue noise code aperture | |
| CN210862922U (en) | Single-pixel spectral imaging system based on tunable optical filter | |
| CN207457047U (en) | A kind of compressed sensing imaging device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |