CN111337130B - Multispectral associated imaging method, device and equipment in push-broom mode - Google Patents
Multispectral associated imaging method, device and equipment in push-broom mode Download PDFInfo
- Publication number
- CN111337130B CN111337130B CN202010184410.1A CN202010184410A CN111337130B CN 111337130 B CN111337130 B CN 111337130B CN 202010184410 A CN202010184410 A CN 202010184410A CN 111337130 B CN111337130 B CN 111337130B
- Authority
- CN
- China
- Prior art keywords
- intensity distribution
- light intensity
- speckle field
- field light
- multispectral
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 54
- 238000001514 detection method Methods 0.000 claims abstract description 59
- 238000000034 method Methods 0.000 claims abstract description 36
- 238000004422 calculation algorithm Methods 0.000 claims description 30
- 238000002310 reflectometry Methods 0.000 claims description 26
- 238000012545 processing Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 9
- 238000000701 chemical imaging Methods 0.000 abstract description 10
- 238000004364 calculation method Methods 0.000 abstract description 3
- 239000011159 matrix material Substances 0.000 description 31
- 238000004088 simulation Methods 0.000 description 22
- 238000010586 diagram Methods 0.000 description 13
- 230000003595 spectral effect Effects 0.000 description 8
- 230000000875 corresponding effect Effects 0.000 description 7
- 238000005094 computer simulation Methods 0.000 description 6
- 238000002474 experimental method Methods 0.000 description 6
- 230000002596 correlated effect Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000010183 spectrum analysis Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2803—Investigating the spectrum using photoelectric array detector
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2823—Imaging spectrometer
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2823—Imaging spectrometer
- G01J2003/2826—Multispectral imaging, e.g. filter imaging
Abstract
The invention provides a multispectral associated imaging method, a multispectral associated imaging device and multispectral associated imaging equipment in a push-broom mode, and relates to the technical field of spectral imaging. Applied to a multispectral associated imaging device, the multispectral associated imaging device comprising: a multi-spectral camera, the method comprising: the method comprises the steps of firstly, detecting the speckle field light intensity distribution of a plurality of frames of images on a detection surface of a photoelectric detection array in a multispectral camera, then, overlapping the detected speckle field light intensities of the plurality of frames of images to obtain the total speckle field light intensity distribution, and finally, obtaining the image of a target object according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution on the detection surface. By adopting the multispectral associated imaging method in the push-broom mode, the total speckle field light intensity distribution of a plurality of frames of images is solved, and then the image of the target object can be obtained through one-time calculation.
Description
Technical Field
The invention relates to the technical field of spectral imaging, in particular to a multispectral associated imaging method, device and equipment in a push-broom mode.
Background
Spectral imaging is an optical detection technology combining spectral analysis and image analysis, and is widely applied to the technical fields of aerospace remote sensing, industry, agriculture, environment and disaster detection, atmospheric detection and the like. In particular, in the technical field of space flight and aviation remote sensing, a remote sensing camera is generally used for spectral imaging in a push-broom mode.
At present, in order to reduce the image blurring effect caused by the motion of a remote sensing camera platform, images corresponding to the light intensity distribution of a speckle field of a plurality of frames of images are firstly reconstructed, then the reconstructed plurality of frames of images are aligned and superposed, and finally a target image is obtained.
However, the speckle field light intensity of each frame of image needs to be reconstructed to obtain the corresponding image information, the reconstruction operation process is complex, and the operation is required for a long time, so that the efficiency of obtaining the target image is reduced.
Disclosure of Invention
The present invention is directed to provide a method, an apparatus, and a device for multispectral correlation imaging in a push-broom mode, which can improve the efficiency of obtaining a target image.
In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solutions:
in a first aspect, an embodiment of the present invention provides a multispectral associated imaging method in a push-broom mode, where the method is applied to a multispectral associated imaging device, and the multispectral associated imaging device includes: a multispectral camera, the method comprising:
acquiring the light intensity distribution of a speckle field calibrated on a detection surface of a photoelectric detection array in the multispectral camera;
acquiring the speckle field light intensity distribution of the multi-frame images detected on the detection surface;
overlapping the speckle field light intensity of the multi-frame images to obtain the total speckle field light intensity distribution;
and acquiring an image of the target object according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution.
Optionally, a difference between a previous frame image and a next frame image in the multi-frame image is a preset column of pixels; the speckle field light intensity of the multi-frame images is superposed to obtain the total speckle field light intensity distribution, which comprises the following steps:
and carrying out staggered superposition on the speckle field light intensity of the multi-frame image according to the preset row of pixels and the frame number of the multi-frame image to obtain the total speckle field light intensity distribution.
Optionally, the acquiring an image of the target object according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution includes:
calculating to obtain the reflectivity intensity distribution of the target object by adopting a preset reconstruction algorithm according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution;
and acquiring an image of the target object according to the reflectivity intensity distribution of the target object.
Optionally, the calculating, according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution, a preset reconstruction algorithm is used to obtain the reflectivity intensity distribution of the target object, including:
and calculating to obtain the reflectivity intensity distribution of the target object by adopting a preset gradient projection reconstruction algorithm according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution.
Optionally, the calculating, according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution, a preset reconstruction algorithm is used to obtain the reflectivity intensity distribution of the target object, including:
and calculating to obtain the reflectivity intensity distribution of the target object by adopting a preset total variation low-rank constraint image reconstruction algorithm according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution.
In a second aspect, an embodiment of the present invention further provides a multispectral associated imaging apparatus in a push-scan mode, where the multispectral associated imaging apparatus is applied to a multispectral associated imaging device, and the multispectral associated imaging device includes: a multispectral camera, the device comprising:
the first acquisition module is used for acquiring the light intensity distribution of a speckle field calibrated on a detection surface of a photoelectric detection array in the multispectral camera;
the second acquisition module is used for acquiring the speckle field light intensity distribution of the multi-frame images detected on the detection surface;
the superposition module is used for superposing the speckle field light intensity of the multi-frame image to obtain the total speckle field light intensity distribution;
and the third acquisition module is used for acquiring the image of the target object according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution.
Optionally, a difference between a previous frame image and a next frame image in the multi-frame image is a preset column of pixels; the superposition module is specifically configured to:
and carrying out staggered superposition on the speckle field light intensity of the multi-frame images according to the preset column of pixels and the frame number of the multi-frame images to obtain the total speckle field light intensity distribution.
Optionally, the third obtaining module is specifically configured to:
calculating to obtain the reflectivity intensity distribution of the target object by adopting a preset reconstruction algorithm according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution;
and acquiring an image of the target object according to the reflectivity intensity distribution of the target object.
Optionally, the third obtaining module is further specifically configured to:
and calculating to obtain the reflectivity intensity distribution of the target object by adopting a preset gradient projection reconstruction algorithm according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution.
Optionally, the third obtaining module is further specifically configured to:
and calculating to obtain the reflectivity intensity distribution of the target object by adopting a preset total variation low-rank constraint image reconstruction algorithm according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution.
In a third aspect, an embodiment of the present invention further provides a processing device, including a memory and a processor, where the memory is connected to the processor, and a computer program operable on the processor is stored in the memory, and when the processor executes the computer program, the steps of the multispectral imaging method in the push-scan mode are implemented.
In a fourth aspect, an embodiment of the present invention further provides a storage medium, where a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the multispectral imaging method in the push-scan mode are performed.
The invention has the beneficial effects that:
the embodiment of the invention provides a multispectral associated imaging method, a device and equipment under a push-broom mode. By adopting the multispectral associated imaging method in the push-broom mode, the total speckle field light intensity distribution of a plurality of frames of images is solved, and then the image of the target object can be obtained through one-time calculation.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a schematic flowchart of a multispectral correlation imaging method in a push-scan mode according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a multispectral correlation imaging system in a push-scan mode according to an embodiment of the present invention;
fig. 3 is a single frame image obtained by a simulation experiment according to an embodiment of the present invention;
fig. 4 is a multi-frame image obtained by a simulation experiment according to an embodiment of the present invention;
fig. 5 is a reconstructed image comparison diagram of a simulation experiment provided in an embodiment of the present invention under the condition of different electron number average values;
fig. 6 is a single frame image obtained by another simulation experiment provided in the embodiment of the present invention;
fig. 7 is a multi-frame image obtained by another simulation experiment according to an embodiment of the present invention;
FIG. 8 is a diagram of a reconstructed image contrast diagram of another simulation experiment provided by an embodiment of the present invention under a condition of different electron number average values;
fig. 9 is a schematic structural diagram of a multispectral correlation imaging device in a push-scan mode according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of a processing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention.
Thus, the following detailed description of the embodiments of the present invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The multispectral correlation imaging method in the push-broom mode provided by the various embodiments described below can be used for multispectral correlation imaging equipment, and the multispectral correlation imaging equipment can be generally installed on a mobile platform such as an unmanned plane or a satellite. At least a multispectral camera may be included for multispectral associated imaging devices on an onboard (e.g., unmanned aircraft) or on-board (e.g., satellite) platform. The method can be realized by a processor connected with the multispectral camera in the multispectral associated imaging equipment, and can also be finished by processing equipment connected with the multispectral associated imaging equipment, wherein the processing equipment can be any one of a notebook computer, a desktop computer, a tablet computer, a server and the like.
Fig. 1 is a schematic flowchart of a multispectral associated imaging method in a push-scan mode according to an embodiment of the present invention, where the method is applied to a multispectral associated imaging device, and the multispectral associated imaging device may include: a multispectral camera, the method may include:
s101, obtaining the light intensity distribution of a speckle field calibrated on a detection surface of a photoelectric detection array in the multispectral camera.
Specifically, the detection surface of the photoelectric detection array can be divided into a plurality of blocks, and each block corresponds to one pixel. For example, there may be m rows and n columns of pixels on the detection surface, i.e., m × n pixels. Each pixel element may include at least one photodetector, also known as a photosensor or an imaging sensor, or the like. A photodetector, which may be a Charge-coupled Device (CCD), is placed at each picture element to record the light intensity at the location of the picture element. The output ends of all the photodetectors on the detection surface of the photodetection array can be connected with the processor of the multispectral associated imaging device or the processing device connected with the multispectral associated imaging device, so that the processor or the processing device can acquire the information of the speckle field light intensity distribution detected on the detection surface. Before the multispectral camera leaves the factory, the detection surface can be calibrated with a set of speckle field light intensity distribution information which can be represented by a calibration matrix A,
the evolution process of the calibration matrix a can be described in the following mathematical language.
Wherein the content of the first and second substances,
in (1)
The s-th wavelength is represented, m and n respectively represent the m-th row and n-th column on the detection surface, and k and l respectively represent the row number and the column number corresponding to the pixel on the detection surface calibrated by generating the matrix. The speckle field light intensity distribution information calibrated on the detection surface can be stored by means of a data tableThe speckle field intensity distribution can be represented in column order in the form of a first matrix, with each element in the second matrix representing a column vector in the first matrix.
S102, obtaining the speckle field light intensity distribution of the multi-frame images detected on the detection surface.
Specifically, it is assumed that the multispectral camera captures r frames of images, each frame of image may form a speckle field on the detection surface, that is, the photodetector on the detection surface may detect speckle field light intensity distribution information of each frame of image, and transmit the speckle field light intensity distribution of each frame of image to the processor or the processing device, where the processor or the processing device may store the speckle field light intensity distribution information corresponding to the frames of image in a preset memory, where the memory may be an internal memory of the multispectral associated imaging device or an external memory of the multispectral associated imaging device, and is not limited herein. The speckle field light intensity distribution of each frame image detected by the detection surface can be represented by a detection matrix Y, and Y is ═ Y1,Y2,…,Yn]TThe evolution process of the detection matrix Y can be described in the following mathematical language.
Wherein, ymnThe speckle field light intensity distribution of each frame of image detected by the detection surface can be stored in a data table mode, the speckle field light intensity distribution in the data table can be represented in a first matrix form according to the sequence of columns, and each element in the second matrix represents a column vector in the first matrix.
S103, overlapping the speckle field light intensity of the multi-frame image to obtain the total speckle field light intensity distribution.
Specifically, after the speckle field light intensity distribution of the r-frame image is obtained, the speckle field light intensities of the front and rear frame images in the multi-frame image can be sequentially superposed to obtain the total speckle field light intensity distribution. In the process of overlapping the speckle field light intensity of the front and rear frame images, a direct overlapping mode can be adopted, a staggered overlapping mode can be adopted according to a preset staggered pixel column, other overlapping modes can be adopted, and the method is not repeated herein.
And S104, acquiring an image of the target object according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution.
Specifically, assuming that there are p × q pixels on the object plane of the target object image, and the target object image contains information of s wavelengths, the target matrix X may be used to represent the information on the target object image,
the evolution process of the target object matrix X can be described in the following mathematical language.
Wherein the content of the first and second substances,
denotes the wavelength λsWhen the target object is at p rows and q columns on the object plane, the information in the image of the target object can also be stored in a data table, the information in the data table can be expressed in the form of a first matrix according to the column sequence, and each element in the second matrix represents a column vector in the first matrix.
According to Y ═ AX + ε, where ε ═ ε1,ε2,…,εn]TIndicating a predetermined error of detection, which is mainly due to noise, and the sources of noise may include dark current noise and readout noise associated with the detector, shot noise associated with the signal intensity, etc. Moreover, the information in the detection matrix Y and the calibration matrix a is known, and epsilon is a preset error and is also known, so that the target object matrix X can be solved according to the relational expression of Y ═ AX + epsilon, and the image of the target object can be acquired.
In summary, the multispectral correlation imaging method in the push-broom mode provided by the invention detects the speckle field light intensity distribution of a plurality of frames of images on the detection surface of the photoelectric detection array in the multispectral camera, then superposes the speckle field light intensities of the detected plurality of frames of images to obtain the total speckle field light intensity distribution, and finally obtains the image of the target object according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution on the detection surface. By adopting the multispectral associated imaging method in the push-broom mode, the total speckle field light intensity distribution of a plurality of frames of images is solved, and then the image of the target object can be obtained through one-time calculation.
Furthermore, the difference between the front frame image and the rear frame image in the multi-frame image is a preset row of pixels, and the speckle field light intensity of the multi-frame image is subjected to dislocation superposition according to the preset row of pixels and the frame number of the multi-frame image, so that the total speckle field light intensity distribution is obtained.
Specifically, the multispectral camera can perform multiple exposure to obtain a multi-frame image, the field of view of the next frame image can move by d rows of pixels relative to the previous frame image, d can take the value of 1, the number of frames of the multi-frame image is r, and r can take the value of 10. Each frame of image can form a speckle field on the detection surface, namely, the photoelectric detector on the detection surface can detect the speckle field light intensity distribution of each frame of image. And carrying out dislocation superposition on the light intensity of the speckle field of the multi-frame image to obtain the total light intensity distribution of the speckle field. When the field of view of the latter frame of image can move 1 row of pixels relative to the former frame of image, the first row of the total speckle field light intensity distribution matrix is the first row of the first frame of image speckle field light intensity distribution matrix, the second row of the total speckle field light intensity distribution matrix is the second row of the first frame of image speckle field light intensity distribution matrix plus the first row of the second frame of image speckle field light intensity distribution matrix, the third row of the total speckle field light intensity distribution matrix is the third row of the first frame of image speckle field light intensity distribution matrix plus the second row of the second frame of image speckle field light intensity distribution matrix plus the first row of the third frame of image speckle field light intensity distribution matrix, and the following is analogized by the rule, and finally the total speckle field light intensity distribution can be obtained.
Further, calculating to obtain the reflectivity intensity distribution of the target object by adopting a preset reconstruction algorithm according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution; and acquiring an image of the target object according to the reflectivity intensity distribution of the target object.
Specifically, according to the principle that the speckle field light intensity distribution has local spatial translation invariance and the condition that the correlation of the speckle field light intensity distribution is greater than ninety percent, the following can be obtained:
Aj,l＝Aj±d,l±d
in the formula, j is the number of columns on the detection surface, l is the number of columns corresponding to the pixels on the detection surface calibrated by the generated calibration matrix A, and d can be obtained as the number of columns of the image shift of the two frames before and after.
Based on the principle of invariance of local space translation, if r times of speckle field light intensity distribution is added in a staggered mode according to the pixel number swept out each time, the following results are obtained:
in the formula, t represents the number of times of detection on the detection surface, and as can be seen from the formula, information in the target object matrix X can be obtained by inverting the reconstruction algorithm, and the image of the target object is restored based on the reflectance intensity distribution information of the target object. The total speckle field light intensity distribution is inverted once through a preset reconstruction algorithm, so that the operation time can be saved, and the acquired image of the target object can be clearer.
In one embodiment, the reflectivity intensity distribution of the target object can be calculated by using a preset gradient projection (GPSR-BB) reconstruction algorithm.
In another embodiment, a preset total variation low RANK constrained image (TV-RANK) reconstruction algorithm may also be adopted, and the reflectivity intensity distribution of the target object is calculated by performing inversion according to the following formula.
Then obtain
Wherein | X' | purple light*Is the nuclear norm of X, μ1、μ2Is a preset weight coefficient, and the weight coefficient,
is the spatial total variation for X.
The following can be used to illustrate the multispectral imaging method in each of the schemes provided in the present application, respectively, with reference to simulation data during computer simulation and experimental data during experiment.
Fig. 2 is a schematic structural diagram of a multispectral correlation imaging system in a push-broom mode according to an embodiment of the present invention, and a simulation and an experiment are performed based on the structural diagram of the system in both a computer simulation process and an experimental process. In this fig. 2 includes: the device comprises a target object 201, an imaging lens 202, a diaphragm 203, a field lens 204, a band-pass filter 205, a spatial random phase modulator 206, a photoelectric detection array 207 and a processor 208, wherein the imaging lens 202 can focus a light path of the target object 201, then unnecessary light is blocked by the diaphragm 203, only the required light is allowed to pass through, the rest light passes through the field lens 204 and the band-pass filter 205 to obtain light in a preset wave band, the spatial random phase modulator 206 modulates the light in the preset wave band to form a speckle field on a detection surface of the photoelectric detection array 207, the processor 208 collects and stores signals detected by the photoelectric detection array, processes the collected signal data, and obtains a multispectral image of the target object through a reconstruction algorithm.
The simulation target object in the computer simulation process can be two horizontal and vertical white strips, the target object comprises 10 wave bands, an image with q pixel elements in S amplitude can be reconstructed by single exposure of a camera, a view field of a next frame image moves d columns of pixels relative to a previous frame image, and images with 10 wave bands (535nm, 545nm, 555nm, 565nm, 575nm, 585nm, 595nm, 605nm, 615nm and 625nm) of the view field are subjected to dislocation splicing and then are used for generating a multispectral image by a gradient projection reconstruction algorithm. The number q of the lines of the single frame view field is 100, the number p of the columns of the single frame view field is 20, the number S of the detection bands is 10, and after 1 detection, the number of the pixels of the swept field of view is q × p 100 × 20 is 2000, as shown in fig. 3, which is a schematic diagram of a single frame image obtained by a simulation experiment provided in the embodiment of the present invention. As shown in fig. 4, fig. 4 is a schematic view of a multi-frame image obtained by a simulation experiment according to an embodiment of the present invention. The target object is photographed 10 times, and 10 frames of images are obtained, wherein the pixel difference d between the two frames of images is 1. After 10 frames of detection, the number of the swept field pixels is q × m, which is 100 × 29, which is 2900.
In order to evaluate the quality of the reconstructed target object image, the quality of the reconstructed target object image is evaluated by using a mean square error mse, which is defined as follows:
in the formula: i is used for representing the ith frame image, r is the shooting frame number, x is the target object and x*Is a reconstructed image. By definition, the smaller the mean square error mse value, the better the quality of the reconstructed image.
The noise model in the computer simulation process is Poisson noise, and the number y of electrons of each pixel and the number n of speckle photons incident to the pixel recorded by the CCDpSatisfies the following conditions:
y＝Gηnp (4)
wherein G is the gain and η is the quantum efficiency.
Under the condition that the electron number average value recorded by the CCD is 200e-, 1000 e-and 1500e-, a reconstruction result of a single-frame image and a result of overlapping and reconstructing speckle light intensity distribution of 10 frames of images in a staggered manner are obtained by utilizing GPSR-BB simulation respectively, and as shown in FIG. 5, an image contrast image is reconstructed under the condition of different electron number average values for a simulation experiment provided by the embodiment of the invention. Simulation results show that when the mean values of the electron numbers recorded by the CCD are 200e-, 1000 e-and 1500e-, the mean square error mse of the reconstruction result of 10 frames of images is respectively reduced by 0.28%, 0.28% and 0.27% compared with the mean square error mse of the reconstruction result of a single frame of images, and the scheme is verified to obtain high-quality reconstructed images under the conditions of different mean values of the electron numbers.
As can be seen from fig. 5, the image signal of any spectral band also spreads to the adjacent spectral band and the spreading phenomenon is reduced as the wavelength interval increases, because the imaging principle is based on the spectral resolution of the correlation of the speckle fields, and the correlation between the spectra is 50% instead of zero when the measurement matrix is calibrated, so that there is a certain correlation between the speckle fields of the adjacent spectral bands.
In order to verify the correctness of the simulation result, an experimental platform can be set up through the structure diagram of the multispectral imaging system in fig. 2, and the experiment is consistent with a target object simulated by a computer simulation and is a strip with two horizontal strips and one vertical strip. A multispectral camera and a red-green-blue (RGB) camera are adopted, wherein the RGB camera can be used as a reference camera, the multispectral camera is a snapshot camera based on phase modulation, the time resolution can reach 120fps, the spatial resolution of the multispectral camera is 597 x 32, the spectral resolution is 10 wave bands (487-633nm), and the wavelength interval is about 15 nm. The photodetectors in photodetector array 207 act as a CCD, which may have an exposure time of 70ms, 365ms, or 545 ms. The exposure interval was 545ms, and the pixel difference between the previous and subsequent frames was 1. The spatial resolution of the RGB camera is 1164 × 114 and the temporal resolution is 120 fps. The two cameras are placed in parallel, the baseline distance is 50mm, and the target scene is located 1780mm in front of the system. The target object information recorded by the RGB camera, the target vertical line and horizontal line spatial resolution are 350 multiplied by 14mm and 114 multiplied by 14mm respectively, and the spectral information of the object is represented by high reflection films of green, red and yellow. A white LED strong-light flashlight is used as a laboratory lighting source, and a single-reflection electric holder is used for driving an imaging system to rotate at a constant speed of 180 degrees/H to perform push-scanning imaging on a target object.
The experimental result shows that under the condition that the average value of the number of electrons recorded by the CCD is 200e-, 1000 e-or 1500e-, the corresponding exposure time is
And
and respectively obtaining the comparison results of the overlapping areas in the single-frame and 10-frame image speckle light intensity distribution dislocation superposition reconstruction. The result shows that the shorter the exposure time of the CCD, the better the deblurring is, and in the experiment, under the condition that the exposure time of the CCD is 70ms, 365ms and 545ms respectively, the moving object image with high signal-to-noise ratio can be reconstructed, and the motion blur can be effectively removed.
When a TV-RANK reconstruction algorithm is adopted, a simulation target object in the computer simulation process can be the same as the GPSR-BB reconstruction algorithm and is two horizontal and vertical white strips, an image with q pixel elements in S frames can be reconstructed by single exposure of a camera, a view field of a next frame of image moves d columns of pixels relative to a previous frame of image, and 10 wave bands (535nm, 545nm, 555nm, 565nm, 575nm, 585nm, 595nm, 605nm, 615nm and 625nm) of the view field are subjected to staggered splicing and then are used for generating a multispectral image by the TV-RANK reconstruction algorithm. The number q of the rows of the single frame view field is 100, the number p of the columns of the single frame view field is 11, the number S of the detection bands is 10, and after 1 detection, the number of the pixels of the swept view field is q × p 100 × 11 1100, as shown in fig. 6, which is a single frame image obtained by another simulation experiment provided in the embodiment of the present invention. As shown in fig. 7, fig. 7 is a multi-frame image obtained by another simulation experiment provided by the embodiment of the present invention. The target object is photographed 10 times, and 10 frames of images are obtained, wherein the pixel difference d between the two frames of images is 1. After 10 frames of detection, the number of the swept field pixels is q × m ═ 100 × 20 ═ 2000.
The quality of the reconstructed target object image can be evaluated by the mean square error mse using the formulas (1), (2) and (3).
Under the condition that the average value of the number of electrons recorded by the CCD is 200e-, 1000 e-and 1500e-, respectively, a reconstruction result of a single-frame image and a result of reconstructing 10 frames of scattered spot light intensity distribution by dislocation superposition are obtained by utilizing a TV-RANK reconstruction algorithm simulation, as shown in FIG. 8, an image contrast image is reconstructed under the condition of different average values of the number of electrons for another simulation experiment provided by the embodiment of the invention. Simulation results show that when the mean values of the electron numbers recorded by the CCD are 200e-, 1000 e-and 1500e-, the mean square error mse of the reconstruction result of 10 frames of images is respectively reduced by 0.11%, 0.09% and 0.08% compared with the mean square error mse of the reconstruction result of a single frame of images, and the scheme is verified to obtain high-quality reconstructed images under the conditions of different mean values of the electron numbers.
As can be seen from fig. 8, the image signal of any spectral band also spreads to adjacent spectral bands and the spreading phenomenon decreases as the wavelength interval increases. Compared with fig. 5, better target object image can be obtained by using the TV-RANK reconstruction algorithm.
In order to verify the correctness of the simulation result, an experimental platform can be set up through the structure diagram of the multispectral imaging system in fig. 2, the experimental platform can be the same as the experimental platform adopting the GPSR-BB reconstruction algorithm, and specific contents are not repeated.
Experimental results show that under the condition that the average value of the number of electrons recorded by the CCD is 200e-, 1000 e-or 1500 e-and corresponding exposure time is 70ms, 365ms or 545ms, comparison results of single-frame speckle light intensity distribution and 10-frame speckle light intensity distribution dislocation superposition reconstruction are obtained through experiments. The results show that the vertical lines of the object in the reconstructed image of 10 frames occupy 10, 11, and 12 pixels, respectively, when the exposure time is 70ms, 365ms, and 545 ms. The result shows that the shorter the exposure time of the CCD, the more obvious the deblurring is, and in the experiment, when the exposure time of the CCD is shorter, the moving object image with high signal-to-noise ratio can be reconstructed, so that the motion blur can be effectively removed.
Fig. 9 is a schematic structural diagram of a multispectral correlated imaging apparatus in a push-scan mode according to an embodiment of the present invention, where the multispectral correlated imaging apparatus is applied to a multispectral correlated imaging device, and the multispectral correlated imaging device includes: a multispectral camera, the apparatus may include:
a first obtaining module 901, configured to obtain a speckle field light intensity distribution calibrated on a detection surface of a photodetection array in the multispectral camera;
a second obtaining module 902, configured to obtain speckle field light intensity distribution of multiple frames of images detected on the detection surface;
a superposition module 903, configured to superpose the speckle field light intensities of the multiple frames of images to obtain a total speckle field light intensity distribution;
and a third obtaining module 904, configured to obtain an image of the target object according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution.
Furthermore, the difference between the front frame image and the rear frame image in the multi-frame image is a preset row of pixels; the overlay module 903 is specifically configured to:
and carrying out staggered superposition on the speckle field light intensity of the multi-frame images according to the preset column of pixels and the frame number of the multi-frame images to obtain the total speckle field light intensity distribution.
Further, the third obtaining module 904 is specifically configured to:
calculating to obtain the reflectivity intensity distribution of the target object by adopting a preset reconstruction algorithm according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution;
and acquiring an image of the target object according to the reflectivity intensity distribution of the target object.
Further, the third obtaining module 904 is further specifically configured to:
and calculating to obtain the reflectivity intensity distribution of the target object by adopting a preset gradient projection reconstruction algorithm according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution.
Further, the third obtaining module 904 is further specifically configured to:
and calculating to obtain the reflectivity intensity distribution of the target object by adopting a preset total variation low-rank constraint image reconstruction algorithm according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution.
The above-mentioned apparatus is used for executing the method provided by the foregoing embodiment, and the implementation principle and technical effect are similar, which are not described herein again.
These above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Fig. 10 is a schematic structural diagram of a processing device according to an embodiment of the present invention, including: memory 1001, processor 1002.
It should be noted that both the memory 1001 and the processor 1002 may be integrated into the processor of the multispectral imaging device, and the memory 1001 may also be a separate external memory, i.e., connected to the processor 208.
The memory 1001 is connected to the processor 1002, the memory 1001 is used for storing a computer program that can be run on the processor 1002, and the processor 1002 is used for implementing the above-mentioned method embodiments when executing the computer program. The specific implementation and technical effects are similar, and are not described herein again.
Optionally, the present invention also provides a storage medium, for example a computer-readable storage medium, comprising a program which, when executed by a processor, is adapted to perform the above-described method embodiments.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Claims (8)
1. A multispectral correlation imaging method in a push-broom mode, wherein the method is applied to a multispectral correlation imaging device, and the multispectral correlation imaging device comprises: a multispectral camera, the method comprising:
acquiring the light intensity distribution of a speckle field calibrated on a detection surface of a photoelectric detection array in the multispectral camera;
acquiring the speckle field light intensity distribution of the multi-frame images detected on the detection surface;
overlapping the speckle field light intensity of the multi-frame images to obtain the total speckle field light intensity distribution;
acquiring an image of a target object according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution;
wherein, the obtaining the image of the target object according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution comprises:
calculating to obtain the reflectivity intensity distribution of the target object by adopting a preset reconstruction algorithm according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution;
and acquiring an image of the target object according to the reflectivity intensity distribution of the target object.
2. The method according to claim 1, wherein the front frame image and the back frame image in the multi-frame image are different by a preset column of pixels; the speckle field light intensity of the multi-frame images is superposed to obtain the total speckle field light intensity distribution, which comprises the following steps:
and carrying out staggered superposition on the speckle field light intensity of the multi-frame image according to the preset row of pixels and the frame number of the multi-frame image to obtain the total speckle field light intensity distribution.
3. The method according to claim 1, wherein the calculating the reflectivity intensity distribution of the target object according to the calibrated speckle field intensity distribution and the total speckle field intensity distribution by using a preset reconstruction algorithm comprises:
and calculating to obtain the reflectivity intensity distribution of the target object by adopting a preset gradient projection reconstruction algorithm according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution.
4. The method according to claim 1, wherein the calculating the reflectivity intensity distribution of the target object according to the calibrated speckle field intensity distribution and the total speckle field intensity distribution by using a preset reconstruction algorithm comprises:
and calculating to obtain the reflectivity intensity distribution of the target object by adopting a preset total variation low-rank constraint image reconstruction algorithm according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution.
5. An apparatus for multispectral correlation imaging in a push-broom mode, the apparatus being applied to a multispectral correlation imaging device, the multispectral correlation imaging device comprising: a multispectral camera, the device comprising:
the first acquisition module is used for acquiring the light intensity distribution of a speckle field calibrated on a detection surface of a photoelectric detection array in the multispectral camera;
the second acquisition module is used for acquiring the speckle field light intensity distribution of the multi-frame images detected on the detection surface;
the superposition module is used for superposing the speckle field light intensity of the multi-frame image to obtain the total speckle field light intensity distribution;
the third acquisition module is used for acquiring an image of the target object according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution;
the third obtaining module is specifically configured to:
calculating to obtain the reflectivity intensity distribution of the target object by adopting a preset reconstruction algorithm according to the calibrated speckle field light intensity distribution and the total speckle field light intensity distribution;
and acquiring an image of the target object according to the reflectivity intensity distribution of the target object.
6. The apparatus according to claim 5, wherein the front frame image and the back frame image in the multi-frame image are different by a preset column of pixels; the superposition module is specifically configured to:
and carrying out staggered superposition on the speckle field light intensity of the multi-frame image according to the preset row of pixels and the frame number of the multi-frame image to obtain the total speckle field light intensity distribution.
7. A processing device, comprising: memory connected to the processor, a processor, in which a computer program is stored that is executable on the processor, the processor implementing the steps of the method according to any one of the preceding claims 1 to 4 when executing the computer program.
8. A storage medium, having stored thereon a computer program which, when executed by a processor, performs the steps of the method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010184410.1A CN111337130B (en) | 2020-03-16 | 2020-03-16 | Multispectral associated imaging method, device and equipment in push-broom mode |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010184410.1A CN111337130B (en) | 2020-03-16 | 2020-03-16 | Multispectral associated imaging method, device and equipment in push-broom mode |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111337130A CN111337130A (en) | 2020-06-26 |
| CN111337130B true CN111337130B (en) | 2022-05-03 |
Family
ID=71180189
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010184410.1A Active CN111337130B (en) | 2020-03-16 | 2020-03-16 | Multispectral associated imaging method, device and equipment in push-broom mode |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111337130B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102621546A (en) * | 2012-03-28 | 2012-08-01 | 浙江大学 | Three-dimensional information obtaining method based on correlated imaging |
| CN109520619A (en) * | 2018-11-26 | 2019-03-26 | 中国科学院上海光学精密机械研究所 | Relevance imaging spectrum camera and its imaging method based on non-rayleigh speckle field |
| CN110285755A (en) * | 2019-07-10 | 2019-09-27 | 吉林大学 | A kind of device and method for screening relevance imaging sampled data |
-
2020
- 2020-03-16 CN CN202010184410.1A patent/CN111337130B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102621546A (en) * | 2012-03-28 | 2012-08-01 | 浙江大学 | Three-dimensional information obtaining method based on correlated imaging |
| CN109520619A (en) * | 2018-11-26 | 2019-03-26 | 中国科学院上海光学精密机械研究所 | Relevance imaging spectrum camera and its imaging method based on non-rayleigh speckle field |
| CN110285755A (en) * | 2019-07-10 | 2019-09-27 | 吉林大学 | A kind of device and method for screening relevance imaging sampled data |
Non-Patent Citations (1)
| Title |
|---|
| 推扫模式下强度关联快照式光谱成像;吴建荣;《第四届高分辨率对地观测学术年会论文集》;20170917;第1-8页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111337130A (en) | 2020-06-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Baek et al. | Single-shot hyperspectral-depth imaging with learned diffractive optics | |
| EP2899690B1 (en) | Random imaging | |
| KR100944462B1 (en) | Satellite image fusion method and system | |
| Wagadarikar et al. | Spectral image estimation for coded aperture snapshot spectral imagers | |
| US9719856B2 (en) | Coded spectral imager | |
| US20140301659A1 (en) | Panchromatic Sharpening Method of Spectral Image Based on Fusion of Overall Structural Information and Spatial Detail Information | |
| US20130011051A1 (en) | Coded aperture imaging | |
| US20010024534A1 (en) | Super resolution methods for electro-optical systems | |
| Tao et al. | Hyperspectral image recovery based on fusion of coded aperture snapshot spectral imaging and RGB images by guided filtering | |
| Zeng et al. | Robust reconstruction with deep learning to handle model mismatch in lensless imaging | |
| Liu et al. | Measurement dimensions compressed spectral imaging with a single point detector | |
| Parmar et al. | Spatio-spectral reconstruction of the multispectral datacube using sparse recovery | |
| Yang et al. | Deep‐learning based on‐chip rapid spectral imaging with high spatial resolution | |
| US9906691B2 (en) | Methods and system for sparse blue sampling | |
| Fortuna et al. | Multivariate image fusion: A pipeline for hyperspectral data enhancement | |
| CN111337130B (en) | Multispectral associated imaging method, device and equipment in push-broom mode | |
| Descour et al. | Nonscanning no-moving-parts imaging spectrometer | |
| Torre et al. | Inverse diffraction for the atmospheric imaging assembly in the solar dynamics observatory | |
| Madhavan et al. | Statistical analysis of the electronic crosstalk correction in Terra MODIS Band 27 | |
| Vaillant et al. | Color correction matrix for sparse RGB-W image sensor without IR cutoff filter | |
| Fu et al. | Sparsity constrained fusion of hyperspectral and multispectral images | |
| Haneishi et al. | Image acquisition technique for high dynamic range scenes using a multiband camera | |
| Eskelinen | Computational methods for hyperspectral imaging using Fabry–Perot interferometers and colour cameras | |
| Chen et al. | A notch-mask and dual-prism system for snapshot spectral imaging | |
| Feng et al. | Compressive spectral integral imaging using a microlens array |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |