CN111397733A - Single/multi-frame snapshot type spectral imaging method, system and medium - Google Patents

Single/multi-frame snapshot type spectral imaging method, system and medium Download PDF

Info

Publication number
CN111397733A
CN111397733A CN202010327175.9A CN202010327175A CN111397733A CN 111397733 A CN111397733 A CN 111397733A CN 202010327175 A CN202010327175 A CN 202010327175A CN 111397733 A CN111397733 A CN 111397733A
Authority
CN
China
Prior art keywords
iteration
spectral
spectral imaging
image
frame snapshot
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010327175.9A
Other languages
Chinese (zh)
Other versions
CN111397733B (en
Inventor
李树涛
谢婷
孙斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hunan University
Original Assignee
Hunan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hunan University filed Critical Hunan University
Priority to CN202010327175.9A priority Critical patent/CN111397733B/en
Publication of CN111397733A publication Critical patent/CN111397733A/en
Application granted granted Critical
Publication of CN111397733B publication Critical patent/CN111397733B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/2823—Imaging spectrometer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J2003/283—Investigating the spectrum computer-interfaced

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Computational Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Algebra (AREA)
  • Spectrometry And Color Measurement (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)

Abstract

The invention discloses a single/multi-frame snapshot type spectral imaging method, a system and a medium, wherein a spectral image is regarded as a unified whole, and the high-dimensional physical characteristics and the global space-spectral correlation of the spectral image are mined by utilizing a tensor low tubular rank constraint model, so that the reconstruction quality of a target scene with rich spectral information or rich spatial details can be greatly improved; considering that the measurement matrix is huge, the inversion of the correlation matrix including the measurement matrix is extremely difficult and the calculation cost is high, the spectral image reconstruction is completed by utilizing the original dual algorithm iterative optimization solution with linear search, no additional matrix-vector multiplication operation is needed, the inversion of the correlation matrix including the measurement matrix can be avoided, and the method has the advantages of high reconstruction precision, low calculation cost and the like.

Description

Single/multi-frame snapshot type spectral imaging method, system and medium
Technical Field
The invention relates to a spectral image imaging technology in the field of computational photography, in particular to a single/multi-frame snapshot type spectral imaging method, a system and a medium, and particularly can reconstruct a spectral image with high precision at low computational cost.
Background
The spectral image belongs to a three-dimensional (3D) cube image, comprises two-dimensional spatial information and one-dimensional spectral information, and is divided into a hyperspectral image and a multispectral image. Compared with a general full-color image and an RGB color image, the spectral image has dozens of to hundreds of wave bands, can provide richer spectral information, and is helpful for judging different substances in a target scene. Therefore, the high/multispectral imaging technology is widely applied to the fields of remote sensing, medical imaging, geological monitoring, biometry, food quality analysis and the like.
To acquire spectral images, conventional spectral imaging systems perform time-sequential scans along a spatial or spectral domain, which is slow. In addition, due to the large amount of acquired data, a large amount of equipment is required for transmission, and a large amount of storage space is occupied, so that various adverse factors prevent the spectral image from being widely applied.
In recent years, with the development of computational imaging technology and compressed sensing theory, the appearance of single/multi-frame snapshot type spectral imaging systems breaks the technical barrier of the traditional imaging systems, which can complete compressed spectral sampling of a target scene at a number far lower than the nyquist sampling number and then reconstruct a spectral image from an underdetermined linear equation set by using a compressed sensing reconstruction algorithm. The most representative of the single/multi-frame snapshot type spectral imaging system is a coded aperture snapshot spectral imaging system (CASSI) proposed by David Brady et al of Duke university, which performs operations such as coding, dispersion and integration on spectral information to realize two-dimensional compression sampling of a three-dimensional spectral image so as to obtain spectral information of a target scene, and the imaging is rapid. The snapshot type spectral imaging is divided into single-frame snapshot type and multi-frame snapshot type imaging, wherein the single-frame snapshot type spectral coding image can only provide limited compression measurement values, and the high-precision reconstruction of the spectral images of hundreds of wave bands is difficult. However, the existing research shows that (see d.kittle, k.choi, a.wadadarikar, d.brady. "multiframeage estimation for coded mapping snapshot spectral images". applied optics.vol.49.2010.pp.6824-6833.) the use of multi-frame snapshot type spectral coding images can increase the data acquisition amount in the reconstruction algorithm and improve the reconstruction effect.
How to accurately reconstruct a three-dimensional spectral image from a single/multi-frame snapshot type spectral coding image is a core problem to be solved by such a single/multi-frame snapshot type spectral imaging system. The existing Reconstruction algorithm starts from a compressive sensing principle, utilizes prior information of a spectral image to construct a target equation with constraints, and then performs optimization solution to reconstruct the spectral image, such as a Total variation constraint method (TV) based on piecewise smoothing, a Gradient Projection method (GPSR) based on orthogonal Sparse transformation, and Sparse constraints on an over-complete dictionary. The total variation constraint method is easy to cause the phenomenon of over-smooth of the image, and the original detail texture is lost. In addition, the spectral image is reconstructed by sparse constraint on the overcomplete dictionary of the conversion domain, although the reconstruction quality is improved to a certain extent, the algorithm converts the three-dimensional spectral image vector into a one-dimensional vector or a two-dimensional matrix for processing, the high-dimensional structural characteristic and the global space-spectrum correlation of the spectral image are damaged, the reconstruction quality is not ideal enough, and the wide application of the single/multi-frame snapshot spectral imaging system in practice is greatly limited.
Disclosure of Invention
The reconstruction algorithm of the existing single/multi-frame snapshot type spectral imaging system is high in calculation cost, and the reconstruction accuracy of a target scene with rich spectral information or spatial details is not ideal because the correlation between the high-dimensional structural characteristics of a spectral image and the global space-spectrum is ignored. The invention can greatly improve the reconstruction quality of target scenes with rich spectrum information or rich space details, does not need additional matrix-vector multiplication operation, can avoid inverting a correlation matrix containing a measurement matrix, and has the advantages of high reconstruction precision, low calculation cost and the like.
In order to solve the technical problems, the invention adopts the technical scheme that:
a single/multi-frame snapshot type spectral imaging method comprises the implementation steps of:
1) inputting a measurement value g and a measurement matrix H of a single/multi-frame snapshot type spectral imaging system aiming at an original hyperspectral image X;
2) generating an initial spectral image X from the measured values g and the measurement matrix H0And an initial auxiliary variable y0The value of the number k of initial iterations is 0 and the initial value tau of the step length tau0Is a value greater than 0, and the initial value theta of the step ratio theta0；
3) Calculating spectral image X of kth iterationk；
4) Selecting step length of k iteration
Figure BDA0002463633460000021
Wherein tau isk-1Denotes the step τ, θ for the k-1 th iterationk-1Represents the step ratio theta of the (k-1) th iteration;
5) calculating a step ratio theta of the kth iterationk＝τk/τk-1；
6) Auxiliary variable y according to the k-th iterationkUpdating the (k + 1) th iteration auxiliary variable yk+1；
7) Judgment of
Figure BDA0002463633460000022
Whether the answer is true or not, if yes, skipping to execute the step 8); otherwise, updating the step length tau of the kth iterationk＝τku, jumping to execute the step 5), wherein β and u are weight coefficients;
8) judging whether a preset termination condition is met, and if the preset termination condition is met, skipping to execute the next step; otherwise, adding 1 to the iteration times k, and skipping to execute the step 3);
9) outputting spectral image X of kth iterationkAs reconstructed spectral imaging.
Optionally, the functional expression of the measurement value g and the measurement matrix H in step 1) is as follows:
g＝[g1；g2；…；gI]
H＝[H1；H2；…；HI]
in the above formula, g1～gIRespectively, measured values H of the single/multi-frame snapshot type spectral imaging system aiming at the ith exposure of the original hyperspectral image X1～HISingle/multi-frame snapshot type spectral imaging system aiming at original hyperspectral imageTaking the measurement matrix of the ith exposure of the X, wherein I is the exposure time, and when the exposure time I is 1, single-frame snapshot spectral imaging is performed; number of exposures I>When the time is 2, multi-frame snapshot type spectral imaging is carried out.
Optionally, generating an initial spectral image X in step 2)0And an initial auxiliary variable y0The function expression of (a) is as follows:
X0＝HTg
y0＝H·vec(X0)-g
in the above formula, HTFor the transpose of the measurement matrix H, g is the measurement value, vec (X)0) Is an initial spectral image X0In vectorized form.
Optionally, calculating the spectral image X of the kth iteration in step 3)kThe function expression of (a) is as follows:
Figure BDA0002463633460000031
in the above formula, λ is a weight coefficient, τk-1Represents the step length tau, | X | | Y of the k-1 th iteration*Tensor nuclear norm, X, representing the original hyperspectral image Xk-1Spectral image representing the k-1 iteration, HTFor the transpose of the measurement matrix H, ykAuxiliary variables representing the kth iteration, U, V are coefficient tensors,
Figure BDA0002463633460000032
representing singular value thresholding operations.
Optionally, the detailed steps of step 6) include:
6.1) updating the auxiliary variables of the kth iteration according to the following equation
Figure BDA0002463633460000033
Figure BDA0002463633460000034
In the above formula, XkIs shown asSpectral image of k iterations, Xk-1Spectral image, theta, representing the k-1 th iterationkRepresents the step ratio of the kth iteration;
6.2) auxiliary variable y at the k-th iteration obtainedkAuxiliary variables for the k-th iteration
Figure BDA0002463633460000035
Based on the step length tau of the selected kth iterationkUpdating the (k + 1) th iteration auxiliary variable y by adopting the following formulak+1；
Figure BDA0002463633460000036
In the above formula, ykThe auxiliary variables, β, representing the kth iteration are all weight coefficients, τkDenotes the step τ of the kth iteration, H denotes the measurement matrix,
Figure BDA0002463633460000037
auxiliary variables representing the kth iteration
Figure BDA0002463633460000038
In vectorized form, g is the measured value.
Alternatively, the functional expression of the termination condition in step 8) is as follows:
||Xk-Xk-1||F≤||Xk-1||F
in the above formula, the tolerance parameter, XkSpectral image, X, representing the kth iterationk-1Spectral image representing the k-1 iteration, | Xk-1||FRepresenting a spectral image Xk-1Norm of, | | Xk-Xk-1||FRepresenting a spectral image XkSubtracting the spectral image Xk-1The norm of the resulting difference.
Optionally, the termination condition in step 8) is that the iteration number K is equal to a preset iteration number threshold Kmax。
In addition, the invention also provides a single/multi-frame snapshot type spectral imaging system, which comprises a computer device which is programmed or configured to execute the steps of the single/multi-frame snapshot type spectral imaging method.
In addition, the invention also provides a single/multi-frame snapshot type spectral imaging system, which comprises a computer device, wherein a computer program which is programmed or configured to execute the single/multi-frame snapshot type spectral imaging method is stored in a memory of the computer device.
Furthermore, the present invention also provides a computer-readable storage medium having stored therein a computer program programmed or configured to execute the single/multiple frame snapshot-based spectral imaging method.
Compared with the prior art, the invention has the following advantages:
1. the method takes the spectral image as a unified whole, utilizes a tensor low-tubular rank constraint model to excavate the high-dimensional physical characteristics and the global space-spectrum correlation of the spectral image, and can greatly improve the reconstruction quality of a target scene with rich spectral information or rich space details; meanwhile, considering that the measurement matrix is huge, the inversion of the correlation matrix including the measurement matrix is extremely difficult and the calculation cost is high, the spectral image reconstruction is completed by utilizing the original dual algorithm iterative optimization solution with linear search, no additional matrix-vector multiplication operation is needed, and the inversion of the correlation matrix including the measurement matrix can be avoided.
2. The method can be used for single-frame snapshot type spectral imaging and also can be used for multi-frame snapshot type spectral imaging.
Drawings
FIG. 1 is a schematic diagram of a basic flow of a method according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of a sampling principle of an original hyperspectral image X in an embodiment of the invention.
Detailed Description
The single/multiframe snapshot type spectral imaging system (single/multiframe snapshot type coded aperture spectral imaging system) mainly comprises an objective lens, a filter, a coded aperture, a relay environment, a dispersion prism, an array detector and other devices, incident light reaches the coded aperture through the objective lens and the filter during sampling to carry out random coding, and then a scene spectral image subjected to coding and light splitting (the dispersion prism) is captured by the array detector to obtain a measured value of the single frame snapshot type spectral imaging system; the multi-frame snapshot type spectral imaging can be realized by carrying out multiple exposure (namely, different coded aperture exposure is used every time) on the same scene, and the measured value of the multi-frame snapshot type spectral imaging system is obtained, wherein different coded apertures are obtained by changing the random coding of each frame on the digital micromirror device. In this embodiment, the measurement value of the single/multi-frame snapshot type spectral imaging system is collectively referred to as g, and the measurement value of the multi-frame snapshot type spectral imaging system can increase the data acquisition amount in the reconstruction algorithm, thereby improving the reconstruction effect.
As shown in fig. 1, the implementation steps of the single/multiple frame snapshot type spectral imaging method of this embodiment include:
1) inputting a measurement value g and a measurement matrix H of a single/multi-frame snapshot type spectral imaging system aiming at an original hyperspectral image X;
2) generating an initial spectral image X from the measured values g and the measurement matrix H0And an initial auxiliary variable y0The value of the number k of initial iterations is 0 and the initial value tau of the step length tau0Is a value greater than 0, and the initial value theta of the step ratio theta0；
3) Calculating spectral image X of kth iterationk；
4) Selecting step length of k iteration
Figure BDA0002463633460000051
Wherein tau isk-1Denotes the step τ, θ for the k-1 th iterationk-1Represents the step ratio theta of the (k-1) th iteration;
5) calculating a step ratio theta of the kth iterationk＝τk/τk-1；
6) Auxiliary variable y according to the k-th iterationkUpdating the (k + 1) th iteration auxiliary variable yk+1；
7) Judgment of
Figure BDA0002463633460000052
Whether the answer is true or not, if yes, skipping to execute the step 8); otherwise, updating the step length tau of the kth iterationk＝τku, jumping to execute the step 5), wherein β and u are weight coefficients;
8) judging whether a preset termination condition is met, and if the preset termination condition is met, skipping to execute the next step; otherwise, adding 1 to the iteration times k, and skipping to execute the step 3);
9) outputting spectral image X of kth iterationkAs reconstructed spectral imaging.
In this embodiment, the functional expression of the measurement value g and the measurement matrix H in step 1) is shown as follows:
g＝[g1；g2；…；gI]
H＝[H1；H2；…；HI]
in the above formula, g1～gIRespectively, measured values H of the single/multi-frame snapshot type spectral imaging system aiming at the ith exposure of the original hyperspectral image X1～HIRespectively measuring matrixes of a single/multi-frame snapshot type spectral imaging system for ith exposure of an original hyperspectral image X, wherein I is the exposure times, and when the exposure times I are 1, single-frame snapshot type spectral imaging is performed; number of exposures I>When the time is 2, multi-frame snapshot type spectral imaging is carried out. Referring to FIG. 2, I exposures are performed on an original hyperspectral image X, and a total of I measurement matrices H can be obtained1～HIAnd its corresponding measured value g1～gI。
Assume that the size of the original hyperspectral image X (target scene) is M × N ×L, wherein M × N represents the spatial resolution size of the spectral image, L represents the number of spectral bands of the spectral image, the spatial index position with (j, l, k) representing the kth spectral band is (j, l), and the code value with (j, l, k) the ith exposure time is recorded as
Figure BDA0002463633460000053
The pixel value of the scene spectrum image with the index position of (j, l, k) is recorded asFj,l,kThen, the discrete mathematical model of the single/multi-frame snapshot type spectral imaging system is:
Figure BDA0002463633460000054
in the above formula, the first and second carbon atoms are,
Figure BDA0002463633460000055
for the ith array detector measurement with spatial index position (j, l), Fj,l,kThe pixel value of the scene spectrum image with the index position of (j, l, k),
Figure BDA0002463633460000056
is the code value with the index position as (j, l, k) the ith exposure time,
Figure BDA0002463633460000057
for imaging system noise, I is the number of exposures (I < L.) thus, the measurement g for a single/multiple frame snapshot spectral imaging system can also be written as follows:
Figure 1
in the above formula, g1～gIRespectively, measured values H of the single/multi-frame snapshot type spectral imaging system aiming at the ith exposure of the original hyperspectral image X1～HIRespectively, a measurement matrix omega of a single/multi-frame snapshot type spectral imaging system aiming at ith exposure of an original hyperspectral image X1～ωIAiming at the noise of the ith exposure of the original hyperspectral image X, I is the exposure times, vec (X) is the vector form of the original hyperspectral image X, and vec (X) is vec ([ f)1,…,fL]) Wherein f iskIs a vectorized version of the k-th spectral band of X.
Thus, the mathematical model of the single/multiple frame snapshot spectral imaging system can be expressed as:
g＝H·vec(X)+ω
in the above formula, g ═ g1；g2；…；gI]，H＝[H1；H2；…；HI]，ω＝[ω1；ω2；…；ωI]。
By utilizing spectral correlation and space-spectrum combined correlation of the spectral image and combining a snapshot type spectral imaging system mathematical model, converting the spectral image reconstruction problem into an optimization problem based on tensor low tubular rank constraint, and obtaining a target optimization function as follows:
Figure BDA0002463633460000062
in the above formula, the first and second carbon atoms are,
Figure BDA0002463633460000063
representing the reconstructed spectral image, g represents the measurement value of the single/multi-frame snapshot type spectral imaging system, H represents the measurement matrix of the single/multi-frame snapshot type spectral imaging system, vec (X) is the vectorization form of the original hyperspectral image X (target scene), and lambda is a weight coefficient,
Figure BDA0002463633460000064
is the square of the norm L2, | | · |. non-woven*Is the norm of the kernel of the tensor,
Figure BDA0002463633460000065
wherein X is U is S is VTWhere S is the kernel tensor, U, V are the coefficient tensors, R denotes the tensor low-tubular rank, R ═ R { R, S (R, R,1) ≠ 0},. is the t-product (t-product) calculation, S (R, R,1) denotes the value of the kernel tensor S at the index position (R, R,1), R denotes the position componentMNI×MNLIs a 'dwarf' huge matrix with the number of columns far larger than the number of rows, and forms a related matrix (H) containing HTH+γI)∈RMNL×MNLEven more huge, the inversion of the correlation matrix including H is extremely difficult, especially for multi-frame snapshot spectral imaging. Wherein gamma is a traditional solution algorithm introduced parameter, and I is an identity matrix. Book (I)In the embodiment, a primal-dual algorithm with linear search is adopted to solve an objective optimization function, and the primal-dual algorithm is used for respectively and optimally solving a primal problem and a dual problem. The method for solving the objective optimization function by adopting the primal-dual algorithm with linear search in the embodiment avoids the inversion of the correlation matrix including H, does not need any additional matrix-vector multiplication operation, and has the advantages of low calculation cost and the like.
Let q (X) ═ λ X | non-phosphor*，
Figure BDA0002463633460000066
The objective optimization function can be converted to the following saddle point problem:
Figure BDA0002463633460000067
in the above formula, M × N represents the spatial resolution of the original hyperspectral image X, L represents the number of spectral segments of the original hyperspectral image X, I is the total number of exposures, H represents a snapshot-type spectral imaging measurement matrix of multiple exposures, vec (X) is a vectorization form of the original hyperspectral image X
Figure BDA0002463633460000071
Is the conjugate function of the convex function f (x), y is the auxiliary variable, and g represents the measured value.
In this embodiment, the initial spectral image X is generated in step 2)0And an initial auxiliary variable y0The function expression of (a) is as follows:
X0＝HTg
y0＝H·vec(X0)-g
in the above formula, HTFor the transpose of the measurement matrix H, g is the measurement value, vec (X)0) Is an initial spectral image X0In vectorized form. In addition, the initial value θ of the step ratio θ in the present embodiment0Is 1, and further includes initializing the parameters u ∈ (0,1), ∈ (0,1), β > 0, and 10-4Regularization coefficient λ, iteration number threshold Kmax500, etc.
And 3) step 6) is a step of solving the saddle point problem by using a primal-dual algorithm with linear search. In this embodiment, the spectral image X of the kth iteration is calculated in step 3)kThe function expression of (a) is as follows:
Figure BDA0002463633460000072
in the above formula, λ is a weight coefficient, τk-1Represents the step length tau, | X | | Y of the k-1 th iteration*Tensor nuclear norm, X, representing the original hyperspectral image Xk-1Spectral image representing the k-1 iteration, HTFor the transpose of the measurement matrix H, ykAuxiliary variables representing the kth iteration, U, V are coefficient tensors,
Figure BDA0002463633460000073
representing singular value thresholding operations. The derivation of the above equation is as follows:
solving the original hyperspectral image X:
Figure BDA0002463633460000074
in the above formula, XkSpectral image representing k iterations, prox represents the near-end operator (X)k -1Spectral image, τ, representing the k-1 th iterationk-1Denotes the step τ, H of the k-1 th iterationTFor the transpose of the measurement matrix H, ykRepresenting the auxiliary variable for the kth iteration. The near-end operator at point V as function q can be expressed as:
Figure BDA0002463633460000075
in the above equation, μ denotes a regularization coefficient, q (x) denotes a function q,
Figure BDA0002463633460000076
represents the square of the distance between the two points X and V. Combine the above two letterThe spectral image X of the kth iteration calculated in the step 3) can be obtained by numerical expressionkAnd wherein:
Xk-1-τk-1HTyk＝U*S*VT,
Figure BDA0002463633460000081
Figure BDA0002463633460000082
in the above formula, Xk-1Indicating the value of the (k-1) th iteration of the spectral image X to be reconstructed, fft indicating Fourier transform, ifft indicating inverse Fourier transform, S being a nuclear tensor, and α indicating any nonnegative real number.
In this embodiment, the detailed steps of step 6) include:
6.1) updating the auxiliary variables of the kth iteration according to the following equation
Figure BDA0002463633460000083
Figure BDA0002463633460000084
In the above formula, XkSpectral image, X, representing the kth iterationk-1Spectral image, theta, representing the k-1 th iterationkRepresents the step ratio of the kth iteration;
6.2) auxiliary variable y at the k-th iteration obtainedkAuxiliary variables for the k-th iteration
Figure BDA0002463633460000085
Based on the step length tau of the k-th iterationkUpdating the (k + 1) th iteration auxiliary variable y by adopting the following formulak+1；
Figure BDA0002463633460000086
In the above formula, ykThe auxiliary variables, β, representing the kth iteration are all weight coefficients, τkDenotes the step τ of the kth iteration, H denotes the measurement matrix,
Figure BDA0002463633460000087
auxiliary variables representing the kth iteration
Figure BDA0002463633460000088
In vectorized form, g is the measured value. Wherein the (k + 1) th iteration auxiliary variable y is updatedk+1The derivation of the functional expression of (a) is as follows:
Figure BDA0002463633460000089
in the above formula, the first and second carbon atoms are,
Figure BDA00024636334600000810
representing function f*(y) at point
Figure BDA00024636334600000811
The near-end operator of (c).
As described above, the conditions for determining whether the determination is true in step 7) are:
Figure BDA00024636334600000812
expressed by the above formula is HTyk+1、HTykOf a distance between
Figure BDA00024636334600000813
Multiple of yk+1、ykThe multiple of the distance therebetween.
The above conditions can be equivalently:
τkσk||HTyk+1-HTyk||2≤2||yk+1-yk||2
in the above formula, σk＝βτk. If this condition is satisfied, then:
Figure BDA0002463633460000091
and because:
Figure BDA0002463633460000092
therefore, let H L, we can get:
Figure BDA0002463633460000093
because of ∈ (0,1), there is τkσkL2Less than or equal to 1. This condition ensures convergence of the algorithm (see a. chambole and t. pack, a first-order private algorithm for dependent schemes with applications to imaging, j. math. imaging. vis.,40(2011), pp. 120-145.).
As an alternative implementation manner, the termination condition in step 8) is that the iteration number K is equal to a preset iteration number threshold KmaxFor example, in the present embodiment, the iteration number threshold K is setmaxTaking the value of 500, the iteration is terminated after 500 iterations.
Further, as another alternative embodiment, the functional expression of the termination condition in step 8) is as follows:
||Xk-Xk-1||F≤||Xk-1||F
in the above formula, the tolerance parameter, XkSpectral image, X, representing the kth iterationk-1Spectral image representing the k-1 iteration, | Xk-1||FRepresenting a spectral image Xk-1Norm of, | | Xk-Xk-1||FRepresenting a spectral image XkSubtracting the spectral image Xk-1The norm of the resulting difference.
In order to obtain the effect of the single/multi-frame snapshot spectral imaging method of the embodiment, simulation experiments and comparative analysis are performed on a single/multi-frame snapshot spectral imaging system, the experimental data are from a shift DC mail data set and a copy data set, a sample Image in an original shift DC mail data set comprises 210 bands, the size is 1208 35307 36191, a part affected by water vapor is removed, the remaining 191 bands are selected as experimental data, the spatial size is 280 × 307, the sample Image in the original copy data set comprises 224 bands, the band severely affected by water vapor is removed, the remaining 188 bands are selected as experimental data, the spatial size is 205 ×, the comparative method adopts an istwt algorithm with TV constraint (see biochas-Dias JM, if the comparison method adopts a shift algorithm, Two-Step Iterative simulation algorithm for reconstruction of storage for storage [ psn ] and mapping information for storage J, the result is greater, the signal-to noise ratio of reconstruction of the single/multi-frame snapshot map imaging system is represented by a signal-noise ratio mapping algorithm, the Peak value of PSNR 5, the result is listed as a reconstruction result of a random mapping table, a Peak value of a single/multiple-frame snapshot map data set, a result of a reconstruction algorithm with a Peak value of a Peak coding algorithm (2007) and a Peak value of a Peak coding result of a reconstruction module under IEEE # 2007, a Peak coding result of a Peak of a single/multiple frame mapping table, a structure of a reconstruction module under a structure of the present embodiment, a system, a random mapping table, a structure of a reconstruction module, a structure of a system, a system under a random mapping table, a random mapping table of a system under a random mapping table of a system under a random.
Table 1: the method of the embodiment is compared with the reconstruction effect of the TwinST algorithm under the Washington DC Mall data set.
Figure BDA0002463633460000101
Table 2: the method of the embodiment is compared with the reconstruction effect of the TwinT algorithm under the Currite data set.
Figure BDA0002463633460000102
As can be seen from tables 1 and 2, the peak snr and the structural similarity of the reconstructed image by the single/multi-frame snapshot spectral imaging method are much higher than those of the twinst algorithm, and especially under the condition that the exposure times are I ═ 2 and 4, respectively, the single/multi-frame snapshot spectral imaging method of the present embodiment has a more significant advantage in terms of the recovery accuracy.
In the single/multi-frame snapshot type spectral imaging method, the spectral images are regarded as a unified whole, the tensor low-tubular rank (tensor low-tube rank) constraint model is used for mining the high-dimensional physical characteristics and the global space-spectral correlation of the spectral images, the reconstruction quality of the target scene with rich spectral information or rich spatial detail can be greatly improved, meanwhile, the original dual algorithm (PDA L) with linear search is used for iterative optimization to complete the spectral image reconstruction, extra matrix-vector operation is needed, the reconstruction of the target scene with rich spectral information or rich spatial detail can be avoided, and the reconstruction accuracy of the target scene with high spectral information or rich spatial detail is not high.
In addition, the present embodiment also provides a single/multiple frame snapshot type spectral imaging system, which includes a computer device programmed or configured to execute the steps of the aforementioned single/multiple frame snapshot type spectral imaging method.
In addition, the embodiment also provides a single/multi-frame snapshot type spectral imaging system, which includes a computer device, and a computer program programmed or configured to execute the single/multi-frame snapshot type spectral imaging method is stored in a memory of the computer device.
Furthermore, the present embodiment also provides a computer-readable storage medium in which a computer program programmed or configured to execute the aforementioned single/multiple frame snapshot-type spectral imaging method is stored.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The present application is directed to methods, apparatus (systems), and computer program products according to embodiments of the application wherein instructions, which execute via a flowchart and/or a processor of the computer program product, create means for implementing functions specified in the flowchart and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may occur to those skilled in the art without departing from the principle of the invention, and are considered to be within the scope of the invention.

Claims (10)

1. A single/multi-frame snapshot type spectral imaging method is characterized by comprising the following implementation steps:
1) inputting a measurement value g and a measurement matrix H of a single/multi-frame snapshot type spectral imaging system aiming at an original hyperspectral image X;
2) generating an initial spectral image X from the measured values g and the measurement matrix H0And an initial auxiliary variable y0The value of the number k of initial iterations is 0 and the initial value tau of the step length tau0Is a value greater than 0, and the initial value theta of the step ratio theta0；
3) Calculating spectral image X of kth iterationk；
4) Selecting step length of k iteration
Figure FDA0002463633450000011
Wherein tau isk-1Denotes the step τ, θ for the k-1 th iterationk-1Represents the step ratio theta of the (k-1) th iteration;
5) calculating a step ratio theta of the kth iterationk＝τk/τk-1；
6) Auxiliary variable y according to the k-th iterationkUpdating the (k + 1) th iteration auxiliary variable yk+1；
7) Judgment of
Figure FDA0002463633450000012
Whether the answer is true or not, if yes, skipping to execute the step 8); otherwise, updating the step length tau of the kth iterationk＝τku, jumping to execute the step 5), wherein β and u are weight coefficients;
8) judging whether a preset termination condition is met, and if the preset termination condition is met, skipping to execute the next step; otherwise, adding 1 to the iteration times k, and skipping to execute the step 3);
9) outputting spectral image X of kth iterationkAs reconstructed spectral imaging.
2. The single/multiple frame snapshot type spectral imaging method according to claim 1, wherein the function expression of the measurement value g and the measurement matrix H in step 1) is as follows:
g＝[g1；g2；…；gI]
H＝[H1；H2；…；HI]
in the above formula, g1～gIRespectively, measured values H of the single/multi-frame snapshot type spectral imaging system aiming at the ith exposure of the original hyperspectral image X1～HIRespectively measuring matrixes of a single/multi-frame snapshot type spectral imaging system for ith exposure of an original hyperspectral image X, wherein I is the exposure times, and when the exposure times I are 1, single-frame snapshot type spectral imaging is performed; number of exposures I>When the time is 2, multi-frame snapshot type spectral imaging is carried out.
3. The method for single/multiple frame snapshot based spectral imaging according to claim 1, wherein an initial spectral image X is generated in step 2)0And an initial auxiliary variable y0The function expression of (a) is as follows:
X0＝HTg
y0＝H·vec(X0)-g
in the above formula, HTFor the transpose of the measurement matrix H, g is the measurement value, vec (X)0) Is an initial spectral image X0In vectorized form.
4. The single/multiple frame snapshot type spectral imaging method according to claim 1, wherein the spectral image X of the kth iteration is calculated in step 3)kThe function expression of (a) is as follows:
Figure FDA0002463633450000021
in the above formula, λ is a weight coefficient, τk-1Represents the step length tau, | X | | Y of the k-1 th iteration*Tensor nuclear norm, X, representing the original hyperspectral image Xk-1Spectral image representing the k-1 iteration, HTFor the transpose of the measurement matrix H, ykAuxiliary variables representing the kth iteration, U, V are coefficient tensors,
Figure FDA0002463633450000022
representing singular value thresholding operations.
5. The single/multiple frame snapshot based spectral imaging method according to claim 1, wherein the detailed step of step 6) comprises:
6.1) updating the auxiliary variables of the kth iteration according to the following equation
Figure FDA0002463633450000023
Figure FDA0002463633450000024
In the above formula, XkSpectral image, X, representing the kth iterationk-1Spectral image, theta, representing the k-1 th iterationkRepresents the step ratio of the kth iteration;
6.2) auxiliary variable y at the k-th iteration obtainedkAuxiliary variables for the k-th iteration
Figure FDA0002463633450000025
Based on the step length tau of the selected kth iterationkUpdating the (k + 1) th iteration auxiliary variable y by adopting the following formulak+1；
Figure FDA0002463633450000026
In the above formula, ykThe auxiliary variables, β, representing the kth iteration are all weight coefficients, τkDenotes the step τ of the kth iteration, H denotes the measurement matrix,
Figure FDA0002463633450000027
auxiliary variables representing the kth iteration
Figure FDA0002463633450000028
In vectorized form, g is the measured value.
6. The method for single/multiple frame snapshot based spectral imaging according to claim 1, wherein the termination condition in step 8) is expressed as a function of the following formula:
||Xk-Xk-1||F≤||Xk-1||F
in the above formula, the tolerance parameter, XkSpectral image, X, representing the kth iterationk-1Spectral image representing the k-1 iteration, | Xk-1||FRepresenting a spectral image Xk-1Norm of, | | Xk-Xk-1||FRepresenting a spectral image XkSubtracting the spectral image Xk-1The norm of the resulting difference.
7. The single/multiple frame snapshot type spectral imaging method according to claim 1, wherein the termination condition in step 8) is that the number of iterations K is equal to a preset threshold number of iterations Kmax。
8. A single/multiple frame snapshot spectral imaging system comprising a computer device, wherein the computer device is programmed or configured to perform the steps of the single/multiple frame snapshot spectral imaging method of any one of claims 1-7.
9. A single/multiple frame snapshot spectral imaging system comprising a computer device, wherein a computer program programmed or configured to perform the single/multiple frame snapshot spectral imaging method of any one of claims 1-7 is stored in a memory of the computer device.
10. A computer-readable storage medium having stored thereon a computer program programmed or configured to perform the single/multiple frame snapshot spectral imaging method of any one of claims 1-7.
CN202010327175.9A 2020-04-23 2020-04-23 Single/multi-frame snapshot type spectral imaging method, system and medium Active CN111397733B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010327175.9A CN111397733B (en) 2020-04-23 2020-04-23 Single/multi-frame snapshot type spectral imaging method, system and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010327175.9A CN111397733B (en) 2020-04-23 2020-04-23 Single/multi-frame snapshot type spectral imaging method, system and medium

Publications (2)

Publication Number Publication Date
CN111397733A true CN111397733A (en) 2020-07-10
CN111397733B CN111397733B (en) 2021-03-02

Family

ID=71435328

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010327175.9A Active CN111397733B (en) 2020-04-23 2020-04-23 Single/multi-frame snapshot type spectral imaging method, system and medium

Country Status (1)

Country Link
CN (1) CN111397733B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115950837A (en) * 2023-03-10 2023-04-11 湖南师范大学 Snapshot type spectral imaging method, system and medium based on plug-and-play prior

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102393248A (en) * 2011-10-26 2012-03-28 中国科学院空间科学与应用研究中心 Time-resolved extreme-low-light multispectral imaging system and method
WO2017017684A1 (en) * 2015-07-30 2017-02-02 Technology Innovation Momentum Fund (Israel) Limited Partnership Spectral imaging method and system
CN106768337A (en) * 2017-02-27 2017-05-31 中国科学院上海高等研究院 A kind of phase reconstruction method in two-dimensional Fourier transform electronic spectrum
CN107084789A (en) * 2017-04-01 2017-08-22 上海理工大学 Single pixel detector spectrum reflectivity reconstructing method based on sparse prior
CN107525588A (en) * 2017-08-16 2017-12-29 北京理工大学 A kind of method for fast reconstruction of the double camera spectrum imaging system based on GPU
US20180045569A1 (en) * 2016-08-12 2018-02-15 Spectral Insights Private Limited Spectral imaging system
US20190078937A1 (en) * 2017-09-13 2019-03-14 Qualcomm Incorporated Directional interpolation and cross-band filtering for hyperspectral imaging
CN110095188A (en) * 2019-04-02 2019-08-06 杭州盗火者科技有限公司 A kind of spectrum restoring method, device and computer readable storage medium
EP3203215B1 (en) * 2016-02-08 2019-08-14 Leibniz-Institut für Astrophysik Potsdam (AIP) Optical imaging spectroscopy of large-area samples
CN110296758A (en) * 2019-05-21 2019-10-01 北京理工大学 Spectrometer and spectral measurement method based on perovskite quantum dot filter coating
CN110458777A (en) * 2019-08-05 2019-11-15 湖南大学 A kind of high spectrum image denoising method, system and medium based on the correction of adaptive order
CN110501072A (en) * 2019-08-26 2019-11-26 北京理工大学 A kind of reconstructing method of the fast illuminated spectrum imaging system based on the constraint of tensor low-rank
CN110926611A (en) * 2019-12-12 2020-03-27 北京理工大学 Noise suppression method applied to compressed sensing spectral imaging system

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102393248A (en) * 2011-10-26 2012-03-28 中国科学院空间科学与应用研究中心 Time-resolved extreme-low-light multispectral imaging system and method
WO2017017684A1 (en) * 2015-07-30 2017-02-02 Technology Innovation Momentum Fund (Israel) Limited Partnership Spectral imaging method and system
EP3203215B1 (en) * 2016-02-08 2019-08-14 Leibniz-Institut für Astrophysik Potsdam (AIP) Optical imaging spectroscopy of large-area samples
US20180045569A1 (en) * 2016-08-12 2018-02-15 Spectral Insights Private Limited Spectral imaging system
CN106768337A (en) * 2017-02-27 2017-05-31 中国科学院上海高等研究院 A kind of phase reconstruction method in two-dimensional Fourier transform electronic spectrum
CN107084789A (en) * 2017-04-01 2017-08-22 上海理工大学 Single pixel detector spectrum reflectivity reconstructing method based on sparse prior
CN107525588A (en) * 2017-08-16 2017-12-29 北京理工大学 A kind of method for fast reconstruction of the double camera spectrum imaging system based on GPU
US20190078937A1 (en) * 2017-09-13 2019-03-14 Qualcomm Incorporated Directional interpolation and cross-band filtering for hyperspectral imaging
CN110095188A (en) * 2019-04-02 2019-08-06 杭州盗火者科技有限公司 A kind of spectrum restoring method, device and computer readable storage medium
CN110296758A (en) * 2019-05-21 2019-10-01 北京理工大学 Spectrometer and spectral measurement method based on perovskite quantum dot filter coating
CN110458777A (en) * 2019-08-05 2019-11-15 湖南大学 A kind of high spectrum image denoising method, system and medium based on the correction of adaptive order
CN110501072A (en) * 2019-08-26 2019-11-26 北京理工大学 A kind of reconstructing method of the fast illuminated spectrum imaging system based on the constraint of tensor low-rank
CN110926611A (en) * 2019-12-12 2020-03-27 北京理工大学 Noise suppression method applied to compressed sensing spectral imaging system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
刘世界 等: ""多帧图像编码孔径光谱成像技术"", 《红外与毫米波学报》 *
王锋 等: ""一种基于压缩传感的超分辨光学三维成像技术"", 《光子学报》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115950837A (en) * 2023-03-10 2023-04-11 湖南师范大学 Snapshot type spectral imaging method, system and medium based on plug-and-play prior

Also Published As

Publication number Publication date
CN111397733B (en) 2021-03-02

Similar Documents

Publication Publication Date Title
CN110501072B (en) Reconstruction method of snapshot type spectral imaging system based on tensor low-rank constraint
CN108133465B (en) Non-convex low-rank relaxation hyperspectral image recovery method based on spatial spectrum weighted TV
CN111369487B (en) Hyperspectral and multispectral image fusion method, system and medium
CN109697697B (en) Reconstruction method of spectral imaging system based on optimization heuristic neural network
Chen et al. Learning memory augmented cascading network for compressed sensing of images
CN110113607B (en) Compressed sensing video reconstruction method based on local and non-local constraints
CN111028172A (en) Hyperspectral image denoising method based on non-convex low-rank matrix approximation without parameters
Ni et al. Example-driven manifold priors for image deconvolution
CN110378975B (en) Compressed coding aperture imaging method and system based on deep neural network
CN103929649B (en) Reconstruction method for video compressed sensing
CN109887050B (en) Coded aperture spectral imaging method based on adaptive dictionary learning
CN115950837B (en) Snapshot type spectrum imaging method, system and medium based on plug-and-play priori
CN109087262B (en) Multi-view spectral image reconstruction method and storage medium
CN108288250A (en) Infrared remote sensing image super-resolution rebuilding method based on TGV regularizations
CN116579966B (en) Image reconstruction method and device, electronic equipment and storage medium
CN111025385A (en) Seismic data reconstruction method based on low rank and sparse constraint
Yang et al. Reconstruction of structurally-incomplete matrices with reweighted low-rank and sparsity priors
CN113222812A (en) Image reconstruction method based on information flow reinforced deep expansion network
CN111397733B (en) Single/multi-frame snapshot type spectral imaging method, system and medium
CN112989593B (en) High-spectrum low-rank tensor fusion calculation imaging method based on double cameras
CN107154061B (en) Regularized decoding method for block compressed sensing
Yuan et al. Compressive sensing via low-rank Gaussian mixture models
CN108460810A (en) A kind of parallel MR image method for fast reconstruction of full variation
CN108288295A (en) The method for fast reconstruction and system of infrared small target image based on structural information
CN110926611A (en) Noise suppression method applied to compressed sensing spectral imaging system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
EE01 Entry into force of recordation of patent licensing contract

Application publication date: 20200710

Assignee: HUNAN G.MAX INTELLIGENT TECHNOLOGY CO.,LTD.

Assignor: HUNAN University

Contract record no.: X2023980042742

Denomination of invention: A single/multi frame snapshot spectral imaging method, system, and medium

Granted publication date: 20210302

License type: Common License

Record date: 20230927

EE01 Entry into force of recordation of patent licensing contract