CN111192196A - Method for improving real-time resolution of hyperspectral image of push-broom spectrometer - Google Patents

Method for improving real-time resolution of hyperspectral image of push-broom spectrometer Download PDF

Info

Publication number
CN111192196A
CN111192196A CN201911333614.0A CN201911333614A CN111192196A CN 111192196 A CN111192196 A CN 111192196A CN 201911333614 A CN201911333614 A CN 201911333614A CN 111192196 A CN111192196 A CN 111192196A
Authority
CN
China
Prior art keywords
matrix
image
sliding window
size
online
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911333614.0A
Other languages
Chinese (zh)
Inventor
宋滢滢
代超
何帆
周振
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Power Health Cloud Technology Co ltd
Original Assignee
China Power Health Cloud Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Power Health Cloud Technology Co ltd filed Critical China Power Health Cloud Technology Co ltd
Priority to CN201911333614.0A priority Critical patent/CN111192196A/en
Publication of CN111192196A publication Critical patent/CN111192196A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4046—Scaling the whole image or part thereof using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution

Abstract

The invention discloses a method for improving real-time resolution of a hyperspectral image of a push-broom spectrometer, which relates to the technical field of tumor detection.

Description

Method for improving real-time resolution of hyperspectral image of push-broom spectrometer
Technical Field
The invention relates to the technical field of tumor detection, in particular to a method for improving the real-time resolution of a hyperspectral image of a push-broom spectrometer.
Background
The hyperspectral imaging can simultaneously obtain a plurality of images in the same scene under different spectrum wave band ranges. Compared with the traditional imaging mode, the hyperspectral image contains abundant frequency spectrum information and is widely applied to the fields of satellite remote sensing, agricultural geology general survey, medical imaging, environment monitoring and the like. However, due to the limitations of imaging sensor technology, hyperspectral imaging often comes at the cost of spatial resolution to obtain richer spectral information.
A pair of hyperspectral images are images containing three dimensions, including two spatial dimensions and one spectral dimension, and due to the fact that spectrums of different chemical components are obviously different, different components can be distinguished through analysis of the hyperspectral images, and therefore the technology is applied to medicine to distinguish normal tissues from pathological tissues.
At present, in the field of medical imaging, because the resolution of equipment is limited, imaging with higher spatial resolution is required to be obtained, an offline deconvolution algorithm is generally used for improving the spatial resolution of an offline hyperspectral image, the existing deconvolution algorithm mainly aims at a whole image, if efficient real-time tumor edge resolution is required in a surgical operation, a push-broom spectrometer is required to be used for irradiating a tumor, the push-broom spectrometer is an online hyperspectral imaging system, a camera of the push-broom spectrometer is pushed and swept in one direction in a tissue area to be irradiated, each time point returns a two-dimensional matrix comprising a spatial dimension and a spectral dimension, but in the prior art, deconvolution processing can be carried out on the image to improve the resolution by waiting for the whole image to be formed to obtain the whole offline hyperspectral image, so that the requirement on the storage of the equipment is higher, the storage space of the equipment is required to be large, in addition, the algorithm adopted by the prior art is more time-consuming and lower in efficiency.
Disclosure of Invention
The invention aims to: in order to solve the problem that in the prior art, deconvolution processing can be carried out on an image only by waiting for the whole image to be formed to obtain a whole off-line hyperspectral image, and the memory and the time of equipment are consumed, the invention provides a method for improving the real-time resolution of a hyperspectral image of a push-broom spectrometer.
The invention specifically adopts the following technical scheme for realizing the purpose:
a method for improving the real-time resolution of a hyperspectral image of a push-broom spectrometer comprises the following steps:
irradiating a tissue area containing a tumor by using a push-broom spectrometer according to each wavelength in a set wavelength range to obtain an observation image at a corresponding moment, and expressing the observation image as a combination of a space blurred image and noise interference based on an offline convolution model to obtain an offline observation image convolution model;
converting an offline observation image convolution model into an online convolution model, wherein the online convolution model comprises an online observation image which corresponds to each wavelength and comprises an online space blurred image and online noise interference, and the online observation image is formed by each row of elements of a two-dimensional matrix with space dimensionality and spectrum dimensionality obtained by observation of a push-broom spectrometer;
vectorizing a two-dimensional matrix at each moment, constructing a sliding window, splicing a plurality of vectorized two-dimensional matrixes in the sliding window, then constructing a cost function, carrying out online deconvolution processing on the spliced vectorized two-dimensional matrixes based on the cost function to optimize the resolution, calculating the cost function by using a gradient descent method, wherein the corresponding vectorized two-dimensional matrix is the optimal vectorized two-dimensional matrix at the moment when the value of the cost function is minimum, and the image corresponding to the optimal vectorized two-dimensional matrix is the hyperspectral image with the resolution optimized at the corresponding moment.
Further, the observation image is an observation two-dimensional matrix having a spatial dimension and a spectral dimension obtained by a push-broom spectrometer pushing a tissue region containing a tumor in one direction at each moment.
Further, the calculation formula of the convolution model of the offline observation image is as follows:
Yp＝H*p*Xp+Ep
wherein, YpAn observation image representing a wavelength p, H*pFor a convolution kernel, XpFor the hyperspectral image to be solved of the wavelength p, EpIs noise interference.
Further, the convolution kernel H*pIs a two-dimensional Gaussian matrix expressed as
Figure BDA0002330333790000021
Where M is the number of rows of the gaussian matrix and L is the number of columns of the gaussian matrix, then the ith column vector is represented as:
Figure BDA0002330333790000022
further, if the push-broom spectrometer has delay in obtaining the observation image, the calculation formula of the online convolution model is:
Figure BDA0002330333790000023
wherein the content of the first and second substances,
Figure BDA0002330333790000024
in the form of an online convolution of the observed image at the delayed time k,
Figure BDA0002330333790000025
in the form of an online convolution of the observed image at the actual instant,
Figure BDA0002330333790000026
is composed of
Figure BDA0002330333790000027
The first row of the normal diagonal matrix is
Figure BDA0002330333790000028
The first column is
Figure BDA0002330333790000029
Figure BDA00023303337900000210
In order to wait for the hyperspectral image to be solved online,
Figure BDA00023303337900000211
is an online noise disturbance.
Further, the constructing of the sliding window specifically includes: and constructing a sliding window Q with the size equal to or larger than the convolution kernel, increasing the length of L-1 on the basis of the length of the sliding window Q to form a new sliding window, deconvoluting the spliced vectorized two-dimensional matrix covered by the new sliding window at each moment to obtain an optimal result after vectorization, and moving the new sliding window to the next moment for calculation after the calculation of the current moment is completed.
Further, the cost function includes a fitting term and three regular terms, and the three regular terms are a temporal regular term, a spatial regular term and a spectral regular term, respectively.
Further, the spatial regularization term includes a first order filter applied to the spatial dimension
Figure BDA00023303337900000212
IPAn identity matrix representing the size of P x P,
Figure BDA0002330333790000031
represents the kronecker product, TNRepresents a constant diagonal matrix of size (N-1) xN, the first row of the constant diagonal matrix being [1, -1,0,.. 0, 0]The first column is [1, 0., 0 ]]；
The spectral regularization term includes a first order filter D applied to the spectral dimensionλ，
Figure BDA0002330333790000032
Figure BDA0002330333790000033
Weight for each spectrum, diag (c)1,...,cP-1) Means to convert c1,...,cP-1Vector transformation into diagonal matrix, TpRepresents a constant diagonal matrix of size (P-1) xP, the first row of the constant diagonal matrix being [1, -1,0,. multidot.0, 0]The first column is [1, 0., 0 ]]；INAn identity matrix of size N × N is represented.
Further, the cost function is calculated as:
Figure BDA0002330333790000034
wherein the content of the first and second substances,
Figure BDA0002330333790000035
representing the decrease of the online spatially blurred image to be updated within the sliding window Q from time k to time k-Q +1,
Figure BDA0002330333790000036
representing the time instant covered by the first convolution kernel within the sliding window Q,
Figure BDA0002330333790000037
in order to fit the terms to each other,
Figure BDA0002330333790000038
a processor of the expected value is represented,
Figure BDA0002330333790000039
for the time regularization term, L1 regularization is performed on all vectorized two-dimensional matrices in the time dimension within the sliding window Q,
Figure BDA00023303337900000310
for the spatial regularization term, L1 regularization is performed on all vectorized two-dimensional matrices in the spatial dimension in the sliding window Q,
Figure BDA00023303337900000311
for spectral regularization term, L2 regularization is performed on all vectorized two-dimensional matrices in the spectral dimension within the sliding window Q, ηt、ηs、ηλThe weight parameters are respectively corresponding to the regular terms, and the images are respectively spliced into a corresponding vector in the cost function, so that the expression of the wavelength p, H, is omittedlFor each wavelength
Figure BDA00023303337900000313
And forming a corresponding block diagonal matrix.
Further, the gradient descent method calculates a cost function, specifically:
firstly, calculating to obtain a sub-gradient of the cost function, wherein the calculation formula is as follows:
Figure BDA00023303337900000312
wherein x'kFor all vectors x in the new sliding windowk，...，xk-Q+1，xk-Q，...，xk-Q-L+2The spliced whole vector, phi and G matrix is constructed in HlDue to x'kIs a mosaic vector comprising Q + L-1 time vector images, so phi and G are both H to be associated with a time instantlApplied to the corresponding vector image, phi and G are both (Q + L-1) PN x (Q + L-1) PN;
wherein the content of the first and second substances,
Figure BDA0002330333790000041
0QPN×(L-1)PNzero matrix, 0, representing QPN x (L-1) PN(L-1)PN×(Q+L-1)PNRepresents a zero matrix of size (L-1) PN × (Q + L-1) PN, when L>When L is, Hl＝0PN×PN，0PN×PNRepresenting a zero matrix with the size of PN multiplied by PN;
g matrix is
Figure BDA0002330333790000042
Λt、Λs、ΛλThe three matrixes respectively correspond to a time regular term, a space regular term and a spectrum regular term,
Figure BDA0002330333790000043
wherein IQRepresents a constant diagonal matrix of size (Q-1) xQ, the first row of the constant diagonal matrix being [1, -1,0,.. 0, 0]The first column is [1, 0., 0 ]]，INPUnit matrix of size NP × NP, 0(Q-1)NP×(L-1)NPA zero matrix representing a size of (Q-1) NP × (L-1) NP;
Figure BDA0002330333790000044
wherein IQAn identity matrix of Q × Q size, 0QP(N-1)×(L-1)PNA zero matrix representing QP (N-1) × (L-1) PN;
Figure BDA0002330333790000045
wherein IQAn identity matrix of Q × Q size, 0Q(P-1)N×(L-1)PNA zero matrix representing a size of Q (P-1) Nx (L-1) PN;
then, a regularized sliding window least mean square model is used for calculation, and the calculation formula is as follows:
Figure BDA0002330333790000046
where ρ ist＝μηt/2，ρs＝μηs/2，
Figure BDA0002330333790000047
Optimizing the hyperspectral image in the next sliding window for the resolution at the moment k +1, wherein mu is a learning rate parameter when the gradient is reduced, and the omega matrix ensures
Figure BDA0002330333790000048
The instantaneous value is no longer changed and,
Figure BDA0002330333790000049
a final value of
Figure BDA0002330333790000051
The first vector of the Q part of the middle sliding window, namely the length of Q is pushed forward from the moment of k +1, namely the k-Q +2 vectors, and the vector is a two-dimensional matrix of the optimal vectorization:
Figure BDA0002330333790000052
wherein
Figure BDA0002330333790000053
Wherein 0PN×(Q-1)PNZero matrix representing PN x (Q-1) PN in size，IPNIdentity matrix representing the size of PN × PN, 0PN×(L-1)PNDenotes a zero matrix of size PN x (L-1) PN.
The invention has the following beneficial effects:
1. according to the invention, a real-time deconvolution algorithm is adopted to perform real-time online deconvolution processing on the two-dimensional matrix containing the tumor tissue and the normal tissue obtained by irradiation of the push-broom spectrometer, so that the image resolution can be effectively improved, the lesion tissue and the normal tissue can be accurately distinguished through the hyperspectral image, a clear boundary line is obtained, and the definition of the tumor hyperspectral image edge is improved.
2. The method has the advantages that the real-time online processing is realized, the deconvolution processing can be carried out on the image without waiting for the whole image to be formed to obtain the whole offline hyperspectral image, and compared with the prior art, the method has low requirements on equipment memory and faster response.
Drawings
FIG. 1 is a schematic diagram of a process for using a push-broom spectrometer according to an embodiment of the present invention.
FIG. 2 is a schematic diagram of a two-dimensional matrix obtained by irradiation with the push-broom spectrometer of FIG. 1.
FIG. 3 is a schematic diagram of an on-line deconvolution process according to an embodiment of the present invention.
Detailed Description
For a better understanding of the present invention by those skilled in the art, the present invention will be described in further detail below with reference to the accompanying drawings and the following examples.
Example 1
The embodiment provides a method for improving the real-time resolution of a hyperspectral image of a push-broom spectrometer, which applies a real-time deconvolution technology of a hyperspectral image, is applied to equipment for detecting tumors in real time by using push-broom hyperspectral imaging in a surgical operation, and if the push-broom spectrometer is used, the spatial resolution is improved, and high-precision tumor boundary identification is realized, and the method specifically comprises the following steps:
s1: utilizing a push-broom spectrometer to irradiate a tissue area containing a tumor according to each wavelength in a set wavelength range to obtain an observation image at a corresponding moment, wherein the observation image is an observation two-dimensional matrix with a space dimension and a spectrum dimension, which is obtained by the push-broom spectrometer pushing the tissue area containing the tumor at each moment according to a direction, as shown in fig. 1 and 2; expressing the observation image as a combination of a space blurred image and noise interference based on an offline convolution model to obtain an offline observation image convolution model, wherein the offline observation image convolution model has a calculation formula as follows:
Yp＝H*p*Xp+Ep
wherein, YpAn observation image representing a wavelength p, H*pFor a convolution kernel, XpFor the hyperspectral image to be solved of the wavelength p, EpIs noise interference; convolution kernel H*pIs a two-dimensional Gaussian matrix expressed as
Figure BDA0002330333790000061
Where M is the number of rows of the gaussian matrix and L is the number of columns of the gaussian matrix, then the ith column vector is represented as:
Figure BDA0002330333790000062
s2: converting an offline observation image convolution model into an online convolution model, wherein the online convolution model comprises an online observation image which corresponds to each wavelength and comprises an online space blurred image and online noise interference, and the online observation image is formed by each row of elements of a two-dimensional matrix with space dimensionality and spectrum dimensionality obtained by observation of a push-broom spectrometer;
if the push-broom spectrometer has delay in obtaining the observation image, the calculation formula of the online convolution model is as follows:
Figure BDA0002330333790000063
wherein the content of the first and second substances,
Figure BDA0002330333790000064
in the form of an online convolution of the observed image at the delayed time k,
Figure BDA0002330333790000065
in the form of an online convolution of the observed image at the actual instant,
Figure BDA0002330333790000066
is composed of
Figure BDA0002330333790000067
The first row of the normal diagonal matrix is
Figure BDA0002330333790000068
The first column is
Figure BDA0002330333790000069
Figure BDA00023303337900000610
In order to wait for the hyperspectral image to be solved online,
Figure BDA00023303337900000611
for online noise interference;
s3: vectorizing a two-dimensional matrix at each moment, constructing a sliding window, splicing a plurality of vectorized two-dimensional matrixes in the sliding window, then constructing a cost function, carrying out online deconvolution processing on the spliced vectorized two-dimensional matrixes based on the cost function to optimize the resolution, calculating the cost function by using a gradient descent method, wherein the corresponding vectorized two-dimensional matrix is the optimal vectorized two-dimensional matrix at the moment when the value of the cost function is minimum, and the image corresponding to the optimal vectorized two-dimensional matrix is the hyperspectral image with the resolution optimized at the corresponding moment;
in this embodiment, the constructing the sliding window specifically includes: constructing a sliding window Q with the size equal to or larger than the convolution kernel, increasing the length of L-1 on the basis of the length of the sliding window Q to form a new sliding window, deconvoluting the spliced vectorized two-dimensional matrix covered by the new sliding window at each moment to obtain an optimal result after vectorization, and moving the new sliding window to the next moment for calculation after the calculation of the current moment is completed;
the cost function comprises a fitting term and three regular terms, wherein the three regular terms are a time regular term, a space regular term and a spectrum regular term respectively;
the spatial regularization term includes a first order filter applied to the spatial dimension
Figure BDA00023303337900000612
IPAn identity matrix representing the size of P x P,
Figure BDA00023303337900000613
represents the kronecker product, TNRepresents a constant diagonal matrix of size (N-1) xN, the first row of the constant diagonal matrix being [1, -1,0,.. 0, 0]The first column is [1, 0., 0 ]]；
The spectral regularization term includes a first order filter D applied to the spectral dimensionλ，
Figure BDA00023303337900000614
c1,...,cP-1Weight for each spectrum, diag (c)1,...,cP-1) Means to convert c1,...,cP-1Vector transformation into diagonal matrix, TpRepresents a constant diagonal matrix of size (P-1) xP, the first row of the constant diagonal matrix being [1, -1,0,. multidot.0, 0]The first column is [1, 0., 0 ]]；INAn identity matrix representing a size of N × N;
therefore, as shown in fig. 3, the cost function constructed in this embodiment is calculated by the following formula:
Figure BDA0002330333790000071
wherein the content of the first and second substances,
Figure BDA0002330333790000072
representing the decrease of the online spatially blurred image to be updated within the sliding window Q from time k to time k-Q +1,
Figure BDA0002330333790000073
indicating the first in the sliding window QThe time instants at which the convolution kernel covers,
Figure BDA0002330333790000074
in order to fit the terms to each other,
Figure BDA0002330333790000075
a processor of the expected value is represented,
Figure BDA0002330333790000076
for the time regularization term, L1 regularization is performed on all vectorized two-dimensional matrices in the time dimension within the sliding window Q,
Figure BDA0002330333790000077
for the spatial regularization term, L1 regularization is performed on all vectorized two-dimensional matrices in the spatial dimension in the sliding window Q,
Figure BDA0002330333790000078
for spectral regularization term, L2 regularization is performed on all vectorized two-dimensional matrices in the spectral dimension within the sliding window Q, ηt、ηs、ηλThe weight parameters are respectively corresponding to the regular terms, and the images are respectively spliced into a corresponding vector in the cost function, so that the expression of the wavelength p, H, is omittedlFor each wavelength
Figure BDA00023303337900000711
Forming a corresponding block diagonal matrix;
then, calculating the cost function by using a gradient descent method, specifically:
firstly, calculating to obtain a sub-gradient of the cost function, wherein the calculation formula is as follows:
Figure BDA0002330333790000079
wherein x'kFor all vectors x in the new sliding windowk，...，xk-Q+1，xk-Q，...，xk-Q-L+2A whole vector spliced by phi and G matrixes is constructed inHlDue to x'kIs a mosaic vector comprising Q + L-1 time vector images, so phi and G are both H to be associated with a time instantlApplied to the corresponding vector image, phi and G are both (Q + L-1) PN x (Q + L-1) PN;
wherein the content of the first and second substances,
Figure BDA00023303337900000710
0QPN×(L-1)PNzero matrix, 0, representing QPN x (L-1) PN(L-1)PN×(Q+L-1)PNRepresents a zero matrix of size (L-1) PN × (Q + L-1) PN, when L>When L is, Hl＝0PN×PN，0PN×PNRepresenting a zero matrix with the size of PN multiplied by PN;
g matrix is
Figure BDA0002330333790000081
Λt、Λs、ΛλThe three matrixes respectively correspond to a time regular term, a space regular term and a spectrum regular term,
Figure BDA0002330333790000082
wherein T isQRepresents a constant diagonal matrix of size (Q-1) xQ, the first row of the constant diagonal matrix being [1, -1,0,.. 0, 0]The first column is [1, 0., 0 ]]，INPUnit matrix of size NP × NP, 0(Q-1)NP×(L-1)NPA zero matrix representing a size of (Q-1) NP × (L-1) NP;
Figure BDA0002330333790000083
wherein IQAn identity matrix of Q × Q size, 0QP(N-1)×(L-1)PNA zero matrix representing QP (N-1) × (L-1) PN;
Figure BDA0002330333790000084
wherein IQAn identity matrix of Q × Q size, 0Q(P-1)N×(L-1)PNA zero matrix representing a size of Q (P-1) Nx (L-1) PN;
and finally, calculating by using a regularized sliding window least mean square model, wherein the calculation formula is as follows:
Figure BDA0002330333790000085
where ρ ist＝μηt/2，ρs＝μηs/2，
Figure BDA0002330333790000086
Optimizing the hyperspectral image in the next sliding window for the resolution at the moment k +1, wherein mu is a learning rate parameter when the gradient is reduced, and the omega matrix ensures
Figure BDA0002330333790000087
The instantaneous value is no longer changed and,
Figure BDA0002330333790000088
a final value of
Figure BDA0002330333790000089
The first vector of the Q part of the middle sliding window, namely the length of Q is pushed forward from the moment of k +1, namely the k-Q +2 vectors, and the vector is a two-dimensional matrix of the optimal vectorization:
Figure BDA00023303337900000810
wherein
Figure BDA0002330333790000091
Wherein 0PN×(Q-1)PNDenotes a zero matrix of size PN x (Q-1) PN, IPNIdentity matrix representing the size of PN × PN, 0PN×(L-1)PNTwo-dimensional matrix representing zero matrix of size PN x (L-1) PN, vectorizing the optimum
Figure BDA0002330333790000092
Recombining to construct a two-dimensional matrix, wherein the image corresponding to the two-dimensional matrix obtained by recombination is a pairIn response to the hyperspectral image with the optimized resolution, the real-time deconvolution algorithm is adopted to perform real-time online deconvolution processing on the two-dimensional matrix containing the tumor tissue and the normal tissue, which is obtained by irradiation of the push-broom spectrometer, so that the image resolution can be effectively improved, lesion tissues and normal tissues can be accurately distinguished through the hyperspectral image, a definite boundary line is obtained, and the definition of the tumor hyperspectral image edge is improved.
The above description is only a preferred embodiment of the present invention, and not intended to limit the present invention, the scope of the present invention is defined by the appended claims, and all structural changes that can be made by using the contents of the description and the drawings of the present invention are intended to be embraced therein.

Claims (10)

1. A method for improving the real-time resolution of a hyperspectral image of a push-broom spectrometer is characterized by comprising the following steps:
irradiating a tissue area containing a tumor by using a push-broom spectrometer according to each wavelength in a set wavelength range to obtain an observation image at a corresponding moment, and expressing the observation image as a combination of a space blurred image and noise interference based on an offline convolution model to obtain an offline observation image convolution model;
converting an offline observation image convolution model into an online convolution model, wherein the online convolution model comprises an online observation image which corresponds to each wavelength and comprises an online space blurred image and online noise interference, and the online observation image is formed by each row of elements of a two-dimensional matrix with space dimensionality and spectrum dimensionality obtained by observation of a push-broom spectrometer;
vectorizing a two-dimensional matrix at each moment, constructing a sliding window, splicing a plurality of vectorized two-dimensional matrixes in the sliding window, then constructing a cost function, carrying out online deconvolution processing on the spliced vectorized two-dimensional matrixes based on the cost function to optimize the resolution, calculating the cost function by using a gradient descent method, wherein the corresponding vectorized two-dimensional matrix is the optimal vectorized two-dimensional matrix at the moment when the value of the cost function is minimum, and the image corresponding to the optimal vectorized two-dimensional matrix is the hyperspectral image with the resolution optimized at the corresponding moment.
2. The method of claim 1, wherein the observation image is an observation two-dimensional matrix having a spatial dimension and a spectral dimension obtained by the push-broom spectrometer pushing a tissue region containing a tumor in one direction at each moment.
3. The method for improving the real-time resolution of the hyperspectral image of the push-broom spectrometer according to claim 1, wherein the calculation formula of the convolution model of the offline observed image is as follows:
Yp＝H*p*Xp+Ep
wherein, YpAn observation image representing a wavelength p, H*pFor a convolution kernel, XpFor the hyperspectral image to be solved of the wavelength p, EpIs noise interference.
4. The method for improving the real-time resolution of the hyperspectral image of the push-broom spectrometer of claim 3, wherein the convolution kernel H is*pIs a two-dimensional Gaussian matrix expressed as
Figure FDA0002330333780000011
Where M is the number of rows of the gaussian matrix and L is the number of columns of the gaussian matrix, then the ith column vector is represented as:
5. the method of claim 4, wherein if the push-broom spectrometer has a delay in obtaining the observation image, the calculation formula of the online convolution model is:
Figure FDA0002330333780000013
wherein the content of the first and second substances,
Figure FDA0002330333780000014
in the form of an online convolution of the observed image at the delayed time k,
Figure FDA0002330333780000015
in the form of an online convolution of the observed image at the actual instant,
Figure FDA0002330333780000016
is composed of
Figure FDA0002330333780000017
The first row of the normal diagonal matrix is
Figure FDA0002330333780000018
The first column is
Figure FDA0002330333780000021
Figure FDA0002330333780000022
In order to wait for the hyperspectral image to be solved online,
Figure FDA0002330333780000023
is an online noise disturbance.
6. The method for improving the real-time resolution of the hyperspectral image of the push-broom spectrometer according to claim 5, wherein the constructing the sliding window specifically comprises: and constructing a sliding window Q with the size equal to or larger than the convolution kernel, increasing the length of L-1 on the basis of the length of the sliding window Q to form a new sliding window, deconvoluting the spliced vectorized two-dimensional matrix covered by the new sliding window at each moment to obtain an optimal result after vectorization, and moving the new sliding window to the next moment for calculation after the calculation of the current moment is completed.
7. The method of claim 6, wherein the cost function comprises a fitting term and three regular terms, wherein the three regular terms are a temporal regular term, a spatial regular term and a spectral regular term.
8. The method of claim 7, wherein the spatial regularization term comprises a first order filter D applied to the spatial dimensionS，
Figure FDA0002330333780000024
IPAn identity matrix representing the size of P x P,
Figure FDA0002330333780000025
represents the kronecker product, TNRepresents a constant diagonal matrix of size (N-1) xN, the first row of the constant diagonal matrix being [1, -1,0,.. 0, 0]The first column is [1, 0., 0 ]]；
The spectral regularization term includes a first order filter D applied to the spectral dimensionλ，
Figure FDA0002330333780000026
c1,...,cP-1Weight for each spectrum, diag (c)1,...,cP-1) Means to convert c1,...,cP-1Vector transformation into diagonal matrix, TpRepresents a constant diagonal matrix of size (P-1) xP, the first row of the constant diagonal matrix being [1, -1,0,. multidot.0, 0]The first column is [1, 0., 0 ]]；INAn identity matrix of size N × N is represented.
9. The method of claim 8, wherein the cost function is calculated by:
Figure FDA0002330333780000027
wherein the content of the first and second substances,
Figure FDA0002330333780000028
representing the decrease of the online spatially blurred image to be updated within the sliding window Q from time k to time k-Q +1,
Figure FDA0002330333780000029
representing the time instant covered by the first convolution kernel within the sliding window Q,
Figure FDA00023303337800000210
in order to fit the terms to each other,
Figure FDA00023303337800000211
a processor of the expected value is represented,
Figure FDA00023303337800000212
for the time regularization term, L1 regularization is performed on all vectorized two-dimensional matrices in the time dimension within the sliding window Q,
Figure FDA00023303337800000213
for the spatial regularization term, L1 regularization is performed on all vectorized two-dimensional matrices in the spatial dimension in the sliding window Q,
Figure FDA0002330333780000031
for spectral regularization term, L2 regularization is performed on all vectorized two-dimensional matrices in the spectral dimension within the sliding window Q, ηt、ηs、ηλThe weight parameters are respectively corresponding to the regular terms, and the images are respectively spliced into a corresponding vector in the cost function, so that the expression of the wavelength p, H, is omittedlFor each wavelength
Figure FDA0002330333780000032
And forming a corresponding block diagonal matrix.
10. The method for improving the real-time resolution of the hyperspectral image of the push-broom spectrometer according to claim 9, wherein the gradient descent method is used for calculating a cost function, and specifically comprises:
firstly, calculating to obtain a sub-gradient of the cost function, wherein the calculation formula is as follows:
Figure FDA0002330333780000033
wherein x'kFor all vectors x in the new sliding windowk，...，xk-Q+1，xk-Q，...，xk-Q-L+2The spliced whole vector, phi and G matrix is constructed in HlDue to x'kIs a mosaic vector comprising Q + L-1 time vector images, so phi and G are both K to correspond to a time instantlApplied to the corresponding vector image, phi and G are both (Q + L-1) PN x (Q + L-1) PN;
wherein the content of the first and second substances,
Figure FDA0002330333780000034
0QPN×(L-1)PNzero matrix, 0, representing QPN x (L-1) PN(L-1)PN×(Q+L-1)PNRepresents a zero matrix of size (L-1) PN × (Q + L-1) PN, when L>When L is, Hl＝0PN×PN，0PN×PNRepresenting a zero matrix with the size of PN multiplied by PN;
g matrix is
Figure FDA0002330333780000035
Λt、Λs、ΛλThe three matrixes respectively correspond to a time regular term, a space regular term and a spectrum regular term,
Figure FDA0002330333780000036
wherein T isQRepresents a constant diagonal matrix of size (Q-1) xQ, the first row of the constant diagonal matrix being [1, -1,0,.. 0, 0]The first column is [1, 0., 0 ]]，INPUnit matrix of size NP × NP, 0(Q-1)NP×(L-1)NPA zero matrix representing a size of (Q-1) NP × (L-1) NP;
Figure FDA0002330333780000037
wherein IQAn identity matrix of Q × Q size, 0QP(N-1)×(L-1)PNA zero matrix representing QP (N-1) × (L-1) PN;
Figure FDA0002330333780000038
wherein IQAn identity matrix of Q × Q size, 0Q(P-1)N×(L-1)PNA zero matrix representing a size of Q (P-1) Nx (L-1) PN;
then, a regularized sliding window least mean square model is used for calculation, and the calculation formula is as follows:
Figure FDA0002330333780000041
where ρ ist＝μηt/2，ρs＝μηs/2，
Figure FDA0002330333780000042
Optimizing the hyperspectral image in the next sliding window for the resolution at the moment k +1, wherein mu is a learning rate parameter when the gradient is reduced, and the omega matrix ensures
Figure FDA0002330333780000043
The instantaneous value is no longer changed and,
Figure FDA0002330333780000044
a final value of
Figure FDA0002330333780000045
The first vector of the Q part of the middle sliding window, namely the length of Q is pushed forward from the moment of k +1, namely the k-Q +2 vectors, and the vector is a two-dimensional matrix of the optimal vectorization:
Figure FDA0002330333780000046
wherein
Figure FDA0002330333780000047
Wherein 0PN×(Q-1)PNDenotes a zero matrix of size PN x (Q-1) PN, IPNIdentity matrix representing the size of PN × PN, 0PN×(L-1)PNDenotes a zero matrix of size PN x (L-1) PN.
CN201911333614.0A 2019-12-23 2019-12-23 Method for improving real-time resolution of hyperspectral image of push-broom spectrometer Pending CN111192196A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911333614.0A CN111192196A (en) 2019-12-23 2019-12-23 Method for improving real-time resolution of hyperspectral image of push-broom spectrometer

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911333614.0A CN111192196A (en) 2019-12-23 2019-12-23 Method for improving real-time resolution of hyperspectral image of push-broom spectrometer

Publications (1)

Publication Number Publication Date
CN111192196A true CN111192196A (en) 2020-05-22

Family

ID=70707418

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911333614.0A Pending CN111192196A (en) 2019-12-23 2019-12-23 Method for improving real-time resolution of hyperspectral image of push-broom spectrometer

Country Status (1)

Country Link
CN (1) CN111192196A (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5469060A (en) * 1993-08-19 1995-11-21 Meyerand; Mary E. Time encoded magnetic resonance imaging
US6704692B1 (en) * 1999-10-25 2004-03-09 The Boeing Company Method and system for tracking multiple objects
CN104794681A (en) * 2015-04-28 2015-07-22 西安电子科技大学 Remote sensing image fusion method based on multi-redundancy dictionary and sparse reconstruction
US20150364305A1 (en) * 2014-06-13 2015-12-17 Agilent Technologies, Inc. Data processing for multiplexed spectrometry
US20160371563A1 (en) * 2015-06-22 2016-12-22 The Johns Hopkins University System and method for structured low-rank matrix factorization: optimality, algorithm, and applications to image processing
US20170284867A1 (en) * 2014-08-29 2017-10-05 Commonwealth Scientific And Industrial Research Organisation Imaging method and apparatus
CN108257107A (en) * 2018-01-11 2018-07-06 天津大学 Image Fusion based on local low-rank matrix and spectral resolution
CN110119780A (en) * 2019-05-10 2019-08-13 西北工业大学 Based on the hyperspectral image super-resolution reconstruction method for generating confrontation network
CN110135379A (en) * 2019-05-21 2019-08-16 中电健康云科技有限公司 Tongue picture dividing method and device

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5469060A (en) * 1993-08-19 1995-11-21 Meyerand; Mary E. Time encoded magnetic resonance imaging
US6704692B1 (en) * 1999-10-25 2004-03-09 The Boeing Company Method and system for tracking multiple objects
US20150364305A1 (en) * 2014-06-13 2015-12-17 Agilent Technologies, Inc. Data processing for multiplexed spectrometry
US20170284867A1 (en) * 2014-08-29 2017-10-05 Commonwealth Scientific And Industrial Research Organisation Imaging method and apparatus
CN104794681A (en) * 2015-04-28 2015-07-22 西安电子科技大学 Remote sensing image fusion method based on multi-redundancy dictionary and sparse reconstruction
US20160371563A1 (en) * 2015-06-22 2016-12-22 The Johns Hopkins University System and method for structured low-rank matrix factorization: optimality, algorithm, and applications to image processing
CN108257107A (en) * 2018-01-11 2018-07-06 天津大学 Image Fusion based on local low-rank matrix and spectral resolution
CN110119780A (en) * 2019-05-10 2019-08-13 西北工业大学 Based on the hyperspectral image super-resolution reconstruction method for generating confrontation network
CN110135379A (en) * 2019-05-21 2019-08-16 中电健康云科技有限公司 Tongue picture dividing method and device

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
ARABLOUEI, REZA；DE HOOG, FRANK: "Hyperspectral Image Recovery via Hybrid Regularization", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》 *
SONG, YINGYING；DJERMOUNE, EL-HADI；CHEN, JIE: "Online Deconvolution for Industrial Hyperspectral Imaging Systems", 《SIAM JOURNAL ON IMAGING SCIENCES》 *
李晶: "基于非线性混合模型的高光谱图像解混技术研究", 《万方》 *
梁瑞宇,邹采荣,王青云,张学武: "基于自适应次梯度投影算法的压缩感知信号重构", 《信号处理》 *
田进,陈秀宏,傅俊鹏,徐德荣: "基于重叠稀疏组深度信念网络的图像识别", 《计算机工程与科学》 *
韩昌: "基于联合稀疏分析模型的遥感图像融合研究", 《湖北:华中科技大学》 *

Similar Documents

Publication Publication Date Title
CN110458777B (en) Hyperspectral image denoising method, system and medium based on adaptive rank correction
CN109145992B (en) Hyperspectral image classification method for cooperatively generating countermeasure network and spatial spectrum combination
CN108389188B (en) Sparse hyperspectral abnormal target detection method
Arad et al. Filter selection for hyperspectral estimation
CN110619263B (en) Hyperspectral remote sensing image anomaly detection method based on low-rank joint collaborative representation
Zhuang et al. Hy-demosaicing: Hyperspectral blind reconstruction from spectral subsampling
CN108288256B (en) Multispectral mosaic image restoration method
CN107316309B (en) Hyperspectral image saliency target detection method based on matrix decomposition
CN103886559B (en) Spectrum image processing method
CN106951854A (en) A kind of SAR image internal wave of ocean parameter extracting method
He et al. DsTer: A dense spectral transformer for remote sensing spectral super-resolution
CN112598069B (en) Hyperspectral target tracking method based on feature extraction and weight coefficient parameter updating
CN110554384A (en) imaging method based on microwave signal
CN114529830A (en) Remote sensing image space-time fusion method based on mixed convolution network
Zhu et al. HCNNet: A hybrid convolutional neural network for spatiotemporal image fusion
CN106504208A (en) Based on orderly minima and the high-spectrum image width destriping method of wavelet filtering
CN111192196A (en) Method for improving real-time resolution of hyperspectral image of push-broom spectrometer
CN112598711A (en) Hyperspectral target tracking method based on joint spectrum dimensionality reduction and feature fusion
Liu et al. Super-Resolution Based on Residual Dense Network for Agricultural Image
CN115578262A (en) Polarization image super-resolution reconstruction method based on AFAN model
Ma et al. A snapshot near-infrared hyperspectral demosaicing method with convolutional neural networks in low illumination environment
CN114972075A (en) Hyperspectral image denoising method based on residual learning and mixed domain attention
CN104008532B (en) A kind of infrared image based on prior probability model removes striping methods
Payeur et al. Correlated Read Noise Reduction in Infrared Arrays Using Deep Learning
CN113325364A (en) Space-time joint direction finding method based on data compression

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200522