CN115330901A - Image reconstruction method and device based on compressed sensing network - Google Patents
Image reconstruction method and device based on compressed sensing network Download PDFInfo
- Publication number
- CN115330901A CN115330901A CN202211264253.0A CN202211264253A CN115330901A CN 115330901 A CN115330901 A CN 115330901A CN 202211264253 A CN202211264253 A CN 202211264253A CN 115330901 A CN115330901 A CN 115330901A
- Authority
- CN
- China
- Prior art keywords
- image signal
- sensing network
- compressed sensing
- reconstruction
- reconstructed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 239000011159 matrix material Substances 0.000 claims abstract description 57
- 238000005070 sampling Methods 0.000 claims abstract description 40
- 238000005259 measurement Methods 0.000 claims description 21
- 150000001875 compounds Chemical class 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 8
- 230000020411 cell activation Effects 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 3
- 238000013528 artificial neural network Methods 0.000 abstract description 11
- 230000000306 recurrent effect Effects 0.000 abstract description 10
- 230000006870 function Effects 0.000 description 19
- 230000008569 process Effects 0.000 description 10
- 230000000694 effects Effects 0.000 description 7
- 230000006872 improvement Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 241000282465 Canis Species 0.000 description 1
- 101100365547 Schizosaccharomyces pombe (strain 972 / ATCC 24843) set11 gene Proteins 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/001—Texturing; Colouring; Generation of texture or colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4084—Scaling of whole images or parts thereof, e.g. expanding or contracting in the transform domain, e.g. fast Fourier transform [FFT] domain scaling
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Image Analysis (AREA)
Abstract
The application discloses an image reconstruction method and device based on a compressed sensing network, which are used for sampling an original image signal to obtain a sampled image signal and further initializing to obtain an initial reconstructed image signal; inputting the sampled image signal and the initial reconstructed image signal into a compressed sensing network, carrying out signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain multilayer reconstructed image signals, and optimizing the reconstructed image signals according to the correlation between two adjacent layers of reconstructed image signals; updating network parameters according to the final reconstructed image signal and the original image signal; the image reconstruction is carried out on the image signal to be reconstructed through the trained compressed sensing network, the technical problems that in the prior art, the reconstruction signal is weighted by adopting a recurrent neural network, a weighting matrix is greatly limited, and the stability of the similarity of each column of signals in the reconstruction signal is poor, so that the stability of the reconstruction result is poor are solved.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an image reconstruction method and apparatus based on a compressed sensing network.
Background
Compressed Sensing (CS) aims at accurately reconstructing high-dimensional signals from a small number of measurements by exploiting signal sparsity and structural priors. In order to further improve reconstruction performance, in the prior art, a recurrent neural network is adopted to weight a reconstruction signal, but the method has great limitation on a weighting matrix and needs to be an orthogonal matrix or a diagonal matrix; moreover, the reconstruction method based on the recurrent neural network designs a model by utilizing the similarity of each column of signals in the reconstructed signals, the similarity is unstable, when the picture content is monotonous, for example, only one cup is provided, the similarity between each column of signals is strong, and when the picture content is complex, the similarity between each column of signals is weak, so that the stability of the obtained reconstruction result is poor.
Disclosure of Invention
The application provides an image reconstruction method and device based on a compressed sensing network, which are used for solving the technical problems that in the prior art, a recurrent neural network is adopted to weight a reconstruction signal, a weighting matrix is greatly limited, and the similarity of each column of signals in the reconstruction signal is utilized to design a model, so that the stability of the similarity of each column of signals is poor, and the stability of a reconstruction result is poor.
In view of this, a first aspect of the present application provides an image reconstruction method based on a compressed sensing network, including:
sampling an original image signal to obtain a sampled image signal;
initializing a reconstructed image signal based on the sampling image signal to obtain an initial reconstructed image signal;
inputting the sampled image signal and the initial reconstructed image signal into a compressed sensing network, performing signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain a multilayer reconstructed image signal, and optimizing the reconstructed image signal according to the correlation between two adjacent layers of reconstructed image signals to obtain a final reconstructed image signal;
calculating a loss value according to the final reconstructed image signal and the original image signal, and updating network parameters of the compressive sensing network according to the loss value to obtain a trained compressive sensing network;
and performing image reconstruction on the image signal to be reconstructed through the trained compressed sensing network to obtain a reconstructed image signal.
Optionally, the sampling the original image signal to obtain a sampled image signal includes:
and sampling the original image signal through a measurement matrix to obtain a sampled image signal.
Optionally, initializing a reconstructed image signal based on the sampled image signal to obtain an initial reconstructed image signal includes:
and initializing a reconstructed image signal based on the sampling image signal, the sparse dictionary and the measurement matrix to obtain an initial reconstructed image signal.
Optionally, the compressed sensing network is a multilayer iterative model, and an objective function corresponding to each layer is as follows:
in the formula (I), the compound is shown in the specification,X k() is as followskThe layer reconstructs the image signal of the image,Yin order to sample the image signal, the image signal is sampled,Ain order to measure the matrix of the measurements,Din order to obtain a sparse dictionary,
for the first non-negative regularization parameter,
for the second non-negative regularization parameter,
in order to be a convolution weighting module, the convolution weighting module,B、Cfor a filter with a convolution kernel size of 3 x 3,Relu() In order to modify the linear cell activation function,Fto predict the matrix, is a learnable parameter of the compressed sensing network,X k(-1) is as followsk-1 layer of reconstructed image signals.
Optionally, the loss function of the compressed sensing network is:
in the formula (I), the compound is shown in the specification,N b as the total number of the sampled image signals,N p in order to compress the number of perceived network layers,Nin order to sample the size of the image signal,
is a firstiThe final reconstructed image signal corresponding to each sampled image,S i is as followsiThe original image signal is then processed to generate a plurality of original image signals,W k() is as followskThe layer convolution weighting module is used for weighting the layer convolution,
is a firstkLayer andW k() a convolution weighting module for forming a symmetrical relation,
is a firstiA reconstructed image signal is inkThe sparse signal of the layer(s),
is a third regularization parameter.
A second aspect of the present application provides an image reconstruction apparatus based on a compressed sensing network, including:
the sampling unit is used for sampling the original image signal to obtain a sampled image signal;
the initialization unit is used for initializing a reconstructed image signal based on the sampling image signal to obtain an initial reconstructed image signal;
the reconstruction optimization unit is used for inputting the sampling image signal and the initial reconstruction image signal into a compressed sensing network, carrying out signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain multilayer reconstruction image signals, and optimizing the reconstruction image signals according to the correlation between two adjacent layers of reconstruction image signals to obtain final reconstruction image signals;
the training unit is used for calculating a loss value according to the final reconstructed image signal and the original image signal, and updating the network parameters of the compressive sensing network according to the loss value to obtain a trained compressive sensing network;
and the reconstruction unit is used for carrying out image reconstruction on the image signal to be reconstructed through the trained compressed sensing network to obtain a reconstructed image signal.
Optionally, the sampling unit is specifically configured to:
and sampling the original image signal through a measurement matrix to obtain a sampled image signal.
Optionally, the initialization unit is specifically configured to:
and initializing a reconstructed image signal based on the sampling image signal, the sparse dictionary and the measurement matrix to obtain an initial reconstructed image signal.
Optionally, the compressed sensing network is a multilayer iterative model, and an objective function corresponding to each layer is as follows:
in the formula (I), the compound is shown in the specification,X k() is a firstkThe layer reconstructs the image signal of the image,Yin order to sample the image signal, the image signal is sampled,Ain order to measure the matrix of the measurements,Din order to be a sparse dictionary,
for the first non-negative regularization parameter,
for the second non-negative regularization parameter,
in order to be a convolution weighting module, the convolution weighting module,B、Cfor convolution kernel size of 33 of the filter of the first and second filters,Relu() In order to modify the linear cell activation function,Ffor the prediction matrix, is a learnable parameter of the compressed sensing network,X k(-1) is as followsk-1 layer reconstructed image signal.
Optionally, the loss function of the compressed sensing network is:
in the formula (I), the compound is shown in the specification,N b to be the total number of the sampled image signals,N p in order to compress the number of perceived network layers,Nin order to sample the size of the image signal,
is a firstiThe final reconstructed image signal corresponding to each sampled image,S i is a firstiThe original image signal is then processed to generate a plurality of original image signals,W k() is as followskA layer convolution weighting module for weighting the layer convolution,
is as followskLayer andW k() a convolution weighting module for forming a symmetrical relationship,
is a firstiA reconstructed image signal is inkThe sparse signal of the layer(s),
is the third regularization parameter.
According to the technical scheme, the method has the following advantages:
the application provides an image reconstruction method based on a compressed sensing network, which comprises the following steps: sampling an original image signal to obtain a sampled image signal; initializing a reconstructed image signal based on the sampled image signal to obtain an initial reconstructed image signal; inputting the sampled image signal and the initial reconstructed image signal into a compressed sensing network, carrying out signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain multilayer reconstructed image signals, and optimizing the reconstructed image signals according to the correlation between two adjacent layers of reconstructed image signals to obtain final reconstructed image signals; calculating a loss value according to the final reconstructed image signal and the original image signal, and updating network parameters of the compressive sensing network through the loss value to obtain a trained compressive sensing network; and carrying out image reconstruction on the image signal to be reconstructed through the trained compressed sensing network to obtain a reconstructed image signal.
According to the method, after a sampling image signal and an initial reconstruction image signal are obtained through an original image signal, the sampling image signal and the initial reconstruction image signal are input into a compressed sensing network for image reconstruction, a network input signal is weighted through a multilayer convolution weighting module, multilayer reconstruction image signals are obtained, and the problem that a weighting matrix is required to be an orthogonal matrix or a diagonal matrix when signals are weighted in the prior art is solved; the reconstructed image signals are optimized through the similarity between two adjacent layers of reconstructed image signals, the correlation between the two adjacent layers of reconstructed image signals is more stable than the correlation between adjacent column signals, so that the stability of a compressed sensing network is improved, the stability of a reconstruction result is further ensured, the reconstruction effect is improved, the technical problem that in the prior art, a recurrent neural network is adopted to weight the reconstructed signals, a weighting matrix is greatly limited, and a model is designed by utilizing the similarity of each column of signals in the reconstructed signals, and the reconstruction effect is poor due to the poor stability of the similarity of each column of signals is solved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the description below are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without inventive labor.
Fig. 1 is a schematic flowchart of an image reconstruction method based on a compressed sensing network according to an embodiment of the present application;
fig. 2 is a schematic diagram of a compressed sensing-based signal processing process according to an embodiment of the present application;
fig. 3 is a schematic diagram of a network model of a compressed sensing network according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an image reconstructing apparatus based on a compressed sensing network according to an embodiment of the present application.
Detailed Description
In order to make those skilled in the art better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
Compressed sensing is an emerging information acquisition and processing theory. The idea behind compressed sensing is that there is a tendency to find ways to perceive data directly from a compressed format, i.e. using a lower sampling rate, rather than first using a high sample and then compressing the data.
Please refer to fig. 2, for oneNOf dimensionKSparse signals, projecting high dimensional signals through a perceptual matrixMOn the aspect of the dimensional space, the device can be used,M<<Ni.e. by
Wherein
Is a measurement matrix. The whole process is to obtain high-dimensional signalsNDimension) compression into low-dimension signals (MDimension(s),ytransmitted as an observation signal, which when it reaches the observation end, can be selected fromyAccurate recovery of sparse signalss。
Existing methods can be largely classified into two categories. The first is a compressed sensing algorithm, which utilizes sparse coefficientsxIs used to identify the true solution from an infinite set of feasible solutions. For example, due to natural signalss(e.g. image signals) are typically sparse in the wavelet domain, i.e.s=DxThis sparse prior can be formulated as a criterion
. However, the manual design of signal priors in reconstruction algorithms is limited to general priors, such as sparsity and partial known support, and therefore only limited recognition capabilities are available.
The second category is learning from a large number of training data sets and deep neural networksyToxDirect mapping of (2). These methods can often achieve better performance in many cases with sufficient training data compared to the CS algorithm. In the prior art, a recurrent neural network is usually adopted for signal reconstruction, and a reconstructed signal is weighted by the recurrent neural network, but the method has great limitation on a weighting matrix which is necessarily an orthogonal matrix or a diagonal matrix, so that the weighting performance is greatly limited; and the network model is designed by utilizing the similarity of each column of signals in the reconstructed signals, the similarity of each column of signals is unstable, when the picture content is monotonous, for example, only one cup exists, the similarity between each column of signals is strong, and when the picture content is complex, the similarity between each column of signals is weak, so that the stability of the obtained reconstructed result is poor. In order to solve the problem, referring to fig. 1, an embodiment of the present application provides an image reconstruction method based on a compressed sensing network, including:
The original image signal can be represented asS=DX，DIn order to obtain a sparse dictionary,Xis a sparse signal. By measuring matricesAFor original image signalSSampling to obtain a sampled image signalYI.e. byY=AS。
And 102, initializing a reconstructed image signal based on the sampled image signal to obtain an initial reconstructed image signal.
Based on sampled image signalsYSparse dictionaryDAnd a measurement matrixAInitializing the reconstructed image signal to obtain an initial reconstructed image signalX (0) I.e. byX (0) =D T A T YAnd T is a matrix transpose.
In addition, the original image signal is processedSMeasurement matrix when samplingAInitializing a sparse dictionary of reconstructed image signalsDMeasuring matrixAThe initial sparse dictionary and the initial measurement matrix are known parameters, and the sparse dictionary is used for image reconstruction in a compressed sensing networkDMeasuring matrixAAnd updating the network parameters belonging to the compressed sensing network in the iterative reconstruction process.
The network model of the compressive sensing network in the embodiment of the present application can refer to fig. 3, and the compressive sensing network includes a convolution weighting moduleW k() 、
And a soft threshold module (by a soft threshold function)soft() Composed of convolution layers), wherein the convolution weighting module consists of convolution layersConvActive layerReluAnd a convolution layerConvAnd (4) stacking. Hypothesis signalX k() There are the following dynamic models:
in the formula (I), the compound is shown in the specification,Fthe prediction matrix is initialized to be an identity matrix, and the compressed sensing networkMatrix learning is carried out through data driving in the process of collateral training,U k(-1) is an error matrix.
Based on the above assumptions, the compressive sensing network model in the embodiment of the present application is constructed as a multi-layer iterative model, and an objective function corresponding to each layer is as follows:
in the formula (I), the compound is shown in the specification,X k() is a firstkThe layer reconstructs the image signal of the image,Yin order to sample the image signal, the image signal is sampled,Ain order to measure the matrix of the measurements,Din order to be a sparse dictionary,
for the first non-negative regularization parameter,
is a second non-negative regularization parameter,
in order to be a convolution weighting module, the convolution weighting module,B、Cfor a filter with a convolution kernel size of 3 x 3,Relu() In order to modify the linear cell activation function,Fto predict the matrix, is a learnable parameter of the compressed sensing network,X k(-1) is as followsk-1 layer of reconstructed image signals.
The two norm terms in the formula (1) are combined to obtain the variable
，
，
For the intermediate parameters, the objective function in equation (1) can be further converted into:
the reconstruction process of a compressed sensing network can be expressed as:
in the formula (I), the compound is shown in the specification,R k() is as followskThe input of the layer convolution weighting module is,
is the step size.
Convert equation (3), i.e.
Will be
And (3) performing expansion operation to obtain:
in the formula, I is an identity matrix;
order to
，
I.e. by
。
In the examples of this application, in combination withkThe convolution weighting module of the layer is used for extracting features, and compared with a common weighting matrix, the signal can be extracted by weighting by using the convolution weighting moduleThe high-dimensional characteristics of the number improve the network reconstruction capability by training and changing the parameters of the convolution kernel,Relu() The activation function will make one output 0, which results in sparsity of the output and reduces the interdependence of the parameters, alleviating the over-fitting problem.
Obtained by a convolution weighting module
，
Is andW k() the relative amount of scalar is calculated,W k() is as followskAnd a layer convolution weighting module. Equation (4) can be converted to:
wherein the parameters
By solving, we can get:
inspired by the reversibility of wavelet transformation, can
Is arranged as and
the pair of the symmetrical structures is formed by a pair of the structures,
and with
Are learnable and are incorporated into the penalty function to enforce during compressive sensing network trainingThe line symmetry constraint can be solved as follows:
to increase the capacity of a compressed sensing network, it is not required
，
And
the same at each layer in a compressed sensing network, i.e. the outputX k() Comprises the following steps:
byX k(-1) Can obtain
Gradient term of
Thereby obtainingX k() Of approximation term
。
Compressive sensing network based on input sampled image signalYAnd an initial reconstructed image signalX (0) Calculating to obtain intermediate parametersR (0) Then will beR (0) Inputting the signal into a first layer convolution weighting module for signal weighting to obtain a first layer reconstructed image signalX (1) Reconstructing an image signal by a first layerX (1) The input signal of the second layer convolution weighting module can be obtained by calculationR (2) Adding the signal by a second layer convolution weighting moduleObtaining a second layer reconstructed image signalX (2) By analogy, when the first one is obtainedkInput signal of a layer convolution weighting moduleR k() When passing throughkThe signal weighting is carried out by the layer convolution weighting module to obtain the secondkLayer reconstructed image signalX k() Until a reconstructed image signal output by the last layer of convolution weighting module is obtainedX Np() ，N p For compressing and sensing the number of network layers, a sparse dictionary is usedDAnd reconstructing the image signalX Np() The final reconstructed image signal can be obtained by reconstruction
。
The image reconstruction process based on compressed sensing can be expressed as:
input the methodX (0) 、YNumber of network layersN p Parameter(s) of
And (3) outputting:
for k=1: N do
return 
in the embodiment of the application, the signal is weighted by the convolution weighting module of the compressed sensing network, and the weighting moment existing in the prior art of signal weighting by adopting the recurrent neural network is convolvedThe array is limited by an orthogonal matrix or a diagonal matrix, the high-dimensional characteristics of the signals are extracted through convolution, and the signals are weighted by combining a convolution weighting module, so that the reconstruction performance of the compressed sensing network is improved; reconstructing the image signal by using two adjacent layers (e.g. secondk1 layer and the firstkLayer reconstruction image signal
) The correlation between the adjacent column signals is more stable than the correlation between the adjacent column signals, and the reconstruction effect is improved.
And 104, calculating a loss value according to the final reconstructed image signal and the original image signal, and updating the network parameters of the compressed sensing network according to the loss value to obtain the trained compressed sensing network.
The loss function of the compressed sensing network in the embodiment of the application is as follows:
in the formula (I), the compound is shown in the specification,N b as the total number of the sampled image signals,N p in order to compress the number of layers in the sensing network,Nin order to sample the size of the image signal,
is as followsiThe final reconstructed image signal corresponding to each sampled image,S i is as followsiThe original image signal is then processed to generate a plurality of original image signals,W k() is as followskA layer convolution weighting module for weighting the layer convolution,
is as followskLayer andW k() a convolution weighting module for forming a symmetrical relationship,
is a firstiA reconstructed image signal is inkThe sparse signal of the layers is such that,
for the third regularization parameter, 0.01 may be set.
The loss function in this application consists of
And
two-part construction, the loss function in the embodiment of the present application being taken into accountL constraint In order to ensure symmetry of the convolution structure in order to solve the reconstructed image signal.
And updating network parameters through the loss value obtained by solving the loss function until the network converges to obtain the trained compressed sensing network. It is understood that when training the compressive sensing network, the raw image signals can be obtained from the image data sets disclosed by Caltech-256, BSDS500, set11, or the like.
And 105, carrying out image reconstruction on the image signal to be reconstructed through the trained compressed sensing network to obtain a reconstructed image signal.
And when an image signal to be reconstructed is received, inputting the image signal to be reconstructed into the trained compressed sensing network for image reconstruction to obtain a reconstructed image signal. For example, the observation terminal receives an observation signalYObserving signals through a trained compressed sensing networkYPerforming image reconstruction to derive a signal from the observation signalYAccurate recovery of sparse signalsS。
In the embodiment of the application, after an original image signal is obtained into a sampling image signal and an initial reconstruction image signal, the sampling image signal and the initial reconstruction image signal are input into a compressed sensing network for image reconstruction, and a network input signal is weighted through a multilayer convolution weighting module to obtain multilayer reconstruction image signals, so that the problem that a weighting matrix is required to be an orthogonal matrix or a diagonal matrix when the signals are weighted in the prior art is solved; the reconstructed image signals are optimized through the similarity between two adjacent layers of reconstructed image signals, the correlation between the two adjacent layers of reconstructed image signals is more stable than the correlation between adjacent column signals, so that the stability of a compressed sensing network is improved, the stability of a reconstruction result is further ensured, the reconstruction effect is improved, the technical problem that in the prior art, a recurrent neural network is adopted to weight the reconstructed signals, a weighting matrix is greatly limited, and a model is designed by utilizing the similarity of each column of signals in the reconstructed signals, and the reconstruction effect is poor due to the poor stability of the similarity of each column of signals is solved.
The above is an embodiment of the image reconstruction method based on the compressive sensing network provided by the present application, and the following is an embodiment of the image reconstruction device based on the compressive sensing network provided by the present application.
Referring to fig. 4, an image reconstructing apparatus based on a compressed sensing network according to an embodiment of the present application includes:
a sampling unit 401, configured to sample an original image signal to obtain a sampled image signal;
an initializing unit 402, configured to initialize a reconstructed image signal based on the sampled image signal, to obtain an initial reconstructed image signal;
the reconstruction optimization unit 403 is configured to input the sampled image signal and the initial reconstructed image signal into the compressed sensing network, perform signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain a multilayer reconstructed image signal, and optimize the reconstructed image signal according to a correlation between two adjacent layers of reconstructed image signals to obtain a final reconstructed image signal;
a training unit 404, configured to calculate a loss value according to the final reconstructed image signal and the original image signal, and update a network parameter of the compressed sensing network according to the loss value to obtain a trained compressed sensing network;
and the reconstructing unit 405 is configured to perform image reconstruction on the image signal to be reconstructed through the trained compressed sensing network to obtain a reconstructed image signal.
As a further improvement, the sampling unit 401 is specifically configured to:
and sampling the original image signal through the measurement matrix to obtain a sampled image signal.
As a further improvement, the initialization unit 402 is specifically configured to:
and initializing a reconstructed image signal based on the sampling image signal, the sparse dictionary and the measurement matrix to obtain an initial reconstructed image signal.
As a further improvement, the compressed sensing network is a multi-layer iterative model, and the corresponding objective function of each layer is as follows:
in the formula (I), the compound is shown in the specification,X k() is a firstkThe layer reconstructs the image signal of the image,Yin order to sample the image signal, the image signal is sampled,Ain order to measure the matrix of the measurements,Din order to be a sparse dictionary,
for the first non-negative regularization parameter,
is a second non-negative regularization parameter,
in order to be a convolution weighting module, the convolution weighting module,B、Cfor a filter with a convolution kernel size of 3 x 3,Relu() In order to modify the linear cell activation function,Fto predict the matrix, is a learnable parameter of the compressed sensing network,X k(-1) is as followsk-1 layer of reconstructed image signals.
As a further improvement, the loss function of the compressed sensing network is:
in the formula (I), the compound is shown in the specification,N b to be the total number of the sampled image signals,N p for compressed sensing networksThe number of layers is equal to or greater than the number of layers,Nin order to sample the size of the image signal,
is as followsiThe final reconstructed image signal corresponding to each sampled image,S i is as followsiThe original image signal is then processed to generate a plurality of original image signals,W k() is as followskA layer convolution weighting module for weighting the layer convolution,
is as followskLayer andW k() a convolution weighting module for forming a symmetrical relationship,
is as followsiA reconstructed image signal is inkThe sparse signal of the layer(s),
is the third regularization parameter.
In the embodiment of the application, after an original image signal is obtained into a sampling image signal and an initial reconstruction image signal, the sampling image signal and the initial reconstruction image signal are input into a compressed sensing network for image reconstruction, and a network input signal is weighted through a multilayer convolution weighting module to obtain multilayer reconstruction image signals, so that the problem that a weighting matrix is required to be an orthogonal matrix or a diagonal matrix when the signals are weighted in the prior art is solved; the reconstructed image signals are optimized through the similarity between two adjacent layers of reconstructed image signals, the correlation between the two adjacent layers of reconstructed image signals is more stable than the correlation between adjacent column signals, so that the stability of a compressed sensing network is improved, the stability of a reconstruction result is further ensured, the reconstruction effect is improved, the technical problem that in the prior art, a recurrent neural network is adopted to weight the reconstructed signals, a weighting matrix is greatly limited, and a model is designed by utilizing the similarity of each column of signals in the reconstructed signals, and the reconstruction effect is poor due to the poor stability of the similarity of each column of signals is solved.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The terms "first," "second," "third," "fourth," and the like in the description of the application and the above-described figures, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the application described herein may be implemented, for example, in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be understood that in the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" is used to describe the association relationship of the associated object, indicating that there may be three relationships, for example, "a and/or B" may indicate: only A, only B and both A and B are present, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of the singular or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may also be implemented in the form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for executing all or part of the steps of the method described in the embodiments of the present application through a computer device (which may be a personal computer, a server, or a network device). And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present application.
Claims (10)
1. An image reconstruction method based on a compressed sensing network is characterized by comprising the following steps:
sampling an original image signal to obtain a sampled image signal;
initializing a reconstructed image signal based on the sampling image signal to obtain an initial reconstructed image signal;
inputting the sampled image signal and the initial reconstructed image signal into a compressed sensing network, performing signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain a multilayer reconstructed image signal, and optimizing the reconstructed image signal according to the correlation between two adjacent layers of reconstructed image signals to obtain a final reconstructed image signal;
calculating a loss value according to the final reconstructed image signal and the original image signal, and updating network parameters of the compressive sensing network according to the loss value to obtain a trained compressive sensing network;
and carrying out image reconstruction on the image signal to be reconstructed through the trained compressed sensing network to obtain a reconstructed image signal.
2. The method for reconstructing an image based on a compressed sensing network according to claim 1, wherein the sampling an original image signal to obtain a sampled image signal comprises:
and sampling the original image signal through a measurement matrix to obtain a sampled image signal.
3. The method according to claim 1, wherein initializing a reconstructed image signal based on the sampled image signal to obtain an initial reconstructed image signal comprises:
and initializing a reconstructed image signal based on the sampling image signal, the sparse dictionary and the measurement matrix to obtain an initial reconstructed image signal.
4. The image reconstruction method based on the compressed sensing network according to claim 1, wherein the compressed sensing network is a multi-layer iterative model, and an objective function corresponding to each layer is as follows:
in the formula (I), the compound is shown in the specification,X k() is as followskThe layer reconstructs the image signal of the image,Yin order to sample the image signal, the image signal is sampled,Ain order to measure the matrix of the measurements,Din order to obtain a sparse dictionary,
is a first non-negative regularization parameter,
for the second non-negative regularization parameter,
in order to be a convolution weighting module, the convolution weighting module,B、Cfor a filter with a convolution kernel size of 3 x 3,Relu() In order to modify the linear cell activation function,Fto predict the matrix, is a learnable parameter of the compressed sensing network,X k(-1) is as followsk-1 layer of reconstructed image signals.
5. The compressed sensing network-based image reconstruction method according to claim 4, wherein the loss function of the compressed sensing network is:
in the formula (I), the compound is shown in the specification,N b to be the total number of the sampled image signals,N p in order to compress the number of layers in the sensing network,Nin order to sample the size of the image signal,
is as followsiThe final reconstructed image signal corresponding to each sampled image,S i is as followsiThe original image signal is then processed to generate a plurality of original image signals,W k() is as followskA layer convolution weighting module for weighting the layer convolution,
is a firstkLayer andW k() a convolution weighting module for forming a symmetrical relation,
is as followsiA reconstructed image signal is inkThe sparse signal of the layer(s),
is the third regularization parameter.
6. An image reconstruction apparatus based on a compressed sensing network, comprising:
the sampling unit is used for sampling the original image signal to obtain a sampled image signal;
the initialization unit is used for initializing a reconstructed image signal based on the sampling image signal to obtain an initial reconstructed image signal;
the reconstruction optimization unit is used for inputting the sampling image signal and the initial reconstruction image signal into a compressed sensing network, carrying out signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain multilayer reconstruction image signals, and optimizing the reconstruction image signals according to the correlation between two adjacent layers of reconstruction image signals to obtain final reconstruction image signals;
the training unit is used for calculating a loss value according to the final reconstructed image signal and the original image signal, and updating the network parameters of the compressed sensing network according to the loss value to obtain a trained compressed sensing network;
and the reconstruction unit is used for carrying out image reconstruction on the image signal to be reconstructed through the trained compressed sensing network to obtain a reconstructed image signal.
7. The compressed sensing network-based image reconstruction device according to claim 6, wherein the sampling unit is specifically configured to:
and sampling the original image signal through a measurement matrix to obtain a sampled image signal.
8. The compressed sensing network-based image reconstruction device according to claim 6, wherein the initialization unit is specifically configured to:
and initializing a reconstructed image signal based on the sampling image signal, the sparse dictionary and the measurement matrix to obtain an initial reconstructed image signal.
9. The device according to claim 6, wherein the compressive sensing network is a multi-layer iterative model, and an objective function corresponding to each layer is:
in the formula (I), the compound is shown in the specification,X k() is as followskThe layer reconstructs the image signal of the image,Yin order to sample the image signal, the image signal is sampled,Ain order to measure the matrix of the measurements,Din order to obtain a sparse dictionary,
is a first non-negative regularization parameter,
for the second non-negative regularization parameter,
in order to be a convolution weighting module, the convolution weighting module,B、Cfor a filter with a convolution kernel size of 3 x 3,Relu() In order to modify the linear cell activation function,Fto predict the matrix, is a learnable parameter of the compressed sensing network,X k(-1) is as followsk-1 layer of reconstructed image signals.
10. The compressed sensing network-based image reconstruction device according to claim 9, wherein the loss function of the compressed sensing network is:
in the formula (I), the compound is shown in the specification,N b as the total number of the sampled image signals,N p in order to compress the number of layers in the sensing network,Nin order to sample the size of the image signal,
is as followsiThe final reconstructed image signal corresponding to each sampled image,S i is as followsiThe original image signal is converted into a digital image signal,W k() is as followskA layer convolution weighting module for weighting the layer convolution,
is as followskLayer andW k() a convolution weighting module for forming a symmetrical relationship,
is as followsiA reconstructed image signal is inkThe sparse signal of the layers is such that,
is a third regularization parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211264253.0A CN115330901B (en) | 2022-10-17 | 2022-10-17 | Image reconstruction method and device based on compressed sensing network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211264253.0A CN115330901B (en) | 2022-10-17 | 2022-10-17 | Image reconstruction method and device based on compressed sensing network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115330901A true CN115330901A (en) | 2022-11-11 |
| CN115330901B CN115330901B (en) | 2023-01-17 |
Family
ID=83915426
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211264253.0A Active CN115330901B (en) | 2022-10-17 | 2022-10-17 | Image reconstruction method and device based on compressed sensing network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115330901B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115861472A (en) * | 2023-02-27 | 2023-03-28 | 广东工业大学 | Image reconstruction method, device, equipment and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102332153A (en) * | 2011-09-13 | 2012-01-25 | 西安电子科技大学 | Kernel regression-based image compression sensing reconstruction method |
| CN111667445A (en) * | 2020-05-29 | 2020-09-15 | 湖北工业大学 | Image compressed sensing reconstruction method based on Attention multi-feature fusion |
| CN112634391A (en) * | 2020-12-29 | 2021-04-09 | 华中科技大学 | Gray level image depth reconstruction and fault diagnosis system based on compressed sensing |
| CN112991472A (en) * | 2021-03-19 | 2021-06-18 | 华南理工大学 | Image compressed sensing reconstruction method based on residual dense threshold network |
| CN113052925A (en) * | 2021-04-02 | 2021-06-29 | 广东工业大学 | Compressed sensing reconstruction method and system based on deep learning |
| WO2022166298A1 (en) * | 2021-02-05 | 2022-08-11 | 歌尔股份有限公司 | Image processing method and apparatus, and electronic device and readable storage medium |
-
2022
- 2022-10-17 CN CN202211264253.0A patent/CN115330901B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102332153A (en) * | 2011-09-13 | 2012-01-25 | 西安电子科技大学 | Kernel regression-based image compression sensing reconstruction method |
| CN111667445A (en) * | 2020-05-29 | 2020-09-15 | 湖北工业大学 | Image compressed sensing reconstruction method based on Attention multi-feature fusion |
| CN112634391A (en) * | 2020-12-29 | 2021-04-09 | 华中科技大学 | Gray level image depth reconstruction and fault diagnosis system based on compressed sensing |
| WO2022166298A1 (en) * | 2021-02-05 | 2022-08-11 | 歌尔股份有限公司 | Image processing method and apparatus, and electronic device and readable storage medium |
| CN112991472A (en) * | 2021-03-19 | 2021-06-18 | 华南理工大学 | Image compressed sensing reconstruction method based on residual dense threshold network |
| CN113052925A (en) * | 2021-04-02 | 2021-06-29 | 广东工业大学 | Compressed sensing reconstruction method and system based on deep learning |
Non-Patent Citations (2)
| Title |
|---|
| JIAN ZHANG 等: "ISTA-Net：Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing", 《COMPUTER VISION FOUNDATION》 * |
| JUN ZHANG 等: "Deep Unfolding With Weighted Minimization for Compressive Sensing", 《IEEE INTERNET OF THINGS IOURNAL》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115861472A (en) * | 2023-02-27 | 2023-03-28 | 广东工业大学 | Image reconstruction method, device, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115330901B (en) | 2023-01-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3678059B1 (en) | Image processing method, image processing apparatus, and a neural network training method | |
| US20160292589A1 (en) | Ultra-high compression of images based on deep learning | |
| CN104112263A (en) | Method for fusing full-color image and multispectral image based on deep neural network | |
| CN106097278B (en) | Sparse model, reconstruction method and dictionary training method of multi-dimensional signal | |
| CN105981050A (en) | Method and system for exacting face features from data of face images | |
| CN110288524B (en) | Deep learning super-resolution method based on enhanced upsampling and discrimination fusion mechanism | |
| CN116051896B (en) | Hyperspectral image classification method of lightweight mixed tensor neural network | |
| CN115330901B (en) | Image reconstruction method and device based on compressed sensing network | |
| CN111698508B (en) | Super-resolution-based image compression method, device and storage medium | |
| CN114743009B (en) | Hyperspectral image band selection method and system and electronic equipment | |
| CN115984110A (en) | Swin-transform-based second-order spectral attention hyperspectral image super-resolution method | |
| Li et al. | Combining synthesis sparse with analysis sparse for single image super-resolution | |
| CN113592769A (en) | Abnormal image detection method, abnormal image model training method, abnormal image detection device, abnormal image model training device and abnormal image model training medium | |
| CN113850182B (en) | DAMR _ DNet-based action recognition method | |
| CN106022358A (en) | Hyper-spectral image classification method and hyper-spectral image classification device | |
| CN113327205B (en) | Phase denoising method based on convolutional neural network | |
| Wolter et al. | Wavelet-packet powered deepfake image detection | |
| CN117892059A (en) | Electric energy quality disturbance identification method based on multi-mode image fusion and ResNetXt-50 | |
| Liu et al. | CNN-Enhanced graph attention network for hyperspectral image super-resolution using non-local self-similarity | |
| CN116708807A (en) | Compression reconstruction method and compression reconstruction device for monitoring video | |
| CN114693547A (en) | Radio frequency image enhancement method and radio frequency image identification method based on image super-resolution | |
| CN114708281A (en) | Image compressed sensing reconstruction method based on self-adaptive non-local feature fusion network | |
| WO2021179117A1 (en) | Method and apparatus for searching number of neural network channels | |
| CN113920015A (en) | Infrared image edge preserving super-resolution reconstruction method based on generation countermeasure network | |
| CN107945131B (en) | Distributed image reconstruction method for radio interference array |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |