CN115330901B - Image reconstruction method and device based on compressed sensing network - Google Patents

Image reconstruction method and device based on compressed sensing network Download PDF

Info

Publication number
CN115330901B
CN115330901B CN202211264253.0A CN202211264253A CN115330901B CN 115330901 B CN115330901 B CN 115330901B CN 202211264253 A CN202211264253 A CN 202211264253A CN 115330901 B CN115330901 B CN 115330901B
Authority
CN
China
Prior art keywords
image signal
sensing network
reconstructed
compressed sensing
reconstruction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211264253.0A
Other languages
Chinese (zh)
Other versions
CN115330901A (en
Inventor
张军
刘忠俊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Provincial Laboratory Of Artificial Intelligence And Digital Economy Guangzhou
Guangdong University of Technology
Original Assignee
Guangdong Provincial Laboratory Of Artificial Intelligence And Digital Economy Guangzhou
Guangdong University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Provincial Laboratory Of Artificial Intelligence And Digital Economy Guangzhou, Guangdong University of Technology filed Critical Guangdong Provincial Laboratory Of Artificial Intelligence And Digital Economy Guangzhou
Priority to CN202211264253.0A priority Critical patent/CN115330901B/en
Publication of CN115330901A publication Critical patent/CN115330901A/en
Application granted granted Critical
Publication of CN115330901B publication Critical patent/CN115330901B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/001—Texturing; Colouring; Generation of texture or colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4084—Scaling of whole images or parts thereof, e.g. expanding or contracting in the transform domain, e.g. fast Fourier transform [FFT] domain scaling

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Image Analysis (AREA)

Abstract

The application discloses an image reconstruction method and device based on a compressed sensing network, which are used for sampling an original image signal to obtain a sampled image signal and further initializing to obtain an initial reconstructed image signal; inputting the sampled image signal and the initial reconstructed image signal into a compressed sensing network, carrying out signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain multilayer reconstructed image signals, and optimizing the reconstructed image signals according to the correlation between two adjacent layers of reconstructed image signals; updating network parameters according to the final reconstructed image signal and the original image signal; the image reconstruction is carried out on the image signal to be reconstructed through the trained compressed sensing network, the technical problems that in the prior art, the weighting matrix is greatly limited by adopting a recurrent neural network to weight the reconstructed signal, and the stability of the reconstruction result is poor due to the poor stability of the similarity of each column of signals when a model is designed by utilizing the similarity of each column of signals in the reconstructed signal are solved.

Description

Image reconstruction method and device based on compressed sensing network
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an image reconstruction method and apparatus based on a compressed sensing network.
Background
Compressed Sensing (CS) aims at accurately reconstructing high-dimensional signals from a small number of measurements by exploiting signal sparsity and structural priors. In order to further improve reconstruction performance, in the prior art, a recurrent neural network is adopted to weight a reconstruction signal, but the method has great limitation on a weighting matrix and needs to be an orthogonal matrix or a diagonal matrix; and the reconstruction method based on the recurrent neural network designs a model by utilizing the similarity of each column of signals in the reconstructed signals, the similarity is unstable, when the picture content is monotonous, for example, only one cup is provided, the similarity between each column of signals is strong, and when the picture content is complex, the similarity between each column of signals is weak, so that the stability of the obtained reconstruction result is poor.
Disclosure of Invention
The application provides an image reconstruction method and device based on a compressed sensing network, which are used for solving the technical problems that in the prior art, a recurrent neural network is adopted to weight a reconstruction signal, a weighting matrix is greatly limited, and a model is designed by utilizing the similarity of each column of signals in the reconstruction signal, and the stability of the similarity of each column of signals is poor, so that the stability of a reconstruction result is poor.
In view of this, a first aspect of the present application provides an image reconstruction method based on a compressed sensing network, including:
sampling an original image signal to obtain a sampled image signal;
initializing a reconstructed image signal based on the sampling image signal to obtain an initial reconstructed image signal;
inputting the sampled image signal and the initial reconstructed image signal into a compressed sensing network, performing signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain a multilayer reconstructed image signal, and optimizing the reconstructed image signal according to the correlation between two adjacent layers of reconstructed image signals to obtain a final reconstructed image signal;
calculating a loss value according to the final reconstructed image signal and the original image signal, and updating network parameters of the compressed sensing network according to the loss value to obtain a trained compressed sensing network;
and carrying out image reconstruction on the image signal to be reconstructed through the trained compressed sensing network to obtain a reconstructed image signal.
Optionally, the sampling the original image signal to obtain a sampled image signal includes:
and sampling the original image signal through a measurement matrix to obtain a sampled image signal.
Optionally, initializing a reconstructed image signal based on the sampled image signal to obtain an initial reconstructed image signal includes:
and initializing a reconstructed image signal based on the sampling image signal, the sparse dictionary and the measurement matrix to obtain an initial reconstructed image signal.
Optionally, the compressed sensing network is a multilayer iterative model, and an objective function corresponding to each layer is as follows:
Figure 652691DEST_PATH_IMAGE001
；
in the formula (I), the compound is shown in the specification,X k() is a firstkThe layers reconstruct the image signal and,Yin order to sample the image signal, the image signal is sampled,Ain order to measure the matrix of the measurements,Din order to be a sparse dictionary,
Figure 632148DEST_PATH_IMAGE002
for the first non-negative regularization parameter,
Figure 975667DEST_PATH_IMAGE003
for the second non-negative regularization parameter,
Figure 433193DEST_PATH_IMAGE004
in order to be a convolution weighting module, the convolution weighting module,B、Ca filter with a convolution kernel size of 3*3,Relu() In order to modify the linear cell activation function,Fto predict the matrix, is a learnable parameter of the compressed sensing network,X k(-1) is as followsk-1 layer of reconstructed image signals.
Optionally, the loss function of the compressed sensing network is:
Figure 247566DEST_PATH_IMAGE005
；
in the formula (I), the compound is shown in the specification,N b to be the total number of the sampled image signals,N p in order to compress the number of layers in the sensing network,Nin order to sample the size of the image signal,
Figure 132345DEST_PATH_IMAGE006
is as followsiThe final reconstructed image signal corresponding to each sampled image,S i is as followsiThe original image signal is then processed to generate a plurality of original image signals,W k() is as followskLayer convolution weighting module，
Figure 461695DEST_PATH_IMAGE007
Is as followskLayer andW k() a convolution weighting module for forming a symmetrical relationship,
Figure 224377DEST_PATH_IMAGE008
is as followsiA reconstructed image signal is inkThe sparse signal of the layer(s),
Figure 893256DEST_PATH_IMAGE009
is the third regularization parameter.
A second aspect of the present application provides an image reconstructing apparatus based on a compressed sensing network, including:
the sampling unit is used for sampling the original image signal to obtain a sampled image signal;
the initialization unit is used for initializing a reconstructed image signal based on the sampling image signal to obtain an initial reconstructed image signal;
the reconstruction optimization unit is used for inputting the sampling image signal and the initial reconstruction image signal into a compressed sensing network, carrying out signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain multilayer reconstruction image signals, and optimizing the reconstruction image signals according to the correlation between two adjacent layers of reconstruction image signals to obtain final reconstruction image signals;
the training unit is used for calculating a loss value according to the final reconstructed image signal and the original image signal, and updating the network parameters of the compressive sensing network according to the loss value to obtain a trained compressive sensing network;
and the reconstruction unit is used for carrying out image reconstruction on the image signal to be reconstructed through the trained compressed sensing network to obtain a reconstructed image signal.
Optionally, the sampling unit is specifically configured to:
and sampling the original image signal through a measurement matrix to obtain a sampled image signal.
Optionally, the initialization unit is specifically configured to:
and initializing a reconstructed image signal based on the sampling image signal, the sparse dictionary and the measurement matrix to obtain an initial reconstructed image signal.
Optionally, the compressed sensing network is a multilayer iterative model, and an objective function corresponding to each layer is as follows:
Figure 948937DEST_PATH_IMAGE001
；
in the formula (I), the compound is shown in the specification,X k() is a firstkThe layers reconstruct the image signal and,Yin order to sample the image signal, the image signal is sampled,Ain order to measure the matrix of the measurements,Din order to be a sparse dictionary,
Figure 765583DEST_PATH_IMAGE002
for the first non-negative regularization parameter,
Figure 564912DEST_PATH_IMAGE003
for the second non-negative regularization parameter,
Figure 88297DEST_PATH_IMAGE004
in order to be a convolution weighting module, the convolution weighting module,B、Ca filter with a convolution kernel size of 3*3,Relu() In order to modify the linear cell activation function,Ffor the prediction matrix, is a learnable parameter of the compressed sensing network,X k(-1) is as followsk-1 layer of reconstructed image signals.
Optionally, the loss function of the compressed sensing network is:
Figure 81923DEST_PATH_IMAGE005
；
in the formula (I), the compound is shown in the specification,N b as the total number of the sampled image signals,N p in order to compress the number of layers in the sensing network,Nin order to sample the size of the image signal,
Figure 385865DEST_PATH_IMAGE006
is as followsiThe final reconstructed image signal corresponding to each sampled image,S i is as followsiThe original image signal is then processed to generate a plurality of original image signals,W k() is a firstkA layer convolution weighting module for weighting the layer convolution,
Figure 723306DEST_PATH_IMAGE007
is as followskLayer andW k() a convolution weighting module for forming a symmetrical relationship,
Figure 366777DEST_PATH_IMAGE008
is as followsiA reconstructed image signal is inkThe sparse signal of the layer(s),
Figure 764260DEST_PATH_IMAGE009
is the third regularization parameter.
According to the technical scheme, the method has the following advantages:
the application provides an image reconstruction method based on a compressed sensing network, which comprises the following steps: sampling an original image signal to obtain a sampled image signal; initializing a reconstructed image signal based on the sampled image signal to obtain an initial reconstructed image signal; inputting the sampled image signal and the initial reconstructed image signal into a compressed sensing network, carrying out signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain multilayer reconstructed image signals, and optimizing the reconstructed image signals according to the correlation between two adjacent layers of reconstructed image signals to obtain final reconstructed image signals; calculating a loss value according to the final reconstructed image signal and the original image signal, and updating network parameters of the compressive sensing network through the loss value to obtain a trained compressive sensing network; and carrying out image reconstruction on the image signal to be reconstructed through the trained compressed sensing network to obtain a reconstructed image signal.
According to the method, after a sampling image signal and an initial reconstruction image signal are obtained through an original image signal, the sampling image signal and the initial reconstruction image signal are input into a compressed sensing network for image reconstruction, a network input signal is weighted through a multilayer convolution weighting module, multilayer reconstruction image signals are obtained, and the problem that a weighting matrix is required to be an orthogonal matrix or a diagonal matrix when signals are weighted in the prior art is solved; the reconstructed image signals are optimized through the similarity between two adjacent layers of reconstructed image signals, the correlation between the two adjacent layers of reconstructed image signals is more stable than the correlation between adjacent column signals, so that the stability of a compressed sensing network is improved, the stability of a reconstruction result is further ensured, the reconstruction effect is improved, the technical problem that in the prior art, a recurrent neural network is adopted to weight the reconstructed signals, a weighting matrix is greatly limited, and a model is designed by utilizing the similarity of each column of signals in the reconstructed signals, and the reconstruction effect is poor due to the poor stability of the similarity of each column of signals is solved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a schematic flowchart of an image reconstruction method based on a compressive sensing network according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram of a compressed sensing-based signal processing process according to an embodiment of the present application;
fig. 3 is a schematic diagram of a network model of a compressed sensing network according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an image reconstructing apparatus based on a compressed sensing network according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
Compressed sensing is an emerging information acquisition and processing theory. The idea behind compressed sensing is that there is a trend to find ways to directly sense data from a compressed format, i.e. using a lower sampling rate, rather than first using a high sample and then compressing the data.
Please refer to fig. 2, for oneNOf dimensionKSparse signals, projecting high-dimensional signals through a perceptual matrixMOn the aspect of the dimensional space, the device can be used,M<<Ni.e. by
Figure 56963DEST_PATH_IMAGE010
Wherein
Figure 198095DEST_PATH_IMAGE011
Is a measurement matrix. The whole process is to obtain high-dimensional signalsNDimension) compression into low-dimension signals (MDimension(s),ytransmitted as an observation signal, which when it reaches the observation end, can be selected fromyAccurate recovery of sparse signalss。
Existing methods can be largely classified into two categories. The first is a compressed sensing algorithm, which utilizes sparse coefficientsxIs used to identify the true solution from an infinite set of feasible solutions. For example, due to natural signalss(e.g. image signals) are typically sparse in the wavelet domain, i.e.s=DxThis sparse prior can be formulated as a criterion
Figure 696072DEST_PATH_IMAGE012
. However, the manual design of signal priors in reconstruction algorithms is limited to general priors, such as sparsity and partial known support, and therefore only limited recognition capabilities are available.
The second category is learning from a large number of training data sets and deep neural networksyToxIs directly mapped. These methods can often achieve better performance in many cases with sufficient training data compared to the CS algorithm. In the prior art, a recurrent neural network is usually adopted for signal reconstruction, and a reconstructed signal is weighted by the recurrent neural network, but the method has great limitation on a weighting matrix which is necessarily an orthogonal matrix or a diagonal matrix, so that the weighting performance is greatly limited; and the network model is designed by utilizing the similarity of each column of signals in the reconstructed signals, the similarity of each column of signals is unstable, when the picture content is monotonous, for example, only one cup exists, the similarity between each column of signals is strong, and when the picture content is complex, the similarity between each column of signals is weak, so that the stability of the obtained reconstructed result is poor. In order to solve the problem, referring to fig. 1, an embodiment of the present application provides an image reconstruction method based on a compressed sensing network, including:
step 101, sampling an original image signal to obtain a sampled image signal.
The original image signal can be represented asS=DX，DIn order to be a sparse dictionary,Xis a sparse signal. By measuring matricesAFor original image signalSSampling to obtain a sampled image signalYI.e. byY=AS。
And 102, initializing a reconstructed image signal based on the sampled image signal to obtain an initial reconstructed image signal.
Based on sampled image signalsYSparse dictionaryDAnd a measurement matrixAInitializing the reconstructed image signal to obtain an initial reconstructed image signalX (0) I.e. byX (0) =D T A T YAnd T is matrix transposition.
In addition, the original image signal is processedSMeasurement matrix when samplingAInitializing a sparse dictionary of reconstructed image signalsDMeasuring matrixAThe initial sparse dictionary and the initial measurement matrix are known parameters, and the sparse dictionary is used for image reconstruction in a compressed sensing networkDMeasuring matrixAAt the iteration of the heavyAnd updating the network parameters belonging to the compressed sensing network in the construction process.
Step 103, inputting the sampled image signal and the initial reconstructed image signal into a compressed sensing network, performing signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain a multilayer reconstructed image signal, and optimizing the reconstructed image signal according to the correlation between two adjacent layers of reconstructed image signals to obtain a final reconstructed image signal.
The network model of the compressive sensing network in the embodiment of the present application can refer to fig. 3, and the compressive sensing network includes a convolution weighting moduleW k() 、
Figure 530036DEST_PATH_IMAGE007
And a soft threshold module (by a soft threshold function)soft() Composed of convolutional layers), wherein the convolutional weighting module consists of convolutional layersConvAn active layerReluAnd a convolution layerConvAnd (4) stacking. Hypothesis signalX k() There are the following dynamic models:
Figure 542991DEST_PATH_IMAGE013
；
in the formula (I), the compound is shown in the specification,Fthe initial value of the prediction matrix is an identity matrix, the matrix learning is carried out through data driving in the process of compressed sensing network training,U k(-1) is an error matrix.
Based on the above assumptions, the compressive sensing network model in the embodiment of the present application is constructed as a multi-layer iterative model, and an objective function corresponding to each layer is as follows:
Figure 989278DEST_PATH_IMAGE001
（1）
in the formula (I), the compound is shown in the specification,X k() is as followskThe layer reconstructs the image signal of the image,Yin order to sample the image signal, the image signal is sampled,Ain order to measure the matrix of the measurements,Din order to be a sparse dictionary,
Figure 404079DEST_PATH_IMAGE002
for the first non-negative regularization parameter,
Figure 346627DEST_PATH_IMAGE003
for the second non-negative regularization parameter,
Figure 846879DEST_PATH_IMAGE004
in order to be a convolution weighting module, the convolution weighting module,B、Ca filter with a convolution kernel size of 3*3,Relu() In order to modify the linear cell activation function,Fto predict the matrix, is a learnable parameter of the compressed sensing network,X k(-1) is as followsk-1 layer of reconstructed image signals.
The two norm terms in the formula (1) are combined to obtain the variable
Figure 329813DEST_PATH_IMAGE014
，
Figure 802382DEST_PATH_IMAGE015
，
Figure 491332DEST_PATH_IMAGE016
For the intermediate parameters, the objective function in equation (1) can be further converted into:
Figure 478880DEST_PATH_IMAGE017
（2）
the reconstruction process of a compressed sensing network can be expressed as:
Figure 765505DEST_PATH_IMAGE018
（3）
Figure 889319DEST_PATH_IMAGE019
（4）
in the formula (I), the compound is shown in the specification,R k() is as followskThe input to the layer convolution weighting module is,
Figure 737451DEST_PATH_IMAGE020
is the step size.
Convert equation (3), i.e.
Figure 149978DEST_PATH_IMAGE021
Will be
Figure 240294DEST_PATH_IMAGE016
The expansion operation yields:
Figure 484193DEST_PATH_IMAGE022
（5）
in the formula, I is a unit matrix;
order to
Figure 736183DEST_PATH_IMAGE023
，
Figure 199788DEST_PATH_IMAGE024
I.e. by
Figure 765898DEST_PATH_IMAGE025
。
In the examples of this application, in combination withkThe convolution weighting module of the layer extracts the characteristics, compared with the common weighting matrix, the convolution weighting module can extract the high-dimensional characteristics of the signal by weighting, the network reconstruction capability is improved by training and changing the parameters of the convolution kernel,Relu() The activation function will make one output 0, which results in sparsity of the output and reduces the interdependence of the parameters, alleviating the over-fitting problem.
Obtained by a convolution weighting module
Figure 129883DEST_PATH_IMAGE026
，
Figure 552775DEST_PATH_IMAGE027
Is andW k() the relative scalar quantity is calculated according to the measured quantity,W k() is as followskLayer convolution weightingAnd (5) modules. Equation (4) can be converted to:
Figure 939894DEST_PATH_IMAGE028
（6）
wherein the parameters
Figure 372012DEST_PATH_IMAGE029
By solving, we can get:
Figure 826389DEST_PATH_IMAGE030
（7）
inspired by the reversibility of wavelet transform, can
Figure 685761DEST_PATH_IMAGE031
Is arranged as and
Figure 294597DEST_PATH_IMAGE032
the structure is in a symmetrical structure, and the structure is,
Figure 530406DEST_PATH_IMAGE031
and
Figure 603404DEST_PATH_IMAGE032
all are learnable, and incorporate it into a loss function during the training of the compressed sensing network to enforce a symmetric constraint, one can solve:
Figure 571360DEST_PATH_IMAGE033
（8）
to increase the capacity of a compressed sensing network, no requirement is made
Figure 231274DEST_PATH_IMAGE031
，
Figure 5195DEST_PATH_IMAGE032
And
Figure 870383DEST_PATH_IMAGE034
the same at each layer in a compressed sensing network, i.e. the outputX k() Comprises the following steps:
Figure 71557DEST_PATH_IMAGE035
（9）
byX k(-1) Can obtain
Figure 717302DEST_PATH_IMAGE036
Gradient term of
Figure 530799DEST_PATH_IMAGE037
Thereby obtainingX k() Approximation term of
Figure 578390DEST_PATH_IMAGE038
。
Compressive sensing network based on input sampled image signalsYAnd an initial reconstructed image signalX (0) Calculating to obtain intermediate parametersR (0) Then will beR (0) Inputting the signal into a first layer convolution weighting module for signal weighting to obtain a first layer reconstructed image signalX (1) Reconstructing an image signal by a first layerX (1) The input signal of the second layer convolution weighting module can be obtained by calculationR (2) Performing signal weighting by a second layer convolution weighting module to obtain a second layer reconstructed image signalX (2) By analogy, when the first one is obtainedkInput signal of a layer convolution weighting moduleR k() When passing throughkThe layer convolution weighting module carries out signal weighting to obtain the secondkLayer reconstruction image signalX k() Until obtaining the reconstructed image signal output by the last layer of convolution weighting moduleX Np() ，N p For compressing and sensing the number of network layers, a sparse dictionary is usedDAnd reconstructing the image signalX Np() The final reconstructed image signal can be obtained by reconstruction
Figure 888148DEST_PATH_IMAGE039
。
The image reconstruction process based on compressed sensing can be expressed as:
input deviceX (0) 、YNumber of layers of networkN p Parameter of
Figure 21190DEST_PATH_IMAGE040
And (3) outputting:
Figure 136913DEST_PATH_IMAGE041
for k=1: N do
Figure 976693DEST_PATH_IMAGE042
Figure 21135DEST_PATH_IMAGE043
return
Figure 375893DEST_PATH_IMAGE039
in the embodiment of the application, the signal is weighted by the convolution weighting module of the compressed sensing network, the limitation that the weighting matrix in the prior art that the cyclic neural network is adopted for signal weighting must be an orthogonal matrix or a diagonal matrix is convoluted, the high-dimensional characteristics of the signal are extracted by convolution, and the signal is weighted by the convolution weighting module, so that the reconstruction performance of the compressed sensing network is improved; reconstructing the image signal using two adjacent layers (e.g. secondk-1 layer and the secondkLayer reconstruction image signal
Figure 295307DEST_PATH_IMAGE044
) The correlation between the adjacent column signals is more stable than the correlation between the adjacent column signals, and the reconstruction effect is improved.
And 104, calculating a loss value according to the final reconstructed image signal and the original image signal, and updating the network parameters of the compressed sensing network according to the loss value to obtain the trained compressed sensing network.
The loss function of the compressed sensing network in the embodiment of the application is as follows:
Figure 51911DEST_PATH_IMAGE005
（10）
in the formula (I), the compound is shown in the specification,N b as the total number of the sampled image signals,N p in order to compress the number of layers in the sensing network,Nin order to sample the size of the image signal,
Figure 703472DEST_PATH_IMAGE006
is as followsiThe final reconstructed image signal corresponding to each sampled image,S i is a firstiThe original image signal is then processed to generate a plurality of original image signals,W k() is as followskThe layer convolution weighting module is used for weighting the layer convolution,
Figure 312570DEST_PATH_IMAGE007
is as followskLayer andW k() a convolution weighting module for forming a symmetrical relationship,
Figure 770096DEST_PATH_IMAGE008
is a firstiA reconstructed image signal is inkThe sparse signal of the layer(s),
Figure 318889DEST_PATH_IMAGE009
for the third regularization parameter, 0.01 may be set.
The loss function in this application consists of
Figure 469248DEST_PATH_IMAGE045
And
Figure 798598DEST_PATH_IMAGE046
two-part construction, the loss function in the embodiment of the present application being taken into accountL constraint Is to ensure the symmetry of the convolution structure so as to solveThe image signal is de-reconstructed.
And updating network parameters through the loss value obtained by solving the loss function until the network converges to obtain the trained compressed sensing network. It is understood that when training the compressive sensing network, the raw image signals can be obtained from the image data sets disclosed by Caltech-256, BSDS500, set11, or the like.
And 105, carrying out image reconstruction on the image signal to be reconstructed through the trained compressed sensing network to obtain a reconstructed image signal.
And when an image signal to be reconstructed is received, inputting the image signal to be reconstructed into a trained compressed sensing network for image reconstruction, and obtaining a reconstructed image signal. For example, the observation terminal receives an observation signalYObserving signals through a trained compressed sensing networkYPerforming image reconstruction to derive a signal from the observation signalYAccurate recovery of sparse signalsS。
In the embodiment of the application, after a sampling image signal and an initial reconstruction image signal are obtained through an original image signal, the sampling image signal and the initial reconstruction image signal are input into a compressed sensing network for image reconstruction, a network input signal is weighted through a multilayer convolution weighting module, and multilayer reconstruction image signals are obtained, so that the problem that a weighting matrix is required to be an orthogonal matrix or a diagonal matrix when signals are weighted in the prior art is solved, high-dimensional characteristics of the signals are extracted through convolution, the signals are weighted through the convolution weighting module, and the reconstruction performance of the compressed sensing network is improved; the reconstructed image signals are optimized through the similarity between two adjacent layers of reconstructed image signals, the correlation between the two adjacent layers of reconstructed image signals is more stable than the correlation between adjacent column signals, so that the stability of a compressed sensing network is improved, the stability of a reconstruction result is further ensured, the reconstruction effect is improved, the technical problem that in the prior art, a recurrent neural network is adopted to weight the reconstructed signals, a weighting matrix is greatly limited, and a model is designed by utilizing the similarity of each column of signals in the reconstructed signals, and the reconstruction effect is poor due to the poor stability of the similarity of each column of signals is solved.
The above is an embodiment of the image reconstruction method based on the compressive sensing network provided by the present application, and the following is an embodiment of the image reconstruction device based on the compressive sensing network provided by the present application.
Referring to fig. 4, an image reconstructing apparatus based on a compressed sensing network according to an embodiment of the present application includes:
a sampling unit 401, configured to sample an original image signal to obtain a sampled image signal;
an initializing unit 402, configured to initialize a reconstructed image signal based on the sampled image signal, so as to obtain an initial reconstructed image signal;
the reconstruction optimization unit 403 is configured to input the sampled image signal and the initial reconstructed image signal into the compressed sensing network, perform signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain a multilayer reconstructed image signal, and optimize the reconstructed image signal according to a correlation between two adjacent layers of reconstructed image signals to obtain a final reconstructed image signal;
a training unit 404, configured to calculate a loss value according to the final reconstructed image signal and the original image signal, and update a network parameter of the compressed sensing network according to the loss value to obtain a trained compressed sensing network;
and the reconstructing unit 405 is configured to perform image reconstruction on the image signal to be reconstructed through the trained compressed sensing network to obtain a reconstructed image signal.
As a further improvement, the sampling unit 401 is specifically configured to:
and sampling the original image signal through the measurement matrix to obtain a sampled image signal.
As a further improvement, the initialization unit 402 is specifically configured to:
and initializing a reconstructed image signal based on the sampling image signal, the sparse dictionary and the measurement matrix to obtain an initial reconstructed image signal.
As a further improvement, the compressed sensing network is a multi-layer iterative model, and the corresponding objective function of each layer is as follows:
Figure 549561DEST_PATH_IMAGE001
；
in the formula (I), the compound is shown in the specification,X k() is a firstkThe layer reconstructs the image signal of the image,Yin order to sample the image signal, the image signal is sampled,Ain order to measure the matrix of the measurements,Din order to be a sparse dictionary,
Figure 952861DEST_PATH_IMAGE002
for the first non-negative regularization parameter,
Figure 274120DEST_PATH_IMAGE003
for the second non-negative regularization parameter,
Figure 90767DEST_PATH_IMAGE004
in order to be a convolution weighting module, the convolution weighting module,B、Ca filter with a convolution kernel size of 3*3,Relu() In order to modify the linear cell activation function,Fto predict the matrix, is a learnable parameter of the compressed sensing network,X k(-1) is as followsk-1 layer reconstructed image signal.
As a further improvement, the loss function of the compressed sensing network is:
Figure 827779DEST_PATH_IMAGE005
；
in the formula (I), the compound is shown in the specification,N b as the total number of the sampled image signals,N p in order to compress the number of layers in the sensing network,Nin order to sample the size of the image signal,
Figure 413481DEST_PATH_IMAGE006
is as followsiThe final reconstructed image signal corresponding to each sampled image,S i is a firstiThe original image signal is then processed to generate a plurality of original image signals,W k() is as followskA layer convolution weighting module for weighting the layer convolution,
Figure 407107DEST_PATH_IMAGE007
is as followskLayer andW k() rolls forming a symmetrical relationshipA product-weighting module for performing a product weighting on the data,
Figure 445470DEST_PATH_IMAGE008
is as followsiA reconstructed image signal is inkThe sparse signal of the layers is such that,
Figure 986173DEST_PATH_IMAGE009
is the third regularization parameter.
In the embodiment of the application, after a sampling image signal and an initial reconstruction image signal are obtained through an original image signal, the sampling image signal and the initial reconstruction image signal are input into a compressed sensing network for image reconstruction, a network input signal is weighted through a multilayer convolution weighting module, and multilayer reconstruction image signals are obtained, so that the problem that a weighting matrix is required to be an orthogonal matrix or a diagonal matrix when signals are weighted in the prior art is solved, high-dimensional characteristics of the signals are extracted through convolution, the signals are weighted through the convolution weighting module, and the reconstruction performance of the compressed sensing network is improved; the reconstructed image signals are optimized through the similarity between two adjacent layers of reconstructed image signals, the correlation between the two adjacent layers of reconstructed image signals is more stable than that between adjacent column signals, so that the stability of a compressed sensing network is improved, the stability of a reconstruction result is further ensured, the reconstruction effect is improved, and the technical problems that in the prior art, a recurrent neural network is adopted to weight the reconstructed signals, a weighting matrix is greatly limited, and a model is designed by utilizing the similarity of each column of signals in the reconstructed signals, and the reconstruction effect is poor due to the poor stability of the similarity of each column of signals are solved.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The terms "first," "second," "third," "fourth," and the like in the description of the application and the above-described figures, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the application described herein may be implemented, for example, in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be understood that, in this application, "at least one" means one or more, "a plurality" means two or more. "and/or" for describing an association relationship of associated objects, indicating that there may be three relationships, e.g., "a and/or B" may indicate: only A, only B and both A and B are present, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of single item(s) or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for executing all or part of the steps of the method described in the embodiments of the present application through a computer device (which may be a personal computer, a server, or a network device). And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims (6)

1. An image reconstruction method based on a compressed sensing network is characterized by comprising the following steps:
sampling an original image signal to obtain a sampled image signal;
initializing a reconstructed image signal based on the sampling image signal to obtain an initial reconstructed image signal;
inputting the sampled image signal and the initial reconstructed image signal into a compressed sensing network, performing signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain a multilayer reconstructed image signal, and optimizing the reconstructed image signal according to the correlation between two adjacent layers of reconstructed image signals to obtain a final reconstructed image signal;
calculating a loss value according to the final reconstructed image signal and the original image signal, and updating network parameters of the compressive sensing network according to the loss value to obtain a trained compressive sensing network;
carrying out image reconstruction on an image signal to be reconstructed through the trained compressed sensing network to obtain a reconstructed image signal;
the compressed sensing network is a multilayer iteration model, and the target function corresponding to each layer is as follows:
Figure 756981DEST_PATH_IMAGE001
；
in the formula (I), the compound is shown in the specification,X k() is a firstkThe layers reconstruct the image signal and,Yin order to sample the image signal, the image signal is sampled,Ain order to measure the matrix of the measurements,Din order to obtain a sparse dictionary,
Figure 573627DEST_PATH_IMAGE002
is a first non-negative regularization parameter,
Figure 183076DEST_PATH_IMAGE003
for the second non-negative regularization parameter,
Figure 519510DEST_PATH_IMAGE004
in order to be a convolution weighting module, the convolution weighting module,B、Ca filter with a convolution kernel size of 3*3,Relu() In order to modify the linear cell activation function,Ffor the prediction matrix, is a learnable parameter of the compressed sensing network,X k(-1) is a firstk-1 layer of reconstructed image signals;
the loss function of the compressed sensing network is as follows:
Figure 11671DEST_PATH_IMAGE005
；
in the formula (I), the compound is shown in the specification,N b as the total number of the sampled image signals,N p in order to compress the number of perceived network layers,Nin order to sample the size of the image signal,
Figure 66346DEST_PATH_IMAGE006
is as followsiThe final reconstructed image signal corresponding to each sampled image,S i is as followsiThe original image signal is then processed to generate a plurality of original image signals,W k() is as followskA layer convolution weighting module for weighting the layer convolution,
Figure 403787DEST_PATH_IMAGE007
is a firstkLayer andW k() a convolution weighting module for forming a symmetrical relationship,
Figure 860307DEST_PATH_IMAGE008
is as followsiA reconstructed image signal is inkThe sparse signal of the layer(s),
Figure 257790DEST_PATH_IMAGE009
is a third regularization parameter.
2. The image reconstruction method based on compressed sensing network according to claim 1, wherein the sampling the original image signal to obtain a sampled image signal comprises:
and sampling the original image signal through a measurement matrix to obtain a sampled image signal.
3. The method according to claim 1, wherein initializing a reconstructed image signal based on the sampled image signal to obtain an initial reconstructed image signal comprises:
and initializing a reconstructed image signal based on the sampling image signal, the sparse dictionary and the measurement matrix to obtain an initial reconstructed image signal.
4. An image reconstruction apparatus based on a compressed sensing network, comprising:
the sampling unit is used for sampling the original image signal to obtain a sampled image signal;
the initialization unit is used for initializing a reconstructed image signal based on the sampling image signal to obtain an initial reconstructed image signal;
the reconstruction optimization unit is used for inputting the sampling image signal and the initial reconstruction image signal into a compressed sensing network, carrying out signal weighting through a multilayer convolution weighting module in the compressed sensing network to obtain multilayer reconstruction image signals, and optimizing the reconstruction image signals according to the correlation between two adjacent layers of reconstruction image signals to obtain final reconstruction image signals;
the training unit is used for calculating a loss value according to the final reconstructed image signal and the original image signal, and updating the network parameters of the compressive sensing network according to the loss value to obtain a trained compressive sensing network;
the reconstruction unit is used for carrying out image reconstruction on the image signal to be reconstructed through the trained compressed sensing network to obtain a reconstructed image signal;
the compressed sensing network is a multilayer iteration model, and the target function corresponding to each layer is as follows:
Figure 820269DEST_PATH_IMAGE001
；
in the formula (I), the compound is shown in the specification,X k() is a firstkThe layers reconstruct the image signal and,Yin order to sample the image signal, the image signal is sampled,Ain order to measure the matrix of the measurements,Din order to be a sparse dictionary,
Figure 961400DEST_PATH_IMAGE002
is a first non-negative regularization parameter,
Figure 272427DEST_PATH_IMAGE003
for the second non-negative regularization parameter,
Figure 106391DEST_PATH_IMAGE004
in order to be a convolution weighting module, the convolution weighting module,B、Ca filter with a convolution kernel size of 3*3,Relu() In order to modify the linear cell activation function,Fto predict the matrix, is a learnable parameter of the compressed sensing network,X k(-1) is as followsk-1 layer of reconstructed image signals;
the loss function of the compressed sensing network is as follows:
Figure 870079DEST_PATH_IMAGE005
；
in the formula (I), the compound is shown in the specification,N b as the total number of the sampled image signals,N p in order to compress the number of layers in the sensing network,Nin order to sample the size of the image signal,
Figure 814901DEST_PATH_IMAGE006
is as followsiThe final reconstructed image signal corresponding to each sampled image,S i is as followsiThe original image signal is converted into a digital image signal,W k() is as followskA layer convolution weighting module for weighting the layer convolution,
Figure 980434DEST_PATH_IMAGE007
is a firstkLayer andW k() a convolution weighting module for forming a symmetrical relation,
Figure 985299DEST_PATH_IMAGE008
is as followsiA reconstructed image signal is inkThe sparse signal of the layers is such that,
Figure 233353DEST_PATH_IMAGE009
is the third regularization parameter.
5. The device according to claim 4, wherein the sampling unit is specifically configured to:
and sampling the original image signal through a measurement matrix to obtain a sampled image signal.
6. The compressed sensing network-based image reconstruction device according to claim 4, wherein the initialization unit is specifically configured to:
and initializing a reconstructed image signal based on the sampling image signal, the sparse dictionary and the measurement matrix to obtain an initial reconstructed image signal.
CN202211264253.0A 2022-10-17 2022-10-17 Image reconstruction method and device based on compressed sensing network Active CN115330901B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211264253.0A CN115330901B (en) 2022-10-17 2022-10-17 Image reconstruction method and device based on compressed sensing network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211264253.0A CN115330901B (en) 2022-10-17 2022-10-17 Image reconstruction method and device based on compressed sensing network

Publications (2)

Publication Number Publication Date
CN115330901A CN115330901A (en) 2022-11-11
CN115330901B true CN115330901B (en) 2023-01-17

Family

ID=83915426

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211264253.0A Active CN115330901B (en) 2022-10-17 2022-10-17 Image reconstruction method and device based on compressed sensing network

Country Status (1)

Country Link
CN (1) CN115330901B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115861472B (en) * 2023-02-27 2023-05-23 广东工业大学 Image reconstruction method, device, equipment and medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102332153A (en) * 2011-09-13 2012-01-25 西安电子科技大学 Kernel regression-based image compression sensing reconstruction method
CN111667445A (en) * 2020-05-29 2020-09-15 湖北工业大学 Image compressed sensing reconstruction method based on Attention multi-feature fusion
WO2022166298A1 (en) * 2021-02-05 2022-08-11 歌尔股份有限公司 Image processing method and apparatus, and electronic device and readable storage medium

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112634391B (en) * 2020-12-29 2023-12-29 华中科技大学 Gray image depth reconstruction and fault diagnosis system based on compressed sensing
CN112991472B (en) * 2021-03-19 2023-12-19 华南理工大学 Image compressed sensing reconstruction method based on residual error dense threshold network
CN113052925A (en) * 2021-04-02 2021-06-29 广东工业大学 Compressed sensing reconstruction method and system based on deep learning

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102332153A (en) * 2011-09-13 2012-01-25 西安电子科技大学 Kernel regression-based image compression sensing reconstruction method
CN111667445A (en) * 2020-05-29 2020-09-15 湖北工业大学 Image compressed sensing reconstruction method based on Attention multi-feature fusion
WO2022166298A1 (en) * 2021-02-05 2022-08-11 歌尔股份有限公司 Image processing method and apparatus, and electronic device and readable storage medium

Also Published As

Publication number Publication date
CN115330901A (en) 2022-11-11

Similar Documents

Publication Publication Date Title
CN110807495B (en) Multi-label classification method, device, electronic equipment and storage medium
WO2023134086A1 (en) Convolutional neural network model pruning method and apparatus, and electronic device and storage medium
CN111954250B (en) Lightweight Wi-Fi behavior sensing method and system
CN111475622A (en) Text classification method, device, terminal and storage medium
CN115330901B (en) Image reconstruction method and device based on compressed sensing network
CN112883227B (en) Video abstract generation method and device based on multi-scale time sequence characteristics
Hosseiny et al. A hyperspectral anomaly detection framework based on segmentation and convolutional neural network algorithms
CN112786160A (en) Multi-image input multi-label gastroscope image classification method based on graph neural network
CN116665065A (en) Cross attention-based high-resolution remote sensing image change detection method
WO2020165490A1 (en) A method, an apparatus and a computer program product for video encoding and video decoding
Wolter et al. Wavelet-packet powered deepfake image detection
CN113850182A (en) Action identification method based on DAMR-3 DNet
CN113592769A (en) Abnormal image detection method, abnormal image model training method, abnormal image detection device, abnormal image model training device and abnormal image model training medium
Zhao et al. Fast hypergraph regularized nonnegative tensor ring decomposition based on low-rank approximation
Cang et al. Research on hyperspectral image reconstruction based on GISMT compressed sensing and interspectral prediction
CN114693547A (en) Radio frequency image enhancement method and radio frequency image identification method based on image super-resolution
Zou et al. Deep learning approach based on tensor-train for sparse signal recovery
WO2021179117A1 (en) Method and apparatus for searching number of neural network channels
CN107945131B (en) Distributed image reconstruction method for radio interference array
CN113299300A (en) Voice enhancement method, device and storage medium
KR20220023841A (en) Magnetic resonance image analysis system and method for alzheimer&#39;s disease classification
Cui et al. Deep Unfolding Network for Image Compressed Sensing by Content-adaptive Gradient Updating and Deformation-invariant Non-local Modeling
Jiang et al. Exploiting prior knowledge in compressed sensing to design robust systems for endoscopy image recovery
CN115797780A (en) Hyperspectral image classification method based on cross-level spectrum-space joint coding learning
CN117292209B (en) Video classification method and device based on space-time enhanced three-dimensional attention re-parameterization

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant