CN114529519A - Image compressed sensing reconstruction method and system based on multi-scale depth cavity residual error network - Google Patents
Image compressed sensing reconstruction method and system based on multi-scale depth cavity residual error network Download PDFInfo
- Publication number
- CN114529519A CN114529519A CN202210083515.7A CN202210083515A CN114529519A CN 114529519 A CN114529519 A CN 114529519A CN 202210083515 A CN202210083515 A CN 202210083515A CN 114529519 A CN114529519 A CN 114529519A
- Authority
- CN
- China
- Prior art keywords
- image
- reconstruction
- network
- residual error
- scale
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000005070 sampling Methods 0.000 claims abstract description 64
- 230000000694 effects Effects 0.000 claims abstract description 17
- 238000012549 training Methods 0.000 claims description 53
- 238000012360 testing method Methods 0.000 claims description 28
- 238000005457 optimization Methods 0.000 claims description 8
- 238000010606 normalization Methods 0.000 claims description 7
- 238000013480 data collection Methods 0.000 claims description 6
- 238000011156 evaluation Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 6
- 230000011218 segmentation Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 11
- 238000005259 measurement Methods 0.000 abstract description 8
- 238000003384 imaging method Methods 0.000 abstract description 6
- 239000011159 matrix material Substances 0.000 abstract description 3
- 238000012545 processing Methods 0.000 abstract description 2
- 239000010410 layer Substances 0.000 description 32
- 238000010586 diagram Methods 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 235000004257 Cordia myxa Nutrition 0.000 description 2
- 244000157795 Cordia myxa Species 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 239000002356 single layer Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10088—Magnetic resonance imaging [MRI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of image processing, and particularly relates to an image compressed sensing reconstruction method and system based on a multi-scale depth cavity residual error network, aiming at image data, a convolution sampling network is used for simulating a traditional compressed sensing measurement process to measure an image to obtain a measured value, and the measured value is subjected to a sub-pixel convolution network to complete the preliminary reconstruction from a measurement vector to an original signal; different cavity convolutions are used in the multi-scale depth cavity residual error network to carry out convolution learning on different scale characteristics, and the receptive field of the network is increased by fusing multi-scale information to capture more context information, so that the imaging effect is improved; and sampling is carried out by using a convolutional layer instead of a traditional random matrix method, so that the correlation between the measured value and the image is improved. The method can solve the problem of image block effect existing in the existing block sampling based method while having high reconstructed image quality, and is convenient for application in actual reconstructed image scenes.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to an image compressed sensing reconstruction method and system based on a multi-scale depth cavity residual error network.
Background
With the advent of the big data information age, the disadvantage of data sampling by means of the traditional sampling theorem becomes more and more obvious. Compressed sensing is an advanced data sampling theory, and sensing of a reconstructed signal is achieved on a low-dimensional measurement value based on compressibility of the signal. However, the conventional compressed sensing reconstruction algorithm based on iterative optimization has high time complexity, and the reconstruction effect is not ideal at a low sampling rate. With the development of deep learning, the time complexity of a reconstruction algorithm is greatly reduced and the reconstruction effect is improved by providing a compressed sensing model based on the deep learning.
Disclosure of Invention
Therefore, the invention provides an image compressed sensing reconstruction method and system based on a multi-scale depth cavity residual error network, wherein a convolution sampling network is used for simulating the traditional compressed sensing measurement process to measure an image to obtain a measured value, and the measured value is subjected to the primary reconstruction from a measurement vector to an original signal through a sub-pixel convolution network; different cavity convolution convolutions are used for the primary reconstruction image to learn different scale characteristics, the receptive field of the network is increased by fusing multi-scale information to capture more context information, the imaging effect is improved, and the practical scene application is facilitated.
According to the design scheme provided by the invention, the image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network comprises the following contents:
collecting image sample data, and dividing the data into a training set and a test set according to a preset proportion;
constructing a multi-scale depth cavity residual error network model for image compressed sensing reconstruction, and training and optimizing the model by using training set and test concentrated data, wherein the multi-scale depth cavity residual error network model comprises the following components: the system comprises a sampling network, an initial reconstruction network and a depth reconstruction network, wherein the sampling network is used for carrying out segmentation sampling on input image data according to a set sampling rate and generating a measured value corresponding to the input, the initial reconstruction network is used for carrying out image reconstruction according to the measured value output by the sampling network and acquiring an initial reconstruction image, and the depth reconstruction network is used for carrying out multi-scale cavity convolution on the initial reconstruction image acquired by the initial reconstruction network to acquire a depth reconstruction image;
and performing compressed sensing reconstruction on the target image data by using the trained and optimized multi-scale depth cavity residual error network model.
As the image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network, further, in the two branch networks, aiming at collected image sample data, data preprocessing is firstly carried out, the image data are adjusted to be uniform in pixel size, and then the image sample data are divided into a training set and a testing set according to a preset proportion.
As the image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network, the sampling network adopts convolution kernels with the size of BxBxl and the step length of BxB to carry out analog sampling and generate the image with the size of BxB
Wherein M, N is the input image data length and width dimensions.
As the image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network of the present invention, further, the initial reconstruction network includes: the system comprises a full connection layer for sampling nodes of measured values generated by a network to obtain the characteristics of the measured values of the input image data, and a sub-pixel volume block for scaling and sub-pixel reconstruction of the characteristics of the measured values of the input image data.
As the image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network, further, the depth reconstruction network comprises: the device comprises a plurality of cavity convolution branches for learning initial reconstruction image features of different sizes, a connecting layer for fusing and merging multi-scale feature information of the plurality of cavity convolution branches, and an output layer for performing convolution and batch normalization operation on the fused multi-scale feature information.
The image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network further adopts a residual error network structure, convolutes and learns the initial reconstruction image characteristics by utilizing the cavity convolution with different rates to acquire multi-scale characteristic information, and performs jump connection between the input and the output of the depth reconstruction network.
The image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network further utilizes an objective function
And utilizing Adam optimization algorithm to train and optimize the multi-scale depth cavity residual error network model, wherein n is the number of training image samples in a training set, and xiIn order to be the original image, the image is processed,
in order to reconstruct the image,
for gradient calculation, ωoAre subscripts.
The image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network further comprises the steps of performing model training learning by using image sample data in a training set, performing model test by using image sample data in a test set, and evaluating the performance of the model after the training test by using a preset evaluation index in the model training optimization.
As the image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network, further, the evaluation index includes: the peak signal-to-noise ratio PSNR and the structural similarity SSIM used for measuring the reconstruction effect of the model image are disclosed, wherein,
p is the number of bits per pixel, MSE represents the mean square error, μ, of the original image x and the reconstructed image yxIs the average value of the original image x, muyTo reconstruct the average value of the image y,
is the variance of x and is the sum of the differences,
variance of y, σxyIs the covariance of x, y, c1To maintain stable under-ripeness.
Further, the invention also provides an image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network, which comprises the following steps: a data collection module, a model training module, and an image reconstruction module, wherein,
the data collection module is used for collecting image sample data and dividing the data into a training set and a test set according to a preset proportion;
the model training module is used for constructing a multi-scale depth cavity residual error network model for image compressed sensing reconstruction and training and optimizing the model by utilizing the training set and the test concentrated data, wherein the multi-scale depth cavity residual error network model comprises the following components: the system comprises a sampling network, an initial reconstruction network and a depth reconstruction network, wherein the sampling network is used for carrying out segmentation sampling on input image data according to a set sampling rate and generating a measured value corresponding to the input, the initial reconstruction network is used for carrying out image reconstruction according to the measured value output by the sampling network and acquiring an initial reconstruction image, and the depth reconstruction network is used for carrying out multi-scale cavity convolution on the initial reconstruction image acquired by the initial reconstruction network to acquire a depth reconstruction image;
and the image reconstruction module is used for performing compressed sensing reconstruction on the target image data by utilizing the trained and optimized multi-scale depth cavity residual error network model.
The invention has the beneficial effects that:
(1) the invention constructs a depth cavity residual error network by using multi-scale cavity convolution. Different scale features are learned through different cavity convolution convolutions, then multi-scale information is fused, the receptive field of the network is increased to capture more context information, and therefore the imaging effect is improved. Meanwhile, the correlation of the measured value and the image is improved by using the convolution layer to replace the conventional random matrix method for sampling. Compared with the traditional compressed sensing algorithm based on block sampling, the method has high reconstructed image quality and solves the problem of image block effect existing in the traditional compressed sensing algorithm based on block sampling. The depth cavity residual error network is verified by experiments on MNIST, fast-MNIST and CelebA data, and the result shows that the method has a reconstruction effect superior to that of the current advanced depth compression sensing algorithm and can be applied to the field of medical MRI; compared with the traditional scanning method, the method for depth compressed sensing can greatly accelerate the imaging speed and shorten the scanning time. And the method has the advantages of obtaining accurate and efficient images and keeping image detail information while having low time cost. The scanning time of the patient is reduced, and meanwhile, the quick diagnosis of a doctor is facilitated.
(2) Empirical test data verifies that compared with a sub-pixel convolution anti-neural network SCGAN, the network structure model used by the invention improves the PSNR on average by 5.9983dB and the SSIM on average by 0.0963 on the MNIST data set. The PSNR increased on average 3.1131dB and the SSIM increased on average 0.0447 on the fast-MNIST data set. The PSNR was improved by 4.029dB on average and SSIM by 0.1683 on average on the CelebA dataset.
Description of the drawings:
FIG. 1 is a schematic diagram of an image compressed sensing reconstruction process based on a multi-scale depth cavity residual error network in an embodiment;
FIG. 2 is a schematic diagram of a multi-scale depth cavity residual error network structure in an embodiment;
FIG. 3 is a schematic diagram of a sampling network and an initial reconstruction network in the embodiment;
fig. 4 is a schematic diagram of a deep reconstruction network structure in the embodiment.
The specific implementation mode is as follows:
in order to make the objects, technical solutions and advantages of the present invention clearer and more obvious, the present invention is further described in detail below with reference to the accompanying drawings and technical solutions.
The embodiment of the invention provides an image compressed sensing reconstruction method based on a multi-scale depth cavity residual error network, which is shown in a figure 1 and comprises the following contents:
s101, collecting image sample data, and dividing the data into a training set and a test set according to a preset proportion;
s102, constructing a multi-scale depth cavity residual error network model for image compressed sensing reconstruction, and training and optimizing the model by using training set and test concentrated data, wherein the multi-scale depth cavity residual error network model comprises: the system comprises a sampling network, an initial reconstruction network and a depth reconstruction network, wherein the sampling network is used for carrying out segmentation sampling on input image data according to a set sampling rate and generating a measured value corresponding to the input, the initial reconstruction network is used for carrying out image reconstruction according to the measured value output by the sampling network and acquiring an initial reconstruction image, and the depth reconstruction network is used for carrying out multi-scale cavity convolution on the initial reconstruction image acquired by the initial reconstruction network to acquire a depth reconstruction image;
s103, carrying out compressed sensing reconstruction on the target image data by using the trained and optimized multi-scale depth cavity residual error network model.
In the embodiment of the scheme, referring to fig. 2, a convolution sampling network is used for simulating a traditional compressed sensing measurement process to measure an image to obtain a measured value, and the measured value is subjected to a sub-pixel convolution network to complete the primary reconstruction from a measurement vector to an original signal; different hole convolutions are used for preliminarily reconstructed images to learn different scale characteristics through convolution, the receptive field of the network is increased by fusing multi-scale information to capture more context information, the imaging effect is improved, and the practical scene application is facilitated.
As the image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network in the embodiment of the invention, further, in the two branch networks, aiming at the collected image sample data, data preprocessing is firstly carried out, the image data is adjusted to be uniform in pixel size, and then the image sample data is divided into a training set and a testing set according to a preset proportion.
202, 599 images with 178 × 218 sizes of CelebA in the public training data set can be respectively used as 162, 770 training sets, 19, 867 training sets and 19, 961 testing sets. Before training, each batch size resizes the picture to 64 × 64 pixels.
As the image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network in the embodiment of the invention, the sampling network adopts convolution kernels with the size of BxBxl and the step length of BxB to perform analog sampling and generate the image compressed sensing reconstruction method with the size of BxB
Wherein M, N is the input image data length and width dimensions. Further, the initial reestablishment network includes: the system comprises a full connection layer for sampling nodes of measured values generated by a network to obtain the characteristics of the measured values of the input image data, and a sub-pixel volume block for scaling and sub-pixel reconstruction of the characteristics of the measured values of the input image data.
The sampling network uses convolutional layers for sampling, and the process can be expressed as ymeas＝ωconvX, wherein ymeasAs a measured value, ωconvThe convolution operation is the weight of the convolution layer, and x is the original image, and the process is shown in fig. 3. The sampling network is composed of 32 single-layer convolutional layers with 32 multiplied by 32 convolutional kernel size and 32 step length. Wherein, the number of output channels is set according to the sampling rate. The reconstruction network receives the measurements generated by the sampling network and transforms the image size to 16 x 64 through a fully connected layer of 16384 nodes. The first hidden layer is a sub-pixel convolution block, and the process of sub-pixel convolution can be expressed as:
wherein T is the pixel of the image, x, y are the coordinates of the image, r is the scaling, c is the number of channels,
is a periodic pixel recombination operation. By this process, H × W × Cr2The image size of the size is increased to Hr × Wr × C. The sub-pixel convolution block is composed of a convolution layer of size 3 × 3, a batch normalization layer, a SELU activation function layer, a sub-pixel convolution layer, and a SELU activation function layer, and its structure can be expressed as [ Conv3×3-BN-SeLU-subpixelConv3×3]The number of output channels is 512. The second and third hidden layers are composed of 3 × 3 convolution layer, batch normalization layer, and activation function layer. The structure of each of which can be expressed as [ Conv3×3-BN-SeLU]And (c). [ Conv ]3×3-tanh-BN-SeLU]The number of output channels is 32 and 3 respectively, and the step length is 1 and 1.
As an image compressed sensing reconstruction method based on a multi-scale depth cavity residual error network in the embodiment of the present invention, further, the depth reconstruction network includes: the device comprises a plurality of cavity convolution branches for learning the initial reconstruction image characteristics with different sizes, a connecting layer for fusing and combining multi-scale characteristic information of the plurality of cavity convolution branches, and an output layer for performing convolution and batch normalization operation on the fused multi-scale characteristic information. Further, the depth reconstruction network adopts a residual error network structure, convolution learning of initial reconstruction image features is performed by utilizing cavity convolution of different sizes to obtain multi-scale feature information, and jumping connection is performed between input and output of the depth reconstruction network. The process of learning the multi-scale feature information by the void convolution can be expressed as follows:
i represents the number of network layers, IiAnd for the output of the i-th network, omega is the weight of the hole convolution, the subscript of the weight is the mark of the hole convolution with different scales, represents the convolution operation, and + represents the connection of the feature maps according to the channels.
Deep reconstruction network receiving initial reconstruction networkAnd (5) connecting the initially reconstructed image and performing depth reconstruction. Referring to fig. 4, the deep reconstruction network is composed of 6 hole convolution blocks, the multi-scale hole convolution block is three branches, each branch is composed of a hole convolution layer and a batch normalization layer, and the structure of the deep reconstruction network can be represented as [ AtrousConv [ ]3×3-SeLU-BN]The hole convolution rates of the three branches are respectively 1, 2 and 3, and the number of channels is 8. And the branches pass through the concat layer to sequentially merge the multi-branch results according to the dimensionality. Merging and entering the next hidden layer, which is composed of a convolution layer and a batch normalization layer and can be expressed as [ Conv3×3-SeLU-BN]The number of channels is 24 and the step size is 1. After passing through the hollow convolution block, the signal enters the next hidden layer and is composed of 3 multiplied by 3 convolution layers, the number of channels is 3, and the step length is 1. The residual network is implemented by making a jump connection between the input and the output of the deep reconstruction network.
The image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network further utilizes an objective function
And utilizing Adam optimization algorithm to train and optimize the multi-scale depth cavity residual error network model, wherein n is the number of training image samples in a training set, and xiIn order to be the original image, the image is processed,
in order to reconstruct the image,
for gradient calculation, ωoAre subscripts.
As the image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network in the embodiment of the invention, further, in the optimization of model training, model training learning is performed by using image sample data in a training set, model testing is performed by using image sample data in a testing set, and the performance of the model after the training test is evaluated by using a preset evaluation index.
In the test process, the size of the batch size can be set to be 8, and the hyper-parameters and the initial learning rate of the network structure are setThe learning rate is 0.01, and the number of iterations of the depth hole residual error network is 20. Target loss function for network training:
where n is the number of training images in the training set, xiIs a function of the original image and is,
to reconstruct an image. Parameters within the partial convolution generation countermeasure network are trained and updated by utilizing an Adam optimization algorithm.
And setting the number of channels of the sampling convolutional layer according to the sampling rate to train image data sets with different sampling rates, obtaining trained compressed sensing reconstruction models with different sampling rates by training the optimal parameters of the learning network model, and storing the models, wherein the storage format can be npz. And in the testing process, the performance of the network is verified by using the peak signal to noise ratio (PSNR) and the Structural Similarity (SSIM) of the evaluation indexes.
The reconstruction effect of the network model is measured by using the peak signal-to-noise ratio (PSNR), wherein the larger the PSNR, the better the reconstruction effect is, and the calculation formula is as follows:
n is the number of bits per pixel, and is generally 8, i.e., the number of pixel gray levels is 256, and the unit is dB. Mean square error
MSE represents the current image
And the mean square error of the reference image f (i, j); m, N are the height and width of the image, respectively.
The reconstruction effect of the network model is measured by using SSIM, wherein the larger SSIM indicates the better reconstruction effect, and the SSIM calculation formula of the two images is as follows for the given images x and y:
wherein muxIs the average value of x, μyIs the average value of y.
Is the variance of x and is the sum of the differences,
variance of y, σxyIs the covariance of x, y, c1＝(k1L)2，c2＝(k2L)2Is used to maintain a stable constant, L is the dynamic range of the pixel value, k1＝0.01，k2＝0.03。
According to the embodiment of the scheme, in the multi-scale depth cavity residual error network, different cavity convolutions are used for convolution learning of different scale characteristics, then multi-scale information is fused, the receptive field of the network is increased to capture more context information, and therefore the imaging effect is improved. Meanwhile, the correlation of the measured value and the image is improved by using the convolution layer to replace the conventional random matrix method for sampling. Compared with the traditional block sampling-based compressed sensing algorithm, the scheme of the scheme has high reconstructed image quality, solves the problem of the image block effect existing in the prior block sampling-based compressed sensing algorithm, and is convenient for application in the actual reconstructed image scene.
Further, based on the above method, an embodiment of the present invention further provides an image compressed sensing reconstruction method based on a multi-scale depth cavity residual error network, including: a data collection module, a model training module, and an image reconstruction module, wherein,
the data collection module is used for collecting image sample data and dividing the data into a training set and a test set according to a preset proportion;
the model training module is used for constructing a multi-scale depth cavity residual error network model for image compressed sensing reconstruction and training and optimizing the model by utilizing the training set and the test concentrated data, wherein the multi-scale depth cavity residual error network model comprises the following components: the system comprises a sampling network, an initial reconstruction network and a depth reconstruction network, wherein the sampling network is used for carrying out segmentation sampling on input image data according to a set sampling rate and generating a measured value corresponding to the input, the initial reconstruction network is used for carrying out image reconstruction according to the measured value output by the sampling network and acquiring an initial reconstruction image, and the depth reconstruction network is used for carrying out multi-scale cavity convolution on the initial reconstruction image acquired by the initial reconstruction network to acquire a depth reconstruction image;
and the image reconstruction module is used for performing compressed sensing reconstruction on the target image data by utilizing the trained and optimized multi-scale depth cavity residual error network model.
Unless specifically stated otherwise, the relative steps, numerical expressions and values of the components and steps set forth in these embodiments do not limit the scope of the present invention.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. An image compressed sensing reconstruction method based on a multi-scale depth cavity residual error network is characterized by comprising the following contents:
collecting image sample data, and dividing the data into a training set and a test set according to a preset proportion;
constructing a multi-scale depth cavity residual error network model for image compressed sensing reconstruction, and training and optimizing the model by using training set and test concentrated data, wherein the multi-scale depth cavity residual error network model comprises the following components: the system comprises a sampling network, an initial reconstruction network and a depth reconstruction network, wherein the sampling network is used for carrying out segmentation sampling on input image data according to a set sampling rate and generating a measured value corresponding to the input, the initial reconstruction network is used for carrying out image reconstruction according to the measured value output by the sampling network and acquiring an initial reconstruction image, and the depth reconstruction network is used for carrying out multi-scale cavity convolution on the initial reconstruction image acquired by the initial reconstruction network to acquire a depth reconstruction image;
and performing compressed sensing reconstruction on the target image data by using the trained and optimized multi-scale depth cavity residual error network model.
2. The method according to claim 1, wherein for the collected image sample data, data preprocessing is performed first, the image data is adjusted to a uniform pixel size, and then the image sample data is divided into a training set and a test set according to a preset ratio.
3. The method according to claim 1, wherein the sampling network performs analog sampling by using a convolution kernel with a size of BxBxl and a step size of BxB, and generates the compressed image with a size of BxB
Wherein M, N is the input image data length and width dimensions.
4. The method according to claim 1, wherein the initial reconstruction network comprises: the system comprises a full connection layer for sampling nodes of measured values generated by a network to obtain the characteristics of the measured values of the input image data, and a sub-pixel volume block for scaling and sub-pixel reconstruction of the characteristics of the measured values of the input image data.
5. The method for image compressed sensing reconstruction based on multi-scale depth hole residual error network according to claim 1, wherein the depth reconstruction network comprises: the device comprises a plurality of cavity convolution branches for learning the initial reconstruction image characteristics with different sizes, a connecting layer for fusing and combining multi-scale characteristic information of the plurality of cavity convolution branches, and an output layer for performing convolution and batch normalization operation on the fused multi-scale characteristic information.
6. The image compressed sensing reconstruction method based on the multi-scale depth cavity residual error network according to claim 1 or 5, characterized in that the depth reconstruction network adopts a residual error network structure, convolutes and learns the initial reconstructed image characteristics by utilizing cavity convolutions with different rates to acquire multi-scale characteristic information, and performs jump connection between the input and the output of the depth reconstruction network.
7. The method for image compressive sensing reconstruction based on multi-scale depth hole residual error network as claimed in claim 1, wherein an objective function is utilized
And utilizing Adam optimization algorithm to train and optimize the multi-scale depth cavity residual error network model, wherein n is the number of training image samples in a training set, and xiIn order to be the original image, the image is processed,
in order to reconstruct the image,
for gradient calculation, ωoAre subscripts.
8. The method for reconstructing image compressed sensing based on multi-scale depth cavity residual error network according to claim 1 or 7, wherein in the optimization of model training, model training learning is performed by using image sample data in a training set, model testing is performed by using image sample data in a testing set, and model performance after the training testing is evaluated by using a preset evaluation index.
9. The method according to claim 8, wherein the evaluation index comprises: the peak signal-to-noise ratio PSNR and the structural similarity SSIM used for measuring the reconstruction effect of the model image are disclosed, wherein,
p is the number of bits per pixel, MSE represents the mean square error, μ, of the original image x and the reconstructed image yxIs the average value of the original image x, muyTo reconstruct the average value of the image y,
is the variance of x and is the sum of the differences,
variance of y, σxyIs the covariance of x, y, c1To maintain stable under-ripeness.
10. An image compressed sensing reconstruction system based on a multi-scale depth cavity residual error network is characterized by comprising: a data collection module, a model training module, and an image reconstruction module, wherein,
the data collection module is used for collecting image sample data and dividing the data into a training set and a test set according to a preset proportion;
the model training module is used for constructing a multi-scale depth cavity residual error network model for image compressed sensing reconstruction and training and optimizing the model by utilizing the training set and the test concentrated data, wherein the multi-scale depth cavity residual error network model comprises the following components: the system comprises a sampling network, an initial reconstruction network and a depth reconstruction network, wherein the sampling network is used for carrying out segmentation sampling on input image data according to a set sampling rate and generating a measured value corresponding to the input, the initial reconstruction network is used for carrying out image reconstruction according to the measured value output by the sampling network and acquiring an initial reconstruction image, and the depth reconstruction network is used for carrying out multi-scale cavity convolution on the initial reconstruction image acquired by the initial reconstruction network to acquire a depth reconstruction image;
and the image reconstruction module is used for performing compressed sensing reconstruction on the target image data by utilizing the trained and optimized multi-scale depth cavity residual error network model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210083515.7A CN114529519B (en) | 2022-01-25 | Image compressed sensing reconstruction method and system based on multi-scale depth cavity residual error network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210083515.7A CN114529519B (en) | 2022-01-25 | Image compressed sensing reconstruction method and system based on multi-scale depth cavity residual error network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114529519A true CN114529519A (en) | 2022-05-24 |
| CN114529519B CN114529519B (en) | 2024-07-12 |
Family
ID=
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116452696A (en) * | 2023-06-16 | 2023-07-18 | 山东省计算中心（国家超级计算济南中心） | Image compressed sensing reconstruction method and system based on double-domain feature sampling |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020056791A1 (en) * | 2018-09-21 | 2020-03-26 | 五邑大学 | Method and apparatus for super-resolution reconstruction of multi-scale dilated convolution neural network |
| CN111667444A (en) * | 2020-05-29 | 2020-09-15 | 湖北工业大学 | Image compressed sensing reconstruction method based on multi-channel residual error network |
| WO2021169408A1 (en) * | 2020-02-26 | 2021-09-02 | 华为技术有限公司 | Image processing method and apparatus, and electronic device and storage medium |
| CN113793263A (en) * | 2021-08-23 | 2021-12-14 | 电子科技大学 | Parallel residual error network high-resolution image reconstruction method for multi-scale cavity convolution |
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020056791A1 (en) * | 2018-09-21 | 2020-03-26 | 五邑大学 | Method and apparatus for super-resolution reconstruction of multi-scale dilated convolution neural network |
| WO2021169408A1 (en) * | 2020-02-26 | 2021-09-02 | 华为技术有限公司 | Image processing method and apparatus, and electronic device and storage medium |
| CN111667444A (en) * | 2020-05-29 | 2020-09-15 | 湖北工业大学 | Image compressed sensing reconstruction method based on multi-channel residual error network |
| CN113793263A (en) * | 2021-08-23 | 2021-12-14 | 电子科技大学 | Parallel residual error network high-resolution image reconstruction method for multi-scale cavity convolution |
Non-Patent Citations (2)
| Title |
|---|
| WENZHE SHI等: "Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network", ARXIV, 23 September 2016 (2016-09-23) * |
| 张焯林;赵建伟;曹飞龙;: "构建带空洞卷积的深度神经网络重建高分辨率图像", 模式识别与人工智能, no. 03, 15 March 2019 (2019-03-15) * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116452696A (en) * | 2023-06-16 | 2023-07-18 | 山东省计算中心（国家超级计算济南中心） | Image compressed sensing reconstruction method and system based on double-domain feature sampling |
| CN116452696B (en) * | 2023-06-16 | 2023-08-29 | 山东省计算中心（国家超级计算济南中心） | Image compressed sensing reconstruction method and system based on double-domain feature sampling |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112116601B (en) | Compressed sensing sampling reconstruction method and system based on generation of countermeasure residual error network | |
| CN110473196B (en) | Abdomen CT image target organ registration method based on deep learning | |
| CN108090871A (en) | A kind of more contrast MR image reconstruction methods based on convolutional neural networks | |
| CN112150568A (en) | Magnetic resonance fingerprint imaging reconstruction method based on Transformer model | |
| CN106780372A (en) | A kind of weight nuclear norm magnetic resonance imaging method for reconstructing sparse based on Generalized Tree | |
| CN111870245B (en) | Cross-contrast-guided ultra-fast nuclear magnetic resonance imaging deep learning method | |
| CN114359283A (en) | Defect detection method based on Transformer and electronic equipment | |
| CN107341776A (en) | Single frames super resolution ratio reconstruction method based on sparse coding and combinatorial mapping | |
| CN110297041A (en) | A kind of 3D woven composite defect inspection method based on FCN and GRU | |
| CN112991483B (en) | Non-local low-rank constraint self-calibration parallel magnetic resonance imaging reconstruction method | |
| CN113850883A (en) | Magnetic particle imaging reconstruction method based on attention mechanism | |
| CN114170088A (en) | Relational reinforcement learning system and method based on graph structure data | |
| CN113269256A (en) | Construction method and application of Misrc-GAN model | |
| CN113870327B (en) | Medical image registration method based on prediction multi-level deformation field | |
| CN117333750A (en) | Spatial registration and local global multi-scale multi-modal medical image fusion method | |
| CN111598966B (en) | Magnetic resonance imaging method and device based on generation countermeasure network | |
| CN116228520A (en) | Image compressed sensing reconstruction method and system based on transform generation countermeasure network | |
| CN116993639A (en) | Visible light and infrared image fusion method based on structural re-parameterization | |
| CN114972332B (en) | Bamboo laminated wood crack detection method based on image super-resolution reconstruction network | |
| CN114529519B (en) | Image compressed sensing reconstruction method and system based on multi-scale depth cavity residual error network | |
| CN114529519A (en) | Image compressed sensing reconstruction method and system based on multi-scale depth cavity residual error network | |
| CN115641263A (en) | Single-power equipment infrared image super-resolution reconstruction method based on deep learning | |
| CN113487507A (en) | Dual-domain recursive network MR reconstruction method based on multi-module feature aggregation | |
| CN114529737A (en) | Optical red footprint image contour extraction method based on GAN network | |
| CN114022362A (en) | Image super-resolution method based on pyramid attention mechanism and symmetric network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |