Image denoising method based on residual learning and convolutional neural network
Technical Field
The invention relates to an image processing technology, in particular to an image denoising method based on residual learning and a convolutional neural network.
Background
In the process of image acquisition, coding and transmission, some noises are introduced for various reasons, which leads to serious reduction of image quality, so that the image is denoised, and the improvement of image quality is an important research part in the image processing technology. Through some noise processing technologies, image noise is removed as much as possible, so that information is obtained more efficiently, and further processing such as feature extraction, signal detection and image compression is facilitated for the image. However, some conventional denoising methods always cause loss of information in the transmission process, and when a convolutional neural network is used for image processing operation, preprocessing operation required for inputting information is greatly relaxed, so that the processing process is simplified. The most key in the Convolutional Neural Network (CNN) is convolution operation, which makes full use of the information of adjacent areas in the picture and greatly reduces the scale of the parameter matrix by means of sparse connection and weight sharing.
However, the convolutional neural network cannot well maintain the image structure information, and in the process of deepening the network depth, the training difficulty is increased and the training effect is reduced, so that research on related contents is urgently needed to improve the maintenance effect of the convolutional neural network structure information.
Disclosure of Invention
The purpose of the invention is as follows: the invention aims to provide an image denoising method based on residual error learning and a convolutional neural network, which can increase the stability of network training, avoid information loss and loss in the transmission process and improve the depth of a convolutional neural network denoising model and the structural information retention effect.
The technical scheme is as follows: the invention discloses an image denoising method based on residual error learning and a convolutional neural network, which comprises the following steps of:
(1) denoising the input noisy image by adopting a convolutional neural network method, and constructing a convolutional neural network model comprising a plurality of convolutional layers;
(2) denoising an input noisy image by adopting a residual error learning method, superposing a network mapping layer on a neural network, and equating an original deep neural network model to a shallow neural network model;
(3) denoising the noisy image by adopting an image denoising method combining a convolutional neural network method and a residual error learning method, adding padding in the convolutional neural network, performing batch normalization operation, adding a shallow-to-deep crossing connection structure in the obtained convolutional neural network, and then training by adopting the convolutional neural network of an Adam algorithm;
(4) and outputting the denoised image.
In the step (2), the residual function formula of the network mapping layer is as follows:
F(x)=H(x)-x
wherein, F(x)Representing a residual function, H(x)Representing a network mapping function, and x representing an identity mapping layer; the depth of the image can be deepened, and the phenomenon of degradation can not occur.
In the step (3), padding is added to the convolutional neural network to fill zero around the image block; not only can prevent the size of the output characteristic graph from changing, but also can repeatedly utilize the edge information.
In the step (3), the normalized specific calculation method is as follows:
wherein, aiIs a certain neuron's original activation value, ai normIs a normalized value, gamma, after a normalization operationi、βiTwo regulating factors of the neuron in the training process are adopted, the regulating factors slightly regulate the numerical values which are normalized to have a mean value of 0 and a variance of 1, and then a condition that the mean value is βiVariance is gammai 2The distribution value, thereby preventing the interference of the internal covariates.
When the batch of normalized training networks is carried out, a neuron set S is selected firstly, and then the mean value and the variance of the neuron activation values are calculated, wherein the specific formula is as follows:
k is equal to S and | | | | S | | | | | m
Wherein m is the size of the set S, and epsilon is a positive number with a numerical value close to zero; avoiding the situation that the gradient is not defined at a certain position, and after the normalization operation, the output result can be normalized to be in the range no matter what the change of the neuron occurs.
In the step (2), the convolutional neural network adopting the Adam algorithm is used for training, specifically, the Adam algorithm is used for updating the weight, the estimation with the deviation is calculated, the estimation after the deviation correction is calculated later is improved, and the parameters in the Adam algorithm are set as default values; the algorithm parameter adjustment process is simple, and most problems in the denoising process can be solved.
And each layer of convolution kernels of the convolution layers in the convolution neural network model is identical in number and size.
And the convolution layer in the convolutional neural network model adopts a ReLU activation function.
In the step (1), the original image is subjected to gray scale processing before the noisy image is input, and is converted into a gray scale image, and the gray scale image is subjected to noise processing to obtain a noisy image and then is input.
The noise adding process is to add Gaussian noise with a fixed variance value in the gray level image, so that the noise removing effect is better.
Has the advantages that: compared with the prior art, the invention has the beneficial effects that: (1) loss and loss of information in the transmission process are avoided, and the depth of a convolutional neural network denoising model and the keeping effect of structural information are improved; (2) the stability of network training is increased; (3) enhancing the sensitivity of the convolutional neural network to the shallow and deep features of the image and improving the learning effect of the drying mapping; (4) the sizes of the feature images after convolution are not changed, and images with the same size are finally output.
Drawings
FIG. 1 is a flowchart of an image denoising method based on residual learning and convolutional neural network according to the present invention;
FIG. 2 is a frame diagram of the image denoising method based on residual learning and convolutional neural network according to the present invention;
FIG. 3 is a gray scale image obtained by performing gray scale processing on an original image according to the present invention;
FIG. 4 is a noisy image obtained after a noise process is applied to a gray scale image in accordance with the present invention;
FIG. 5 is a diagram showing a structure of a residual error unit according to the present invention;
FIG. 6 is a diagram illustrating the effect of denoising a noisy image according to the present invention;
Detailed Description
The invention is described in further detail below with reference to specific embodiments and the attached drawing figures.
As shown in fig. 1, the image denoising method based on residual learning and convolutional neural network of the present invention includes the following steps:
(1) denoising the input noisy image by adopting a convolutional neural network method, and constructing a convolutional neural network model comprising a plurality of convolutional layers; the convolution kernels of each layer of convolution layers in the convolution neural network model are identical in number and size, convolution is carried out by adopting convolution kernels of 3 x 3, and the number of the convolution kernels of each layer is 64; a convolution layer in the convolutional neural network model adopts a ReLU activation function;
(2) denoising an input noisy image by adopting a residual error learning method, superposing a network mapping layer on a neural network, and equating an original deep neural network model to a shallow neural network model, so that the depth is deepened, and the image is not degraded;
(3) denoising the noisy image by adopting an image denoising method combining a convolutional neural network method and a residual error learning method, adding padding in the convolutional neural network, performing batch normalization operation, adding a shallow-to-deep crossing connection structure in the obtained convolutional neural network, and then training by adopting the convolutional neural network of an Adam algorithm;
the padding added in the convolutional neural network is zero padding around the image block, so that the size change of the output feature map can be prevented, and the edge information can be used for multiple times. Because the size of the adopted convolution kernel is 3 multiplied by 3, and padding with the size of 1 is added to each layer, the size of the feature image after convolution is unchanged, and finally the image with the same size can be output;
the specific calculation method of the normalization is as follows:
wherein, a
iFor a certain value of the original activation of a neuron,
is a normalized value, gamma, after a normalization operation
i、β
iTwo regulating factors of the neuron in the training process are adopted, the regulating factors slightly regulate the numerical values which are normalized to have a mean value of 0 and a variance of 1, and then a condition that the mean value is β
iVariance is gamma
i 2The value of the distribution, thereby preventing interference of the internal covariates;
when the training network is normalized in batches, a neuron set S is selected firstly, and then the mean value and the variance of the neuron activation value are calculated, wherein the specific formula is as follows:
k is equal to S and | | | | S | | | | | m
Wherein m is the size of the set S, and epsilon is a positive number with a numerical value close to zero; avoiding the situation that the gradient is not defined at a certain position;
after standardized operation, no matter what the neuron changes, the output result is normalized to the range, in addition, the translation parameter and the scaling parameter can be self-adaptively learned, the introduced parameters are few, and the parameters can be updated by using back propagation, so that the convergence speed of neural network training can be accelerated to a certain extent, and the degree of dependence on the network parameter initialization process is reduced;
as shown in fig. 2, the head-to-tail crossing connection from the shallow layer to the deep layer is added to the convolutional neural network, so that the problem that the shallow image features are lost due to the higher-dimensional feature image obtained after the input image is subjected to multiple convolution operations in the forward propagation process in the network can be avoided, the shallow information of the image can be retained, the sensitivity of the convolutional neural network to the shallow and deep image features can be enhanced, and the learning effect of denoising mapping can be improved; in addition, the convolutional neural network adopting the Adam algorithm is trained, specifically, the weight is updated by adopting the Adam algorithm, the estimation with deviation is calculated, the estimation after the deviation correction is calculated later is improved, and the parameters in the Adam algorithm are set as default values;
(4) and outputting the denoised image.
As shown in fig. 2, the convolutional neural network model of the present invention only includes a plurality of convolutional layers, because when denoising is performed using the convolutional neural network, the dimensionality reduction process of the pooling operation filters out part of the features, which adversely affects the denoising, and therefore, the network model of the present invention removes the pooling layer and only retains the convolutional layers.
As shown in fig. 3 and 4, in step (1), before the noisy image is input, the original image is subjected to gray scale processing to be converted into a gray scale image, and the gray scale image is subjected to noise processing to obtain a noisy image, and then the noisy image is input into each model to be subjected to noise removal processing respectively. In this embodiment, the noise addition process is to add gaussian noise having a variance value of 25 to the grayscale image.
In order to increase the depth of the network and ensure that the network does not degenerate, the residual is used in deep learning, an identity mapping layer (y-x layer) is superimposed on a neural network, and the original deep neural network model is equivalent to a shallow neural network, so that the depth is deepened, and the phenomenon of degeneration does not occur. Since it is not easy to directly implement an identity mapping function h (x) ═ x in the neural network layer, the mapping of the network is changed to h (x) ═ f (x) + x, as shown in fig. 5, the model includes two mapping, one is identification mapping performed by using a short connection structure on the right side, and the other is residual part f (x) of residual mapping, and the model can convert the problem into learning a residual function, so the residual function formula of the network mapping layer in step (2) is expressed as:
F(x)=H(x)-x
wherein, F(x)Representing a residual function, H(x)Representing the network mapping function and x representing the identity mapping layer.
The present invention is compared with a three-dimensional block matching algorithm (BM3D), a convolutional neural network denoising method including 5 convolutional layers (novel method 1), and a convolutional neural network denoising method including 19 convolutional layers (novel method 2):
in the above four comparison methods, the sizes of the convolution kernels are all 3 × 3. After the same group of gray level images are denoised, the peak signal-to-noise ratio (PSNR) and the used time are compared, and the results clearly show that the PSNR is obviously improved and the consumed time is obviously reduced by adopting the denoising method disclosed by the invention, so that the image denoising method based on residual learning and the convolutional neural network has the best denoising effect, as shown in fig. 6, the residual learning is combined into the convolutional neural network, the network depth can be expanded, the high-dimensional characteristic is extracted, the network degradation and the loss and loss of information in the transmission project are effectively avoided, the noise elimination is considered, and the depth of the denoising model of the convolutional neural network and the maintenance effect of structural information are improved.