CN111145125A - Image denoising method based on residual learning and convolutional neural network - Google Patents

Image denoising method based on residual learning and convolutional neural network Download PDF

Info

Publication number
CN111145125A
CN111145125A CN201911391449.4A CN201911391449A CN111145125A CN 111145125 A CN111145125 A CN 111145125A CN 201911391449 A CN201911391449 A CN 201911391449A CN 111145125 A CN111145125 A CN 111145125A
Authority
CN
China
Prior art keywords
neural network
convolutional neural
image
denoising
residual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911391449.4A
Other languages
Chinese (zh)
Inventor
周先春
吴静
翟靖宇
徐新菊
葛超
吴婷
陈铭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Information Science and Technology
Original Assignee
Nanjing University of Information Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Information Science and Technology filed Critical Nanjing University of Information Science and Technology
Priority to CN201911391449.4A priority Critical patent/CN111145125A/en
Publication of CN111145125A publication Critical patent/CN111145125A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses an image denoising method based on residual error learning and a convolutional neural network, which comprises the following steps: firstly, denoising an input noisy image by respectively adopting a convolutional neural network method and a residual error learning method; denoising the noisy image by adopting a method combining a convolutional neural network method and a residual error learning method, adding padding in the convolutional neural network, carrying out batch standardization operation, adding a shallow-to-deep spanning connection structure in the network, and training by adopting a convolutional neural network of an Adam algorithm; and finally, outputting the denoised image. The method of the invention can not only expand the network depth, effectively avoid network degradation and information loss and loss in the transmission project, and improve the depth of the denoising model of the convolutional neural network and the retention effect of the structural information.

Description

Image denoising method based on residual learning and convolutional neural network
Technical Field
The invention relates to an image processing technology, in particular to an image denoising method based on residual learning and a convolutional neural network.
Background
In the process of image acquisition, coding and transmission, some noises are introduced for various reasons, which leads to serious reduction of image quality, so that the image is denoised, and the improvement of image quality is an important research part in the image processing technology. Through some noise processing technologies, image noise is removed as much as possible, so that information is obtained more efficiently, and further processing such as feature extraction, signal detection and image compression is facilitated for the image. However, some conventional denoising methods always cause loss of information in the transmission process, and when a convolutional neural network is used for image processing operation, preprocessing operation required for inputting information is greatly relaxed, so that the processing process is simplified. The most key in the Convolutional Neural Network (CNN) is convolution operation, which makes full use of the information of adjacent areas in the picture and greatly reduces the scale of the parameter matrix by means of sparse connection and weight sharing.
However, the convolutional neural network cannot well maintain the image structure information, and in the process of deepening the network depth, the training difficulty is increased and the training effect is reduced, so that research on related contents is urgently needed to improve the maintenance effect of the convolutional neural network structure information.
Disclosure of Invention
The purpose of the invention is as follows: the invention aims to provide an image denoising method based on residual error learning and a convolutional neural network, which can increase the stability of network training, avoid information loss and loss in the transmission process and improve the depth of a convolutional neural network denoising model and the structural information retention effect.
The technical scheme is as follows: the invention discloses an image denoising method based on residual error learning and a convolutional neural network, which comprises the following steps of:
(1) denoising the input noisy image by adopting a convolutional neural network method, and constructing a convolutional neural network model comprising a plurality of convolutional layers;
(2) denoising an input noisy image by adopting a residual error learning method, superposing a network mapping layer on a neural network, and equating an original deep neural network model to a shallow neural network model;
(3) denoising the noisy image by adopting an image denoising method combining a convolutional neural network method and a residual error learning method, adding padding in the convolutional neural network, performing batch normalization operation, adding a shallow-to-deep crossing connection structure in the obtained convolutional neural network, and then training by adopting the convolutional neural network of an Adam algorithm;
(4) and outputting the denoised image.
In the step (2), the residual function formula of the network mapping layer is as follows:
F(x)=H(x)-x
wherein, F(x)Representing a residual function, H(x)Representing a network mapping function, and x representing an identity mapping layer; the depth of the image can be deepened, and the phenomenon of degradation can not occur.
In the step (3), padding is added to the convolutional neural network to fill zero around the image block; not only can prevent the size of the output characteristic graph from changing, but also can repeatedly utilize the edge information.
In the step (3), the normalized specific calculation method is as follows:
Figure BDA0002345087500000021
wherein, aiIs a certain neuron's original activation value, ai normIs a normalized value, gamma, after a normalization operationi、βiTwo regulating factors of the neuron in the training process are adopted, the regulating factors slightly regulate the numerical values which are normalized to have a mean value of 0 and a variance of 1, and then a condition that the mean value is βiVariance is gammai 2The distribution value, thereby preventing the interference of the internal covariates.
When the batch of normalized training networks is carried out, a neuron set S is selected firstly, and then the mean value and the variance of the neuron activation values are calculated, wherein the specific formula is as follows:
Figure BDA0002345087500000022
k is equal to S and | | | | S | | | | | m
Figure BDA0002345087500000023
Wherein m is the size of the set S, and epsilon is a positive number with a numerical value close to zero; avoiding the situation that the gradient is not defined at a certain position, and after the normalization operation, the output result can be normalized to be in the range no matter what the change of the neuron occurs.
In the step (2), the convolutional neural network adopting the Adam algorithm is used for training, specifically, the Adam algorithm is used for updating the weight, the estimation with the deviation is calculated, the estimation after the deviation correction is calculated later is improved, and the parameters in the Adam algorithm are set as default values; the algorithm parameter adjustment process is simple, and most problems in the denoising process can be solved.
And each layer of convolution kernels of the convolution layers in the convolution neural network model is identical in number and size.
And the convolution layer in the convolutional neural network model adopts a ReLU activation function.
In the step (1), the original image is subjected to gray scale processing before the noisy image is input, and is converted into a gray scale image, and the gray scale image is subjected to noise processing to obtain a noisy image and then is input.
The noise adding process is to add Gaussian noise with a fixed variance value in the gray level image, so that the noise removing effect is better.
Has the advantages that: compared with the prior art, the invention has the beneficial effects that: (1) loss and loss of information in the transmission process are avoided, and the depth of a convolutional neural network denoising model and the keeping effect of structural information are improved; (2) the stability of network training is increased; (3) enhancing the sensitivity of the convolutional neural network to the shallow and deep features of the image and improving the learning effect of the drying mapping; (4) the sizes of the feature images after convolution are not changed, and images with the same size are finally output.
Drawings
FIG. 1 is a flowchart of an image denoising method based on residual learning and convolutional neural network according to the present invention;
FIG. 2 is a frame diagram of the image denoising method based on residual learning and convolutional neural network according to the present invention;
FIG. 3 is a gray scale image obtained by performing gray scale processing on an original image according to the present invention;
FIG. 4 is a noisy image obtained after a noise process is applied to a gray scale image in accordance with the present invention;
FIG. 5 is a diagram showing a structure of a residual error unit according to the present invention;
FIG. 6 is a diagram illustrating the effect of denoising a noisy image according to the present invention;
Detailed Description
The invention is described in further detail below with reference to specific embodiments and the attached drawing figures.
As shown in fig. 1, the image denoising method based on residual learning and convolutional neural network of the present invention includes the following steps:
(1) denoising the input noisy image by adopting a convolutional neural network method, and constructing a convolutional neural network model comprising a plurality of convolutional layers; the convolution kernels of each layer of convolution layers in the convolution neural network model are identical in number and size, convolution is carried out by adopting convolution kernels of 3 x 3, and the number of the convolution kernels of each layer is 64; a convolution layer in the convolutional neural network model adopts a ReLU activation function;
(2) denoising an input noisy image by adopting a residual error learning method, superposing a network mapping layer on a neural network, and equating an original deep neural network model to a shallow neural network model, so that the depth is deepened, and the image is not degraded;
(3) denoising the noisy image by adopting an image denoising method combining a convolutional neural network method and a residual error learning method, adding padding in the convolutional neural network, performing batch normalization operation, adding a shallow-to-deep crossing connection structure in the obtained convolutional neural network, and then training by adopting the convolutional neural network of an Adam algorithm;
the padding added in the convolutional neural network is zero padding around the image block, so that the size change of the output feature map can be prevented, and the edge information can be used for multiple times. Because the size of the adopted convolution kernel is 3 multiplied by 3, and padding with the size of 1 is added to each layer, the size of the feature image after convolution is unchanged, and finally the image with the same size can be output;
the specific calculation method of the normalization is as follows:
Figure BDA0002345087500000041
wherein, aiFor a certain value of the original activation of a neuron,
Figure BDA0002345087500000042
is a normalized value, gamma, after a normalization operationi、βiTwo regulating factors of the neuron in the training process are adopted, the regulating factors slightly regulate the numerical values which are normalized to have a mean value of 0 and a variance of 1, and then a condition that the mean value is βiVariance is gammai 2The value of the distribution, thereby preventing interference of the internal covariates;
when the training network is normalized in batches, a neuron set S is selected firstly, and then the mean value and the variance of the neuron activation value are calculated, wherein the specific formula is as follows:
Figure BDA0002345087500000043
k is equal to S and | | | | S | | | | | m
Figure BDA0002345087500000044
Wherein m is the size of the set S, and epsilon is a positive number with a numerical value close to zero; avoiding the situation that the gradient is not defined at a certain position;
after standardized operation, no matter what the neuron changes, the output result is normalized to the range, in addition, the translation parameter and the scaling parameter can be self-adaptively learned, the introduced parameters are few, and the parameters can be updated by using back propagation, so that the convergence speed of neural network training can be accelerated to a certain extent, and the degree of dependence on the network parameter initialization process is reduced;
as shown in fig. 2, the head-to-tail crossing connection from the shallow layer to the deep layer is added to the convolutional neural network, so that the problem that the shallow image features are lost due to the higher-dimensional feature image obtained after the input image is subjected to multiple convolution operations in the forward propagation process in the network can be avoided, the shallow information of the image can be retained, the sensitivity of the convolutional neural network to the shallow and deep image features can be enhanced, and the learning effect of denoising mapping can be improved; in addition, the convolutional neural network adopting the Adam algorithm is trained, specifically, the weight is updated by adopting the Adam algorithm, the estimation with deviation is calculated, the estimation after the deviation correction is calculated later is improved, and the parameters in the Adam algorithm are set as default values;
(4) and outputting the denoised image.
As shown in fig. 2, the convolutional neural network model of the present invention only includes a plurality of convolutional layers, because when denoising is performed using the convolutional neural network, the dimensionality reduction process of the pooling operation filters out part of the features, which adversely affects the denoising, and therefore, the network model of the present invention removes the pooling layer and only retains the convolutional layers.
As shown in fig. 3 and 4, in step (1), before the noisy image is input, the original image is subjected to gray scale processing to be converted into a gray scale image, and the gray scale image is subjected to noise processing to obtain a noisy image, and then the noisy image is input into each model to be subjected to noise removal processing respectively. In this embodiment, the noise addition process is to add gaussian noise having a variance value of 25 to the grayscale image.
In order to increase the depth of the network and ensure that the network does not degenerate, the residual is used in deep learning, an identity mapping layer (y-x layer) is superimposed on a neural network, and the original deep neural network model is equivalent to a shallow neural network, so that the depth is deepened, and the phenomenon of degeneration does not occur. Since it is not easy to directly implement an identity mapping function h (x) ═ x in the neural network layer, the mapping of the network is changed to h (x) ═ f (x) + x, as shown in fig. 5, the model includes two mapping, one is identification mapping performed by using a short connection structure on the right side, and the other is residual part f (x) of residual mapping, and the model can convert the problem into learning a residual function, so the residual function formula of the network mapping layer in step (2) is expressed as:
F(x)=H(x)-x
wherein, F(x)Representing a residual function, H(x)Representing the network mapping function and x representing the identity mapping layer.
The present invention is compared with a three-dimensional block matching algorithm (BM3D), a convolutional neural network denoising method including 5 convolutional layers (novel method 1), and a convolutional neural network denoising method including 19 convolutional layers (novel method 2):
Figure BDA0002345087500000051
in the above four comparison methods, the sizes of the convolution kernels are all 3 × 3. After the same group of gray level images are denoised, the peak signal-to-noise ratio (PSNR) and the used time are compared, and the results clearly show that the PSNR is obviously improved and the consumed time is obviously reduced by adopting the denoising method disclosed by the invention, so that the image denoising method based on residual learning and the convolutional neural network has the best denoising effect, as shown in fig. 6, the residual learning is combined into the convolutional neural network, the network depth can be expanded, the high-dimensional characteristic is extracted, the network degradation and the loss and loss of information in the transmission project are effectively avoided, the noise elimination is considered, and the depth of the denoising model of the convolutional neural network and the maintenance effect of structural information are improved.

Claims (10)

1. An image denoising method based on residual learning and a convolutional neural network is characterized by comprising the following steps:
(1) denoising the input noisy image by adopting a convolutional neural network method, and constructing a convolutional neural network model comprising a plurality of convolutional layers;
(2) denoising an input noisy image by adopting a residual error learning method, superposing a network mapping layer on a neural network, and equating an original deep neural network model to a shallow neural network model;
(3) denoising the noisy image by adopting an image denoising method combining a convolutional neural network method and a residual error learning method, adding padding in the convolutional neural network, performing batch normalization operation, adding a shallow-to-deep crossing connection structure in the obtained convolutional neural network, and then training by adopting the convolutional neural network of an Adam algorithm;
(4) and outputting the denoised image.
2. The method for denoising an image based on residual error learning and convolutional neural network of claim 1, wherein in step (2), the residual error function formula of the network mapping layer is:
F(x)=H(x)-x
wherein, F(x)Representing a residual function, H(x)Representing the network mapping function and x representing the identity mapping layer.
3. The image denoising method based on residual learning and convolutional neural network of claim 1, wherein: in the step (3), padding is added to the convolutional neural network to fill zeros around the image block.
4. The image denoising method based on residual learning and convolutional neural network of claim 1, wherein: in the step (3), the normalized specific calculation method is as follows:
Figure FDA0002345087490000011
wherein, aiFor a certain value of the original activation of a neuron,
Figure FDA0002345087490000012
is a normalized value, gamma, after a normalization operationi、βiTwo regulating factors of the neuron in the training process are adopted, the regulating factors slightly regulate the numerical values which are normalized to have a mean value of 0 and a variance of 1, and then a condition that the mean value is βiVariance is gammai 2The value of the distribution.
5. The image denoising method based on residual learning and convolutional neural network of claim 1, wherein: when the batch of normalized training networks is carried out, a neuron set S is selected firstly, and then the mean value and the variance of the neuron activation values are calculated, wherein the specific formula is as follows:
Figure FDA0002345087490000013
Figure FDA0002345087490000021
where m is the size of the set S and ε is a positive number with a value close to zero.
6. The image denoising method based on residual error learning and convolutional neural network of claim 1, wherein in the step (2), the convolutional neural network training using Adam algorithm is specifically to use Adam algorithm to update weights, and by calculating the estimation with deviation, the estimation after the deviation correction is calculated later is improved, and the parameters in Adam algorithm are set as default values.
7. The image denoising method based on residual learning and convolutional neural network of claim 1, wherein: and each layer of convolution kernels of the convolution layers in the convolution neural network model is identical in number and size.
8. The image denoising method based on residual learning and convolutional neural network of claim 1, wherein: and the convolution layer in the convolutional neural network model adopts a ReLU activation function.
9. The image denoising method based on residual learning and convolutional neural network of claim 1, wherein: in the step (1), before the noisy image is input, the original image is subjected to gray scale processing to be converted into a gray scale image, and the gray scale image is subjected to noise processing to obtain a noisy image and then input.
10. The image denoising method based on residual learning and convolutional neural network of claim 9, wherein: the noise adding process is to add Gaussian noise with a fixed variance value in the gray level image.
CN201911391449.4A 2019-12-30 2019-12-30 Image denoising method based on residual learning and convolutional neural network Pending CN111145125A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911391449.4A CN111145125A (en) 2019-12-30 2019-12-30 Image denoising method based on residual learning and convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911391449.4A CN111145125A (en) 2019-12-30 2019-12-30 Image denoising method based on residual learning and convolutional neural network

Publications (1)

Publication Number Publication Date
CN111145125A true CN111145125A (en) 2020-05-12

Family

ID=70521721

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911391449.4A Pending CN111145125A (en) 2019-12-30 2019-12-30 Image denoising method based on residual learning and convolutional neural network

Country Status (1)

Country Link
CN (1) CN111145125A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111580162A (en) * 2020-05-21 2020-08-25 长江大学 Seismic data random noise suppression method based on residual convolutional neural network
CN114845042A (en) * 2022-03-14 2022-08-02 南京大学 Camera automatic focusing method based on image information entropy
CN116703772A (en) * 2023-06-15 2023-09-05 山东财经大学 Image denoising method, system and terminal based on adaptive interpolation algorithm

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080240607A1 (en) * 2007-02-28 2008-10-02 Microsoft Corporation Image Deblurring with Blurred/Noisy Image Pairs
CN108564553A (en) * 2018-05-07 2018-09-21 南方医科大学 Low-dose CT image noise suppression method based on convolutional neural networks
CN109978778A (en) * 2019-03-06 2019-07-05 浙江工业大学 Convolutional neural networks medicine CT image denoising method based on residual error study

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080240607A1 (en) * 2007-02-28 2008-10-02 Microsoft Corporation Image Deblurring with Blurred/Noisy Image Pairs
CN108564553A (en) * 2018-05-07 2018-09-21 南方医科大学 Low-dose CT image noise suppression method based on convolutional neural networks
CN109978778A (en) * 2019-03-06 2019-07-05 浙江工业大学 Convolutional neural networks medicine CT image denoising method based on residual error study

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
XIAO-JIAO MAO等: "Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections" *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111580162A (en) * 2020-05-21 2020-08-25 长江大学 Seismic data random noise suppression method based on residual convolutional neural network
CN114845042A (en) * 2022-03-14 2022-08-02 南京大学 Camera automatic focusing method based on image information entropy
CN114845042B (en) * 2022-03-14 2023-03-10 南京大学 Camera automatic focusing method based on image information entropy
CN116703772A (en) * 2023-06-15 2023-09-05 山东财经大学 Image denoising method, system and terminal based on adaptive interpolation algorithm
CN116703772B (en) * 2023-06-15 2024-03-15 山东财经大学 Image denoising method, system and terminal based on adaptive interpolation algorithm

Similar Documents

Publication Publication Date Title
CN110599409B (en) Convolutional neural network image denoising method based on multi-scale convolutional groups and parallel
CN110781895B (en) Image semantic segmentation method based on convolutional neural network
CN109087273B (en) Image restoration method, storage medium and system based on enhanced neural network
CN109961396B (en) Image super-resolution reconstruction method based on convolutional neural network
CN111145125A (en) Image denoising method based on residual learning and convolutional neural network
WO2018214671A1 (en) Image distortion correction method and device and electronic device
CN109410127A (en) A kind of image de-noising method based on deep learning and multi-scale image enhancing
Wang et al. Channel and space attention neural network for image denoising
CN113763268B (en) Blind restoration method and system for face image
CN112581397B (en) Degraded image restoration method, system, medium and equipment based on image priori information
CN113808036B (en) Low-illumination image enhancement and denoising method based on Retinex model
CN116051408B (en) Image depth denoising method based on residual error self-coding
CN110111266B (en) Approximate information transfer algorithm improvement method based on deep learning denoising
CN113256508A (en) Improved wavelet transform and convolution neural network image denoising method
CN112215054A (en) Depth generation countermeasure method for underwater acoustic signal denoising
CN114897694A (en) Image super-resolution reconstruction method based on mixed attention and double-layer supervision
CN113191983A (en) Image denoising method and device based on deep learning attention mechanism
CN115358910A (en) Digital watermark attack method and system based on convolutional neural network denoising algorithm
CN114359076A (en) Self-adaptive weighted Gaussian curvature filtering method based on image edge indication function
CN118172290B (en) Multi-stage adaptive CNN and hybrid transducer-based Thangka image restoration method, system and storage medium
CN112927137A (en) Method, device and storage medium for acquiring blind super-resolution image
Chen et al. Image denoising via generative adversarial networks with detail loss
CN113379641A (en) Single image rain removing method and system based on self-coding convolutional neural network
CN114998666B (en) Degradation kernel extraction method for image blind super-resolution enhancement network
CN111325690A (en) Self-adaptive underwater image enhancement method based on differential evolution algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200512