CN110503608B - Image denoising method based on multi-view convolutional neural network - Google Patents
Image denoising method based on multi-view convolutional neural network Download PDFInfo
- Publication number
- CN110503608B CN110503608B CN201910632475.5A CN201910632475A CN110503608B CN 110503608 B CN110503608 B CN 110503608B CN 201910632475 A CN201910632475 A CN 201910632475A CN 110503608 B CN110503608 B CN 110503608B
- Authority
- CN
- China
- Prior art keywords
- network
- image
- denoising
- multiplied
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 21
- 230000006870 function Effects 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 3
- 238000012512 characterization method Methods 0.000 claims 1
- 230000009466 transformation Effects 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 11
- 238000003745 diagnosis Methods 0.000 abstract description 4
- 238000004364 calculation method Methods 0.000 abstract description 3
- 230000007547 defect Effects 0.000 abstract description 3
- 239000013589 supplement Substances 0.000 abstract description 3
- 230000004927 fusion Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 241000282326 Felis catus Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000012850 discrimination method Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention discloses an image denoising method based on a multi-view convolutional neural network. The invention uses INBN technology to replace BN to accelerate the convergence of the denoising network; the method can effectively supplement the defects of BN and accelerate network convergence, and can process real noise images, blind noise and Gaussian noise. The invention only uses 20 layers of networks to carry out denoising, thereby reducing the calculation cost of the networks. In addition, the invention uses a new technology GF technology to better transform linear data into nonlinear data; the denoising model is better trained with a Smooth function. In addition, it employs feature fusion from multiple perspectives to enhance network performance. The invention has important significance for disaster relief, aviation exploration and medical diagnosis in reality.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an image denoising method based on a multi-view convolutional neural network.
Background
Digital image devices have been widely used in various fields such as disease diagnosis, personal identification, and disaster relief. However, when the image device is used to take pictures, the pictures are often affected by camera shake, low light, rainy days, etc., which causes the taken pictures to be unclear, the image denoising technology is to restore the unclear images to high-definition images, this process is called a reversible process, and the denoising mainly depends on a method of y=x+μ, where y is a noisy image, x is a restored clean image, and μ is noise. From the bayesian perspective, a priori knowledge is critical to image denoising, while some students have done much work in this respect, such as sparse methods are very robust to image denoising tasks. The sparse method is optimized by non-logic adaptivity and is applied to the denoising task. Dictionary learning is used to remove noise and also effectively reduce computational costs. The total variation regularization method can make the image smoother, which is beneficial to restoring a clean image. Furthermore, markov, weighted kernel norm minimization and 3-dimensional block matching filter methods are the dominant denoising methods. While these methods have achieved good performance in image denoising, they still face the following challenges:
(1) These methods require manual parameter settings to obtain optimal performance;
(2) These methods require complex optimization algorithms at the test stage, which greatly increases the computational cost of these methods;
(3) This approach only trains a model for one case to solve this problem, e.g. gaussian noise with a noise level of 25, and they only have a model to solve this problem, and noise images in life are complex, which greatly limits their application range.
In recent years, deep learning has become more popular for reasons of image processing units (Graphic Processing Unit, GPUs) and big data. Among them, convolutional neural network (Convolutional Neural Network) is a typical deep learning technique, which is also called as more and more popular for the following reasons:
(1) The structure of CNN is end-to-end connection, it is very flexible, it can set up the structure to the characteristic of the task;
(2) CNNs can rely on basic plugins, including linear correction units (Rectifier Linear Unit, reLU) and convolutional layers (Conv);
(3) CNN relies on GPU to perform parallel computation, greatly improving the operation efficiency.
The CNN has strong self-learning capability, does not need manual parameter adjustment, and can rapidly process images by means of a GPU, so that the CNN is also an effective image processing recovery method, such as: the SRCNN network uses three layers to process super resolution tasks. Although it is better than the traditional method in super-resolution task, the network depth is reduced when exceeding three layers of performances, so the method lacks flexibility. Subsequently, CNN has also made a breakthrough in image denoising, such as DnCNN first uses CNN in image denoising and uses a model to handle multiple tasks, gaussian denoising, super resolution and restoring compressed images. Ffdnat uses the noise map and the noise image together as inputs to the denoising network, and the method effectively processes blind noise. IRCNN combines the optimization algorithm and the discrimination method for the first time, and the method has a certain meaning in processing real noise. The MLWC combines spatial domain features with CNN to solve the tasks of super resolution, denoising and the like. He Kaiming proposes to reuse + operation to improve the image denoising performance. All of the above approaches have made some progress in image denoising, but the following challenges remain unsolved:
(1) Most of the methods use BN technology, the BN technology is very dependent on the size of the batch, and when the batch is large, the BN technology has better performance; when the batch is smaller, the BN technical performance is degraded. Therefore BN technology does not have very good robustness;
(2) The above-mentioned method cannot use one model to process multiple tasks such as real noise, gaussian noise and blind noise;
(3) The above-mentioned methods are partly by deepening the network layer number, and partly by repeatedly utilizing + operation to raise the denoising performance, which greatly increases the calculation cost of the network.
Disclosure of Invention
The invention aims to solve the technical problem of providing an image denoising method based on a multi-view convolutional neural network, which can effectively supplement the defects of BN and accelerate network convergence and can process real noise images, blind noise and Gaussian noise. Has important significance for disaster relief, aviation exploration and medical diagnosis in reality.
The invention is realized in the following way: the image denoising method based on the multi-view convolutional neural network comprises the following steps of:
1) After the features of the original image are extracted by an FFT algorithm, image reconstruction is carried out, and the reconstructed image and the original image are segmented and then are used as the input of a denoising network;
2) And after the reconstructed image and the original image are segmented through a denoising network, adding the output corresponding to the FFT with the output of the original image corresponding network, and outputting a denoised clean image.
The denoising network consists of 19 layers, wherein the 1 st layer consists of a convolution layer and a ReLU; layers 2-18 consist of convolutional layers, INBN and GF functions, and layer 19 consists of convolutional layers; and adding the obtained network output corresponding to the FFT with the network output corresponding to the original image through denoising network processing, and taking the added network output as the input of the 19 th convolution layer.
The input size of the network is 256 multiplied by 1 multiplied by 40, the output size is 256 multiplied by 1 multiplied by 40, the convolution size is 3 multiplied by 3, wherein 256 multiplied by 1 multiplied by 40 is represented by the batch size being 256, the output channel is 1, and the height and the width are 40.
The INBN layer is formed by combining half of channels of a convolution layer through IN and the other half of channels of the convolution layer through BN by +operation; wherein IN is represented by formula (1):
in the formula (1), μ is an average value, σ is a standard deviation, ε is a constant, and H is a height W is a depth;
BN is represented by formula (2):
(2) The representative average (3) represents the variance (4) represents the normalization (5) represents the reconstruction of the data.
The GF function is GF (x) =relu (x) ×tanh (x), where ReLU is Φ (x) =max (0, x), and the function of ReLU is to convert the linear converted data into nonlinear data.
Wherein, tanh (x) is
The denoising model is trained using a smoothfunction as an objective function, the smoothfunction being as shown in equation (6):
and (3) image reconstruction: after extracting the characteristics of the image, multiplying the characteristics by the parameters of model training to obtain a characteristic diagram. The feature map is distinguished from the original map, and the feature map is the extraction of the key of the original map, for example: a human face image is extracted by the feature, and an image band can be reconstructed to represent the whole human face image by using the features of eyes, nose, mouth and the like.
Image blocking: the whole image is divided into a plurality of small blocks, so that direct local characteristics of the image can be extracted quickly.
The input is to input the FFT feature image and the original image, and to divide the feature image and the original image into blocks.
The invention uses INBN technology to replace BN to accelerate the convergence of the denoising network; the method can effectively supplement the defects of BN and accelerate network convergence, and can process real noise images, blind noise and Gaussian noise. The invention only uses 20 layers of networks to carry out denoising, thereby reducing the calculation cost of the networks. In addition, the invention uses a new technology GF technology to better transform linear data into nonlinear data; the denoising model is better trained with a Smooth function. In addition, it employs feature fusion from multiple perspectives to enhance network performance. The invention has important significance for disaster relief, aviation exploration and medical diagnosis in reality.
Drawings
FIG. 1 is an overall flow chart of a network of the present invention;
FIG. 2 is a block diagram of a denoising network according to the present invention;
FIG. 3 is an image of the original noise image and the FFT extracted spatial domain features according to an embodiment of the present invention;
FIG. 4 is a block diagram of 2 blocks in an original noise block image according to an embodiment of the present invention;
FIG. 5 is a 2-block image of an FFT image block image according to an embodiment of the invention;
fig. 6 is a clean image 2 block image of an embodiment of the present invention.
Fig. 7 is a comparison of an original image, an image of FFT extracted spatial features, and a restored clean image in an embodiment of the present invention.
Detailed Description
Embodiments of the invention: the image denoising method based on the multi-view convolutional neural network comprises the step of taking a Smooth function as an objective function of training a denoising network. Take gaussian noise level 75 as an example.
The method comprises the following steps:
1) The original image is subjected to feature extraction by FFT and then subjected to image reconstruction, the reconstructed image is segmented, and the original image is segmented to be used as the input of a network, as shown in figure 3.
2) And after the reconstructed image and the original image are segmented through a denoising network, adding the output corresponding to the FFT with the output of the original image corresponding network, and outputting a denoised clean image.
The network is composed of 19 layers, wherein layer 1 is composed of a convolution layer and a ReLU; layers 2-18 consist of convolutional layers, INBN and GF functions, and layer 19 consists of convolutional layers; step 1 is performed using a "+" operation, and features obtained in fig. 2 are fused together and then connected to the layer 19 convolution layer.
The input size of the network is 256 multiplied by 1 multiplied by 40, the output size is 256 multiplied by 1 multiplied by 40, the convolution size is 3 multiplied by 3, wherein 256 multiplied by 1 multiplied by 40 is represented by the batch size being 256, the output channel is 1, and the height and the width are 40.
The INBN layer is formed by combining half of channels of a convolution layer through IN and the other half of channels of the convolution layer through BN after the channels of the convolution layer pass through cat+ operation; wherein IN is represented by formula (1):
in the formula (1), μ is an average value, σ is a standard deviation, ε is a constant, and H is a height W is a depth;
BN is represented by formula (2):
(2) The representative average (3) represents the variance (4) represents the normalization (5) represents the reconstruction of the data.
The GF function is GF (x) =relu (x) ×tanh (x), where ReLU is Φ (x) =max (0, x), and the function of ReLU is to convert the linear converted data into nonlinear data.
Wherein, tanh (x) is
The invention uses the Smooth function as an objective function to train the denoising model, and the Smooth can enable the image to be smoother.
Claims (4)
1. The image denoising method based on the multi-view convolutional neural network is characterized by comprising the following steps of:
1) After the features of the original image are extracted by an FFT algorithm, image reconstruction is carried out, and the reconstructed image and the original image are segmented and then are used as the input of a denoising network;
2) After the reconstructed image and the original image are segmented through a denoising network, adding the network output corresponding to the FFT and the network output corresponding to the original image, and outputting a denoised clean image;
the denoising network consists of 19 layers, wherein the 1 st layer consists of a convolution layer and a ReLU; layers 2-18 consist of convolutional layers, INBN and GF functions, and layer 19 consists of convolutional layers; adding the obtained network output corresponding to the FFT with the network output corresponding to the original image through denoising network processing, and taking the added network output as the input of a layer 19 convolution layer;
the INBN layer is formed by combining half of channels of a convolution layer through IN and the other half of channels of the convolution layer through BN by +operation; wherein IN is represented by formula (1):
in the formula (1), μ is an average value, σ is a standard deviation, ε is a constant, and H is a height W is a depth;
BN is represented by formula (2):
(2) The representative average (3) represents the variance (4) represents the normalization (5) represents the reconstruction of the data.
2. The multi-view convolutional neural network-based image denoising method as claimed in claim 1, wherein: the input size of the network is 256 multiplied by 1 multiplied by 40, the output size is 256 multiplied by 1 multiplied by 40, the convolution size is 3 multiplied by 3, wherein 256 multiplied by 1 multiplied by 40 is represented by the batch size being 256, the output channel is 1, and the height and the width are 40.
3. The multi-view convolutional neural network-based image denoising method as claimed in claim 1, wherein: the GF function is GF (x) =ReLU (x) ×Tanh (x), wherein ReLU is phi (x) =max (0, x), and the function of ReLU is to perform nonlinear transformation on the data and increase the nonlinear characterization capability of the data, wherein tan h (x) is
4. The multi-view convolutional neural network-based image denoising method as claimed in claim 1, wherein: the denoising model is trained using a smoothfunction as an objective function, the smoothfunction being as shown in equation (6):
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910632475.5A CN110503608B (en) | 2019-07-13 | 2019-07-13 | Image denoising method based on multi-view convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910632475.5A CN110503608B (en) | 2019-07-13 | 2019-07-13 | Image denoising method based on multi-view convolutional neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110503608A CN110503608A (en) | 2019-11-26 |
| CN110503608B true CN110503608B (en) | 2023-08-08 |
Family
ID=68585482
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910632475.5A Active CN110503608B (en) | 2019-07-13 | 2019-07-13 | Image denoising method based on multi-view convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110503608B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112883759B (en) * | 2019-11-29 | 2023-09-26 | 杭州海康威视数字技术股份有限公司 | Method for detecting image noise of biological feature part |
| CN111045084B (en) * | 2020-01-06 | 2021-12-07 | 中国石油化工股份有限公司 | Multi-wave self-adaptive subtraction method based on prediction feature extraction |
| CN112634159B (en) * | 2020-12-23 | 2022-07-26 | 中国海洋大学 | Hyperspectral image denoising method based on blind noise estimation |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6207957B1 (en) * | 1998-09-18 | 2001-03-27 | The Regents Of The University Of California | System to quantify gamma-ray radial energy deposition in semiconductor detectors |
| CN108460726A (en) * | 2018-03-26 | 2018-08-28 | 厦门大学 | A kind of magnetic resonance image super-resolution reconstruction method based on enhancing recurrence residual error network |
| CN108986047A (en) * | 2018-07-13 | 2018-12-11 | 中国科学技术大学 | Image denoising method |
| CN109410127A (en) * | 2018-09-17 | 2019-03-01 | 西安电子科技大学 | A kind of image de-noising method based on deep learning and multi-scale image enhancing |
| CN109658344A (en) * | 2018-11-12 | 2019-04-19 | 哈尔滨工业大学（深圳） | Image de-noising method, device, equipment and storage medium based on deep learning |
-
2019
- 2019-07-13 CN CN201910632475.5A patent/CN110503608B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6207957B1 (en) * | 1998-09-18 | 2001-03-27 | The Regents Of The University Of California | System to quantify gamma-ray radial energy deposition in semiconductor detectors |
| CN108460726A (en) * | 2018-03-26 | 2018-08-28 | 厦门大学 | A kind of magnetic resonance image super-resolution reconstruction method based on enhancing recurrence residual error network |
| CN108986047A (en) * | 2018-07-13 | 2018-12-11 | 中国科学技术大学 | Image denoising method |
| CN109410127A (en) * | 2018-09-17 | 2019-03-01 | 西安电子科技大学 | A kind of image de-noising method based on deep learning and multi-scale image enhancing |
| CN109658344A (en) * | 2018-11-12 | 2019-04-19 | 哈尔滨工业大学（深圳） | Image de-noising method, device, equipment and storage medium based on deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 基于深度学习的乳腺X射线影像分类方法研究;孙利雷等;《计算机工程与应用》;20181101(第21期);13-19 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110503608A (en) | 2019-11-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Dong et al. | Deep spatial–spectral representation learning for hyperspectral image denoising | |
| CN114140353B (en) | Swin-Transformer image denoising method and system based on channel attention | |
| CN110738605B (en) | Image denoising method, system, equipment and medium based on transfer learning | |
| WO2020087607A1 (en) | Bi-skip-net-based image deblurring method | |
| CN110503608B (en) | Image denoising method based on multi-view convolutional neural network | |
| CN111275643B (en) | Real noise blind denoising network system and method based on channel and space attention | |
| Yin et al. | Highly accurate image reconstruction for multimodal noise suppression using semisupervised learning on big data | |
| CN111091503B (en) | Image defocusing and blurring method based on deep learning | |
| CN111062880A (en) | Underwater image real-time enhancement method based on condition generation countermeasure network | |
| CN111161360B (en) | Image defogging method of end-to-end network based on Retinex theory | |
| Zhao et al. | Invertible image decolorization | |
| CN110796622B (en) | Image bit enhancement method based on multi-layer characteristics of series neural network | |
| CN111861894B (en) | Image motion blur removing method based on generation type countermeasure network | |
| CN112257766B (en) | Shadow recognition detection method in natural scene based on frequency domain filtering processing | |
| CN112270654A (en) | Image denoising method based on multi-channel GAN | |
| CN108765330B (en) | Image denoising method and device based on global and local prior joint constraint | |
| CN113362250B (en) | Image denoising method and system based on dual-tree quaternary wavelet and deep learning | |
| CN112991199B (en) | Image high-low frequency decomposition noise removal method based on residual dense network | |
| CN114972107A (en) | Low-illumination image enhancement method based on multi-scale stacked attention network | |
| CN105957025A (en) | Inconsistent image blind restoration method based on sparse representation | |
| CN113362338B (en) | Rail segmentation method, device, computer equipment and rail segmentation processing system | |
| CN115018726A (en) | U-Net-based image non-uniform blur kernel estimation method | |
| Choi et al. | Fast super-resolution algorithm using ELBP classifier | |
| Tian et al. | A modeling method for face image deblurring | |
| Bera et al. | A lightweight convolutional neural network for image denoising with fine details preservation capability |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |