CN110503608A - The image de-noising method of convolutional neural networks based on multi-angle of view - Google Patents

The image de-noising method of convolutional neural networks based on multi-angle of view Download PDF

Info

Publication number
CN110503608A
CN110503608A CN201910632475.5A CN201910632475A CN110503608A CN 110503608 A CN110503608 A CN 110503608A CN 201910632475 A CN201910632475 A CN 201910632475A CN 110503608 A CN110503608 A CN 110503608A
Authority
CN
China
Prior art keywords
image
network
angle
view
denoising
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910632475.5A
Other languages
Chinese (zh)
Other versions
CN110503608B (en
Inventor
徐勇
孙利雷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guizhou University
Original Assignee
Guizhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guizhou University filed Critical Guizhou University
Priority to CN201910632475.5A priority Critical patent/CN110503608B/en
Publication of CN110503608A publication Critical patent/CN110503608A/en
Application granted granted Critical
Publication of CN110503608B publication Critical patent/CN110503608B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20021Dividing image into blocks, subimages or windows
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20048Transform domain processing
    • G06T2207/20056Discrete and fast Fourier transform, [DFT, FFT]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30196Human being; Person
    • G06T2207/30201Face
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02TCLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00Road transport of goods or passengers
    • Y02T10/10Internal combustion engine [ICE] based vehicles
    • Y02T10/40Engine management systems

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a kind of image de-noising methods of convolutional neural networks based on multi-angle of view.The present invention is accelerated to denoise the convergence of network instead of BN with INBN technology;This method can effectively mend the deficiency with regard to BN and accelerate network convergence, can handle real noise image, blind noise and Gaussian noise.The present invention is only denoised with 20 layers of network, reduces the calculating cost of network.In addition, linear data is more preferably transformed to nonlinear data with a kind of new technology GF technology by the present invention;Denoising model is preferably trained with Smooth function.In addition, it enhances network performance using the Fusion Features of multi-angle of view.The present invention diagnoses a disease and is of great significance for disaster relief, aviation exploration and medical treatment in reality.

Description

The image de-noising method of convolutional neural networks based on multi-angle of view
Technical field
The present invention relates to technical field of image processing, are based especially on the image denoising side of the convolutional neural networks of multi-angle of view Method.
Background technique
Digital image device has been widely applied to multiple fields, such as medical diagnosis on disease, person identification and calamity Hardly possible rescue.And vision facilities is usually influenced by camera shake, low light, greasy weather rainy day etc. in shooting photo, this causes to shoot The photo come is unintelligible, and Image Denoising Technology is exactly these not to be known image restoring to highly clearly image, this process Referred to as reversible process, it is y=x+ μ that it, which denoises the method relied primarily on, and wherein y is noise image, and x is the clean figure recovered to Picture, μ are noises.From the perspective of Bayes, priori knowledge is crucial for image denoising, while some scholars are at this Aspect has done many work, as Sparse methods have good robustness for image denoising task.It is adaptive with NOT logic Property optimizes Sparse methods and applies in denoising task.Dictionary learning is used to removal noise, can also efficiently reduce calculating Cost.Full variational regularization method can make image become more smooth, this is conducive to restore clean image.In addition, Ma Erke The Block- matching filter method of husband, Weighted Kernel norm minimum and 3 dimensions are the denoising methods of mainstream.Although these methods are in image Fine performance is had been achieved in denoising, but these methods still suffer from following challenge:
(1) these methods need artificial manual setting parameter to obtain optimal performance;
(2) these methods need to need complicated optimization algorithm in test phase, this greatly increases the calculating of these methods Cost;
(3) this method can only solve this problem for a kind of situation one model of training, if level of noise is 25 Gaussian noise, they can only have a model to solve this problem, and noise image is complicated in life, this is limited significantly Their application range.
In recent years, deep learning is due to image processing unit (Graphic Processing Unit, GPU) and big data The reason of become more and more popular.Wherein convolutional neural networks (Convolutional Neural Network) are typical Depth learning technology, its also known as the reason of becoming more and more popular following points:
(1) structure of CNN is end-to-end connection, it is very flexibly, it can set for the characteristic of task Determine structure;
(2) CNN can realize regularization by basic plug-in unit, and basic plug-in unit includes linear amending unit (Rectifier Linear Unit, ReLU) and convolutional layer (Convolution, Conv);
(3) CNN carries out parallel computation by GPU, greatly improves operational efficiency.
CNN is not since with powerful self-learning capability, it needs to adjust ginseng manually, while it can quickly be handled by GPU Image, therefore it is also a kind of effectively processing image recovery method, such as: SRCNN network carries out processing super-resolution using three layers Task.Although it is better than conventional method in super-resolution task, when network depth is more than that three layers of performance will decline, so side Method lacks flexibility.Then, CNN also progress of making a breakthrough property on image denoising, as CNN is used image for the first time by DnCNN Denoising, and multiple tasks are handled with a model: Gauss denoising, super-resolution and recovery compression image.FFDNet noise The input as denoising network, this method effectively handle blind noise together for mapping and noise image.IRCNN for the first time will optimization Algorithm and method of discrimination are combined together, and this method has definite meaning in processing real noise.MLWC by space domain characteristic with CNN in conjunction with come solve super-resolution and denoising etc. tasks.He Kaiming proposition is recycled+is operated to promote the performance of image denoising. The above method all obtains certain progress in terms of image denoising, but still has following challenge not yet to solve:
(1) most of method uses BN technology more than, and BN technology relies on the size of batch very much, when batch is big, BN Contrast in technical performance is good;When batch is smaller, the decline of BN technical performance.Therefore BN technology does not have good robust Property;
(2) it is referred to above to method cannot be more with a model treatment real noise, Gaussian noise and blind noise etc. A task;
(3) it is by repeatedly being mentioned using+operation that method part referred to above, which is by deepening the network number of plies, Part Methods, High denoising performance, this greatly increases the calculating cost of network.
Summary of the invention
Technical problem to be solved by the invention is to provide a kind of image denoisings of convolutional neural networks based on multi-angle of view Method, this method can effectively mend the deficiency with regard to BN and accelerate network convergence, can handle real noise image, blind noise and height This noise.Disaster relief, aviation exploration and medical treatment in reality are diagnosed a disease and be of great significance.
The present invention is implemented as follows: the image de-noising method of the convolutional neural networks based on multi-angle of view, includes following step It is rapid to carry out:
1) after original image being extracted feature with fft algorithm, then image reconstruction is carried out, by reconstructed image and original image piecemeal Afterwards collectively as the input of denoising network;
2) reconstructed image and original image piecemeal are by denoising network, by the corresponding output of FFT and original image corresponding network After output is added, the clean image of denoising is exported.
The denoising network is formed by 19 layers, and the 1st layer is made of convolutional layer and ReLU;2-18 layers by convolutional layer, INBN and GF function composition, the 19th layer is made of convolutional layer;It is by denoising network processes, the corresponding network of obtained FFT is defeated Input after being added out with the output of original image corresponding network, as the 19th layer of convolutional layer.
The input size of the network is 256 × 1 × 40 × 40, and output size is 256 × 1 × 40 × 40, and convolution is big Small is 3 × 3, wherein 256 × 1 × 40 × 40 to represent batchsize be 256, output channel 1 is highly 40 with width.
The INBN layer is the channel half an of convolutional layer by IN, the other half by passing through+operation again after BN It is merged together;The wherein expression of IN such as formula (1):
In formula (1), μ is average value, and σ is standard deviation, and ε is constant, and H is that height W is depth;
Shown in the expression of BN such as formula (2):
(2) represent average value (3) represent variance (4) represent normalization (5) represent data reconstruction.
The GF function is GF (x)=ReLU (x) × Tanh (x), and wherein ReLU is φ (x)=max (0, x), ReLU Effect be exactly the data of linear transfor be nonlinear data.
Wherein, tanh (x) is
Smooth function is used as objective function to train denoising model, shown in Smooth function such as formula (6):
Image reconstruction: be multiplied and can be obtained with feature with the parameter of model training training after the feature for extracting image To characteristic pattern.Characteristic pattern and original image be it is distinguishing, characteristic pattern is the extraction to original image key, such as: a secondary facial image, I Extraction feature, can with human eye, nose, mouth feature etc. reconstruct a sub-picture band indicate whole picture facial image.
Image block: being that entire image is divided into multiple fritters, helps the direct local feature of rapidly extracting image in this way.
Input is to input FFT characteristic image with original image, and characteristic image and original image are carried out piecemeal.
The present invention is accelerated to denoise the convergence of network instead of BN with INBN technology;This method can effectively be mended with regard to BN not Foot and quickening network convergence, can handle real noise image, blind noise and Gaussian noise.The present invention only uses 20 layers of network It is denoised, reduces the calculating cost of network.In addition, the present invention is with a kind of new technology GF technology come more preferably linear data It is transformed to nonlinear data;Denoising model is preferably trained with Smooth function.In addition, it is melted using the feature of multi-angle of view It closes to enhance network performance.The present invention diagnoses a disease and is of great significance for disaster relief, aviation exploration and medical treatment in reality.
Detailed description of the invention
Fig. 1 is the overall flow figure of network of the invention;
Fig. 2 is the structure chart of denoising network of the invention;
Fig. 3 is the former noise image of the embodiment of the present invention and the image of FFT extraction space domain characteristic;
Fig. 4 is 2 pieces of figures in the former noise block image of the embodiment of the present invention;
Fig. 5 is 2 blocks of images in the FFT image block image of the embodiment of the present invention;
Fig. 6 is 2 blocks of images of clean image of the embodiment of the present invention.
Fig. 7 is the ratio of the clean image of the original image of the embodiment of the present invention, the image of FFT extraction space characteristics and recovery Compared with figure.
Specific embodiment
The embodiment of the present invention: the image de-noising method of the convolutional neural networks based on multi-angle of view, this implementation is by Smooth Objective function of the function as training denoising network.By taking Gaussian noise rank 75 as an example.
Include the following steps:
1) original image is subjected to image reconstruction after FFT carries out feature extraction, reconstructed image piecemeal, also original image Input of the piecemeal as network, as shown in Figure 3.
2) reconstructed image and original image piecemeal are by denoising network, by the corresponding output of FFT and original image corresponding network After output is added, the clean image of denoising is exported.
The network is formed by 19 layers, and the 1st layer is made of convolutional layer and ReLU;2-18 layers by convolutional layer, INBN It is formed with GF function, the 19th layer is made of convolutional layer;It is operated with "+" by step 1, obtains Fusion Features as shown in Figure 2 to together It is connected afterwards with the 19th layer of convolutional layer.
The input size of the network is 256 × 1 × 40 × 40, and output size is 256 × 1 × 40 × 40, and convolution is big Small is 3 × 3, wherein 256 × 1 × 40 × 40 to represent batchsize be 256, output channel 1 is highly 40 with width.
The INBN layer is the channel half an of convolutional layer by IN, the other half by passing through cat+ again after BN Operation is merged together;The wherein expression of IN such as formula (1):
In formula (1), μ is average value, and σ is standard deviation, and ε is constant, and H is that height W is depth;
Shown in the expression of BN such as formula (2):
(2) represent average value (3) represent variance (4) represent normalization (5) represent data reconstruction.
The GF function is GF (x)=ReLU (x) × Tanh (x), and wherein ReLU is φ (x)=max (0, x), ReLU Effect be exactly the data of linear transfor be nonlinear data.
Wherein, tanh (x) is
The present invention uses Smooth function as objective function to train denoising model, and Smooth can be such that image becomes more Smoothly.

Claims (6)

1. a kind of image de-noising method of the convolutional neural networks based on multi-angle of view, which is characterized in that comprise the following steps progress:
1) after original image being extracted feature with fft algorithm, then image reconstruction is carried out, it will be total after reconstructed image and original image piecemeal With the input as denoising network;
2) reconstructed image and original image piecemeal are by denoising network, by the corresponding network output of FFT and original image corresponding network After output is added, the clean image of denoising is exported.
2. the image de-noising method of the convolutional neural networks according to claim 1 based on multi-angle of view, it is characterised in that: institute The denoising network stated is formed by 19 layers, and the 1st layer is made of convolutional layer and ReLU;2-18 layers by convolutional layer, INBN and GF letter Array is at the 19th layer is made of convolutional layer;By denoising network processes, by the corresponding network output of obtained FFT and original image After corresponding network output is added, the input as the 19th layer of convolutional layer.
3. the image de-noising method of the convolutional neural networks according to claim 1 based on multi-angle of view, it is characterised in that: institute The input size for the network stated is 256 × 1 × 40 × 40, and output size is 256 × 1 × 40 × 40, and convolution size is 3 × 3, In 256 × 1 × 40 × 40 to represent batchsize be 256, output channel 1 is highly 40 with width.
4. the image de-noising method of the convolutional neural networks according to claim 2 based on multi-angle of view, it is characterised in that: institute The INBN layer stated is the channel half an of convolutional layer by IN, the other half is merged together by passing through again after BN+operating; The wherein expression of IN such as formula (1):
In formula (1), μ is average value, and σ is standard deviation, and ε is constant, and H is that height W is depth;
Shown in the expression of BN such as formula (2):
(2) represent average value (3) represent variance (4) represent normalization (5) represent data reconstruction.
5. the image de-noising method of the convolutional neural networks according to claim 2 based on multi-angle of view, it is characterised in that: institute The GF function stated is GF (x)=ReLU (x) × Tanh (x), and wherein ReLU is φ (x)=max (0, x), and the effect of ReLU is exactly It is nonlinear data the data of linear transfor.Wherein, tanh (x) is
6. the image de-noising method of the convolutional neural networks according to claim 2 based on multi-angle of view, it is characterised in that: make Smooth function is used as objective function to train denoising model, shown in Smooth function such as formula (6):
CN201910632475.5A 2019-07-13 2019-07-13 Image denoising method based on multi-view convolutional neural network Active CN110503608B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910632475.5A CN110503608B (en) 2019-07-13 2019-07-13 Image denoising method based on multi-view convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910632475.5A CN110503608B (en) 2019-07-13 2019-07-13 Image denoising method based on multi-view convolutional neural network

Publications (2)

Publication Number Publication Date
CN110503608A true CN110503608A (en) 2019-11-26
CN110503608B CN110503608B (en) 2023-08-08

Family

ID=68585482

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910632475.5A Active CN110503608B (en) 2019-07-13 2019-07-13 Image denoising method based on multi-view convolutional neural network

Country Status (1)

Country Link
CN (1) CN110503608B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111045084A (en) * 2020-01-06 2020-04-21 中国石油化工股份有限公司 Multi-wave self-adaptive subtraction method based on prediction feature extraction
CN112634159A (en) * 2020-12-23 2021-04-09 中国海洋大学 Hyperspectral image denoising method based on blind noise estimation
CN112883759A (en) * 2019-11-29 2021-06-01 杭州海康威视数字技术股份有限公司 Method for detecting image noise of biological characteristic part

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6207957B1 (en) * 1998-09-18 2001-03-27 The Regents Of The University Of California System to quantify gamma-ray radial energy deposition in semiconductor detectors
CN108460726A (en) * 2018-03-26 2018-08-28 厦门大学 A kind of magnetic resonance image super-resolution reconstruction method based on enhancing recurrence residual error network
CN108986047A (en) * 2018-07-13 2018-12-11 中国科学技术大学 Image denoising method
CN109410127A (en) * 2018-09-17 2019-03-01 西安电子科技大学 A kind of image de-noising method based on deep learning and multi-scale image enhancing
CN109658344A (en) * 2018-11-12 2019-04-19 哈尔滨工业大学(深圳) Image de-noising method, device, equipment and storage medium based on deep learning

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6207957B1 (en) * 1998-09-18 2001-03-27 The Regents Of The University Of California System to quantify gamma-ray radial energy deposition in semiconductor detectors
CN108460726A (en) * 2018-03-26 2018-08-28 厦门大学 A kind of magnetic resonance image super-resolution reconstruction method based on enhancing recurrence residual error network
CN108986047A (en) * 2018-07-13 2018-12-11 中国科学技术大学 Image denoising method
CN109410127A (en) * 2018-09-17 2019-03-01 西安电子科技大学 A kind of image de-noising method based on deep learning and multi-scale image enhancing
CN109658344A (en) * 2018-11-12 2019-04-19 哈尔滨工业大学(深圳) Image de-noising method, device, equipment and storage medium based on deep learning

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
HAROLD C. BURGER 等: ""Image denoising: Can plain neural networks compete with BM3D"", 《HTTPS://IEEEXPLORE.IEEE.ORG/STAMP/STAMP.JSP?TP=&ARNUMBER=6247952》 *
KUANG GONG: ""PET Image Denoising Using a Deep Neural Network Through Fine Tuning"", 《HTTPS://IEEEXPLORE.IEEE.ORG/STAMP/STAMP.JSP?TP=&ARNUMBER=8502864》 *
孙利雷等: "基于深度学习的乳腺X射线影像分类方法研究", 《计算机工程与应用》 *
新智元: ""【实战】利用卷积自编码器实现图片降噪(代码开源)"", 《HTTPS://WWW.SOHU.COM/A/157810470_473283》 *
胡拓: ""结合图像降噪处理的低剂量CT重建"", 《中国优秀硕士学位论文全文数据库 医药卫生科技辑》 *
马国喻: "一种求解差分方程的好方法――生成函数法", 《北京化工大学学报(自然科学版)》 *
马红强等: "基于改进栈式稀疏去噪自编码器的自适应图像去噪", 《光学学报》 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112883759A (en) * 2019-11-29 2021-06-01 杭州海康威视数字技术股份有限公司 Method for detecting image noise of biological characteristic part
CN112883759B (en) * 2019-11-29 2023-09-26 杭州海康威视数字技术股份有限公司 Method for detecting image noise of biological feature part
CN111045084A (en) * 2020-01-06 2020-04-21 中国石油化工股份有限公司 Multi-wave self-adaptive subtraction method based on prediction feature extraction
CN112634159A (en) * 2020-12-23 2021-04-09 中国海洋大学 Hyperspectral image denoising method based on blind noise estimation
CN112634159B (en) * 2020-12-23 2022-07-26 中国海洋大学 Hyperspectral image denoising method based on blind noise estimation

Also Published As

Publication number Publication date
CN110503608B (en) 2023-08-08

Similar Documents

Publication Publication Date Title
WO2023092813A1 (en) Swin-transformer image denoising method and system based on channel attention
Tian et al. Deep learning on image denoising: An overview
Remez et al. Class-aware fully convolutional Gaussian and Poisson denoising
CN108520503B (en) Face defect image restoration method based on self-encoder and generation countermeasure network
CN110782399B (en) Image deblurring method based on multitasking CNN
WO2020087607A1 (en) Bi-skip-net-based image deblurring method
CN111127308B (en) Mirror image feature rearrangement restoration method for single sample face recognition under partial shielding
Liu et al. Knowledge-driven deep unrolling for robust image layer separation
CN108765338A (en) Spatial target images restored method based on convolution own coding convolutional neural networks
CN111915530A (en) End-to-end-based haze concentration self-adaptive neural network image defogging method
Huang et al. Deep hyperspectral image fusion network with iterative spatio-spectral regularization
CN111161360B (en) Image defogging method of end-to-end network based on Retinex theory
CN110232653A (en) The quick light-duty intensive residual error network of super-resolution rebuilding
CN111161178A (en) Single low-light image enhancement method based on generation type countermeasure network
CN110503608A (en) The image de-noising method of convolutional neural networks based on multi-angle of view
CN111861894A (en) Image motion blur removing method based on generating type countermeasure network
CN112270654A (en) Image denoising method based on multi-channel GAN
CN112991199B (en) Image high-low frequency decomposition noise removal method based on residual dense network
CN113450290A (en) Low-illumination image enhancement method and system based on image inpainting technology
CN107194380A (en) The depth convolutional network and learning method of a kind of complex scene human face identification
CN114202460A (en) Super-resolution high-definition reconstruction method, system and equipment facing different damage images
CN104200194B (en) A kind of face identification method
CN116703750A (en) Image defogging method and system based on edge attention and multi-order differential loss
Liang et al. Deep convolution neural networks for automatic eyeglasses removal
CN113378980B (en) Mask face shielding recovery method based on self-adaptive context attention mechanism

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant