CN107507141A - A kind of image recovery method based on adaptive residual error neutral net - Google Patents
A kind of image recovery method based on adaptive residual error neutral net Download PDFInfo
- Publication number
- CN107507141A CN107507141A CN201710666509.3A CN201710666509A CN107507141A CN 107507141 A CN107507141 A CN 107507141A CN 201710666509 A CN201710666509 A CN 201710666509A CN 107507141 A CN107507141 A CN 107507141A
- Authority
- CN
- China
- Prior art keywords
- image
- residual error
- network model
- adaptive residual
- resolution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003044 adaptive effect Effects 0.000 title claims abstract description 83
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000011084 recovery Methods 0.000 title claims abstract description 28
- 230000007935 neutral effect Effects 0.000 title claims abstract description 20
- 230000000694 effects Effects 0.000 claims abstract description 56
- 238000012549 training Methods 0.000 claims abstract description 54
- 238000003062 neural network model Methods 0.000 claims abstract description 29
- 210000005036 nerve Anatomy 0.000 claims abstract description 17
- 230000006870 function Effects 0.000 claims description 27
- 238000013507 mapping Methods 0.000 claims description 12
- 238000000576 coating method Methods 0.000 claims description 11
- 239000011248 coating agent Substances 0.000 claims description 8
- 230000001537 neural effect Effects 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 6
- 210000002569 neuron Anatomy 0.000 claims description 5
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 4
- 210000004205 output neuron Anatomy 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 abstract description 4
- 230000008569 process Effects 0.000 description 9
- 238000012360 testing method Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000000903 blocking effect Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 239000000654 additive Substances 0.000 description 2
- 230000000996 additive effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000004880 explosion Methods 0.000 description 2
- 210000004218 nerve net Anatomy 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 101100365548 Caenorhabditis elegans set-14 gene Proteins 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 210000002364 input neuron Anatomy 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of image recovery method based on adaptive residual error neutral net, including：Adaptive residual error neural network model is built, the adaptive residual error neutral net includes the multiple adaptive residual units being serially connected；The training set for image denoising, Image Super-resolution and image deblocking effect problem is chosen respectively, and corresponding training parameter is set respectively；According to the adaptive residual error neural network model and the training parameter for image denoising, Image Super-resolution and image deblocking effect problem, corresponding target nerve network model is respectively trained as target to minimize loss function；The target nerve network model for image denoising, Image Super-resolution and image deblocking effect problem obtained according to training, is input to corresponding target nerve network model by pending image, exports the image of corresponding high quality.The present invention can significantly improve PSNR, SSIM and visual effect of image, and recovery effect is good, speed is fast, strong robustness.
Description
Technical field
The present invention relates to computer vision and digital image processing field, more particularly to one kind is based on adaptive residual error nerve
The image recovery method of network.
Background technology
Image restoration, be computer vision and image procossing a classics and it is basic the problem of, be solve it is many related
The pretreatment indispensability process of problem, its purpose are to recover the image X of potential high quality, the mistake from low-quality image Y
Journey is represented by：Y=AX+N, wherein, N be typically considered additive white Gaussian noise (Additive White Gassian,
AWG), this is a typical ill linear inverse problem.If picture breakdown source only has white Gaussian noise, the problem becomes
Into image denoising；If without noise, and A is a down-sampling factor, then the problem becomes Image Super-resolution；If do not make an uproar
Sound, and A is a JPEG compression mass parameter, then the problem becomes image deblocking effect.The method for solving these problems is main
There are traditional algorithm and neural net method popular in the recent period.The existing typical solution party for these three different applications
Method is as follows.
Image denoising, the basis of many methods is non local self similarity (Nonlocal Self-Similarity, NSS) original
Reason.Wherein, an important method is Block-matching and 3D filtering (BM3D)：A 3D was established before this
Cube NSS image blocks, collaboration filtering then is carried out to image block in sparse 3D transform domains；Except utilizing noise image
NSS blocks, it is the NSS blocks using clean image that another, which has effective method, and its representational method has Patch Group
Prior based Denoising (PGPD), it is to utilize the Patch Group based drawn from clean image
Gaussian Mixture Model (PG-GMM) model carrys out the structure of approximate fits noise image, and then carries out denoising.
But these methods do not have the NSS for making full use of noise image and clean image simultaneously, the model for causing to obtain is less smart
Really；The denoising process of these other methods needs the substantial amounts of time, and does not possess robustness to the noise and resolution ratio of image,
Lack actual application value.Therefore, deep learning is applied in image denoising；Wherein representational is denoising autocoding
Machine (Denoising Auto-Encoder, DAE), it is noisy image to set input, exports the clean image for estimation,
Input and output are connected by autocoding；Another representational method is multi-layer perception (MLP) (Multi-Layer
Perception, MLP), by dilated data set, good effect is obtained, but it is actually a kind of fully connected network
Network, depth is 3 layers, and is free of convolutional layer, so the ability of network is not so strong.
Image Super-resolution, the most frequently used method are that fixed neighborhood returns (anchored neighborhood
Regression, ANR), then this method calculates phase using the sparse dictionary learnt, fixed regressor to dictionary atom
The embeded matrix answered, this method significantly reduce time complexity while the effect obtained；The principle is based on later
Improved method have A+, it combines ANR and simple function (simple functions, SF) thought, learns to scheme using the former
As feature and fixed regressor, but the study of regressor is as SF, is based on substantial amounts of training material；Therefore A+ methods
Effect is also very good.Later combined optimization regressor (jointly optimized regressors, JOR) proposes that study is fixed
The regressor of quantity, and can according to input picture carry out it is adaptively selected, be also in this way it is highly effective with
Efficiently.Deep learning is also applied to Image Super-resolution, and Typical Representative is SRCNN, and it is one three layers of convolutional layer, respectively
Effect be：Convolutional neural networks are applied to Image Super-resolution by feature extraction, Nonlinear Mapping and reconstruction layer, this method first
In, there is very strong reference, but the ability of the network of this method is also and less strong.
The generation of image deblocking effect, wherein blocking effect is due to the coarse quantization in JPEG compression change in process domain.Therefore
Substantial amounts of method is all based on the filtering in discrete cosine transform (Discrete Cosine Transformation, DCT) domain, its
Middle ANCE is typically to represent, and this method optimizes the conversion coefficient of overlapping block, this method using non local piece of original image
Achieve good effect.In addition, that earliest Application of Neural Network Technology deblocking effect is ARCNN, and this method is similar
SRCNN, build 4 layers of convolutional neural networks and carry out mapping study, the prediction of high quality graphic is carried out by the model trained, because
The shortcomings that this this method, is also similar to ARCNN.
Current method is substantially the one of which for three kinds of problems for being only capable of solving image, and lacking one kind can solve simultaneously
Certainly these three of image restore the method for problem.
The disclosure of background above technology contents is only used for design and the technical scheme that auxiliary understands the present invention, and it is not necessarily
Belong to the prior art of present patent application, no tangible proof show the above present patent application the applying date
In the case of disclosed, above-mentioned background technology should not be taken to evaluate the novelty and creativeness of the application.
The content of the invention
In order to solve the above technical problems, the present invention proposes a kind of image restoration side based on adaptive residual error neutral net
Method, Y-PSNR (PSNR), structural similarity (SSIM) and the visual effect of image can be significantly improved, there is recovery effect
Good, speed is fast and the advantages that strong robustness.
In order to achieve the above object, the present invention uses following technical scheme：
The invention discloses a kind of image recovery method based on adaptive residual error neutral net, comprise the following steps：
A1：Adaptive residual error neural network model is built, the adaptive residual error neutral net is more including being serially connected
Individual adaptive residual unit, wherein each adaptive residual error neural unit includes multiple convolutional layers, multiple active coatings, adaptive
Should be jumped connection unit, wherein after each active coating is correspondingly arranged at each convolutional layer respectively；
A2：The training set for image denoising, Image Super-resolution and image deblocking effect problem is chosen respectively, and is set respectively
Put the training for image denoising, Image Super-resolution and image deblocking effect problem of the adaptive residual error neural network model
Parameter；
A3：Gone according to the adaptive residual error neural network model and for image denoising, Image Super-resolution and image
The training parameter of blocking effect problem, the adaptive residual error neural network model is respectively trained as target to minimize loss function
To form the target nerve network model for image denoising, Image Super-resolution and image deblocking effect problem respectively；
A4：The target for image denoising, Image Super-resolution and image deblocking effect problem obtained according to training
Neural network model, pending image is input to corresponding target nerve network model, exports the figure of corresponding high quality
Picture.
Preferably, the active coating is PReLU functions.
Preferably, the adaptive residual error neural unit specifically include two 3 × 3 convolutional layers and be serially connected 6 from
Residual unit is adapted to, the convolutional layer of two of which 3 × 3 is connected to the head and the tail two of the adaptive residual unit of 6 be serially connected
End.
Preferably, the multiple convolutional layer in each residual unit is made up of two 3 × 3 convolutional layers.
Preferably, the adaptive jump connection unit is that two parameter alphas and β will be inputted and output connects by introducing
Get up, wherein α is used to control the influence being originally inputted to deep layer network below, and β is used to control current output layer to deep layer below
The influence of network.
Preferably, the training set in step A2 for image denoising, Image Super-resolution and image deblocking effect problem is distinguished
Including multiple low-quality images and corresponding high quality graphic, the pin of the adaptive residual error neural network model is being set respectively
To also including before the training parameter of image denoising, Image Super-resolution and image deblocking effect problem：By the low-quality image
Multiple low-quality image blocks and multiple high quality graphic blocks are divided into respectively with the high quality graphic, wherein the low quality figure
As the quantity and size of block and the high quality graphic block are all identical.
Preferably, wherein before for image denoising and image deblocking effect problem arrange parameter, by the low-quality spirogram
Low-quality image block as being divided into 50 × 50, the high quality graphic is divided into 50 × 50 high quality graphic block；In pin
Before training parameter is set to image super-resolution problem, the low-quality image is divided into 33 × 33 low-quality image
Block, the high quality graphic is divided into 33 × 33 high quality graphic block.
Preferably, the loss function in step A3 is MSE functions：
Wherein, Xi、YiThe low-quality image block and high quality graphic block of image in the training set respectively chosen, θ represent power
Weight, n represent the number of image block, the mapping of the low-quality image that F function representations train to high quality graphic.
Preferably, it is described adaptive residual in step A3 during the adaptive residual error neural network model is trained
The weight θ of poor neutral net initialization uses Xavier methods：Wherein, ninFor input neuron
Number, noutFor the number of output neuron；Minimize loss function and use Adam optimization methods.
Preferably, the target for image denoising, Image Super-resolution and image deblocking effect problem in step A3
Neural network model is respectively according to the minimum loss letter for image denoising, Image Super-resolution and image deblocking effect problem
Number obtains the weights of corresponding convolutional layers to establish.
Compared with prior art, the beneficial effects of the present invention are：The present invention based on adaptive residual error neutral net
Image recovery method, convolutional layer, activation are introduced in each adaptive residual unit in adaptive residual error neural network model
The learning ability of layer and adaptive jump connection unit, greatly strength neural network, and can surpass for image denoising, image
Differentiate and image deblocking effect problem sets up accurate mapping of the low-quality image to high quality graphic respectively, can finally pass through
The effective mapping established is handled the image under corresponding recovery problem, to obtain the image of high quality；Pass through the present invention's
Image recovery method, Y-PSNR (PSNR), structural similarity (SSIM) and the visual effect of image can be significantly improved, and
And efficiency, quality and the robustness of image restoration are greatly lifted, there is far-reaching meaning in computer vision and image processing field
Justice.
In further scheme, the present invention can also have the advantages that：
In adaptive residual unit in the adaptive residual error neural network model that the present invention is built after each convolutional layer
Active coating is PReLU functions, can be avoided all putting anon-normal element by introducing customized parameter from PReLU functions
Zero, 0 neuron is adaptively retained less than, good effect is obtained in terms of formal neuron is retained, contrasts ReLU function energy
More effective feature is enough filtered out, and then effectively avoids gradient explosion issues and accelerates training process.
Processing includes mutually string in adaptive residual unit in the adaptive residual error neural network model that the present invention is built
Multiple adaptive residual units of connection, one is also connected respectively at the head and the tail both ends for the multiple adaptive residual units being serially connected
3 × 3 convolutional layers, further increase network depth, improve effect.
Multiple convolutional layers in adaptive residual unit in the adaptive residual error neural network model that the present invention is built are equal
From 3 × 3 convolution kernel so that need not introduce pond layer can just be easy to train and have enough abilities to obtain figure well
Picture recovery effect, so as to avoid because introducing pond layer so that inaccurate, effect variation of model etc. is asked caused by parameter reduction
Topic.
Adaptive jump in adaptive residual unit in the adaptive residual error neural network model that the present invention is built connects
Order member sets up the selectable contact for being input to output by introducing two parameter alphas and β, remains more input pictures
Detailed information, more conducively increase network training process convergence rate；And can be by adaptively coordinating two parameters
Size, to obtain optimal numerical value so that overall network performance is best.
Brief description of the drawings
Fig. 1 is the flow chart of the image recovery method based on adaptive residual error neutral net of the preferred embodiment of the present invention；
Fig. 2 is the internal structure schematic diagram of the adaptive residual error neutral net of the preferred embodiment of the present invention；
Fig. 3 is the structural representation of the adaptive residual unit in Fig. 2；
Fig. 4 is the schematic diagram of the PReLU functions in Fig. 3.
Embodiment
Below against accompanying drawing and with reference to preferred embodiment, the invention will be further described.
As shown in figure 1, the preferred embodiment of the present invention discloses a kind of image restoration based on adaptive residual error neutral net
Method, comprise the following steps：
A1：Adaptive residual error neural network model is built, wherein adaptive residual error neutral net is more including being serially connected
Individual adaptive residual unit, each adaptive residual error neural unit include multiple convolutional layers, multiple active coatings, the adaptive company of jump
Order member, wherein after each active coating is correspondingly arranged at each convolutional layer respectively；
As shown in Fig. 2 in the present embodiment, adaptive residual error neutral net includes two 3 × 3 convolutional layers and is serially connected
6 adaptive residual units, the convolutional layer of two of which 3 × 3 is connected to the adaptive residual unit of 6 be serially connected
Head and the tail both ends.As shown in figure 3, adaptive residual unit is by corresponding active coating after 2 convolutional layers, each convolutional layer and adaptively
The connection unit (shortcut) that jumps is formed；The convolution kernel size of wherein 2 convolutional layers is 3 × 3, from the convolutional layer of the size
One side has the effect of extraction feature well, and another aspect parameter is also no so much, and amount of calculation is also just little, convenient real
It is existing.There is corresponding active coating behind each convolutional layer, hidden in the present embodiment from (as shown in Figure 4) be used as of PReLU functions
The activation primitive of layer, can be avoided, by anon-normal element whole zero setting, protecting from PReLU functions by introducing customized parameter
Good effect is obtained in terms of staying formal neuron, and then effectively avoids gradient explosion issues and accelerates training process.This implementation
The adaptive jump connection unit in each adaptive residual unit in example is not directly to take unit mapping will input and defeated
Go out to connect, but introduce two parameter alphas and β, wherein α for controlling the influence being originally inputted to deep layer network below, β is used
To control influence of the current output layer to deep layer network below, by adaptively coordinating the size of two parameters, to obtain most
Good numerical value so that overall network performance is best；The figure of more inputs is delivered by the adaptive jump connection unit
As information, extraction of the enhancing network to feature.
Under normal circumstances, convolutional neural networks can include slowization layer, and pond layer is generally possible to equivalent to very strong regular terms
Reduce next layer of input size, reduce amount of calculation and number of parameters.But by from suitable big in the preferred embodiment of the present invention
The number of plies and convolution kernel size of small convolutional layer, avoid and introduce too big amount of calculation and parameter, therefore pond need not be introduced
Layer, avoid because introducing pond layer causes the problems such as model is inaccurate, effect is deteriorated caused by parameter reduction.The present invention
The adaptive residual error neutral net being made up of in embodiment this 6 adaptive residual units and two convolutional layers is selected to be easy to
Training, and can enough have enough abilities to obtain good recovery effect.
A2：The training set and test set for image denoising, Image Super-resolution and image deblocking effect problem are chosen respectively,
And set respectively adaptive residual error neural network model for image denoising, Image Super-resolution and image deblocking effect problem
Training parameter；
Image denoising problem is wherein directed to, 500 images chosen in BSD500 have respectively as training set, every image
Corresponding noise image and clean image；Then the instruction for image denoising problem of adaptive residual error neural network model is set
Practice parameter, including the image number of blocks of each input model training, the size of input picture block and output image block, image depth
Degree, learning rate etc..To increase data set, the noise image in training set and clean image are divided into same resolution ratio respectively
Image block；And it is that " SAME " (i.e. by Effect of Interpolation, the size of image will not be according to the size of convolution kernel to set padding
Reduce, that is, input consistent with output size), therefore the noise image for being divided into noise image and clean image respectively in the present invention
The quantity and size of block and clean image block, wherein noise image block and clean image block are all identical, and increase data set can have
Avoid to effect the over-fitting in training process.In the present embodiment, the noise image in training set is divided into 50 × 50
Noise image block, clean image is divided into 50 × 50 clean image block so that can preferably be caught in training pattern
Catch the structural information and detailed information of image；The quantity of the image block of each input model training is for 128 (in other embodiment
In, the arbitrary value in 100~200 can also be taken)；Due to being directed to the denoising of gray-scale map, picture depth is set to 1；Study speed
Rate is set to 0.001 (in other embodiments, can also take the arbitrary value in 0.1~0.001), every time rate of decay during training
It is set to 0.9 (arbitrary value in 0.1~0.9 in other embodiments, can also be taken)；500 times are often trained (in other embodiment
In, can also be 500~1000 times) once tested, the related of model is changed according to effect of the model on test set and joined
Number.Wherein, test set can also be chosen while training set is chosen, can select that denoising field is conventional in test set 15
Image, every image in test set similarly include noise image and corresponding clean image.
For Image Super-resolution problem, 91 images of conventional high quality are chosen as training set, every image difference
There are corresponding low-resolution image and high-definition picture；Then surpassing for image for adaptive residual error neural network model is set
The training parameter of resolution problems, including the image number of blocks of each input model training, input picture block and output image block
Size, picture depth, learning rate etc..Wherein be also provided with padding for " SAME " to increase data set, avoid in training process
Over-fitting.In the present embodiment, the low-resolution image in training set is divided into 33 × 33 low-resolution image
Block, high-definition picture is divided into 33 × 33 high-definition picture block so that can preferably catch in training pattern
The structural information and detailed information of image；Other training parameters are identical with the foregoing training parameter for image denoising problem.Survey
Examination collection chooses Set5, Set14, BSD100 or Urban100 image set.
For image deblocking effect (JPEG deblocking effects), selection and the foregoing instruction for image denoising problem of training set
Practice collection, training parameter is identical with the foregoing training parameter for image denoising problem.Test set chooses Classic5 or LIVE1 figures
Image set.
A3：Imitated according to adaptive residual error neural network model and for image denoising, Image Super-resolution and image deblocking
The training parameter of problem is answered, adaptive residual error neural network model is respectively trained with shape respectively as target to minimize loss function
Into the target nerve network model for image denoising, Image Super-resolution and image deblocking effect problem；
Specifically, it can be obtained according to minimum loss function and be imitated for image denoising, Image Super-resolution and image deblocking
The adaptive residual error parameter of problem and the weight and deviation of convolutional layer are answered, so as to establish respectively for image denoising, image
The target nerve network model of super-resolution and image deblocking effect problem.
Wherein loss function (Loss functions) elects MSE functions as：
Wherein, for image denoising problem, Xi、YiThe noise image block of image in the training set respectively chosen and dry
Net image block, θ represent weight, and n represents the number of image block, and the noise image that F function representations train reflects to clean image
Penetrate；For Image Super-resolution problem, Xi、YiThe low-resolution image block and high-resolution of image in the training set respectively chosen
Rate image block, θ represent weight, and n represents the number of image block, the low-resolution image that F function representations train to high-resolution
The mapping of image；For image deblocking effect problem, Xi、YiThe low-quality image block of image in the training set respectively chosen
With high quality graphic block, θ represents weight, and n represents the number of image block, and the low-quality image that F function representations train is to high-quality
The mapping of spirogram picture；.
Because Y-PSNR (PSNR) formula is：
Wherein, MAX is typically the gray level of image, typically takes 255, as can be seen from the above equation, constantly minimizes loss letter
Number is obtained with high Y-PSNR (PSNR) value, i.e. the quality of image is higher.
In the present embodiment, minimize loss function and use Adam optimization methods, wherein Adam optimization methods calculation
It is that per time step iteration once, (first and second is dynamic for the subduplicate attenuation of average gradient of calculating and average gradient
Amount estimation), the first momentum can not short decay over time, due to the first and second momentum initial value be 0, then cause some power
Weight coefficient is changed into 0；Therefore it can effectively avoid optimization process from entering locally optimal solution, and accelerate optimal speed, it is complete to obtain
Office's optimal solution.
In the present embodiment, the weight θ of adaptive residual error neural network model initialization uses Xavier methods：Wherein, ninTo input the number of neuron, noutFor the number of output neuron, weight θ initialization clothes
From the distribution of 0 average and particular variance；Operated by these, can greatly reduce the complexity of network, improve the training of network
Efficiency.
A4：The target nerve for image denoising, Image Super-resolution and image deblocking effect problem obtained according to training
Network model, pending image is input to corresponding target nerve network model, exports the image of corresponding high quality.
Wherein, for image denoising problem, can be chosen comprising a variety of noise variances in the training set in step A2
Multiple images, multiple images of a variety of noise variances are respectively trained in step A3 adaptive residual error neural network model formed it is more
Target nerve network model under noise variance corresponding to kind.Pending image is input in step A4 corresponding to the image
Noise variance under target nerve network model, you can predict corresponding to clean image, export denoising after image.For
Image Super-resolution and image deblocking effect problem, similarly can be with as it was previously stated, corresponding choose includes the more of a variety of level problems
Open image.
In an example, in the case where noise variance is 25, the noise image PSNR of one 256 × 256 is
20.25, after model maps, the PSNR of clean image is 29.18；When the down-sampling factor is 2, one 256 × 256 low
Image in different resolution PSNR is 27.43, and after model maps, the PSNR of high-definition picture is 32.92；In JPEG quality coefficients
For 10 when, the obvious image PSNR of 256 × 256 blocking effects be 24.33, after model maps, high quality graphic
PSNR is 24.97；Adaptive residual error network model under the different application trained by the present invention, drastically increases image
Quality, visual effect are also satisfactory.
According to the image recovery method of the present invention, the target nerve network model that can be trained in advance, target nerve net
Network model is the end-to-end direct mapping by input low-quality image to outputting high quality image, passes through the target nerve net
The speed that network model is restored to image is exceedingly fast, and the high quality graphic of a reconstruction was just obtained less than 0.5 second, there is very strong reality
With value, very big application will be had in the occasion for needing to restore in real time；Except the advantages that speed is fast, recovery effect is good, this hair
Bright to also have very strong robustness, for different image problems, the time of recovery and effect there is no change.Therefore, originally
Invent that the adaptive residual error neutral net recovery effect that is provided is good, speed is fast, strong robustness, there is very strong practicality and in real time
Property, wide market, especially to the good occasion of requirement of real-time.
The image recovery method based on adaptive residual error neutral net of the preferred embodiments of the present invention, can accurately be learned
Practise the reflecting from low-quality image to high quality graphic for being directed to image denoising, Image Super-resolution and image deblocking effect problem respectively
Penetrate, by the learning ability of convolutional layer, the screening reserve capability of PReLU functions, the adaptive feature obtained, it is established that input
To the mapping of output, so as to carry out the prediction and estimation of high quality graphic by the mapping learnt.
Above content is to combine specific preferred embodiment further description made for the present invention, it is impossible to is assert
The specific implementation of the present invention is confined to these explanations.For those skilled in the art, do not taking off
On the premise of from present inventive concept, some equivalent substitutes or obvious modification can also be made, and performance or purposes are identical, all should
When being considered as belonging to protection scope of the present invention.
Claims (10)
1. a kind of image recovery method based on adaptive residual error neutral net, it is characterised in that comprise the following steps：
A1：Build adaptive residual error neural network model, the adaptive residual error neutral net include being serially connected it is multiple from
Residual unit is adapted to, wherein each adaptive residual error neural unit includes multiple convolutional layers, multiple active coatings, adaptively jumped
Jump connection unit, wherein after each active coating is correspondingly arranged at each convolutional layer respectively；
A2：The training set for image denoising, Image Super-resolution and image deblocking effect problem is chosen respectively, and institute is set respectively
The training for image denoising, Image Super-resolution and image deblocking effect problem for stating adaptive residual error neural network model is joined
Number；
A3：Imitated according to the adaptive residual error neural network model and for image denoising, Image Super-resolution and image deblocking
The training parameter of problem is answered, the adaptive residual error neural network model is respectively trained to divide as target to minimize loss function
The target nerve network model of image denoising, Image Super-resolution and image deblocking effect problem Xing Cheng be directed to；
A4：The target nerve for image denoising, Image Super-resolution and image deblocking effect problem obtained according to training
Network model, pending image is input to corresponding target nerve network model, exports the image of corresponding high quality.
2. image recovery method according to claim 1, it is characterised in that the active coating is PReLU functions.
3. image recovery method according to claim 1, it is characterised in that the adaptive residual error neural unit specifically wraps
Two 3 × 3 convolutional layers and 6 adaptive residual units being serially connected are included, the convolutional layer of two of which 3 × 3 is connected to phase
The head and the tail both ends for the 6 adaptive residual units mutually connected.
4. image recovery method according to claim 1, it is characterised in that the multiple in each residual unit
Convolutional layer is made up of two 3 × 3 convolutional layers.
5. image recovery method according to claim 1, it is characterised in that the adaptive jump connection unit is to pass through
Introduce two parameter alphas and β will be inputted and output connects, wherein α is used for control and is originally inputted to deep layer network below
Influence, β is used to control influence of the current output layer to deep layer network below.
6. image recovery method according to claim 1, it is characterised in that surpass in step A2 for image denoising, image
Differentiate and the training set of image deblocking effect problem includes multiple low-quality images and corresponding high quality graphic respectively, respectively
The instruction for image denoising, Image Super-resolution and image deblocking effect problem of the adaptive residual error neural network model is set
Also include before practicing parameter：By the low-quality image and the high quality graphic be divided into respectively multiple low-quality image blocks and
Multiple high quality graphic blocks, wherein the quantity and size of the low-quality image block and the high quality graphic block are all identical.
7. image recovery method according to claim 6, it is characterised in that wherein for image denoising and image deblocking
Before effect problem arrange parameter, the low-quality image is divided into 50 × 50 low-quality image block, by the high-quality
High quality graphic block as being divided into 50 × 50；, will be described low before for image super-resolution problem, training parameter is set
Quality image is divided into 33 × 33 low-quality image block, and the high quality graphic is divided into 33 × 33 high quality graphic
Block.
8. image recovery method according to claim 1, it is characterised in that the loss function in step A3 is MSE functions：
<mrow>
<mi>L</mi>
<mrow>
<mo>(</mo>
<mi>&theta;</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mfrac>
<mn>1</mn>
<mrow>
<mn>2</mn>
<mi>n</mi>
</mrow>
</mfrac>
<munderover>
<mo>&Sigma;</mo>
<mrow>
<mi>i</mi>
<mo>=</mo>
<mn>1</mn>
</mrow>
<mi>n</mi>
</munderover>
<mo>|</mo>
<mo>|</mo>
<mi>F</mi>
<mrow>
<mo>(</mo>
<msub>
<mi>X</mi>
<mi>i</mi>
</msub>
<mo>,</mo>
<mi>&theta;</mi>
<mo>)</mo>
</mrow>
<mo>-</mo>
<msub>
<mi>Y</mi>
<mi>i</mi>
</msub>
<mo>|</mo>
<msubsup>
<mo>|</mo>
<mn>2</mn>
<mn>2</mn>
</msubsup>
</mrow>
Wherein, Xi、YiThe low-quality image block and high quality graphic block of image in the training set respectively chosen, θ represent weight, n
Represent the number of image block, the mapping of the low-quality image that F function representations train to high quality graphic.
9. image recovery method according to claim 8, it is characterised in that training the adaptive residual error in step A3
During neural network model, the weight θ of adaptive residual error neutral net initialization uses Xavier methods：Wherein, ninTo input the number of neuron, noutFor the number of output neuron；Minimize loss letter
Number uses Adam optimization methods.
10. image recovery method according to claim 8 or claim 9, it is characterised in that in step A3 for image denoising,
Image Super-resolution and the target nerve network model of image deblocking effect problem are respectively according to for image denoising, image
The minimum loss function of super-resolution and image deblocking effect problem obtains the weight of corresponding convolutional layer to establish.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710666509.3A CN107507141A (en) | 2017-08-07 | 2017-08-07 | A kind of image recovery method based on adaptive residual error neutral net |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710666509.3A CN107507141A (en) | 2017-08-07 | 2017-08-07 | A kind of image recovery method based on adaptive residual error neutral net |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107507141A true CN107507141A (en) | 2017-12-22 |
Family
ID=60690200
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710666509.3A Pending CN107507141A (en) | 2017-08-07 | 2017-08-07 | A kind of image recovery method based on adaptive residual error neutral net |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107507141A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108235058A (en) * | 2018-01-12 | 2018-06-29 | 广州华多网络科技有限公司 | Video quality processing method, storage medium and terminal |
| CN108876737A (en) * | 2018-06-06 | 2018-11-23 | 武汉大学 | A kind of image de-noising method of joint residual error study and structural similarity |
| CN108921785A (en) * | 2018-06-14 | 2018-11-30 | 厦门大学 | Super resolution ratio reconstruction method based on wavelet packet |
| CN109035163A (en) * | 2018-07-09 | 2018-12-18 | 南京信息工程大学 | A kind of adaptive denoising method based on deep learning |
| CN109284719A (en) * | 2018-09-28 | 2019-01-29 | 成都臻识科技发展有限公司 | A kind of primary data processing method and system based on machine learning |
| CN109658344A (en) * | 2018-11-12 | 2019-04-19 | 哈尔滨工业大学（深圳） | Image de-noising method, device, equipment and storage medium based on deep learning |
| CN110148091A (en) * | 2019-04-10 | 2019-08-20 | 深圳市未来媒体技术研究院 | Neural network model and image super-resolution method based on non local attention mechanism |
| CN110223273A (en) * | 2019-05-16 | 2019-09-10 | 天津大学 | A kind of image repair evidence collecting method of combination discrete cosine transform and neural network |
| CN111028174A (en) * | 2019-12-10 | 2020-04-17 | 深圳先进技术研究院 | Multi-dimensional image restoration method and equipment based on residual connection |
| WO2020103171A1 (en) * | 2018-11-21 | 2020-05-28 | 北京大学深圳研究生院 | Bi-level optimization method for image deblurring |
| CN111353504A (en) * | 2020-03-02 | 2020-06-30 | 济南大学 | Source and machine identification method based on image block diversity selection and residual prediction module |
| CN112907464A (en) * | 2021-02-01 | 2021-06-04 | 涂可致 | Underwater thermal disturbance image restoration method |
| CN114299550A (en) * | 2022-01-05 | 2022-04-08 | 南通理工学院 | Method for defending against noninductive noise attack in pedestrian re-identification system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106204467A (en) * | 2016-06-27 | 2016-12-07 | 深圳市未来媒体技术研究院 | A kind of image de-noising method based on cascade residual error neutral net |
| WO2017116812A1 (en) * | 2015-12-28 | 2017-07-06 | Microsoft Technology Licensing, Llc | Linearly augmented neural network |
-
2017
- 2017-08-07 CN CN201710666509.3A patent/CN107507141A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017116812A1 (en) * | 2015-12-28 | 2017-07-06 | Microsoft Technology Licensing, Llc | Linearly augmented neural network |
| CN106204467A (en) * | 2016-06-27 | 2016-12-07 | 深圳市未来媒体技术研究院 | A kind of image de-noising method based on cascade residual error neutral net |
Non-Patent Citations (3)
| Title |
|---|
| KAI ZHANG: "Beyond a Gaussian Denoiser:Residual Learning of Deep CNN for Image Denoising", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》 * |
| KAIMING HE 等: "Deep Residual Learning for Image Recognition", 《2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| 李鸣 等: "基于卷积神经网络迭代优化的图像分类算法", 《计算机工程与设计》 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108235058A (en) * | 2018-01-12 | 2018-06-29 | 广州华多网络科技有限公司 | Video quality processing method, storage medium and terminal |
| CN108876737A (en) * | 2018-06-06 | 2018-11-23 | 武汉大学 | A kind of image de-noising method of joint residual error study and structural similarity |
| CN108876737B (en) * | 2018-06-06 | 2021-08-03 | 武汉大学 | Image denoising method combining residual learning and structural similarity |
| CN108921785B (en) * | 2018-06-14 | 2020-07-07 | 厦门大学 | Super-resolution reconstruction method based on wavelet packet |
| CN108921785A (en) * | 2018-06-14 | 2018-11-30 | 厦门大学 | Super resolution ratio reconstruction method based on wavelet packet |
| CN109035163A (en) * | 2018-07-09 | 2018-12-18 | 南京信息工程大学 | A kind of adaptive denoising method based on deep learning |
| CN109035163B (en) * | 2018-07-09 | 2022-02-15 | 南京信息工程大学 | Self-adaptive image denoising method based on deep learning |
| CN109284719A (en) * | 2018-09-28 | 2019-01-29 | 成都臻识科技发展有限公司 | A kind of primary data processing method and system based on machine learning |
| CN109658344A (en) * | 2018-11-12 | 2019-04-19 | 哈尔滨工业大学（深圳） | Image de-noising method, device, equipment and storage medium based on deep learning |
| WO2020103171A1 (en) * | 2018-11-21 | 2020-05-28 | 北京大学深圳研究生院 | Bi-level optimization method for image deblurring |
| CN110148091A (en) * | 2019-04-10 | 2019-08-20 | 深圳市未来媒体技术研究院 | Neural network model and image super-resolution method based on non local attention mechanism |
| CN110223273A (en) * | 2019-05-16 | 2019-09-10 | 天津大学 | A kind of image repair evidence collecting method of combination discrete cosine transform and neural network |
| CN111028174A (en) * | 2019-12-10 | 2020-04-17 | 深圳先进技术研究院 | Multi-dimensional image restoration method and equipment based on residual connection |
| WO2021115053A1 (en) * | 2019-12-10 | 2021-06-17 | 深圳先进技术研究院 | Residual connection-based multi-dimensional image restoration method and device |
| CN111028174B (en) * | 2019-12-10 | 2023-08-04 | 深圳先进技术研究院 | Multi-dimensional image restoration method and device based on residual connection |
| CN111353504A (en) * | 2020-03-02 | 2020-06-30 | 济南大学 | Source and machine identification method based on image block diversity selection and residual prediction module |
| CN111353504B (en) * | 2020-03-02 | 2023-05-26 | 济南大学 | Source camera identification method based on image block diversity selection and residual prediction module |
| CN112907464A (en) * | 2021-02-01 | 2021-06-04 | 涂可致 | Underwater thermal disturbance image restoration method |
| CN114299550A (en) * | 2022-01-05 | 2022-04-08 | 南通理工学院 | Method for defending against noninductive noise attack in pedestrian re-identification system |
| CN114299550B (en) * | 2022-01-05 | 2024-02-27 | 南通理工学院 | Defending method for noise-free attack in pedestrian re-recognition system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107507141A (en) | A kind of image recovery method based on adaptive residual error neutral net | |
| CN106204468B (en) | A kind of image de-noising method based on ReLU convolutional neural networks | |
| CN106204467A (en) | A kind of image de-noising method based on cascade residual error neutral net | |
| WO2023092813A1 (en) | Swin-transformer image denoising method and system based on channel attention | |
| CN106683067B (en) | Deep learning super-resolution reconstruction method based on residual sub-images | |
| CN107463989B (en) | A kind of image based on deep learning goes compression artefacts method | |
| CN106408522A (en) | Image de-noising method based on convolution pair neural network | |
| CN109727195B (en) | Image super-resolution reconstruction method | |
| CN110223234A (en) | Depth residual error network image super resolution ratio reconstruction method based on cascade shrinkage expansion | |
| CN114049261B (en) | Image super-resolution reconstruction method focusing on foreground information | |
| CN110675321A (en) | Super-resolution image reconstruction method based on progressive depth residual error network | |
| CN110689495B (en) | Image restoration method for deep learning | |
| CN108921789A (en) | Super-resolution image reconstruction method based on recurrence residual error network | |
| CN107784628A (en) | A kind of super-resolution implementation method based on reconstruction optimization and deep neural network | |
| CN111784619A (en) | Fusion method of infrared and visible light images | |
| CN112884650B (en) | Image mixing super-resolution method based on self-adaptive texture distillation | |
| CN107734333A (en) | A kind of method for improving video error concealing effect using network is generated | |
| CN112991231B (en) | Single-image super-image and perception image enhancement joint task learning system | |
| CN107392868A (en) | Compression binocular image quality enhancement method and device based on full convolutional neural networks | |
| CN111126599A (en) | Neural network weight initialization method based on transfer learning | |
| CN112785539A (en) | Multi-focus image fusion method based on image adaptive decomposition and parameter adaptive | |
| CN109118428A (en) | A kind of image super-resolution rebuilding method based on feature enhancing | |
| CN109272450B (en) | Image super-resolution method based on convolutional neural network | |
| Liu et al. | Facial image inpainting using multi-level generative network | |
| Wang et al. | Frequency compensated diffusion model for real-scene dehazing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Zhang Yongbing Inventor after: Ji Xiangyang Inventor after: Sun Lulu Inventor after: Wang Haoqian Inventor after: Wang Xingzheng Inventor after: Dai Qionghai Inventor before: Zhang Yongbing Inventor before: Sun Lulu Inventor before: Wang Haoqian Inventor before: Wang Xingzheng Inventor before: Dai Qionghai |
|
| CB03 | Change of inventor or designer information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171222 |
|
| RJ01 | Rejection of invention patent application after publication |