CN110378844A - Motion blur method is gone based on the multiple dimensioned Image Blind for generating confrontation network is recycled - Google Patents
Motion blur method is gone based on the multiple dimensioned Image Blind for generating confrontation network is recycled Download PDFInfo
- Publication number
- CN110378844A CN110378844A CN201910515590.4A CN201910515590A CN110378844A CN 110378844 A CN110378844 A CN 110378844A CN 201910515590 A CN201910515590 A CN 201910515590A CN 110378844 A CN110378844 A CN 110378844A
- Authority
- CN
- China
- Prior art keywords
- image
- size
- convolutional layer
- module
- residual error
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 230000033001 locomotion Effects 0.000 title claims abstract description 16
- 238000004064 recycling Methods 0.000 claims abstract description 7
- 238000012549 training Methods 0.000 claims description 34
- 230000006870 function Effects 0.000 claims description 30
- 230000004913 activation Effects 0.000 claims description 27
- 230000015654 memory Effects 0.000 claims description 15
- 230000004069 differentiation Effects 0.000 claims description 7
- 230000007423 decrease Effects 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 238000003475 lamination Methods 0.000 claims description 5
- 238000009826 distribution Methods 0.000 claims description 4
- 238000005549 size reduction Methods 0.000 claims description 4
- 239000011248 coating agent Substances 0.000 claims description 3
- 238000000576 coating method Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 230000000644 propagated effect Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 3
- 238000011084 recovery Methods 0.000 abstract description 3
- 230000008485 antagonism Effects 0.000 abstract 1
- 238000011478 gradient descent method Methods 0.000 abstract 1
- 230000006403 short-term memory Effects 0.000 description 3
- 239000004744 fabric Substances 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000001556 precipitation Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
Motion blur method is gone based on the Image Blind for recycling multiple dimensioned generation confrontation network the invention discloses a kind of.The method of the present invention constructs corresponding decision device to recycle multiple dimensioned encoder and decoder as generator.Antagonism to generate image and clear image is lost, multiple dimensioned mean square error and multi-scale gradient error are as the loss function for generating confrontation network, optimizes loss function with gradient descent method.The present invention eliminates complicated fuzzy core estimation procedure with the relationship generated between the confrontation corresponding clear image of e-learning motion blur image.The method of the present invention can extract the edge feature of image, have simpler network structure, less parameter, and the network model is easier to train, and recovery effect is preferable.
Description
Technical field
The invention belongs to technical field of image processing, are related to a kind of based on the Image Blind for recycling multiple dimensioned generation confrontation network
Go motion blur method.
Background technique
Due to being difficult to keep relative static conditions between capture apparatus and imaging object, the movement mould of image will cause
Paste.But in fields such as daily life, traffic safety, medicine, military investigations, capable of obtaining a width, clearly image all seems
It is particularly important.
The fuzzy of moving image can be regarded as after clear image and a two-dimensional linear convolution of functions operation by additivity
Noise pollution and formed.The linear function is referred to as point spread function or convolution kernel, it contains the fuzzy message of image.Figure
The blind deblurring of picture refers in the case where fuzzy manner unknown (i.e. fuzzy core is unknown), only relies on the information of blurred picture itself
To restore original clear image.In the blind deblurring of single width moving image, the fuzzy core and its size of blurred picture are unknown,
This can all influence the accuracy of fuzzy kernel estimates, and then influence final recovery effect.
Summary of the invention
The purpose of the present invention is to image motions to obscure this feature, provides a kind of based on the multiple dimensioned generation pair of circulation
The Image Blind of anti-network goes motion blur method, and this method can estimate clear image without ambiguous estimation core.
The present invention specifically includes the following steps:
Step (1) constructs arbiter D；
The arbiter D is made of nine convolutional layers, a full articulamentum and a Sigmoid active coating, and input is big
The small color image for being 256 × 256.
Each convolutional layer is all made of LeakyReLU as activation primitive: first layer has 32 convolution kernels, each convolution kernel ruler
Very little is 5 × 5, step-length 2, and zero filling width (zero-padding) is 2；The second layer has 64 convolution kernels, each convolution kernel size
It is 5 × 5, step-length 1, zero filling width is 2；Third layer has 64 convolution kernels, and each convolution kernel is filled out having a size of 5 × 5, step-length 2
Zero width is 2；4th layer has 128 convolution kernels, and for each convolution kernel having a size of 5 × 5, step-length 1, zero filling width is 2；Layer 5
There are 128 convolution kernels, for each convolution kernel having a size of 5 × 5, step-length 4, zero filling width is 2；Layer 6 has 256 convolution kernels, often
For a convolution kernel having a size of 5 × 5, step-length 1, zero filling width is 2；Layer 7 has 256 convolution kernels, and each convolution kernel is having a size of 5
× 5, step-length 4, zero filling width is 2；8th layer has 512 convolution kernels, and each convolution kernel is having a size of 5 × 5, step-length 1, zero filling
Width is 2；9th layer has 512 convolution kernels, and for each convolution kernel having a size of 4 × 4, step-length 4, zero filling width is 0.
It is 512 that the convolution of the last layer, which is exported through input channel number, the full articulamentum that output channel number is 1, obtains 1 often
Number exports the probability of judgement after the activation of Sigmoid function.
Step (2) constructs generator G；
The generator G includes the sub-network of cascade three scales, and each sub-network includes 1 input module, 2
Coding module, the 1 long short-term memory of convolution (ConvLSTM) module of cascade, 2 decoder modules and 1 output module；Each mould
All contain residual error module in block, the residual error module cascades a convolution kernel by a convolutional layer and forms, and convolutional layer is to improve
Linear unit (Rectified Linear Unit, ReLU) is used as activation primitive；The convolution kernel of residual error module cascade it is defeated
It is the output of residual error module after being added out with the input of residual error module.
The input module includes an independent convolutional layer and the identical residual error module of three structures, independent convolution
The layer and nuclear volume of the convolutional layer convolution kernel of residual error module is 32, size is 5 × 5, step-length 1, zero filling width are 2, it is independent
Convolutional layer in use ReLU function as activation primitive.
First coding module includes an independent convolutional layer and the identical residual error module of three structures, independent convolutional layer
And the quantity of the convolutional layer convolution kernel of residual error module is 64, size is 5 × 5, step-length 2, zero filling width are 2, independent volume
Use ReLU function as activation primitive in lamination.
Second coding module includes an independent convolutional layer and the identical residual error module of three structures, independent convolutional layer
And the quantity of the convolutional layer convolution kernel of residual error module is 128, size is 5 × 5, step-length 2, zero filling width are 2, independent volume
Use ReLU function as activation primitive in lamination.
Input of the memory cell state output as decoder module in the long memory module in short-term of the convolution, convolution length
When memory module hidden state output and the hidden state input phase of the long memory module in short-term of convolution in next scale sub-network
Even；For the last one scale, the long output of memory module hidden state in short-term of convolution is not connect with other modules.
The structure of the long short-term memory of convolution (ConvLSTM) module is shown in Shi X, Chen Z, Hao W, et
al.Convolutional LSTM Network:a machine learning approach for precipitation
nowcasting[C]//International Conference on Neural Information Processing
Systems.2015, the page number: 802-810.
First decoder module includes the identical residual error module of three structures and an independent convolutional layer, independent convolutional layer
And the quantity of the convolutional layer convolution kernel of residual error module is 128, size is 5 × 5, step-length 2, zero filling width are 2, residual error module
Use ReLU function as activation primitive in cascade independent convolutional layer afterwards.
Second decoder module includes the identical residual error module of three structures and an independent convolutional layer, independent convolutional layer
And the quantity of the convolutional layer convolution kernel of residual error module is 64, size is 5 × 5, step-length 2, zero filling width are 2, residual error module
Use ReLU function as activation primitive in cascade independent convolutional layer afterwards.
The output module includes the identical residual error module of three structures and an independent convolutional layer, independent convolution
Layer and residual error module convolutional layer convolution kernel quantity be 32, size is 5 × 5, step-length 1, zero filling width are 2, residual error mould
Use ReLU function as activation primitive in cascade independent convolutional layer after block.
The generator for exporting third level scale exports image L3, size is 64 × 64, L3It obtains through up-sampling having a size of 128
× 128 image, as the input of second level scale, the generator of the second level scale of output 128 × 128 exports image L2；L2
Input of the image as first order scale having a size of 256 × 256 is obtained through up-sampling, exports 256 × 256 first order scale
Generator export image L1, as result images of deblurring.In the sub-network of cascade three scales, three sub-networks pair
Structure, port number, the convolution kernel size answered are all the same.In three channels of color image RGB, each channel weight is shared.
Step (3) randomly selects m (m >=16) blurred picture and corresponding clear image from training dataset T, and
Random cropping is separately constituted at 256 × 256 square region for trained fuzzy graph image set B and corresponding clear image collection
S, the amount of images of obtained B and S are m, and every image is 256 × 256 3 Channel Color images.By blurred picture
Collect B and input generator, obtains having the color image that m sizes are 256 × 256 in generator output image set L, L.
Generator output image set L and corresponding clear image collection S is successively used as the input of arbiter by step (4), is sentenced
Other device is sequentially output two groups of confidence levels as a result, every group of confidence level includes m probability value, determines that every image inputted is clear with this
Clear image generates image: if probability value is greater than 0.5, being determined as clear image；Probability value is less than or equal to 0.5, then is determined as
Generate image.
Step (5) constructs the loss function of training generator, loss function are as follows: ldb=lE+α1lgrad+α2ladv；
Wherein α1、α2For the regularization coefficient greater than 0, lEImage set L and corresponding clear image collection S is exported for generator
Between mean square error, it may be assumed that
Wherein, Li、SiGenerator output image and the clear image being illustrated respectively on the i-th scale, NiIt indicates in the i-th ruler
The number of pixels in all channels on degree image, i=1,2,3；It is multiple dimensioned by the image 3 times down-sampled figures for obtaining size reduction
Picture, wherein first order scale is the image of full size size, and since the second level, the size of every first order image is upper level image
The width of size, each half of height.
lgradFor gradient imageWithBetween gradient error, it may be assumed that
L in formulai(dx) and Li(dy) respectively indicate the horizontal gradient and vertical gradient of Li, Si(dx) and Si(dy) respectively indicate
SiHorizontal gradient and vertical gradient.
ladvThe differentiation error of image set L and corresponding clear image collection S are exported for generator, it may be assumed that
S~p (S) indicates that clear image s is taken from clear image collection S, p (S) and indicates the probability point of clear image collection S in formula
Cloth；B~p (B) indicates that blurred picture b is taken from fuzzy graph image set B, p (B) and indicates the probability distribution of fuzzy graph image set B；
D (s) indicates arbiter to the differentiation probability of input picture s, and G (b) expression is generated by input picture b through generator
Result images, E [] are indicated to taking expectation in bracket.
Step (6) is input to image is generated in arbiter together with clear image, and is updated using gradient decline iteration each
Weight parameter in layer network, continues to optimize ladv, until arbiter can not differentiate that the image of input is to generate image or clear
Image, that is, the probability value obtained and 0.5 difference variation be less than thr, 0.01≤thr≤0.08, at this time arbiter training terminate.
Step (7) is according to loss function ldb=lE+α1lgrad+α2ladvTraining generator, is input to generation for blurred picture
It in device, is obtained by propagated forward and generates image, compare the otherness for generating image and clear image, decline iteration using gradient
Update the weight parameter in each layer network, continuous loss function ldb=lE+α1lgrad+α2ladv, until Maker model training rank
The training set total losses functional value l of sectiondbVariation is less than threshold value Th, 0.001≤Th≤0.01, and generator training at this time terminates.
The step of step (8) repetition training process (3)~step (7), until the training set of Maker model training stage
Total losses functional value ldbVariation is less than threshold value Th, i.e. arbiter can not determine that the image of input is clear image or generation figure
Picture assert that Maker model and arbiter model training have reached convergence, blurred picture is input in generator at this time, obtains
The de-blurred image of estimation.
The method of the present invention learns the relationship between the corresponding clear image of motion blur image with deep learning method,
Eliminate complicated fuzzy core estimation procedure.By the comparative training of a large amount of blurred pictures and clear image, institute's climbing form type can be with
The edge feature of image is extracted, there is simpler network structure, less parameter, and the network model is easier to train,
And recovery effect is preferable.
Specific embodiment
Specific implementation of the invention will be described further below.
Fuzzy graph image set B is input in generator G, is obtained generator output image set L and is obtained as the input of arbiter D
To the differentiation result of arbiter.Similarly, clear image collection S is also used as the input of arbiter, obtains differentiating result.The judgement result
It indicates to determine that input is from clear image collection or generating image set is then determined as clear image if it is determined that result is greater than 0.5
Collect S；Otherwise, it is determined that exporting image set L for generator.The error for calculating the judgement result and true tag data, utilizes gradient
Descent algorithm optimizes arbiter, then calculates the error mean for generating image and clear image, is optimized using gradient descent algorithm
Generator.Alternative optimization arbiter and generator, until model is restrained.In experiment of the invention, totally 40 Wan Cihou, mould are trained
Type is restrained.
Motion blur method is gone based on the multiple dimensioned Image Blind for generating confrontation network is recycled, the specific steps are as follows:
S1, building arbiter D: arbiter D are by nine convolutional layers, a full articulamentum and a Sigmoid active coating group
At the color image that input size is 256 × 256.
Each convolutional layer is all made of LeakyReLU as activation primitive: first layer has 32 convolution kernels, each convolution kernel ruler
Very little is 5 × 5, step-length 2, and zero filling width is 2；The second layer has 64 convolution kernels, and each convolution kernel is having a size of 5 × 5, step-length 1,
Zero filling width is 2；Third layer has 64 convolution kernels, and for each convolution kernel having a size of 5 × 5, step-length 2, zero filling width is 2；4th
Layer has 128 convolution kernels, and for each convolution kernel having a size of 5 × 5, step-length 1, zero filling width is 2；Layer 5 has 128 convolution kernels,
For each convolution kernel having a size of 5 × 5, step-length 4, zero filling width is 2；Layer 6 has 256 convolution kernels, each convolution kernel having a size of
5 × 5, step-length 1, zero filling width is 2；Layer 7 has 256 convolution kernels, and each convolution kernel is filled out having a size of 5 × 5, step-length 4
Zero width is 2；8th layer has 512 convolution kernels, and for each convolution kernel having a size of 5 × 5, step-length 1, zero filling width is 2；9th layer
There are 512 convolution kernels, for each convolution kernel having a size of 4 × 4, step-length 4, zero filling width is 0.
It is 512 that the convolution of the last layer, which is exported through input channel number, the full articulamentum that output channel number is 1, obtains 1 often
Number exports the probability of judgement after the activation of Sigmoid function.
S2, building generator G: generator G include the sub-network of cascade three scales, and each sub-network includes 1 defeated
Enter module, 2 coding modules, 1 convolution of cascade long memory module, 2 decoder modules and 1 output module in short-term；Each mould
All contain residual error module in block, the residual error module cascades a convolution kernel by a convolutional layer and forms, and convolutional layer is to improve
ELU is as activation primitive for Linear unit R；After the output of the convolution kernel of residual error module cascade is added with the input of residual error module
The as output of residual error module.
Input module include an independent convolutional layer and the identical residual error module of three structures, independent convolutional layer and
The nuclear volume of the convolutional layer convolution kernel of residual error module is 32, size is 5 × 5, step-length 1, zero filling width are 2, independent convolution
Use ReLU function as activation primitive in layer.
First coding module includes an independent convolutional layer and the identical residual error module of three structures, independent convolutional layer
And the quantity of the convolutional layer convolution kernel of residual error module is 64, size is 5 × 5, step-length 2, zero filling width are 2, independent volume
Use ReLU function as activation primitive in lamination.
Second coding module includes an independent convolutional layer and the identical residual error module of three structures, independent convolutional layer
And the quantity of the convolutional layer convolution kernel of residual error module is 128, size is 5 × 5, step-length 2, zero filling width are 2, independent volume
Use ReLU function as activation primitive in lamination.
Input of the memory cell state output as decoder module in the long memory module in short-term of convolution, the long short-term memory of convolution
The hidden state output of module is connected with the hidden state input of the long memory module in short-term of convolution in next scale sub-network；For
The last one scale, the long output of memory module hidden state in short-term of convolution are not connect with other modules.
First decoder module includes the identical residual error module of three structures and an independent convolutional layer, independent convolutional layer
And the quantity of the convolutional layer convolution kernel of residual error module is 128, size is 5 × 5, step-length 2, zero filling width are 2, residual error module
Use ReLU function as activation primitive in cascade independent convolutional layer afterwards.
Second decoder module includes the identical residual error module of three structures and an independent convolutional layer, independent convolutional layer
And the quantity of the convolutional layer convolution kernel of residual error module is 64, size is 5 × 5, step-length 2, zero filling width are 2, residual error module
Use ReLU function as activation primitive in cascade independent convolutional layer afterwards.
Output module include the identical residual error module of three structures and an independent convolutional layer, independent convolutional layer and
The quantity of the convolutional layer convolution kernel of residual error module is 32, size is 5 × 5, step-length 1, zero filling width are 2, residual error module rear class
Use ReLU function as activation primitive in the independent convolutional layer of connection.
The generator for exporting third level scale exports image L3, size is 64 × 64, L3It obtains through up-sampling having a size of 128
× 128 image, as the input of second level scale, the generator of the second level scale of output 128 × 128 exports image L2；L2
Input of the image as first order scale having a size of 256 × 256 is obtained through up-sampling, exports 256 × 256 first order scale
Generator export image L1, as result images of deblurring.In the sub-network of cascade three scales, three sub-networks pair
Structure, port number, the convolution kernel size answered are all the same.In three channels of color image RGB, each channel weight is shared.
S3, m (m=16) blurred picture and corresponding clear image is randomly selected from training dataset T, and cut out at random
It is cut into 256 × 256 square region, is separately constituted for trained fuzzy graph image set B and corresponding clear image collection S, at this time B
Image with S is 256 × 256 3 Channel Color images.Fuzzy graph image set B is inputted into generator, obtains generator output figure
Image set L.
S4, the input that generator output image set L and corresponding clear image collection S is successively used as to arbiter, arbiter
Two groups of confidence levels are sequentially output as a result, every group of confidence level includes 16 probability values, determine that the image of every input is clear with this
Image generates image: if probability value is greater than 0.5, being determined as clear image；Otherwise, then it is judged to generating image.
S5, the loss function for constructing training generator, loss function are as follows: ldb=lE+α1lgrad+α2ladv。α1、α2For canonical
Term coefficient, α1=10-2, α2=10-4。lEThe mean square error between image set L and corresponding clear image collection S is exported for generator
Difference:
Wherein, Li、SiThe generator output image and clear figure being illustrated respectively on the i-th scale
Picture, NiThe number of pixels in expression all channels on the i-th scale image, i=1,2,3；It is multiple dimensioned to be adopted by being dropped three times to image
Sample obtains the image of size reduction, and first order scale is the image of full size size, since the second level, the ruler of every first order image
The very little width for upper level picture size, each half of height.
lgradFor gradient imageWithBetween gradient error, it may be assumed that
In formula, Li(dx) and Li(dy) respectively indicate the horizontal gradient and vertical gradient of Li, Si(dx) and Si(dy) respectively indicate
SiHorizontal gradient and vertical gradient；ladvThe differentiation error of image set L and corresponding clear image collection S are exported for generator,
That is:
S~p (S) indicates that clear image s is taken from clear image collection S, p (S) and indicates the probability point of clear image collection S in formula
Cloth；B~p (B) indicates that blurred picture b is taken from fuzzy graph image set B, p (B) and indicates the probability distribution of fuzzy graph image set B；
D (s) indicates arbiter to the differentiation probability of input picture s, and G (b) expression is generated by input picture b through generator
Result images, E [] are indicated to taking expectation in bracket.
S6, generation image is input in arbiter together with clear image, gradient decline iteration is utilized to update each layer net
Weight parameter in network, continues to optimize ladv, until arbiter can not differentiate that the image of input is to generate image or clear figure
Picture, that is, the probability value obtained and 0.5 difference variation be less than given threshold 0.05, at this time arbiter training terminates.
S7, according to loss function ldb=lE+α1lgrad+α2ladvTraining generator, blurred picture is input in generator,
It is obtained by propagated forward and generates image, compare the otherness for generating image and clear image, updated using gradient decline iteration
Weight parameter in each layer network, continuous loss function ldb=lE+α1lgrad+α2ladv, until the Maker model training stage
Training set total losses functional value ldbVariation is less than given threshold 0.005, and generator training at this time terminates.
Step S3 to the S7 of S8, repetition training process, until the training set total losses function of Maker model training stage
Value ldbVariation is less than threshold value 0.005, i.e. arbiter can not determine that the image of input is clear image or generates image, assert life
It grows up to be a useful person model and arbiter model training has reached convergence, blurred picture is input in generator at this time, obtains going for estimation
Blurred picture.
Claims (6)
1. going motion blur method based on the multiple dimensioned Image Blind for generating confrontation network is recycled, it is characterised in that comprise the concrete steps that:
Step (1) constructs arbiter D:
The arbiter D is made of nine convolutional layers, a full articulamentum and a Sigmoid active coating, and input size is
256 × 256 color image；
Each convolutional layer is all made of LeakyReLU as activation primitive: first layer has 32 convolution kernels, each convolution kernel having a size of
5 × 5, step-length 2, zero filling width is 2；The second layer has 64 convolution kernels, and each convolution kernel is having a size of 5 × 5, step-length 1, zero filling
Width is 2；Third layer has 64 convolution kernels, and for each convolution kernel having a size of 5 × 5, step-length 2, zero filling width is 2；4th layer has
128 convolution kernels, for each convolution kernel having a size of 5 × 5, step-length 1, zero filling width is 2；Layer 5 has 128 convolution kernels, each
For convolution kernel having a size of 5 × 5, step-length 4, zero filling width is 2；Layer 6 has 256 convolution kernels, each convolution kernel having a size of 5 ×
5, step-length 1, zero filling width is 2；Layer 7 has 256 convolution kernels, and each convolution kernel is having a size of 5 × 5, and step-length 4, zero filling is wide
Degree is 2；8th layer has 512 convolution kernels, and for each convolution kernel having a size of 5 × 5, step-length 1, zero filling width is 2；9th layer has
512 convolution kernels, for each convolution kernel having a size of 4 × 4, step-length 4, zero filling width is 0；
It is 512 that the convolution of the last layer, which is exported through input channel number, the full articulamentum that output channel number is 1, obtains 1 constant,
The probability of judgement is exported after the activation of Sigmoid function；
Step (2) constructs generator G:
The generator G includes the sub-network of cascade three scales, and each sub-network includes 1 input module, 2 codings
Module, 1 convolution of cascade long memory module, 2 decoder modules and 1 output module in short-term；Contain residual error in each module
Module, the residual error module cascade a convolution kernel by a convolutional layer and form, and convolutional layer is with modified linear unit ReLU
As activation primitive；The output of the convolution kernel of residual error module cascade is residual error module after being added with the input of residual error module
Output；
The input module include an independent convolutional layer and the identical residual error module of three structures, independent convolutional layer with
And the nuclear volume of the convolutional layer convolution kernel of residual error module is 32, size is 5 × 5, step-length 1, zero filling width are 2, independent volume
Use modified linear unit ReLU as activation primitive in lamination；
First coding module include an independent convolutional layer and the identical residual error module of three structures, independent convolutional layer and
The quantity of the convolutional layer convolution kernel of residual error module is 64, size is 5 × 5, step-length 2, zero filling width are 2, independent convolutional layer
It is middle to use modified linear unit ReLU as activation primitive；
Second coding module include an independent convolutional layer and the identical residual error module of three structures, independent convolutional layer and
The quantity of the convolutional layer convolution kernel of residual error module is 128, size is 5 × 5, step-length 2, zero filling width are 2, independent convolutional layer
It is middle to use modified linear unit ReLU as activation primitive；
Input of the memory cell state output as decoder module in the long memory module in short-term of the convolution, convolution length are remembered in short-term
The hidden state output for recalling module is connected with the hidden state input of the long memory module in short-term of convolution in next scale sub-network；It is right
In the last one scale, the long output of memory module hidden state in short-term of convolution is not connect with other modules；
First decoder module include the identical residual error module of three structures and an independent convolutional layer, independent convolutional layer and
The quantity of the convolutional layer convolution kernel of residual error module is 128, size is 5 × 5, step-length 2, zero filling width are 2, residual error module rear class
Use modified linear unit ReLU as activation primitive in the independent convolutional layer of connection；
Second decoder module include the identical residual error module of three structures and an independent convolutional layer, independent convolutional layer and
The quantity of the convolutional layer convolution kernel of residual error module is 64, size is 5 × 5, step-length 2, zero filling width are 2, residual error module rear class
Use modified linear unit ReLU as activation primitive in the independent convolutional layer of connection；
The output module include the identical residual error module of three structures and an independent convolutional layer, independent convolutional layer with
And the quantity of the convolutional layer convolution kernel of residual error module is 32, size is 5 × 5, step-length 1, zero filling width are 2, after residual error module
Use modified linear unit ReLU as activation primitive in cascade independent convolutional layer；
The generator for exporting third level scale exports image L3, size is 64 × 64, L3It obtains through up-sampling having a size of 128 × 128
Image, as the input of second level scale, the generator of the second level scale of output 128 × 128 exports image L2；L2Through upper
Sampling obtains input of the image as first order scale having a size of 256 × 256, exports the life of 256 × 256 first order scale
It grows up to be a useful person and exports image L1, as result images of deblurring；
Step (3) randomly selects m blurred picture and corresponding clear images from training dataset T, and random cropping at
256 × 256 square region, separately constitute for trained fuzzy graph image set B and corresponding clear image collection S, obtained B and
The amount of images of S is m, and every image is 256 × 256 3 Channel Color images；Fuzzy graph image set B is inputted and is generated
Device obtains having the color image that m sizes are 256 × 256 in generator output image set L, L；
Generator output image set L and corresponding clear image collection S is successively used as the input of arbiter, arbiter by step (4)
Two groups of confidence levels are sequentially output as a result, every group of confidence level includes m probability value, determine that every image inputted is clearly to scheme with this
Picture generates image: if probability value is greater than 0.5, being determined as clear image；Probability value is less than or equal to 0.5, then is judged to generating
Image；
Step (5) constructs the loss function of training generator, loss function are as follows: ldb=lE+α1lgrad+α2ladv；
Wherein α1、α2For the regularization coefficient greater than 0, lEIt is exported between image set L and corresponding clear image collection S for generator
Mean square error, it may be assumed thatLi、SiThe generator output image that is illustrated respectively on the i-th scale and clear
Image, NiThe number of pixels in expression all channels on the i-th scale image, i=1,2,3；It is multiple dimensioned by being adopted to 3 drops of image
Sample obtains the image of size reduction；
lgradFor gradient imageWithBetween gradient error, it may be assumed that
Li(dx) and Li(dy) respectively indicate Li's
Horizontal gradient and vertical gradient, Si(dx) and Si(dy) respectively indicate SiHorizontal gradient and vertical gradient；
ladvThe differentiation error of image set L and corresponding clear image collection S are exported for generator, it may be assumed that
S~p (S) indicates that clear image s is taken from clear image collection
The probability distribution of S, p (S) expression clear image collection S；B~p (B) indicates that blurred picture b is taken from fuzzy graph image set B, p (B) table
Show the probability distribution of fuzzy graph image set B；D (s) indicates arbiter to the differentiation probability of input picture s, and G (b) indicates to be schemed by input
As the result images that b is generated through generator, E [] is indicated to taking expectation in bracket；
Step (6) is input to image is generated in arbiter together with clear image, and updates each layer net using gradient decline iteration
Weight parameter in network, continues to optimize ladv, until arbiter can not differentiate that the image of input is to generate image or clear figure
Picture, that is, the probability value obtained and 0.5 difference variation be less than thr, at this time arbiter training terminates；
Step (7) is according to loss function ldb=lE+α1lgrad+α2ladvTraining generator, is input to generator for blurred picture
In, it is obtained by propagated forward and generates image, compare the otherness for generating image and clear image, more using gradient decline iteration
Weight parameter in new each layer network, continues to optimize loss function ldb=lE+α1lgrad+α2ladv, until Maker model training
The training set total losses functional value l in stagedbVariation is less than threshold value Th, and generator training at this time terminates；
The step of step (8) repetition training process (3)~step (7), until the training lump of Maker model training stage is damaged
Lose functional value ldbVariation is less than threshold value Th, i.e. arbiter can not determine that the image of input is clear image or generates image, recognizes
Determine Maker model and arbiter model training has reached convergence, blurred picture is input in generator at this time, is estimated
De-blurred image.
2. motion blur method is gone based on the Image Blind for recycling multiple dimensioned generation confrontation network as described in claim 1, it is special
Sign is: in the sub-network of cascade three scales of step (2), the corresponding structure of three sub-networks, port number, convolution kernel size
It is all the same；In three channels of color image RGB, each channel weight is shared.
3. motion blur method is gone based on the Image Blind for recycling multiple dimensioned generation confrontation network as described in claim 1, it is special
Sign is: in step (3) and (4), m >=16.
4. motion blur method is gone based on the Image Blind for recycling multiple dimensioned generation confrontation network as described in claim 1, it is special
Sign is: multiple dimensioned by the way that image, repeatedly the down-sampled image for obtaining size reduction, first order scale are original in step (5)
The image of size, since the second level, the size of every first order image is the width of upper level picture size, height each one
Half.
5. motion blur method is gone based on the Image Blind for recycling multiple dimensioned generation confrontation network as described in claim 1, it is special
Sign is: in step (6), 0.01≤thr≤0.08.
6. motion blur method is gone based on the Image Blind for recycling multiple dimensioned generation confrontation network as described in claim 1, it is special
Sign is: in step (7) and (8), 0.001≤Th≤0.01.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910515590.4A CN110378844B (en) | 2019-06-14 | 2019-06-14 | Image blind motion blur removing method based on cyclic multi-scale generation countermeasure network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910515590.4A CN110378844B (en) | 2019-06-14 | 2019-06-14 | Image blind motion blur removing method based on cyclic multi-scale generation countermeasure network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110378844A true CN110378844A (en) | 2019-10-25 |
| CN110378844B CN110378844B (en) | 2021-04-09 |
Family
ID=68250306
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910515590.4A Expired - Fee Related CN110378844B (en) | 2019-06-14 | 2019-06-14 | Image blind motion blur removing method based on cyclic multi-scale generation countermeasure network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110378844B (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110910442A (en) * | 2019-11-29 | 2020-03-24 | 华南理工大学 | Machine vision size detection method for high-speed moving object based on kernel-free image restoration |
| CN111028177A (en) * | 2019-12-12 | 2020-04-17 | 武汉大学 | Edge-based deep learning image motion blur removing method |
| CN111199522A (en) * | 2019-12-24 | 2020-05-26 | 重庆邮电大学 | Single-image blind motion blur removing method for generating countermeasure network based on multi-scale residual errors |
| CN111223062A (en) * | 2020-01-08 | 2020-06-02 | 西安电子科技大学 | Image deblurring method based on generation countermeasure network |
| CN111292262A (en) * | 2020-01-19 | 2020-06-16 | 腾讯科技（深圳）有限公司 | Image processing method, image processing apparatus, electronic device, and storage medium |
| CN111340716A (en) * | 2019-11-20 | 2020-06-26 | 电子科技大学成都学院 | Image deblurring method for improving dual-discrimination countermeasure network model |
| CN111489304A (en) * | 2020-03-27 | 2020-08-04 | 天津大学 | Image deblurring method based on attention mechanism |
| CN111583143A (en) * | 2020-04-30 | 2020-08-25 | 广州大学 | Complex image deblurring method |
| CN111681188A (en) * | 2020-06-15 | 2020-09-18 | 青海民族大学 | Image deblurring method based on combination of image pixel prior and image gradient prior |
| CN111986275A (en) * | 2020-07-31 | 2020-11-24 | 西安理工大学 | Inverse halftoning method for multi-modal halftone image |
| CN112241939A (en) * | 2020-10-15 | 2021-01-19 | 天津大学 | Light-weight rain removing method based on multi-scale and non-local |
| CN112258425A (en) * | 2020-11-24 | 2021-01-22 | 中电万维信息技术有限责任公司 | Two-dimensional code image sharpening and deblurring processing method |
| CN112329932A (en) * | 2020-10-30 | 2021-02-05 | 深圳市优必选科技股份有限公司 | Training method and device for generating countermeasure network and terminal equipment |
| CN112435187A (en) * | 2020-11-23 | 2021-03-02 | 浙江工业大学 | Single-image blind motion blur removing method for generating countermeasure network based on aggregation residual |
| CN112508817A (en) * | 2020-12-16 | 2021-03-16 | 西北工业大学 | Image motion blind deblurring method based on loop generation countermeasure network |
| CN112541864A (en) * | 2020-09-25 | 2021-03-23 | 中国石油大学(华东) | Image restoration method based on multi-scale generation type confrontation network model |
| CN112614072A (en) * | 2020-12-29 | 2021-04-06 | 北京航空航天大学合肥创新研究院 | Image restoration method and device, image restoration equipment and storage medium |
| CN112634163A (en) * | 2020-12-29 | 2021-04-09 | 南京大学 | Method for removing image motion blur based on improved cycle generation countermeasure network |
| CN112686119A (en) * | 2020-12-25 | 2021-04-20 | 陕西师范大学 | License plate motion blurred image processing method based on self-attention generation countermeasure network |
| CN113012074A (en) * | 2021-04-21 | 2021-06-22 | 山东新一代信息产业技术研究院有限公司 | Intelligent image processing method suitable for low-illumination environment |
| CN113129237A (en) * | 2021-04-26 | 2021-07-16 | 广西师范大学 | Depth image deblurring method based on multi-scale fusion coding network |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108320274A (en) * | 2018-01-26 | 2018-07-24 | 东华大学 | It is a kind of to recycle the infrared video colorization method for generating confrontation network based on binary channels |
| CN109035149A (en) * | 2018-03-13 | 2018-12-18 | 杭州电子科技大学 | A kind of license plate image based on deep learning goes motion blur method |
-
2019
- 2019-06-14 CN CN201910515590.4A patent/CN110378844B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108320274A (en) * | 2018-01-26 | 2018-07-24 | 东华大学 | It is a kind of to recycle the infrared video colorization method for generating confrontation network based on binary channels |
| CN109035149A (en) * | 2018-03-13 | 2018-12-18 | 杭州电子科技大学 | A kind of license plate image based on deep learning goes motion blur method |
Non-Patent Citations (6)
| Title |
|---|
| XIN TAO等: "Scale-recurrent Network for Deep Image Deblurring", 《2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| YI CHEN等: "Motion Deblurring Via Using Generative Adversarial Networks For Space-Based Imaging", 《SERA 2018》 * |
| 毛勇: "模糊车牌图像的盲去模糊算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 毛勇等: "基于深度学习的车牌图像去运动模糊技术", 《杭州电子科技大学学报(自然科学版)》 * |
| 陈富成等: "基于生成对抗网络的图像盲去运动模糊算法", 《软件导刊》 * |
| 鲍宗袍: "运动图像盲去模糊技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111340716A (en) * | 2019-11-20 | 2020-06-26 | 电子科技大学成都学院 | Image deblurring method for improving dual-discrimination countermeasure network model |
| CN111340716B (en) * | 2019-11-20 | 2022-12-27 | 电子科技大学成都学院 | Image deblurring method for improving double-discrimination countermeasure network model |
| CN110910442A (en) * | 2019-11-29 | 2020-03-24 | 华南理工大学 | Machine vision size detection method for high-speed moving object based on kernel-free image restoration |
| CN110910442B (en) * | 2019-11-29 | 2023-09-29 | 华南理工大学 | High-speed moving object machine vision size detection method based on kernel-free image restoration |
| CN111028177A (en) * | 2019-12-12 | 2020-04-17 | 武汉大学 | Edge-based deep learning image motion blur removing method |
| CN111028177B (en) * | 2019-12-12 | 2023-07-21 | 武汉大学 | Edge-based deep learning image motion blur removing method |
| CN111199522B (en) * | 2019-12-24 | 2024-02-09 | 芽米科技(广州)有限公司 | Single-image blind removal motion blurring method for generating countermeasure network based on multi-scale residual error |
| CN111199522A (en) * | 2019-12-24 | 2020-05-26 | 重庆邮电大学 | Single-image blind motion blur removing method for generating countermeasure network based on multi-scale residual errors |
| CN111223062A (en) * | 2020-01-08 | 2020-06-02 | 西安电子科技大学 | Image deblurring method based on generation countermeasure network |
| CN111292262A (en) * | 2020-01-19 | 2020-06-16 | 腾讯科技（深圳）有限公司 | Image processing method, image processing apparatus, electronic device, and storage medium |
| CN111292262B (en) * | 2020-01-19 | 2023-10-13 | 腾讯科技（深圳）有限公司 | Image processing method, device, electronic equipment and storage medium |
| CN111489304A (en) * | 2020-03-27 | 2020-08-04 | 天津大学 | Image deblurring method based on attention mechanism |
| CN111489304B (en) * | 2020-03-27 | 2022-04-26 | 天津大学 | Image deblurring method based on attention mechanism |
| CN111583143A (en) * | 2020-04-30 | 2020-08-25 | 广州大学 | Complex image deblurring method |
| CN111681188B (en) * | 2020-06-15 | 2022-06-17 | 青海民族大学 | Image deblurring method based on combination of image pixel prior and image gradient prior |
| CN111681188A (en) * | 2020-06-15 | 2020-09-18 | 青海民族大学 | Image deblurring method based on combination of image pixel prior and image gradient prior |
| CN111986275A (en) * | 2020-07-31 | 2020-11-24 | 西安理工大学 | Inverse halftoning method for multi-modal halftone image |
| CN111986275B (en) * | 2020-07-31 | 2024-06-11 | 广州嘉尔日用制品有限公司 | Inverse halftoning method for multi-mode halftone image |
| CN112541864A (en) * | 2020-09-25 | 2021-03-23 | 中国石油大学(华东) | Image restoration method based on multi-scale generation type confrontation network model |
| CN112241939A (en) * | 2020-10-15 | 2021-01-19 | 天津大学 | Light-weight rain removing method based on multi-scale and non-local |
| CN112241939B (en) * | 2020-10-15 | 2023-05-30 | 天津大学 | Multi-scale and non-local-based light rain removal method |
| CN112329932A (en) * | 2020-10-30 | 2021-02-05 | 深圳市优必选科技股份有限公司 | Training method and device for generating countermeasure network and terminal equipment |
| CN112435187A (en) * | 2020-11-23 | 2021-03-02 | 浙江工业大学 | Single-image blind motion blur removing method for generating countermeasure network based on aggregation residual |
| CN112258425A (en) * | 2020-11-24 | 2021-01-22 | 中电万维信息技术有限责任公司 | Two-dimensional code image sharpening and deblurring processing method |
| CN112508817B (en) * | 2020-12-16 | 2024-05-14 | 西北工业大学 | Image motion blind deblurring method based on cyclic generation countermeasure network |
| CN112508817A (en) * | 2020-12-16 | 2021-03-16 | 西北工业大学 | Image motion blind deblurring method based on loop generation countermeasure network |
| CN112686119B (en) * | 2020-12-25 | 2022-12-09 | 陕西师范大学 | License plate motion blurred image processing method based on self-attention generation countermeasure network |
| CN112686119A (en) * | 2020-12-25 | 2021-04-20 | 陕西师范大学 | License plate motion blurred image processing method based on self-attention generation countermeasure network |
| CN112614072B (en) * | 2020-12-29 | 2022-05-17 | 北京航空航天大学合肥创新研究院 | Image restoration method and device, image restoration equipment and storage medium |
| CN112634163A (en) * | 2020-12-29 | 2021-04-09 | 南京大学 | Method for removing image motion blur based on improved cycle generation countermeasure network |
| CN112614072A (en) * | 2020-12-29 | 2021-04-06 | 北京航空航天大学合肥创新研究院 | Image restoration method and device, image restoration equipment and storage medium |
| CN113012074B (en) * | 2021-04-21 | 2023-03-24 | 山东新一代信息产业技术研究院有限公司 | Intelligent image processing method suitable for low-illumination environment |
| CN113012074A (en) * | 2021-04-21 | 2021-06-22 | 山东新一代信息产业技术研究院有限公司 | Intelligent image processing method suitable for low-illumination environment |
| CN113129237A (en) * | 2021-04-26 | 2021-07-16 | 广西师范大学 | Depth image deblurring method based on multi-scale fusion coding network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110378844B (en) | 2021-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110378844A (en) | Motion blur method is gone based on the multiple dimensioned Image Blind for generating confrontation network is recycled | |
| CN113469356B (en) | Improved VGG16 network pig identity recognition method based on transfer learning | |
| CN109671023B (en) | Face image super-resolution secondary reconstruction method | |
| CN106204468B (en) | A kind of image de-noising method based on ReLU convolutional neural networks | |
| CN106780543B (en) | A kind of double frame estimating depths and movement technique based on convolutional neural networks | |
| CN108647639A (en) | Real-time body's skeletal joint point detecting method | |
| CN109035149A (en) | A kind of license plate image based on deep learning goes motion blur method | |
| CN107767413A (en) | A kind of image depth estimation method based on convolutional neural networks | |
| CN112541877B (en) | Defuzzification method, system, equipment and medium for generating countermeasure network based on condition | |
| CN106981080A (en) | Night unmanned vehicle scene depth method of estimation based on infrared image and radar data | |
| CN110889343A (en) | Crowd density estimation method and device based on attention type deep neural network | |
| CN112580473B (en) | Video super-resolution reconstruction method integrating motion characteristics | |
| CN109003234B (en) | For the fuzzy core calculation method of motion blur image restoration | |
| CN111242846A (en) | Fine-grained scale image super-resolution method based on non-local enhancement network | |
| CN113379771B (en) | Hierarchical human body analysis semantic segmentation method with edge constraint | |
| CN110705340B (en) | Crowd counting method based on attention neural network field | |
| CN112634163A (en) | Method for removing image motion blur based on improved cycle generation countermeasure network | |
| CN107977930A (en) | A kind of image super-resolution method and its system | |
| CN107845065A (en) | Super-resolution image reconstruction method and device | |
| CN113554653A (en) | Semantic segmentation method for long-tail distribution of point cloud data based on mutual information calibration | |
| CN115484410A (en) | Event camera video reconstruction method based on deep learning | |
| CN116385281A (en) | Remote sensing image denoising method based on real noise model and generated countermeasure network | |
| CN110675311A (en) | Sketch generation method and device under sketch order constraint and storage medium | |
| CN118229569A (en) | Document image transmission removing method and device based on fuzzy diffusion model | |
| CN111260706B (en) | Dense depth map calculation method based on monocular camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210409 |