CN109035149A - A kind of license plate image based on deep learning goes motion blur method - Google Patents
A kind of license plate image based on deep learning goes motion blur method Download PDFInfo
- Publication number
- CN109035149A CN109035149A CN201810205661.6A CN201810205661A CN109035149A CN 109035149 A CN109035149 A CN 109035149A CN 201810205661 A CN201810205661 A CN 201810205661A CN 109035149 A CN109035149 A CN 109035149A
- Authority
- CN
- China
- Prior art keywords
- image
- license plate
- size
- minibatch
- convolution kernel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000033001 locomotion Effects 0.000 title claims abstract description 30
- 238000000034 method Methods 0.000 title claims abstract description 22
- 238000013135 deep learning Methods 0.000 title claims abstract description 11
- 238000012549 training Methods 0.000 claims abstract description 44
- 238000012360 testing method Methods 0.000 claims abstract description 12
- 230000011218 segmentation Effects 0.000 claims abstract description 4
- 239000011248 coating agent Substances 0.000 claims description 11
- 238000000576 coating method Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 10
- 238000003475 lamination Methods 0.000 claims description 9
- 230000004069 differentiation Effects 0.000 claims description 7
- 238000009826 distribution Methods 0.000 claims description 4
- 241001269238 Data Species 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 3
- 238000013480 data collection Methods 0.000 claims description 3
- 238000005549 size reduction Methods 0.000 claims description 2
- 238000011084 recovery Methods 0.000 abstract description 6
- 238000004904 shortening Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 8
- 230000000694 effects Effects 0.000 description 4
- 238000005457 optimization Methods 0.000 description 4
- 238000011160 research Methods 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 239000008199 coating composition Substances 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 230000001627 detrimental effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The present invention proposes that a kind of license plate image based on deep learning goes motion blur method.The present invention is divided into data set pretreatment stage, training stage and test phase.In data set pretreatment stage, determines the license plate area in image, segmentation characters on license plate and normalized images size, add Gaussian noise, obtain training set.In the training stage, motion blur model is removed using confrontation e-learning image is generated, using the mean square error of network restoration result, gradient error and the linear of error three is differentiated and alternately trains arbiter and generator as network losses.In test phase, segmentation characters on license plate and successively as the input of generator, deblurring result is combined to obtain deblurring license plate image according to characters on license plate original order.Model proposed by the invention effectively constrains the edge of license plate image, so that improving license plate image goes the quality of motion blur, while shortening the time of recovery.
Description
Technical field
The invention belongs to technical field of image processing, be related to it is a kind of to the license plate image deblurring method for having motion blur,
Specifically a kind of license plate image based on deep learning goes motion blur method.
Background technique
Since speed is too fast, captures the influence of the factors such as device hardware limitation and light environment, captures image and exist centainly
Motion blur, affect the accurate acquisition of license plate number, bring detrimental effect to the management of urban transportation.License plate goes to transport
Dynamic model paste refers to the efficient operational performance using computer, is answered by intelligent algorithm the license plate image there are motion blur
Original obtains clearly license plate image.An important link of the Car license recognition as traffic administration, license plate go motion blur advantageous
In the identification for improving license plate.
Traditional license plate goes motion blur technology to be broadly divided into two processes: estimation motion blur core and non-blind deblurring.
Estimation motion blur core refers to that estimation obtains the kernel function of motion blur from blurred picture, this process is entirely removing movement mould
Extremely important during paste, can the quality of fuzzy core directly determines effectively restore clear image.Non-blind deblurring refers to root
Deblurring is carried out to blurred picture according to known fuzzy core, this method passes through nearly research in 50 years, there are more technical literature quilts
It proposes.
In recent years, domestic and foreign scholars carried out in-depth study and discussion for the estimation of motion blur core, and Pan et al. is proposed
l0Estimation image itself and its gradient among regularization constraint, but when image is there are when less zero value pixels, image itself
Sparse constraint does not work, and causes the recovering quality in license plate deblurring not high；Fang et al. proposes l1Regular terms constrains image
Sparsity, improve the quality of license plate deblurring, but there are partial pixel distortions when restored image；Song et al. is then proposed
Using the l of analysis dictionary1Sparse regularization constraint image, and Recognition Algorithm of License Plate optimization deblurring effect is introduced, but license plate
The cyclic process of deblurring and Car license recognition takes long time entire license plate recuperation.
In recent years, the extensive use in the fast development and computer vision field of deep learning, image remove movement mould
Paste problem is widely studied, and Svobode et al. proposes the model restored using convolutional neural networks model training license plate image,
This method is limited to motor racing direction and the length of a small range, and the license plate being unsuitable under complex situations removes motion blur；Nah
Et al. then propose using generate confrontation network processes dynamic scene under motion blur, but restore picture edge characteristic not
Obviously, however it remains motion blur.
The method of deep learning eliminates the process of ambiguous estimation core, and recovery speed is relatively fast, has a clear superiority,
Therefore deep learning is applied in license plate image deblurring research by the present invention, to obtain better recovery effect.
Summary of the invention
The object of the present invention is to provide a kind of methods that effective license plate image removes motion blur, preferably to extract image
Feature so that improving image goes motion blur quality, while shortening recovery time.
Step of the present invention is divided into data set pretreatment, training stage and test phase, detail and is described as follows.
Step 1, data set pretreatment:
1-1, data set include clear image blurred picture pair, for demarcating the position of license plate in clearly vehicle image；
Complete license plate area is plucked out from clearly vehicle image and corresponding fuzzy vehicle image, obtains the clear figure of license plate area
Picture and blurred picture pair；
1-2, license plate is divided to obtain the single character data collection of 34 classes (0 and I is incorporated into as 0 and 1) according to character width, and unites
One returns generalized to 16*48*3 size, wherein 16,48 and 3 respectively indicate width, height and the Color Channel number of image；
1-3, all fuzzy license plate character pictures are added into mean value as μ, standard deviation is the Gaussian noise of σ, by clear vehicle
The pixel value of board character picture and fuzzy license plate character picture both maps in [0,1]；
1-4, N is randomly selected from every a kind of data that 34 class character datas are concentrated1To as training dataset T, N2To work
For validation data set V；
Step 2, the operation of training stage:
2-1, minibatch blurred pictures and corresponding clear images are randomly selected from training dataset T, and random
16*16 square region is cut as training fuzzy graph image set B and clear image collection S, training fuzzy graph image set B at this time and clear
Image set S is minibatch*3*16*16, and minibatch indicates the quantity of image, and 3 indicate the Color Channel number of image,
16*16 is the size of image；Fuzzy graph image set B is inputted into generator, obtains the minibatch*3*16*16's of generator output
Image set L；
2-2, the input that the output image set L of generator and corresponding clear image collection S are successively used as to arbiter differentiate
Device is sequentially output two groups of confidence levels as a result, every group of confidence level includes minibatch probability value, determines every figure inputted with this
Seem the image that clear image still generates, if probability value is greater than 0.5, is judged to clear image；Otherwise, it is judged to generate image；
2-3, the mean square error l generated between image set L and clear image collection S is calculatedmse, it may be assumed that
Wherein, ck、wk、hkRespectively indicate Color Channel number, width and the height of each scale image in multiple dimensioned generator
It spends, here all ck3, K is taken to indicate scale series, LkIt is k-th of scale image that multiple dimensioned generator generates, SkIt is corresponding
K-th of scale clear image；It is multiple dimensioned by image repeatedly the down-sampled image for obtaining size reduction, with piece image difference
The quantity of size indicates the series of scale, and wherein first order scale is the image of full size size, since the second level, every level-one
The size of image is the width of upper level picture size, each half of height, as upper level image size a quarter；
2-4, the gradient image ▽ for generating image set L and clear image collection S is calculatedLAnd ▽S, and calculate between gradient image
Gradient error lgrad, it may be assumed that
Wherein, gradient image passes through respectively calculates horizontal direction gradient value dxWith the gradient value d of vertical directionyAbsolute value it
With obtain, it may be assumed that
2-5, the differentiation error l for generating image set L and clear image collection S is calculatedadv, it may be assumed that
In formula, s~p (S) indicates that clear image s is taken from clear image collection S, and wherein p (S) indicates clear image collection S's
Probability distribution, b~p (B) indicate that blurred picture b is taken from fuzzy graph image set B, p (B) and indicates the probability distribution of fuzzy graph image set B,
G (im) indicates that the result images that input picture im is generated through generator, D (im) indicate that arbiter is general to the differentiation of input picture im
Rate, E (x) expression take expectation to x；
2-6, total loss function l is calculateddbSize, it may be assumed that
ldb=lmse+α1lgrad+α2ladv (5)
Wherein α1、α2For the regularization coefficient greater than 0；
2-7, the formula according to step 2-5 (4) differentiate error ladvOptimize arbiter；
2-8, the formula according to step 2-6 (5) total losses function ldbOptimize generator；
2-9, minibatch blurred pictures and corresponding clear image group again are randomly selected from training dataset T
At fuzzy graph image set B and clear image collection S, pass through step 2-1 to step 2-8 repetition training；
2-10, when trained total amount of data reaches the amount of images of training set T, will verifying collection V blurred picture successively
The generator and arbiter currently trained are inputted, the average total losses l of verifying collection is calculateddb, optimize arbiter and generator；
2-11, repetition training is carried out by step 2-1 to step 2-10, until the training lump of the model training stage is damaged
Lose functional value ldbVariation is less than threshold value Th, then assert that the model training has reached convergence；
The operation of step 3, test phase:
3-1, license plate area is plucked out from the test license plate image of motion blur；
3-2, license plate area is divided into single characters on license plate according to character pitch and return generalized to for 16*48*3 it is big
It is small, wherein 16,48 and 3 respectively indicate width, height and the Color Channel number of image；
3-3, the trained model of load, the test license plate character picture that input segmentation obtains, the vehicle after obtaining deblurring
Board character picture；
3-4, the characters on license plate after deblurring according to former order is combined to obtain whole picture license plate image after deblurring；
Arbiter described in step 2-5, specific as follows:
Arbiter is made of seven convolutional layers, a full articulamentum and a Sigmoid active coating；Firstly,
The image set of minibatch*3*16*16 inputs the arbiter, and first layer is the convolutional layer of 32*3*3, there is the convolution in 32 channels
Core, each convolution kernel are 1 having a size of 3*3, step-length 2, the width of edge zero padding, output size minibatch*32*8*8；
The second layer is the convolutional layer of 64*3*3, has the convolution kernel in 64 channels, each convolution kernel is having a size of 3*3, step-length 1, edge zero padding
Width be 1, output size minibatch*64*8*8；Third layer is the convolutional layer of 64*3*3, there is the convolution kernel in 64 channels,
Each convolution kernel is 1 having a size of 3*3, step-length 2, the width of edge zero padding, output size minibatch*64*4*4；4th layer
For the convolutional layer of 128*3*3, there is the convolution kernel in 128 channels, each convolution kernel is having a size of 3*3, step-length 1, the width of edge zero padding
Degree is 1, output size minibatch*128*4*4；Layer 5 is the convolutional layer of 128*3*3, there is the convolution kernel in 128 channels,
Each convolution kernel is 1 having a size of 3*3, step-length 2, the width of edge zero padding, output size minibatch*128*2*2；The
Six layers of convolutional layer for 256*3*3 has the convolution kernel in 256 channels, each convolution kernel is having a size of 3*3, step-length 1, edge zero padding
Width be 1, output size minibatch*256*2*2；Layer 7 is the convolutional layer of 256*2*2, there is the convolution in 256 channels
Core, each convolution kernel is having a size of 2*2, edge not zero padding, output size minibatch*256*1*1；The last layer convolutional layer
Output through input channel number be 256, the full articulamentum that output channel number is 1 obtains minibatch constant, through Sigmoid
The probability of minibatch judgement of output after function activation then determines that corresponding input picture is clear if it is determined that probability is greater than 0.5
Clear image；Otherwise, it is determined that generate image；
Generator described in step 2-6, specific as follows:
Generator is made of the network of three scales, and thick scale network is connected by warp lamination and upper level scale network
Connect, wherein warp lamination port number be 3, convolution kernel 5*5, step-length 1, zero padding width be 2；In the network of single scale
In, the size that image is output and input remains unchanged, and the network under the scale is the convolution in 64 channels, 5*5 size first
The convolutional layer that core, step-length 1, zero padding width are 2, if the scale is k, which is minibatch*64*wk*
Hk, followed by a ReLU active coating, followed by 19 duplicate residual error subnets, wherein each residual error subnet includes two points
Branch, a branch is made of the ReLU active coating after two convolutional layers and first convolutional layer, wherein convolution layer parameter be all provided with for
64 channels, the convolution kernel of 5*5 size, step-length 1, zero padding width are 2, and another branch is then without any structural unit, two
Branch completes channel before the output of residual error subnet and cascades, and exports the image set in 128 channels；Individually last in scale network
A convolutional layer and a ReLU active coating are connected to after a residual error subnet, wherein convolutional layer includes the convolution kernel in 3 channels, each
Convolution kernel size is 5*5, step-length 1, zero padding width are 2, if the scale is k, output size minibatch*3*wk*hk.
The present invention eliminates complicated fuzzy kernel estimates mistake with the deblurring process of deep learning method study image
Journey.By the comparative training of a large amount of blurred pictures and clear image, institute's climbing form type can extract the edge feature of image, to go
Except the motion blur in license plate image, the average every figure recovery of the present invention is only needed 2.5 seconds, and recovery effect is preferable.
Detailed description of the invention
Fig. 1 is training process flow chart of the invention；
Fig. 2 is arbiter network model of the invention；
Fig. 3 is generator network model of the invention；
Residual error sub-network structures in the generator of the present invention of the position Fig. 4；
Specific embodiment
Specific implementation of the invention is described further below in conjunction with attached drawing.
Fig. 1 is the flow chart for generating the confrontation network training stage.Fuzzy graph image set B is input in generator G, is generated
Image set L generates input of the image set L as arbiter D, obtains the differentiation of arbiter as a result, similarly, clear image collection S
As the input of arbiter, obtain differentiating result.The judgement result indicates to determine that input is from clear image collection or generation
Image set, if it is determined that result > 0.5, then be determined as clear image collection S；Otherwise, it is determined that generate image set L.Calculate the judgement knot
The error of fruit and true tag data optimizes arbiter using gradient descent algorithm, then calculates and generate image and clear image
Error mean, utilize gradient descent algorithm optimize generator.Alternative optimization arbiter and generator, until model is restrained.?
In experiment of the invention, after totally 600000 times, model is restrained for training.
License plate image deblurring operation of the invention the following steps are included:
One, data preprocessing phase:
S1, nominal data concentrate the license plate image region of clear vehicle image, from data set clear vehicle image and
License plate area is plucked out in the vehicle image of corresponding motion blur, obtains clear image and the blurred picture pair of license plate.
S2, license plate image is divided to obtain the single character data collection of 34 classes (O and I are incorporated into as 0 and 1) according to character width,
And uniformly return generalized to 16*48*3 size, wherein 16,48 and 3 respectively indicate width, height and the Color Channel number of image.
S3, mean value is added as μ=0 to the obtained fuzzy license plate character picture of step S2, standard deviation is the Gauss of σ=0.01
Pixel value is simultaneously mapped in [0,1] by noise.
S4, the every a kind of data concentrated from the 34 class character datas of step S3 randomly select N1=500 to as training number
According to collection T, N2=100 is to as validation data set V.
Two, the training stage:
S1, minibatch=128 blurred pictures and corresponding clear images are randomly selected from training dataset T, and
The image of random cropping 16*16 square region is as training input sample collection B and clear sample set S, image set B and S at this time
It is minibatch*3*16*16.Blurred picture sample set B is inputted into generator, obtains the minibatch* of generator output
The image set L of 3*16*16 size.
S2, the output image set L of the obtained generator of step S1 and corresponding clear image collection S are successively regard as arbiter
Input, arbiter is sequentially output two groups of confidence levels as a result, every group of confidence level includes minibatch probability value, is determined with this
The image of every input is the image of clear image or generation, if probability value is judged to clear image greater than 0.5；Otherwise, sentence
To generate image.
S3, the mean square error lmse generated between image set L and clear image collection S that step S1 is calculated according to formula (1),
Middle K=3, ck=3,K=1,2,3.
S4, the gradient map image set ▽ for generating image set L and clear image collection S that step S1 is calculated according to formula (3)LAnd ▽S,
And the gradient error lgrad between gradient image is calculated according to formula (2).
S5, the differentiation error ladv for generating image set L and clear image collection S that step S1 is calculated according to formula (4).
S6, according to formula (5) the loss function size that the linear read group total of the result of step S3, S4, S5 is total, wherein canonical
Term coefficient α1=10-2, α2=10-4。
S7, arbiter (as shown in Figure 2) is optimized according to the differentiation error ladv of step S5.Arbiter by seven convolutional layers,
One full articulamentum and a Sigmoid active coating composition.The image set of minibatch*3*16*16 inputs the arbiter, the
One layer of convolutional layer for the input of 3 channels, the output of 32 channels, each convolution kernel is having a size of 3*3, step-length 2, the width of edge zero padding
It is 1, output size minibatch*32*8*8；The second layer is the convolutional layer of the input of 32 channels, the output of 64 channels, each convolution
Core is 1 having a size of 3*3, step-length 1, the width of edge zero padding, output size minibatch*64*8*8；Third layer is 64 logical
The convolutional layer of road input, the output of 64 channels, each convolution kernel are 1 having a size of 3*3, step-length 2, the width of edge zero padding, output
Size minibatch*64*4*4；4th layer for 64 channels input, 128 channels output convolutional layer, each convolution kernel having a size of
3*3, step-length 1, the width of edge zero padding are 1, output size 128*128*4*4；Layer 5 is that 128 channels input, 128 lead to
The convolutional layer of road output, each convolution kernel is having a size of 3*3, and the width of step-length 2, edge zero padding is 1, and output size is
minibatch*128*2*2；Layer 6 is the convolutional layer of the input of 128 channels, the output of 256 channels, and each convolution kernel is having a size of 3*
3, step-length 1, the width of edge zero padding is 1, output size minibatch*256*2*2；Layer 7 be 256 channels input,
The convolutional layer of 256 channels output, each convolution kernel is having a size of 2*2, edge not zero padding, output size minibatch*256*1*
1.The output of the last layer convolutional layer obtains minibatch through the full articulamentum (fc module) of input channel 256, output channel 1
A constant, the probability of minibatch judgement of output then determines if it is determined that probability is greater than 0.5 after the activation of Sigmoid function
Corresponding input picture is clear image；Otherwise, it is determined that generate image.
S8, total losses function ldb optimization generator (as shown in Figure 3) being calculated according to step S6.Generator is by three
The network of a scale grade is constituted, the output (Output) of thick scale network by the output of warp lamination (Deconv module) with it is upper
The input of level-one scale network is completed channel under the effect of Concate module and is cascaded, and wherein warp lamination includes the convolution in 3 channels
Core, the size of each convolution kernel is 5*5, step-length 1, zero padding width are 2.In the network of single scale, image input and it is defeated
Size out remains unchanged, and the network under the scale is a convolutional layer, the convolution kernel including 64 channels, each convolution kernel first
Size is 5*5, step-length 1, zero padding width are 2, if the scale is k, which is minibatch*64*wk*
Then hk connects a ReLU active coating, next connect 19 duplicate residual error subnets (Residual Block module), wherein
Each residual error subnet (as shown in Figure 4) includes Liang Ge branch.One branch is made of convolutional layer, ReLU active coating and convolutional layer,
Wherein each convolutional layer includes the convolution kernel in 64 channels, and each convolution kernel size is 5*5, and step-length 1, zero padding width is 2；Separately
Then without any structural unit, Liang Tiao branch is then defeated through Concate module completion channel cascade in residual error subnet for one branch
Out.It is connected to a convolutional layer and a ReLU active coating after the last one residual error subnet in single scale network, wherein rolling up
Lamination includes the convolution kernel in 3 channels, and each convolution kernel size is 5*5, step-length 1, zero padding width are 2, defeated if the scale is k
Size is minibatch*3*wk*hk out.In conclusion generator is by third level scale from the point of view of the input and output of generator
The fuzzy graph image set B3 of network inputs minibatch*3*4*4 exports the generation image set L3 of third level scale, and L3 is through deconvolution
The image set and the blurred picture B2 under the scale of the second level of layer output minibatch*3*8*8 cascades to obtain minibatch*6*8*
Input of 8 image set as second level scale network exports the generation image set L2 of minibatch*3*8*8, and L2 is through warp
The image set and the blurred picture B1 under first order scale of lamination output minibatch*3*16*16 cascades to obtain minibatch*
Input of the image set of 6*16*16 as first order scale network exports the generation image set L1 of minibatch*3*16*16,
The as result figure image set of deblurring.
S9, minibatch blurred pictures and corresponding clear image reformulation are randomly selected from training dataset T
Fuzzy graph image set B and clear image collection S, repetition training step S1 to S8.
S10, when trained total amount of data reaches the quantity of training set T, the blurred picture of verifying collection V is sequentially input and works as
Preceding trained generator and arbiter calculate the average total losses ldb of verifying collection V according to step S3, S4, S5, S6, according to step
Rapid S7 and S8 optimization arbiter and generator.
Step S1 to the S10 of S11, repetition training process, until the training set total losses ldb of the model training stage changes
Less than threshold value Th=5 × 10-3, then assert that the model training has reached convergence.
Three, test phase:
S1, license plate area is plucked out from the test license plate image of motion blur.
S2, the license plate area image that step S1 is obtained is divided into single characters on license plate image according to character pitch and is returned
Generalized is to 16*48*3 size.
S3, the trained model of load, the characters on license plate image that input step S2 is divided, the vehicle after obtaining deblurring
Board character picture.
S4, the characters on license plate after deblurring that step S3 is obtained according to former order is combined to obtain whole picture vehicle after deblurring
Board image.
Claims (3)
1. a kind of license plate image based on deep learning goes motion blur method, it is characterised in that including following operation:
Step 1, data set pretreatment:
1-1, data set include clear image blurred picture pair, for demarcating the position of license plate in clearly vehicle image；From clear
Pluck out complete license plate area in clear vehicle image and corresponding fuzzy vehicle image, obtain license plate area clear image and
Blurred picture pair；
1-2, license plate is divided to obtain the single character data collection of 34 classes (O and I are incorporated into as 0 and 1) according to character width, and uniformly returns
Generalized is to 16*48*3 size, wherein 16,48 and 3 respectively indicate width, height and the port number of image；
1-3, all fuzzy license plate character pictures are added into mean value as μ, standard deviation is the Gaussian noise of σ, by clear license plate word
The pixel value of symbol image and fuzzy license plate character picture both maps in [0,1]；
1-4, N is randomly selected from every a kind of data that 34 class character datas are concentrated1To as training dataset T, N2To as testing
Demonstrate,prove data set V；
Step 2, the operation of training stage:
2-1, minibatch blurred pictures and corresponding clear images, and random cropping are randomly selected from training dataset T
16*16 square region is as training fuzzy graph image set B and clear image collection S, training fuzzy graph image set B and clear image at this time
Collection S is minibatch*3*16*16, and minibatch indicates the quantity of image, and 3 indicate the Color Channel number of image, 16*16
For the size of image；Fuzzy graph image set B is inputted into generator, obtains the image of the minibatch*3*16*16 of generator output
Collect L；
2-2, the input that the output image set L of generator and corresponding clear image collection S are successively used as to arbiter, arbiter according to
Two groups of confidence levels of secondary output determine that every image inputted is with this as a result, every group of confidence level includes minibatch probability value
The image that clear image still generates is judged to clear image if probability value is greater than 0.5；Otherwise, it is judged to generate image；
2-3, the mean square error l generated between image set L and clear image collection S is calculatedmse, it may be assumed that
Wherein, ck、wk、hkRespectively indicate Color Channel number, width and the height of each scale image in multiple dimensioned generator, K
Indicate scale series, LkIt is k-th of scale image that multiple dimensioned generator generates, SkIt is corresponding k-th of scale clear image；
It is multiple dimensioned by indicating scale with the various sizes of quantity of piece image to the multiple down-sampled image for obtaining size reduction of image
Series, wherein first order scale is the image of full size size, and since the second level, the size of every first order image is upper level
The width of picture size, each half of height, as upper level image size a quarter；
2-4, the gradient image ▽ for generating image set L and clear image collection S is calculatedLAnd ▽S, and calculate the ladder between gradient image
Spend error lgrad, it may be assumed that
Wherein, gradient image passes through respectively calculates horizontal direction gradient value dxWith the gradient value d of vertical directionyThe sum of absolute value
It arrives, it may be assumed that
▽=| dx|+|dy|(3)
2-5, the differentiation error l for generating image set L and clear image collection S is calculatedadv, it may be assumed that
In formula, s~p (S) indicates that clear image s is taken from clear image collection S, and wherein p (S) indicates the probability of clear image collection S
Distribution, b~p (B) indicate that blurred picture b is taken from fuzzy graph image set B, p (B) and indicates the probability distribution of fuzzy graph image set B, G
(im) indicate that the result images that input picture im is generated through generator, D (im) indicate that arbiter is general to the differentiation of input picture im
Rate, E (x) expression take expectation to x；
2-6, total loss function l is calculateddbSize, it may be assumed that
ldb=lmse+α1lgrad+α2ladv (5)
Wherein α1、α2For the regularization coefficient greater than 0；
2-7, the formula according to step 2-5 (4) differentiate error ladvOptimize arbiter；
2-8, the formula according to step 2-6 (5) total losses function ldbOptimize generator；
2-9, minibatch blurred pictures and corresponding clear image reformulation mould are randomly selected from training dataset T
Image set B and clear image collection S is pasted, step 2-1 to step 2-8 repetition training is passed through；
2-10, when trained total amount of data reaches the amount of images of training set T, will verifying collection V blurred picture sequentially input
The generator and arbiter currently trained calculate the average total losses l of verifying collectiondb, optimize arbiter and generator；
2-11, repetition training is carried out by step 2-1 to step 2-10, until the training set total losses letter of the model training stage
Numerical value ldbVariation is less than threshold value Th, then assert that the model training has reached convergence；
The operation of step 3, test phase:
3-1, license plate area is plucked out from the test license plate image of motion blur；
3-2, license plate area is divided into single characters on license plate according to character pitch and returns generalized to for 16*48*3 size,
In 16,48 and 3 width, height and the Color Channel numbers for respectively indicating image；
3-3, the trained model of load, the test license plate character picture that input segmentation obtains, the license plate word after obtaining deblurring
Accord with image；
3-4, the characters on license plate after deblurring according to former order is combined to obtain whole picture license plate image after deblurring；
2. a kind of license plate image based on deep learning according to claim 1 goes motion blur method, it is characterised in that
Arbiter described in step 2-5, specific as follows:
Arbiter is made of seven convolutional layers, a full articulamentum and a Sigmoid active coating；Firstly, minibatch*3*
The image set of 16*16 inputs the arbiter, and first layer is the convolutional layer of 32*3*3, there is the convolution kernel in 32 channels, each convolution kernel
Having a size of 3*3, step-length 2, the width of edge zero padding is 1, output size minibatch*32*8*8；The second layer is 64*3*3
Convolutional layer, have a convolution kernel in 64 channels, each convolution kernel is 1 having a size of 3*3, step-length 1, the width of edge zero padding, output
Size is minibatch*64*8*8；Third layer is the convolutional layer of 64*3*3, there is the convolution kernel in 64 channels, each convolution kernel size
For 3*3, step-length 2, the width of edge zero padding is 1, output size minibatch*64*4*4；The 4th layer of volume for 128*3*3
Lamination has the convolution kernel in 128 channels, and each convolution kernel is 1 having a size of 3*3, step-length 1, the width of edge zero padding, output size
For minibatch*128*4*4；Layer 5 is the convolutional layer of 128*3*3, there is the convolution kernel in 128 channels, each convolution kernel size
For 3*3, step-length 2, the width of edge zero padding is 1, output size minibatch*128*2*2；Layer 6 is 256*3*3's
Convolutional layer has the convolution kernel in 256 channels, and each convolution kernel is 1 having a size of 3*3, step-length 1, the width of edge zero padding, and output is big
Small is minibatch*256*2*2；Layer 7 is the convolutional layer of 256*2*2, there is the convolution kernel in 256 channels, each convolution kernel ruler
Very little is 2*2, edge not zero padding, output size minibatch*256*1*1；The output of the last layer convolutional layer is through input channel
The full articulamentum that number is 256, output channel number is 1 obtains minibatch constant, exports after the activation of Sigmoid function
The probability of minibatch judgement then determines that corresponding input picture is clear image if it is determined that probability is greater than 0.5；Otherwise, sentence
It is set to generation image；
3. a kind of license plate image based on deep learning according to claim 1 goes motion blur method, it is characterised in that
Arbiter described in step 2-6, specific as follows:
Generator is made of the network of three scales, and thick scale network is connected to the network by warp lamination and upper level scale,
Middle warp lamination port number be 3, convolution kernel 5*5, step-length 1, zero padding width be 2；In the network of single scale, image
The size output and input remains unchanged, and the network under the scale is 64 channels, the convolution kernel of 5*5 size, step-length first
The convolutional layer for being 2 for 1, zero padding width, if the scale is k, which is minibatch*64*wk*hk, then
It is a ReLU active coating, followed by 19 duplicate residual error subnets, wherein each residual error subnet includes Liang Tiao branch, one
Branch is made of the ReLU active coating after two convolutional layers and first convolutional layer, wherein convolution layer parameter be all provided with for 64 channels,
Convolution kernel, step-length 1, the zero padding width of 5*5 size are 2, and another branch is then without any structural unit, and Liang Tiao branch is residual
Channel cascade is completed before poor subnet output, exports the image set in 128 channels；The last one residual error in single scale network
A convolutional layer and a ReLU active coating are connected to after net, wherein convolutional layer includes the convolution kernel in 3 channels, and each convolution kernel is big
Small is 5*5, step-length 1, zero padding width are 2, if the scale is k, output size minibatch*3*wk*hk.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810205661.6A CN109035149B (en) | 2018-03-13 | 2018-03-13 | License plate image motion blur removing method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810205661.6A CN109035149B (en) | 2018-03-13 | 2018-03-13 | License plate image motion blur removing method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109035149A true CN109035149A (en) | 2018-12-18 |
| CN109035149B CN109035149B (en) | 2021-07-09 |
Family
ID=64630269
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810205661.6A Active CN109035149B (en) | 2018-03-13 | 2018-03-13 | License plate image motion blur removing method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109035149B (en) |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109740607A (en) * | 2018-12-26 | 2019-05-10 | 南京互连智能科技有限公司 | The incomplete region detection of character picture and incomplete character picture restoration methods |
| CN109800682A (en) * | 2018-12-29 | 2019-05-24 | 桂林电子科技大学 | Driver attributes' recognition methods and Related product |
| CN109903237A (en) * | 2019-01-23 | 2019-06-18 | 复旦大学 | A kind of multiple dimensioned facial image deblurring algorithm based on the low high frequency of separation |
| CN109949905A (en) * | 2019-02-28 | 2019-06-28 | 福建自贸区试验区厦门片区Manteia数据科技有限公司 | A kind of 3-dimensional dose distribution forecasting method based on target area |
| CN110188752A (en) * | 2019-05-20 | 2019-08-30 | 南京邮电大学 | The deblurring identifying system and deblurring recognition methods of fuzzy license plate under monitor video |
| CN110223259A (en) * | 2019-06-14 | 2019-09-10 | 华北电力大学（保定） | A kind of road traffic fuzzy image enhancement method based on production confrontation network |
| CN110378844A (en) * | 2019-06-14 | 2019-10-25 | 杭州电子科技大学 | Motion blur method is gone based on the multiple dimensioned Image Blind for generating confrontation network is recycled |
| CN111028177A (en) * | 2019-12-12 | 2020-04-17 | 武汉大学 | Edge-based deep learning image motion blur removing method |
| CN111209850A (en) * | 2020-01-04 | 2020-05-29 | 圣点世纪科技股份有限公司 | Method for generating applicable multi-device identification finger vein image based on improved cGAN network |
| CN111275637A (en) * | 2020-01-15 | 2020-06-12 | 北京工业大学 | Non-uniform motion blurred image self-adaptive restoration method based on attention model |
| CN111310862A (en) * | 2020-03-27 | 2020-06-19 | 西安电子科技大学 | Deep neural network license plate positioning method based on image enhancement in complex environment |
| CN111583124A (en) * | 2019-02-19 | 2020-08-25 | 宁波舜宇光电信息有限公司 | Method, device, system and storage medium for deblurring images |
| CN111681188A (en) * | 2020-06-15 | 2020-09-18 | 青海民族大学 | Image deblurring method based on combination of image pixel prior and image gradient prior |
| CN112070176A (en) * | 2020-09-18 | 2020-12-11 | 福州大学 | Cutting-free end-to-end license plate recognition method |
| CN112419171A (en) * | 2020-10-28 | 2021-02-26 | 云南电网有限责任公司昆明供电局 | Image restoration method for multi-residual-block conditional generation countermeasure network |
| CN112686119A (en) * | 2020-12-25 | 2021-04-20 | 陕西师范大学 | License plate motion blurred image processing method based on self-attention generation countermeasure network |
| CN112949636A (en) * | 2021-03-31 | 2021-06-11 | 上海电机学院 | License plate super-resolution identification method and system and computer readable medium |
| CN113129240A (en) * | 2021-05-19 | 2021-07-16 | 广西师范大学 | Motion blur removing method for industrial packaging characters |
| CN113409200A (en) * | 2020-03-17 | 2021-09-17 | 通用汽车环球科技运作有限责任公司 | System and method for image deblurring in a vehicle |
| CN113434491A (en) * | 2021-06-18 | 2021-09-24 | 深圳市曙光信息技术有限公司 | Character model data cleaning method, system and medium for deep learning OCR recognition |
| CN114612580A (en) * | 2022-03-15 | 2022-06-10 | 中国人民解放军国防科技大学 | High-definition imaging method for low-quality camera |
| CN114821449A (en) * | 2022-06-27 | 2022-07-29 | 松立控股集团股份有限公司 | License plate image processing method based on attention mechanism |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107154023A (en) * | 2017-05-17 | 2017-09-12 | 电子科技大学 | Face super-resolution reconstruction method based on generation confrontation network and sub-pix convolution |
| CN107590774A (en) * | 2017-09-18 | 2018-01-16 | 北京邮电大学 | A kind of car plate clarification method and device based on generation confrontation network |
| CN107704857A (en) * | 2017-09-25 | 2018-02-16 | 北京邮电大学 | A kind of lightweight licence plate recognition method and device end to end |
-
2018
- 2018-03-13 CN CN201810205661.6A patent/CN109035149B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107154023A (en) * | 2017-05-17 | 2017-09-12 | 电子科技大学 | Face super-resolution reconstruction method based on generation confrontation network and sub-pix convolution |
| CN107590774A (en) * | 2017-09-18 | 2018-01-16 | 北京邮电大学 | A kind of car plate clarification method and device based on generation confrontation network |
| CN107704857A (en) * | 2017-09-25 | 2018-02-16 | 北京邮电大学 | A kind of lightweight licence plate recognition method and device end to end |
Non-Patent Citations (1)
| Title |
|---|
| T M NIMISHA ET AL.: "Blur-Invariant Deep Learning for Blind-Deblurring", 《2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION》 * |
Cited By (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109740607B (en) * | 2018-12-26 | 2023-07-21 | 南京互连智能科技有限公司 | Incomplete region detection and incomplete character image restoration method for character image |
| CN109740607A (en) * | 2018-12-26 | 2019-05-10 | 南京互连智能科技有限公司 | The incomplete region detection of character picture and incomplete character picture restoration methods |
| CN109800682A (en) * | 2018-12-29 | 2019-05-24 | 桂林电子科技大学 | Driver attributes' recognition methods and Related product |
| CN109903237A (en) * | 2019-01-23 | 2019-06-18 | 复旦大学 | A kind of multiple dimensioned facial image deblurring algorithm based on the low high frequency of separation |
| CN109903237B (en) * | 2019-01-23 | 2023-04-07 | 复旦大学 | Multi-scale face image deblurring method based on low and high frequency separation |
| CN111583124A (en) * | 2019-02-19 | 2020-08-25 | 宁波舜宇光电信息有限公司 | Method, device, system and storage medium for deblurring images |
| CN109949905A (en) * | 2019-02-28 | 2019-06-28 | 福建自贸区试验区厦门片区Manteia数据科技有限公司 | A kind of 3-dimensional dose distribution forecasting method based on target area |
| CN110188752A (en) * | 2019-05-20 | 2019-08-30 | 南京邮电大学 | The deblurring identifying system and deblurring recognition methods of fuzzy license plate under monitor video |
| CN110378844A (en) * | 2019-06-14 | 2019-10-25 | 杭州电子科技大学 | Motion blur method is gone based on the multiple dimensioned Image Blind for generating confrontation network is recycled |
| CN110223259A (en) * | 2019-06-14 | 2019-09-10 | 华北电力大学（保定） | A kind of road traffic fuzzy image enhancement method based on production confrontation network |
| CN110378844B (en) * | 2019-06-14 | 2021-04-09 | 杭州电子科技大学 | Image blind motion blur removing method based on cyclic multi-scale generation countermeasure network |
| CN111028177A (en) * | 2019-12-12 | 2020-04-17 | 武汉大学 | Edge-based deep learning image motion blur removing method |
| CN111028177B (en) * | 2019-12-12 | 2023-07-21 | 武汉大学 | Edge-based deep learning image motion blur removing method |
| CN111209850A (en) * | 2020-01-04 | 2020-05-29 | 圣点世纪科技股份有限公司 | Method for generating applicable multi-device identification finger vein image based on improved cGAN network |
| CN111275637B (en) * | 2020-01-15 | 2024-01-30 | 北京工业大学 | Attention model-based non-uniform motion blurred image self-adaptive restoration method |
| CN111275637A (en) * | 2020-01-15 | 2020-06-12 | 北京工业大学 | Non-uniform motion blurred image self-adaptive restoration method based on attention model |
| CN113409200A (en) * | 2020-03-17 | 2021-09-17 | 通用汽车环球科技运作有限责任公司 | System and method for image deblurring in a vehicle |
| CN113409200B (en) * | 2020-03-17 | 2023-09-29 | 通用汽车环球科技运作有限责任公司 | System and method for image deblurring in a vehicle |
| CN111310862B (en) * | 2020-03-27 | 2024-02-09 | 西安电子科技大学 | Image enhancement-based deep neural network license plate positioning method in complex environment |
| CN111310862A (en) * | 2020-03-27 | 2020-06-19 | 西安电子科技大学 | Deep neural network license plate positioning method based on image enhancement in complex environment |
| CN111681188A (en) * | 2020-06-15 | 2020-09-18 | 青海民族大学 | Image deblurring method based on combination of image pixel prior and image gradient prior |
| CN111681188B (en) * | 2020-06-15 | 2022-06-17 | 青海民族大学 | Image deblurring method based on combination of image pixel prior and image gradient prior |
| CN112070176A (en) * | 2020-09-18 | 2020-12-11 | 福州大学 | Cutting-free end-to-end license plate recognition method |
| CN112419171A (en) * | 2020-10-28 | 2021-02-26 | 云南电网有限责任公司昆明供电局 | Image restoration method for multi-residual-block conditional generation countermeasure network |
| CN112686119A (en) * | 2020-12-25 | 2021-04-20 | 陕西师范大学 | License plate motion blurred image processing method based on self-attention generation countermeasure network |
| CN112686119B (en) * | 2020-12-25 | 2022-12-09 | 陕西师范大学 | License plate motion blurred image processing method based on self-attention generation countermeasure network |
| CN112949636B (en) * | 2021-03-31 | 2023-05-30 | 上海电机学院 | License plate super-resolution recognition method, system and computer readable medium |
| CN112949636A (en) * | 2021-03-31 | 2021-06-11 | 上海电机学院 | License plate super-resolution identification method and system and computer readable medium |
| CN113129240B (en) * | 2021-05-19 | 2023-07-25 | 广西师范大学 | Motion blur removing method for industrial packaging characters |
| CN113129240A (en) * | 2021-05-19 | 2021-07-16 | 广西师范大学 | Motion blur removing method for industrial packaging characters |
| CN113434491B (en) * | 2021-06-18 | 2022-09-02 | 深圳市曙光信息技术有限公司 | Character model data cleaning method, system and medium for deep learning OCR recognition |
| CN113434491A (en) * | 2021-06-18 | 2021-09-24 | 深圳市曙光信息技术有限公司 | Character model data cleaning method, system and medium for deep learning OCR recognition |
| CN114612580A (en) * | 2022-03-15 | 2022-06-10 | 中国人民解放军国防科技大学 | High-definition imaging method for low-quality camera |
| CN114821449A (en) * | 2022-06-27 | 2022-07-29 | 松立控股集团股份有限公司 | License plate image processing method based on attention mechanism |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109035149B (en) | 2021-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109035149A (en) | A kind of license plate image based on deep learning goes motion blur method | |
| CN104077613B (en) | Crowd density estimation method based on cascaded multilevel convolution neural network | |
| CN111126472A (en) | Improved target detection method based on SSD | |
| CN110378844A (en) | Motion blur method is gone based on the multiple dimensioned Image Blind for generating confrontation network is recycled | |
| CN110929602A (en) | Foundation cloud picture cloud shape identification method based on convolutional neural network | |
| CN108921196A (en) | A kind of semantic segmentation method for improving full convolutional neural networks | |
| CN110532946B (en) | Method for identifying axle type of green-traffic vehicle based on convolutional neural network | |
| CN113780132B (en) | Lane line detection method based on convolutional neural network | |
| CN111832453B (en) | Unmanned scene real-time semantic segmentation method based on two-way deep neural network | |
| CN111611998A (en) | Adaptive feature block extraction method based on candidate region area and width and height | |
| CN113255837A (en) | Improved CenterNet network-based target detection method in industrial environment | |
| CN112489050A (en) | Semi-supervised instance segmentation algorithm based on feature migration | |
| CN110827312A (en) | Learning method based on cooperative visual attention neural network | |
| CN112232351A (en) | License plate recognition system based on deep neural network | |
| CN109191392A (en) | A kind of image super-resolution reconstructing method of semantic segmentation driving | |
| CN111881716A (en) | Pedestrian re-identification method based on multi-view-angle generation countermeasure network | |
| CN107705323A (en) | A kind of level set target tracking method based on convolutional neural networks | |
| CN114973011A (en) | High-resolution remote sensing image building extraction method based on deep learning | |
| CN111507184B (en) | Human body posture detection method based on parallel cavity convolution and body structure constraint | |
| CN114463492A (en) | Adaptive channel attention three-dimensional reconstruction method based on deep learning | |
| CN109859222A (en) | Edge extracting method and system based on cascade neural network | |
| CN115797808A (en) | Unmanned aerial vehicle inspection defect image identification method, system, device and medium | |
| CN111462090A (en) | Multi-scale image target detection method | |
| CN108154488B (en) | A kind of image motion ambiguity removal method based on specific image block analysis | |
| CN110136098B (en) | Cable sequence detection method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |