CN111639542A - License plate recognition method, device, equipment and medium - Google Patents
License plate recognition method, device, equipment and medium Download PDFInfo
- Publication number
- CN111639542A CN111639542A CN202010372209.6A CN202010372209A CN111639542A CN 111639542 A CN111639542 A CN 111639542A CN 202010372209 A CN202010372209 A CN 202010372209A CN 111639542 A CN111639542 A CN 111639542A
- Authority
- CN
- China
- Prior art keywords
- image
- license plate
- foggy
- model
- edge gradient
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000010606 normalization Methods 0.000 claims description 51
- 238000012549 training Methods 0.000 claims description 29
- 238000012545 processing Methods 0.000 claims description 27
- 238000004590 computer program Methods 0.000 claims description 5
- 238000001228 spectrum Methods 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims 4
- 239000000779 smoke Substances 0.000 description 12
- 230000008569 process Effects 0.000 description 9
- 238000004422 calculation algorithm Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 241000820057 Ithone Species 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000002834 transmittance Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a license plate recognition method, a license plate recognition device, license plate recognition equipment and a license plate recognition medium. The method comprises the following steps: acquiring a to-be-recognized image obtained by shooting a target license plate in a foggy environment; inputting the image to be recognized into a pre-trained generative confrontation network to obtain an edge gradient image corresponding to the image to be recognized; determining a target area image corresponding to the target license plate from the edge gradient image; and identifying characters of the target license plate from the target area image. According to the license plate recognition method, the license plate recognition device, the license plate recognition equipment and the license plate recognition medium, the license plate recognition accuracy can be improved.
Description
Technical Field
The present invention relates to the field of communications, and in particular, to a license plate recognition method, apparatus, device, and medium.
Background
The license plate identification is an application of a computer video image identification technology in vehicle license plate identification, and the license plate identification technology requires that a moving vehicle license plate can be extracted and identified from a complex background, and information such as a vehicle license plate number can be identified.
However, the license plate of the vehicle in the image collected in the foggy environment is hidden and disturbed by the fog and the like, and the recognition accuracy is low.
Disclosure of Invention
The license plate recognition method, the license plate recognition device, the license plate recognition equipment and the license plate recognition medium can improve the license plate recognition accuracy.
In a first aspect, a license plate recognition method is provided, including: acquiring a to-be-recognized image obtained by shooting a target license plate in a foggy environment; inputting an image to be recognized into a pre-trained generative confrontation network to obtain an edge gradient image corresponding to the image to be recognized; determining a target area image corresponding to a target license plate from the edge gradient image; and identifying characters of the target license plate from the target area image.
In an optional embodiment, the method further comprises: the method comprises the steps of training a generative countermeasure network by utilizing a plurality of foggy image samples containing license plates and first edge gradient images which respectively correspond to the foggy image samples one by one.
In an optional embodiment, the method further comprises: acquiring a plurality of foggy image samples containing license plates and fog-free image samples which respectively correspond to the foggy image samples one by one; carrying out image enhancement processing on the fog-free image sample corresponding to each fog-containing image sample to obtain an enhanced image-free image sample; and extracting the edge gradient characteristics of each pixel point in the fog-free image sample after the enhancement processing to obtain a first edge gradient image corresponding to each fog-containing image sample.
In an alternative embodiment, the generative confrontation network includes an image generation model and an image discrimination model, and the method further includes: acquiring a plurality of foggy image samples containing license plates and second edge gradient images which respectively correspond to the foggy image samples one to one; for each of a plurality of foggy image samples containing license plates, performing the following operations: inputting each foggy image sample into an image generation model to obtain a first edge gradient image corresponding to each foggy image sample; calculating to obtain a weighted total loss value of each hazy image sample based on the first edge gradient image and the second edge gradient image of each hazy image sample; and adjusting the network parameters of the image generation model and the network parameters of the image discrimination model according to the weighted total loss value.
In an optional embodiment, adjusting the network parameters of the image generation model and the network parameters of the image discrimination model according to the weighted total loss value includes: under the condition of keeping the network parameters of the image generation model unchanged, adjusting the network parameters of the image discrimination model to enable the weighted total loss value to reach the minimum value; and adjusting the network parameters of the image generation model according to the image discrimination model and the truth discrimination result of the first edge gradient image until the discrimination module reaches Nash balance.
In an alternative embodiment, the generative confrontation network includes an image generation model and an image discrimination model, and the method further includes: constructing an image generation model, wherein the image generation model comprises a first normalization layer, a first convolution layer, a second normalization layer, a second convolution layer, a third normalization layer, a third convolution layer, a fourth normalization layer and a fourth convolution layer which are sequentially arranged; constructing an image discrimination model, wherein the image discrimination model comprises a fifth convolution layer, a sixth convolution layer, a fifth normalization layer, a seventh convolution layer, a sixth normalization layer, an eighth convolution layer, a seventh normalization layer and a spectrum normalization layer which are sequentially arranged; and connecting the fourth convolution layer with the fifth convolution layer to obtain the generative countermeasure network.
In a second aspect, there is provided a license plate recognition device, comprising: the first image acquisition module is used for acquiring an image to be recognized which is obtained by shooting a target license plate in a foggy environment; the first image processing module is used for inputting the image to be recognized into a pre-trained generative confrontation network to obtain an edge gradient image corresponding to the image to be recognized; the second image processing module is used for determining a target area image corresponding to the target license plate in the edge gradient image; and the license plate recognition module is used for recognizing the characters of the target license plate in the target area image.
In an alternative embodiment, the apparatus further comprises: and the model training module is used for training the generative confrontation network by utilizing a plurality of foggy image samples containing license plates and edge gradient images which respectively correspond to the foggy image samples one by one.
In an alternative embodiment, the apparatus further comprises: the second image acquisition module is used for acquiring a plurality of foggy image samples containing license plates and fogless image samples which respectively correspond to the foggy image samples one by one; and the third image processing module is used for carrying out image enhancement processing on the fog-free image sample corresponding to each foggy image sample to obtain a first edge gradient image corresponding to each foggy image sample.
In an alternative embodiment, the generative confrontation network includes an image generation model and an image discrimination model, and the apparatus further includes: the third image acquisition module is used for acquiring a plurality of foggy image samples containing license plates and second edge gradient images which respectively correspond to the foggy image samples one by one; the model training module is used for executing the following operations aiming at each foggy image sample in a plurality of foggy image samples containing license plates: inputting each foggy image sample into an image generation model to obtain a first edge gradient image corresponding to each foggy image sample; calculating to obtain a weighted total loss value of each hazy image sample based on the first edge gradient image and the second edge gradient image of each hazy image sample; and adjusting the network parameters of the image generation model and the network parameters of the image discrimination model according to the weighted total loss value.
In an alternative embodiment, the model training module is specifically configured to: under the condition of keeping the network parameters of the image generation model unchanged, adjusting the network parameters of the image discrimination model to enable the weighted total loss value to reach the minimum value; and adjusting the network parameters of the image generation model according to the truth judgment result of the image judgment model on the first edge gradient image until the judgment module reaches Nash balance.
In an alternative embodiment, the generative confrontation network includes an image generation model and an image discrimination model, and the apparatus further includes: the image generation module comprises a first normalization layer, a first convolution layer, a second normalization layer, a second convolution layer, a third normalization layer, a third convolution layer, a fourth normalization layer and a fourth convolution layer which are sequentially arranged; the second model building module is used for building an image discrimination model, and the image discrimination model comprises a fifth convolution layer, a sixth convolution layer, a fifth normalization layer, a seventh convolution layer, a sixth normalization layer, an eighth convolution layer, a seventh normalization layer and a spectrum normalization layer which are sequentially arranged; and the model acquisition module is used for connecting the fourth convolution layer with the fifth convolution layer to obtain the generative confrontation network.
In a third aspect, a license plate recognition apparatus is provided, including: a memory for storing a program;
and a processor, configured to execute a program stored in the memory to perform the license plate recognition method provided in the first aspect or any optional implementation manner of the first aspect.
In a fourth aspect, a computer storage medium is provided, where computer program instructions are stored on the computer storage medium, and when executed by a processor, the computer program instructions implement the license plate recognition method provided in the first aspect or any optional implementation manner of the first aspect.
According to the license plate recognition method, the license plate recognition device, the license plate recognition equipment and the license plate recognition media in the embodiment of the invention, the image to be recognized can be processed by utilizing the generating countermeasure network to obtain the edge gradient image corresponding to the image to be recognized, and the characters of the target license plate are recognized based on the edge gradient image. Considering that a target license plate image in an image to be recognized, which is shot in a foggy environment, can be covered by fog, smoke, haze and the like, the generated countermeasure network can enhance the image in the image to be recognized, and the influence of the fog, the smoke, the haze and the like on the image is reduced, so that the recognition precision is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a license plate recognition method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a license plate recognition device according to an embodiment of the present invention;
fig. 3 is a structural diagram of an exemplary hardware architecture of a license plate recognition device according to an embodiment of the present invention.
Detailed Description
Features and exemplary embodiments of various aspects of the present invention will be described in detail below, and in order to make objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not to be construed as limiting the invention. It will be apparent to one skilled in the art that the present invention may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the present invention.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
In the process of recognizing the license plate, the license plate may be shielded by smoke and the like to influence the recognition accuracy. Therefore, a technical scheme capable of identifying the license plate in a foggy environment is needed to improve the license plate identification precision.
In a feasible license plate recognition scheme, the collected image can be subjected to defogging treatment. Specifically, a dark channel prior defogging method can be adopted to perform defogging processing on a captured image of a foggy automobile. The method for defogging the dark channel prior image comprises the following specific steps: the method comprises the following steps of firstly, dividing an input foggy day automobile image into 15 × 15 color blocks, and solving local and global dark primary color maps. And secondly, assuming that atmospheric light components are known, and estimating the atmospheric transmittance through a dark channel prior theory. And a third step of estimating the atmospheric light component by dark channel prior. And fourthly, restoring a fog-free image. And fifthly, carrying out image enhancement and color image recovery on the recovered fog-free image by using a multi-scale Retinex algorithm. And a sixth step of decomposing the defogged image J into R, G, B three grayscale images and respectively enhancing the grayscale images. And seventhly, combining the three gray level images through a multi-scale Retinex algorithm to obtain an enhanced image.
However, the dark channel prior defogging algorithm is based on the assumed prior theory, but the prior information is not sufficient, various prior assumptions are often accompanied by new problems while solving a certain class of problems, and it is very difficult to artificially analyze and find an accurate prior model. For example, for an object with a color close to atmospheric color in a filled area, if there is no shadow coverage, the defogging effect of this method is ineffective, and the dark channel value is large because of the high values of all three color channels.
In addition, the defogging algorithm and the license plate positioning algorithm are calculated respectively in the method, repeated calculation is caused by the method, the calculation cost is increased, and the calculation time is too long.
For better understanding of the present invention, a license plate recognition method, an apparatus, a device and a medium according to embodiments of the present invention will be described in detail below with reference to the accompanying drawings, and it should be noted that these embodiments are not intended to limit the scope of the present disclosure.
Fig. 1 is a schematic flow chart illustrating a license plate recognition method according to an embodiment of the present invention. As shown in fig. 1, the license plate recognition method 100 in the present embodiment may include the following steps S110 to S140.
And S110, acquiring an image to be recognized obtained by shooting a target license plate in a foggy environment. The image to be recognized may be acquired by an apparatus having an image acquisition function, such as a camera or a monitoring apparatus. The foggy environment may be a foggy day, a haze day, or other environments where smoke exists, and is not particularly limited.
And S120, inputting the image to be recognized into a pre-trained generative confrontation network to obtain an edge gradient image corresponding to the image to be recognized. The edge gradient image corresponding to the image to be identified contains information of the edge gradient characteristics of each pixel point.
A generative countermeasure network is targeted. The generative confrontation network can learn the mapping relation between the foggy image and the edge gradient map through pre-training. The relatively real image edge gradient image information can be obtained by utilizing the generative countermeasure network, the dependence on prior information such as a foggy weather atmosphere model and the like is avoided, and meanwhile, the calculated amount of image processing is reduced.
The following sections will specifically describe the generative countermeasure network.
First, a specific description is given of a specific structure of the generative countermeasure network. The generative confrontation network comprises an image generation model and an image discrimination model. At this time, the edge gradient image acquired in S120 is output by the image generation model.
The image generation model comprises a first normalization layer, a first convolution layer, a second normalization layer, a second convolution layer, a third normalization layer, a third convolution layer, a fourth normalization layer and a fourth convolution layer which are sequentially arranged. Each layer of initial parameters of the generative countermeasure network specifically includes: the sizes of convolution kernels of the first convolution layer, the second convolution layer, the third convolution layer and the fourth convolution layer are all set to be 5 multiplied by 5, the step length is all set to be 2, and the edge filling mode is all set to be SAME. The mean values of the first normalization layer, the second normalization layer and the third normalization layer are all set to be 0, and the variances are all set to be 1.
The image discrimination model includes a fifth convolution layer, a sixth convolution layer, a fifth normalization layer, a seventh convolution layer, a sixth normalization layer, an eighth convolution layer, a seventh normalization layer, and a spectral normalization layer, which are arranged in order.
When the generative countermeasure network is constructed, the image generation model may be constructed first, the image discrimination model may be constructed second, and finally the fourth convolution layer of the image generation model and the fifth convolution layer of the image discrimination model may be connected.
Next, the specific training procedure for the generative confrontation network is described below.
In some embodiments, before using the generative warfare network in S120, a step of training the generative warfare network is further included. Specifically, the training step specifically includes: the method comprises the steps of training a generative countermeasure network by utilizing a plurality of foggy image samples containing license plates and first edge gradient images which respectively correspond to the foggy image samples one by one.
In some embodiments, generating a competing network specifically includes the following two steps.
The method comprises the following steps of firstly, obtaining a plurality of foggy image samples containing license plates and first edge gradient images which respectively correspond to the foggy image samples one by one.
In one embodiment, step one may specifically include the following three substeps.
The method comprises a first substep of obtaining a plurality of foggy image samples containing license plates and fogless image samples which respectively correspond to the fogless image samples one by one. The license plate in the foggy image is completely or partially covered by fog, smoke, haze and the like, and the license plate in the non-image is not covered by the fog, the smoke, the haze and the like. For example, a foggy image and a fogless image taken of the same license plate may be treated as a set of corresponding foggy image samples and fogless image samples. Alternatively, the corresponding foggy and fogless image samples differ only in the presence or absence of masking by fog, smoke, haze, etc.
In a specific embodiment, a plurality of foggy image samples containing license plates and fogless image samples respectively corresponding to the plurality of foggy image samples in a one-to-one manner can be acquired through a public image data set or a web crawler technology.
At this time, since the training data is acquired by the public image data set or web crawler technology, it is authentic. The training data can be regarded as prior information which is the relation between real data and is not assumed, so that the defogging method and the defogging device do not depend on the assumed prior information to be used for defogging, and have wider application range.
And in the second substep, carrying out image enhancement processing on the fog-free image sample corresponding to each fog-containing image sample to obtain an enhanced image-free image sample. The image enhancement processing technology can make the original unclear image clear or emphasize some interesting features by purposefully emphasizing the overall or local characteristics of the image, enlarge the difference between different object features in the image, inhibit the uninteresting features, improve the image quality, enrich the information content and enhance the image interpretation and identification effects. For example, the image enhancement method may be a contrast enhancement method, a gray scale conversion method, or the like.
In one embodiment, to further improve the accuracy, the fog-free image sample after the enhancement processing may be further preprocessed, such as graying processing, median filtering processing, and the like.
And a third substep of extracting the edge gradient characteristics of each pixel point in the fog-free image sample after the enhancement processing to obtain a first edge gradient image corresponding to each fog-containing image sample. Illustratively, a CANNY edge extraction method or the like may be utilized.
And secondly, executing the following three substeps for each fogging image sample in the plurality of fogging image samples containing the license plate.
And a first substep of inputting each foggy image sample into an image generation model to obtain a second edge gradient image corresponding to each foggy image sample.
And a second substep of calculating a weighted total loss value of each hazy image sample based on the first edge gradient image of each hazy image sample and the second edge gradient image of each hazy image sample.
Wherein, if each foggy image sample is the ith one of the above foggy image samples including the license plate as the training set, the weighted total loss value of each foggy image sample satisfies the formula (1):
wherein each of the foggy image samples is λ1And λ2Respectively a first weight coefficient and a second weight coefficient, lambda1And λ2Is at [0,1 ]]Two fractions in the range, and λ1And λ2The sum is equal to 1.
A distance loss value representing a distance between the first edge gradient image and the second edge gradient image of each of the hazy image samples,
and indicating the perception loss value of the first edge gradient image and the second edge gradient image of each hazy image sample.
Wherein the distance loss value
Is calculated by the formula (2):
wherein m represents the total number of channels of each of the fog image samples, w and h represent the width and height of each of the fog image samples, respectively, n represents the total number of pixels of each of the fog image samples, j represents the serial number of the pixels of each of the fog image samples,yi,ja pixel value, x, of the jth pixel in the first edge gradient image representing each of the fog image samplesi,jThe pixel value of the jth pixel in the second edge gradient image representing each of the hazy image samples.
Wherein, the perception loss value is calculated by formula (3):
wherein f represents the total number of channels of the deep feature map corresponding to each foggy image sample, g and d represent the width and height of the deep feature map corresponding to each foggy image sample, u represents the total number of pixels of the deep feature map corresponding to each foggy image sample, k represents the serial number of the pixels in the deep feature map corresponding to each foggy image sample, y represents the serial number of the pixels in the deep feature map corresponding to each foggy image samplei,kRepresenting the value, x, of the k-th pixel in the deep feature map of the first edge gradient image corresponding to each of the fog image samplesi,kThe pixel value of the k-th pixel in the deep feature map of each of the fog image samples is represented.
And a third substep of adjusting the network parameters of the image generation model and the network parameters of the image discrimination model according to the weighted total loss value obtained by the formula (1).
In one embodiment, a detailed implementation of the parameter adjustment substep is as follows.
Firstly, under the condition of keeping the network parameters of the image generation model unchanged, adjusting the network parameters of the image discrimination model to enable the weighted total loss value of each foggy image sample to reach the minimum value.
Illustratively, the network parameters of the image discrimination model may be adjusted using a stochastic gradient descent algorithm.
Specifically, the update formula of the network parameters using the stochastic gradient descent algorithm is shown in formula (4):
θv+1＝θv-L′(θV) (4)
wherein, thetaV+1Image discrimination model after v +1 th updateNetwork parameter of thetaVRepresents the network parameters of the image discrimination model after the v-th update, L 'represents the partial derivative operation, L' (θ:)V) The loss value L (theta) is expressed in the network parameter thetavThe partial derivative value of time.
And secondly, adjusting the network parameters of the image generation model according to the image discrimination model and the truth discrimination result of the first edge gradient image until the discrimination module reaches Nash balance. Illustratively, the discrimination module reaches Nash equilibrium at a discrimination probability of 0.5.
And S130, determining a target area image corresponding to the target license plate from the edge gradient image obtained in the S120. The target area image is the area occupied by the target license plate on the edge gradient image.
In some embodiments, the target license plate in the image may have a certain deviation from the real license plate in shape due to the influence of the shooting angle, smoke refraction and other factors. For example, the target area image determined in S130 may be non-horizontal or a parallelogram. Therefore, in order to improve the recognition accuracy, after the target area image is acquired, it may be horizontally rectified.
S140, identifying characters of the target license plate from the target area image. Specifically, the characters of the target license plate may include a license plate number composed of one or more of chinese characters, english, and numbers.
As an example, a Tesseract engine (i.e., a type of text recognition engine) may be used to recognize characters of a target license plate from a target area image. The JTessBoxEditor training tool can be used for carrying out sample training on the Tesseract engine so as to improve the character recognition accuracy.
According to the license plate recognition method, the license plate recognition device, the license plate recognition equipment and the license plate recognition media in the embodiment of the invention, the image to be recognized can be processed by utilizing the generating countermeasure network to obtain the edge gradient image corresponding to the image to be recognized, and the characters of the target license plate are recognized based on the edge gradient image. Considering that a target license plate image in an image to be recognized, which is shot in a foggy environment, can be covered by fog, smoke, haze and the like, the generated countermeasure network can enhance the image in the image to be recognized, and the influence of the fog, the smoke, the haze and the like on the image is reduced, so that the recognition precision is improved.
In addition, the embodiment of the invention transfers a large amount of operation cost to the training process of the network. In practical application, the trained generative confrontation network is only required to be directly called to generate an edge gradient map containing the target license plate, and a fog-free clear image is obtained without defogging. Thus reducing the operation time and reducing the amount of operations.
An apparatus according to an embodiment of the present invention will be described in detail below with reference to the accompanying drawings.
Based on the same inventive concept, the embodiment of the invention provides a license plate recognition device. Fig. 2 is a schematic structural diagram of a license plate recognition device according to an embodiment of the present invention. As shown in fig. 2, the license plate recognition device 200 includes a first image acquisition module 210, a first image processing module 220, a second image processing module 230, and a license plate recognition module 240.
The first image obtaining module 210 is configured to obtain an image to be recognized obtained by shooting a target license plate in a foggy environment.
The first image processing module 220 is configured to input the image to be recognized into a pre-trained generative confrontation network, so as to obtain an edge gradient image corresponding to the image to be recognized.
And the second image processing module 230 is configured to determine a target area image corresponding to the target license plate in the edge gradient image.
And the license plate recognition module 240 is configured to recognize characters of the target license plate in the target area image.
In some embodiments, the license plate recognition device 200 further includes a model training module.
The model training module is used for training the generative countermeasure network by utilizing a plurality of foggy image samples containing license plates and edge gradient images which respectively correspond to the foggy image samples one by one.
In some embodiments, the license plate recognition device 200 further includes a second image acquisition module and a third image processing module.
The second image acquisition module is used for acquiring a plurality of foggy image samples containing license plates and fog-free image samples which respectively correspond to the foggy image samples one to one.
The third image processing module is used for carrying out image enhancement processing on the fog-free image sample corresponding to each foggy image sample to obtain a first edge gradient image corresponding to each foggy image sample.
In some embodiments, the generative confrontation network includes an image generation model and an image discrimination model. Accordingly, the license plate recognition device 200 further includes a third image acquisition module and a model training module.
The third image acquisition module is used for acquiring a plurality of foggy image samples containing license plates and first edge gradient images which respectively correspond to the foggy image samples one to one.
The model training module is used for executing the following operations for each foggy image sample in a plurality of foggy image samples containing license plates: and inputting each foggy image sample into an image generation model to obtain a second edge gradient image corresponding to each foggy image sample. And calculating to obtain a weighted total loss value of each hazy image sample based on the first edge gradient image and the second edge gradient image of each hazy image sample. And adjusting the network parameters of the image generation model and the network parameters of the image discrimination model according to the weighted total loss value.
In some embodiments, the model training module is specifically configured to adjust the network parameters of the image discrimination model under the condition that the network parameters of the image generation model are kept unchanged, so that the weighted total loss value reaches a minimum value.
And the model training module is further specifically used for adjusting the network parameters of the image generation model according to the truth judgment result of the image judgment model on the first edge gradient image until the judgment module reaches Nash balance.
In some embodiments, the generative confrontation network includes an image generation model and an image discrimination model. Accordingly, the license plate recognition device 200 may further include a first model building module, a second model building module, and a model obtaining module.
The first model building module is used for building an image generation model. The image generation model comprises a first normalization layer, a first convolution layer, a second normalization layer, a second convolution layer, a third normalization layer, a third convolution layer, a fourth normalization layer and a fourth convolution layer which are sequentially arranged.
The second model building module is used for building an image discrimination model, and the image discrimination model comprises a fifth convolution layer, a sixth convolution layer, a fifth normalization layer, a seventh convolution layer, a sixth normalization layer, an eighth convolution layer, a seventh normalization layer and a spectrum normalization layer which are sequentially arranged.
The model obtaining module is used for connecting the fourth convolution layer with the fifth convolution layer to obtain the generative confrontation network.
According to the license plate recognition device provided by the embodiment of the invention, the image to be recognized can be processed by utilizing the generating countermeasure network to obtain the edge gradient image corresponding to the image to be recognized, and the characters of the target license plate are recognized based on the edge gradient image. Considering that a target license plate image in an image to be recognized, which is shot in a foggy environment, can be covered by fog, smoke, haze and the like, the generated countermeasure network can enhance the image in the image to be recognized, and the influence of the fog, the smoke, the haze and the like on the image is reduced, so that the recognition precision is improved.
Other details of the license plate recognition device according to the embodiment of the present invention are similar to the license plate recognition method described above with reference to the example shown in fig. 1, and can achieve the corresponding technical effects, and are not repeated herein for brevity.
Fig. 3 is a structural diagram of an exemplary hardware architecture of a license plate recognition device according to an embodiment of the present invention.
As shown in fig. 3, the license plate recognition device 300 includes an input device 301, an input interface 302, a central processor 303, a memory 304, an output interface 305, and an output device 306. The input interface 302, the central processing unit 303, the memory 304, and the output interface 305 are connected to each other through a bus 310, and the input device 301 and the output device 306 are connected to the bus 310 through the input interface 302 and the output interface 305, respectively, and further connected to other components of the license plate recognition device 300.
Specifically, the input device 301 receives input information from the outside and transmits the input information to the central processor 303 through the input interface 302; central processor 303 processes the input information based on computer-executable instructions stored in memory 304 to generate output information, stores the output information temporarily or permanently in memory 304, and then transmits the output information to output device 306 through output interface 305; the output device 306 outputs the output information to the outside of the license plate recognition device 300 for use by the user.
That is, the license plate recognition apparatus shown in fig. 3 may also be implemented to include: a memory storing computer-executable instructions; and a processor which, when executing computer executable instructions, may implement the method of the license plate recognition device described in connection with fig. 2.
In one embodiment, the license plate recognition device 300 shown in fig. 3 may be implemented as a device that may include: a memory for storing a program; and the processor is used for operating the program stored in the memory so as to execute the license plate recognition method provided by the embodiment of the invention.
The embodiment of the invention also provides a computer storage medium, wherein computer program instructions are stored on the computer storage medium, and when being executed by the processor, the computer program instructions realize the license plate recognition method of the embodiment of the invention.
It is to be understood that the invention is not limited to the specific arrangements and instrumentality described above and shown in the drawings. A detailed description of known methods is omitted herein for the sake of brevity. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present invention are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications and additions or change the order between the steps after comprehending the spirit of the present invention.
The functional blocks shown in the above structural block diagrams may be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, plug-in, function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine-readable medium or transmitted by a data signal carried in a carrier wave over a transmission medium or a communication link. A "machine-readable medium" may include any medium that can store or transfer information. Examples of a machine-readable medium include electronic circuits, semiconductor memory devices, ROM, flash memory, Erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, Radio Frequency (RF) links, and so forth. The code segments may be downloaded via computer networks such as the internet, intranet, etc.
As will be apparent to those skilled in the art, for convenience and brevity of description, the specific working processes of the systems, modules and units described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Claims (14)
1. A license plate recognition method is characterized by comprising the following steps:
acquiring a to-be-recognized image obtained by shooting a target license plate in a foggy environment;
inputting the image to be recognized into a pre-trained generative confrontation network to obtain an edge gradient image corresponding to the image to be recognized;
determining a target area image corresponding to the target license plate from the edge gradient image;
and identifying characters of the target license plate from the target area image.
2. The method of claim 1, further comprising:
and training the generative countermeasure network by utilizing a plurality of foggy image samples containing license plates and first edge gradient images which respectively correspond to the foggy image samples one by one.
3. The method of claim 2, wherein before the training of the generative warfare network using the plurality of foggy image samples containing the license plate and the first edge gradient image in one-to-one correspondence with the plurality of foggy image samples, the method further comprises:
acquiring a plurality of foggy image samples containing license plates and fog-free image samples which respectively correspond to the foggy image samples one by one;
carrying out image enhancement processing on the fog-free image sample corresponding to each fog-containing image sample to obtain an enhanced image-free image sample;
and extracting the edge gradient characteristics of each pixel point in the enhanced fog-free image sample to obtain a first edge gradient image corresponding to each fog-containing image sample.
4. The method of claim 1, wherein the generative confrontation network comprises an image generation model and an image discrimination model, the method further comprising:
acquiring a plurality of foggy image samples containing license plates and second edge gradient images which are respectively in one-to-one correspondence with the foggy image samples;
for each of the plurality of foggy image samples including the license plate, performing the following operations:
inputting each foggy image sample into the image generation model to obtain a first edge gradient image corresponding to each foggy image sample;
calculating to obtain a weighted total loss value of each hazy image sample based on the first edge gradient image and the second edge gradient image of each hazy image sample;
and adjusting the network parameters of the image generation model and the network parameters of the image discrimination model according to the weighted total loss value.
5. The method of claim 4, wherein adjusting the network parameters of the image generation model and the network parameters of the image discrimination model according to the weighted total loss value comprises:
under the condition of keeping the network parameters of the image generation model unchanged, adjusting the network parameters of the image discrimination model to enable the weighted total loss value to reach the minimum value;
and adjusting the network parameters of the image generation model according to the image discrimination model and the truth discrimination result of the first edge gradient image until the discrimination module reaches Nash balance.
6. The method of claim 1, wherein the generative confrontation network comprises an image generation model and an image discrimination model, the method further comprising:
constructing the image generation model, wherein the image generation model comprises a first normalization layer, a first convolution layer, a second normalization layer, a second convolution layer, a third normalization layer, a third convolution layer, a fourth normalization layer and a fourth convolution layer which are sequentially arranged;
constructing the image discrimination model, wherein the image discrimination model comprises a fifth convolution layer, a sixth convolution layer, a fifth normalization layer, a seventh convolution layer, a sixth normalization layer, an eighth convolution layer, a seventh normalization layer and a spectrum normalization layer which are sequentially arranged;
and connecting the fourth convolution layer with the fifth convolution layer to obtain the generative countermeasure network.
7. A license plate recognition device, the device comprising:
the first image acquisition module is used for acquiring an image to be recognized which is obtained by shooting a target license plate in a foggy environment;
the first image processing module is used for inputting the image to be recognized into a pre-trained generative confrontation network to obtain an edge gradient image corresponding to the image to be recognized;
the second image processing module is used for determining a target area image corresponding to the target license plate in the edge gradient image;
and the license plate recognition module is used for recognizing the characters of the target license plate in the target area image.
8. The apparatus of claim 7, further comprising:
and the model training module is used for training the generative countermeasure network by utilizing a plurality of foggy image samples containing license plates and edge gradient images which respectively correspond to the foggy image samples one by one.
9. The apparatus of claim 8, further comprising:
the second image acquisition module is used for acquiring a plurality of foggy image samples containing license plates and fog-free image samples which respectively correspond to the foggy image samples one by one;
and the third image processing module is used for carrying out image enhancement processing on the fog-free image sample corresponding to each foggy image sample to obtain a first edge gradient image corresponding to each foggy image sample.
10. The apparatus of claim 7, wherein the generative confrontation network comprises an image generation model and an image discrimination model, the apparatus further comprising:
the third image acquisition module is used for acquiring a plurality of foggy image samples containing license plates and second edge gradient images which respectively correspond to the foggy image samples one by one;
a model training module, configured to perform the following operations for each of the plurality of foggy image samples including the license plate:
inputting each foggy image sample into the image generation model to obtain a first edge gradient image corresponding to each foggy image sample;
calculating to obtain a weighted total loss value of each hazy image sample based on the first edge gradient image and the second edge gradient image of each hazy image sample;
and adjusting the network parameters of the image generation model and the network parameters of the image discrimination model according to the weighted total loss value.
11. The apparatus of claim 10, wherein the model training module is specifically configured to:
under the condition of keeping the network parameters of the image generation model unchanged, adjusting the network parameters of the image discrimination model to enable the weighted total loss value to reach the minimum value;
and adjusting the network parameters of the image generation model according to the truth judgment result of the first edge gradient image by the image judgment model until the judgment module reaches Nash balance.
12. The apparatus of claim 7, wherein the generative confrontation network comprises an image generation model and an image discrimination model, the apparatus further comprising:
the image generation module comprises a first model construction module, a second model construction module and a third model construction module, wherein the first model construction module is used for constructing the image generation model, and the image generation model comprises a first normalization layer, a first convolution layer, a second normalization layer, a second convolution layer, a third normalization layer, a third convolution layer, a fourth normalization layer and a fourth convolution layer which are sequentially arranged;
the second model building module is used for building the image discrimination model, and the image discrimination model comprises a fifth convolution layer, a sixth convolution layer, a fifth normalization layer, a seventh convolution layer, a sixth normalization layer, an eighth convolution layer, a seventh normalization layer and a spectrum normalization layer which are sequentially arranged;
and the model acquisition module is used for connecting the fourth convolution layer with the fifth convolution layer to obtain the generative confrontation network.
13. A license plate recognition apparatus, characterized in that the apparatus comprises:
a memory for storing a program;
a processor for executing the program stored in the memory to perform the license plate recognition method of any one of claims 1-6.
14. A computer storage medium having computer program instructions stored thereon, which when executed by a processor, implement the license plate recognition method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010372209.6A CN111639542A (en) | 2020-05-06 | 2020-05-06 | License plate recognition method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010372209.6A CN111639542A (en) | 2020-05-06 | 2020-05-06 | License plate recognition method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111639542A true CN111639542A (en) | 2020-09-08 |
Family
ID=72330955
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010372209.6A Pending CN111639542A (en) | 2020-05-06 | 2020-05-06 | License plate recognition method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111639542A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116110053A (en) * | 2023-04-13 | 2023-05-12 | 济宁能源发展集团有限公司 | Container surface information detection method based on image recognition |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107563385A (en) * | 2017-09-02 | 2018-01-09 | 西安电子科技大学 | License plate character recognition method based on depth convolution production confrontation network |
| CN109190621A (en) * | 2018-09-06 | 2019-01-11 | 东北大学 | Greasy weather automatic license plate recognition method |
| CN109509156A (en) * | 2018-10-31 | 2019-03-22 | 聚时科技(上海)有限公司 | A kind of image defogging processing method based on generation confrontation model |
| US20190095730A1 (en) * | 2017-09-25 | 2019-03-28 | Beijing University Of Posts And Telecommunications | End-To-End Lightweight Method And Apparatus For License Plate Recognition |
| CN109949242A (en) * | 2019-03-19 | 2019-06-28 | 内蒙古工业大学 | The generation method of image defogging model, device and image defogging method, device |
| CN110225350A (en) * | 2019-05-30 | 2019-09-10 | 西安电子科技大学 | Natural image compression method based on production confrontation network |
| CN110443166A (en) * | 2019-07-23 | 2019-11-12 | 淮阴工学院 | A kind of licence plate recognition method of haze weather |
-
2020
- 2020-05-06 CN CN202010372209.6A patent/CN111639542A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107563385A (en) * | 2017-09-02 | 2018-01-09 | 西安电子科技大学 | License plate character recognition method based on depth convolution production confrontation network |
| US20190095730A1 (en) * | 2017-09-25 | 2019-03-28 | Beijing University Of Posts And Telecommunications | End-To-End Lightweight Method And Apparatus For License Plate Recognition |
| CN109190621A (en) * | 2018-09-06 | 2019-01-11 | 东北大学 | Greasy weather automatic license plate recognition method |
| CN109509156A (en) * | 2018-10-31 | 2019-03-22 | 聚时科技(上海)有限公司 | A kind of image defogging processing method based on generation confrontation model |
| CN109949242A (en) * | 2019-03-19 | 2019-06-28 | 内蒙古工业大学 | The generation method of image defogging model, device and image defogging method, device |
| CN110225350A (en) * | 2019-05-30 | 2019-09-10 | 西安电子科技大学 | Natural image compression method based on production confrontation network |
| CN110443166A (en) * | 2019-07-23 | 2019-11-12 | 淮阴工学院 | A kind of licence plate recognition method of haze weather |
Non-Patent Citations (3)
| Title |
|---|
| HE ZHANG: "《Image De-raining Using a Conditional Generative》", 《ARXIV:1701.O5957V4[CS.CV]》 * |
| MINGHUI ZHANG: "《JOINT LICENSE PLATE SUPER-RESOLUTION AND RECOGNITION IN》", 《2018ICASSP》 * |
| 陈玮: "《基于生成对抗网络的图像去雾算法》", 《信息与控制》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116110053A (en) * | 2023-04-13 | 2023-05-12 | 济宁能源发展集团有限公司 | Container surface information detection method based on image recognition |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104700099B (en) | The method and apparatus for recognizing traffic sign | |
| CN107563433B (en) | Infrared small target detection method based on convolutional neural network | |
| CN111783551B (en) | Countermeasure sample defense method based on Bayesian convolutional neural network | |
| CN111754519B (en) | Class activation mapping-based countermeasure method | |
| CN110853074B (en) | Video target detection network system for enhancing targets by utilizing optical flow | |
| CN110765948A (en) | Target detection and identification method and system based on unmanned aerial vehicle | |
| CN109523013A (en) | A kind of air particle pollution level estimation method based on shallow-layer convolutional neural networks | |
| CN106600560A (en) | Image defogging method for automobile data recorder | |
| CN106680775A (en) | Method and system for automatically identifying radar signal modulation modes | |
| CN113591967A (en) | Image processing method, device and equipment and computer storage medium | |
| CN108229274B (en) | Method and device for training multilayer neural network model and recognizing road characteristics | |
| CN108805832B (en) | Improved gray projection image stabilizing method suitable for tunnel environment characteristics | |
| CN111507337A (en) | License plate recognition method based on hybrid neural network | |
| CN111340041A (en) | License plate recognition method and device based on deep learning | |
| CN112686835A (en) | Road obstacle detection device, method and computer-readable storage medium | |
| CN114757854B (en) | Night vision image quality improving method, device and equipment based on multispectral analysis | |
| CN117611494B (en) | Panchromatic remote sensing image thin cloud removing method | |
| CN116091462A (en) | Physical countermeasure attack method for unmanned aerial vehicle target detection system | |
| CN113435407A (en) | Small target identification method and device for power transmission system | |
| CN113780110A (en) | Method and device for detecting weak and small targets in image sequence in real time | |
| CN111539250B (en) | Image fog concentration estimation method, system and terminal based on neural network | |
| CN115272876A (en) | Remote sensing image ship target detection method based on deep learning | |
| CN113989612A (en) | Remote sensing image target detection method based on attention and generation countermeasure network | |
| CN111598793A (en) | Method and system for defogging image of power transmission line and storage medium | |
| CN115100068A (en) | Infrared image correction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200908 |