CN111046891A - Training method of license plate recognition model, and license plate recognition method and device - Google Patents
Training method of license plate recognition model, and license plate recognition method and device Download PDFInfo
- Publication number
- CN111046891A CN111046891A CN201811184621.4A CN201811184621A CN111046891A CN 111046891 A CN111046891 A CN 111046891A CN 201811184621 A CN201811184621 A CN 201811184621A CN 111046891 A CN111046891 A CN 111046891A
- Authority
- CN
- China
- Prior art keywords
- license plate
- sample
- training
- recognition
- recognition model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 151
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000001514 detection method Methods 0.000 claims description 18
- 238000012937 correction Methods 0.000 claims description 17
- 230000009466 transformation Effects 0.000 claims description 15
- 238000000605 extraction Methods 0.000 claims description 7
- 238000010276 construction Methods 0.000 claims description 5
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 11
- 230000003321 amplification Effects 0.000 description 6
- 238000003199 nucleic acid amplification method Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 3
- 238000002360 preparation method Methods 0.000 description 3
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000013145 classification model Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/245—Aligning, centring, orientation detection or correction of the image by locating a pattern; Special marks for positioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/146—Aligning or centring of the image pick-up or image-field
- G06V30/1475—Inclination or skew detection or correction of characters or of image to be recognised
- G06V30/1478—Inclination or skew detection or correction of characters or of image to be recognised of characters or characters lines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Character Discrimination (AREA)
- Traffic Control Systems (AREA)
Abstract
The application discloses a training method of a license plate recognition model, a license plate recognition method and a license plate recognition device. The training method of the license plate recognition model provided by the application comprises the following steps: constructing a training sample set; the training sample set comprises a plurality of sample subsets, and license plate types of the license plate samples in each sample subset are the same; the license plate sample comprises a pre-collected original license plate sample and an amplified license plate sample obtained by amplifying the original license plate sample; and selecting a target number of license plate samples from each sample subset, and training a pre-constructed license plate recognition model by using the selected license plate samples until the license plate recognition model is converged. The training method of the license plate recognition model, the license plate recognition method and the device can improve the generalization procedure of the trained license plate recognition model and improve the recognition accuracy rate of the trained license plate recognition model.
Description
Technical Field
The application relates to the field of image recognition, in particular to a license plate recognition model training method, a license plate recognition method and a license plate recognition device.
Background
The license plate recognition is an important component of an intelligent traffic system, and plays an important role in vehicle management, vehicle monitoring, traffic flow monitoring, traffic control, stolen vehicle and special vehicle discrimination and the like. At present, a license plate recognition model which is trained in advance is often adopted to recognize the license plate.
The related technology discloses a training method of a license plate recognition model, which comprises the following steps: constructing a license plate recognition model; constructing a training sample set; and training the license plate recognition model by using the training sample set to obtain the trained license plate recognition model. When the license plate recognition model is trained by using the training sample set, randomly selecting a specified number of license plate samples from the training sample set as a sub-training sample set for the training of the round in the process of each round of iterative training.
When the license plate recognition model is trained by adopting the method, the sub-training sample set is randomly selected from the training samples in the process of each round of iterative training. Thus, for a certain type of license plate type, if the number of license plate samples belonging to the type of license plate type is small, in the process of one iterative training, the sub-training sample set may not contain the license plate samples belonging to the type of license plate, that is, in the training process, the license plate samples of the type of license plate are not trained. Therefore, the generalization degree of the trained license plate recognition model is low, and the recognition accuracy of the trained license plate recognition model to the license plates of the license plate types is low.
Disclosure of Invention
The application provides a training method of a license plate recognition model, a license plate recognition method and a device, and aims to solve the problems that a license plate recognition model trained by the existing training method is low in generalization program and low in recognition accuracy rate of license plates of certain license plate types.
In a first aspect, the present application provides a method for training a license plate recognition model, where the method includes:
constructing a training sample set; the training sample set comprises a plurality of sample subsets, and license plate types of the license plate samples in each sample subset are the same; the license plate sample comprises a pre-collected original license plate sample and an amplified license plate sample obtained by amplifying the original license plate sample;
and selecting a target number of license plate samples from each sample subset, and training a pre-constructed license plate recognition model by using the selected license plate samples until the license plate recognition model is converged.
The second aspect of the application provides a training device for a license plate recognition model, which comprises a construction module and a training module; wherein the content of the first and second substances,
the construction module is used for constructing a training sample set; the training sample set comprises a plurality of sample subsets, and license plate types of the license plate samples in each sample subset are the same; the license plate sample comprises a pre-collected original license plate sample and an amplified license plate sample obtained by amplifying the original license plate sample;
the training module is used for selecting a target number of license plate samples from each sample subset, and training a pre-constructed license plate recognition model by using the selected license plate samples until the license plate recognition model converges.
A third aspect of the present application provides a license plate recognition method, including:
inputting an image to be recognized into a trained license plate recognition model, wherein the license plate recognition model is obtained by adopting license plate sample training containing an amplified license plate sample, the license plate recognition model comprises a license plate area detection network, an inclination correction network and a recognition network, a target area where a license plate is located is positioned by the license plate recognition model through the license plate area detection network, the inclination correction network is used for carrying out space transformation on the target area to obtain a transformed target area, the transformed target area is subjected to feature extraction through the recognition network to obtain a first feature map, and a recognition result is output after the first feature map is recognized;
and acquiring a recognition result output by the license plate recognition model.
A fourth aspect of the present application provides a license plate recognition device, comprising a recognition module and an acquisition module, wherein,
the recognition module is used for inputting an image to be recognized into a trained license plate recognition model, the license plate recognition model is obtained by adopting a license plate sample training containing an amplified license plate sample, the license plate recognition model comprises a license plate region detection network, an inclination correction network and a recognition network, a target region where a license plate is located by the license plate recognition model through the license plate region detection network, the target region is subjected to space transformation through the inclination correction network to obtain a transformed target region, the transformed target region is subjected to feature extraction through the recognition network to obtain a first feature map, and a recognition result is output after the first feature map is recognized;
the acquisition module is used for acquiring the recognition result output by the license plate recognition model.
According to the training method and device for the license plate recognition model, a training sample set is constructed, wherein the training sample set comprises a plurality of sample subsets, and license plate types of all license plate samples in each sample subset are the same; the license plate sample comprises a pre-collected original license plate sample and an amplified license plate sample obtained by amplifying the original license plate sample; and then selecting a target number of license plate samples from each sample subset, and training a pre-constructed license plate recognition model by using the selected license plate samples until the license plate recognition model is converged. Therefore, the sub-training sample set used in each round of training can be guaranteed to contain the license plate sample of each type of vehicle, and each type of license plate can be further guaranteed to be trained in each round of training. Therefore, the generalization degree of the trained license plate recognition model can be improved, and the recognition preparation rate of the trained license plate recognition model on license plates of various vehicle types is improved.
Drawings
FIG. 1 is a schematic view of a license plate shown in an exemplary embodiment of the present application;
FIG. 2 is a flowchart of a first embodiment of a license plate recognition model training method provided in the present application;
FIG. 3 is a schematic diagram of an augmented license plate sample obtained by character transformation according to an exemplary embodiment of the present application;
fig. 4 is a schematic structural diagram of a license plate recognition model according to an exemplary embodiment of the present application;
FIG. 5 is a flowchart illustrating an implementation of selecting a target number of license plate samples from each sample subset according to an exemplary embodiment of the present application;
fig. 6 is a flowchart of a first embodiment of a license plate recognition method provided in the present application;
fig. 7 is a schematic diagram illustrating an implementation of a license plate recognition model for recognizing an image to be recognized according to an exemplary embodiment of the present application;
FIG. 8 is a schematic structural diagram of a first embodiment of a license plate recognition model training device provided in the present application;
fig. 9 is a schematic structural diagram of a license plate recognition device according to a first embodiment of the present disclosure.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
License plates, as "identification cards" for vehicles, are commonly used to identify motor vehicles. At present, a license plate recognition technology is widely applied to scenes such as a gate, a parking lot and intelligent traffic to acquire a license plate of a motor vehicle in the scene and manage the motor vehicle.
Most license plates consist of numbers and letters, but the license plates of some countries contain special characters besides the numbers and the letters, and although the positions of the special characters appearing in the license plates are not fixed, the license plates have fixed modes. Fig. 1 is a schematic diagram of a license plate according to an exemplary embodiment of the present disclosure. Referring to fig. 1, chinese license plates are classified into two license plate types (a type and B type in fig. 1), korean license plates are classified into five license plate types, and thailand license plates are classified into three license plate types, respectively, and the like. It should be noted that the license plate type refers to a type mark of a country or a region to which the license plate belongs and a subclass to which the license plate belongs.
At present, a license plate recognition model which is trained in advance is often adopted to recognize the license plate. The related technology discloses a training method of a license plate recognition model, which comprises the following steps: constructing a license plate recognition model; constructing a training sample set; and training the license plate recognition model by using the training sample set to obtain the trained license plate recognition model. When the license plate recognition model is trained by using the training sample set, randomly selecting a specified number of license plate samples from the training sample set as a sub-training sample set for the training of the round in the process of each round of iterative training.
When the license plate recognition model is trained by adopting the method, the sub-training sample set is randomly selected from the training samples in the process of each round of iterative training. Thus, for a certain type of license plate type, if the number of license plate samples belonging to the type of license plate is small, in the process of one iterative training, the sub-training sample set may not contain the license plate sample belonging to the type of license plate, that is, in the training process, the license plate sample of the type of license plate is not trained. Therefore, the generalization degree of the trained license plate recognition model is low, and the recognition accuracy of the trained license plate recognition model to the license plates of the license plate types is low.
The application provides a training method of a license plate recognition model, a license plate recognition method and a device, and aims to solve the problems that a license plate recognition model trained by the existing training method is low in generalization program and low in recognition accuracy rate of license plates of certain license plate types.
Several specific embodiments are given below to describe the technical solutions of the present application in detail, and these specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.
Fig. 2 is a flowchart of a first embodiment of a license plate recognition model training method provided in the present application. Referring to fig. 2, the method provided in the present application may include:
s201, constructing a training sample set; the training sample set comprises a plurality of sample subsets, and license plate types of the license plate samples in each sample subset are the same; the license plate sample comprises a pre-collected original license plate sample and an amplified license plate sample obtained by amplifying the original license plate sample.
Further, the specific implementation process of this step may include: collecting an original license plate sample; according to the license plate type marked on the original license plate sample in advance; classifying and storing each original sample according to the license plate type to obtain a plurality of sample subsets; counting the number of license plate samples contained in each sample subset, and performing sample amplification on the license plate samples contained in the sample subsets when the number of the license plate samples contained in the sample subsets is smaller than a preset threshold value, so that the amplified license plate samples obtained after sample amplification are stored in the sample subsets. Finally, all sample subsets are determined as a constructed training sample set.
Optionally, in a possible implementation manner of the present application, the amplified license plate sample is obtained by at least one of the following license plate sample amplification methods: affine transformation, perturbation clipping, and character exchange.
It should be noted that, for a specific implementation principle and implementation method of the radial transformation, the perturbation clipping and the character exchange, reference may be made to the description in the related art, and details are not described herein again. For example, fig. 3 is a schematic diagram of an augmented license plate sample obtained by character transformation according to an exemplary embodiment of the present application. Referring to fig. 3, for a license plate type with a few license plate samples, sample amplification can be performed on an original license plate sample through character transformation to obtain an amplified license plate sample, so as to increase the number of license plate samples of the license plate type.
Thus, when the number of license plate samples belonging to a certain vehicle class is small, the number of license plate samples belonging to the vehicle class can be increased by sample amplification. Therefore, after the license plate recognition model is trained by adopting the training sample set, the generalization degree of the trained license plate recognition model can be improved and the recognition accuracy rate of the trained license plate recognition model can be improved due to the fact that the training samples are sufficient.
S202, selecting a target number of license plate samples from each sample subset, and training a pre-constructed license plate recognition model by using the selected license plate samples until the license plate recognition model is converged.
It should be noted that fig. 4 is a schematic structural diagram of a license plate recognition model according to an exemplary embodiment of the present application. Referring to fig. 4, the license plate recognition model provided in the present application is composed of three cascaded neural networks, which are a license plate area detection network, a tilt correction network, and a recognition network, respectively, wherein the license plate area detection network is configured to locate a target area where a license plate is located from an image to be recognized; the tilt correction network is configured to perform spatial transformation on the target region; the identification network can be used for identifying the license plate according to actual needs and outputting the text content of the license plate and/or the type of the license plate to which the license plate belongs. In the present embodiment, this is not limited. I.e. the identification network comprises at least one of the following networks: a character recognition network and a license plate type recognition network. The character recognition network is used for recognizing the license plate and outputting the text content of the license plate; and the license plate type identification network is used for outputting the license plate type of the license plate.
Specifically, fig. 5 is a flowchart illustrating an implementation of selecting a target number of license plate samples from each sample subset according to an exemplary embodiment of the present application. Referring to fig. 5, selecting a target number of license plate samples from each sample subset may include:
s501, calculating the proportion of the license plate samples of the license plate types in the training sample set aiming at each license plate type.
For example, in one embodiment, the training sample set includes 8 sample subsets, and the license plate types of the license plate samples in each sample subset are the same. In this step, the number of license plate samples included in each sample subset can be counted, so as to obtain the total number of license plate samples included in the training sample set, thereby obtaining the proportion of the license plate samples of each type of license plate in the training sample set. For example, table 1 shows a ratio of license plate samples of each type of license plate in a training sample set obtained by calculation according to an exemplary embodiment, where mi is a number of license plate samples included in an ith sample subset (corresponding to the ith type of license plate); and bi is the proportion of the license plate samples of the ith type of license plates in the training sample set obtained by calculation.
TABLE 1 proportion of license plate samples of each type of license plate type in training sample set
| |
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
| mi | 1000 | 200 | 3000 | 4000 | 100 | 450 | 100 | 900 |
| bi | 0.1 | 0.02 | 0.31 | 0.41 | 0.01 | 0.05 | 0.01 | 0.09 |
S502, determining the target number of the license plate samples of the license plate type in the sub-training sample set for the training according to the number of the license plate samples required to be contained in the preset sub-training sample set for the training of the current round and the calculated proportion of the license plate samples of the license plate type in the training sample set.
Specifically, the number of license plate samples to be included in the preset sub-training sample set for the current round of training is set according to actual needs, and in this embodiment, this is not limited thereto, for example, in an embodiment, the number of license plate samples to be included in the preset sub-training sample set for the current round of training is 100.
Further, the target number of the license plate samples of the license plate types in the sub-training sample set for the training is determined according to the following formula:
Ai＝[x*bi]
wherein x is the number of license plate samples required to be contained in a preset sub-training sample set for the training of the current round; a is the target number of license plate samples of the ith type of license plate in the sub-training sample set for the training of the round.
With reference to the above example, for the license plate category 1, it is determined that the target number of the license plate samples of the license plate type in the sub-training sample set for training in the current round is 10. And aiming at the license plate types from 2 to 8, determining that the number of targets of the license plate samples of the license plate types in the sub-training sample set for training in the round is respectively 2, 31, 41, 1, 5, 1 and 9.
S503, selecting the target number of license plate samples from the sample subsets corresponding to the license plate types.
With reference to the above example, 10 license plate samples are selected from the sample subset 1, 2, 31, 41, 1, 5, 1, and 9 license plate samples are selected from the sample subsets 2 to 8, and a total of 100 license plate samples are selected, where the 100 license plate samples constitute the sub-training sample set.
It should be noted that, in the method provided in this embodiment, when the sub-training samples for training in the current round are selected from the training sample set, the sub-training samples are not randomly selected, but a target number of license plate samples are selected from each sample subset. Therefore, the license plate samples of each type of license plate can be contained in the sub-training samples, and each type of license plate can be trained in each round of training. Therefore, the generalization degree of the trained license plate recognition model can be improved, and the recognition preparation rate of the trained license plate recognition model on license plates of various vehicle types is improved.
It should be noted that the convergence of the license plate recognition model may be that the number of iterations reaches a preset number or that the loss is smaller than a preset threshold. In the present embodiment, this is not limited.
Optionally, in a possible implementation manner of the present application, the loss function of the license plate recognition model that is trained and constructed in advance by using the selected license plate sample is as follows:
FL(pt)＝-ai(1-pt)rlog(pt)
wherein, the FL (p)t) The error corresponding to the ith license plate sample is obtained;
said p istThe predicted value of the ith license plate sample is obtained;
a is aiThe loss weight corresponding to the license plate type to which the ith license plate sample belongs; the number of the license plate samples of the license plate type contained in the training sample set is larger, and the loss weight corresponding to the license plate type is smaller;
the r is a specified value.
Specifically, the specified value is set according to actual needs, and in this embodiment, this is not limited, for example, in one embodiment, the specified value is 2.
It should be noted that, the loss weight corresponding to the license plate type to which the ith license plate sample belongs is preset. For each type of license plate type, if the number of the license plate samples of the license plate type contained in the training sample set is larger, the loss weight corresponding to the license plate type is smaller. That is, if the number of license plate samples of the license plate type contained in the training sample set is larger, a smaller loss weight is given to the license plate type; and if the number of the license plate samples of the license plate type contained in the training sample set is smaller, giving a larger loss weight to the license plate type. Therefore, when the license plate samples of all license plate types are unbalanced, the method can make the license plate samples of all license plate types contribute to the due loss in the training process, and the problem that the license plate samples of a certain type are covered by the license plate samples of other types when the license plate samples of the certain type are few is avoided. Therefore, the generalization degree of the trained license plate recognition model can be improved, and the recognition accuracy rate of the trained license plate recognition model is improved.
According to the training method of the license plate recognition model, the generalization program of the trained license plate recognition model can be improved by changing the loss function, and the recognition accuracy of the trained license plate recognition model is improved.
In the method provided by the embodiment, a training sample set is constructed, wherein the training sample set comprises a plurality of sample subsets, and license plate types of the license plate samples in each sample subset are the same; the license plate sample comprises a pre-collected original license plate sample and an amplified license plate sample obtained by amplifying the original license plate sample; and then selecting a target number of license plate samples from each sample subset, and training a pre-constructed license plate recognition model by using the selected license plate samples until the license plate recognition model is converged. Therefore, the sub-training sample set used in each round of training can be guaranteed to contain the license plate sample of each type of vehicle, and each type of license plate can be further guaranteed to be trained in each round of training. Therefore, the generalization degree of the trained license plate recognition model can be improved, and the recognition preparation rate of the trained license plate recognition model on license plates of various vehicle types is improved.
The above introduces the training method of the license plate recognition model provided by the present application, and the following introduces the license plate recognition method provided by the present application:
fig. 6 is a flowchart of a first embodiment of a license plate recognition method provided in the present application. Referring to fig. 6, the license plate recognition method provided by the present application may include:
s601, inputting an image to be recognized into a trained license plate recognition model, wherein the license plate recognition model is obtained by adopting license plate sample training containing an amplified license plate sample, the license plate recognition model comprises a license plate area detection network, an inclination correction network and a recognition network, a target area where a license plate is located through the license plate area detection network by the license plate recognition model, space transformation is carried out on the target area through the inclination correction network to obtain a transformed target area, feature extraction is carried out on the transformed target area through the recognition network to obtain a first feature map, and a recognition result is output after the first feature map is recognized.
And S602, obtaining the recognition result output by the license plate recognition model.
Specifically, fig. 7 is an implementation schematic diagram of a license plate recognition model for recognizing an image to be recognized according to an exemplary embodiment of the present application. Referring to fig. 6 and 7, the step of recognizing the image to be recognized by the license plate recognition model may include: positioning a target area where a license plate is located from an image to be recognized; carrying out space transformation on the target area to obtain a transformed target area; and performing feature extraction on the transformed target region to obtain a first feature map, and outputting an identification result after identifying the first feature map.
The license plate area detection network may be an FR-CNN network or a YOLO network. The specific structure and implementation principle of the license plate area detection network may be described in the related art, and are not described herein again.
Further, the tilt correction network may be an STN network, and for the specific structure and implementation principle of the network, reference may be made to the description in the related art, and details are not described here.
Further, in a possible implementation manner of the present application, the recognizing network may include a character recognizing network, and the outputting the recognition result after recognizing the first feature map includes:
and serializing the first feature graph to obtain a feature sequence, coding the feature sequence to obtain a coding result, and decoding the coding result to output the identified license plate number.
Specifically, the character recognition network may be an attention model, and the specific implementation principle of recognizing the first feature map by using the attention model may refer to description in related art, which is not described herein again.
In addition, in another possible implementation manner of the present application, the recognition network may include a license plate type recognition network, and the outputting a recognition result after recognizing the first feature map includes:
and after classifying the first characteristic graph, outputting the probability that the license plate in the image to be recognized belongs to each preset license plate type, and determining the preset license plate type corresponding to the maximum probability as the license plate type to which the license plate in the image to be recognized belongs.
Specifically, the license plate type recognition network may be a classification model, and a specific implementation principle of recognizing the first feature map by using the classification model may refer to description in related art, which is not described herein again.
According to the license plate recognition method provided by the embodiment, the license plate recognition model is obtained by adopting the license plate sample training containing the amplified license plate sample, so that the recognition accuracy can be improved when the license plate recognition model is used for recognition.
The following introduces a license plate recognition model training device and a license plate recognition device provided by the application:
fig. 8 is a schematic structural diagram of a first embodiment of a training device for a license plate recognition model provided in the present application. Referring to fig. 8, the apparatus provided in this embodiment may include a construction module 810 and a training module 820; the constructing module 810 is configured to construct a training sample set; the training sample set comprises a plurality of sample subsets, and license plate types of the license plate samples in each sample subset are the same; the license plate sample comprises a pre-collected original license plate sample and an amplified license plate sample obtained by amplifying the original license plate sample;
the training module 820 is configured to select a target number of license plate samples from each sample subset, and train a pre-constructed license plate recognition model using the selected license plate samples until the license plate recognition model converges.
Further, the training module 820 is specifically configured to:
calculating the proportion of the license plate samples of the license plate types in the training sample set aiming at each license plate type;
determining the target number of the license plate samples of the license plate type in the sub-training sample set for the training of the round according to the number of the license plate samples required to be contained in the preset sub-training sample set for the training of the round and the calculated proportion of the license plate samples of the license plate type in the training sample set;
and selecting the target number of license plate samples from the sample subset corresponding to the type of license plate.
Further, the amplified license plate sample is obtained by at least one of the following license plate sample amplification methods: affine transformation, perturbation clipping, and character exchange.
Further, the license plate recognition model comprises a license plate region detection network, a tilt correction network and a recognition network; the identification network comprises at least one of: a character recognition network and a license plate type recognition network.
Fig. 9 is a schematic structural diagram of a license plate recognition device according to a first embodiment of the present disclosure. Referring to fig. 9, the present application provides a license plate recognition apparatus, including a recognition module 910 and an obtaining module 920, wherein,
the recognition module 910 is configured to input an image to be recognized into a trained license plate recognition model, where the license plate recognition model is obtained by training a license plate sample including an amplified license plate sample, the license plate recognition model includes a license plate region detection network, a tilt correction network and a recognition network, and the license plate recognition model locates a target region where a license plate is located through the license plate region detection network, performs spatial transformation on the target region through the tilt correction network to obtain a transformed target region, performs feature extraction on the transformed target region through the recognition network to obtain a first feature map, and outputs a recognition result after recognizing the first feature map;
the obtaining module 920 is configured to obtain a recognition result output by the license plate recognition model.
Further, the recognizing network includes a character recognizing network, and the outputting a recognition result after recognizing the first feature map includes:
and serializing the first feature graph to obtain a feature sequence, coding the feature sequence to obtain a coding result, and decoding the coding result to output the identified license plate number.
Further, the recognizing network includes a license plate type recognizing network, and the outputting a recognition result after recognizing the first feature map includes:
and after classifying the first characteristic graph, outputting the probability that the license plate in the image to be recognized belongs to each preset license plate type, and determining the preset license plate type corresponding to the maximum probability as the license plate type to which the license plate in the image to be recognized belongs.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the scope of protection of the present application.
Claims (10)
1. A training method of a license plate recognition model is characterized by comprising the following steps:
constructing a training sample set; the training sample set comprises a plurality of sample subsets, and license plate types of the license plate samples in each sample subset are the same; the license plate sample comprises a pre-collected original license plate sample and an amplified license plate sample obtained by amplifying the original license plate sample;
and selecting a target number of license plate samples from each sample subset, and training a pre-constructed license plate recognition model by using the selected license plate samples until the license plate recognition model is converged.
2. The method of claim 1, wherein selecting a target number of license plate samples from each sample subset comprises:
calculating the proportion of the license plate samples of the license plate types in the training sample set aiming at each license plate type;
determining the target number of the license plate samples of the license plate type in the sub-training sample set for the training of the round according to the number of the license plate samples required to be contained in the preset sub-training sample set for the training of the round and the calculated proportion of the license plate samples of the license plate type in the training sample set;
and selecting the target number of license plate samples from the sample subset corresponding to the type of license plate.
3. The method of claim 1, wherein the amplified license plate sample is obtained by at least one of the following methods for amplifying a license plate sample: affine transformation, perturbation clipping, and character exchange.
4. The method of claim 1, wherein the license plate recognition model comprises a license plate region detection network, a tilt correction network, and a recognition network; the identification network comprises at least one of: a character recognition network and a license plate type recognition network.
5. A license plate recognition method is characterized by comprising the following steps:
inputting an image to be recognized into a trained license plate recognition model, wherein the license plate recognition model is obtained by adopting license plate sample training containing an amplified license plate sample, the license plate recognition model comprises a license plate area detection network, an inclination correction network and a recognition network, a target area where a license plate is located is positioned by the license plate recognition model through the license plate area detection network, the inclination correction network is used for carrying out space transformation on the target area to obtain a transformed target area, the transformed target area is subjected to feature extraction through the recognition network to obtain a first feature map, and a recognition result is output after the first feature map is recognized;
and acquiring a recognition result output by the license plate recognition model.
6. The method according to claim 5, wherein the recognition network comprises a character recognition network, and the outputting the recognition result after recognizing the first feature map comprises:
and serializing the first feature graph to obtain a feature sequence, coding the feature sequence to obtain a coding result, and decoding the coding result to output the identified license plate number.
7. The method according to claim 5 or 6, wherein the recognition network comprises a license plate type recognition network, and the outputting a recognition result after the recognizing the first feature map comprises:
and after classifying the first characteristic graph, outputting the probability that the license plate in the image to be recognized belongs to each preset license plate type, and determining the preset license plate type corresponding to the maximum probability as the license plate type to which the license plate in the image to be recognized belongs.
8. A training device for a license plate recognition model is characterized by comprising a construction module and a training module, wherein,
the construction module is used for constructing a training sample set; the training sample set comprises a plurality of sample subsets, and license plate types of the license plate samples in each sample subset are the same; the license plate sample comprises a pre-collected original license plate sample and an amplified license plate sample obtained by amplifying the original license plate sample;
the training module is used for selecting a target number of license plate samples from each sample subset, and training a pre-constructed license plate recognition model by using the selected license plate samples until the license plate recognition model converges.
9. A license plate recognition device is characterized by comprising a recognition module and an acquisition module, wherein,
the recognition module is used for inputting an image to be recognized into a trained license plate recognition model, the license plate recognition model is obtained by adopting a license plate sample training containing an amplified license plate sample, the license plate recognition model comprises a license plate region detection network, an inclination correction network and a recognition network, a target region where a license plate is located by the license plate recognition model through the license plate region detection network, the target region is subjected to space transformation through the inclination correction network to obtain a transformed target region, the transformed target region is subjected to feature extraction through the recognition network to obtain a first feature map, and a recognition result is output after the first feature map is recognized;
the acquisition module is used for acquiring the recognition result output by the license plate recognition model.
10. The apparatus according to claim 9, wherein the recognition network comprises a character recognition network, and the outputting the recognition result after recognizing the first feature map comprises:
and serializing the first feature graph to obtain a feature sequence, coding the feature sequence to obtain a coding result, and decoding the coding result to output the identified license plate number.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811184621.4A CN111046891A (en) | 2018-10-11 | 2018-10-11 | Training method of license plate recognition model, and license plate recognition method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811184621.4A CN111046891A (en) | 2018-10-11 | 2018-10-11 | Training method of license plate recognition model, and license plate recognition method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111046891A true CN111046891A (en) | 2020-04-21 |
Family
ID=70229197
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811184621.4A Pending CN111046891A (en) | 2018-10-11 | 2018-10-11 | Training method of license plate recognition model, and license plate recognition method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111046891A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112580643A (en) * | 2020-12-09 | 2021-03-30 | 浙江智慧视频安防创新中心有限公司 | License plate recognition method and device based on deep learning and storage medium |
Citations (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1419214A (en) * | 2002-12-23 | 2003-05-21 | 北京邮电大学 | Automatic training pattern selector for character recognition and method thereof |
| US20130097103A1 (en) * | 2011-10-14 | 2013-04-18 | International Business Machines Corporation | Techniques for Generating Balanced and Class-Independent Training Data From Unlabeled Data Set |
| CN104462301A (en) * | 2014-11-28 | 2015-03-25 | 北京奇虎科技有限公司 | Network data processing method and device |
| CN104616029A (en) * | 2014-12-29 | 2015-05-13 | 小米科技有限责任公司 | Data classification method and device |
| CN105373606A (en) * | 2015-11-11 | 2016-03-02 | 重庆邮电大学 | Unbalanced data sampling method in improved C4.5 decision tree algorithm |
| US9418305B1 (en) * | 2015-04-29 | 2016-08-16 | Xerox Corporation | Segmentation free approach to automatic license plate recognition |
| CN106295541A (en) * | 2016-08-03 | 2017-01-04 | 乐视控股（北京）有限公司 | Vehicle type recognition method and system |
| CN106529598A (en) * | 2016-11-11 | 2017-03-22 | 北京工业大学 | Classification method and system based on imbalanced medical image data set |
| KR20170034258A (en) * | 2015-09-18 | 2017-03-28 | 삼성전자주식회사 | Model training method and apparatus, and data recognizing method |
| CN106548196A (en) * | 2016-10-20 | 2017-03-29 | 中国科学院深圳先进技术研究院 | A kind of random forest sampling approach and device for non-equilibrium data |
| CN106548210A (en) * | 2016-10-31 | 2017-03-29 | 腾讯科技（深圳）有限公司 | Machine learning model training method and device |
| CN106778583A (en) * | 2016-12-07 | 2017-05-31 | 北京理工大学 | Vehicle attribute recognition methods and device based on convolutional neural networks |
| CN106874907A (en) * | 2017-01-19 | 2017-06-20 | 博康智能信息技术有限公司北京海淀分公司 | A kind of method and device for setting up Car license recognition model |
| US20170323202A1 (en) * | 2016-05-06 | 2017-11-09 | Fujitsu Limited | Recognition apparatus based on deep neural network, training apparatus and methods thereof |
| CN107506759A (en) * | 2016-06-14 | 2017-12-22 | 杭州海康威视数字技术股份有限公司 | A kind of motor vehicle brand identification method and device |
| CN107563385A (en) * | 2017-09-02 | 2018-01-09 | 西安电子科技大学 | License plate character recognition method based on depth convolution production confrontation network |
| CN107590500A (en) * | 2017-07-20 | 2018-01-16 | 济南中维世纪科技有限公司 | A kind of color recognizing for vehicle id method and device based on color projection classification |
| WO2018090771A1 (en) * | 2016-11-16 | 2018-05-24 | 杭州海康威视数字技术股份有限公司 | Vehicle license plate recognition method and apparatus |
| CN108171175A (en) * | 2017-12-29 | 2018-06-15 | 苏州科达科技股份有限公司 | A kind of deep learning sample enhancing system and its operation method |
| CN108171280A (en) * | 2018-01-31 | 2018-06-15 | 国信优易数据有限公司 | A kind of grader construction method and the method for prediction classification |
| CN108229676A (en) * | 2017-05-18 | 2018-06-29 | 北京市商汤科技开发有限公司 | The training method and device of neural network model, electronic equipment and computer storage media |
| CN108389201A (en) * | 2018-03-16 | 2018-08-10 | 北京推想科技有限公司 | The good pernicious sorting technique of Lung neoplasm based on 3D convolutional neural networks and deep learning |
| CN108460328A (en) * | 2018-01-15 | 2018-08-28 | 浙江工业大学 | A kind of fake-licensed car detection method based on multitask convolutional neural networks |
| CN108460421A (en) * | 2018-03-13 | 2018-08-28 | 中南大学 | The sorting technique of unbalanced data |
| CN108520220A (en) * | 2018-03-30 | 2018-09-11 | 百度在线网络技术（北京）有限公司 | model generating method and device |
| CN108537223A (en) * | 2018-04-16 | 2018-09-14 | 广东工业大学 | A kind of detection method of license plate, system and equipment and storage medium |
| CN108596199A (en) * | 2017-12-29 | 2018-09-28 | 北京交通大学 | Unbalanced data classification method based on EasyEnsemble algorithms and SMOTE algorithms |
| CN108628971A (en) * | 2018-04-24 | 2018-10-09 | 深圳前海微众银行股份有限公司 | File classification method, text classifier and the storage medium of imbalanced data sets |
-
2018
- 2018-10-11 CN CN201811184621.4A patent/CN111046891A/en active Pending
Patent Citations (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1419214A (en) * | 2002-12-23 | 2003-05-21 | 北京邮电大学 | Automatic training pattern selector for character recognition and method thereof |
| US20130097103A1 (en) * | 2011-10-14 | 2013-04-18 | International Business Machines Corporation | Techniques for Generating Balanced and Class-Independent Training Data From Unlabeled Data Set |
| CN104462301A (en) * | 2014-11-28 | 2015-03-25 | 北京奇虎科技有限公司 | Network data processing method and device |
| CN104616029A (en) * | 2014-12-29 | 2015-05-13 | 小米科技有限责任公司 | Data classification method and device |
| US9418305B1 (en) * | 2015-04-29 | 2016-08-16 | Xerox Corporation | Segmentation free approach to automatic license plate recognition |
| KR20170034258A (en) * | 2015-09-18 | 2017-03-28 | 삼성전자주식회사 | Model training method and apparatus, and data recognizing method |
| CN105373606A (en) * | 2015-11-11 | 2016-03-02 | 重庆邮电大学 | Unbalanced data sampling method in improved C4.5 decision tree algorithm |
| US20170323202A1 (en) * | 2016-05-06 | 2017-11-09 | Fujitsu Limited | Recognition apparatus based on deep neural network, training apparatus and methods thereof |
| CN107506759A (en) * | 2016-06-14 | 2017-12-22 | 杭州海康威视数字技术股份有限公司 | A kind of motor vehicle brand identification method and device |
| CN106295541A (en) * | 2016-08-03 | 2017-01-04 | 乐视控股（北京）有限公司 | Vehicle type recognition method and system |
| CN106548196A (en) * | 2016-10-20 | 2017-03-29 | 中国科学院深圳先进技术研究院 | A kind of random forest sampling approach and device for non-equilibrium data |
| CN106548210A (en) * | 2016-10-31 | 2017-03-29 | 腾讯科技（深圳）有限公司 | Machine learning model training method and device |
| CN106529598A (en) * | 2016-11-11 | 2017-03-22 | 北京工业大学 | Classification method and system based on imbalanced medical image data set |
| WO2018090771A1 (en) * | 2016-11-16 | 2018-05-24 | 杭州海康威视数字技术股份有限公司 | Vehicle license plate recognition method and apparatus |
| CN106778583A (en) * | 2016-12-07 | 2017-05-31 | 北京理工大学 | Vehicle attribute recognition methods and device based on convolutional neural networks |
| CN106874907A (en) * | 2017-01-19 | 2017-06-20 | 博康智能信息技术有限公司北京海淀分公司 | A kind of method and device for setting up Car license recognition model |
| CN108229676A (en) * | 2017-05-18 | 2018-06-29 | 北京市商汤科技开发有限公司 | The training method and device of neural network model, electronic equipment and computer storage media |
| CN107590500A (en) * | 2017-07-20 | 2018-01-16 | 济南中维世纪科技有限公司 | A kind of color recognizing for vehicle id method and device based on color projection classification |
| CN107563385A (en) * | 2017-09-02 | 2018-01-09 | 西安电子科技大学 | License plate character recognition method based on depth convolution production confrontation network |
| CN108171175A (en) * | 2017-12-29 | 2018-06-15 | 苏州科达科技股份有限公司 | A kind of deep learning sample enhancing system and its operation method |
| CN108596199A (en) * | 2017-12-29 | 2018-09-28 | 北京交通大学 | Unbalanced data classification method based on EasyEnsemble algorithms and SMOTE algorithms |
| CN108460328A (en) * | 2018-01-15 | 2018-08-28 | 浙江工业大学 | A kind of fake-licensed car detection method based on multitask convolutional neural networks |
| CN108171280A (en) * | 2018-01-31 | 2018-06-15 | 国信优易数据有限公司 | A kind of grader construction method and the method for prediction classification |
| CN108460421A (en) * | 2018-03-13 | 2018-08-28 | 中南大学 | The sorting technique of unbalanced data |
| CN108389201A (en) * | 2018-03-16 | 2018-08-10 | 北京推想科技有限公司 | The good pernicious sorting technique of Lung neoplasm based on 3D convolutional neural networks and deep learning |
| CN108520220A (en) * | 2018-03-30 | 2018-09-11 | 百度在线网络技术（北京）有限公司 | model generating method and device |
| CN108537223A (en) * | 2018-04-16 | 2018-09-14 | 广东工业大学 | A kind of detection method of license plate, system and equipment and storage medium |
| CN108628971A (en) * | 2018-04-24 | 2018-10-09 | 深圳前海微众银行股份有限公司 | File classification method, text classifier and the storage medium of imbalanced data sets |
Non-Patent Citations (4)
| Title |
|---|
| 何斯迈;金羽佳;王华;葛冬冬;: "在线学习方法综述:汤普森抽样和其他方法", 运筹学学报, no. 04 * |
| 何艳芳，石丹林主编: "银行客户信用评级", 31 March 2002, 北京：中国商业出版社 , 2002.03, pages: 421 - 425 * |
| 袁公萍; 汤一平; 韩旺明; 陈麒: "基于深度卷积神经网络的车型识别方法", 《浙江大学学报(工学版)》 * |
| 袁公萍;汤一平;韩旺明;陈麒;: "基于深度卷积神经网络的车型识别方法", 浙江大学学报(工学版), no. 04 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112580643A (en) * | 2020-12-09 | 2021-03-30 | 浙江智慧视频安防创新中心有限公司 | License plate recognition method and device based on deep learning and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108846835B (en) | Image change detection method based on depth separable convolutional network | |
| CN109523790B (en) | License plate error correction method based on license plate similarity | |
| CN109271991B (en) | License plate detection method based on deep learning | |
| CN105512660A (en) | License number identification method and device | |
| CN109325418A (en) | Based on pedestrian recognition method under the road traffic environment for improving YOLOv3 | |
| CN102388392A (en) | Pattern recognition device | |
| CN103164701B (en) | Handwritten Numeral Recognition Method and device | |
| CN110956169A (en) | License plate recognition method and device and electronic equipment | |
| CN102346847A (en) | License plate character recognizing method of support vector machine | |
| CN109685956B (en) | Method, device, equipment and storage medium for scanning and unlocking shared bicycle license plate | |
| CN111126122B (en) | Face recognition algorithm evaluation method and device | |
| CN110503099B (en) | Information identification method based on deep learning and related equipment | |
| CN109740040B (en) | Verification code identification method, device, storage medium and computer equipment | |
| CN110633289A (en) | License plate number determining method, server and parking management system | |
| CN111507385B (en) | Extensible network attack behavior classification method | |
| CN106033443B (en) | A kind of expanding query method and device in vehicle retrieval | |
| CN107292302A (en) | Detect the method and system of point of interest in picture | |
| CN113468539A (en) | Attack program identification method based on vulnerability attack database and decision tree | |
| CN108537223B (en) | License plate detection method, system and equipment and storage medium | |
| CN111858977B (en) | Bill information acquisition method, device, computer equipment and storage medium | |
| CN110969176B (en) | License plate sample amplification method and device and computer equipment | |
| CN111046891A (en) | Training method of license plate recognition model, and license plate recognition method and device | |
| CN116524725B (en) | Intelligent driving traffic sign image data identification system | |
| CN111126420B (en) | Method and device for establishing identification model | |
| CN116127337B (en) | Risk mining method, device, storage medium and equipment based on position and image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |