CN110443245B - License plate region positioning method, device and equipment in non-limited scene - Google Patents
License plate region positioning method, device and equipment in non-limited scene Download PDFInfo
- Publication number
- CN110443245B CN110443245B CN201910747526.9A CN201910747526A CN110443245B CN 110443245 B CN110443245 B CN 110443245B CN 201910747526 A CN201910747526 A CN 201910747526A CN 110443245 B CN110443245 B CN 110443245B
- Authority
- CN
- China
- Prior art keywords
- image
- vehicle
- license plate
- size
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 20
- 238000003062 neural network model Methods 0.000 claims abstract description 31
- 238000001514 detection method Methods 0.000 claims abstract description 22
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims abstract description 20
- 230000009466 transformation Effects 0.000 claims abstract description 17
- 238000013528 artificial neural network Methods 0.000 claims description 27
- 238000013527 convolutional neural network Methods 0.000 claims description 11
- 238000007781 pre-processing Methods 0.000 claims description 7
- 230000005284 excitation Effects 0.000 claims description 6
- 238000012549 training Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims 1
- 230000006870 function Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/02—Affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/32—Normalisation of the pattern dimensions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a device and equipment for positioning a license plate region in a non-limited scene, wherein an image of a vehicle is collected, the image is input into a preset neural network model, and a vehicle image containing the vehicle image is obtained, wherein the preset neural network model is used for identifying that the image of the vehicle exists in the image; according to the ratio between the size of the license plate and the vehicle boundary frame, zooming the image size of the vehicle image to obtain a vehicle image with uniform size; the vehicle image is input into a full convolution network, wherein the full convolution network identifies and determines the area of the license plate in the vehicle image, the full convolution network is used for calculating the probability that a unit corresponding to a target point in the vehicle image covers the license plate, and the full convolution network is used for outputting affine transformation parameters to correct the size of the license plate area, so that the problem of inaccurate detection of the license plate area in an unlimited scene is solved, and the accuracy of the license plate positioning area is improved.
Description
Technical Field
The invention relates to the field of image recognition, in particular to a method, a device and equipment for positioning a license plate region in an unlimited scene.
Background
Most of common commercial software is used for identifying license plates under the condition of limited scenes, such as monitoring of parking lots and toll stations, the scenes are fixed, and the requirements on the shooting angle and distance of vehicles are high. At present, many license plate recognition systems have good recognition effect on a license plate photo shot on the front side, but the license plate recognition technology in a non-limited scene, such as expressway monitoring, mobile phone image shooting and the like, cannot reach practical standards. And the license plate detection technology under the non-limiting scene is a necessary premise for license plate identification. Only if the license plate area is correctly detected in the image can the license plate identification process be carried out.
Aiming at the problem that the detection of the license plate area in the non-limited scene is inaccurate in the related technology, an effective solution is not provided at present.
Disclosure of Invention
The invention provides a method, a device and equipment for positioning a license plate region in a non-limited scene, aiming at the problem of inaccurate detection of the license plate region in the non-limited scene in the related art.
According to one aspect of the invention, a method for positioning a license plate region in an unrestricted scene is provided, which is characterized by comprising the following steps:
acquiring an image of a vehicle, inputting the image into a preset neural network model, and acquiring a vehicle image containing the vehicle image, wherein the preset neural network model is used for identifying that the image of the vehicle exists in the image;
according to the ratio between the size of the license plate and the vehicle boundary frame, zooming the image size of the vehicle image to obtain a vehicle image with a uniform size;
and inputting the vehicle image into a full convolution network, wherein the full convolution network identifies and determines the area of the license plate in the vehicle image, the full convolution network is used for calculating the probability that the unit corresponding to the target point in the vehicle image covers the license plate, and the full convolution network is used for outputting affine transformation parameters to correct the size of the license plate area.
Further, the preset neural network comprises a YOLO network and a shallow neural network, wherein the shallow neural network comprises a shallow convolutional neural network with 2 convolutional layers and 1 full-link layer.
Further, the scaling of the image size of the vehicle image according to the ratio between the size of the license plate and the vehicle bounding box to obtain a vehicle image with a uniform size includes:
amplifying the vehicle image under the condition that the ratio between the size of the license plate and the vehicle boundary frame is lower than a preset first threshold value;
reducing the vehicle image under the condition that the ratio between the size of the license plate and the vehicle boundary frame is larger than a preset second threshold value;
unifying the size of the vehicle image according to the preset first threshold value and the preset second threshold value.
Further, the full convolution network comprises a first convolution module and a second convolution module, wherein the first convolution module is used for calculating the probability that the target point corresponding unit in the vehicle image covers the license plate, a Softmax function is used as an excitation function, and the second convolution module is used for obtaining the affine transformation parameters without using the excitation function.
Further, the method of expanding the training data set of the image input to the preset neural network model includes at least one of:
adjusting the license plate center in the image to be an image center;
scaling the license plate in the image to match its width to a value between 40px and 208 px;
randomly performing 3D angle rotation on the image;
mirroring the image with a 50% probability;
randomly translating to move the license plate from the image center;
cutting the image by taking the license plate as a center;
modifying a Hue Saturation Value (HSV) color space of the image.
According to another aspect of the present invention, there is also provided a license plate region locating device for locating a license plate region in an unlimited scene, the device including:
the input module is used for acquiring an image of a vehicle, inputting the image into a preset neural network model and acquiring a vehicle image containing the vehicle image, wherein the preset neural network model is used for identifying that the image of the vehicle exists in the image;
the preprocessing module is used for scaling the image size of the vehicle image according to the ratio between the size of the license plate and the vehicle boundary frame to obtain the vehicle image with uniform size;
the recognition detection module is used for inputting the vehicle image into a full convolution network, wherein the full convolution network recognizes and determines the area of the license plate in the vehicle image, the full convolution network is used for calculating the probability that the unit corresponding to the target point in the vehicle image covers the license plate, and the full convolution network is used for outputting affine transformation parameters to correct the size of the license plate area.
Further, the preset neural network comprises a YOLO network and a shallow neural network, wherein the shallow neural network comprises a shallow convolutional neural network with 2 convolutional layers and 1 full-link layer.
Further, the preprocessing module is further configured to enlarge the vehicle image when a ratio between a size of a license plate and a vehicle boundary frame is lower than a preset first threshold, reduce the vehicle image when the ratio between the size of the license plate and the vehicle boundary frame is greater than a preset second threshold, and unify sizes of the vehicle image according to the preset first threshold and the preset second threshold.
According to another aspect of the present invention, there is also provided a license plate recognition apparatus, the apparatus including a camera and a processor,
the camera is used for collecting images of a vehicle, the processor inputs the images into a preset neural network model to obtain vehicle images containing the vehicle images, and the preset neural network model is used for identifying that the images of the vehicle exist in the images;
the processor is further used for scaling the image size of the vehicle image according to the ratio between the size of the license plate and the vehicle boundary frame to obtain a vehicle image with a uniform size;
the processor is further configured to input the vehicle image into a full convolution network, wherein the full convolution network identifies and determines an area of a license plate in the vehicle image, wherein the full convolution network is configured to calculate a probability that a unit corresponding to a target point in the vehicle image covers the license plate, and the full convolution network is configured to output an affine transformation parameter to correct the size of the license plate area.
Further, the preset neural network comprises a YOLO network and a shallow neural network, wherein the shallow neural network comprises a shallow convolutional neural network with 2 convolutional layers and 1 full-link layer.
According to the invention, the image of the vehicle is collected, the image is input into the preset neural network model, and the vehicle image containing the vehicle image is obtained, wherein the preset neural network model is used for identifying that the image of the vehicle exists in the image; according to the ratio between the size of the license plate and the vehicle boundary frame, zooming the image size of the vehicle image to obtain a vehicle image with uniform size; the vehicle image is input into a full convolution network, wherein the full convolution network identifies and determines the area of the license plate in the vehicle image, the full convolution network is used for calculating the probability that a unit corresponding to a target point in the vehicle image covers the license plate, and the full convolution network is used for outputting affine transformation parameters to correct the size of the license plate area, so that the problem of inaccurate detection of the license plate area in an unlimited scene is solved, and the accuracy of the license plate positioning area is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
fig. 1 is a flowchart of a license plate region locating method in an unrestricted scene according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a vehicle detection neural network concept according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a full convolutional network LPnet architecture according to an embodiment of the present invention;
FIG. 4 is a block diagram of a license plate location device according to an embodiment of the present invention;
fig. 5 is a block diagram of a license plate recognition device according to an embodiment of the present invention.
Detailed Description
The invention will be described in detail hereinafter with reference to the accompanying drawings in conjunction with embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
In this embodiment, a method for positioning a license plate region in an unrestricted scene is provided, and fig. 1 is a flowchart of a method for positioning a license plate region in an unrestricted scene according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
step S102, collecting an image of a vehicle, inputting the image into a preset neural network model, and acquiring a vehicle image containing the vehicle image, wherein the preset neural network model is used for identifying that the image of the vehicle exists in the image;
step S104, zooming the image size of the vehicle image according to the ratio between the size of the license plate and the vehicle boundary frame to obtain the vehicle image with uniform size;
step S106, inputting the vehicle image into a full convolution network, wherein the full convolution network identifies and determines the area of the license plate in the vehicle image, the full convolution network is used for calculating the probability that the unit corresponding to the target point in the vehicle image covers the license plate, and the full convolution network is used for outputting affine transformation parameters to correct the size of the license plate area.
Through the steps, the image of the vehicle is collected, the image is input into a preset neural network model, and the vehicle image containing the vehicle image is obtained, wherein the preset neural network model is used for identifying that the image of the vehicle exists in the image; according to the ratio between the size of the license plate and the vehicle boundary frame, zooming the image size of the vehicle image to obtain a vehicle image with uniform size; the vehicle image is input into a full convolution network, wherein the full convolution network identifies and determines the area of the license plate in the vehicle image, the problem of inaccuracy in detection of the license plate area in an unlimited scene is solved, and the accuracy of the license plate positioning area is improved.
In this embodiment, a higher recognition rate is required for vehicle image detection, because any vehicle missing detection directly results in missing detection of the whole license plate, and in addition, a shorter forward propagation time of the vehicle detection network is required, so that higher real-time performance is achieved. The preset neural network model may use a YOLO network, which has the advantages of fast execution speed and high recognition accuracy, and for the YOLO network, in the present embodiment, outputs related to vehicles (i.e., cars and buses) are combined and other categories are ignored.
In order to improve the recognition accuracy of the vehicle detection network on a certain scene, a shallow neural network is added behind the YOLO network in the embodiment. Fig. 2 is a schematic diagram of the principle of a vehicle detection neural network according to an embodiment of the present invention, in which the shallow neural network includes a shallow convolutional neural network of 2 convolutional layers and 1 fully-connected layer, and features output by the YOLO fully-connected layer are converted into a two-dimensional space as an input image, and the shallow convolutional neural network is supervised-trained. By training the shallow convolutional neural network, the complex relationship between high-level abstract features can be learned. The structure and parameters of the shallow convolutional neural network are that the layer 1 and the layer 2 are convolutional layers, the sizes of convolutional kernels are both 5 multiplied by 5, the number of convolutional kernels of the layer 1 is 96, and the number of convolutional kernels of the layer 2 is 72. Then followed by 1 fully connected layer, size 1000.
In an embodiment of the present invention, the scaling of the image size of the vehicle image according to the ratio between the size of the license plate and the vehicle bounding box, and the obtaining of the vehicle image with a uniform size may include:
amplifying the vehicle image under the condition that the ratio between the size of the license plate and the vehicle boundary frame is lower than a preset first threshold value;
reducing the vehicle image under the condition that the ratio between the size of the license plate and the vehicle boundary frame is larger than a preset second threshold value;
unifying the size of the vehicle image according to the preset first threshold and the preset second threshold.
In the embodiment of the invention, the detected vehicle images are sent to the subsequent license plate recognition and detection after the vehicle image size is preprocessed. The ratio between the license plate size and the vehicle bounding box is high in front and rear elevation views of the vehicle. The vehicle bounding box tends to be larger for squinting/side viewing, so the ratio is much smaller. In order to ensure the identifiability of the license plate region, when the vehicle picture has an angle, the picture is amplified to amplify the license plate region; when the vehicle photo is originally large, the size is correspondingly reduced. This is to pre-process the vehicle image size. The scaling factor calculation method of the vehicle image is as follows formula 1:
wherein, WvAnd HvWidth and height, respectively, of the vehicle bounding box, DminMinimum size pixel value, D, for vehicle image sizemaxMaximum size pixel value for vehicle image size, where Dmin≤fsc*min(Wv，Hv)≤Dmax. On an experimental basis, D was chosen in order to maintain a good balance between accuracy and run timemin240px and Dmax580 px. After the scaling calculation, the pictures containing the vehicle images are unified into a picture size of 240 × 580.
In this embodiment, fig. 3 is a schematic diagram of a full convolutional network LPnet architecture according to an embodiment of the present invention, and as shown in fig. 3, the full convolutional network LPnet forward propagation generates a feature map, and a probability of an encoding object and parameters of affine transformation. And if the target probability of the unit taking (m, n) as the center is greater than the detected threshold value, the license plate is in the area covered by the unit, and the output affine transformation parameters are used for correcting the license plate.
In order to ensure the speed of detecting the license plate, the embodiment of the invention selects a MobileNet V2 algorithm model, and the network is designed for deep learning application of a mobile terminal and an embedded terminal, so that the ideal speed requirement can be achieved on a processor. In terms of detection, the embodiment envisages having two convolution modules in parallel (convolution kernel CONV 3 × 3, 2): the first convolution module is: for calculating the probability of the license plate, a softmax function is used as the excitation function. The second convolution module is: for obtaining affine parameters, no excitation function is used.
For the LPnet framework in the implementation, a Loss function is designed, and the function has the functions of detecting the license plate and outputting affine parameters. Let pi＝[xi，yi]TAnd i 1.. 4, which represent the four corners of the license plate. The size of the domestic license plate is 440mm multiplied by 140mm, and under the proportion, the vertex coordinates of the standard license plate are set as follows: q. q.s1＝[-0.5，0.3]T，q2＝[0.5，0.3]T，q3＝[0.5，-0.3]Tq4＝[-0.5，-0.3]T。
For an input image of height H and width W, the network output feature map consists of an mxnx8 convolution, where M ═ H/Ns，，N＝W/Ns，NsThe scale of the image reduction after forward propagation. For the cell corresponding to each point (m, n) in the feature map, there are eight values to be estimated: the first two values (v)1And v2) Probability of being target and non-target, the last six values (v)3To v8) For constructing local affine transformations TmnLocal affine transformation TmnThe following equation 2 is calculated:
to match the network output resolution, point piThe coordinates need to be rescaled and re-centered from each point (m, n) in the feature map. Realized by a normalization function.
Where α is a proportionality constant used to normalize the coordinates to a range of 0-1 values.
Assuming that a license plate target exists at the corresponding unit of (m, n), the first part of the loss function is the expression of the error between the standard license plate coordinates after the license plate affine correction as the formula 4:
the second part of the loss function is the expression of equation 5 to determine the probability of the presence or absence of a license plate in the (m, n) corresponding cell.
fprobs(m,n)＝logloss(IIobj，v1)+logloss(1-IIobj，v2) Equation 5
Wherein IobjIs the object indicator function, returns 1 if there is a license plate in the (m, n) corresponding cell, otherwise logoss (y, p) ═ ylog (p).
The final loss function is expressed by the combination defined in the formula as in formula 6.
In an embodiment of the present invention, the method of expanding the training data set of the image input to the preset neural network model includes at least one of:
adjusting the license plate center in the image into an image center;
scaling the license plate in the image to match the width with a value between 40px and 208 px;
randomly performing 3D angle rotation on the image;
mirroring the image with a 50% probability;
randomly translating to move the license plate from the center of the image;
cutting the image by taking the license plate as a center;
the hue saturation value HSV color space of the image is modified.
Fig. 4 is a block diagram of a license plate region locating device according to an embodiment of the present invention, as shown in fig. 4, the device is used for locating a license plate region, and the device includes: an input module 42, a pre-processing module 44, and an identification detection module 46.
The input module 42 is configured to acquire an image of a vehicle, input the image into a preset neural network model, and acquire a vehicle image including the vehicle image, where the preset neural network model is used to identify that the image of the vehicle is included in the image;
the preprocessing module 44 is configured to perform image size scaling on the vehicle image according to a ratio between a size of a license plate and a vehicle bounding box, and obtain a vehicle image with a uniform size;
and the identification detection module 46 is configured to input the vehicle image into a full convolution network, where the full convolution network identifies and determines an area of the license plate in the vehicle image, the full convolution network is configured to calculate a probability that a unit corresponding to a target point in the vehicle image covers the license plate, and the full convolution network is configured to output an affine transformation parameter to correct the size of the license plate area.
Through the device, the input module 42 collects images of the vehicle, the images are input into the preset neural network model to obtain vehicle images containing the vehicle images, the preprocessing module 44 scales the image sizes of the vehicle images according to the ratio between the size of the license plate and the vehicle boundary frame to obtain vehicle images with uniform sizes, and the recognition and detection module 46 inputs the vehicle images into the full convolution network, wherein the full convolution network recognizes and determines the area of the license plate in the vehicle images, so that the problem of inaccurate detection of the license plate area in an unrestricted scene is solved, and the accuracy of the license plate positioning area is improved.
In the implementation process of the license plate region locating device of the embodiment, as described in the above embodiment, the preset neural network includes a YOLO network and a shallow neural network, wherein the shallow neural network includes a shallow convolutional neural network of 2 convolutional layers and 1 fully-connected layer.
Fig. 5 is a block diagram illustrating a license plate recognition apparatus according to an embodiment of the present invention, as shown in fig. 5, the license plate recognition apparatus 500 includes a camera 52 and a processor 54,
the camera 52 is configured to capture an image of a vehicle, and the processor 54 inputs the image into a preset neural network model to obtain a vehicle image including the vehicle image, where the preset neural network model is configured to identify that the image of the vehicle is included in the image;
the processor 54 is further configured to scale the image size of the vehicle image according to the ratio between the size of the license plate and the vehicle bounding box, and obtain a vehicle image with a uniform size;
the processor 55 is further configured to input the vehicle image into a full convolution network, wherein the full convolution network identifies and determines a region of the license plate in the vehicle image, wherein the full convolution network is configured to calculate a probability that a unit corresponding to a target point in the vehicle image covers the license plate, and the full convolution network is configured to output an affine transformation parameter to correct a size of the license plate region.
Through the equipment, the full convolution network identifies and determines the license plate in the area of the vehicle image, so that the problem of inaccuracy in detection of the license plate area in an unlimited scene is solved, and the accuracy of the license plate positioning area is improved.
In another embodiment, a software is provided, which is used to execute the technical solutions described in the above embodiments and the preferred embodiments.
In another embodiment, a storage medium is provided, wherein the software is stored in the storage medium, and the storage medium includes, but is not limited to, an optical disc, a floppy disc, a hard disc, a rewritable memory, and the like.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (7)
1. A method for positioning a license plate region in an unlimited scene is characterized by comprising the following steps:
collecting an image of a vehicle, inputting the image into a preset neural network model, obtaining a vehicle image containing the vehicle image, wherein the preset neural network model is used for identifying that the image of the vehicle exists in the image, the preset neural network model comprises a YOLO network and a shallow neural network, the features output by a full connection layer of the YOLO network are converted into a two-dimensional space to be used as an input image of the shallow neural network, the size of the full connection layer of the YOLO network is 1024, the conversion is carried out on the neural network into a two-dimensional space of 32 multiplied by 32, wherein the shallow neural network is a shallow convolutional neural network comprising 2 convolutional layers and 1 full-connection layer, the sizes of convolutional kernels of the 1 st convolutional layer and the 2 nd convolutional layer are both 5 multiplied by 5, the number of convolutional kernels of the 1 st convolutional layer is 96, the number of convolutional kernels of the 2 nd convolutional layer is 72, and the size of the full-connection layer is 1000;
according to the ratio between the size of the license plate and the vehicle boundary frame, zooming the image size of the vehicle image to obtain a vehicle image with a uniform size;
and inputting the vehicle image into a full convolution network, wherein the full convolution network identifies and determines the area of the license plate in the vehicle image, the full convolution network is used for calculating the probability that the unit corresponding to the target point in the vehicle image covers the license plate, and the full convolution network is used for outputting affine transformation parameters to correct the size of the license plate area.
2. The method of claim 1, wherein the scaling of the image size of the vehicle image according to the ratio between the size of the license plate and the vehicle bounding box to obtain a uniform size of the vehicle image comprises:
amplifying the vehicle image under the condition that the ratio between the size of the license plate and the vehicle boundary frame is lower than a preset first threshold value;
reducing the vehicle image under the condition that the ratio between the size of the license plate and the vehicle boundary frame is larger than a preset second threshold value;
unifying the size of the vehicle image according to the preset first threshold value and the preset second threshold value.
3. The method according to any one of claims 1 to 2,
the full convolution network comprises a first convolution module and a second convolution module, wherein the first convolution module is used for calculating the probability that the target point corresponding unit in the vehicle image covers the license plate, a Softmax function is used as an excitation function, and the second convolution module is used for obtaining the affine transformation parameters and does not use the excitation function.
4. The method of any one of claims 1 to 2, wherein the method of expanding the training data set of the image input to the pre-set neural network model comprises at least one of:
adjusting the license plate center in the image to be an image center;
scaling the license plate in the image to match its width to a value between 40px and 208 px;
randomly performing 3D angle rotation on the image;
mirroring the image with a 50% probability;
randomly translating to move the license plate from the image center;
cutting the image by taking the license plate as a center;
modifying a Hue Saturation Value (HSV) color space of the image.
5. A device for locating a license plate region in an unlimited scene, the device comprising:
an input module, configured to acquire an image of a vehicle, input the image into a preset neural network model, and obtain a vehicle image including the vehicle image, where the preset neural network model is used to identify that the image of the vehicle exists in the image, the preset neural network model includes a YOLO network and a shallow neural network, a feature output by a full connection layer of the YOLO network is converted into a two-dimensional space as an input image of the shallow neural network, a size of the full connection layer of the YOLO network is 1024, which is converted into the two-dimensional space as 32 × 32, where the shallow neural network is a shallow convolutional neural network including 2 convolutional layers and 1 full connection layer, sizes of convolutional cores of the 1-th convolutional layer and the 2-th convolutional layer are both 5 × 5, the number of convolutional cores of the 1-th convolutional layer is 96, and the number of convolutional cores of the 2-th convolutional layer is 72, the size of the fully connected layer is 1000;
the preprocessing module is used for scaling the image size of the vehicle image according to the ratio between the size of the license plate and the vehicle boundary frame to obtain the vehicle image with uniform size;
the recognition detection module is used for inputting the vehicle image into a full convolution network, wherein the full convolution network recognizes and determines the area of the license plate in the vehicle image, the full convolution network is used for calculating the probability that the unit corresponding to the target point in the vehicle image covers the license plate, and the full convolution network is used for outputting affine transformation parameters to correct the size of the license plate area.
6. The device of claim 5, wherein the preprocessing module is further configured to enlarge the vehicle image if a ratio between a size of a license plate and a vehicle bounding box is lower than a preset first threshold, reduce the vehicle image if the ratio between the size of the license plate and the vehicle bounding box is greater than a preset second threshold, and unify the size of the vehicle image according to the preset first threshold and the preset second threshold.
7. A license plate recognition device is characterized by comprising a camera and a processor,
the camera is used for acquiring an image of a vehicle, the processor inputs the image into a preset neural network model to acquire a vehicle image containing the vehicle image, wherein the preset neural network model is used for identifying that the image of the vehicle exists in the image, the preset neural network comprises a YOLO network and a shallow neural network, the feature output by a full connecting layer of the YOLO network is converted into a two-dimensional space to be used as the input image of the shallow neural network, the size of the full connecting layer of the YOLO network is 1024, the full connecting layer of the YOLO network is converted into the two-dimensional space to be 32 x 32, the shallow neural network is a shallow convolutional neural network comprising 2 convolutional layers and 1 full connecting layer, the convolutional cores of the 1 convolutional layer and the 2 convolutional layer are both 5 x 5, the convolutional core number of the 1 convolutional layer is 96, the convolutional core number of the 2 convolutional layer is 72, the size of the fully connected layer is 1000;
the processor is further used for scaling the image size of the vehicle image according to the ratio between the size of the license plate and the vehicle boundary frame to obtain a vehicle image with a uniform size;
the processor is further configured to input the vehicle image into a full convolution network, wherein the full convolution network identifies and determines an area of a license plate in the vehicle image, wherein the full convolution network is configured to calculate a probability that a unit corresponding to a target point in the vehicle image covers the license plate, and the full convolution network is configured to output an affine transformation parameter to correct the size of the license plate area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910747526.9A CN110443245B (en) | 2019-08-14 | 2019-08-14 | License plate region positioning method, device and equipment in non-limited scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910747526.9A CN110443245B (en) | 2019-08-14 | 2019-08-14 | License plate region positioning method, device and equipment in non-limited scene |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110443245A CN110443245A (en) | 2019-11-12 |
| CN110443245B true CN110443245B (en) | 2022-02-15 |
Family
ID=68435360
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910747526.9A Active CN110443245B (en) | 2019-08-14 | 2019-08-14 | License plate region positioning method, device and equipment in non-limited scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110443245B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111079744B (en) * | 2019-12-06 | 2020-09-01 | 鲁东大学 | Intelligent vehicle license plate identification method and device suitable for complex illumination environment |
| CN111797713B (en) * | 2020-06-16 | 2024-08-06 | 浙江大华技术股份有限公司 | License plate recognition method and photographing equipment |
| CN112560608B (en) * | 2020-12-05 | 2024-05-24 | 江苏爱科赛尔云数据科技有限公司 | Vehicle license plate recognition method |
| CN113628206B (en) * | 2021-08-25 | 2022-08-12 | 深圳市捷顺科技实业股份有限公司 | License plate detection method, device and medium |
| CN113780278A (en) * | 2021-09-10 | 2021-12-10 | 北京精英路通科技有限公司 | Method and device for identifying license plate content, electronic equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103065135A (en) * | 2013-01-25 | 2013-04-24 | 上海理工大学 | License number matching algorithm based on digital image processing |
| CN104408430A (en) * | 2014-12-01 | 2015-03-11 | 广东中星电子有限公司 | License plate positioning method and device |
| CN106778745A (en) * | 2016-12-23 | 2017-05-31 | 深圳先进技术研究院 | A kind of licence plate recognition method and device, user equipment |
| CN106874907A (en) * | 2017-01-19 | 2017-06-20 | 博康智能信息技术有限公司北京海淀分公司 | A kind of method and device for setting up Car license recognition model |
| CN108548820A (en) * | 2018-03-28 | 2018-09-18 | 浙江理工大学 | Cosmetics paper labels defect inspection method |
| CN110008360A (en) * | 2019-04-09 | 2019-07-12 | 河北工业大学 | Vehicle target image data base method for building up comprising specific background image |
| CN110059683A (en) * | 2019-04-15 | 2019-07-26 | 广州广电银通金融电子科技有限公司 | A kind of license plate sloped antidote of wide-angle based on end-to-end neural network |
-
2019
- 2019-08-14 CN CN201910747526.9A patent/CN110443245B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103065135A (en) * | 2013-01-25 | 2013-04-24 | 上海理工大学 | License number matching algorithm based on digital image processing |
| CN104408430A (en) * | 2014-12-01 | 2015-03-11 | 广东中星电子有限公司 | License plate positioning method and device |
| CN106778745A (en) * | 2016-12-23 | 2017-05-31 | 深圳先进技术研究院 | A kind of licence plate recognition method and device, user equipment |
| CN106874907A (en) * | 2017-01-19 | 2017-06-20 | 博康智能信息技术有限公司北京海淀分公司 | A kind of method and device for setting up Car license recognition model |
| CN108548820A (en) * | 2018-03-28 | 2018-09-18 | 浙江理工大学 | Cosmetics paper labels defect inspection method |
| CN110008360A (en) * | 2019-04-09 | 2019-07-12 | 河北工业大学 | Vehicle target image data base method for building up comprising specific background image |
| CN110059683A (en) * | 2019-04-15 | 2019-07-26 | 广州广电银通金融电子科技有限公司 | A kind of license plate sloped antidote of wide-angle based on end-to-end neural network |
Non-Patent Citations (3)
| Title |
|---|
| "YOLO9000:Better, Faster, Stronger";Joseph Redmon et al;《arXiv》;20161225;全文 * |
| "You Only Look Once:Unified, Real-Time Object Detection";Joseph Redmon et al;《arXiv》;20160409;全文 * |
| S＇ergio Montazzolli Silva et al."License Plate Detection and Recognition in Unconstrained Scenarios".《ECCV》.2018, * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110443245A (en) | 2019-11-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110443245B (en) | License plate region positioning method, device and equipment in non-limited scene | |
| CN110378837B (en) | Target detection method and device based on fish-eye camera and storage medium | |
| CN108960211A (en) | A kind of multiple target human body attitude detection method and system | |
| CN109741241B (en) | Fisheye image processing method, device, equipment and storage medium | |
| CN111160202A (en) | AR equipment-based identity verification method, AR equipment-based identity verification device, AR equipment-based identity verification equipment and storage medium | |
| CN112201078A (en) | Automatic parking space detection method based on graph neural network | |
| CN113192646A (en) | Target detection model construction method and different target distance monitoring method and device | |
| CN112101113B (en) | Lightweight unmanned aerial vehicle image small target detection method | |
| CN110827320A (en) | Target tracking method and device based on time sequence prediction | |
| CN113570530A (en) | Image fusion method and device, computer readable storage medium and electronic equipment | |
| CN113538523B (en) | Parking space detection tracking method, electronic equipment and vehicle | |
| CN118376355A (en) | Intelligent detection method for shield tunnel lining leakage water | |
| US20140055644A1 (en) | Apparatus and method for extracting object | |
| CN109376653B (en) | Method, apparatus, device and medium for locating vehicle | |
| CN115205793B (en) | Electric power machine room smoke detection method and device based on deep learning secondary confirmation | |
| CN114724119B (en) | Lane line extraction method, lane line detection device, and storage medium | |
| CN114463685B (en) | Behavior recognition method, behavior recognition device, electronic equipment and storage medium | |
| CN112364693B (en) | Binocular vision-based obstacle recognition method, device, equipment and storage medium | |
| CN114972945A (en) | Multi-machine-position information fusion vehicle identification method, system, equipment and storage medium | |
| CN112347853A (en) | License plate data desensitization method based on video, storage medium and server | |
| CN114360044A (en) | Gesture recognition method and device, terminal equipment and computer readable storage medium | |
| CN110827348A (en) | Self-checking processing method, device and equipment for position of camera equipment and storage medium | |
| CN112197747B (en) | Method and apparatus for assisting target detection using wireless positioning | |
| CN118518093B (en) | Laser SLAM method, equipment and medium based on multi-frame space occupancy rate | |
| CN118172763B (en) | Object detection method, intelligent device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 5288, building e, 555 Dongchuan Road, Minhang District, Shanghai 200241 Applicant after: Shanghai Shimao Internet of Things Technology Co.,Ltd. Address before: World Trade Building, 55 Weifang West Road, Pudong New Area, Shanghai, 200120 Applicant before: Shanghai Shimao Internet of Things Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |