CN111914837A - License plate detection method, device, equipment and storage medium - Google Patents
License plate detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN111914837A CN111914837A CN202010666508.0A CN202010666508A CN111914837A CN 111914837 A CN111914837 A CN 111914837A CN 202010666508 A CN202010666508 A CN 202010666508A CN 111914837 A CN111914837 A CN 111914837A
- Authority
- CN
- China
- Prior art keywords
- license plate
- image
- target
- preset number
- video frames
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 144
- 239000002131 composite material Substances 0.000 claims description 20
- 238000000034 method Methods 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 15
- 238000006748 scratching Methods 0.000 claims description 9
- 230000002393 scratching effect Effects 0.000 claims description 9
- 238000010586 diagram Methods 0.000 description 21
- 238000013527 convolutional neural network Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 7
- 239000003086 colorant Substances 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 6
- 230000002194 synthesizing effect Effects 0.000 description 6
- 239000000463 material Substances 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000012958 reprocessing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
Abstract
The invention discloses a license plate detection method, a license plate detection device, license plate detection equipment and a storage medium based on YOLO target detection, wherein the license plate detection method comprises the following steps: continuously intercepting a preset number of video frames in a video stream to be detected; detecting a first object target in a first frame image in the preset number of video frames in a YOLO target detection mode to obtain a first positioning result corresponding to the first object target; and step three, detecting a second object target in non-first frame images in the preset number of video frames according to the range determined by the first positioning result, and obtaining a second positioning result corresponding to the second object target in the non-first frame images until the preset number of video frames are detected. The invention can improve the detection capability of the small target and reduce the misjudgment while achieving high-speed detection.
Description
Technical Field
The invention relates to a target detection technology in the field of image recognition, in particular to a license plate detection method, a license plate detection device, license plate detection equipment and a storage medium based on YOLO target detection.
Background
Target detection is one of important research directions in the field of computer vision, and the traditional target detection method is to use a classifier to classify features after extracting the features by constructing a feature descriptor, so as to realize target detection, such as HOG (Histogram of Oriented Gradient) and SVM (Support Vector Machine). With the excellent performance of deep learning in the image classification field, the convolutional neural network is beginning to be widely used in various fields of computer vision, and the realization of target detection by using deep learning in the target detection field also becomes a new direction.
The existing technical scheme for realizing target detection by using deep learning is listed as follows:
the classical model of target detection includes R-CNN (Region CNN) and Faster R-CNN (fast R-CNN). The fast R-CNN deep learning target detection algorithm proposed in 2015 is higher than the conventional method of combining a feature descriptor and a classifier in terms of mAP (mean average precision). However, the fast R-CNN still has the problem of slow detection speed.
The Object Detection algorithm YOLO (You see Only Once) was proposed in 2016, and its nomenclature comes from the article "You Only Look one: Unifield, Real-Time Object Detection, You see Only Once: unified real-time target detection ". The same year as the industry also proposed a target detection algorithm SSD (Single-shot detector Single deep neural network). The YOLO and the SSD finish target detection in a regression mode, so that the target detection in a deep learning mode achieves real-time detection speed.
The Faster R-CNN belongs to a two-step target detection algorithm, and target detection is completed through two steps of classification and regression. YOLO and SSD belong to a one-stage target detection algorithm, target detection is directly realized through single-step regression, but the problem that small target detection capability is poor while high speed is achieved is solved. In practical engineering application, the situation that the proportion of the target to be detected in the image is small is more common.
Although the SSD can achieve a real-time effect on some GPU (graphics processing unit) servers, the SSD has too many model parameters and an excessive operating memory footprint, and cannot operate on a GPU device with a small display memory capacity or a mobile embedded device such as an ARM (Advanced RISC Machine).
At present, in the method for positioning the license plate by adopting a target detection algorithm, the adopted algorithm comprises the traditional computer vision and deep learning, but no matter which algorithm is adopted, the image is directly positioned, and the license plate is positioned from the whole image. The image is directly positioned, and the license plate is positioned from the whole image, which has the following defects: it is easy to misjudge. Since street view data includes many regions having the same image characteristics, such as advertisement signs or traffic signs, which are rectangular or rectangular and printed with characters and numbers, in addition to the license plate, the prior art schemes are prone to misjudge these regions, i.e., misrecognize the advertisement signs or traffic signs as license plates. In addition, other regions have too many possibilities and therefore have a high probability of being misjudged.
Disclosure of Invention
In order to solve the above problems, the present invention provides a license plate detection method, apparatus, device and storage medium based on YOLO target detection, which can improve the detection capability of small targets and reduce misjudgment while achieving high-speed detection.
In order to achieve the above object, the present invention provides a license plate detection method based on YOLO target detection, including:
continuously intercepting a preset number of video frames in a video stream to be detected;
detecting a first object target in a first frame image in the preset number of video frames in a YOLO target detection mode to obtain a first positioning result corresponding to the first object target;
and step three, detecting a second object target in non-first frame images in the preset number of video frames according to the range determined by the first positioning result, and obtaining a second positioning result corresponding to the second object target in the non-first frame images until the preset number of video frames are detected.
The license plate detection method further comprises the following steps:
and step four, judging whether the video stream to be detected is intercepted completely, if so, ending the process, otherwise, returning to the step one.
In the license plate detection method, in the second step, the YOLO target detection method for detecting the first object target is based on a model trained by a first data set.
In the license plate detection method, the first object target is an automobile target, and the first data set is a data set formed by a first composite image obtained by combining a background image and an automobile image.
In the license plate detection method, the first synthesized image is generated in a manner that: and scratching the automobile region in the background image, and putting different types of automobile pictures into the scratching region in the background image to synthesize a corresponding first synthesized image.
In the above license plate detection method, in the third step, the YOLO target detection method for detecting the second object target is to use a model trained by a second data set.
In the above license plate detection method, the second object target is a license plate target, and the second data set is a data set formed by a second synthesized image obtained by synthesizing the vehicle image and the license plate image.
In the license plate detection method, the second synthesized image is generated in a manner that: and scratching the license plate region in the automobile image, and putting different types of license plate pictures into the scratched region in the automobile image to synthesize a corresponding second synthesized image.
The invention also provides a license plate detection device based on the YOLO target detection, which comprises the following components:
the device comprises an intercepting unit, a processing unit and a processing unit, wherein the intercepting unit is used for continuously intercepting a preset number of video frames in a video stream to be detected;
a first detection unit, configured to detect a first object target in a first frame image of the preset number of video frames in a YOLO target detection manner, and obtain a first positioning result corresponding to the first object target;
and the second detection unit is used for detecting a second object target in the non-first frame image in the preset number of video frames according to the range determined by the first positioning result to obtain a second positioning result corresponding to the second object target in the non-first frame image until the preset number of video frames are detected.
In the above license plate detection device, further comprising:
and the circulating control unit is used for judging whether the video stream to be detected is intercepted completely, if so, ending the intercepting, otherwise, continuously intercepting by the intercepting unit.
The invention also provides a license plate detection device based on the YOLO target detection, which comprises the following components:
the shooting module is used for obtaining a video stream to be detected;
the cache module is used for continuously intercepting a preset number of video frames in the video stream to be detected;
the detection processing module is used for detecting a first object target in a first frame image in the preset number of video frames in a YOLO target detection mode to obtain a first positioning result corresponding to the first object target; and detecting a second object target in non-first frame images in the preset number of video frames according to the range determined by the first positioning result to obtain a second positioning result corresponding to the second object target in the non-first frame images until the preset number of video frames are detected.
The invention also provides a computer storage medium for license plate detection based on the YOLO target detection, which stores computer executable instructions, wherein the computer executable instructions are set as follows:
continuously intercepting a preset number of video frames in a video stream to be detected;
detecting a first object target in a first frame image in the preset number of video frames in a YOLO target detection mode to obtain a first positioning result corresponding to the first object target;
and detecting a second object target in non-first frame images in the preset number of video frames according to the range determined by the first positioning result to obtain a second positioning result corresponding to the second object target in the non-first frame images until the preset number of video frames are detected.
By adopting the invention, each frame of image is only detected once, which belongs to single step target detection and can ensure that the YOLO is used for high-speed detection.
The invention is described in detail below with reference to the drawings and specific examples, but the invention is not limited thereto.
Drawings
FIG. 1 is a flow chart of a license plate detection method in an embodiment of the invention;
FIG. 2 is a diagram illustrating a background image according to an embodiment of the present invention;
FIG. 3 is a schematic illustration of car area matting in a background image in an embodiment of the invention;
FIG. 4 is a schematic diagram of adding car images in car matting areas according to an embodiment of the present invention;
FIG. 5 is another schematic diagram of adding car images in car matting areas in an embodiment of the invention;
FIG. 6 is a schematic diagram of a background car image in an embodiment of the present invention;
FIG. 7 is a schematic diagram of removing a license plate region from a background car image according to an embodiment of the present disclosure;
FIG. 8 is a schematic diagram of adding a license plate image in a cutout license plate region according to an embodiment of the present disclosure;
FIG. 9 is a schematic diagram of a license plate detection apparatus according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of a license plate detection apparatus according to an embodiment of the present disclosure;
FIG. 11 is a diagram illustrating another background image according to an embodiment of the present invention;
FIG. 12 is a schematic illustration of car area matting in another background image in an embodiment of the invention;
FIG. 13 is a schematic view of adding a car image in a bus picking area in an embodiment of the present invention;
FIG. 14 is another schematic view of adding a car image in a cutout bus area in an embodiment of the present invention;
FIG. 15 is a schematic view of another background car image in an embodiment of the present invention;
FIG. 16 is a schematic diagram of removing a license plate region from another background car image according to an embodiment of the present invention;
FIG. 17 is a schematic diagram of adding a license plate image in a cutout license plate region according to an embodiment of the present invention;
fig. 18 is a schematic diagram of adding another license plate image to the scratch-out license plate region in the embodiment of the present invention.
Wherein the reference numerals are as follows:
Second detecting unit 303
Photographing module 401
Detailed Description
The following detailed description of the embodiments of the present invention with reference to the drawings and specific examples is provided for further understanding the objects, aspects and effects of the present invention, but not for limiting the scope of the appended claims.
FIG. 1 is a flow chart of a license plate detection method in an embodiment of the invention; as shown in fig. 1, an embodiment of the present invention provides a license plate detection method based on YOLO target detection, including:
Therefore, in the method embodiment of the invention, the first object target is positioned and detected on the first frame image, and the second object target is identified and detected on the subsequent frame image according to the positioning result, so that each frame image is only detected once, which belongs to single-step target detection and can ensure high-speed detection. The subsequent frame image is used for detecting the second object target in the positioning range and is not used for detecting the whole image, so that the data processing amount is reduced, the interference is prevented, the detection is quicker and more detailed, the detection capability of the small target is improved, and the misjudgment can be reduced.
Specifically, in the license plate recognition process, the embodiment of the method of the invention firstly detects the automobile in the first frame image, obtains the position range of the automobile in the first frame image, and then detects the license plate in the position range in the second frame image. Therefore, each frame of image is only detected once, and invalid detection outside the range of the automobile can be avoided when the license plate is detected, so that the license plate detection is more rapid and accurate. Of course, since the car is a moving object, the position range in the first frame image is slightly deviated from the position range in the second frame image, but since the video stream is shot faster, such slight deviation does not affect the detection accuracy of the license plate, as long as the magnitude of the selected preset number of video frames is small enough, for example, 5 frames, 10 frames or 15 frames, the license plate can be detected in the subsequent frames according to the car range determined by the first frame.
Referring to fig. 1, in another embodiment of the present invention, a license plate detection method further includes:
and 104, judging whether the video stream to be detected is intercepted completely, if so, ending the process, otherwise, returning to the step 101. This step is provided in order to ensure complete detection of the entire video stream to be detected.
In one embodiment of the method of the present invention, in the step 102, the YOLO target detection method for detecting the first object target is a model trained by a first data set. The first object target is a car target, and the first data set is a data set formed by a first composite image obtained by combining a background image and a car image.
In an embodiment of the method of the present invention, the first composite image is generated in a manner that: and scratching the automobile region in the background image, and putting different types of automobile pictures into the scratching region in the background image to synthesize a corresponding first synthesized image.
Fig. 2 is a schematic diagram of a background image in the embodiment of the present invention, and fig. 3 is a schematic diagram of car region removal in the background image in the embodiment of the present invention, as shown in fig. 2 and fig. 3, after the car region in the background picture fig. 2 is removed, fig. 3 is obtained. Then, different automotive graphics are added to the cutout areas in fig. 3 to form fig. 4 and 5. Of course, in the embodiment of the present invention, the background image and the car image are all thousands of images, and are synthesized by a random selection method of software, thousands of synthesized images similar to fig. 4 and 5 can be obtained, and these synthesized images form the first data set, and the YOLO target detection method for car detection uses a model trained by the first data set. Therefore, the original data set is used as a material to be synthesized into a synthesized data set with controllable quantity and consistent quantity for each category again by utilizing a mode of artificially synthesizing the data set, so that the prediction accuracy of the model for each category can be greatly improved.
In one embodiment of the method, in the step 103, the YOLO target detection method for detecting the second object target is a model trained by a second data set. The second object target is a license plate target, and the second data set is a data set formed by a second composite image obtained by combining the automobile image and the license plate image.
In an embodiment of the method of the present invention, the second composite image is generated in a manner that: and scratching the license plate region in the automobile image, and putting different types of license plate pictures into the scratched region in the automobile image to synthesize a corresponding second synthesized image.
FIG. 6 is a schematic diagram of a background car image in an embodiment of the present invention; FIG. 7 is a schematic diagram of removing a license plate region from a background car image according to an embodiment of the present disclosure; as shown in fig. 6 and 7, the license plate region in the background car picture fig. 6 is removed to obtain fig. 7. Then, a different license plate graphic is added to the cutout area in FIG. 7 to form FIG. 8. Certainly, in the embodiment of the present invention, the background car image and the license plate image are thousands of images, and are synthesized in a random software selection manner, thousands of synthesized images similar to fig. 8 can be obtained, and these synthesized images form the second data set, and the model trained by the second data set is used in the YOLO target detection manner for car detection.
Therefore, in the embodiment of the present invention, the processing of the data set related to the YOLO target detection method includes data enhancement and synthesizing the data set. The data enhancement is to generate license plates with different colors and sizes by a program as a composite data set. Since most public data sets are characterized by data imbalances, i.e., different classes of data differ in number by a factor of several or more than ten. For example, the number of images in the data set BDD100K for the car category is 1021857, the number of images in the bus category is 16505, and the number of images in the truck category is 42963. The deep learning model trained by the unbalanced data sets has better training effect for more categories, and the detection effect is obviously better than that of other categories.
In the embodiment of the present invention, the original images of all data are reprocessed, and first, the pixel values in the bounding boxes of all categories in each original image are set to 0 and stored as, for example, the background image of fig. 3 or fig. 7; storing the bounding boxes of all the categories as a single object image; according to the preset quantity, for a specific category, the object images stored in the boundary frame are randomly selected, the background images are randomly selected, the image values of the randomly selected position areas in the background images are replaced by the object images, and the quantity of the object images can be in a self-defined range. For example, in the embodiment of the present invention, a composite image of ten thousand car categories is generated, and the composite image is formed by combining a background image (e.g., car image with the license plate area removed as shown in fig. 7) and one or more object images (e.g., license plate images with different colors). The background image is stored with the pixel values within the bounding box of all classes in the original image set to 0. The object image is the image of the original image intercepted by the bounding box of different types. The composite image is a randomly selected background image, and within a quantity range, the range can be set by itself, for example, eight to twelve randomly selected object images of the same category, and the original pixel values of the original background image are replaced by the object images at the randomly selected positions in the background image.
Because the data set of the collected original images has unbalanced factors, such as too many images of cars and too few images of engineering vehicles and special vehicles, or too many images of common license plates and too few images of special license plates, and the like, the adoption of the original image reprocessing mode can enable the number of unbalanced data sets of various types to be detected to be controllably balanced, such as the number of car images, engineering vehicle images and special vehicle images in the data sets to be balanced, thereby preventing the special vehicles from being identified and increasing the detection and identification effects.
In the embodiment of the present invention, the object detection framework and the model architecture can use the object detection framework of YOLOv2, and the model is mobilent v 1. This section can be used alternatively as long as it is a one-stage object detection algorithm, for example, replacing YOLOv2 with an SSD object detection framework. Other configurations suitable for edge devices can be used for the model configuration, for example, instead of mobilene v1, mobilene v2 can be used instead of mobilene v1, as long as it is a small-capacity model configuration. In addition, when detecting vehicles of different vehicle types, the first model used is trained with a composite data set containing the different vehicle types; when detecting license plates of different colors and shapes, the second model used is trained with a synthetic data set containing license plates of color and shape. It should be noted that the forms of the first model and the second model in the embodiment of the present invention are not limited by the scope of the example, and may also be other types of vehicle data sets or license plate data sets.
From the above, the embodiment of the present invention adopts data enhancement and artificially synthesizes the data set with balanced quantity by the original data set with unbalanced quantity, thereby improving the accuracy. In the embodiment of the invention, the specific area containing different types of vehicles in the image is detected by using an object detection method, and the model used in the method is trained by using a composite data set containing different types of vehicles; and detecting the license plate by using an object detection method, wherein the model used in the method is trained by a synthetic data set containing different colors and shapes, and the license plate is identified aiming at the specific area containing different types of vehicles.
Fig. 9 is a schematic diagram of a license plate detection device in an embodiment of the present invention, and as shown in fig. 9, an embodiment of the present invention provides a license plate detection device based on YOLO target detection, including:
an intercepting unit 301, configured to continuously intercept a preset number of video frames in a video stream to be detected;
a first detecting unit 302, configured to detect a first object target in a first frame image of the preset number of video frames in a YOLO target detection manner, so as to obtain a first positioning result corresponding to the first object target;
a second detecting unit 303, configured to detect a second object target in a non-first frame image of the preset number of video frames according to the range determined by the first positioning result, to obtain a second positioning result corresponding to the second object target in the non-first frame image, until the preset number of video frames are detected.
In an embodiment of the present invention, the license plate detecting apparatus further includes:
and the loop control unit 304 is configured to determine whether the video stream to be detected is intercepted, if so, the process is ended, otherwise, the interception unit continues intercepting the video stream.
Therefore, because the known technology of directly detecting the license plate by using the object detection is easy to determine that the traffic sign, the signboard advertisement and the like are rectangular or rectangular objects as the positive sample by mistake, the embodiment of the detection device of the invention firstly detects the vehicle, then uses the vehicle as a whole, and carries out the object detection once again to detect the license plate, thereby effectively reducing the probability of misjudging other rectangular or rectangular objects.
Fig. 10 is a schematic diagram of a license plate detection device in an embodiment of the present invention, and as shown in fig. 10, an embodiment of the present invention further provides a license plate detection device based on YOLO target detection, including:
a shooting module 401, configured to obtain a video stream to be detected;
a caching module 402, configured to continuously intercept a preset number of video frames in the video stream to be detected;
a detection processing module 403, configured to detect a first object target in a first frame image of the preset number of video frames in a YOLO target detection manner, so as to obtain a first positioning result corresponding to the first object target; and detecting a second object target in non-first frame images in the preset number of video frames according to the range determined by the first positioning result to obtain a second positioning result corresponding to the second object target in the non-first frame images until the preset number of video frames are detected.
The shooting module 401 may be a video shooting device such as a video camera. The buffer module 402 may be a storage device such as a RAM or a ROM, and the detection processing module 403 may be a computer or a GPU (graphics processing unit), an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or the like, which completes actual operation processing.
The embodiment of the invention also provides a computer storage medium for license plate detection based on the YOLO target detection, which stores computer executable instructions, wherein the computer executable instructions are set as follows:
continuously intercepting a preset number of video frames in a video stream to be detected;
detecting a first object target in a first frame image in the preset number of video frames in a YOLO target detection mode to obtain a first positioning result corresponding to the first object target;
and detecting a second object target in non-first frame images in the preset number of video frames according to the range determined by the first positioning result to obtain a second positioning result corresponding to the second object target in the non-first frame images until the preset number of video frames are detected.
To further illustrate the manner in which the present invention artificially synthesizes a data set, the present invention provides an embodiment in which the first synthesized image is synthesized with another new background image.
Fig. 11 is a schematic diagram of another background image in the embodiment of the present invention, where there is a bus in the street in fig. 11, and fig. 12 is a schematic diagram of a region where the bus is cut out from the another background image in the embodiment of the present invention, as shown in fig. 11 and fig. 12, after the region of the bus in the another background image fig. 11 is cut out, fig. 12 is obtained. Then, different car graphics are added to the area of FIG. 12 where the bus is to be removed to form FIGS. 13 and 14. Of course, in the embodiment of the present invention, the background image and the car image are all thousands of images, and the synthesis is performed in a manner randomly selected by software, so that thousands of synthesized images similar to fig. 13 and 14 can be obtained, the synthesized images form a first data set, and the YOLO target detection method for detecting the car uses a model trained by the first data set. Therefore, the original data set is used as a material to be synthesized into a synthesized data set with controllable quantity and consistent quantity for each category again by utilizing a mode of artificially synthesizing the data set, so that the prediction accuracy of the model for each category can be greatly improved.
To further illustrate the manner in which the present invention artificially synthesizes a data set, the present invention provides an embodiment in which a second synthesized image is synthesized with another background car image.
FIG. 15 is a schematic view of another background car image in an embodiment of the present invention; FIG. 16 is a schematic diagram of removing a license plate region from another background car image according to an embodiment of the present invention; as shown in fig. 15 and 16, the license plate area in the picture of the backdrop all-terrain vehicle 15 is removed to obtain a picture 16. Then, a different license plate graphic is added to the cutout area in FIG. 16 to form FIG. 17. Certainly, in the embodiment of the present invention, the background car image and the license plate image are thousands of images, and are synthesized in a random selection manner by software, so that thousands of synthesized images similar to fig. 17 can be obtained, for example, another different license plate is added to form the synthesized image of fig. 18. These composite images constitute a second data set, and the YOLO target detection method for vehicle detection employs a model trained from this second data set.
In the embodiment of the present invention, the original images of all data are reprocessed, and first, the pixel values in the bounding boxes of all categories in each original image are set to 0 and stored as the background image in fig. 11 or fig. 15, for example; storing the bounding boxes of all the categories as a single object image; according to the preset quantity, for a specific category, the object images stored in the boundary frame are randomly selected, the background images are randomly selected, the image values of the randomly selected position areas in the background images are replaced by the object images, and the quantity of the object images can be in a self-defined range. For example, in the embodiment of the present invention, a composite image of ten thousand car categories is generated, and the composite image is formed by combining a background image (e.g., car image with the license plate area removed as shown in fig. 16) and one or more object images (e.g., license plate images with different colors). The background image is stored with the pixel values within the bounding box of all classes in the original image set to 0. The object image is the image of the original image intercepted by the bounding box of different types. The composite image is a randomly selected background image, and within a quantity range, the range can be set by itself, for example, eight to twelve randomly selected object images of the same category, and the original pixel values of the original background image are replaced by the object images at the randomly selected positions in the background image.
Therefore, in the embodiment of the present invention, the processing of the data set related to the YOLO target detection method includes data enhancement and synthesizing the data set. The data enhancement is to generate license plates with different colors and sizes by a program as a composite data set. Since most public data sets are characterized by data imbalances, i.e., different classes of data differ in number by a factor of several or more than ten. For example, the number of images in the vehicle category in the data set BDD100K is two million, the number of images in the bus category is twenty thousand, and the number of images in the truck category is fifty thousand. The deep learning model trained by the unbalanced data sets has better training effect for more categories, and the detection effect is obviously better than that of other categories.
Therefore, the invention has the following advantages:
1) in the embodiment of the invention, each frame of image is only detected once, which belongs to single-step target detection and can ensure that YOLO is used for high-speed detection. According to the invention, the automobile is detected only in the first frame image, and after the automobile is positioned, the license plate is detected in the non-first frame image, so that the interference of frame graphics except the automobile image to the license plate recognition can be prevented, the misjudgment probability can be reduced, and the recognition capability of the YOLO to the small target is improved.
2) According to the method, the original data set is used as a material in a mode of artificially synthesizing the data set, and the synthesized data set with controllable quantity and quite consistent quantity for each category is synthesized again, so that the prediction accuracy of the model for each category can be greatly improved.
The present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof, and it should be understood that various changes and modifications can be effected therein by one skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (12)
1. A license plate detection method based on YOLO target detection is characterized by comprising the following steps:
continuously intercepting a preset number of video frames in a video stream to be detected;
detecting a first object target in a first frame image in the preset number of video frames in a YOLO target detection mode to obtain a first positioning result corresponding to the first object target;
and step three, detecting a second object target in non-first frame images in the preset number of video frames according to the range determined by the first positioning result, and obtaining a second positioning result corresponding to the second object target in the non-first frame images until the preset number of video frames are detected.
2. The license plate detection method of claim 1, further comprising:
and step four, judging whether the video stream to be detected is intercepted completely, if so, ending the process, otherwise, returning to the step one.
3. The license plate detecting method of claim 2,
in the second step, the YOLO target detection method for detecting the first object target is to use a model trained by a first data set.
4. The license plate detecting method of claim 3, wherein the first object target is a car target, and the first data set is a data set formed by a first composite image obtained by combining a background image and a car image.
5. The license plate detection method of claim 4, wherein the first composite image is generated in a manner that: and scratching the automobile region in the background image, and putting different types of automobile pictures into the scratching region in the background image to synthesize a corresponding first synthesized image.
6. The license plate detecting method of claim 3,
in the third step, the YOLO target detection method for detecting the second object target is a model trained by a second data set.
7. The license plate detection method of claim 5, wherein the second object is a license plate object, and the second data set is a data set composed of a second composite image obtained by compositing the vehicle image and the license plate image.
8. The license plate detection method of claim 4, wherein the second composite image is generated in a manner that: and scratching the license plate region in the automobile image, and putting different types of license plate pictures into the scratched region in the automobile image to synthesize a corresponding second synthesized image.
9. A license plate detection device based on YOLO target detection is characterized by comprising:
the device comprises an intercepting unit, a processing unit and a processing unit, wherein the intercepting unit is used for continuously intercepting a preset number of video frames in a video stream to be detected;
a first detection unit, configured to detect a first object target in a first frame image of the preset number of video frames in a YOLO target detection manner, and obtain a first positioning result corresponding to the first object target;
and the second detection unit is used for detecting a second object target in the non-first frame image in the preset number of video frames according to the range determined by the first positioning result to obtain a second positioning result corresponding to the second object target in the non-first frame image until the preset number of video frames are detected.
10. The license plate detecting device of claim 9, further comprising:
and the circulating control unit is used for judging whether the video stream to be detected is intercepted completely, if so, ending the intercepting, otherwise, continuously intercepting by the intercepting unit.
11. A license plate detection device based on YOLO target detection is characterized by comprising:
the shooting module is used for obtaining a video stream to be detected;
the cache module is used for continuously intercepting a preset number of video frames in the video stream to be detected;
the detection processing module is used for detecting a first object target in a first frame image in the preset number of video frames in a YOLO target detection mode to obtain a first positioning result corresponding to the first object target; and detecting a second object target in non-first frame images in the preset number of video frames according to the range determined by the first positioning result to obtain a second positioning result corresponding to the second object target in the non-first frame images until the preset number of video frames are detected.
12. A computer storage medium for license plate detection based on YOLO target detection, having stored thereon computer-executable instructions configured to:
continuously intercepting a preset number of video frames in a video stream to be detected;
detecting a first object target in a first frame image in the preset number of video frames in a YOLO target detection mode to obtain a first positioning result corresponding to the first object target;
and detecting a second object target in non-first frame images in the preset number of video frames according to the range determined by the first positioning result to obtain a second positioning result corresponding to the second object target in the non-first frame images until the preset number of video frames are detected.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010666508.0A CN111914837A (en) | 2020-07-10 | 2020-07-10 | License plate detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010666508.0A CN111914837A (en) | 2020-07-10 | 2020-07-10 | License plate detection method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111914837A true CN111914837A (en) | 2020-11-10 |
Family
ID=73226436
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010666508.0A Pending CN111914837A (en) | 2020-07-10 | 2020-07-10 | License plate detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111914837A (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101848019B1 (en) * | 2016-12-27 | 2018-04-11 | 아주대학교산학협력단 | Method and Apparatus for Detecting Vehicle License Plate by Detecting Vehicle Area |
| CN108681693A (en) * | 2018-04-12 | 2018-10-19 | 南昌大学 | Licence plate recognition method based on trusted area |
| CN108810616A (en) * | 2018-05-31 | 2018-11-13 | 广州虎牙信息科技有限公司 | Object localization method, image display method, device, equipment and storage medium |
| CN109598246A (en) * | 2018-12-07 | 2019-04-09 | 广东亿迅科技有限公司 | Vehicle enters and leaves detection method, device, computer equipment and storage medium |
| CN109740571A (en) * | 2019-01-22 | 2019-05-10 | 南京旷云科技有限公司 | The method of Image Acquisition, the method, apparatus of image procossing and electronic equipment |
| CN109871763A (en) * | 2019-01-16 | 2019-06-11 | 清华大学 | A kind of specific objective tracking based on YOLO |
| CN109993138A (en) * | 2019-04-08 | 2019-07-09 | 北京易华录信息技术股份有限公司 | A kind of car plate detection and recognition methods and device |
| CN110826415A (en) * | 2019-10-11 | 2020-02-21 | 上海眼控科技股份有限公司 | Method and device for re-identifying vehicles in scene image |
| CN111062885A (en) * | 2019-12-09 | 2020-04-24 | 中国科学院自动化研究所 | Mark detection model training and mark detection method based on multi-stage transfer learning |
| CN111311634A (en) * | 2020-01-23 | 2020-06-19 | 支付宝实验室(新加坡)有限公司 | Face image detection method, device and equipment |
| CN111368830A (en) * | 2020-03-03 | 2020-07-03 | 西北工业大学 | License plate detection and identification method based on multi-video frame information and nuclear phase light filtering algorithm |
-
2020
- 2020-07-10 CN CN202010666508.0A patent/CN111914837A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101848019B1 (en) * | 2016-12-27 | 2018-04-11 | 아주대학교산학협력단 | Method and Apparatus for Detecting Vehicle License Plate by Detecting Vehicle Area |
| CN108681693A (en) * | 2018-04-12 | 2018-10-19 | 南昌大学 | Licence plate recognition method based on trusted area |
| CN108810616A (en) * | 2018-05-31 | 2018-11-13 | 广州虎牙信息科技有限公司 | Object localization method, image display method, device, equipment and storage medium |
| CN109598246A (en) * | 2018-12-07 | 2019-04-09 | 广东亿迅科技有限公司 | Vehicle enters and leaves detection method, device, computer equipment and storage medium |
| CN109871763A (en) * | 2019-01-16 | 2019-06-11 | 清华大学 | A kind of specific objective tracking based on YOLO |
| CN109740571A (en) * | 2019-01-22 | 2019-05-10 | 南京旷云科技有限公司 | The method of Image Acquisition, the method, apparatus of image procossing and electronic equipment |
| CN109993138A (en) * | 2019-04-08 | 2019-07-09 | 北京易华录信息技术股份有限公司 | A kind of car plate detection and recognition methods and device |
| CN110826415A (en) * | 2019-10-11 | 2020-02-21 | 上海眼控科技股份有限公司 | Method and device for re-identifying vehicles in scene image |
| CN111062885A (en) * | 2019-12-09 | 2020-04-24 | 中国科学院自动化研究所 | Mark detection model training and mark detection method based on multi-stage transfer learning |
| CN111311634A (en) * | 2020-01-23 | 2020-06-19 | 支付宝实验室(新加坡)有限公司 | Face image detection method, device and equipment |
| CN111368830A (en) * | 2020-03-03 | 2020-07-03 | 西北工业大学 | License plate detection and identification method based on multi-video frame information and nuclear phase light filtering algorithm |
Non-Patent Citations (1)
| Title |
|---|
| 傅鹏;谢世朋;: "基于级联卷积神经网络的车牌定位", 计算机技术与发展, no. 01, 19 October 2017 (2017-10-19) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112686812B (en) | Bank card inclination correction detection method and device, readable storage medium and terminal | |
| Liu et al. | Hybrid cascade structure for license plate detection in large visual surveillance scenes | |
| Chen et al. | Detection and Recognition of Traffic Signs Based on HSV Vision Model and Shape features. | |
| Ardianto et al. | Real-time traffic sign recognition using color segmentation and SVM | |
| US20090110286A1 (en) | Detection method | |
| CN106709530A (en) | License plate recognition method based on video | |
| CN105354573B (en) | A kind of container licence plate recognition method and system | |
| Cao et al. | Traffic sign detection and recognition using multi-scale fusion and prime sample attention | |
| Shah et al. | OCR-based chassis-number recognition using artificial neural networks | |
| CN107704797B (en) | Real-time detection method, system and equipment based on pedestrians and vehicles in security video | |
| CN111046746A (en) | License plate detection method and device | |
| Liu et al. | Research on vehicle object detection algorithm based on improved YOLOv3 algorithm | |
| CN110473255B (en) | Ship mooring post positioning method based on multiple grid division | |
| CN112347967B (en) | Pedestrian detection method fusing motion information in complex scene | |
| CN114550134A (en) | Deep learning-based traffic sign detection and identification method | |
| CN111914837A (en) | License plate detection method, device, equipment and storage medium | |
| Biao et al. | Improvements on Vibe algorithm for detecting foreground objects | |
| He et al. | Combining global and local features for detection of license plates in video | |
| Oluchi et al. | Development of a Nigeria vehicle license plate detection system | |
| Voles et al. | Target identification in a complex maritime scene | |
| CN108734158A (en) | A kind of real-time train number identification method and device | |
| KR20220043316A (en) | License plate detection apparatus and method using image warping | |
| CN113128500A (en) | Mask-RCNN-based non-motor vehicle license plate recognition method and system | |
| Hoang et al. | Low cost hardware implementation for traffic sign detection system | |
| Pu | Research on Chinese license plate recognition algorithm based on convolution neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20201127 Address after: Room 206, 2 / F, building C, phase I, Zhongguancun Software Park, No. 8, Dongbei Wangxi Road, Haidian District, Beijing 100094 Applicant after: Canaan Bright Sight Co.,Ltd. Address before: 100094 Zhongguancun Software Park Phase I Building C Block 101, 8 Wangxi Road, Haidian District, Beijing Applicant before: Canaan Creative Co.,Ltd. |