CN109255350B - New energy license plate detection method based on video monitoring - Google Patents
New energy license plate detection method based on video monitoring Download PDFInfo
- Publication number
- CN109255350B CN109255350B CN201810993897.0A CN201810993897A CN109255350B CN 109255350 B CN109255350 B CN 109255350B CN 201810993897 A CN201810993897 A CN 201810993897A CN 109255350 B CN109255350 B CN 109255350B
- Authority
- CN
- China
- Prior art keywords
- license plate
- new energy
- detection
- candidate frames
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/63—Scene text, e.g. street names
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
Abstract
The invention discloses a new energy license plate detection method based on video monitoring, which comprises the following steps: collecting a large number of new energy vehicle pictures containing license plate areas as a data set of a training model, and performing artificial punctuation on each picture in the data set; constructing a new energy license plate detection training neural network, inputting the data set into the neural network for training to obtain a new energy license plate detection model; acquiring new energy vehicle images from the monitoring video, establishing a new energy vehicle image set containing license plate position coordinate information, and naming target vehicle images according to the time sequence of the target vehicles in the video; extracting the characteristics of the image to be detected through a neural network, sending the characteristics into a detection model for judgment, and outputting the confidence coefficient of the license plate image candidate area in the vehicle image according to the model; and finally, calculating the optimal position coordinate point of the license plate according to the obtained license plate candidate region based on the confidence coefficient.
Description
Technical Field
The invention belongs to the technical field of digital security license plate detection, and particularly relates to a new energy license plate detection method based on video monitoring.
Background
The increasing number of motor vehicles brings serious challenges to supervision and management of traffic departments, an intelligent traffic system is particularly important, and a license plate under a monitoring scene is used as an identity mark of a vehicle, so that an automatic license plate detection and recognition (ALPR) technology becomes a key technology in the intelligent traffic system, and characters on the license plate are recognized through various image processing and computer vision technologies, so that each vehicle is recognized. License plate recognition generally comprises license plate detection, character segmentation and recognition; the license plate detection is a basic component of license plate recognition, and the performance of the license plate detection in the aspects of detection precision and running efficiency determines the overall accuracy and processing speed of the whole recognition system to a great extent, so that the performance of an intelligent traffic system in a smart city is influenced, and the license plate detection is an important functional module of a modern intelligent traffic system.
However, due to the influence of factors such as natural environment and monitoring equipment, the acquired license plate images have both clear high-quality images and fuzzy low-quality images, and therefore, for license plate images of different qualities, a detection algorithm capable of correctly extracting license plate information needs to be designed.
At present, the idea of license plate detection and positioning is divided into two stages: coarse detection and accurate positioning. The purpose of rough detection is to extract candidate regions from the input image, and accurate positioning is to screen out the real license plate image from the candidate regions. In the whole positioning process, the position of rough detection is more important, and the rough detection focuses on the recall ratio, namely, the candidate region extracted in the rough detection stage needs to contain all license plate images in the input image as much as possible.
The rough inspection mainly comprises the following methods: (1) the method based on the edge features comprises the following steps: the general process of using edge features to perform coarse license plate inspection is as follows: carrying out gray processing on an input image; extracting vertical edges by using a Sobel operator; carrying out binarization on the edge image; closing the binarized image; according to the method, adjacent vertical edge pixel points are connected by using a closing operation, isolated edge pixel points are eliminated by using an opening operation, but the time complexity of morphological operation is too high to be used for real-time detection of a license plate image; (2) maximum Stable Extremum Region (MSER) algorithm: firstly, carrying out binarization processing on an image by using a series of gray thresholds; obtaining corresponding black areas and white areas for the binary images obtained by each threshold; the region which keeps stable shape in a wider gray threshold range is MSER; the method can realize high positioning accuracy in a relatively simple scene, but is difficult to detect the scene that some areas of the license plate are polluted; (3) method based on color features: the HSV color model can position most license plates with bright colors, but the method is greatly influenced by the quality of the image, an edge detection positioning method is usually adopted after the color method, areas with more vertical edges in the image are positioned through vertical edge detection, and the combination of the two methods can obtain the area where the license plate is located in the vehicle image and judge the real license plate position.
The patent with the application number of CN201710531085.X discloses a license plate detection and recognition method based on a deep learning convolutional neural network, which comprises the steps of firstly using a constructed automatic storage system to classify images containing license plates in the real world, collecting enough license plates and cut character images in different illumination, visual angles and scenes, then using a series of deep neural networks to train license plate detection and recognition, using the cut characters to separately detect and recognize the obtained model, and finally combining the obtained model into a result. The patent with the application number of CN201710187201.0 discloses a license plate detection method based on a partitioned license plate region regression technology, which comprises the following steps: carrying out vehicle detection to obtain a target vehicle, and determining the whole license plate detection area by the target vehicle; dividing the license plate detection area into n small blocks, wherein the small blocks are partially overlapped; fitting by using a first deep neural network learning model to obtain a rough license plate region, and meanwhile, obtaining the credibility that the region is a real license plate; and fusing to obtain a final license plate region according to the relationship between the position of the license plate region and the reliability.
For the vehicle images in the monitoring scene, the problems of high calculation cost, long time consumption, background interference and the like exist in the detection mode of positioning the license plate by adopting two steps of rough detection and fine detection, and meanwhile, the problems are more obvious due to the appearance of the license plate of the new energy vehicle; the new energy automobile license plate is greatly different from the common automobile license plate, the number of the new energy automobile license plate is increased by one digit, compared with the common automobile license plate, the number of the new energy automobile license plate is increased from 5 digits to 6 digits, so that certain inconvenience occurs in traffic management and vehicle management, the number of the new energy automobile license plate is lengthened, the conventional license plate detection technology cannot be completely detected, the detection effect and robustness of the traditional detection method for the blue license plate automobile are improved, and in sum, the detection method capable of effectively detecting the new energy automobile license plate is urgently needed at present.
Disclosure of Invention
The invention aims to solve the problems in the prior art and provides a new energy license plate detection method based on video monitoring.
In order to achieve the purpose, the invention adopts the technical scheme that:
a new energy license plate detection method based on video monitoring comprises the following steps:
s1, collecting a plurality of new energy vehicle pictures containing license plate areas, performing manual accurate punctuation work on the license plate areas of the new energy vehicles in each picture, and establishing a data set of a training model;
s2, constructing a new energy license plate detection training neural network, wherein the neural network comprises a feature extraction module, a feature fusion module and a prediction result output module; inputting the established data set into a neural network to obtain a new energy license plate detection model;
s3, acquiring a frame of image containing a vehicle and a license plate from a monitoring video, detecting and screening the vehicle in the frame of image, screening out a new energy vehicle image as an image to be detected, obtaining a plurality of images to be detected, naming the plurality of images to be detected according to the time sequence of the new energy vehicle in the monitoring video frame sequence, and establishing a new energy vehicle image set to be detected;
s4, detecting license plate regions of the new energy vehicle image to be detected according to the new energy license plate detection model to obtain candidate frames based on different confidence degrees;
and S5, distributing different weights to the candidate frames according to the candidate frames with different confidence degrees, merging the candidate frames, performing accurate regression on the license plate coordinates, generating accurate position coordinate information, and obtaining a final new energy license plate detection result.
The method comprises the steps of training through a certain amount of data to obtain a new energy license plate detection model with a high positioning rate in a monitored environment, extracting a target vehicle image, extracting new energy license plate features by using a convolutional neural network, inputting the new energy license plate features into the trained detection model to generate candidate frames with different confidence degrees, merging based on weight scores by using output frames with the current overlapping area (IOU) of two high-resolution frames being larger than a threshold value of 0.9, and outputting predicted new energy license plate coordinate information through a neural network.
Specifically, in step S1, the manual precise punctuation work includes: sequentially marking four points of the upper left corner, the upper right corner, the lower right corner and the lower left corner of the new energy license plate region clockwise; the data set is eight pieces of numerical information txt format text containing four coordinate points.
Specifically, in step S2:
the feature extraction module extracts the features of different granularities of the license plate layer by layer in a top-down mode; the feature extraction module adopts 10 convolution layers of 4 convolution blocks, the first convolution block and the second convolution block respectively comprise 2 convolution layers and 1 pooling layer, and the third convolution block and the fourth convolution block respectively comprise 3 convolution layers and 1 pooling layer; the rolling block extracts features from the training data set and outputs 3 feature maps;
the feature fusion module fuses the extracted features from bottom to top; the output of the volume block is characterized by fi(i ═ 1, 2, 3), the sizes of the 3 feature maps being 1/16, 1/8, 1/4, respectively, the size of the input image; the formula for feature fusion is as follows:
wherein, giReference feature graph representing fusion, hiIs a new feature map after fusion, and a feature map g of the previous layer is obtainedi-1The size of the graph is amplified by one time through upsampling and then is compared with the feature graph f of the current layeriPerforming fusion, reducing the number of channels and the calculation amount by using conv1 x 1, and fusing the characteristic information by using conv3 x 3; at the last fusion operationUsing a conv3 x 3 to generate a final fused feature map, and transmitting the feature map into an output layer;
the prediction result output module comprises 7 channels and is used for outputting a plurality of candidate frames, confidence values corresponding to the candidate frames and the geometric shapes of the new energy license plate areas; the system comprises 1 channel, a detection frame and a plurality of channels, wherein the 1 channel is used for judging whether a new energy license plate character area is in the detection frame; the 2 channels are used for judging whether the candidate frame belongs to the boundary pixels of the new energy license plate character area frame; the 4 channels are used to output 2 vertex coordinates corresponding to the predicted boundary pixels.
Further, the new energy license plate detection model is as follows:
(plate_box，score)＝plate_detection(src)＝Conv10(src)+network(Feature)+plate_box_pred(side_pix)
the method comprises the steps that a plate _ box represents coordinate positions of all detected new energy license plates, score represents a final confidence score of a corresponding license plate region, plate _ detection () represents a detection positioning model of the new energy license plates, Conv10() represents 10-layer convolutional neural network extraction features, network () represents a feature fusion network, and plate _ box _ pred () represents a coordinate regression.
Specifically, in step S3, the size of the obtained one frame of image is 2592 × 2048 pixels;
the step S3 further includes: all images in the new energy vehicle image set to be monitored are subjected to normalization operation, and the size of all the images is unified to 256 × 256 pixel points.
Specifically, in step S5, the merging of the candidate frames includes the following steps:
s51, sorting all candidate frames output by the detection model according to the confidence score, and selecting the candidate frames corresponding to the highest score and the second highest score respectively;
s52, traversing the rest candidate frames, and if the candidate frames with the overlapping area larger than the set threshold value exist, merging the current two high-score candidate frames based on the weight score;
and S53, continuously selecting two candidate frames with the highest scores from the rest candidate frames, and repeating the step S52 until no candidate frame with the overlapping area of the current two high-score candidate frames larger than the set threshold value of 0.9 exists in the rest candidate frames.
Further, in step S52, the formula of the weight score-based merge is:
m＝weighted_merge(g，p)
mi＝V(g)gi+V(p)pi
V(m)＝V(g)+V(p)
wherein m isiIs one of the coordinates of the m-region frame candidates, v (m) is the confidence score of the frame candidate m, the shape of the quadrangle is determined by means of weighted averaging, and p and g are the two frame candidate regions.
Compared with the prior art, the invention has the beneficial effects that: (1) according to the invention, the target vehicle is detected through the video frame sequence, the time track information of the target vehicle can be tracked and determined, the new energy license plate is quickly and accurately positioned, and an image to be detected is provided for further detecting the license plate; (2) according to the invention, a neural network is constructed, and a large number of new energy vehicle pictures containing license plate areas are collected as training data to be trained, so that a new energy license plate detection model is obtained, the detection model can effectively detect new energy license plate images which are slightly fuzzy and have a certain inclination angle, the calculation complexity is low, the detection time is short, and the most appropriate license plate position coordinate information can be quickly obtained; (3) the method comprises the steps of extracting different 'granularity' features of the license plate layer by layer from top to bottom through a constructed convolutional neural network, performing up-sampling on the features of each layer from bottom to top to fuse the features of each layer, generating candidate frames based on different confidence degrees according to the fused features, distributing different weight values to the corresponding candidate frames according to the different confidence degrees, and finally combining the candidate frames to obtain a new energy license plate detection model with higher positioning rate in a monitoring scene; the geometric shape of the candidate frame can be rotated according to specific practical application, and the robustness is high.
Drawings
Fig. 1 is a schematic flow chart of a new energy license plate detection method based on video monitoring according to the present embodiment;
FIG. 2 is a schematic diagram of a new energy license plate detection training network model in this embodiment;
fig. 3 is a real image of a new energy vehicle image set named in the order of video time frames obtained from a surveillance video in the present embodiment;
FIG. 4 is a diagram of a sample for manual labeling of new energy license plate positions of a training set sample in this embodiment;
FIG. 5 is a block diagram of a candidate frame and a boundary pixel set region in this embodiment, which are obtained by a detection model and based on different confidence levels;
fig. 6 is a real image of a new energy license plate detection result finally obtained in this embodiment.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it is obvious that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, the embodiment provides a new energy license plate detection method based on video monitoring, including: detecting a target vehicle through a video frame sequence, and establishing a vehicle image information set; extracting different 'granularity' features of the license plate layer by layer from top to bottom through a built convolutional neural network, performing up-sampling from bottom to top to fuse the features of each layer, generating a candidate frame based on confidence according to the fused features, finally distributing different weight values to two candidate frames corresponding to the confidence, and combining coordinate position information of four points in a final license plate region; and training a new energy vehicle data set of a small database to obtain a new energy license plate detection model with a high positioning rate in a monitoring scene, and realizing new energy license plate detection and accurate positioning.
A new energy license plate detection method based on video monitoring specifically comprises the following steps:
s1, collecting 5146 new energy vehicle pictures containing license plate regions as a data set for establishing a training new energy license plate detection model, performing manual accurate punctuation work on the license plate regions of the new energy vehicles in each picture, performing punctuation operation on new energy license plate vertexes in a clockwise direction, and establishing the data set of the training model, wherein the data set is shown in FIG. 2;
s2, constructing a new energy license plate detection training neural network, wherein the neural network comprises a feature extraction module, a feature fusion module and a prediction result output module; inputting the established data set into a neural network to obtain a new energy license plate detection model, as shown in fig. 3;
s3, acquiring a frame of image containing a vehicle and a license plate from a monitoring video, detecting and screening the vehicle in the frame of image, screening out a new energy vehicle image as an image to be detected, obtaining a plurality of images to be detected, naming the plurality of images to be detected according to the time sequence of the new energy vehicle in the monitoring video frame sequence, and establishing a new energy vehicle image set to be detected, as shown in FIG. 4;
s4, detecting license plate regions of the new energy vehicle image to be detected according to the new energy license plate detection model to obtain candidate frames and boundary pixel set regions based on different confidence degrees, as shown in FIG. 5;
and S5, distributing different weights to the candidate frames according to the candidate frames with different confidence degrees, merging the candidate frames, performing accurate regression on the license plate coordinates, generating accurate position coordinate information, and obtaining a final new energy license plate detection result, wherein the final new energy license plate detection result is shown in FIG. 6.
Specifically, in step S1, the manual precise punctuation work includes: sequentially marking four points of the upper left corner, the upper right corner, the lower right corner and the lower left corner of the new energy license plate region clockwise; the data set is eight pieces of numerical information txt format text containing four coordinate points.
Specifically, in step S2, the feature extraction module extracts the features of different particle sizes of the license plate layer by layer in a top-down manner; the feature extraction module adopts 10 convolution layers of 4 convolution blocks, the first convolution block and the second convolution block respectively comprise 2 convolution layers and 1 pooling layer, and the third convolution block and the fourth convolution block respectively comprise 3 convolution layers and 1 pooling layer; the rolling block extracts features from the training data set and outputs 3 feature maps;
the feature fusion module fuses the extracted features from bottom to top; the output of the volume block is characterized by fi(i ═ 1, 2, 3), the sizes of the 3 feature maps being 1/16, 1/8, 1/4, respectively, the size of the input image; the formula for feature fusion is as follows:
wherein, giReference feature graph representing fusion, hiIs a new feature map after fusion, and a feature map g of the previous layer is obtainedi-1The size of the graph is amplified by one time through upsampling and then is compared with the feature graph f of the current layeriPerforming fusion, reducing the number of channels and the calculation amount by using conv1 x 1, and fusing the characteristic information by using conv3 x 3; in the final blend operation, a conv3 x 3 is used to generate the final blended feature map and passed into the output layer;
the prediction result output module comprises 7 channels and is used for outputting a plurality of candidate frames, confidence values corresponding to the candidate frames and the geometric shapes of the new energy license plate areas; the system comprises 1 channel, a detection frame and a plurality of channels, wherein the 1 channel is used for judging whether a new energy license plate character area is in the detection frame; the 2 channels are used for judging whether the candidate frame belongs to the boundary pixels of the new energy license plate character area frame; the 4 channels are used to output 2 vertex coordinates corresponding to the predicted boundary pixels.
The activation function in the feature of extracting the license plate picture by adopting the ten-layer convolutional neural network adopts a sigmoid activation function and a Relu activation function, and the formula is as follows:
ReLU(x)＝max(0，x)
the super parameters of the learning rate, the scaling ratio, the weight attenuation rate and the learning attenuation rate are adjusted manually and then are as follows: 0.001, 0.2, 0.0005 and 0.1;
further, the new energy license plate detection model is as follows:
(plate_box，score)＝plate_detection(src)＝Conv10(src)+network(Feature)+plate_box_pred(side_pix)
the method comprises the steps that a plate _ box represents coordinate positions of all detected new energy license plates, score represents a final confidence score of a corresponding license plate region, plate _ detection () represents a detection positioning model of the new energy license plates, Conv10() represents 10-layer convolutional neural network extraction features, network () represents a feature fusion network, and plate _ box _ pred () represents a coordinate regressor and is used for obtaining the license plate regions and the final score.
Specifically, in step S3, specifically, in step S3, the size of the obtained one frame of image is 2592 × 2048 pixels;
the step S3 further includes: all images in the new energy vehicle image set to be monitored are subjected to normalization operation, and the size of all the images is unified to 256 × 256 pixel points.
Further, in step S3, feature maps of different scales are extracted according to each layer network, which are 128 × 3, 64 × 3, and 32 × 3 in sequence; and (3) replaying the image into the half size of the original image through the feature graph output by the feature extraction module and the 2 x 2 upper pooling layer from bottom to top in sequence, and finally regressing to obtain a candidate frame of the license plate detection area through the operation that the concat layer is connected with the feature graph.
Specifically, in step S5, the merging of the candidate frames includes the following steps:
s51, sorting all candidate frames output by the detection model according to the confidence score, and selecting the candidate frames corresponding to the highest score and the second highest score respectively;
s52, traversing the rest candidate frames, and if the candidate frames with the overlapping area larger than the set threshold value exist, merging the current two high-score candidate frames based on the weight score;
and S53, continuously selecting two candidate frames with the highest scores from the rest candidate frames, and repeating the step S52 until no candidate frame with the overlapping area of the current two high-score candidate frames larger than the set threshold value of 0.9 exists in the rest candidate frames.
Further, in step S52, the formula of the weight score-based merge is:
m＝weighted_merge(g，p)
mi＝V(g)gi+V(p)pi
V(m)＝V(g)+V(p)
wherein m isiIs one of the coordinates of the m-region frame candidates, v (m) is the confidence score of the frame candidate m, the shape of the quadrangle is determined by means of weighted averaging, and p and g are the two frame candidate regions.
Further, in step S5, combining the coordinates of the quadrangles of the new energy license plate candidate frames, performing weighted average through the confidence scores of the two given quadrangle candidate frames, sorting all candidate frames output by the detection model according to the confidence scores, and selecting the highest score, the second highest score and the frames corresponding to the highest score and the second highest score; and then, traversing the rest of the frames, and if the overlapping area (IOU) of the current two high-level frames is larger than the output frame with the threshold value threshold being 0.9, merging based on the weight scores, wherein each window gets a score by being distinguished from the standard NMS program, a sliding window causes the condition that many windows are included or most of the windows are crossed, and only the candidate frame with the highest confidence score is selected as the final result to be output.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (6)
1. A new energy license plate detection method based on video monitoring is characterized by comprising the following steps:
s1, collecting a plurality of new energy vehicle pictures containing license plate areas, performing manual accurate punctuation work on the license plate areas of the new energy vehicles in each picture, and establishing a data set of a training model;
s2, constructing a new energy license plate detection training neural network, wherein the neural network comprises a feature extraction module, a feature fusion module and a prediction result output module; inputting the established data set into a neural network to obtain a new energy license plate detection model;
s3, acquiring a frame of image containing a vehicle and a license plate from a monitoring video, detecting and screening the vehicle in the frame of image, screening out a new energy vehicle image as an image to be detected, obtaining a plurality of images to be detected, naming the plurality of images to be detected according to the time sequence of the new energy vehicle in the monitoring video frame sequence, and establishing a new energy vehicle image set to be detected;
s4, detecting license plate regions of the new energy vehicle image to be detected according to the new energy license plate detection model to obtain candidate frames based on different confidence degrees;
s5, distributing different weights to the candidate frames according to the candidate frames with different confidence degrees, merging the candidate frames, performing accurate regression on license plate coordinates, generating accurate position coordinate information, and obtaining a final new energy license plate detection result;
in the step S2, the feature extraction module extracts the features of different granularity of the license plate layer by layer in a top-down mode; the feature extraction module adopts 10 convolution layers of 4 convolution blocks, the first convolution block and the second convolution block respectively comprise 2 convolution layers and 1 pooling layer, and the third convolution block and the fourth convolution block respectively comprise 3 convolution layers and 1 pooling layer; the rolling block extracts features from the training data set and outputs 3 feature maps;
the feature fusion module fuses the extracted features from bottom to top; the output of the volume block is characterized by fiWherein i is 1, 2, 3, and the sizes of the 3 feature maps are 1/16, 1/8, 1/4 of the size of the input image; the formula for feature fusion is as follows:
wherein, giReference feature graph representing fusion, hiIs a new feature map after fusion, and a feature map g of the previous layer is obtainedi-1The size of the graph is amplified by one time through upsampling and then is compared with the feature graph f of the current layeriPerforming fusion, reducing the number of channels and the calculation amount by using conv1 x 1, and fusing the characteristic information by using conv3 x 3; in the final blend operation, a conv3 x 3 is used to generate the final blended feature map and passed into the output layer;
the prediction result output module comprises 7 channels and is used for outputting a plurality of candidate frames, confidence values corresponding to the candidate frames and the geometric shapes of the new energy license plate areas; the system comprises 1 channel, a detection frame and a plurality of channels, wherein the 1 channel is used for judging whether a new energy license plate character area is in the detection frame; the 2 channels are used for judging whether the candidate frame belongs to the boundary pixels of the new energy license plate character area frame; the 4 channels are used to output 2 vertex coordinates corresponding to the predicted boundary pixels.
2. The method for detecting the license plate of the new energy source based on the video monitoring as claimed in claim 1, wherein in the step S1, the manual precise punctuation work comprises: sequentially marking four points of the upper left corner, the upper right corner, the lower right corner and the lower left corner of the new energy license plate region clockwise; the data set is eight pieces of numerical information txt format text containing four coordinate points.
3. The method according to claim 1, wherein in step S2, the new energy license plate detection model is:
(plate_box,score)＝plate_detection(src)＝Conv10(src)+network(Feature)+plate_box_pred(side_pix)
the method comprises the steps that src represents a new energy vehicle picture containing a license plate region, plate _ box represents the coordinate positions of all detected new energy license plates, score represents the final confidence score of the corresponding license plate region, plate _ detection () represents a detection positioning model of the new energy license plates, Conv10() represents 10-layer convolutional neural network extraction features, network () represents a feature fusion network, and plate _ box _ pred () represents a coordinate regressor.
4. The method according to claim 1, wherein in step S3, the size of the acquired frame of image is 2592 × 2048 pixels;
the step S3 further includes: all images in the new energy vehicle image set to be monitored are subjected to normalization operation, and the size of all the images is unified to 256 × 256 pixel points.
5. The method according to claim 1, wherein in step S5, the merging of the candidate frames comprises the following steps:
s51, sorting all candidate frames output by the detection model according to the confidence score, and selecting the candidate frames corresponding to the highest score and the second highest score respectively;
s52, traversing the rest candidate frames, and if the candidate frames with the overlapping area larger than the set threshold value exist, merging the current two high-score candidate frames based on the weight score;
s53, two candidate frames with the highest score are selected from the remaining candidate frames, and step S52 is repeated until there is no candidate frame with an overlapping area with the current two high-score candidate frames larger than the set threshold value.
6. The method according to claim 5, wherein in step S52, the formula based on the weight score combination is:
m＝weighted_merge(g,p)
mi＝V(g)gi+V(p)pi
V(m)＝V(g)+V(p)
wherein m isiIs one of the coordinates of the m-region frame candidates, v (m) is the confidence score of the frame candidate m, the shape of the quadrangle is determined by means of weighted averaging, and p and g are the two frame candidate regions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810993897.0A CN109255350B (en) | 2018-08-29 | 2018-08-29 | New energy license plate detection method based on video monitoring |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810993897.0A CN109255350B (en) | 2018-08-29 | 2018-08-29 | New energy license plate detection method based on video monitoring |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109255350A CN109255350A (en) | 2019-01-22 |
| CN109255350B true CN109255350B (en) | 2021-12-10 |
Family
ID=65048841
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810993897.0A Active CN109255350B (en) | 2018-08-29 | 2018-08-29 | New energy license plate detection method based on video monitoring |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109255350B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110047102A (en) * | 2019-04-18 | 2019-07-23 | 北京字节跳动网络技术有限公司 | Methods, devices and systems for output information |
| CN110135412B (en) * | 2019-04-30 | 2021-05-11 | 北京邮电大学 | Business card recognition method and device |
| CN110705548A (en) * | 2019-09-09 | 2020-01-17 | 创新奇智(南京)科技有限公司 | Coarse-to-fine license plate detection algorithm and system thereof |
| CN111126383A (en) * | 2019-12-06 | 2020-05-08 | 广州烽火众智数字技术有限公司 | License plate detection method, system, device and storage medium |
| CN111127440A (en) * | 2019-12-22 | 2020-05-08 | 上海眼控科技股份有限公司 | Tire specification detection method and device, electronic equipment and readable storage medium |
| CN111797829A (en) * | 2020-06-24 | 2020-10-20 | 浙江大华技术股份有限公司 | License plate detection method and device, electronic equipment and storage medium |
| CN112289022B (en) * | 2020-09-29 | 2022-02-25 | 西安电子科技大学 | Black smoke vehicle detection and judgment method and system based on space-time background comparison |
| CN112215123B (en) * | 2020-10-09 | 2022-10-25 | 腾讯科技（深圳）有限公司 | Target detection method, device and storage medium |
| CN112232237B (en) * | 2020-10-20 | 2024-03-12 | 城云科技（中国）有限公司 | Method, system, computer device and storage medium for monitoring vehicle flow |
| CN113283349A (en) * | 2021-05-28 | 2021-08-20 | 中国公路工程咨询集团有限公司 | Traffic infrastructure construction target monitoring system and method based on target anchor frame optimization strategy |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106022232A (en) * | 2016-05-12 | 2016-10-12 | 成都新舟锐视科技有限公司 | License plate detection method based on deep learning |
| CN108009543A (en) * | 2017-11-29 | 2018-05-08 | 深圳市华尊科技股份有限公司 | A kind of licence plate recognition method and device |
| CN108304808A (en) * | 2018-02-06 | 2018-07-20 | 广东顺德西安交通大学研究院 | A kind of monitor video method for checking object based on space time information Yu depth network |
-
2018
- 2018-08-29 CN CN201810993897.0A patent/CN109255350B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106022232A (en) * | 2016-05-12 | 2016-10-12 | 成都新舟锐视科技有限公司 | License plate detection method based on deep learning |
| CN108009543A (en) * | 2017-11-29 | 2018-05-08 | 深圳市华尊科技股份有限公司 | A kind of licence plate recognition method and device |
| CN108304808A (en) * | 2018-02-06 | 2018-07-20 | 广东顺德西安交通大学研究院 | A kind of monitor video method for checking object based on space time information Yu depth network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109255350A (en) | 2019-01-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109255350B (en) | New energy license plate detection method based on video monitoring | |
| CN109977812B (en) | Vehicle-mounted video target detection method based on deep learning | |
| CN106599792B (en) | Method for detecting hand driving violation behavior | |
| CN104978567B (en) | Vehicle checking method based on scene classification | |
| CN107122777A (en) | A kind of vehicle analysis system and analysis method based on video file | |
| CN110866430B (en) | License plate recognition method and device | |
| CN104077577A (en) | Trademark detection method based on convolutional neural network | |
| CN109086687A (en) | The traffic sign recognition method of HOG-MBLBP fusion feature based on PCA dimensionality reduction | |
| CN106529532A (en) | License plate identification system based on integral feature channels and gray projection | |
| CN104050450A (en) | Vehicle license plate recognition method based on video | |
| CN101266654A (en) | Image text location method and device based on connective component and support vector machine | |
| CN103824081A (en) | Method for detecting rapid robustness traffic signs on outdoor bad illumination condition | |
| CN111027475A (en) | Real-time traffic signal lamp identification method based on vision | |
| CN117094914B (en) | Smart city road monitoring system based on computer vision | |
| CN110689003A (en) | Low-illumination imaging license plate recognition method and system, computer equipment and storage medium | |
| CN110969164A (en) | Low-illumination imaging license plate recognition method and device based on deep learning end-to-end | |
| CN112528917A (en) | Zebra crossing region identification method and device, electronic equipment and storage medium | |
| CN114723709A (en) | Tunnel disease detection method and device and electronic equipment | |
| CN112613392A (en) | Lane line detection method, device and system based on semantic segmentation and storage medium | |
| CN108073940A (en) | A kind of method of 3D object instance object detections in unstructured moving grids | |
| Ghahremannezhad et al. | Automatic road detection in traffic videos | |
| CN113657305A (en) | Video-based intelligent detection method for blackness level of black smoke vehicle and Ringelmann | |
| CN103680145A (en) | Automatic pedestrian and vehicle recognition method based on local image characteristics | |
| CN107392115B (en) | Traffic sign identification method based on hierarchical feature extraction | |
| Hommos et al. | Hd Qatari ANPR system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |