CN109034171B - Method and device for detecting unlicensed vehicles in video stream - Google Patents
Method and device for detecting unlicensed vehicles in video stream Download PDFInfo
- Publication number
- CN109034171B CN109034171B CN201810821013.3A CN201810821013A CN109034171B CN 109034171 B CN109034171 B CN 109034171B CN 201810821013 A CN201810821013 A CN 201810821013A CN 109034171 B CN109034171 B CN 109034171B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- classification model
- license plate
- background
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07B—TICKET-ISSUING APPARATUS; FARE-REGISTERING APPARATUS; FRANKING APPARATUS
- G07B15/00—Arrangements or apparatus for collecting fares, tolls or entrance fees at one or more control points
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Abstract
The invention provides a method and a device for detecting a unlicensed vehicle in a video stream. The method for detecting the unlicensed vehicle comprises the following steps: obtaining a training sample set for training a linear SVM classification model from a plurality of video segments of the video stream; extracting HOG characteristics of all positive and negative samples from the obtained training sample set; performing off-line training on the SVM classification model by using the extracted HOG characteristics to obtain an initially optimized classification model; the video information in the video stream is used for training the initially optimized classification model again on line to obtain an optimal classification model; and carrying out vehicle detection on the video stream by utilizing the optimal classification model. The method and the device for detecting the unlicensed vehicle not only can provide help for the automatic charging of the entrance and the exit of the parking lot, but also can effectively adapt to the complex changes of various traffic scenes.
Description
Technical Field
The invention relates to an intelligent traffic technology, in particular to a method and a device for detecting a unlicensed vehicle in a video stream.
Background
At present, two modes are mainly adopted for managing vehicles at the entrance and the exit of a parking lot. One mode is a ground sense triggered snapshot mode. Ground induction triggering needs to be carried out by laying ground induction coils on the ground, so that construction is inconvenient and cost is high. Moreover, some scenarios do not meet the requirements for triggering. Particularly, any ironwork can be captured through the coil area, so that more mistaken captures can be caused, for example, bicycles, tricycles, motorcycles and the like can cause disorder of the entrance and exit of the parking lot, more interference information can be caused, and the difficulty is increased for the processing of a later-period charging system. The other mode is a mode of manually controlling the capturing and the rod lifting. In a parking lot management system, the ultimate aim is to carry out unattended operation and completely accessible toll payment, and the scheme runs counter to the aim.
Disclosure of Invention
The invention provides a method and a device for detecting a unlicensed vehicle in a video stream, aiming at detecting the unlicensed vehicle at an entrance and an exit of a parking lot, providing information for automatic passing of an entering vehicle and matching of a vehicle without license plate number information on the exit, and reducing the manual workload.
According to one aspect of the invention, a method of unlicensed vehicle detection in a video stream is provided. The method for detecting the unlicensed vehicle comprises the following steps of: obtaining a training sample set for training a linear Support Vector Machine (SVM) classification model from a plurality of video segments of the video stream, wherein the training sample set comprises positive samples serving as vehicle samples and negative samples serving as non-vehicle samples; extracting directional Gradient Histogram (HOG) features of all positive and negative samples from the obtained training sample set; performing off-line training on the SVM classification model by using the extracted HOG characteristics to obtain an initially optimized classification model; the video information in the video stream is used for training the initially optimized classification model again on line to obtain an optimal classification model; and carrying out vehicle detection on the video stream by utilizing the optimal classification model.
In an improved embodiment of the present invention, the step of extracting the HOG features of all positive and negative samples from the obtained training sample set includes: graying the sample image; normalizing the color space of the image by adopting a gamma correction method; calculating a gradient for each pixel in the image; dividing the image into cells; counting the gradient histogram of each unit, namely obtaining a descriptor of each unit; combining the description substrings of all the units in a block to obtain the HOG characteristics of the block; and connecting HOG features of all blocks in the image in series to obtain a feature vector for training.
In an improved embodiment of the present invention, the step of training the initially optimized classification model online again using the video information in the video stream includes: according to the analysis of the video information, extracting a negative sample from a scene without a vehicle, and extracting a region around a license plate as a positive sample by taking the position of the license plate as a reference; retraining the initially optimized classification model using the extracted positive and negative samples.
In an improved embodiment of the present invention, the step of performing vehicle detection on the video stream by using an optimal classification model includes: identifying the license plate and judging whether the license plate exists or not; under the condition that a license plate result exists, automatically intercepting a vehicle head image by taking the position of the license plate as a standard, and outputting the license plate result; under the condition of no license plate result, judging whether a background model is established; under the condition that a background model is not established, local Binary Pattern (LBP) characteristics are applied to carry out background modeling; under the condition that a background model is established, judging whether a vehicle without a license plate result exists by using a classification model and the background model; and under the condition that the vehicle without the license plate exists, outputting the detected vehicle as the unlicensed vehicle.
In an improved embodiment of the invention, the SVM classification model is a two-classification model with respect to the target and the background.
According to another aspect of the invention, an apparatus for detecting a unlicensed vehicle in a video stream is provided. The unlicensed vehicle detecting device includes: a training sample set obtaining module, configured to obtain a training sample set for training a linear SVM classification model from a plurality of video segments of the video stream, where the training sample set includes positive samples as vehicle samples and negative samples as non-vehicle samples; the HOG feature extraction module is used for extracting HOG features of all positive and negative samples from the obtained training sample set; the offline training module is used for performing offline training on the SVM classification model by using the extracted HOG characteristics to obtain an initially optimized classification model; the online training module is used for training the initially optimized classification model again online by utilizing the video information in the video stream to obtain an optimal classification model; and the detection module is used for detecting the vehicle in the video stream by utilizing the optimal classification model.
In an improved embodiment of the present invention, the HOG feature extraction module is further configured to: graying the sample image; normalizing the color space of the image by adopting a gamma correction method; calculating a gradient for each pixel in the image; dividing the image into cells; counting the gradient histogram of each unit, namely obtaining a descriptor of each unit; combining the description substrings of all the units in a block to obtain the HOG characteristics of the block; and connecting HOG features of all blocks in the image in series to obtain a feature vector for training.
In an improved embodiment of the present invention, the online training module is further configured to: according to the analysis of the video information, extracting a negative sample from a scene without a vehicle, and extracting a region around a license plate as a positive sample by taking the position of the license plate as a reference; retraining the initially optimized classification model using the extracted positive and negative samples.
In an improved embodiment of the present invention, the detection module is further configured to: identifying the license plate and judging whether the license plate exists or not; under the condition that a license plate result exists, automatically intercepting a vehicle head image by taking the position of the license plate as a standard, and outputting the license plate result; under the condition of no license plate result, judging whether a background model is established; under the condition that a background model is not established, the LBP characteristics are used for background modeling; under the condition that a background model is established, judging whether a vehicle without a license plate result exists by using the classification model and the background model; and under the condition that the vehicle without the license plate result exists, outputting the detected vehicle as the unlicensed vehicle.
In an improved embodiment of the invention, the SVM classification model is a binary classification model with respect to the target and the background.
According to another aspect of the present invention, there is also provided a computer storage medium having stored therein computer program code which, when executed, implements a method of detecting a unlicensed vehicle in a video stream as claimed in any one of claims 1 to 5.
In the method and the device for detecting the unlicensed vehicles in the video stream, the unlicensed vehicles in the video are detected by using an HOG + SVM target detection method and are fused with the license plate result, so that the correct output of the unlicensed vehicles is achieved, and the help is provided for the automatic charging of the entrance and the exit of the parking lot. Meanwhile, an online training method is adopted, so that the model is further enhanced, and the method can effectively adapt to the complex changes of various traffic scenes, such as the environmental conditions of night illumination, moving shadows, severe weather and the like.
It will be appreciated by those skilled in the art that the objects and advantages that can be achieved with the present invention are not limited to the specific details set forth above, and that these and other objects that can be achieved with the present invention will be more clearly understood from the detailed description that follows.
Also, it is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, as claimed.
Drawings
Further objects, features and advantages of the present invention will become apparent from the following description of embodiments of the invention, with reference to the accompanying drawings, in which:
FIG. 1 is a flow chart of a method of unlicensed vehicle detection according to an exemplary embodiment of the present invention;
fig. 2 is a block diagram of a unlicensed vehicle detection apparatus according to an exemplary embodiment of the present invention.
Detailed Description
Preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While the preferred embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
The invention provides a method and a device for detecting a unlicensed vehicle in a video stream based on HOG + SVM aiming at the defects of the existing parking lot entrance and exit management. The method comprises the steps of detecting the unlicensed vehicles in the video by using an HOG + SVM target detection method, fusing a license plate result with the unlicensed vehicles, achieving correct output of the unlicensed information vehicles, and providing help for automatic charging at an entrance and an exit of a parking lot. Meanwhile, an online training method is adopted, so that the model is further enhanced, and the method can effectively adapt to the complex changes of various traffic scenes, such as the environmental conditions of night illumination, moving shadows, severe weather and the like.
In order that the technical solutions of the present invention will be more clearly understood, the present invention will be described in detail below with reference to the accompanying drawings in conjunction with specific embodiments.
FIG. 1 shows a flow diagram of a method of unlicensed vehicle detection according to an exemplary embodiment of the present invention. As shown in fig. 1, the unlicensed vehicle detection method includes a training sample set obtaining step S1, an HOG feature extraction step S2, an offline training step S3, an online training step S4, and a detection step S5. The respective steps are specifically described below.
Step S1: a training sample set for training a linear SVM classification model is obtained from a plurality of video segments of an acquired video stream. The labeling of the training positive sample (i.e., the vehicle sample) is performed based on the vehicle sample collected by the camera at the entrance/exit of the parking lot. The annotated positive samples include vehicle samples of various poses and lighting conditions. According to the scene characteristics of the scene shot in the scene, the marked area is mainly selected from the car face part, not the whole car body, and comprises a partial background area. In addition, a method of sliding a window is used for extracting negative samples (namely, samples of non-vehicles, which can comprise three-wheeled vehicles, pedestrians, motorcycles, buildings, the ground and any objects except vehicles appearing on a shot picture) by using a jpeg flow collected by a camera when the non-vehicles pass through the parking lot entrance and exit. Thus, the obtained training sample set comprises positive samples as vehicle samples and negative samples as non-vehicle samples.
Step S2: HOG features of all positive and negative samples are extracted from the training sample set obtained in step S1. First, a sample image is grayed, and here, the image needs to be regarded as a three-dimensional image. Then, the grayed image is normalized in color space by a gamma correction method. The purpose of normalization is to adjust the contrast of the image, reduce the influence caused by local shadows and illumination changes of the image, and simultaneously suppress the interference of noise. Next, a gradient is calculated for each pixel in the normalized image, including the gradient magnitude and direction. This is done to capture contour information and further attenuate the disturbance of illumination. Next, the image is divided into small cells (cells), for example, each cell being 8*8 pixels in size. Then, the gradient histogram of each cell is counted, i.e., a descriptor (descriptor) of each cell is obtained. Then, every predetermined number of cells are grouped into a block (block), and the descriptors of all cells in the block are connected in series to obtain the HOG feature descriptor of the block. And finally, connecting HOG feature descriptors of all blocks in the image in series to obtain a feature vector for training. In consideration of time and effect, the inventor selects the size of a training sample to be 128 x 64 pixels, the size of each cell is 8*8 pixels, 2*2 cells form a block, the selection of the angle range is 0-360 degrees, 18 equal divisions are carried out, and finally 7560-dimensional feature vectors are obtained.
And step S3: and (3) performing off-line training on the SVM classification model by using the HOG characteristics extracted in the step (S2) to obtain an initially optimized classification model. Here, the SVM classification model is a linear binary classification model. In this step, first, parameters including parameters, iteration times, errors, and the like of the SVM classification model are set. Then, based on the set parameters, the HOG features of a certain number (for example, 1000 to 10000) of training samples randomly extracted from the training sample set are input into the SVM classification model for training, and an initially optimized classification model is obtained.
And step S4: and (4) re-training the initially optimized classification model on line by using video information in the video stream to obtain an optimal classification model. Specifically, first, according to the analysis of the video information, a negative sample is extracted from a scene when there is no vehicle, and the region around the license plate is extracted as a positive sample with the position of the license plate as a reference. Then, the initially optimized classification model is retrained by using the extracted positive and negative samples, so that the detection rate can be improved and the false detection rate can be reduced.
Step S5: and (5) carrying out vehicle detection on the video stream by using the optimal classification model obtained in the step (S4). In the step, firstly, license plate recognition is carried out, and whether a license plate exists or not is judged. And under the condition that a license plate result exists, automatically intercepting the vehicle head image by taking the position of the license plate as a standard, and outputting the license plate result. However, in the case of no license plate result, it is determined whether a background model has been established.
On the other hand, if the background model is established, the classification model and the background model are used for judging whether vehicles without license plate results exist. When there is a vehicle with no license plate result, the detected vehicle is output as a non-license vehicle.
Therefore, in the detection step S5, the vehicle detection result and the license plate result are fused. If a vehicle passes through the classification model and the background modeling, the license plate result needs to be preferentially output for the vehicle with the license plate, and if the license plate result does not exist in the process, the detected vehicle can be output as the vehicle without the license plate. Therefore, the fact that a vehicle comes in and has a license plate number is guaranteed, a license plate number result is output, and a vehicle picture is captured without the license plate number.
By adopting the method for detecting the unlicensed vehicles, license plate recognition results and continuous video streams at the entrance and the exit of the parking lot can be given, and output of the unlicensed vehicles is completed. Meanwhile, vehicles without license plates and with license plate numbers can be automatically lifted and released at the entrance of the parking lot. In addition, the vehicle with the license plate number can be further automatically charged through license plate number matching at the exit of the parking lot, and for the vehicle without the license plate number, the vehicle without the license plate number information at the entrance is searched for manual searching or automatic matching is carried out, and the entrance information is found for charging.
The invention also provides a device for realizing the method for detecting the unlicensed vehicle. Fig. 2 shows a block diagram of the unlicensed vehicle detection apparatus 100 according to an exemplary embodiment of the present invention. As shown in fig. 2, the unlicensed vehicle detection apparatus 100 includes a training sample set obtaining module 101, an HOG feature extraction module 102, an offline training module 103, an online training module 104, and a detection module 105.
The training sample set obtaining module 101 is configured to obtain a training sample set for training a linear SVM classification model from a plurality of video segments of a video stream. The labeling of the training positive sample (i.e., the vehicle sample) is performed based on the vehicle sample collected by the camera at the entrance/exit of the parking lot. The annotated positive samples include vehicle samples of various poses and lighting conditions. According to the scene characteristics of the scene shot in the scene, the marked area is mainly selected from the car face part, not the whole car body, and comprises a partial background area. In addition, a method of sliding a window is used for extracting negative samples (namely samples of non-vehicles, which can comprise three-wheel vehicles, pedestrians, motorcycles, buildings, the ground and any objects except vehicles appearing on a shot picture) by utilizing a jpeg stream collected by a camera at the entrance and exit of the parking lot when the non-vehicles pass by. Thus, the obtained training sample set comprises positive samples as vehicle samples and negative samples as non-vehicle samples.
The HOG feature extraction module 102 is configured to extract HOG features of all positive and negative samples from the training sample set obtained by the training sample set obtaining module 101. First, a sample image is grayed, and here, the image needs to be regarded as a three-dimensional image. Then, the grayed image is normalized in color space by a gamma correction method. The normalization is to adjust the contrast of the image, reduce the influence caused by the local shadow and illumination change of the image, and simultaneously suppress the noise interference. Next, a gradient is calculated for each pixel in the normalized image, including the gradient magnitude and direction. This is done to capture contour information and further attenuate the disturbance of illumination. Next, the image is divided into small cells (cells), for example, each cell being 8*8 pixels in size. Then, the gradient histogram of each cell is counted, i.e., a descriptor (descriptor) of each cell is obtained. Then, every predetermined number of cells are grouped into a block (block), and the descriptors of all cells in a block are concatenated to obtain the HOG feature descriptor of the block. And finally, connecting HOG feature descriptors of all blocks in the image in series to obtain a feature vector for training. Considering the comprehensive time and effect, the inventor selects the size of a training sample to be 128 × 64 pixels, the size of each cell is 8*8 pixels, 2*2 cells form a block, the angle range is selected to be 0-360 degrees, 18 equal divisions, and finally 7560-dimensional feature vectors are obtained.
The offline training module 103 is configured to perform offline training on the SVM classification model by using the HOG features extracted by the HOG feature extraction module 102, so as to obtain an initially optimized classification model. Here, the SVM classification model is a linear binary classification model. In this step, first, parameters including parameters of the SVM classification model, the number of iterations, errors, and the like are set. Then, based on the set parameters, the HOG features of a certain number (for example, 1000 to 10000) of training samples randomly extracted from the training sample set are input into the SVM classification model for training, and an initially optimized classification model is obtained.
The online training module 104 is configured to train the initially optimized classification model again online using the video information in the video stream to obtain an optimal classification model. Specifically, first, according to the analysis of the video information, a negative sample is extracted from a scene when there is no vehicle, and the region around the license plate is extracted as a positive sample with the position of the license plate as a reference. Then, the initially optimized classification model is retrained by using the extracted positive and negative samples, so that the detection rate can be improved and the false detection rate can be reduced.
The detection module 105 is configured to perform vehicle detection on the video stream by using the optimal classification model obtained by the online training module 104. First, the detection module 105 performs license plate recognition and determines whether there is a license plate result. And under the condition that a license plate result exists, automatically intercepting the vehicle head image by taking the position of the license plate as a standard, and outputting the license plate result. However, in the case of no license plate result, it is determined whether a background model has been established.
If a background model is not established, the detection module 105 uses LBP features for background modeling. Specifically, a small area is selected near the ideal vehicle output position, and the LBP feature is used for background modeling. In other words, a simulated trigger region is established, and when an object passes through the region, the LBP characteristic value deviates from the background value, and when the object leaves, the value in the background state can be restored. The LBP is extracted by first converting the original image into an LBP map, then counting the LBP histogram of the LBP map, and representing the original image by the histogram in the form of the vector. The LBP characteristics can overcome the interference of illumination, and the triggering is sensitive and accurate. The purpose of this is two: i) When a vehicle passes by, processing multiple frames of pictures, and obtaining multiple detection windows by using a classification model, but for practical application, the same vehicle only needs to pass by one result, namely the vehicle is required to be divided into a process from appearance to disappearance, and in the process, the result is output at a fixed position as far as possible; ii) the result is output only when the analog trigger area changes, and the result output at a fixed position is ensured as much as possible. The vehicle appearance to disappearance segmentation process using LBP feature background modeling is as follows: 1) If there are continuous multiframes without motion in the analog trigger area, extracting the LBP histogram of the current area to show that the background is established. If no motion exists, continuously updating the background characteristic value; 2) If the motion is detected, calculating the distance between LBP histograms of the foreground and the background of the current simulation trigger area, if the distance is greater than a fixed threshold, considering that the current area is covered, possibly the shielding of a vehicle or the shielding of other interference, and judging whether the vehicle passes through or not according to the result of a single-frame picture; 3) And continuously calculating the distance of the LBP histograms of the foreground and the background of the current analog trigger area, and if the distance lasts for a plurality of frames and is smaller than a fixed threshold value, considering that the current vehicle leaves. For example, when a vehicle comes in, the distance between the LBP histogram of the analog trigger area calculated at the current frame and the background histogram may be larger than a specified threshold, which represents that the area is covered. If the vehicle walks, the distance between the LBP histogram of the simulation trigger area and the background histogram is smaller than a specified threshold value, which indicates that the background is recovered, and the triggering of the next vehicle is allowed only if the background state is recovered.
On the other hand, if the background model is established, whether the vehicle without the license plate result exists is judged by using the classification model and the background model. If there is a vehicle with no license plate, the detected vehicle is output as a non-license plate.
As an example, the unlicensed vehicle detection apparatus of the present invention may comprise a memory having computer program code stored thereon and a processor, which when configured to execute the computer program code stored on the memory, implements steps S1-S5 of the unlicensed vehicle detection method as previously described.
In conclusion, the method and the device for detecting the unlicensed vehicles in the embodiment use an HOG + SVM target detection method to detect the unlicensed vehicles in the video, and fuse the detection result with the license plate result, so that the vehicles with the unlicensed information can be correctly output, and help is provided for automatic charging at the entrance and the exit of the parking lot. Meanwhile, an online training method is adopted, so that the model is further enhanced, and the method can effectively adapt to the complex changes of various traffic scenes, such as the environmental conditions of night illumination, moving shadows, severe weather and the like.
Portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following technologies, which are well known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
The logic and/or steps represented in the flowcharts or otherwise described herein, such as an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.
Features that are described and/or illustrated above with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
Claims (9)
1. A method for detecting a unlicensed vehicle in a video stream, comprising the steps of:
obtaining a training sample set for training a linear SVM classification model from a plurality of video segments of a video stream, the training sample set comprising positive samples as vehicle samples and negative samples as non-vehicle samples;
extracting HOG characteristics of all positive and negative samples from the obtained training sample set;
performing off-line training on the SVM classification model by using the extracted HOG characteristics to obtain an initially optimized classification model;
the video information in the video stream is used for training the initially optimized classification model on line again to obtain an optimal classification model;
carrying out vehicle detection on the video stream by utilizing an optimal classification model; the step of using the optimal classification model to detect the vehicle in the video stream comprises:
identifying the license plate and judging whether the license plate exists or not;
under the condition that a license plate result exists, automatically intercepting a vehicle head image by taking the position of the license plate as a standard, and outputting the license plate result;
under the condition of no license plate result, judging whether a background model is established;
under the condition that a background model is not established, the LBP characteristics are used for background modeling;
under the condition that a background model is established, judging whether a vehicle without a license plate result exists by using the classification model and the background model; specifically, an area is selected as a simulation trigger area at an output position of the vehicle, and the LBP characteristics of the simulation trigger area are used for background modeling; when a target passes through the simulation trigger area, the LBP characteristic value deviates from a background value, and when the target leaves, the value in the background state can be recovered, so that the process of dividing the appearance of the vehicle into disappearance is realized;
the vehicle appearance to disappearance segmentation process using LBP feature background modeling is as follows:
if continuous multiframes do not move in the analog trigger area, extracting an LBP histogram of the current area to show that a background is established; if no movement exists, continuously updating the background characteristic value;
if the motion is detected, calculating the distance between LBP histograms of the foreground and the background of the current simulation trigger area, if the distance is larger than a fixed threshold value, considering that the current area is covered, judging that the current area is sheltered by a vehicle or other interferences, and judging whether the vehicle passes through according to an optimal classification model; under the condition that the vehicle without the license plate result exists, outputting the detected vehicle as a license plate-free vehicle;
continuously calculating the distance between the LBP histograms of the foreground and the background of the current analog trigger area, and if the distance is less than a fixed threshold value for a plurality of continuous frames, considering that the current vehicle leaves;
the same vehicle is guaranteed to pass through, and only one result is achieved.
2. The method of claim 1, wherein the step of extracting the HOG features of all the positive and negative samples from the obtained training sample set comprises:
graying the sample image;
normalizing the color space of the image by adopting a gamma correction method;
calculating a gradient for each pixel in the image;
dividing the image into cells;
counting the gradient histogram of each unit, namely obtaining a descriptor of each unit;
combining each predetermined number of cells into a block, and obtaining HOG characteristics of the block by sub-linking the description strings of all cells in the block;
and connecting HOG characteristics of all blocks in the image in series to obtain a characteristic vector for training.
3. The method of claim 1, wherein the step of retraining the initially optimized classification model online using the video information in the video stream comprises:
according to the analysis of the video information, extracting a negative sample from a scene without a vehicle, and extracting a region around a license plate as a positive sample by taking the position of the license plate as a reference;
retraining the initially optimized classification model using the extracted positive and negative samples.
4. The unlicensed vehicle detection method of claim 1, wherein the SVM classification model is a binary classification model with respect to a target and a background.
5. An arrangement for detecting a unlicensed vehicle in a video stream, said arrangement comprising:
a training sample set obtaining module, configured to obtain a training sample set for training a linear SVM classification model from a plurality of video segments of the video stream, where the training sample set includes positive samples as vehicle samples and negative samples as non-vehicle samples;
the HOG feature extraction module is used for extracting HOG features of all positive and negative samples from the obtained training sample set;
the off-line training module is used for carrying out off-line training on the SVM classification model by using the extracted HOG characteristics so as to obtain an initially optimized classification model;
the online training module is used for carrying out online training on the initially optimized classification model again by utilizing the video information in the video stream to obtain an optimal classification model;
the detection module is used for detecting the vehicle for the video stream by utilizing the optimal classification model; and is further configured to:
identifying the license plate and judging whether the license plate exists or not;
under the condition that a license plate result exists, automatically intercepting a vehicle head image by taking the position of the license plate as a standard, and outputting the license plate result;
under the condition of no license plate result, judging whether a background model is established;
under the condition that a background model is not established, the LBP characteristics are used for background modeling;
under the condition that a background model is established, judging whether a vehicle without a license plate result exists by using the classification model and the background model; specifically, an area is selected as a simulation trigger area at the output position of the vehicle, and the LBP characteristics of the simulation trigger area are used for background modeling; when a target passes through the simulation trigger area, the LBP characteristic value deviates from a background value, and when the target leaves, the value in the background state can be recovered, so that the process of dividing the appearance of the vehicle into disappearance is realized;
the vehicle appearance to disappearance segmentation process using LBP feature background modeling is as follows:
if continuous multiframes do not move in the analog trigger area, extracting an LBP histogram of the current area to show that a background is established; if no motion exists, continuously updating the background characteristic value;
if the motion is detected, calculating the distance between LBP histograms of the foreground and the background of the current simulation trigger area, if the distance is larger than a fixed threshold value, considering that the current area is covered, judging that the current area is sheltered by a vehicle or other interferences, and judging whether the vehicle passes through according to an optimal classification model;
under the condition that the vehicle without the license plate result exists, outputting the detected vehicle as a license plate-free vehicle;
continuously calculating the distance between the LBP histograms of the foreground and the background of the current analog trigger area, and if the distance is less than a fixed threshold value for a plurality of continuous frames, considering that the current vehicle leaves;
the same vehicle is guaranteed to pass through, and only one result is achieved.
6. The unlicensed vehicle detection apparatus of claim 5, wherein the HOG feature extraction module is further configured to:
graying the sample image;
normalizing the color space of the image by adopting a gamma correction method;
calculating a gradient for each pixel in the image;
dividing the image into cells;
counting the gradient histogram of each unit, namely obtaining a descriptor of each unit;
combining the description substrings of all the units in a block to obtain the HOG characteristics of the block;
and connecting HOG features of all blocks in the image in series to obtain a feature vector for training.
7. The unlicensed vehicle detection device of claim 5, wherein the online training module is further configured to:
according to the analysis of the video information, extracting a negative sample from a scene without a vehicle, and extracting a region around a license plate as a positive sample by taking the position of the license plate as a reference;
retraining the initially optimized classification model using the extracted positive and negative samples.
8. The unlicensed vehicle detection device of claim 5, wherein the SVM classification model is a binary classification model with respect to target and background.
9. A computer storage medium, characterized in that a computer program code is stored in the computer storage medium, which computer program code, when executed, implements a method for detecting a unlicensed vehicle in a video stream according to any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810821013.3A CN109034171B (en) | 2018-07-24 | 2018-07-24 | Method and device for detecting unlicensed vehicles in video stream |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810821013.3A CN109034171B (en) | 2018-07-24 | 2018-07-24 | Method and device for detecting unlicensed vehicles in video stream |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109034171A CN109034171A (en) | 2018-12-18 |
| CN109034171B true CN109034171B (en) | 2022-10-04 |
Family
ID=64645606
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810821013.3A Active CN109034171B (en) | 2018-07-24 | 2018-07-24 | Method and device for detecting unlicensed vehicles in video stream |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109034171B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113256991B (en) * | 2021-07-14 | 2021-09-24 | 智广海联(天津)大数据技术有限公司 | Traffic flow statistics-based automobile plant peripheral road congestion management system and method |
| CN113793508B (en) * | 2021-09-27 | 2023-06-16 | 深圳市芊熠智能硬件有限公司 | Anti-interference rapid detection method for entrance and exit unlicensed vehicle |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104820998A (en) * | 2015-05-27 | 2015-08-05 | 成都通甲优博科技有限责任公司 | Human body detection and tracking method and device based on unmanned aerial vehicle mobile platform |
| CN105702048A (en) * | 2016-03-23 | 2016-06-22 | 武汉理工大学 | Automobile-data-recorder-based illegal lane occupation identification system and method for automobile on highway |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101647370B1 (en) * | 2014-11-26 | 2016-08-10 | 휴앤에스(주) | road traffic information management system for g using camera and radar |
-
2018
- 2018-07-24 CN CN201810821013.3A patent/CN109034171B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104820998A (en) * | 2015-05-27 | 2015-08-05 | 成都通甲优博科技有限责任公司 | Human body detection and tracking method and device based on unmanned aerial vehicle mobile platform |
| CN105702048A (en) * | 2016-03-23 | 2016-06-22 | 武汉理工大学 | Automobile-data-recorder-based illegal lane occupation identification system and method for automobile on highway |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109034171A (en) | 2018-12-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021085848A1 (en) | Signal control apparatus and signal control method based on reinforcement learning | |
| US20190122059A1 (en) | Signal light detection | |
| CN103279756B (en) | Vehicle detection based on integrated classifier analyzes system and determination method thereof | |
| US8447139B2 (en) | Object recognition using Haar features and histograms of oriented gradients | |
| CN110929593B (en) | Real-time significance pedestrian detection method based on detail discrimination | |
| KR101848019B1 (en) | Method and Apparatus for Detecting Vehicle License Plate by Detecting Vehicle Area | |
| CN107273832B (en) | License plate recognition method and system based on integral channel characteristics and convolutional neural network | |
| Peng et al. | Drone-based vacant parking space detection | |
| CN104615986A (en) | Method for utilizing multiple detectors to conduct pedestrian detection on video images of scene change | |
| Momin et al. | Vehicle detection and attribute based search of vehicles in video surveillance system | |
| Zhang et al. | License plate localization in unconstrained scenes using a two-stage CNN-RNN | |
| CN104978567A (en) | Vehicle detection method based on scenario classification | |
| CN113128507B (en) | License plate recognition method and device, electronic equipment and storage medium | |
| Saran et al. | Traffic video surveillance: Vehicle detection and classification | |
| Vignesh et al. | Abnormal event detection on BMTT-PETS 2017 surveillance challenge | |
| Yao et al. | Coupled multivehicle detection and classification with prior objectness measure | |
| CN109034171B (en) | Method and device for detecting unlicensed vehicles in video stream | |
| CN113255634A (en) | Vehicle-mounted mobile terminal target detection method based on improved Yolov5 | |
| Dousai et al. | Detecting humans in search and rescue operations based on ensemble learning | |
| Lin et al. | A lightweight, high-performance multi-angle license plate recognition model | |
| CN114550023A (en) | Traffic target static information extraction device | |
| CN111178359A (en) | License plate number recognition method, device and equipment and computer storage medium | |
| CN113112479A (en) | Progressive target detection method and device based on key block extraction | |
| CN110728229B (en) | Image processing method, device, equipment and storage medium | |
| CN110765940B (en) | Target object statistical method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |