CN113033275A - Vehicle lane-changing non-turn signal lamp analysis system based on deep learning - Google Patents
Vehicle lane-changing non-turn signal lamp analysis system based on deep learning Download PDFInfo
- Publication number
- CN113033275A CN113033275A CN202011285646.0A CN202011285646A CN113033275A CN 113033275 A CN113033275 A CN 113033275A CN 202011285646 A CN202011285646 A CN 202011285646A CN 113033275 A CN113033275 A CN 113033275A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- illegal
- lane
- information
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
- G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention discloses a deep learning-based lane-changing steering-lamp-free analysis system for vehicles, which is characterized by comprising a video stream access module, a GPU (graphics processing unit) graphic analysis module and an illegal information combination uploading module, wherein the GPU graphic analysis module comprises a vehicle information extraction module and an illegal behavior judgment module; the invention has the beneficial effects that: and comparing and verifying the analysis result by adopting an artificial intelligence technology, detecting whether the vehicle has the illegal behavior of changing lanes without turning a steering lamp, recording information such as driver information, license plate, body color, vehicle logo, time, place and the like according to needs, and outputting the information to an illegal information platform. The product adopts a national standard communication protocol and a video decoding algorithm, improves the compatibility of the product, and can be compatible with all domestic mainstream real-time monitoring systems and video snapshot systems.
Description
Technical Field
The invention relates to the field of road traffic, in particular to a system and a method for analyzing a vehicle lane changing turn-free turn signal lamp based on deep learning.
Background
Along with the popularization of automobiles, the traffic flow on the road surface is more and more while enjoying a great deal of convenience brought by automobile traveling, and traffic jam and even traffic accidents are easily caused. Therefore, related parts establish a plurality of traffic laws, increase the penalty of illegal laws and standardize the behavior of the car owner. For example, an expressway is the only exit at a lower high speed, and when the expressway is ready to drive into the expressway, a turn light needs to be turned on in advance. The vehicle behind or the vehicle beside is prompted to avoid, otherwise, the vehicle is easy to collide with the rear vehicle, and serious traffic accidents are caused. For the expressway, the vehicles run at a high speed, so that the expressway is very dangerous to go at a high speed, and the penalty for the illegal activities on the expressway is very strict.
In urban roads, according to the regulations of the currently implemented road traffic safety regulations, when a vehicle enters or exits an annular intersection, a corresponding turn light is turned on according to the driving direction, when the vehicle is in driving, the vehicle is not smooth in traffic and needs to change lanes, whether adjacent lanes are idle or not is observed through a rearview mirror, the corresponding turn light is turned on in advance under the condition that the normal driving of the vehicles in other lanes is not hindered, then the lanes are changed, when the vehicle turns around, a left turn light is turned on, whether the vehicle comes in front of or behind the vehicle is observed, and then the vehicle turns around again. Under default conditions (such as lane change, turning and the like), no turn signal lamp is turned to indicate that the lamp light is used in violation of regulations, and according to item 5 of the declaration and use regulations of motor vehicle driver's license (road traffic safety illegal behavior score value): the use of light is not as prescribed and can be penalized.
Although it is illegal to do not turn left lights, the conventional video monitoring technology cannot automatically identify the behavior of the vehicle, and thus, off-site illegal evidence collection cannot be performed. It can only be applied by road traffic police on site, so many drivers have a lucky psychology, and can do the job only by paying attention to the situation of side rear when changing lanes, and it is not so called to turn on the turn signal.
In conclusion, the management of the behavior of changing the lane of the vehicle without turning on the turn signal lamp has many problems, the monitoring points in the construction of the smart city are more and more at present, and the supervision of the behavior of changing the lane of the vehicle without turning on the turn signal lamp is directly related to the road traffic safety. The imaging snapshot mode used in the prior art can not accurately identify illegal behaviors of vehicles changing lanes without turning on the turn lights. The illegal behavior that the vehicle does not turn on the turn signal lamp when changing the lane is caused, and the management and the complaint can not be completely and effectively carried out.
Disclosure of Invention
In order to solve the defects of the prior art, the application provides a vehicle lane-changing turn-on-no-turn-signal analysis system based on deep learning.
The technical scheme of the invention is as follows:
the system for analyzing the lane-changing turning-free turn lights of the vehicle based on deep learning is characterized by comprising a video stream access module, a GPU (graphics processing unit) graphic analysis module and an illegal information combination uploading module;
the video stream access module judges whether a communication network is normal or not, whether a front-end camera works normally or not and whether a video code stream is normal or not through detection;
the GPU graphic analysis module comprises a vehicle information extraction module and an illegal behavior judgment module;
the vehicle information extraction module extracts vehicle information through a deep learning algorithm, wherein the vehicle information comprises a vehicle type, a vehicle logo, a license plate position, a vehicle body color and a steering lamp position, and accurately positions the position of a vehicle in an image;
the illegal behavior judgment module combines the extracted position of the steering lamp with a video image to obtain a behavior judgment image, the behavior judgment image is used as input to be sent into a deep learning algorithm for illegal behavior analysis, and forward propagation is carried out on an illegal behavior analysis network to finally judge whether the steering lamp is turned on or not when the vehicle changes lanes;
the illegal information combination uploading module identifies the license plate of an illegal vehicle, combines the license plate photo, the vehicle photo and the video image and uploads the license plate photo, the vehicle photo and the video image to an upper-layer server.
The analysis method of the deep learning-based analysis system for the lane-changing and turn-free lights of the vehicle is characterized by comprising the following steps of:
1) the video stream firstly passes through a video stream access module which is responsible for docking various front-end cameras and detecting image data transmitted by a network, wherein the video stream mainly comprises three functions of network detection, camera check and code stream detection;
2) after the video stream is ensured to be complete and effective, the video stream is decomposed into single-frame images and is transmitted to a GPU (graphics processing unit) image analysis module, wherein the vehicle information extraction module processes various information of vehicles in the single-frame images through a deep learning algorithm; the illegal behavior judgment module combines the extracted position of the steering lamp with a video image to obtain a behavior judgment image, the behavior judgment image is taken as input and sent into a neural network of a deep learning algorithm, the neural network receives image information as input and carries out forward propagation of the network, and finally an illegal judgment result is output at the end of the network; if illegal behaviors exist, inputting the license plate region information and the vehicle body related information into an illegal information combination uploading module;
3) the illegal information combination uploading module receives the information of the illegal behavior judging module, integrates the related information of the illegal vehicles, identifies the license plate of the license plate area, and uploads the integrated information to the upper-layer server.
The analysis method of the deep learning-based analysis system for the lane-changing and turn-lighting-free of the vehicle is characterized in that the illegal behavior judgment module in the step 2) specifically comprises the following steps:
2.1) lane calibration: calibrating the number of lanes, the virtual lines and the solid lines of the lanes in the video frame image picture;
2.2) vehicle tracking: the method comprises the steps that a typical track analysis method is utilized to carry out motion description on a video frame image vehicle to judge whether the vehicle has lane changing behavior, and if the vehicle is judged to have lane changing in a solid line, license plate region information and vehicle body related information are directly input into an illegal information combination uploading module; if the lane change of the vehicle at the dotted line is judged, entering the step 2.3);
2.3) vehicle type judgment: the method comprises the following steps of identifying and judging the vehicle type of a dotted-line lane-changing vehicle, and symmetrically dividing the vehicle into a left part and a right part so as to quickly identify the positions of left and right steering lamps of different vehicles;
2.4) judging a steering lamp: judging whether a steering lamp of a lane-changing vehicle is turned on or not through an HSI color model or an HSV color model;
2.5) shadow elimination: analyzing the vehicle by establishing a height comparison model to eliminate the vehicle shadow;
2.6) illegal vehicle tracking: and judging whether the vehicles in two adjacent frames are the same vehicle or not by using the spatial distance so as to complete the tracking of the vehicles in the time domain.
The analysis method of the deep learning-based analysis system for the lane-changing and turn-lighting-free vehicle is characterized in that the starting process of vehicle tracking in the step 2.2) is as follows: under the condition that no vehicle travels in a visual field, the gray value of a pixel of a background road surface in an image is Gb, the gray value Gv of the pixel of the image of the vehicle traveling usually has a difference value with Gb and is larger than Gb, the number N of the pixels with the gray value larger than TH in a lane marking line is calculated by setting a reasonable threshold TH, and when the vehicle passes through the visual field, the gray value Gv of the pixel of the vehicle traveling is larger than TH, the N value is increased to a set value, so that the vehicle is determined and recorded.
The analysis method of the deep learning-based vehicle lane-changing non-driving turn light analysis system is characterized in that when two vehicles exist in one lane at the same time during vehicle tracking in the step 2.2), the vehicles are roughly divided through the combination of time domain and space domain analysis algorithms, so that the regions of the vehicles are divided from a complex traffic scene.
The analysis method of the deep learning-based vehicle lane-changing and turn-lamp-free analysis system is characterized in that in the step 2.2), in order to obtain a better typical track, the track samples are subjected to segment fitting, and finally combined to obtain the typical track of the whole segment of track, so that errors can be eliminated to the maximum extent, and the finally optimized typical track is a curve capable of reflecting the behavior characteristics.
The analysis method of the deep learning-based lane-changing and turn-lamp-free analysis system for the vehicle is characterized in that the typical track is utilized in the step 2.3), model matching is carried out on the detected vehicle track, and the vehicle type is rapidly identified; and (3) model matching process: firstly, setting a threshold value, judging whether the detected track floats around a typical track motion model, and if so, considering the model to be matched, thus obtaining the behavior of the detected track, wherein the difference is within the set threshold value; in order to ensure the effectiveness of the threshold, the typical model is subjected to pattern learning, and the number of times of model matching is increased, which means that the track of the vehicle sample is also increased, so that the longer the pattern learning time is, the more the typical model can represent the characteristics of the behaviors, and the more accurate the matching result is.
The analysis method of the deep learning-based analysis system for the lane-changing and turn-free lights of the vehicle is characterized in that the establishment process of the high-degree comparison model in the step 2.5) is as follows:
2.5.1) selecting the most common and smallest vehicle in the video to track, and uniformly extracting a single-frame image with the vehicle;
2.5.2) sequentially recording the position information of the target vehicle in each image, wherein the position information comprises a vertical coordinate Y and a vertical coordinate height difference Delta Y of the vehicle in the image;
2.5.3) making a fitting curve of the height difference Delta Y of the ordinate relative to the ordinate Y, and calculating an approximate trend line equation of the fitting curve, namely a longitudinal height comparison model;
2.5.4) the height of the detected moving target is judged, if the height condition of the height comparison model is not met, the processing is not carried out, and the shadow influence is filtered.
The analysis method of the deep learning-based vehicle lane-changing and turn-free lamp analysis system is characterized in that the neural network structure of the deep learning algorithm in the step 2) comprises a convolution layer structure of basic feature extraction, a convolution layer structure of multi-scale feature extraction, a feature extraction layer, a pooling layer and detection output; when the image is input into a network for forward transmission, the convolution layer structure extracted by basic features is input into the convolution layer structure extracted by multi-scale features, the features in the basic feature extraction structure are input into a feature extraction layer for combination by combining the features of the multi-scale extraction structure, and the output combined features are input into a pooling layer; finally, inputting the output characteristics of the pooling layer into the final detection output, and outputting an illegal judgment result; if the output 1 is the vehicle lane change and the turn light is not turned on; and if the output is 0, the vehicle normally runs, and finally the behavior judgment of the vehicle steering lamp is obtained.
The analysis method of the deep learning-based lane-changing and turn-free lamp analysis system is characterized in that the related information of the illegal vehicle in the step 4) comprises three captured pictures of the tail or head of the vehicle, one close-up picture of the vehicle number plate and a video recording of the whole lane-changing process of the motor vehicle, so that a punnable illegal evidence chain is formed.
The invention has the beneficial effects that:
1) deep learning technology: the deep learning technology is used, so that the efficiency is high; further training can be performed according to the user data, and the efficiency is improved.
2) The problem of off-site evidence collection is solved: the system can utilize the data of the existing electronic police reverse gate, and a camera does not need to be erected again; the system utilizes the artificial intelligence technology to analyze all vehicles passing through the card port, detects whether the vehicles have the behavior of changing lanes without turning on the turn lights, and then records illegal and illegal vehicles according to the requirements.
3) Powerful system functions: and comparing and verifying the analysis result by adopting an artificial intelligence technology, detecting whether the driver of the vehicle has illegal behaviors of changing lanes and not hitting a steering lamp, recording information of the driver, the license plate, the color of the vehicle body, the vehicle logo, time, place and the like according to needs, and outputting the information to an illegal information platform.
4) Excellent product compatibility: the product adopts a national standard communication protocol and a video decoding algorithm, improves the compatibility of the product, and can be compatible with all domestic mainstream real-time monitoring systems and video snapshot systems.
Drawings
FIG. 1 is a schematic diagram of the network architecture of the present invention;
FIG. 2 is a diagram showing the detection results of lane-changing turn signals;
FIG. 3 is a diagram showing the detection results of lane-change turn-off-free indicator lights.
Detailed Description
The invention is further described below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention.
The system utilizes a deep learning technology to analyze all vehicles passing through a forward video in an original electronic police reverse checkpoint system, detects whether a driver of the vehicle has illegal behaviors of changing lanes and not driving a steering lamp, and then records information such as a license plate, a vehicle body color, a vehicle logo, time, a place and the like according to needs. The method solves the problem that the illegal act of turning the steering lamp is not performed when the vehicle changes the lane, and the off-site evidence obtaining of the special illegal act cannot be accurately identified.
A deep learning-based analysis system for a vehicle lane-changing turn-signal-free turn signal lamp mainly comprises a video stream access module, a vehicle information extraction module, an illegal behavior judgment module and an illegal information combination uploading module.
The video stream access module judges whether the network is normal or not, whether the front-end camera works normally or not and whether the video stream is normal or not through detection; the method comprises the steps that when a network is not connected or network delay exceeds a preset value, the network is judged to be abnormal, the current image scene change, the interference, the image distortion and the blurring of a camera are judged to be abnormal, and the code stream loss and the frame skipping are judged to be abnormal;
the video stream access module is formed by packaging SDK access components which are publicly provided by camera manufacturers, samples are taken, transcoding is carried out through operational amplifier, the samples are input into the GPU graphic analysis module for analysis, and fault function judgment processing is carried out through sampling software; the GPU graphic analysis module is a GPU processor chip of the Invida NVIDIA company, integrates video processing and conversion functions on the chip, and supports large-scale single-precision floating point number calculation; the deep learning algorithm is a DeepMindAI core of Google, deep learning operation is centralized in a GPU, the operation mode of a back-end processing program CPU + GPU can fully call the operation resources of a server, and the multi-thread mode is cooperated.
The vehicle information extraction module extracts vehicle information through a deep learning algorithm, wherein the vehicle information comprises vehicle body color, vehicle type, vehicle logo, license plate position and steering lamp position, and accurately positions the position of a vehicle in an image;
the illegal behavior judgment module combines the extracted position of the steering lamp with a video image to obtain a behavior judgment image, the behavior judgment image is used as input to be sent into a deep learning algorithm for illegal behavior analysis, and forward propagation is carried out on an illegal behavior analysis network to finally judge whether the steering lamp is turned on or not when the vehicle changes lanes;
the illegal vehicle license plate combination uploading module identifies the illegal vehicle license plate, combines the driver and the vehicle information which are judged to be illegal by the illegal behavior judging module, and uploads the evidence information to the upper-layer server.
The network structure of the deep learning algorithm comprises a convolution layer structure for basic feature extraction, a convolution layer structure for multi-scale feature extraction, a feature extraction layer, a pooling layer and detection output. In the image input network, firstly, inputting the convolution layer structure subjected to basic feature extraction into the convolution layer structure subjected to multi-scale feature extraction, inputting the features in the basic feature extraction structure into the feature extraction layer for combination by combining the features of the multi-scale extraction structure, and inputting the output combined features into the pooling layer; finally, inputting the output characteristics of the pooling layer into the final detection output, and outputting an illegal judgment result; if the output 1 is the vehicle lane change and the turn light is not turned on; if the output is 0, the vehicle normally runs, finally the behavior judgment of the vehicle steering lamp is obtained, and finally various classification and detection tasks in the system are completed, as shown in fig. 1.
The working process of the system is as follows:
the video stream firstly passes through a video stream access module which is responsible for docking various front-end cameras and detecting image data transmitted by a network, wherein the video stream mainly comprises three functions of network detection, camera check and code stream detection; after the video stream is ensured to be complete and effective, the video stream is decomposed into single-frame images and is transmitted to a GPU (graphics processing Unit) image analysis module, and various information of vehicles in the images is processed by a vehicle information extraction module, wherein the information mainly comprises vehicle detection, vehicle turn light detection, vehicle body information detection and license plate detection; the vehicle detection is used for detecting the accurate position of the vehicle, the vehicle turn light detection aims at extracting a vehicle turn light region, the vehicle turn light region is combined with the video frame image, the turn light image is extracted and input into the illegal behavior judgment module; the vehicle body information detection is carried out along with the vehicle detection, and mainly extracts vehicle related information including vehicle types, vehicle body colors and vehicle logos; the license plate detection is responsible for extracting the position of the license plate in the image; the illegal judging module takes the car window image as input, analyzes the behavior of the driver through an artificial neural network and judges whether the illegal behavior exists; if illegal behaviors exist, inputting the license plate region information and the vehicle body related information into an illegal information combination uploading module; the illegal information combination uploading module receives the information of the illegal judging module, integrates the related information of the illegal vehicles, identifies the license plate of the license plate area and uploads the integrated information to the upper-layer server;
the analysis method of the analysis system for the lane changing and turn-free lights of the vehicle based on deep learning comprises the following steps:
the system is an artificial intelligence platform for detecting and identifying the traffic target by utilizing the deep learning video analysis technology. The device can monitor and record all-weather illegal behaviors of passing motor vehicles on roads and turnstiles without turning on the turn lights. The system automatically analyzes, identifies and obtains evidence of traffic illegal behaviors such as no turn lights on the road driving lane change and the like according to the existing and subsequently added high-definition video monitoring equipment.
The system analyzes the running process of the motor vehicle through videos, and in the process of changing the lane of the motor vehicle, three pictures of the tail or the head of the vehicle are captured, and 1 vehicle number plate close-up is added. And the illegal photo framing is randomly snapshot according to the actual lane changing position of the vehicle, and meanwhile, video recording is carried out on the whole lane changing process of the motor vehicle. When vehicles pass through the lane, the system automatically detects and identifies the number plate number, the color and the type of the passing vehicles, captures a group of high-definition images and videos of the vehicles for continuously changing the lane, and obtains evidence videos and pictures of illegal lane changing of the motor vehicles through secondary screening to form a punishable illegal evidence chain.
The method comprises the following concrete steps:
1) the video stream firstly passes through a video stream access module which is responsible for docking various front-end cameras and detecting image data transmitted by a network, wherein the video stream mainly comprises three functions of network detection, camera check and code stream detection;
2) after the video stream is ensured to be complete and effective, the video stream is decomposed into single-frame images and is transmitted to a GPU (graphics processing Unit) image analysis module, and various information of vehicles in the images is processed by a vehicle information extraction module, wherein the information mainly comprises vehicle detection, vehicle window detection, vehicle body information detection and license plate detection; the vehicle detection is used for detecting the accurate position of the vehicle, the vehicle window detection aims at extracting a vehicle window area, the vehicle window area is combined with the video frame image, the vehicle window image is extracted and input into the illegal behavior judgment module; the vehicle body information detection is carried out along with the vehicle detection, and mainly extracts vehicle related information including vehicle types, vehicle body colors and vehicle logos; the license plate detection is responsible for extracting the position of the license plate in the image; the illegal behavior judgment module takes the vehicle window image as input and sends the vehicle window image into a neural network of a deep learning algorithm, the neural network receives image information as input and carries out forward propagation of the network, and finally an illegal judgment result is output at the tail of the network; if illegal behaviors exist, inputting the license plate region information and the vehicle body related information into an illegal information combination uploading module;
the specific logic steps are as follows:
2.1) lane calibration: calibrating the number of lanes, the virtual lines and the solid lines of the lanes in a single-frame image picture;
2.2) vehicle tracking: the method comprises the steps that a typical track analysis method is used for describing the motion of a vehicle to judge whether the vehicle has lane changing behavior, and if the vehicle is judged to have lane changing in a solid line, license plate region information and vehicle body related information are directly input into an illegal information combination uploading module; if the lane change of the vehicle at the dotted line is judged, entering the step 2.3);
starting process of vehicle tracking in step 2.2): under the condition that no vehicle travels in a visual field, pixel gray values Gb of a background road surface in an image are distributed in a small range, and the pixel gray values Gv of the image of the vehicle traveling usually have a difference value with Gb and are larger than Gb; by setting a reasonable threshold TH (Gv > TH > Gb), the number N of pixels with the gray value larger than TH in the sign line is calculated, the N value is small, the fact that no moving vehicle enters a visual field is indicated, when the vehicle passes through the visual field, the gray value Gv of the pixels of the moving vehicle is larger than TH, the N value can be increased to a set value, and therefore the vehicle can be determined and recorded.
The tracking of the individual vehicle behaviors in the step 2.2) is mainly completed by analyzing the tracks, the final processing result of the track analysis is the completion of the clustering of the tracks,
firstly, establishing a model, and describing a motion model by adopting a typical track, wherein the typical track can well reflect the characteristics of the track and is the most vivid expression of the track; the typical track is mainly obtained by fitting the track samples, a single fitting curve cannot represent the characteristics of the whole section, in order to obtain a better typical track, the track samples can be subjected to sectional fitting, and finally combined to obtain the typical track of the whole section of track, and errors can be eliminated to the maximum extent; the final optimized typical trajectory is not a series of points, but a curve reflecting the behavior characteristics.
And 2.2) roughly dividing the vehicles in the extracted traffic scene video image when the vehicles are tracked in the step 2.2), for example, when two vehicles exist in one lane at the same time, and dividing the regions which are possible to be the vehicles from the complex traffic scene so as to carry out subsequent detection and tracking operations. The method comprises time domain, space domain and time-space domain analysis of the video image sequence.
2.3) vehicle type judgment: the method comprises the following steps of identifying and judging the vehicle type of a dotted-line lane-changing vehicle, and symmetrically dividing the vehicle into a left part and a right part so as to quickly identify the positions of left and right steering lamps of different vehicles;
and 2.3) carrying out model matching on the detected vehicle track by using the typical model, so as to realize the identification of the vehicle behavior. The principle of model matching is simple: first a threshold is set, and it is observed whether the detected trajectory floats around the representative model, and the difference is within the set threshold, and if so, the model is deemed to match, thus deriving the behavior of the detected trajectory. The types of thresholds are two: one is the position difference between the locus point and the typical model, the other is the probability judgment of the locus, and the threshold value is set to a certain probability value.
The key point of the matching process is the setting of a threshold, and a proper threshold needs to be selected, so that the matching result has no misjudgment or missing judgment. In order to ensure the validity of the threshold, one effective method is to perform pattern learning on the model. The fact that the number of times of model matching is increased means that the track of a vehicle sample is also increased, so that the longer the pattern learning time is, the more the typical model can represent the characteristics of the behaviors, and the more accurate the matching result is.
2.4) judging a steering lamp: judging whether a steering lamp of a lane-changing vehicle is turned on or not through an HSI color model or an HSV color model;
2.5) shadow elimination: analyzing the vehicle by establishing a height comparison model to eliminate the vehicle shadow;
shadow elimination principle:
the longitudinal height of the shadow of the vehicle in the video is much smaller than the height of the vehicle body, a height comparison model in the time interval is set according to the phenomenon, the height of the detected moving target is judged, and if the specific height condition is not met, the moving target is not processed, so that the shadow influence can be simply filtered; in order to avoid the phenomenon of missing judgment, the smallest vehicle needs to be selected for sampling and tracking to obtain a height comparison model.
The specific process comprises the following steps:
2.5.1) selecting the most common and smallest vehicle in the video for tracking, and uniformly extracting the video frame with the vehicle;
2.5.2) recording the position information of the target vehicle in each image in turn, wherein the position information comprises a red prompt box as shown in figures 2 and 3: ordinate Y, ordinate height DeltaY;
2.5.3) making a fitting curve of the height delta Y relative to a vertical coordinate Y, and calculating an approximate trend line equation of the fitting curve, namely a longitudinal height comparison model; however, in order to prevent the missing judgment caused by the fluctuation of the longitudinal height, a certain amount is usually required to be subtracted from the equation to increase the robustness;
2.6) illegal vehicle tracking: and judging whether the vehicles in two adjacent frames are the same vehicle or not by using the spatial distance so as to complete the tracking of the vehicles in the time domain.
After detecting the illegal vehicle, the illegal vehicle is tracked, and the vehicle tracking method follows a basic principle, namely that whether the vehicles in two adjacent frames are the same vehicle is judged according to the space distance, so that the vehicle tracking in the time domain is completed. The method has the advantage that even in the case of mutual occlusion between vehicles, a large part of the characteristics of the vehicles are visible, and can provide a basis for the tracking process. The motion limitation is used as the basis of clustering, namely, the features belonging to the same vehicle always move at the same speed, thereby obtaining better effect.
3) And the illegal information combination uploading module receives the information of the illegal judging module, integrates the related information of the illegal driver, identifies the license plate in the license plate area and uploads the integrated information to the upper-layer server.
The embodiments described in this specification are merely illustrative of implementations of the inventive concept and the scope of the present invention should not be considered limited to the specific forms set forth in the embodiments but rather by the equivalents thereof as may occur to those skilled in the art upon consideration of the present inventive concept.
Claims (10)
1. The system for analyzing the lane-changing turning-free turn lights of the vehicle based on deep learning is characterized by comprising a video stream access module, a GPU (graphics processing unit) graphic analysis module and an illegal information combination uploading module;
the video stream access module judges whether a communication network is normal or not, whether a front-end camera works normally or not and whether a video code stream is normal or not through detection;
the GPU graphic analysis module comprises a vehicle information extraction module and an illegal behavior judgment module;
the vehicle information extraction module extracts vehicle information through a deep learning algorithm, wherein the vehicle information comprises a vehicle type, a vehicle logo, a license plate position, a vehicle body color and a steering lamp position, and accurately positions the position of a vehicle in an image;
the illegal behavior judgment module combines the extracted position of the steering lamp with a video image to obtain a behavior judgment image, the behavior judgment image is used as input to be sent into a deep learning algorithm for illegal behavior analysis, and forward propagation is carried out on an illegal behavior analysis network to finally judge whether the steering lamp is turned on or not when the vehicle changes lanes;
the illegal information combination uploading module identifies the license plate of an illegal vehicle, combines the license plate photo, the vehicle photo and the video image and uploads the license plate photo, the vehicle photo and the video image to an upper-layer server.
2. The analysis method of the deep learning based lane-changing turn-signal-free analysis system for the vehicle according to claim 1, characterized by comprising the following steps:
1) the video stream firstly passes through a video stream access module which is responsible for docking various front-end cameras and detecting image data transmitted by a network, wherein the video stream mainly comprises three functions of network detection, camera check and code stream detection;
2) after the video stream is ensured to be complete and effective, the video stream is decomposed into single-frame images and is transmitted to a GPU (graphics processing unit) image analysis module, wherein the vehicle information extraction module processes various information of vehicles in the single-frame images through a deep learning algorithm; the illegal behavior judgment module combines the extracted position of the steering lamp with a video image to obtain a behavior judgment image, the behavior judgment image is taken as input and sent into a neural network of a deep learning algorithm, the neural network receives image information as input and carries out forward propagation of the network, and finally an illegal judgment result is output at the end of the network; if illegal behaviors exist, inputting the license plate region information and the vehicle body related information into an illegal information combination uploading module;
3) the illegal information combination uploading module receives the information of the illegal behavior judging module, integrates the related information of the illegal vehicles, identifies the license plate of the license plate area, and uploads the integrated information to the upper-layer server.
3. The analysis method of the deep learning-based analysis system for the lane-changing and turn-lighting-free of vehicle according to claim 2, wherein the illegal behavior judgment module in the step 2) is implemented by the following steps:
2.1) lane calibration: calibrating the number of lanes, the virtual lines and the solid lines of the lanes in the video frame image picture;
2.2) vehicle tracking: the method comprises the steps that a typical track analysis method is utilized to carry out motion description on a video frame image vehicle to judge whether the vehicle has lane changing behavior, and if the vehicle is judged to have lane changing in a solid line, license plate region information and vehicle body related information are directly input into an illegal information combination uploading module; if the lane change of the vehicle at the dotted line is judged, entering the step 2.3);
2.3) vehicle type judgment: the method comprises the following steps of identifying and judging the vehicle type of a dotted-line lane-changing vehicle, and symmetrically dividing the vehicle into a left part and a right part so as to quickly identify the positions of left and right steering lamps of different vehicles;
2.4) judging a steering lamp: judging whether a steering lamp of a lane-changing vehicle is turned on or not through an HSI color model or an HSV color model;
2.5) shadow elimination: analyzing the vehicle by establishing a height comparison model to eliminate the vehicle shadow;
2.6) illegal vehicle tracking: and judging whether the vehicles in two adjacent frames are the same vehicle or not by using the spatial distance so as to complete the tracking of the vehicles in the time domain.
4. The analysis method of the deep learning-based lane-changing and turn-free lamp analysis system for the vehicle according to claim 3, wherein the starting process of the vehicle tracking in the step 2.2) is as follows: under the condition that no vehicle travels in a visual field, the gray value of a pixel of a background road surface in an image is Gb, the gray value Gv of the pixel of the image of the vehicle traveling usually has a difference value with Gb and is larger than Gb, the number N of the pixels with the gray value larger than TH in a lane marking line is calculated by setting a reasonable threshold TH, and when the vehicle passes through the visual field, the gray value Gv of the pixel of the vehicle traveling is larger than TH, the N value is increased to a set value, so that the vehicle is determined and recorded.
5. The analysis method of the deep learning-based vehicle lane-changing and turn-not-turning lamp analysis system according to claim 3, wherein in the step 2.2), when the vehicle tracks, for example, when two vehicles exist in a lane at the same time, the vehicle is roughly divided by combining a time domain and a space domain analysis algorithm, so that the region of the vehicle is divided from a complex traffic scene.
6. The analysis method of the deep learning-based analysis system for the lane-changing and turn-lighting-free vehicle according to claim 3, wherein in the step 2.2), in order to obtain a better typical track, the track samples are subjected to segment fitting, and finally combined to obtain a typical track of the whole track, so that errors can be eliminated to the maximum extent, and the finally optimized typical track is a curve capable of reflecting the behavior characteristics.
7. The analysis method of the deep learning-based lane-changing and turn-lighting-free analysis system for the vehicle according to claim 3, wherein the step 2.3) is implemented by using a typical track to perform model matching on the detected vehicle track, so as to realize rapid recognition on the vehicle type; and (3) model matching process: firstly, setting a threshold value, judging whether the detected track floats around a typical track motion model, and if so, considering the model to be matched, thus obtaining the behavior of the detected track, wherein the difference is within the set threshold value; in order to ensure the effectiveness of the threshold, the typical model is subjected to pattern learning, and the number of times of model matching is increased, which means that the track of the vehicle sample is also increased, so that the longer the pattern learning time is, the more the typical model can represent the characteristics of the behaviors, and the more accurate the matching result is.
8. The analysis method of the deep learning-based vehicle lane-changing and turn-free lamp analysis system according to claim 3, wherein the high degree comparison model establishment process in the step 2.5) is as follows:
2.5.1) selecting the most common and smallest vehicle in the video to track, and uniformly extracting a single-frame image with the vehicle;
2.5.2) sequentially recording the position information of the target vehicle in each image, wherein the position information comprises a vertical coordinate Y and a vertical coordinate height difference Delta Y of the vehicle in the image;
2.5.3) making a fitting curve of the height difference Delta Y of the ordinate relative to the ordinate Y, and calculating an approximate trend line equation of the fitting curve, namely a longitudinal height comparison model;
2.5.4) the height of the detected moving target is judged, if the height condition of the height comparison model is not met, the processing is not carried out, and the shadow influence is filtered.
9. The analysis method of the deep learning-based lane-changing and turn-free lamp analysis system of the vehicle according to claim 3, wherein the neural network structure of the deep learning algorithm in the step 2) comprises a convolution layer structure of basic feature extraction, a convolution layer structure of multi-scale feature extraction, a feature extraction layer, a pooling layer and detection output; when the image is input into a network for forward transmission, the convolution layer structure extracted by basic features is input into the convolution layer structure extracted by multi-scale features, the features in the basic feature extraction structure are input into a feature extraction layer for combination by combining the features of the multi-scale extraction structure, and the output combined features are input into a pooling layer; finally, inputting the output characteristics of the pooling layer into the final detection output, and outputting an illegal judgment result; if the output 1 is the vehicle lane change and the turn light is not turned on; and if the output is 0, the vehicle normally runs, and finally the behavior judgment of the vehicle steering lamp is obtained.
10. The analysis method of the deep learning-based lane-changing and turn-off-free lamp analysis system of the vehicle as claimed in claim 3, wherein the related information of the illegal vehicle in the step 3) comprises three captured pictures of the tail or head of the vehicle, one close-up picture of the vehicle number plate and a video recording of the whole lane-changing process of the motor vehicle, so as to form a punnable illegal evidence chain.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011285646.0A CN113033275B (en) | 2020-11-17 | 2020-11-17 | Vehicle lane-changing non-turn signal lamp analysis system based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011285646.0A CN113033275B (en) | 2020-11-17 | 2020-11-17 | Vehicle lane-changing non-turn signal lamp analysis system based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113033275A true CN113033275A (en) | 2021-06-25 |
| CN113033275B CN113033275B (en) | 2023-04-18 |
Family
ID=76459007
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011285646.0A Active CN113033275B (en) | 2020-11-17 | 2020-11-17 | Vehicle lane-changing non-turn signal lamp analysis system based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113033275B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113505663A (en) * | 2021-08-16 | 2021-10-15 | 浙江浩腾电子科技股份有限公司 | Electric bicycle red light running video analysis and identification method based on artificial intelligence |
| CN114241774A (en) * | 2021-12-28 | 2022-03-25 | 高新兴智联科技有限公司 | Method and device for recording vehicle law violation evidence |
| CN114241449A (en) * | 2022-02-24 | 2022-03-25 | 南京乾创电子科技有限公司 | Vehicle lane-changing no-lighting illegal snapshot system and method based on edge equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105975915A (en) * | 2016-04-28 | 2016-09-28 | 大连理工大学 | Front vehicle parameter identification method based on multitask convolution nerve network |
| CN106128096A (en) * | 2016-06-29 | 2016-11-16 | 安徽金赛弗信息技术有限公司 | A kind of motor vehicles is not opened steering indicating light by regulation and is broken rules and regulations lane change intelligent identification Method and device thereof |
| CN106652468A (en) * | 2016-12-09 | 2017-05-10 | 武汉极目智能技术有限公司 | Device and method for detection of violation of front vehicle and early warning of violation of vehicle on road |
| CN109741608A (en) * | 2019-01-29 | 2019-05-10 | 浙江浩腾电子科技股份有限公司 | Motor vehicle based on deep learning, which is turned right, gives precedence to pedestrian's analysis capturing system and method |
| CN109919069A (en) * | 2019-02-27 | 2019-06-21 | 浙江浩腾电子科技股份有限公司 | Oversize vehicle analysis system based on deep learning |
| CN110399766A (en) * | 2019-01-28 | 2019-11-01 | 浙江浩腾电子科技股份有限公司 | Smoking testing and analysis system based on deep learning |
-
2020
- 2020-11-17 CN CN202011285646.0A patent/CN113033275B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105975915A (en) * | 2016-04-28 | 2016-09-28 | 大连理工大学 | Front vehicle parameter identification method based on multitask convolution nerve network |
| CN106128096A (en) * | 2016-06-29 | 2016-11-16 | 安徽金赛弗信息技术有限公司 | A kind of motor vehicles is not opened steering indicating light by regulation and is broken rules and regulations lane change intelligent identification Method and device thereof |
| CN106652468A (en) * | 2016-12-09 | 2017-05-10 | 武汉极目智能技术有限公司 | Device and method for detection of violation of front vehicle and early warning of violation of vehicle on road |
| CN110399766A (en) * | 2019-01-28 | 2019-11-01 | 浙江浩腾电子科技股份有限公司 | Smoking testing and analysis system based on deep learning |
| CN109741608A (en) * | 2019-01-29 | 2019-05-10 | 浙江浩腾电子科技股份有限公司 | Motor vehicle based on deep learning, which is turned right, gives precedence to pedestrian's analysis capturing system and method |
| CN109919069A (en) * | 2019-02-27 | 2019-06-21 | 浙江浩腾电子科技股份有限公司 | Oversize vehicle analysis system based on deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 温聪等: "基于图像处理的行车安全辅助系统", 《自动化技术与应用》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113505663A (en) * | 2021-08-16 | 2021-10-15 | 浙江浩腾电子科技股份有限公司 | Electric bicycle red light running video analysis and identification method based on artificial intelligence |
| CN114241774A (en) * | 2021-12-28 | 2022-03-25 | 高新兴智联科技有限公司 | Method and device for recording vehicle law violation evidence |
| CN114241774B (en) * | 2021-12-28 | 2023-09-19 | 高新兴智联科技股份有限公司 | Method and device for recording vehicle illegal evidence |
| CN114241449A (en) * | 2022-02-24 | 2022-03-25 | 南京乾创电子科技有限公司 | Vehicle lane-changing no-lighting illegal snapshot system and method based on edge equipment |
| CN114241449B (en) * | 2022-02-24 | 2022-11-04 | 南京乾创电子科技有限公司 | Vehicle lane-changing no-lighting illegal snapshot system and method based on edge equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113033275B (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113033275B (en) | Vehicle lane-changing non-turn signal lamp analysis system based on deep learning | |
| CN106652465B (en) | Method and system for identifying abnormal driving behaviors on road | |
| US10885777B2 (en) | Multiple exposure event determination | |
| CN111815959B (en) | Vehicle violation detection method and device and computer readable storage medium | |
| CN102765365B (en) | Pedestrian detection method based on machine vision and pedestrian anti-collision warning system based on machine vision | |
| KR100969995B1 (en) | System of traffic conflict decision for signalized intersections using image processing technique | |
| CN109299674B (en) | Tunnel illegal lane change detection method based on car lamp | |
| US20160034778A1 (en) | Method for detecting traffic violation | |
| KR101742490B1 (en) | System for inspecting vehicle in violation by intervention and the method thereof | |
| CN110298300B (en) | Method for detecting vehicle illegal line pressing | |
| CN102332209A (en) | Automobile violation video monitoring method | |
| CN113055649A (en) | Tunnel intelligent video monitoring method and device, intelligent terminal and storage medium | |
| US11745749B2 (en) | Vehicular system for testing performance of object detection algorithms | |
| CN105574502A (en) | Automatic detection method for violation behaviors of self-service card sender | |
| CN113160575A (en) | Traffic violation detection method and system for non-motor vehicles and drivers | |
| CN114781479A (en) | Traffic incident detection method and device | |
| US11482012B2 (en) | Method for driving assistance and mobile device using the method | |
| CN112528759A (en) | Traffic violation behavior detection method based on computer vision | |
| Munajat et al. | Vehicle detection and tracking based on corner and lines adjacent detection features | |
| CN106340187A (en) | Camera license plate recognition system and recognition implementation method | |
| CN115240435A (en) | AI technology-based vehicle illegal driving detection method and device | |
| CN113177443A (en) | Method for intelligently identifying road traffic violation based on image vision | |
| Srivastava et al. | Automatic traffic surveillance system Utilizing object detection and image processing | |
| CN113487878A (en) | Motor vehicle illegal line pressing running detection method and system | |
| CN112633062A (en) | Deep learning bus lane occupation detection method based on embedded terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |