CN106874863B - Vehicle illegal parking and reverse running detection method based on deep convolutional neural network - Google Patents

Vehicle illegal parking and reverse running detection method based on deep convolutional neural network Download PDF

Info

Publication number
CN106874863B
CN106874863B CN201710059676.1A CN201710059676A CN106874863B CN 106874863 B CN106874863 B CN 106874863B CN 201710059676 A CN201710059676 A CN 201710059676A CN 106874863 B CN106874863 B CN 106874863B
Authority
CN
China
Prior art keywords
road surface
road
vehicle
detection
roi
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710059676.1A
Other languages
Chinese (zh)
Other versions
CN106874863A (en
Inventor
阮雅端
高妍
赵博睿
陈金艳
陈启美
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University
Original Assignee
Nanjing University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University filed Critical Nanjing University
Priority to CN201710059676.1A priority Critical patent/CN106874863B/en
Publication of CN106874863A publication Critical patent/CN106874863A/en
Application granted granted Critical
Publication of CN106874863B publication Critical patent/CN106874863B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

The method for detecting the vehicle illegal parking and reverse running based on the deep convolutional neural network comprises the steps of using a mobile terminal detection point as a road camera, obtaining image information through the mobile terminal detection point through the camera, introducing deep learning into road surface event recognition and improving the road surface event recognition, and therefore the road event recognition accuracy is remarkably improved. The method utilizes the convolution neural network to analyze the acquired image, divides the road surface ROI into a plurality of networks, constructs a road surface-non-road surface identification model, and reversely identifies targets such as illegal parking of a highway, reverse driving of vehicles and the like through a non-road surface grid. The method is applied to non-real-time tasks such as road surface violation detection, vehicle back-driving detection and the like, fully utilizes the characteristics and advantages of the mobile internet, and realizes the detection of road surface events such as regional high-coverage vehicle violation, vehicle back-driving and the like at low cost.

Description

Vehicle illegal parking and reverse running detection method based on deep convolutional neural network
Technical Field
The invention belongs to the technical field of video detection of depth features, relates to detection of road surface events, and provides a vehicle illegal parking and converse driving detection method based on a Deep convolutional neural network (Deep-CNN), wherein the vehicle illegal parking or converse driving detection is carried out based on a Deep-CNN road surface reverse recognition model, and data analysis and data mining are carried out on detection results.
Background
Road accidents, traffic congestion and environmental pollution are common problems in the current road traffic development. Road traffic safety conditions are great, and an Intelligent Transportation System (ITS) is an effective means for improving utilization efficiency of road facilities, relieving traffic jam and reducing incidence of traffic accidents. The method has the advantages that traffic flow parameters such as road speed and flow are sensed in real time through a computer vision technology, real-time road conditions are provided, the traffic state and travel time of a road network are predicted by combining historical data, manual monitoring is replaced by automatic video analysis, road abnormal events including road surface residue detection, expressway illegal parking detection and other high-risk events are detected from massive videos, and the method has very important significance for improving the road informatization level and the public service capacity.
Disclosure of Invention
The invention aims to solve the problems that: with the great increase of the number of road monitoring videos, the effective management of the existing video resources cannot be realized only by manpower. The traffic monitoring video is automatically analyzed through computer video analysis, traffic parameters are extracted, abnormal events are automatically found and actively reported, the labor cost of traffic management can be greatly reduced, and the management level and the emergency response capability of the events are improved.
The technical scheme of the invention is as follows: the method for detecting the illegal parking and the reverse running of the vehicle based on the Deep convolutional neural network comprises the following steps of based on a Deep convolutional neural network Deep-CNN network model, dividing a road surface region of interest into a plurality of networks by the Deep convolutional neural network Deep-CNN network model, constructing a road surface-non-road surface identification model, and reversely identifying targets such as illegal parking and reverse running of a highway road surface through a non-road surface grid, wherein the method comprises the following steps:
step 1: training a road model, acquiring a video image of a road camera, and displaying a region of interest ROI of the road in a video windowRoad surfaceThe method comprises the steps of performing gridding segmentation into a plurality of small blocks, using the small blocks as a training set of a Deep-CNN network model after standardization, firstly adopting an unsupervised method to train and obtain image characteristics during training, setting labels after clustering, marking road surface types in a manual mode, and distinguishing road surfaces from non-road surfaces to obtain a road surface-non-road surface recognition model;
step 2: the method comprises the steps of training a non-road surface foreground model, combining grid pictures divided into non-road surfaces into candidate targets according to connected regions, adding the candidate targets into a training library, training by adopting a Deep-CNN network model again, and training road surface targets in a grading mode, wherein the road surface targets comprise vehicles, road surface remnants and pedestrians to obtain a foreground identification model;
step 3: detecting a foreground target, namely detecting and classifying the foreground target by using a Deep-CNN network model and an SVM classifier on a real-time video image on the basis of the identification models of step1 and step2, firstly identifying a non-road surface of the road surface, and then identifying the type of the foreground target;
step 4: behavior analysis, namely performing illegal parking and converse running detection and identification on the vehicles in the road event according to the context information of the foreground target in the video image sequence on the basis of foreground target classification and identificationAnd
Figure BDA0001218382640000023
are respectively T0And T0+tThe position of the kth foreground candidate target at the moment is calculated, and the Euclidean distance of the foreground candidate target in the image area displacement is calculated
Figure BDA0001218382640000024
Figure BDA0001218382640000025
And acquiring the motion state and direction of the kth foreground candidate target, and further determining whether the target has a stop state or a retrograde motion state.
The vehicle illegal parking or reverse driving detection specifically comprises the following steps:
step1.1: setting forbidden region ROI on road surface in real-time video imageForbidden to useAfter the video window image is gridded, classifying according to a pavement-non-pavement identification model;
step1.2: connected ROIForbidden to useNon-pavement gridding picture I in areai,jGenerating a candidate object Ok；
Step1.3: for candidate target OkClassifying and identifying, if the vehicle is a vehicle, locking the vehicle as a vehicle target;
step1.4: calculating an initial time T0And T after time T0+tComparing the target displacement distance and direction of the vehicle with the direction of the vehicle on the normal road;
step1.5: and judging the illegal parking or the reverse driving behavior of the vehicle.
Further, Step1 and Step2 are specifically:
1) setting region of interest ROIRoad surface: acquiring a video image of a road or street view monitor to obtain a road or street view video frame image, extracting 4 points of the boundary opposite angle of an attention area on a current frame image according to the actual road or street view condition, performing straight line fitting calculation on the extracted points to form a fork-shaped structure, wherein the fork-shaped structure is used as a detected interest area ROIRoad surfaceI.e. the effective detection area;
2) filling the non-detection area with water: non-ROIRoad surfaceFilling the non-monitoring area with water, and falling to ROI after fillingRoad surfaceThe pixel mean value of the grid picture outside the area is 0, and the subsequent processing is not carried out after the direct filtering;
3) detecting region ROIRoad surfaceGridding and blocking, classifying the gridding picture into a road surface or a non-road surface through Deep-CNN, and classifying the ROIRoad surfaceThe non-road grid pictures in the blocks are connected and marked as Ip,qI.e. by
Figure BDA0001218382640000021
Road surface, Ip,qAnd forming candidate targets, sending the candidate targets into a classifier, and classifying the candidate targets into vehicles, pedestrians or road surface remnants.
The invention fully utilizes the existing video monitoring facilities and mass video data, can save hardware investment to the maximum extent, obtains richer and more visual traffic data, and meets the data/information requirements of traffic management and public service.
The invention establishes a Deep-CNN-based reverse pavement identification model, utilizes the Deep-CNN pavement model to solve the problems of moving cameras and image target detection, and is applied to tasks such as vehicle illegal parking detection, reverse driving detection and the like. Compared with the traditional technologies such as a ground sensing coil and a radar, the monitoring video contains image information of a road surface, vehicles, road surface remnants and the like, the characteristics are extracted through the CNN neural network, abnormal event information such as illegal parking, retrograde motion and the like of the vehicles on the road surface is detected based on Deep-CNN, road accidents are identified, image-text alarming is sent in time, and road event information richer than that of traditional vehicle detection equipment is provided.
The method comprises the steps of deploying a mobile terminal in a large range in a monitoring area to form a mobile terminal detection point, carrying out interactive calibration through mobile communication equipment or road monitoring, determining camera parameters, using the camera to collect images, further analyzing the collected images by using a CNN (neural network) to obtain ROI (region of interest) data of each current image, dividing a road ROI (region of interest) area into a plurality of networks, constructing a road-non-road recognition model, and reversely recognizing targets such as illegal parking, reverse driving and the like through a non-road grid.
The detection point of the mobile terminal is a road camera, the price is low, the advantages of the mobile internet are fully utilized, expensive monitoring instrument equipment is not needed, only the existing camera on the road is needed, the mobile terminal can be deployed in a large range, and the on-site detection road surface information is transmitted to the server through the mobile internet. The characteristics and advantages of the mobile internet are fully utilized, and the detection of the regional high-coverage-rate road surface event is realized at low cost.
Drawings
FIG. 1 is a detection flow chart of a road surface event reverse identification based on Deep-CNN in the invention
FIG. 2 shows the principle of Deep-CNN model used in the present invention. (a) Softplus and ReLU activation function (b) Deep-CNN network architecture.
FIG. 3 is a detection diagram of Deep-CNN pavement identification model according to the present invention. (a) Dividing ROI detection regions, (b) filling and clearing non-ROI regions by a flooding method, (c) gridding Deep-CNN input and target detection.
Fig. 4 is a road surface detection training diagram of a single-frame photo of the mobile terminal of the present invention. (a) And (b) carrying out picture processing on the handheld mobile terminal, and (b) detecting the road surface in the ROI area of the lane.
Fig. 5 shows the detection effect of the Deep-CNN road surface reverse recognition model based on contraband and contraband in the selected scene of the highway of ninglian province in Jiangsu province, which is implemented by the invention, (a) road surface roi, (b) contraband detection, (c) contraband detection, (d) and contraband detection.
Detailed Description
The invention introduces deep learning into road surface event recognition and improves the same, and can obviously improve the road event recognition accuracy. Considering that vehicles are static targets, the traditional background modeling method is not suitable for static target detection, and road remains are difficult to construct a training set by using a prior model, so that the road remains which are also static targets can be confused with illegal vehicles. The invention establishes a pavement reverse recognition model based on a Deep convolutional neural network Deep-CNN, wherein a mobile terminal detection point is a road camera, the mobile terminal detection point acquires image information through the camera, the CNN neural network is used for analyzing the acquired image, a pavement ROI area is divided into a plurality of networks, a pavement-non-pavement recognition model is established, and illegal parking and reverse driving are reversely recognized through a non-pavement mesh.
The invention constructs a hierarchical road event identification framework based on Deep-CNN network:
the basic idea is layered recognition, and a pavement target object is reversely recognized through pavement-non-pavement recognition, wherein the pavement reverse recognition process comprises the following steps:
firstly, performing pavement model training, namely gridding and dividing a Region of Interest (ROI) of a video window into a plurality of small blocks, standardizing the small blocks to be used as a training set of a Deep-CNN network, and forcibly marking the pavement model as a single type which is easy to cause overfitting by taking the fact that the pavement model is composed of pavement asphalt, pavement lanes, guardrails and the like and the appearance difference of the image is large, so that an unsupervised method is adopted to train and obtain image characteristics, and the pavement is marked in a multi-type label manual mode after clustering, so that the pavement and the non-pavement are accurately distinguished.
And then, performing non-road surface foreground model training, combining the grid picture blocks divided into non-road surfaces into candidate targets according to connected regions, adding the candidate targets into a training library, performing Deep-CNN network training again, and training road surface targets including vehicles, road surface remnants, pedestrians and the like in a grading manner.
When the road is detected in real time, the foreground target detection is firstly carried out, and the detection and classification of the moving and static foreground targets are realized on the basis of the previous road surface identification model.
And finally, performing behavior analysis, and performing vehicle illegal parking detection and vehicle converse detection according to the context information of the target in the video image sequence on the basis of foreground target detection.
The vehicle violation detection and the reverse detection are specifically realized in the following steps:
step1.1: setting forbidden region ROI on road surface in real-time video imageForbidden to useAfter the video window image is gridded, classifying according to a pavement-non-pavement identification model;
step1.2: connected ROIForbidden to useNon-pavement gridding picture I in areai,jGenerating a candidate object Ok；
Step1.3: for candidate target OkClassifying and identifying, if the vehicle is a vehicle, locking the vehicle as a vehicle target;
step1.4: calculating an initial time T0And T after time T0+tComparing the target displacement distance and direction of the vehicle with the direction of the vehicle on the normal road;
step1.5: and judging the illegal parking or the reverse driving behavior of the vehicle.
The practice of the invention is further illustrated by the following specific examples.
The flow diagram of the invention is shown in fig. 1, and the video picture is obtained by the mobile terminal road camera, and the CNN processing training classification is carried out to detect the road event:
step 1: the method comprises the steps of training a pavement model, namely segmenting a Region of Interest (ROI) of a video window into a plurality of small blocks, standardizing the small blocks to serve as a training set of a Deep-CNN network, and forcibly marking the pavement model as a single type which is easy to cause overfitting by taking the pavement model into consideration that the pavement model is composed of pavement asphalt, pavement lanes, guardrails and the like, so that unsupervised method training is adopted to obtain image features, and after clustering, the pavement is marked manually by a plurality of types of labels, so that the pavement and the non-pavement are accurately distinguished;
step 2: the non-road surface foreground model training, namely combining the divided small blocks divided into the non-road surface into candidate targets according to the connected regions, adding the candidate targets into a training library, performing Deep-CNN network training again, and training road surface targets including vehicles, road surface remnants, pedestrians and the like in a grading manner;
step 3: foreground target detection, namely realizing the detection and classification of moving and static foreground targets on the basis of step1 and step2 road surface identification models;
step 4: and behavior analysis, namely performing road event identification including vehicle illegal parking detection and vehicle converse running detection according to the context information on the basis of foreground target classification identification.
The network model principle of Deep convolutional neural network Deep-CNN is shown in fig. 2, and includes:
1) biophylactic nerve activation function:
the neuroscientist thinks that the human brain signal receiving process is the activation response of the nerve touch to the external input signal in essence, and the neuron node excitation can be regarded as the linear function activation, and the node value is 1 when the external stimulation is reflected, otherwise, the node value is 0. The energy measurement experiment of brain waves stimulated by input shows that the response mode of external stimulation and neuron nodes is sparse, namely only a part of neuron nodes are activated under a large number of stimulation signals, and only about 1-4% of neurons are in an active state at the same time when external information is received. In 2001, the neuroscientists Dayan and Abott suggested that the brain neuron signal reception and activation response model was modeled from a biological perspective. For ease of computation, Glorot et al propose a corrective nonlinear activation function Softplus, adapted to a machine learning multi-layer neural network, approximating a surrogate neuron model.
The Softplus function is defined as:
SortPlus(x)＝log(1+exp(x)) (1)
the corrective Linear activation functions (relus) are a Linear, simplified version of Softplus, defined as:
ReLU(x)＝max(0，x) (2)
the ReLU function is very simple to derive, where the derivative is 1 when x >0, otherwise 0, and the derivation formula can be expressed as:
the distribution of the curves of Softplus and ReLU activation functions is shown in FIG. 2(a)
2) Network architecture
Fig. 2(b) shows a multi-layer CNN network structure including basic layers such as an input layer, a hidden layer, and an output layer, where the hidden layer is formed by stacking a plurality of convolution-pooling sublayers, and the output of the previous layer is used as the input of the next layer. And activating the nodes by using the ReLU function after convolution calculation, wherein due to the derivative characteristics of the ReLU function, the weight of part of the nodes of which the initial values are subjected to the ReLU gradient calculation is set to be 0, and the part of the nodes which are weighted by 0 values are not activated, so that the network structure has sparsity. The sparse network structure conforms to the essence of biological nerves, has practical advantages in mathematical calculation, such as reduction of parameter complexity, improvement of operation efficiency and effective prevention of overfitting.
The whole process comprises the following specific operations:
the Deep-CNN pavement identification model is realized based on Caffe, candidate targets are extracted reversely by the pavement model, and functions of illegal parking detection, reverse running detection and the like are realized after classification.
Assuming that the input image size is w × h, the image is gridded into (w/s) × (h/s) small blocks using a step s, represented as a matrix sequence aij( i 1,2,3 … w/s; j 1,2,3 … h/s), the pixel width of step s is about 1/4 the width of the lane.
Detecting a region of interest ROIRoad surfacePoints { x1, y 1; x2, y 2; x3, y 3; x4.y 4; x1, y1} enclosed detection area, then AROI∈ROIRoad surfaceA matrix grid within the detection area is detected.
A of the road surface modelROISending the image blocks into Deep-CNN training, using an unsupervised method, extracting the characteristics of the pavement modules, clustering the pavement into sub-types of asphalt surfaces, lane mark, guardrails and the like, and marking the sub-types as pavement 1 to pavement n (n)<4). In order to improve the coverage of the pavement model samples, a partial pixel sampling method of downward and rightward offset is adopted in the gridding image process, and 3 x (w/s) x (h/s) pavement samples are obtained in total.
After the input image is gridded, a corresponding classification matrix is constructed, and a grid picture in a non-detection area of the road surface directly uses the overflowing waterSetting the pixel to 0, non-coverage area grid matrix
Figure BDA0001218382640000061
The method is initialized to-1, and after the grid image blocks of the detection area are classified by a Deep-CNN pavement model, a classification result matrix is output
Figure BDA0001218382640000062
Because the prior knowledge is difficult to construct the road surface remnant experience model, the invention adopts the road surface-non-road recognition to reversely realize the foreground detection and classification. Extracting foreground from CROIThe connected matrix sub-block of 1 constitutes a candidate target, and the prospect is clustered into vehicles, pedestrians, other targets (possibly road surface remnants) using a Deep-CNN target network.
FIG. 3 illustrates a Deep-CNN road model implementation process, in which an interest region, i.e., a detection region, is marked in FIG. 3(a), 4 vertexes are used to generate a closed ROI detection region, in which FIG. 3(b) is filled with non-detection region by flooding water, and after filling by flooding water, a grid image block I is formedi,jThe mean value of the grid picture pixels falling outside the ROI area is 0, and the grid picture pixels can be directly filtered before being sent to Deep-CNN. And classifying the ROI picture blocks in the detection area into a road surface or a non-road surface through Deep-CNN. ROI (region of interest)Road surfaceThe non-road grid pictures in the blocks are connected and marked as Ip,qI.e. by
Figure BDA0001218382640000066
Road surface, as shown by the bold marked area in FIG. 3(c), Ip,qAnd forming candidate targets, sending the candidate targets into a classifier, and classifying the candidate targets into vehicles, pedestrians or other undefined targets (namely road surface remnants).
Is provided with
Figure BDA0001218382640000063
Andare respectively T0And T0+tCalculating the position of the kth foreground candidate target at the moment, and calculating the position of the foreground candidate target in the imageEuclidean distance of region displacement
Figure BDA0001218382640000065
And acquiring the motion state and direction of the kth foreground candidate target, and further determining whether the target has a stop state or a retrograde motion state.
Fig. 4 demonstrates the road surface detection situation of the mobile photo of the handheld terminal, fig. 4(a) is an original photo, a lane line straight line detection is used for extracting a ROI detection region, and a non-ROI region is filled to black by using a water-spreading method so as to reduce the burden of a classifier, as shown in fig. 4(b), a road surface launching well cover, a vertical cone roadblock and a deceleration strip are all detected as non-road surfaces. For conventional pavement facilities such as well covers, roadblocks and the like, foreground images can be extracted to a training sample set to directly train a detection model.
Based on the above models, fig. 5(b), (c), and (d) respectively realize forbidden zone violation detection, vehicle retrograde motion detection, and vehicle retrograde motion detection.
Potential safety hazards are easily caused by behaviors such as parking or retrograde motion at the exit of the ramp of the expressway, parking in an emergency lane and the like, and the implementation of a Deep-CNN restricted zone parking detection model is demonstrated in figure 5. Marking out forbidden region ROI in road surfaceForbidden to useFor example, in the areas such as fishbone zebra crossing and emergency lane at the front end of the exit ramp road junction, the detection area image grid A is adoptedROIAnd (5) sending the road surface into Deep-CNN road surface model training to generate a road surface template of an exclusion zone, as shown in figure 5 (a). And identifying the vehicle in the detected foreground target area by the Deep-CNN foreground classifier again. With vehicle targetsNumber of rows M of matrix spanning gridrowFor the displacement threshold, assume the target
Figure BDA0001218382640000072
If the motion tracking is smaller than the set threshold value within the interval time t, the vehicle is judged to be parked, otherwise, the moving distance and the moving direction of the vehicle are calculated, if the motion direction dirctv is opposite to the driving direction dirctflow, the vehicle is judged to be in the reverse direction, and the parking and the reverse direction detection are respectively carried out in the images (b) to (d) in the image 5. Wherein the violation threshold is t>20S (500 Frames), Mrow<3, retrograde threshold Mrow>The direction of motion is dirctv-dirctflow, 3.

Claims (3)

1. The method for detecting the illegal parking and the reverse driving of the vehicle based on the Deep convolutional neural network is characterized in that a road surface region of interest is divided into a plurality of grids based on a Deep convolutional neural network Deep-CNN network model, a road surface-non-road surface recognition model is constructed, and illegal parking or reverse driving targets are reversely recognized through the non-road surface grids, and the method comprises the following steps:
step 1: training a road model, acquiring a video image of a road camera, and acquiring a region of interest (ROI) of the road surface in the video imageRoad surfaceThe method comprises the steps of performing gridding segmentation into a plurality of small blocks, using the small blocks as a training set of a Deep-CNN network model after standardization, firstly adopting an unsupervised method to train and obtain image characteristics during training, setting labels after clustering, marking road surface types in a manual mode, and distinguishing road surfaces from non-road surfaces to obtain a road surface-non-road surface recognition model;
step 2: the method comprises the steps of training a non-road surface foreground model, combining grid pictures divided into non-road surfaces into candidate targets according to connected regions, adding the candidate targets into a training library, training by adopting a Deep-CNN network model again, and training road surface targets in a grading mode, wherein the road surface targets comprise vehicles, road surface remnants and pedestrians to obtain a foreground identification model;
step 3: detecting a foreground target, namely detecting and classifying the foreground target by using a Deep-CNN network model and an SVM classifier on a real-time video image on the basis of the identification models of step1 and step2, firstly identifying a non-road surface of the road surface, and then identifying the type of the foreground target;
step 4: behavior analysis, namely performing illegal parking and converse running detection and identification on the vehicles in the road event according to the context information of the foreground target in the video image sequence on the basis of foreground target classification and identificationAnd
Figure FDA0002294064970000012
are respectively T0And T0+tTime of dayThe position of the kth foreground candidate target, and the Euclidean distance of the foreground candidate target in the image area displacement are calculated
Figure FDA0002294064970000014
And acquiring the motion state and direction of the kth foreground candidate target, and further determining whether the target has a stop state or a retrograde motion state.
2. The vehicle illegal parking converse running detection method based on the deep convolutional neural network as claimed in claim 1, wherein the detection and identification of the illegal parking converse running of the vehicles at the road event are specifically as follows:
step4.1: setting forbidden region ROI on road surface in real-time video imageForbidden to useAfter the video image is gridded, classifying according to a pavement-non-pavement identification model;
step4.2: connected ROIForbidden to useNon-pavement gridding picture I in areai,jGenerating a candidate object Ok；
Step4.3: for candidate target OkClassifying and identifying, if the vehicle is a vehicle, locking the vehicle as a vehicle target;
step4.4: calculating an initial time T0And T after time T0+tComparing the target displacement distance and direction of the vehicle with the direction of the vehicle on the normal road;
step4.5: and judging the illegal parking or the reverse driving behavior of the vehicle.
3. The method for detecting the vehicle illegal parking and the reverse driving based on the deep convolutional neural network as claimed in claim 1, wherein Step1 and Step2 are specifically as follows:
1) setting region of interest ROIRoad surface: collecting video images of a road or street view monitor to obtain each frame of road or street view video images, extracting boundary diagonal angles of an attention area from a current frame of image according to actual road or street view conditions, performing straight line fitting calculation on the extracted points to form a fork-shaped structureROI as a region of interest for detection within a fork structureRoad surfaceI.e. the effective detection area;
2) filling the non-detection area with water: non-ROIRoad surfaceFilling the region to be detected with overflowing water, and allowing the region to fall into ROI after fillingRoad surfaceThe pixel mean value of the grid picture outside the area is 0, and the subsequent processing is not carried out after the direct filtering;
3) detecting region ROIRoad surfaceGridding and blocking, classifying the gridding picture into a road surface or a non-road surface through Deep-CNN, and classifying the ROIRoad surfaceThe non-road grid pictures in the blocks are connected and marked as Ip,qI.e. by
Figure FDA0002294064970000021
Ip,qAnd forming candidate targets, sending the candidate targets into a classifier, and classifying the candidate targets into vehicles, pedestrians or road surface remnants.
CN201710059676.1A 2017-01-24 2017-01-24 Vehicle illegal parking and reverse running detection method based on deep convolutional neural network Active CN106874863B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710059676.1A CN106874863B (en) 2017-01-24 2017-01-24 Vehicle illegal parking and reverse running detection method based on deep convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710059676.1A CN106874863B (en) 2017-01-24 2017-01-24 Vehicle illegal parking and reverse running detection method based on deep convolutional neural network

Publications (2)

Publication Number Publication Date
CN106874863A CN106874863A (en) 2017-06-20
CN106874863B true CN106874863B (en) 2020-02-07

Family

ID=59159242

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710059676.1A Active CN106874863B (en) 2017-01-24 2017-01-24 Vehicle illegal parking and reverse running detection method based on deep convolutional neural network

Country Status (1)

Country Link
CN (1) CN106874863B (en)

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107491765A (en) * 2017-08-29 2017-12-19 张羽 A kind of shared bicycle and its parking identification device and system
CN109493606B (en) * 2017-09-11 2020-10-16 中国移动通信集团浙江有限公司 Method and system for identifying illegal parking vehicles on expressway
CN109523570B (en) * 2017-09-20 2021-01-22 杭州海康威视数字技术股份有限公司 Motion parameter calculation method and device
CN108256554A (en) * 2017-12-20 2018-07-06 深圳市金溢科技股份有限公司 Vehicle reverse stopping judgment method, server and system based on deep learning
SG11201909815RA (en) * 2018-03-12 2019-11-28 Jayant Ratti On-demand artificial intelligence and roadway stewardship system
CN110555347B (en) * 2018-06-01 2022-06-03 杭州海康威视数字技术股份有限公司 Vehicle target identification method and device with dangerous cargo-carrying behavior and electronic equipment
CN109063612A (en) * 2018-07-19 2018-12-21 中智城信息技术有限公司 City intelligent red line management method and machine readable storage medium
CN109670404B (en) * 2018-11-23 2023-07-11 江苏理工学院 Road ponding image detection early warning method based on hybrid model
CN109815831B (en) * 2018-12-28 2021-03-23 东软睿驰汽车技术（沈阳）有限公司 Vehicle orientation obtaining method and related device
CN109949578B (en) * 2018-12-31 2020-11-24 上海眼控科技股份有限公司 Vehicle line pressing violation automatic auditing method based on deep learning
CN109919053A (en) * 2019-02-24 2019-06-21 太原理工大学 A kind of deep learning vehicle parking detection method based on monitor video
CN109887281B (en) * 2019-03-01 2021-03-26 北京云星宇交通科技股份有限公司 Method and system for monitoring traffic incident
CN110399803B (en) * 2019-07-01 2022-04-22 北京邮电大学 Vehicle detection method and device
CN110580570B (en) * 2019-08-14 2021-01-15 平安国际智慧城市科技股份有限公司 Law enforcement analysis method, device and medium
CN110619750B (en) * 2019-08-15 2020-09-11 重庆特斯联智慧科技股份有限公司 Intelligent aerial photography identification method and system for illegal parking vehicle
CN110929676A (en) * 2019-12-04 2020-03-27 浙江工业大学 Deep learning-based real-time detection method for illegal turning around
CN111126171A (en) * 2019-12-04 2020-05-08 江西洪都航空工业集团有限责任公司 Vehicle reverse running detection method and system
CN111178215B (en) * 2019-12-23 2024-03-08 深圳成谷科技有限公司 Sensor data fusion processing method and device
CN111209880A (en) * 2020-01-13 2020-05-29 南京新一代人工智能研究院有限公司 Vehicle behavior identification method and device
CN113327414B (en) * 2020-02-28 2023-06-20 深圳市丰驰顺行信息技术有限公司 Vehicle reverse running detection method, device, computer equipment and storage medium
CN111368687B (en) * 2020-02-28 2022-07-19 成都市微泊科技有限公司 Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation
CN112085952B (en) * 2020-09-07 2022-06-03 平安科技（深圳）有限公司 Method and device for monitoring vehicle data, computer equipment and storage medium
CN112270668B (en) * 2020-11-06 2021-09-21 威海世一电子有限公司 Suspended cable detection method and system and electronic equipment
CN113378642B (en) * 2021-05-12 2022-05-06 三峡大学 Method for detecting illegal occupation buildings in rural areas
CN114360209B (en) * 2022-01-17 2023-06-23 常州信息职业技术学院 Video behavior recognition security system based on deep learning
CN115984221B (en) * 2023-01-03 2023-08-04 广州新粤交通技术有限公司 Road marking restoration and identification method, device, equipment and storage medium thereof

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101937614A (en) * 2010-06-12 2011-01-05 北京中科卓视科技有限责任公司 Plug and play comprehensive traffic detection system
CN102184547A (en) * 2011-03-28 2011-09-14 长安大学 Video-based vehicle reverse driving event detecting method
CN104809470A (en) * 2015-04-23 2015-07-29 杭州中威电子股份有限公司 Vehicle converse running detection device and method based on SVM
CN104809443A (en) * 2015-05-05 2015-07-29 上海交通大学 Convolutional neural network-based license plate detection method and system
CN105185108A (en) * 2015-08-06 2015-12-23 苏州市世跃智能科技有限公司 Automatic snapshot system of illegal parking at yellow grid lines

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120148092A1 (en) * 2010-12-09 2012-06-14 Gorilla Technology Inc. Automatic traffic violation detection system and method of the same
US9514391B2 (en) * 2015-04-20 2016-12-06 Xerox Corporation Fisher vectors meet neural networks: a hybrid visual classification architecture

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101937614A (en) * 2010-06-12 2011-01-05 北京中科卓视科技有限责任公司 Plug and play comprehensive traffic detection system
CN102184547A (en) * 2011-03-28 2011-09-14 长安大学 Video-based vehicle reverse driving event detecting method
CN104809470A (en) * 2015-04-23 2015-07-29 杭州中威电子股份有限公司 Vehicle converse running detection device and method based on SVM
CN104809443A (en) * 2015-05-05 2015-07-29 上海交通大学 Convolutional neural network-based license plate detection method and system
CN105185108A (en) * 2015-08-06 2015-12-23 苏州市世跃智能科技有限公司 Automatic snapshot system of illegal parking at yellow grid lines

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
From On-Road to Off:Transfer Learning Within a Deep Convolutional Neural Network for Segmentation and Classification of Off-Road Scenes;Christopher J.Holder et al.;《European Conference on Computer Vision》;20160918;第149-162页 *
基于图模型与卷积神经网络的交通标志识别方法;刘占文 等;《交通运输工程学报》;20161031;第16卷(第5期);第122-131页 *

Also Published As

Publication number Publication date
CN106874863A (en) 2017-06-20

Similar Documents

Publication Publication Date Title
CN106874863B (en) Vehicle illegal parking and reverse running detection method based on deep convolutional neural network
CN106845424B (en) Pavement remnant detection method based on deep convolutional network
CN111368687B (en) Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation
CN109670404B (en) Road ponding image detection early warning method based on hybrid model
WO2022126669A1 (en) Subway pedestrian flow network fusion method based on video pedestrian recognition, and pedestrian flow prediction method
KR102197946B1 (en) object recognition and counting method using deep learning artificial intelligence technology
Zheng et al. A novel vehicle detection method with high resolution highway aerial image
CN110738857B (en) Vehicle violation evidence obtaining method, device and equipment
CN103986910A (en) Method and system for passenger flow statistics based on cameras with intelligent analysis function
CN103208190B (en) Traffic flow detection method based on object detection
WO2013186662A1 (en) Multi-cue object detection and analysis
CN109448365A (en) Across the scale space base land regions road traffic system integrated supervision method of one kind
CN105574488A (en) Low-altitude aerial infrared image based pedestrian detection method
US20230306573A1 (en) Systems and methods for assessing infrastructure
CN104320617A (en) All-weather video monitoring method based on deep learning
Bharadwaj et al. Traffic data analysis using image processing technique on Delhi–Gurgaon expressway
CN113450573A (en) Traffic monitoring method and traffic monitoring system based on unmanned aerial vehicle image recognition
Yeshwanth et al. Estimation of intersection traffic density on decentralized architectures with deep networks
CN114372503A (en) Cluster vehicle motion trail prediction method
Sanjeewani et al. Learning and analysis of AusRAP attributes from digital video recording for road safety
WO2021236549A1 (en) Systems and methods for creating and/or analyzing three-dimensional models of infrastructure assets
Neto et al. Computer-vision-based surveillance of intelligent transportation systems
KR20220071822A (en) Identification system and method of illegal parking and stopping vehicle numbers using drone images and artificial intelligence technology
CN113850111A (en) Road condition identification method and system based on semantic segmentation and neural network technology
CN113850112A (en) Road condition identification method and system based on twin neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant