CN112289037A - Motor vehicle illegal parking detection method and system based on high visual angle under complex environment - Google Patents

Motor vehicle illegal parking detection method and system based on high visual angle under complex environment Download PDF

Info

Publication number
CN112289037A
CN112289037A CN202011176098.8A CN202011176098A CN112289037A CN 112289037 A CN112289037 A CN 112289037A CN 202011176098 A CN202011176098 A CN 202011176098A CN 112289037 A CN112289037 A CN 112289037A
Authority
CN
China
Prior art keywords
motor vehicle
illegal parking
vehicle
detection
license plate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011176098.8A
Other languages
Chinese (zh)
Other versions
CN112289037B (en
Inventor
何小峰
张堃
冯文宇
朱远璠
张宇豪
祁晖
陆贝洋
何秀平
戴璐
陈建锋
张树
涂鑫涛
徐沛霞
刘志诚
黄宇煦
韩宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nantong Zhongtie Huayu Electrics Co ltd
Nantong University
Original Assignee
Nantong Zhongtie Huayu Electrics Co ltd
Nantong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nantong Zhongtie Huayu Electrics Co ltd, Nantong University filed Critical Nantong Zhongtie Huayu Electrics Co ltd
Priority to CN202011176098.8A priority Critical patent/CN112289037B/en
Publication of CN112289037A publication Critical patent/CN112289037A/en
Application granted granted Critical
Publication of CN112289037B publication Critical patent/CN112289037B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
    • G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Abstract

The invention discloses a motor vehicle illegal parking detection method and system based on a high visual angle under a complex environment, which comprises the following steps: receiving a motor vehicle illegal parking detection request transmitted by a system, and acquiring motor vehicle monitoring video information corresponding to the motor vehicle illegal parking detection request; extracting a vehicle image key frame in the motor vehicle parking monitoring video information; inputting the vehicle image into a detection model for processing to obtain a detection result of illegal parking of the motor vehicle; when the detection result of the motor vehicle illegal parking is in an abnormal state, acquiring license plate image information of the motor vehicle; obtaining the license plate number of the vehicle through a license plate recognition module; and determining the identity information of the driver, and performing management and control processing on the driver according to the identity information of the driver. The method and the device can quickly identify whether the motor vehicle is illegal to park in the forbidden parking area, save the management and control cost of illegal parking detection of the motor vehicle, and improve the management and control efficiency and operability of illegal parking detection of the motor vehicle.

Description

Motor vehicle illegal parking detection method and system based on high visual angle under complex environment
Technical Field
The invention belongs to the technical field of image processing and machine learning, and particularly relates to a light-weight and high-precision motor vehicle illegal parking detection method based on high-viewing-angle monitoring.
Background
With the improvement of the living standard of residents, nowadays, basically every family has a private car. Therefore, the phenomenon of vehicle parking is also endless, and the vehicle parking not only can cause the taste and image of the city to be influenced, but also can increase the occurrence of collision accidents and rear-end accidents, and can also cause unnecessary blockage to influence the normal traffic of other vehicles and pedestrians.
In the face of such a situation, a traffic police usually patrols and fines and deducts the points of the illegal vehicles, but most of the illegal vehicles still cannot be punished due to limited police force. Therefore, the vehicle illegal parking detection method based on the monitoring video is provided, the vehicle illegal parking is detected through monitoring, the detection precision is guaranteed, and meanwhile, the investment of manpower and material resources is greatly reduced. A license plate detection method is also integrated, license plate numbers of vehicles violating parking are identified, and punishment education is performed on the vehicle owners.
Compared with the problem of small storage capacity of the traditional computing technology, the cloud computing technology can be used for infinite storage, the cost of a computer and software can be reduced, and meanwhile, the data reliability is improved, and the compatibility of a document format is improved.
In the field of vehicle illegal parking event detection by using an object interaction model, a vehicle illegal parking event detection method based on the object interaction model is provided, and the method has the following defects: the traffic sign which prohibits parking is identified based on the color and the shape, and the method has high requirements on a camera and is easily influenced by the color of the surrounding environment. An SVM algorithm is adopted when a license plate is recognized, and the algorithm has the main problems that input data need to be comprehensively marked during training, and large-scale training samples are difficult to implement.
In the sidewalk illegal parking detection text based on target detection and semantic segmentation, a sidewalk parking detection algorithm based on deep learning and semantic segmentation algorithms is provided. The method has the following defects: the scheme is only used for detecting whether the vehicle stops on the pedestrian crossing or not, and the application range is narrow. In the selection of the training data set, only three types of coaches, tricycles and buses are adopted, other vehicle types are not involved, and the problem of recognition of certain pictures is caused. When the image is subjected to semantic segmentation, the image is only divided into three types, namely a road, a sidewalk and an automobile, the obtained result of the classification method is rough, and the accuracy and the speed of the classification method are still to be improved in the case of some very complex urban roads. And the key license plate recognition is lacked, and subsequent related punishment cannot be given to the parking violation owner.
In the vehicle illegal parking detection evidence obtaining literature based on the quad-rotor unmanned aerial vehicle, a vehicle illegal parking detection evidence obtaining method based on the quad-rotor unmanned aerial vehicle is provided. The method has the following defects: when the illegal vehicle is identified, each frame of image is required to be identified, the memory space is occupied, and the identification efficiency is low. Moreover, the cruising ability of the unmanned aerial vehicle is poor, the flying noise is large, and the scheme cost is increased.
Disclosure of Invention
The purpose of the invention is as follows: the invention aims to overcome the defects in the prior art, provides a motor vehicle illegal parking detection method and system based on a high visual angle in a complex environment, and aims to solve the technical problems of high cost, low efficiency, poor flexibility and the like of the motor vehicle illegal parking detection method in the prior art.
The technical scheme is as follows: the invention discloses a motor vehicle illegal parking detection method and system based on a high visual angle under a complex environment, which comprises the following steps:
s1, receiving a motor vehicle illegal parking detection request transmitted by a system, and acquiring motor vehicle monitoring video information corresponding to the motor vehicle illegal parking detection request;
s2, extracting vehicle image key frames in the motor vehicle monitoring video information;
s3, uploading the images of the key frames of the vehicle images to a system cloud;
s4, inputting the image of the vehicle image key frame uploaded to the cloud of the system into a motor vehicle illegal parking detection model for processing to obtain a motor vehicle illegal parking detection result;
s5, dividing the motor vehicle illegal parking detection result into two states: a normal state and an abnormal state;
s6, when the detection result of the motor vehicle illegal parking is in an abnormal state, determining the characteristic information of the motor vehicle according to the vehicle image key frame, and focusing a monitoring camera on the motor vehicle to obtain the license plate image information of the motor vehicle;
s7, extracting a license plate image key frame of the motor vehicle from the license plate image information of the motor vehicle;
s8, inputting the key frame image of the license plate image of the motor vehicle into a license plate detection model of the motor vehicle for processing to obtain the license plate number of the motor vehicle;
s9, after obtaining the license plate number of the illegal parking vehicle, uploading the license plate number to a system cloud end to determine the identity information of the driver of the illegal parking vehicle, and performing management and control processing on the driver according to the identity information of the driver.
Further, the step S1 specifically includes:
(a) receiving the motor vehicle illegal parking detection request, and continuously shooting at a certain initial frame rate through an installed monitoring acquisition device to obtain monitoring video information of the motor vehicle;
(b) judging whether a shooting detection blind area which is not covered by the monitoring acquisition device exists according to the monitoring video information of the shooting area;
(c) if the shooting detection blind area which is not covered by the monitoring acquisition device exists, calling other cameras closest to the monitoring acquisition device to shoot the shooting detection blind area in multiple angles, obtaining the multiple-view motor vehicle illegal parking detection video information, and combining the multiple-view motor vehicle illegal parking detection video information to serve as the motor vehicle monitoring video information corresponding to the motor vehicle illegal parking detection request.
Further, in step S5: extracting a plurality of same image key frames of the motor vehicle from the monitoring video information of the motor vehicle, comparing the image key frames, and judging the motor vehicle to be in a normal state if the system judges that the motor vehicle is in a driving state; if the system determines that the motor vehicle is in a stopped-running state, it is determined as an abnormal state.
Further, the specific step of the management and control processing in step S9 includes: and when the motor vehicle illegal parking detection result indicates that the motor vehicle is illegally parked, the system automatically notifies a traffic police nearby the motor vehicle, uploads the illegal record of the vehicle driver to a system cloud end, and relevant mechanisms give education and management to the illegal vehicle driver.
Furthermore, the monitoring acquisition device comprises a lamp post, a monitoring camera arranged on the lamp post and an LED display screen arranged on the lamp post, wherein the monitoring camera is used for acquiring video monitoring information of illegal motor vehicle parking, and the LED display screen is used for reminding a motor vehicle driver of the condition that the automobile vehicle driver is forbidden to park in the area or the speed limit condition of the area, so that the smooth road traffic is ensured and the public traffic safety is improved.
Further, the motor vehicle illegal parking detection model is built based on an improved YOLOV3-TINY backbone network, appropriate anchor boxes are obtained by optimizing a K-means algorithm, a CBAM attention module and a CSPNET-NEW network are added into the convolutional layer, an activation function is modified into a Mish activation function, a CIOU loss function is used, and pruning is conducted on part of the convolutional layer to complete optimization of model precision and quantification of the model.
Further, obtaining suitable anchor boxes by optimizing the K-means algorithm specifically includes:
(a) firstly, calculating the IOU values of a bounding box of a group route and k anchor boxes;
(b) initializing a clustering center;
(c) the distance between each ground channel and k anchor boxes is calculated using the following distance formula:
d(box，anchor)＝1-IO U(box，anchor)
wherein, the box and the anchor are respectively a boundingbox and k anchor boxes of a group channel; for each group channel, selecting the index of the group channel with the minimum distance from the anchor box, and storing the index;
(d) if the index of each group channel with the minimum distance from the anchor box is the same as the index of the previous group channel, finishing clustering; if not, the index of the closest group to each anchor box is updated.
Further, the CBAM attention module includes two independent modules: a channel attention module and a spatial attention module.
Furthermore, the CSPNET-NEW network integrates the gradient change into the feature map from beginning to end, and changes the leak activation function of the original convolutional layer into a Mish activation function.
Further, the CIOU loss function takes into account the distance, overlap ratio and scale between the target and the frame, so that the regression of the target frame becomes more stable:
Figure BDA0002748709190000051
wherein, bgtRespectively representing the central points of the prediction frame and the real frame, wherein rho represents the Euclidean distance between the two central points, and c represents the diagonal distance of the minimum closure area which can simultaneously contain the prediction frame and the real frame;
in addition, the CIOU loss function also considers the aspect ratio of the three elements of the bounding box regression, and also adds an influence factor in the penalty term:
Figure BDA0002748709190000052
where a is a parameter of the penalty term,
Figure BDA0002748709190000053
v measure length and widthSimilarity of ratio, defined as
Figure BDA0002748709190000054
Further, the pruning completion model comprises:
uniformly clipping the parameters of two detection branches of the model by respectively adopting a certain numerical clipping rate in a clipping mode based on sensitivity, and calculating the L1 norm of a convolution kernel w _ L ^ k corresponding to the connection by a method for evaluating the contribution of a certain connection in the following calculation mode:
Figure BDA0002748709190000061
wherein the content of the first and second substances,
Figure BDA0002748709190000062
for the loss value, N is the sample,
Figure BDA0002748709190000063
is the true value of the ith sample,
Figure BDA0002748709190000064
is the L1 norm of the weight.
The invention also discloses a system for detecting illegal parking of motor vehicles based on high visual angle in complex environment, which comprises:
the request receiving module is used for receiving a motor vehicle illegal parking detection request and acquiring the motor vehicle monitoring video information corresponding to the motor vehicle illegal parking detection request;
the video extraction module is used for extracting vehicle image key frames in the motor vehicle illegal parking monitoring video information;
the illegal parking detection module is used for obtaining the illegal parking detection result of the motor vehicle according to the processing result of the vehicle key frame image detection;
and the license plate recognition module is used for extracting the license plate picture information of the illegal vehicle and determining the identity information of the driver according to the recognized license plate information.
Has the advantages that: the detection method and the detection system can find the illegal parking vehicles in time, assist relevant departments to search the illegal parking vehicles, save the management and control cost of the illegal parking detection of the motor vehicles, and improve the management and control efficiency and operability of the illegal parking detection of the motor vehicles.
Drawings
FIG. 1 is a main workflow diagram of a method for detecting illegal parking of a motor vehicle according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a monitoring and collecting device according to an embodiment of the present invention;
FIG. 3 is a backbone network of an illegal motor vehicle parking detection model of the present invention;
FIG. 4 is a schematic diagram of the CMBA attention module in the motor vehicle illegal parking detection model of the present invention;
FIG. 5 is a schematic diagram of a channel attention module in a CMBA attention module in a motor vehicle illegal parking detection model according to the present invention;
FIG. 6 is a schematic diagram of a spatial attention module in a CMBA attention module in the motor vehicle illegal parking detection model according to the present invention;
FIG. 7 is a schematic diagram of a CSPNET-NEW network structure in the illegal motor vehicle parking detection model according to the present invention;
fig. 8 is a schematic diagram of the recognition result in the illegal parking detection model of the motor vehicle according to the invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
As shown in fig. 1, the method and system for detecting illegal parking of a motor vehicle based on a high viewing angle in a complex environment according to the embodiment of the present invention includes the following steps:
s1, receiving a motor vehicle illegal parking detection request transmitted by a system, and acquiring motor vehicle monitoring video information corresponding to the motor vehicle illegal parking detection request;
s2, extracting vehicle image key frames in the motor vehicle monitoring video information;
s3, uploading the images of the key frames of the vehicle images to a system cloud, and calculating by the system cloud to greatly reduce the time required by image processing;
s4, inputting the image of the vehicle image key frame uploaded to the cloud of the system into a motor vehicle illegal parking detection model for processing to obtain a motor vehicle illegal parking detection result;
s5, dividing the motor vehicle illegal parking detection result into two states: a normal state and an abnormal state; according to the processing result of the key frame image identified by the camera, different states are divided into: and (3) normal state: mainly defined as the non-violation parking of the motor vehicle; abnormal state: mainly defined as the illegal parking of the motor vehicle;
s6, when the detection result of the motor vehicle illegal parking is in an abnormal state, determining the characteristic information of the motor vehicle according to the vehicle image key frame, and focusing a monitoring camera on the motor vehicle to obtain the license plate image information of the motor vehicle;
s7, extracting a license plate image key frame of the motor vehicle from the license plate image information of the motor vehicle;
s8, inputting the key frame image of the license plate image of the motor vehicle into a license plate detection model of the motor vehicle for processing to obtain the license plate number of the motor vehicle;
s9, after obtaining the license plate number of the illegal parking vehicle, uploading the license plate number to a system cloud end to determine the identity information of the driver of the illegal parking vehicle, and performing management and control processing on the driver according to the identity information of the driver.
Preferably, in this embodiment, in order to improve the accuracy of the monitoring video information of the motor vehicle, the step S1 specifically includes:
(a) receiving the motor vehicle illegal parking detection request, and continuously shooting at a certain initial frame rate through an installed monitoring acquisition device to obtain monitoring video information of the motor vehicle;
(b) judging whether a shooting detection blind area which is not covered by the monitoring acquisition device exists according to the monitoring video information of the shooting area;
(c) if the shooting detection blind area which is not covered by the monitoring acquisition device exists, calling other cameras closest to the monitoring acquisition device to shoot the shooting detection blind area in multiple angles, obtaining the multiple-view motor vehicle illegal parking detection video information, and combining the multiple-view motor vehicle illegal parking detection video information to serve as the motor vehicle monitoring video information corresponding to the motor vehicle illegal parking detection request.
In this embodiment, preferably, the monitoring and collecting device, as shown in an implementation manner shown in fig. 2, includes a lamp post, a monitoring camera installed at a high position of the lamp post, and an LED display screen installed on the lamp post, where the monitoring camera installed at the high position of the lamp post is used to collect video monitoring information of illegal parking of the motor vehicle, and the LED display screen installed on the lamp post is used to remind a driver of the motor vehicle of the situation that parking is prohibited in the area or the speed of the area is limited, so as to ensure smooth road traffic and improve public traffic safety.
In this embodiment, preferably, in step S5: extracting a plurality of same image key frames of the motor vehicle from the monitoring video information of the motor vehicle, comparing the image key frames, and judging the motor vehicle to be in a normal state if the system judges that the motor vehicle is in a driving state; if the system determines that the motor vehicle is in a stopped-running state, it is determined as an abnormal state.
In this embodiment, preferably, the specific step of the management and control processing in step S9 includes: and when the motor vehicle illegal parking detection result indicates that the motor vehicle is illegally parked, the system automatically notifies a traffic police nearby the motor vehicle, uploads the illegal record of the vehicle driver to a system cloud end, and relevant mechanisms give education and management to the illegal vehicle driver.
In this embodiment, preferably, the illegal parking detection model of the motor vehicle adopted by the invention is built based on an improved YOLOV3-TINY backbone network, as shown in fig. 3, suitable anchor boxes are obtained by optimizing a K-means algorithm, a CBAM attention module and a CSPNET-NEW network are added to the convolutional layer, an activation function is modified into a mix activation function, a CIOU loss function is used, and pruning is performed on part of the convolutional layer to complete optimization of model precision and quantization of the model.
The following provides a detailed description of the various parts of the illegal parking detection model of a motor vehicle according to the present invention:
(1) improved Yolov3-TINY backbone network
YOLOV3 uses an anchor box for best matching the width and height of the target of detection when performing bounding box prediction, and since the sizes of the motor vehicles to be detected are different, the YOLOV3 is adapted to the anchor boxes with different sizes.
For computer vision, it is easier to understand a ground truth (real label), a label artificially marked for each target. But after adding the prior box concept, each anchor box is considered as a training sample in the training set. Therefore, to train the target model, a label for each prior box needs to be labeled, where the label includes two parts: class labels and offsets.
During training, the prior frame is also used for processing to find the position grid points of the real labels. The width and height of countless targets can be predicted according to a certain grid of a certain characteristic layer, and prediction is carried out by referring to the size of a prior frame.
Taking a priori frame preset by YOLOV3 as an example, each layer of network outputs feature maps of 3 scales, which are respectively 13 × 13, 26 × 26 and 52 × 52, corresponding to 9 priori frames, and each scale equally divides 3 priori frames. On the smallest 13 × 13 feature map, the largest prior box (116 × 90), (156 × 198), (373 × 326) is applied because its receptive field is the largest.
Using K-means clustering, the IOU (cross-over ratio) values of the anchor box and K prior boxes of a label data are first calculated. Since the detection object is a small target and the length and width of the target detection frame are uniform, K in the tentative K-means model is 6. Particularly, when a cluster center is initialized, if k prior frames exist, the width and the height of 1 anchor frame are randomly selected from r real labels as the cluster center, and the shortest distance between r-1 real labels and the current existing cluster center (namely the distance to the nearest cluster center) is calculated. The width and height of the real label with the largest distance is selected as the second center of the cluster. The above steps are then repeated until 6 cluster centers are found.
The distance between each real tag and k prior boxes is calculated using distance equation (1):
d(box，anchor)＝1-IOU(box，anchor) (1)
wherein, box and anchor are respectively an anchor frame and k prior frames of the real label.
And for each real label, selecting the index of the real label with the minimum distance from the prior frame, and storing the index.
Specifically, when calculating the distance between each real tag and K prior boxes, two algorithms of elkan K-Means are used that optimize the amount of distance calculation. The first algorithm is for one sample point x and two cluster centers μj1，μj2. The distance D (j) between every two centroids is calculated in advance1,j2) Then if the calculation finds 2D (x, j)1)≤D(j1，j2) Then D (x, j) is known1)≤D(x，j2) At this time, D (x, j) does not need to be calculated again2) The distance calculation is omitted. The second algorithm is for two cluster centers muj1，μj2To obtain D (x, j)2)≥max{0,D(x，j1)-D(j1，j2)}。
If the index of each current real label with the minimum distance from the prior frame is the same as the index of the last time, finishing clustering; if not, the index of the nearest real label to each prior frame is updated.
(2) CBAM attention Module
The CBAM attention module is added into the motor vehicle illegal parking detection model to improve the detection precision and quantify the model.
The CBAM attention module is a simple but effective attention module. For an intermediate feature map, attention weights can be deduced sequentially along two dimensions of space and channels, and then the attention weights are multiplied by the original feature map to perform adaptive adjustment on features. The whole process is divided into two independent modules: a channel attention module and a spatial attention module. Therefore, parameters and calculated amount can be saved, and the module can be used as a plug-and-play module to improve the detection precision of the model.
In the motor vehicle illegal parking detection model of the present invention, the CBAM attention module is added to the second layer of convolutional layers, wherein:
fig. 5 shows a channel attention module. Given the H × W × C feature F, performing global average pooling and maximum pooling on the space respectively yields two 1 × 1 × C channel features. Then, the neurons are sent into a shared two-layer neural network, the number of the neurons in the first layer is C/r, the activation function is Relu, and the number of the neurons in the second layer is C. And adding the two obtained features, obtaining a weight coefficient Mc through a Sigmoid activation function, and finally multiplying the weight coefficient by the original feature to obtain a new feature.
Fig. 6 shows a spatial attention module. Given the H multiplied by W multiplied by C characteristic F, the intermediate characteristic is processed differently by two pooling modes, namely, the average pooling and the maximum pooling are carried out on the channel sequentially, so as to obtain two H multiplied by W multiplied by 1 channel characteristics. And splicing the two layers of characteristics together, and passing through a convolution layer of 7 multiplied by 7 to obtain the weight coefficient Ms, wherein the activation function is Sigmoid. And finally, multiplying the number by the characteristic F by taking the weight to obtain a new characteristic.
Combining fig. 5 and 6 is the CBAM module of fig. 4.
(3) CSPNET-NEW network
As shown in fig. 7, a NEW CSPNET-NEW structure is obtained by improving CSPNET (cross Stage Partial network) to enhance the feature extraction, so as to obtain higher accuracy, and by integrating the gradient change into the feature map from beginning to end, the accuracy can be ensured while the calculation amount is reduced. Unlike CSPNET, CSPNET-NEW first changes all the leak activation functions of the original convolutional layer to Mish activation functions.
The Mish activation function is shown in equation (2):
f(x)＝x*tanh(log(1+ex)) (2)
mish is a self-regularizing non-monotonic neural activation function, and a smooth activation function allows better information to enter the neural network, resulting in better accuracy and generalization.
In particular, in the original large residual structure, two superposed resblocks are added to better extract more abstract features and ensure less parameters.
(4) CIOU loss function
Generally, the IOU loss function is used for the target detection loss function, and because the IOU is a ratio concept, when the prediction frame and the target frame are not intersected and the IOU (A, B) is 0, the distance between the A and the B cannot be reflected, the loss function is not conducive at this time, and the IOU loss function cannot optimize the condition that the two frames are not intersected.
Assuming that the sizes of the prediction box and the target box are determined, as long as the intersection value of the two boxes is determined, and the IoU value is the same, the IoU value cannot reflect how the two boxes intersect.
It is insensitive to the range of the target object. Based on the reason, the proposed DIOU is more in line with the mechanism of target frame regression than the IOU, and the distance, the overlapping rate and the scale between the target and the frame are taken into consideration, so that the target frame regression becomes more stable, and the problems of divergence and the like in the training process like the IOU do not occur, as shown in formula (3):
Figure BDA0002748709190000131
b and bgt represent central points of the prediction frame and the real frame respectively, rho represents the Euclidean distance between the two central points, and c represents the diagonal distance of the minimum closure area which can contain the prediction frame and the real frame simultaneously.
In addition, the CIOU loss function also considers the aspect ratio of the bounding box regression three elements, and also adds an influence factor to the penalty term, as shown in formula (4):
Figure BDA0002748709190000132
where a is a parameter of the penalty term,
Figure BDA0002748709190000133
v measures the similarity of the aspect ratio, defined as
Figure BDA0002748709190000134
(5) Pruning quantification model
Uniformly clipping the parameters of two detection branches of the model by respectively adopting clipping rates of 0.9 and 0.9 by adopting a clipping mode based on sensitivity, and calculating the L1 norm of a convolution kernel w _ L ^ k corresponding to the connection by a method for evaluating the contribution degree of a certain connection in the way of formula (5):
Figure BDA0002748709190000135
wherein the content of the first and second substances,
Figure BDA0002748709190000141
for the loss value, N is the sample,
Figure BDA0002748709190000142
is the true value of the ith sample,
Figure BDA0002748709190000143
is the L1 norm of the weight.
The sampling includes two assumption principles:
the convolution kernels are sorted from high to low in the parameter of one convolution layer, and the weight occupied by the convolution kernels at the back is smaller.
And secondly, the two convolutional layers cut the convolutional kernels with the same proportion, and the sensitivity of the convolutional layers which have larger influence on the model precision is relatively high.
The clipping rate of a convolutional layer is inversely proportional to its sensitivity, and lower convolutional kernels are clipped preferentially.
In addition, the invention also provides a system for detecting illegal parking of motor vehicles based on a high visual angle under a complex environment, which comprises the following steps:
the request receiving module is used for receiving a motor vehicle illegal parking detection request and acquiring the motor vehicle monitoring video information corresponding to the motor vehicle illegal parking detection request;
the video extraction module is used for extracting vehicle image key frames in the motor vehicle illegal parking monitoring video information;
the illegal parking detection module is used for obtaining the illegal parking detection result of the motor vehicle according to the processing result of the vehicle key frame image detection;
and the license plate recognition module is used for extracting the license plate picture information of the illegal vehicle and determining the identity information of the driver according to the recognized license plate information.
The detection method and the detection system can find the illegal parking vehicles in time, assist relevant departments to search the illegal parking vehicles, save the management and control cost of the illegal parking detection of the motor vehicles, and improve the management and control efficiency and operability of the illegal parking detection of the motor vehicles.
Although the present invention has been described with reference to a preferred embodiment, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (12)

1. A motor vehicle illegal parking detection method and system based on a high visual angle under a complex environment are characterized in that: the method comprises the following steps:
s1, receiving a motor vehicle illegal parking detection request transmitted by a system, and acquiring motor vehicle monitoring video information corresponding to the motor vehicle illegal parking detection request;
s2, extracting vehicle image key frames in the motor vehicle monitoring video information;
s3, uploading the images of the key frames of the vehicle images to a system cloud;
s4, inputting the image of the vehicle image key frame uploaded to the cloud of the system into a motor vehicle illegal parking detection model for processing to obtain a motor vehicle illegal parking detection result;
s5, dividing the motor vehicle illegal parking detection result into two states: a normal state and an abnormal state;
s6, when the detection result of the motor vehicle illegal parking is in an abnormal state, determining the characteristic information of the motor vehicle according to the vehicle image key frame, and focusing a monitoring camera on the motor vehicle to obtain the license plate image information of the motor vehicle;
s7, extracting a license plate image key frame of the motor vehicle from the license plate image information of the motor vehicle;
s8, inputting the key frame image of the license plate image of the motor vehicle into a license plate detection model of the motor vehicle for processing to obtain the license plate number of the motor vehicle;
s9, after obtaining the license plate number of the illegal parking vehicle, uploading the license plate number to a system cloud end to determine the identity information of the driver of the illegal parking vehicle, and performing management and control processing on the driver according to the identity information of the driver.
2. The method and system for detecting illegal parking of motor vehicles based on high visual angle under complex environment according to claim 1 are characterized in that: the step S1 specifically includes:
(a) receiving the motor vehicle illegal parking detection request, and continuously shooting at a certain initial frame rate through an installed monitoring acquisition device to obtain monitoring video information of the motor vehicle;
(b) judging whether a shooting detection blind area which is not covered by the monitoring acquisition device exists according to the monitoring video information of the shooting area;
(c) if the shooting detection blind area which is not covered by the monitoring acquisition device exists, calling other cameras closest to the monitoring acquisition device to shoot the shooting detection blind area in multiple angles, obtaining the multiple-view motor vehicle illegal parking detection video information, and combining the multiple-view motor vehicle illegal parking detection video information to serve as the motor vehicle monitoring video information corresponding to the motor vehicle illegal parking detection request.
3. The method and system for detecting illegal parking of motor vehicles based on high visual angle under complex environment according to claim 1 are characterized in that: in the step S5: extracting a plurality of same image key frames of the motor vehicle from the monitoring video information of the motor vehicle, comparing the image key frames, and judging the motor vehicle to be in a normal state if the system judges that the motor vehicle is in a driving state; if the system determines that the motor vehicle is in a stopped-running state, it is determined as an abnormal state.
4. The method and system for detecting illegal parking of motor vehicles based on high visual angle under complex environment according to claim 1 are characterized in that: the specific steps of the management and control processing in step S9 include: and when the motor vehicle illegal parking detection result indicates that the motor vehicle is illegally parked, the system automatically notifies a traffic police nearby the motor vehicle, uploads the illegal record of the vehicle driver to a system cloud end, and relevant mechanisms give education and management to the illegal vehicle driver.
5. The method and the system for detecting illegal parking of motor vehicles based on high visual angle under complex environment according to claim 2 are characterized in that: the monitoring and collecting device comprises a lamp post, a monitoring camera arranged on the lamp post and an LED display screen arranged on the lamp post, wherein the monitoring camera is used for collecting video monitoring information of illegal motor vehicle parking, and the LED display screen is used for reminding a motor vehicle driver that the motor vehicle driver is forbidden to park in the area or the speed limit condition of the area is avoided, so that smooth road traffic is guaranteed and public traffic safety is improved.
6. The method and the system for detecting illegal parking of motor vehicles based on high visual angle under complex environment according to any one of claims 1 to 5 are characterized in that: the detection model for detecting the illegal parking of the motor vehicle is built based on an improved YOLOV3-TINY main network, a proper anchor boxes are obtained by optimizing a K-means algorithm, a CBAM attention module and a CSPNET-NEW network are added into a convolutional layer, an activation function is modified into a Mish activation function, a CIOU loss function is used, and pruning is carried out on part of the convolutional layer to complete optimization of model precision and quantification of the model.
7. The method and system for detecting illegal parking of motor vehicles based on high visual angle under complex environment according to claim 6 are characterized in that: obtaining suitable anchor boxes by optimizing the K-means algorithm specifically comprises the following steps:
(a) firstly, calculating the IOU values of a bounding box of a group route and k anchor boxes;
(b) initializing a clustering center;
(c) the distance between each ground channel and k anchor boxes is calculated using the following distance formula:
d(box，anchor)＝1-IO U(box，anchor)
wherein, the box and the anchor are respectively a boundingbox and k anchor boxes of a group channel; for each group channel, selecting the index of the group channel with the minimum distance from the anchor box, and storing the index;
(d) if the index of each group channel with the minimum distance from the anchor box is the same as the index of the previous group channel, finishing clustering; if not, the index of the closest group to each anchor box is updated.
8. The method and system for detecting illegal parking of motor vehicles based on high visual angle under complex environment according to claim 6 are characterized in that: the CBAM attention module includes two independent modules: a channel attention module and a spatial attention module.
9. The method and system for detecting illegal parking of motor vehicles based on high visual angle under complex environment according to claim 6 are characterized in that: the CSPNET-NEW network integrates the gradient change into a characteristic diagram from beginning to end, and completely changes the leak activation function of the original convolutional layer into a Mish activation function.
10. The method and system for detecting illegal parking of motor vehicles based on high visual angle under complex environment according to claim 6 are characterized in that: the CIOU loss function takes the distance, the overlapping rate and the scale between the target and the frame into consideration, so that the regression of the target frame becomes more stable:
Figure FDA0002748709180000041
wherein, bgtRespectively representing the central points of the prediction frame and the real frame, wherein rho represents the Euclidean distance between the two central points, and c represents the diagonal distance of the minimum closure area which can simultaneously contain the prediction frame and the real frame;
in addition, the CIOU loss function also considers the aspect ratio of the three elements of the bounding box regression, and also adds an influence factor in the penalty term:
Figure FDA0002748709180000042
where a is a parameter of the penalty term,
Figure FDA0002748709180000043
v measures the similarity of the aspect ratio, defined as
Figure FDA0002748709180000044
11. The method and system for detecting illegal parking of motor vehicles based on high visual angle under complex environment according to claim 6 are characterized in that: the pruning completion model comprises:
uniformly clipping the parameters of two detection branches of the model by respectively adopting a certain numerical clipping rate in a clipping mode based on sensitivity, and calculating the L1 norm of a convolution kernel w _ L ^ k corresponding to the connection by a method for evaluating the contribution of a certain connection in the following calculation mode:
Figure FDA0002748709180000051
wherein the content of the first and second substances,
Figure FDA0002748709180000052
for the loss value, N is the sample,
Figure FDA0002748709180000053
is the true value of the ith sample,
Figure FDA0002748709180000054
is the L1 norm of the weight.
12. A motor vehicle illegal parking detection system based on a high visual angle under a complex environment is characterized in that: the method comprises the following steps:
the request receiving module is used for receiving a motor vehicle illegal parking detection request and acquiring the motor vehicle monitoring video information corresponding to the motor vehicle illegal parking detection request;
the video extraction module is used for extracting vehicle image key frames in the motor vehicle illegal parking monitoring video information;
the illegal parking detection module is used for obtaining the illegal parking detection result of the motor vehicle according to the processing result of the vehicle key frame image detection;
and the license plate recognition module is used for extracting the license plate picture information of the illegal vehicle and determining the identity information of the driver according to the recognized license plate information.
CN202011176098.8A 2020-10-29 2020-10-29 Motor vehicle illegal parking detection method and system based on high visual angle under complex environment Active CN112289037B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011176098.8A CN112289037B (en) 2020-10-29 2020-10-29 Motor vehicle illegal parking detection method and system based on high visual angle under complex environment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011176098.8A CN112289037B (en) 2020-10-29 2020-10-29 Motor vehicle illegal parking detection method and system based on high visual angle under complex environment

Publications (2)

Publication Number Publication Date
CN112289037A true CN112289037A (en) 2021-01-29
CN112289037B CN112289037B (en) 2022-06-07

Family

ID=74373296

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011176098.8A Active CN112289037B (en) 2020-10-29 2020-10-29 Motor vehicle illegal parking detection method and system based on high visual angle under complex environment

Country Status (1)

Country Link
CN (1) CN112289037B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112950942A (en) * 2021-02-19 2021-06-11 合肥海赛信息科技有限公司 Intelligent detection method for motor vehicle illegal parking based on video analysis
CN115019547A (en) * 2022-05-31 2022-09-06 深圳云游四海信息科技有限公司 Nonstandard in-road parking detection method and system
CN116152719A (en) * 2023-03-15 2023-05-23 南通大学 Parking management method based on high-order video

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107609491A (en) * 2017-08-23 2018-01-19 中国科学院声学研究所 A kind of vehicle peccancy parking detection method based on convolutional neural networks
CN109615868A (en) * 2018-12-20 2019-04-12 北京以萨技术股份有限公司 A kind of video frequency vehicle based on deep learning is separated to stop detection method
CN110298307A (en) * 2019-06-27 2019-10-01 浙江工业大学 A kind of exception parking real-time detection method based on deep learning
CN110782673A (en) * 2019-10-26 2020-02-11 江苏看见云软件科技有限公司 Vehicle violation identification and detection system based on unmanned aerial vehicle shooting cloud computing
CN111429726A (en) * 2020-04-13 2020-07-17 城云科技（中国）有限公司 Monitoring video illegal parking vehicle detection and management method and corresponding system
CN111539359A (en) * 2020-04-28 2020-08-14 浙江工商大学 Illegal parking detection method based on deep learning
CN111597902A (en) * 2020-04-16 2020-08-28 浙江工业大学 Motor vehicle illegal parking monitoring method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107609491A (en) * 2017-08-23 2018-01-19 中国科学院声学研究所 A kind of vehicle peccancy parking detection method based on convolutional neural networks
CN109615868A (en) * 2018-12-20 2019-04-12 北京以萨技术股份有限公司 A kind of video frequency vehicle based on deep learning is separated to stop detection method
CN110298307A (en) * 2019-06-27 2019-10-01 浙江工业大学 A kind of exception parking real-time detection method based on deep learning
CN110782673A (en) * 2019-10-26 2020-02-11 江苏看见云软件科技有限公司 Vehicle violation identification and detection system based on unmanned aerial vehicle shooting cloud computing
CN111429726A (en) * 2020-04-13 2020-07-17 城云科技（中国）有限公司 Monitoring video illegal parking vehicle detection and management method and corresponding system
CN111597902A (en) * 2020-04-16 2020-08-28 浙江工业大学 Motor vehicle illegal parking monitoring method
CN111539359A (en) * 2020-04-28 2020-08-14 浙江工商大学 Illegal parking detection method based on deep learning

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112950942A (en) * 2021-02-19 2021-06-11 合肥海赛信息科技有限公司 Intelligent detection method for motor vehicle illegal parking based on video analysis
CN115019547A (en) * 2022-05-31 2022-09-06 深圳云游四海信息科技有限公司 Nonstandard in-road parking detection method and system
CN116152719A (en) * 2023-03-15 2023-05-23 南通大学 Parking management method based on high-order video

Also Published As

Publication number Publication date
CN112289037B (en) 2022-06-07

Similar Documents

Publication Publication Date Title
CN111368687B (en) Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation
CN112289037B (en) Motor vehicle illegal parking detection method and system based on high visual angle under complex environment
CN103824452B (en) A kind of peccancy parking detector based on panoramic vision of lightweight
CN110717387B (en) Real-time vehicle detection method based on unmanned aerial vehicle platform
CN112712057B (en) Traffic signal identification method and device, electronic equipment and storage medium
CN111931683B (en) Image recognition method, device and computer readable storage medium
CN102902983B (en) A kind of taxi identification method based on support vector machine
JP6700373B2 (en) Apparatus and method for learning object image packaging for artificial intelligence of video animation
CN114418895A (en) Driving assistance method and device, vehicle-mounted device and storage medium
CN113160575A (en) Traffic violation detection method and system for non-motor vehicles and drivers
CN112990065A (en) Optimized YOLOv5 model-based vehicle classification detection method
CN112084890A (en) Multi-scale traffic signal sign identification method based on GMM and CQFL
Ketcham et al. Recognizing the Illegal Parking Patterns of Cars on the Road in Front of the Bus Stop Using the Support Vector Machine
CN114049572A (en) Detection method for identifying small target
CN114048536A (en) Road structure prediction and target detection method based on multitask neural network
CN114092902A (en) Method and device for detecting violation behaviors of electric bicycle
CN117115690A (en) Unmanned aerial vehicle traffic target detection method and system based on deep learning and shallow feature enhancement
CN116630702A (en) Pavement adhesion coefficient prediction method based on semantic segmentation network
CN113963350A (en) Vehicle identification detection method, system, computer equipment, storage medium and terminal
CN113147781A (en) Intelligent driving information display system for automobile
Sisias et al. Preliminary results of a proposed CNN framework for use in motorway applicable detection systems
Roubil et al. A unified framework on reading variable size traffic sign boards using deep neural network
CN116541715B (en) Target detection method, training method of model, target detection system and device
Chen et al. Automatic recognition of driving events based on dashcam videos
Agrawal et al. Automated Car Parking System Using Deep Convolutional Neural Networks

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant