CN111091061B - Vehicle scratch detection method based on video analysis - Google Patents

Vehicle scratch detection method based on video analysis Download PDF

Info

Publication number
CN111091061B
CN111091061B CN201911144384.3A CN201911144384A CN111091061B CN 111091061 B CN111091061 B CN 111091061B CN 201911144384 A CN201911144384 A CN 201911144384A CN 111091061 B CN111091061 B CN 111091061B
Authority
CN
China
Prior art keywords
vehicle
vehicles
passing
area
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911144384.3A
Other languages
Chinese (zh)
Other versions
CN111091061A (en
Inventor
高飞
葛逸凡
卢书芳
陆佳炜
程振波
肖刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University of Technology ZJUT
Original Assignee
Zhejiang University of Technology ZJUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University of Technology ZJUT filed Critical Zhejiang University of Technology ZJUT
Priority to CN201911144384.3A priority Critical patent/CN111091061B/en
Publication of CN111091061A publication Critical patent/CN111091061A/en
Application granted granted Critical
Publication of CN111091061B publication Critical patent/CN111091061B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles

Abstract

The invention provides a vehicle scratch detection method based on video analysis, in particular to a vehicle scratch detection method based on a video semantic segmentation technology. The invention detects the vehicle in the video through video analysis by taking a semantic segmentation network and a tracking algorithm as a basis, tracks and judges the vehicle, and is used for solving the problems of low detection efficiency and high false detection rate in the prior art.

Description

Vehicle scratch detection method based on video analysis
Technical Field
The invention relates to the technical field of video semantic segmentation, in particular to a vehicle scratch detection method based on video analysis through a video semantic segmentation technology.
Background
With the development of the times and the improvement of technologies, automobiles have become a necessary travel tool for each family. However, with the increase of the number of vehicles, the number of parking spaces on the roadside is increased, the width of the road is narrowed, and hidden troubles are brought to the fact that the parked vehicles are scratched by the passing vehicles. The information of scratching the vehicle is found out by manually checking the monitoring video, so that time and labor are wasted, and the cost is too high. Therefore, a method for vehicle tracking, vehicle scratch detection and vehicle scratch information preservation based on video analysis is indispensable.
Currently, in the field of detecting vehicle scratches, the following methods are commonly used for detection: if a convolutional neural network for target detection is used for detecting the vehicle under video and judging vehicle scratch, the method is high in detection efficiency and improves the flexibility of detection operation, but the position of the vehicle is only framed by a rectangular selection frame, so that the contour information of the vehicle cannot be reflected, and the false detection rate of judgment of vehicle scratch is further high. Meanwhile, the convolutional neural network used in the prior art is lagged behind, the efficiency is low, and the convolutional neural network is not suitable for training and using a large number of pictures.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention aims to provide a vehicle scratch detection method based on video analysis. The method takes a semantic segmentation network and a tracking algorithm as a basis, detects the vehicles in the video through video analysis, tracks and judges the vehicles, and is used for solving the problems of low detection efficiency and high false detection rate in the prior art.
The vehicle scratch detection method based on video analysis is characterized by comprising the following steps of:
step 1: a rectangular frame is used for marking a parking area and a vehicle passing area in a monitoring video;
step 2: defining a set of parked vehicles S ═ SiI | ═ 1,2, …, I }, and the passing vehicle set G ═ G { (G)j1,2, …, J, where siThe minimum bounding rectangle of the ith vehicle in S, I represents the number of vehicles in the parking vehicle set, gjRepresenting the minimum bounding rectangle of the jth vehicle in G, wherein J represents the number of vehicles in the vehicle set; initially, both S and G are empty sets;
and step 3: obtaining a frame of image from a monitoring video buffer queue;
and 4, step 4: and (3) adopting a semantic segmentation method based on a deep learning technology to detect vehicles in the passing area from the image in the step (3), and recording the result as C ═ Ck1, 2., K }, where c iskA minimum bounding rectangle representing the K-th detected vehicle, K representing the number of detected vehicles; for each ckThe logic is as follows: if c iskIs completely contained in the parking area and is connected with all siIf formula (1) is satisfied, the reaction mixture willckAdding into S; otherwise, if ckNot completely contained in the parking area and corresponding to all gjIf formula (2) is satisfied, c iskAdding the mixture into G;
Figure GDA0003387976550000021
Figure GDA0003387976550000022
wherein A represents an area calculation function;
and 5: tracking and updating each vehicle in the parking vehicle set S in the step 4, and if the vehicle exceeds the range of the parking area, moving the vehicle out of the S;
step 6: tracking and updating G for each vehicle in the vehicle passing set G in the step 4, and if the vehicle exceeds the range of the vehicle passing area, moving the vehicle out of G;
and 7: for any vehicle S in SiAnd any vehicle G in GjMake the judgement of scraping, concrete step is:
step 7.1: if siAnd gjIf the two do not intersect, the result is judged as siAnd gjNo scratch; if siAnd gjIf the two are crossed, the next step of judgment is carried out in the step 7.2;
step 7.2: definition siIs SPi＝{spil|l＝1,2,…,LiIn which, spilDenotes siL is the first boundary point, LiDenotes siThe number of boundary points of (a); definition of gjIs GPj＝{gpjt|t＝1,2,…,MjIn which gp isjtDenotes gjT-th boundary point of (1), MjDenotes gjThe number of boundary points of (a); if SPiAnd GPjIf there is no identical element in the sequence, s is determinediAnd gjNo scratch; if SPiAnd GPjIf the same element exists in the above-mentioned formula, the formula is judged as siAnd gjScratching and recording siAnd gjThe distance between the respective center points, denoted dijSaving the frame image;
step 7.3: repeating steps 7.1-7.2 to define s in the last frameiAnd gjThe distance between the respective center points is
Figure GDA0003387976550000031
In a new frame, d is comparedijAnd
Figure GDA0003387976550000032
size of (1), if
Figure GDA0003387976550000033
Then d will beijIs updated to
Figure GDA0003387976550000034
And saving the frame image; if it is
Figure GDA0003387976550000035
D is not updatedijThe value of (d) and the saved image.
Compared with the prior art, the method has the following beneficial effects:
the vehicle detection method takes the semantic segmentation technology as the basis of vehicle detection, can describe the vehicle contour information instead of a simple rectangular selection frame, and can improve the accuracy of judging whether to scratch or not; when the information of the scratched vehicle is stored, the optimal image is stored in an iteration mode, and a better visualization effect is guaranteed.
Drawings
Fig. 1 is a calibration image of positioning information of a parking detection area and a passing detection area in a surveillance video.
In the figure: 1-a parking detection area and 2-a vehicle passing detection area.
Detailed Description
The following describes in detail a specific implementation method of the vehicle scratch detection method by video semantic segmentation technology according to the present invention with reference to an embodiment.
The invention discloses a vehicle scratch detection method based on video analysis, namely a vehicle scratch detection method based on a video semantic segmentation technology, which specifically comprises the following steps:
step 1: a rectangular frame is used for calibrating a parking detection area 1 and a vehicle passing detection area 2 in a monitoring video, and the calibration condition is shown in figure 1;
step 2: defining a set of parked vehicles S ═ SiI | ═ 1,2, …, I }, and the passing vehicle set G ═ G { (G)j1,2, …, J, where siThe minimum bounding rectangle of the ith vehicle in the S is represented, I represents the number of elements in the parking vehicle set, and gjRepresenting the minimum circumscribed rectangle of the jth vehicle in G, wherein J represents the number of elements in the vehicle set; initially, both S and G are empty sets;
and step 3: obtaining a frame of image from a monitoring video buffer queue;
and 4, step 4: and (3) adopting a semantic segmentation method based on a deep learning technology to detect vehicles in the passing area from the image in the step (3), and recording the result as C ═ Ck1, 2., K }, where c iskA minimum bounding rectangle representing the K-th detected vehicle, K representing the number of detected vehicles; for each ckThe logic is as follows: if c iskIs completely contained in the parking area and is connected with all siIf formula (1) is satisfied, c iskAdding into S; otherwise, if ckNot completely contained in the parking area and corresponding to all gjIf formula (2) is satisfied, c iskAdding the mixture into G;
Figure GDA0003387976550000041
Figure GDA0003387976550000051
wherein A represents an area calculation function;
the vehicle detection method based on semantic segmentation specifically comprises the following steps:
step 4.1: acquiring a large number of pictures under a monitoring view angle, labeling the outline information of the pictures in the acquired pictures by vehicles to obtain a vehicle semantic segmentation data set, and calculating the following steps according to the following steps of 3: 1 dividing the data set into a training set and a test set;
step 4.2: inputting the divided training set into a first convolutional neural network for feature extraction, and inputting the extracted features into a second convolutional neural network to realize parameter training of semantic segmentation; inputting the divided test set into a first convolutional neural network and a second convolutional neural network, and comparing the detection effect of the parameters obtained through training with the test set picture calibration information to realize parameter optimization;
step 4.3: as a preferred scheme of the present invention, 10 layers of convolutional neural networks are selected as a first convolutional neural network for feature extraction, and sequentially include a first convolutional layer, a second bottleneck layer, a third bottleneck layer, a fourth bottleneck layer, a fifth bottleneck layer, a sixth bottleneck layer, a seventh bottleneck layer, an eighth bottleneck layer, a ninth convolutional layer, and a tenth pooling layer; the picture input dimension of the first convolutional neural network is 224 × 224 × 3, the picture output dimension of the first convolutional neural network is 1 × 1 × 1280, the picture output dimension of the first convolutional layer is 112 × 112 × 32, the picture output dimension of the second bottleneck layer is 112 × 112 × 16, the picture output dimension of the third bottleneck layer is 56 × 56 × 24, the picture output dimension of the fourth bottleneck layer is 28 × 28 × 32, the picture output dimension of the fifth bottleneck layer is 14 × 14 × 64, the picture output dimension of the sixth bottleneck layer is 14 × 14 × 96, the picture output dimension of the seventh bottleneck layer is 7 × 7 × 160, the picture output dimension of the eighth bottleneck layer is 7 × 7 × 320, the picture output dimension of the ninth convolutional layer is 7 × 1280, and the picture output dimension of the tenth pooling layer is 1 × 1 × 1280; the bottleneck layer is different from other nonlinear layers and contains all necessary information, and the condition of information loss can not occur when the ReLU and other activation functions are used;
step 4.4: as a preferred scheme of the invention, the second convolutional neural network adopts a symmetrical structure of a coder and a decoder to realize end-to-end pixel-level image segmentation; the second convolutional neural network takes the first convolutional neural network as a pre-trained network, takes three convolutional layers with steps as spatial paths, and then fuses the output characteristics of the two components through a characteristic fusion module to make final prediction; at the level of the representation of the features, the two networks are not characterized identically: the first convolutional neural network is used as the feature output by the pre-trained network to mainly encode the contextual information, the spatial information captured by the spatial path encodes most detailed information, and the two kinds of feature information cannot be simply weighted; in order to solve the problem, under the condition that different levels of characteristics are given, the second convolution neural network characteristic fusion module firstly connects the first convolution neural network as the characteristics output by the pre-training network and the characteristics output by the space path, then, through the scale of batch normalization balance characteristics, the connected characteristics are pooled into a characteristic vector in the next step, and a weight vector is calculated, wherein the weight vector can re-weight the characteristics to play the roles of characteristic selection and combination; the second convolutional neural network supervises the output of the whole second convolutional neural network through the main loss function by training of the auxiliary loss function supervision model, and supervises the output of the first convolutional neural network through adding two special auxiliary loss functions, just like multi-layer supervision, by using formula (3) to balance the weight of the main loss function and the auxiliary loss function, wherein the main loss function and the auxiliary loss function both use formula (4):
Figure GDA0003387976550000061
wherein X, W are the weights of the main loss function and the auxiliary loss function, respectively, K represents the number of total samples, lpAs an output function of the entire second convolutional neural network,/iSupervision functions, X, being auxiliary loss functionsiWeight of the principal loss function for the ith sample, WiAnd alpha is a weight value for balancing the main loss function and the auxiliary loss function.
Figure GDA0003387976550000071
Wherein N represents the number of the input sample, y represents the expected output of the second convolutional neural network, a represents the actual output of the second convolutional neural network, ln represents the logarithm operation, and when loss is less than 0.001, the training of the input image is completed.
Step 4.5: inputting the training set and the test set into a convolutional neural network to complete parameter training of vehicle detection;
and 5: tracking and updating each vehicle in the parking vehicle set S in the step 4, and if the vehicle exceeds the range of the parking area, moving the vehicle out of the S;
step 6: tracking and updating G for each vehicle in the vehicle passing set G in the step 4, and if the vehicle exceeds the range of the vehicle passing area, moving the vehicle out of G;
and 7: for any vehicle S in SiAnd any vehicle G in GjMake the judgement of scraping, concrete step is:
step 7.1: if siAnd gjIf the two do not intersect, the result is judged as siAnd gjNo scratch; if siAnd gjIf the two are crossed, the next step of judgment is carried out in the step 7.2;
step 7.2: definition siIs SPi＝{spil|l＝1,2,…,LiIn which, spilDenotes siL is the first boundary point, LiDenotes siThe number of boundary points of (a); definition of gjIs GPj＝{gpjt|t＝1,2,…,MjIn which gp isjtDenotes gjT-th boundary point of (1), MjDenotes gjThe number of boundary points of (a); if SPiAnd GPjIf there is no identical element in the sequence, s is determinediAnd gjNo scratch; if SPiAnd GPjIf the same element exists in the above-mentioned formula, the formula is judged as siAnd gjScratching and recording siAnd gjThe distance between the respective center points, denoted dijSaving the frame image;
step 7.3: repeating steps 7.1-7.2 to define s in the last frameiAnd gjThe distance between the respective center points is
Figure GDA0003387976550000081
In a new frame, d is comparedijAnd
Figure GDA0003387976550000082
size of (1), if
Figure GDA0003387976550000083
Then d will beijIs updated to
Figure GDA0003387976550000084
And saving the frame image; if it is
Figure GDA0003387976550000085
D is not updatedijThe value of (d) and the saved image.
The matters set forth in the examples are illustrative only of the principles of the invention and its efficacy and are not intended to be limiting thereof. Any person skilled in the relevant art can modify or change the above-described embodiments without departing from the spirit and scope of the invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.

Claims (1)

1. A vehicle scratch detection method based on video analysis is characterized by comprising the following steps:
step 1: a rectangular frame is used for marking a parking area and a vehicle passing area in a monitoring video;
step 2: defining a set of parked vehicles S ═ SiI | ═ 1,2, …, I }, and the passing vehicle set G ═ G { (G)j1,2, …, J, where siThe minimum bounding rectangle of the ith vehicle in S, I represents the number of vehicles in the parking vehicle set, gjThe minimum bounding rectangle for the jth vehicle in G,j represents the number of vehicles in the passing vehicle set; initially, both S and G are empty sets;
and step 3: obtaining a frame of image from a monitoring video buffer queue;
and 4, step 4: and (3) adopting a semantic segmentation method based on a deep learning technology to detect vehicles in the passing area from the image in the step (3), and recording the result as C ═ Ck1, 2., K }, where c iskA minimum bounding rectangle representing the K-th detected vehicle, K representing the number of detected vehicles; for each ckThe logic is as follows: if c iskIs completely contained in the parking area and is connected with all siIf formula (1) is satisfied, c iskAdding into S; otherwise, if ckNot completely contained in the parking area and corresponding to all gjIf formula (2) is satisfied, c iskAdding the mixture into G;
Figure FDA0003387976540000011
Figure FDA0003387976540000012
wherein A represents an area calculation function;
and 5: tracking and updating each vehicle in the parking vehicle set S in the step 4, and if the vehicle exceeds the range of the parking area, moving the vehicle out of the S;
step 6: tracking and updating G for each vehicle in the vehicle passing set G in the step 4, and if the vehicle exceeds the range of the vehicle passing area, moving the vehicle out of G;
and 7: for any vehicle S in SiAnd any vehicle G in GjMake the judgement of scraping, concrete step is:
step 7.1: if siAnd gjIf the two do not intersect, the result is judged as siAnd gjNo scratch; if siAnd gjIf the two are crossed, the next step of judgment is carried out in the step 7.2;
step 7.2: definition siIs SPi＝{spil|l＝1,2,…,LiIn which, spilDenotes siL is the first boundary point, LiDenotes siThe number of boundary points of (a); definition of gjIs GPj＝{gpjt|t＝1,2,…,MjIn which gp isjtDenotes gjT-th boundary point of (1), MjDenotes gjThe number of boundary points of (a); if SPiAnd GPjIf there is no identical element in the sequence, s is determinediAnd gjNo scratch; if SPiAnd GPjIf the same element exists in the above-mentioned formula, the formula is judged as siAnd gjScratching and recording siAnd gjThe distance between the respective center points, denoted dijSaving the frame image;
step 7.3: repeating steps 7.1-7.2 to define s in the last frameiAnd gjThe distance between the respective center points is
Figure FDA0003387976540000021
In a new frame, d is comparedijAnd
Figure FDA0003387976540000022
size of (1), if
Figure FDA0003387976540000023
Then d will beijIs updated to
Figure FDA0003387976540000024
And saving the frame image; if it is
Figure FDA0003387976540000025
D is not updatedijThe value of (d) and the saved image.
CN201911144384.3A 2019-11-20 2019-11-20 Vehicle scratch detection method based on video analysis Active CN111091061B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911144384.3A CN111091061B (en) 2019-11-20 2019-11-20 Vehicle scratch detection method based on video analysis

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911144384.3A CN111091061B (en) 2019-11-20 2019-11-20 Vehicle scratch detection method based on video analysis

Publications (2)

Publication Number Publication Date
CN111091061A CN111091061A (en) 2020-05-01
CN111091061B true CN111091061B (en) 2022-02-15

Family

ID=70394035

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911144384.3A Active CN111091061B (en) 2019-11-20 2019-11-20 Vehicle scratch detection method based on video analysis

Country Status (1)

Country Link
CN (1) CN111091061B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111653103A (en) * 2020-05-07 2020-09-11 浙江大华技术股份有限公司 Target object identification method and device

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103226891B (en) * 2013-03-26 2015-05-06 中山大学 Video-based vehicle collision accident detection method and system
CN107657237B (en) * 2017-09-28 2020-03-31 东南大学 Automobile collision detection method and system based on deep learning
CN107945566A (en) * 2017-11-17 2018-04-20 张慧 Curb parking management system and method based on multiple target tracking and deep learning
CN108898054B (en) * 2018-05-24 2020-08-07 合肥工业大学 Safety belt detection method based on semantic segmentation
CN109190523B (en) * 2018-08-17 2021-06-04 武汉大学 Vehicle detection tracking early warning method based on vision
CN109829403B (en) * 2019-01-22 2020-10-16 淮阴工学院 Vehicle anti-collision early warning method and system based on deep learning

Also Published As

Publication number Publication date
CN111091061A (en) 2020-05-01

Similar Documents

Publication Publication Date Title
CN111091105B (en) Remote sensing image target detection method based on new frame regression loss function
CN106228125B (en) Method for detecting lane lines based on integrated study cascade classifier
CN111598030B (en) Method and system for detecting and segmenting vehicle in aerial image
CN112750150B (en) Vehicle flow statistical method based on vehicle detection and multi-target tracking
CN113780296B (en) Remote sensing image semantic segmentation method and system based on multi-scale information fusion
CN109858389B (en) Vertical ladder people counting method and system based on deep learning
CN110189304B (en) Optical remote sensing image target on-line rapid detection method based on artificial intelligence
CN112418212B (en) YOLOv3 algorithm based on EIoU improvement
CN111126359A (en) High-definition image small target detection method based on self-encoder and YOLO algorithm
CN108428248B (en) Vehicle window positioning method, system, equipment and storage medium
CN112633149B (en) Domain-adaptive foggy-day image target detection method and device
CN113052159A (en) Image identification method, device, equipment and computer storage medium
CN111738114A (en) Vehicle target detection method based on anchor-free accurate sampling remote sensing image
CN111611925A (en) Building detection and identification method and device
CN110909656B (en) Pedestrian detection method and system integrating radar and camera
CN117079132A (en) Remote sensing image target detection method based on Gaussian distance loss
CN115424237A (en) Forward vehicle identification and distance detection method based on deep learning
CN112101114A (en) Video target detection method, device, equipment and storage medium
CN111833353A (en) Hyperspectral target detection method based on image segmentation
CN115527096A (en) Small target detection method based on improved YOLOv5
CN111091061B (en) Vehicle scratch detection method based on video analysis
CN116861262B (en) Perception model training method and device, electronic equipment and storage medium
CN112085001A (en) Tunnel recognition model and method based on multi-scale edge feature detection
CN111666988A (en) Target detection algorithm based on multi-layer information fusion
CN116363610A (en) Improved YOLOv 5-based aerial vehicle rotating target detection method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant