CN117237883A - Traffic illegal behavior detection method and system based on visual target tracking - Google Patents
Traffic illegal behavior detection method and system based on visual target tracking Download PDFInfo
- Publication number
- CN117237883A CN117237883A CN202311198251.0A CN202311198251A CN117237883A CN 117237883 A CN117237883 A CN 117237883A CN 202311198251 A CN202311198251 A CN 202311198251A CN 117237883 A CN117237883 A CN 117237883A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- illegal
- target
- tracking
- detection frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 242
- 230000000007 visual effect Effects 0.000 title claims abstract description 23
- 238000000034 method Methods 0.000 claims abstract description 55
- 238000001914 filtration Methods 0.000 claims abstract description 46
- 239000011159 matrix material Substances 0.000 claims description 37
- 239000013598 vector Substances 0.000 claims description 10
- 230000004044 response Effects 0.000 claims description 7
- 238000003708 edge detection Methods 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 4
- 238000009499 grossing Methods 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 16
- 230000002688 persistence Effects 0.000 abstract description 2
- 230000006399 behavior Effects 0.000 description 36
- 238000004590 computer program Methods 0.000 description 11
- 238000012545 processing Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 238000007726 management method Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 238000003860 storage Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000006698 induction Effects 0.000 description 3
- 230000033228 biological regulation Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000012550 audit Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a traffic offence detection method and system based on visual target tracking. The method comprises the steps of detecting traffic illegal vehicles by using a visual detection algorithm, tracking the illegal vehicles by using a Kalman filtering method under the condition that a license plate photo of the illegal vehicles cannot be clearly shot, judging the tracking state, and fitting the positions of the vehicles to continuously track the vehicles when a target part is shielded; when the target tracking is lost, re-tracking operation is further carried out, the persistence and the robustness of the target tracking are improved until the license plate of the traffic illegal vehicle can be clearly shot, the video of the traffic illegal vehicle is recorded in the whole process, and meanwhile, a plurality of pictures such as the illegal state, the illegal state termination, the clear license plate and the like are shot, so that the reliable detection of the traffic illegal state is realized.
Description
Technical Field
The invention relates to the technical field of image recognition processing, in particular to a traffic illegal behavior detection method and system based on visual target tracking.
Background
The traffic violation detection system is based on a video sequence acquired by a camera, judges whether the vehicle has illegal behaviors or not through technologies such as image processing, computer vision and the like, and submits the illegal behaviors to a traffic management department for checking. It has a crucial role in traffic regulation execution and maintenance of traffic order.
At present, traffic violation detection systems have been widely used at home and abroad, and can be mainly divided into three types: the induction type illegal detection system is characterized in that coils are arranged on the ground in advance, and when a vehicle runs through, induction current signals are generated, a camera is controlled to take a candid photograph, and the induction type illegal detection system is mainly used for detecting the traffic behavior violating traffic signal lamps at intersections; the visual illegal detection directly utilizes an algorithm to identify illegal behaviors in the image, and the illegal behaviors are captured in time; the radar-assisted detection system is mainly used for measuring the speed and photographing. The method needs to acquire the images of the illegal vehicles as punishment basis and needs to ensure that the license plate numbers of the vehicles are clearly visible, however, the images meeting the requirements cannot be acquired during illegal behaviors, such as shielding of the illegal vehicle license plates by other vehicles, insufficient distance between the vehicles and the cameras, clear shooting, and the like. In addition, under the background that navigation software is continuously perfected, the positions of a plurality of illegal monitoring cameras are well known, and a few drivers stop illegal behaviors by utilizing the limitation of the imaging definition of the monitoring cameras on the license plates of vehicles, and recover the illegal states after leaving the monitoring range so as to avoid punishment and get privates from the illegal behaviors. This phenomenon is most serious in taking emergency lanes and bus lanes. Such behavior brings great challenges to the authority of traffic regulations and also serves as hidden danger for trip safety.
Disclosure of Invention
The invention aims to provide a traffic illegal action detection method and system based on visual target tracking, which can realize the reliability detection of traffic illegal actions.
In order to achieve the above object, the present invention provides the following solutions:
a traffic offence detection method based on visual target tracking, the method comprising:
determining an illegal detection area, capturing an illegal vehicle in the illegal detection area based on a target detection algorithm, acquiring a clear vehicle license plate of the illegal vehicle, and outputting an illegal vehicle illegal behavior image and vehicle license plate information;
when the clear license plate of the illegal vehicle cannot be obtained, extracting Harris corner points based on the captured illegal vehicle image to serve as characteristic points of the illegal vehicle, and carrying out target tracking on the illegal vehicle by utilizing Kalman filtering;
when a preset detection frame is carried out, matching a predicted target frame and feature points of the illegal vehicle is carried out on the illegal vehicle, and the tracking state of the illegal vehicle is judged according to a matching result;
when the tracking state is that target tracking is normal, acquiring clear license plates in a vehicle target detection frame of a current detection frame, and outputting the illegal vehicle illegal behavior image and the license plate information; if the clear license plate of the illegal vehicle cannot be obtained, returning to the step of tracking the target of the illegal vehicle by using Kalman filtering;
When the tracking state is that the target part is blocked, determining the fitting position of the illegal vehicle of the current detection frame according to the characteristic points of the illegal vehicle of the current detection frame and the last detection frame, and returning to the step of 'performing target tracking on the illegal vehicle by using Kalman filtering';
when the tracking state is that the target tracking is lost, in the next detection frame, re-tracking the illegal vehicle is carried out based on the position and the size of the vehicle target detection frame of the last frame before the loss;
and returning to the step of 'tracking the target of the illegal vehicle by using Kalman filtering' after the re-tracking is successful.
Optionally, when detecting the rule-breaking behavior, determining the rule-breaking detection area specifically includes:
image clipping is carried out on the basis of the image shot by the illegal detection system to obtain a pavement area image;
performing color space conversion on the pavement area image, and performing edge detection on the converted image by using a Canny operator;
extracting straight line segments by Hough transformation according to an edge detection result, determining lane boundaries according to the extracted straight line segments, and obtaining a plurality of lane areas;
optionally, one of the lane areas is the violation detection area.
Optionally, extracting Harris corner points as feature points of the illegal vehicle based on the captured illegal vehicle image specifically includes:
carrying out gray scale treatment on the illegal vehicle image, and respectively calculating the gradient in the x direction and the gradient in the y direction in the vehicle target detection frame through a difference operator;
constructing a first matrix based on the gradient in the x-direction and the gradient in the y-direction;
performing Gaussian smoothing filtering on elements in the first matrix;
selecting the size of a window and the corresponding weight, and calculating a second matrix according to the size of the window, the weight and the first matrix;
calculating corresponding Harris response values at each pixel point in the illegal vehicle image according to the second matrix;
and screening pixel points with the Harris response value larger than a threshold value as the Harris corner points to obtain the illegal vehicle feature points.
Optionally, matching the predicted target frame with the feature points of the illegal vehicle specifically includes:
in the current detection frame, carrying out vehicle detection in the road range by utilizing the target detection algorithm to obtain a plurality of vehicle detection frames corresponding to the current detection frame;
the vehicle target frame offset speed vector obtained by the Kalman filtering is acted on a vehicle target detection frame extracted from the previous detection frame and the illegal vehicle characteristic points to obtain a predicted target frame of the current detection frame and predicted positions of the illegal vehicle characteristic points;
Calculating the intersection ratio of the predicted target frame and each vehicle detection frame;
judging whether the cross-over ratio is larger than a cross-over ratio threshold value or not to obtain a first judging result;
if the first judgment result is yes, a vehicle target detection frame corresponding to the illegal vehicle is obtained, and feature points of the illegal vehicle in the vehicle target detection frame are extracted;
performing nearest neighbor association matching on the illegal vehicle characteristic points in the vehicle target detection frame and the illegal vehicle characteristic points in the prediction target frame to obtain characteristic point matching coefficients;
if the first judgment result is negative, expanding the current prediction target frame with a preset expansion coefficient, and extracting the illegal vehicle characteristic points in the expanded prediction frame;
performing nearest neighbor association matching on the illegal vehicle feature points in the predicted target frame before and after expansion to obtain the feature point matching coefficients;
when the characteristic point matching coefficient is larger than the coefficient threshold value, matching is successful;
and when the characteristic point matching coefficient is smaller than or equal to a coefficient threshold value, matching fails.
Optionally, performing nearest neighbor association matching on the feature points of the illegal vehicle in the vehicle target detection frame and the feature points of the illegal vehicle in the prediction target frame to obtain feature point matching coefficients, which specifically includes:
Calculating the Euclidean distance between each illegal vehicle characteristic point in the vehicle target detection frame and the prediction target frame, and constructing an association cost matrix;
comparing matrix elements in the association cost matrix with a preset association threshold;
regarding two feature points corresponding to the matrix elements exceeding the preset association threshold as nearest neighbor association matching point pairs;
and calculating the characteristic point matching coefficient according to the number of nearest neighbor associated matching point pairs, the number of illegal vehicle characteristic points of the previous detection frame and the number of illegal vehicle characteristic points of the current detection frame.
Optionally, determining the fitting position of the illegal vehicle of the current detection frame according to the characteristic points of the illegal vehicle of the current detection frame and the previous detection frame specifically includes:
calculating the relative positions of the matching feature points of the previous detection frame and the center point of the target frame, the positions of the matching feature points of the current detection frame in the illegal vehicle image coordinates, and constructing an equation set by combining the illegal vehicle center point coordinates of the current detection frame; the equation set comprises n equations; n is the number of the matched feature points; the matching feature points are nearest-neighbor association matching point pairs obtained after nearest-neighbor association matching;
And solving the equation set by using a least square method to obtain the fitting position of the illegal vehicle of the current detection frame.
Optionally, judging the tracking state of the illegal vehicle according to the matching result specifically includes:
when the first judgment result is yes and the second judgment result is yes, the tracking state is that the target tracking is normal;
when the first judgment result is NO and the second judgment result is yes, the tracking state is the target part shielding;
and when the first judgment result is NO and the second judgment result is NO, the tracking state is the target tracking loss.
Optionally, the re-tracking of the illegal vehicle is performed based on the position and the size of the vehicle target detection frame of the last frame before the loss, which specifically includes:
acquiring the position and the size of a vehicle target detection frame of the last frame before the loss;
determining the movement direction of the illegal vehicle by using the velocity vector when the Kalman filtering is used for tracking the target;
determining a target potential occurrence domain according to the vehicle target detection frame of the last frame before the loss and the movement direction of the illegal vehicle;
calculating the relative positions of the feature points of the illegal vehicles in the vehicle target detection frame of the last frame before the loss and the center point of the vehicle target detection frame, and marking the relative positions as first relative positions;
Calculating the intersection ratio of each vehicle detection frame in the next detection frame and the target potential occurrence domain, and marking the intersection ratio as a heavy tracking intersection ratio;
when the vehicle target detection frame with the re-tracking intersection ratio being larger than the re-tracking intersection ratio threshold exists, extracting the illegal vehicle characteristic points in the vehicle target detection frame during re-tracking, calculating the relative coordinates of the illegal vehicle characteristic points extracted during re-tracking and the center point of the vehicle target detection frame during re-tracking, and recording the relative coordinates as a second relative position;
performing feature point matching according to the first relative position and the second relative position by using a nearest neighbor correlation matching method to obtain a feature point matching coefficient during re-tracking;
when the characteristic point matching coefficient in the re-tracking is larger than the coefficient threshold in the re-tracking, recapturing the illegal vehicle in the potential target occurrence domain;
and correcting the vehicle position by utilizing a vehicle target detection frame for capturing the illegal vehicle currently, and adding a re-tracking mark to the re-captured illegal vehicle.
Optionally, determining the target potential occurrence domain according to the vehicle target detection frame of the last frame before the loss and the movement direction of the illegal vehicle specifically includes:
Calculating the intersection point of the detection frame center speed reverse ray of the vehicle target detection frame of the last frame before the loss and the rear frame line of the detection frame, and taking a perpendicular line of the lane line through the intersection point;
judging the lane where the illegal vehicle is located according to the coordinates of the detection frame lines at two sides, and extracting lane boundary lines;
and regarding the area formed by the lane boundary line, the perpendicular line of the lane line and the visual field edge as the target potential occurrence domain.
The invention also provides a traffic offence detection system based on visual target tracking, which comprises:
the area determining and target detecting module is used for determining an illegal detecting area, capturing illegal vehicles in the illegal detecting area based on a target detecting algorithm, obtaining clear vehicle license plates of the illegal vehicles, and outputting illegal vehicle illegal behavior images and vehicle license plate information;
the feature extraction and target tracking module is used for extracting Harris corner points based on the captured illegal vehicle images to serve as illegal vehicle feature points when clear license plates of the illegal vehicles cannot be obtained, and carrying out target tracking on the illegal vehicles by utilizing Kalman filtering;
the matching and tracking state judging module is used for carrying out matching between a predicted target frame and the feature points of the illegal vehicle when the preset detection frame is carried out, and judging the tracking state of the illegal vehicle according to a matching result;
The vehicle license plate acquisition and target tracking module is used for acquiring clear vehicle license plates in a vehicle target detection frame of a current detection frame when the tracking state is that target tracking is normal, and outputting the illegal vehicle illegal behavior image and the vehicle license plate information; if the clear license plate of the illegal vehicle cannot be obtained, performing target tracking on the illegal vehicle by using Kalman filtering in a matching and tracking state judging module;
the position fitting and target tracking module is used for determining the fitting position of the illegal vehicle of the current detection frame according to the characteristic points of the illegal vehicle of the current detection frame and the last detection frame when the tracking state is partially blocked, and executing 'utilizing Kalman filtering to track the target of the illegal vehicle' in the matching and tracking state judging module;
the re-tracking module is used for carrying out re-tracking on the illegal vehicle in the next detection frame based on the position and the size of the vehicle target detection frame of the last frame before the loss when the tracking state is that the target tracking is lost; and after the re-tracking is successful, executing target tracking on the illegal vehicle by utilizing Kalman filtering in the matching and tracking state judging module.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the invention provides a traffic offence detection method and system based on visual target tracking. The method comprises the steps of detecting the traffic illegal vehicles by using a visual detection algorithm, continuously tracking the traffic illegal vehicles by adopting a Kalman filtering method under the condition that the license plate photos of the traffic illegal vehicles cannot be clearly shot, keeping tracking the traffic illegal vehicles when the traffic illegal vehicles stop the traffic illegal behaviors by changing lanes and the like, judging the tracking state, further carrying out re-tracking operation when the target tracking is lost, improving the sustainability and the robustness of the target tracking until the license plates of the traffic illegal vehicles can be clearly shot, recording the videos of the traffic illegal vehicles in the whole process, and simultaneously shooting a plurality of photos of the illegal vehicles, such as illegal termination, clear license plates and the like, so as to realize reliable detection of the traffic illegal behaviors.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a traffic offence detection method based on visual target tracking according to a first embodiment of the present invention;
FIG. 2 is a Kalman filtering tracking process according to a first embodiment of the present invention;
FIG. 3 is a tracking state determination flow provided in a first embodiment of the present invention;
FIG. 4 is a schematic illustration of a first embodiment of the present invention showing a vehicle target that is not accurately detected due to partial occlusion;
fig. 5 is a flowchart illustrating a re-tracking implementation procedure according to a first embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention aims to provide a traffic illegal behavior detection method and system based on visual target tracking. When a remote traffic illegal vehicle is detected by utilizing a vision detection algorithm, under the condition that a license plate photo of the illegal vehicle cannot be clearly shot, the traffic illegal vehicle is continuously tracked by adopting a Kalman filtering method, when the traffic illegal vehicle is close to a traffic illegal detection area where the license plate of the vehicle can be clearly shot, the traffic illegal behavior is stopped by lane change and other behaviors, the tracking of the illegal vehicle is also kept, and in order to cope with the situation that a target is partially shielded and tracking is lost during tracking, a nearest neighbor matching based on Harris angular points and a target re-tracking module adopting a potential occurrence domain are respectively designed, so that the persistence and the robustness of target tracking are improved until the license plate of the traffic illegal vehicle can be clearly shot, the video of the traffic illegal behavior vehicle is recorded in the whole process, and meanwhile, a plurality of photos such as the illegal vehicle license plate, the illegal termination, clear license plate and the like are shot, so that the reliable detection of the traffic illegal behavior is realized.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
Example 1
As shown in fig. 1, the present embodiment provides a traffic offence detection method based on visual target tracking, which includes:
s1: determining an illegal detection area, capturing an illegal vehicle in the illegal detection area based on a target detection algorithm, acquiring a clear vehicle license plate of the illegal vehicle, and outputting an illegal vehicle illegal behavior image and vehicle license plate information.
In step S1, when detecting the illegal behavior of the illegal occupation road, the Hough transformation is used to detect the lane line, and the illegal detection tracking start area (illegal detection area) is defined. The tracking initiation area is defined for calibration operations of the violation detection system after altering the camera position, representing a set of positions where the vehicle should not be present.
The invention is mainly set aiming at illegal behavior of illegal occupation, so that lane line detection is needed to be carried out after the camera position is changed each time, and an illegal area is redefined. The specific demarcation method in this case is as follows:
firstly, the image is cut out to obtain a pavement area based on the color characteristics, and Gaussian blur is carried out to reduce noise. And performing image clipping based on the image shot by the illegal detection system to obtain a pavement area image.
The image is color space converted. The lane line colors are mainly yellow and white, so that the original image RGB color space is converted into the YcbCr color space so as to distinguish the difference between the two lane lines and the road surface. The conversion method comprises the following steps:
converting the image into a gray image, and performing edge detection by using a Canny operator, wherein whether the pixel is an edge or not is judged according to the pixel gradient, and only the pixel larger than a threshold value is accepted.
And finally, extracting straight line segments by using Hough transformation, fitting a straight line with similar distance in Hough space into a straight line as a lane boundary, dividing the road into a plurality of lane areas, numbering from left to right, and selecting the corresponding number as a tracking initial area of the illegal detection system according to road setting.
For other traffic violations, the method for dividing the starting area of the violation is different, but the subsequent processing steps are basically consistent, for example, when the behavior of running a red light is monitored, a proper range in front of a stop line of the vehicle when the red light is monitored can be selected as a detection area.
In step S1, an illegal target is captured in a delimited area based on a target detection algorithm, and acquisition of a license plate of a vehicle is attempted.
And (3) detecting a vehicle target by using a target detection algorithm, such as yolov5, faster-RCNN and the like, calculating the proportion of the intersection of a detection frame and the tracking initial region obtained in the step (1) to the detection frame, screening illegal targets in the tracking initial region obtained in the step (1), photographing as an illegal behavior photo and recording a timestamp when the illegal vehicle target is detected, and identifying the vehicle license in a target bounding box of the illegal target, wherein if the illegal photo is successfully identified, the illegal target is directly used as evidence of the illegal photo.
S2: when the clear license plate of the illegal vehicle cannot be obtained, extracting Harris corner points based on the captured illegal vehicle image to serve as characteristic points of the illegal vehicle, and carrying out target tracking on the illegal vehicle by utilizing Kalman filtering.
If the license plate of the vehicle cannot be identified, the Harris corner point is extracted from the acquired target detection frame by using a plesey algorithm and used as a vehicle characteristic point, the basic idea is to use a fixed window to slide in any direction on an image, compare the gray level change degree of pixels in the window before and after sliding, and if the gray level change is larger when the window slides in any direction, the Harris corner point exists in the window. The pixel gray level variation of the sliding window is represented by the following formula:
where [ u, v ] is the offset of window W; (x, y) is the pixel coordinate position corresponding to window WW; i (x, y) is the image gray value of the pixel coordinate position (x, y); i (x+u, y+v) is the image gray value after the shift at the pixel coordinate position (x, y); w (x, y) is a window weight function.
The specific process for extracting Harris corner points is as follows:
a) Gray processing is carried out on the illegal vehicle image, and gradient I in the x direction in a vehicle target detection frame is calculated through a difference operator x And gradient I in y-direction y 。
I x ＝I×[-1,0,1]
I y ＝I×[-1,0,1] T
b) Gradient I based on x-direction x And gradient I in y-direction y A first matrix N is constructed.
Calculating the gradient product of image direction to form matrix
c) And carrying out Gaussian smoothing filtering on the elements in the first matrix N.
The four elements of the first matrix N are subjected to Gaussian smoothing filtering, and a discrete two-dimensional zero-mean Gaussian function is used as follows:
d) The size of the window W and the corresponding weight W (x, y) are selected and a second matrix M is calculated from the size of the window, the weight and the first matrix N.
e) And calculating corresponding Harris response values at each pixel point in the illegal vehicle image according to the second matrix.
Calculating corresponding Harris response values R=det (M) -k (trace (M)) at each pixel point (x, y) 2 。
f) And screening pixel points with the Harris response value larger than a threshold value as the Harris corner points to obtain the illegal vehicle feature points.
After the feature points are extracted, target tracking is executed:
for illegal targets of which the license plate numbers cannot be immediately detected, a Kalman filtering method is used for predicting the movement trend of the vehicle in the next frame, the targets are tracked, and the tracking process is shown in figure 2.
The Kalman filtering for predicting the target position can be divided into two steps of prediction updating and measurement updating. Updating the state vector and the covariance matrix of the error of the state vector by predicting, updating and receiving the observation input at the moment t; the measurement update obtains the prediction of the system state at the next moment according to the state equation and estimates the covariance matrix of the state vector error. The process comprises the following formulas:
Wherein x and z are state variables and measurement variables respectively, are four-dimensional vectors formed by target frame positions x, y and target frame speeds dx, dy, P is an error covariance matrix, A is a state matrix, B is a control matrix, H is a gain matrix, R represents measurement noise, Q represents observation noise, K k Representing the kalman gain coefficient.
S3: and when the detection frame is carried out, matching the predicted target frame and the feature points of the illegal vehicle is carried out on the illegal vehicle, and the tracking state of the illegal vehicle is judged according to the matching result.
In step S3, the matching of the prediction target frame and the feature points of the illegal vehicle is performed on the illegal vehicle, which specifically includes:
(1) And in the current detection frame, carrying out vehicle detection in the road range by utilizing the target detection algorithm to obtain a plurality of vehicle detection frames corresponding to the current detection frame.
In view of the high computational power requirements of the target detection algorithm, feature point extraction and matching steps, frame-by-frame detection can affect the sampling frequency of the system. The invention adopts Kalman filtering to predict the target position in real time, and sets detection frames at intervals of a certain frame number, and detects and matches illegal vehicles to correct the accumulated error of the Kalman filtering.
And when the detection frame is reached, carrying out vehicle detection in the road range by using a target detection algorithm to obtain a plurality of vehicle target frames.
(2) And acting the vehicle target frame offset speed vector obtained by Kalman filtering on the vehicle target detection frame extracted from the previous detection frame and the illegal vehicle characteristic point to obtain a predicted target frame of the current detection frame and a predicted position of the illegal vehicle characteristic point.
As shown in fig. 3, the tracking state judging process is based on the above prediction and detection results, and uses the nearest neighbor matching method to obtain Harris feature point matching coefficients, and completes matching of the target frame according to the intersection ratio of the prediction frame and the detection frame, and comprehensively judges the state of the target vehicle, wherein the state comprises three types of tracking normal, partial shielding and target loss. The specific process is as follows steps (3) to (9).
(3) And calculating the intersection ratio IOU of the predicted target frame and each vehicle detection frame. And judging whether the cross-over ratio IOU is larger than a cross-over ratio threshold value or not, and obtaining a first judging result.
(4) And if the first judgment result is yes, obtaining a vehicle target detection frame corresponding to the illegal vehicle, and extracting the characteristic points of the illegal vehicle in the vehicle target detection frame.
(5) And carrying out nearest neighbor association matching on the illegal vehicle characteristic points in the vehicle target detection frame and the illegal vehicle characteristic points in the prediction target frame to obtain characteristic point matching coefficients.
If a detection target frame meeting the requirements exists, extracting Harris corner points as characteristic points by using a plesey algorithm in the range of the detection target frame, and carrying out nearest neighbor association matching with the characteristic points in the prediction frame. And calculating Euclidean distance between each point as an association cost matrix, starting from the minimum element in the matrix, carrying out one-to-one matching on each point and the nearest neighbor point, setting an association threshold as a distance maximum value limit of neighbor point matching in the matching process, and calculating the number n of successfully matched points. Setting the number of feature points obtained from the previous frame as X, detecting the number of feature points in the currently examined target frame as Y, and calculating the matching coefficient of the feature pointsThe obtained characteristic point matching coefficients are specifically:
1) And calculating Euclidean distance between each illegal vehicle characteristic point in the vehicle target detection frame and the prediction target frame, and constructing an association cost matrix.
2) And comparing matrix elements in the association cost matrix with a preset association threshold.
3) And regarding the two feature points corresponding to the matrix elements exceeding the preset association threshold as nearest neighbor association matching point pairs.
4) And calculating the characteristic point matching coefficient according to the number of nearest neighbor associated matching point pairs, the number of illegal vehicle characteristic points of the previous detection frame and the number of illegal vehicle characteristic points of the current detection frame.
(7) If the first judgment result is negative, expanding the current prediction target frame with a preset expansion coefficient, and extracting the illegal vehicle characteristic points in the expanded prediction frame.
(8) And carrying out nearest neighbor association matching on the illegal vehicle characteristic points in the predicted target frame before and after expansion to obtain the characteristic point matching coefficient.
If there is no target frame meeting the requirement of the cross-over ratio IOU, predicting the target frame to the current frame by using the expansion coefficient k p And (3) expanding, extracting Harris corner points, matching feature points according to the nearest neighbor association matching method in the previous step, and calculating feature point matching coefficients.
When the matching method is different from the nearest neighbor association matching method, the feature points matched are the feature points of the illegal vehicles in the predicted target frames before and after expansion.
(9) When the characteristic point matching coefficient is larger than the coefficient threshold value, matching is successful; and when the characteristic point matching coefficient is smaller than or equal to a coefficient threshold value, matching fails.
Judging whether the characteristic point matching coefficient is larger than a threshold thr match If the feature point matching is larger than the threshold value, the feature point matching is successful, otherwise, the feature point matching fails.
S31: when the tracking state is that target tracking is normal, acquiring clear license plates in a vehicle target detection frame of a current detection frame, and outputting the illegal vehicle illegal behavior image and the license plate information; and if the clear license plate of the illegal vehicle cannot be acquired, returning to the step S2 of carrying out target tracking on the illegal vehicle by using Kalman filtering.
When the IOU matching and the feature point matching are both larger than the threshold value, the condition tracking of the illegal target is normal, if the condition tracking is successful, the vehicle license plate information is tried to be acquired in a detection frame, if the condition is successfully acquired, a photo is taken as a clear license plate evidence photo, a timestamp is recorded, the acquisition of a video frame is stopped, the stored video and the recorded photos of the illegal act, such as illegal termination and license plate cleaning, are output, otherwise, the detection frame target frame is used for correcting the predicted position, if the detection frame leaves the illegal area for the first time, the photo is taken as an illegal act termination photo, the target tracking is carried out on the illegal vehicle by using Kalman filtering in the step S2, and the next tracking detection period is started.
S32: when the tracking state is that the target part is blocked, determining the fitting position of the illegal vehicle of the current detection frame according to the characteristic points of the illegal vehicle of the current detection frame and the last detection frame, and returning to the step S2 of carrying out target tracking on the illegal vehicle by utilizing Kalman filtering.
When the IOU matching is smaller than the threshold value, but the characteristic point matching extracted by using the current frame prediction frame position is larger than the threshold value, the fact that the vehicle is at the tracking prediction position is indicated, but partial shielding is caused by the factors of the front vehicle or the environment, and the target detection algorithm cannot identify the tracked target. A vehicle with its left side blocked by a truck as shown in fig. 4.
In this case, since the interval time between the two detection frames is short, the feature point relative position change of the road vehicle due to the visual angle change is ignored, and the feature point relative to the vehicle position between the front and rear detection frames can be regarded as unchanged, so that the target vehicle position of the current frame can be reversely deduced through the feature point position.
In the feature point matching pair (nearest-neighbor association matching point pair), the relative position (x) between each feature point of the last detection frame and the center point of the target frame (the center point of the vehicle target detection frame or the center point of the predicted target frame) is calculated i ,y i ) The position (u) of each corresponding feature point of the current frame in the image coordinates i ,v i ) Let the vehicle center point of the current frame be (U, V), obtain the equation set from the above-mentioned geometric relationship:
and (2) the equation set has only two unknowns, the fitting position of the target vehicle of the current detection frame can be obtained by solving the equation set by using a least square method, the target tracking of the illegal vehicle is carried out by utilizing Kalman filtering in the step (S2), and the next tracking detection period is entered.
S33: when the tracking state is that the target tracking is lost, in the next detection frame, re-tracking the illegal vehicle is carried out based on the position and the size of the vehicle target detection frame of the last frame before the loss; and returning to the step S2 to perform target tracking on the illegal vehicle by using Kalman filtering after the re-tracking is successful.
When the feature point matching is smaller than the threshold value, the system is not capable of reliably tracking the target, and the target vehicle is possibly blocked seriously by other vehicles or the tracking module is abnormal. And judging the target state as a lost target no matter how the IOU matching result is, starting to record the lost duration of the lost target, and performing re-tracking operation.
And for judging the lost target, searching and matching the next detection frame by the re-tracking module, taking the constraint effect of the lane on the running of the vehicle into consideration, and in order to reduce the probability of mismatching, obtaining the potential occurrence domain of the target according to the position size of the target frame of the previous frame, and constraining the searching range.
As shown in fig. 5, the specific steps for re-tracking an illegal vehicle are as follows:
(1) Acquiring the position and the size of a vehicle target detection frame of the last frame before the loss;
(2) And determining the movement direction of the illegal vehicle by using the velocity vector when the Kalman filtering is used for tracking the target.
(3) And determining a target potential occurrence domain according to the vehicle target detection frame of the last frame before the loss and the movement direction of the illegal vehicle.
And when the detection frame crosses two lanes, selecting the outer boundary lines of the two lanes, and taking the region formed by the boundary lines, the perpendicular line of the lane lines and the visual field edge as the potential occurrence domain of the target.
(4) And calculating the relative positions of the feature points of the illegal vehicles in the vehicle target detection frame of the last frame before the loss and the center point of the vehicle target detection frame, and marking the relative positions as first relative positions.
(5) And calculating the intersection ratio of each vehicle detection frame in the next detection frame and the target potential occurrence domain, and recording the intersection ratio as a heavy tracking intersection ratio.
(6) And when the vehicle target detection frame with the re-tracking intersection ratio being larger than the re-tracking intersection ratio threshold exists, extracting the illegal vehicle characteristic points in the vehicle target detection frame during re-tracking, calculating the relative coordinates of the illegal vehicle characteristic points extracted during re-tracking and the central point of the vehicle target detection frame during re-tracking, and recording the relative coordinates as a second relative position.
(7) And carrying out characteristic point matching according to the first relative position and the second relative position by utilizing a nearest neighbor correlation matching method to obtain a characteristic point matching coefficient during re-tracking.
(8) When the characteristic point matching coefficient in the re-tracking is larger than the coefficient threshold in the re-tracking, recapturing the illegal vehicle in the potential target occurrence domain; and correcting the vehicle position by utilizing a vehicle target detection frame for capturing the illegal vehicle currently, and adding a re-tracking mark to the re-captured illegal vehicle. And returning to the step S2 of tracking the target of the illegal vehicle by using Kalman filtering, starting the next tracking detection period, and verifying the heavy tracking reliability by using a vehicle tracking state detection module.
In this embodiment, when detecting that the license plate of the illegal target vehicle, the target vehicle moves to the edge of the camera field of view or the target loss time exceeds a threshold, the stored video and the photos of the illegal behavior, the illegal termination, the clear license plate and the like recorded in the step S2 and the step S3 are derived, and classified according to different target tracking conditions.
1) The tracking result of the target vehicle which does not have the loss condition in the process and detects the license plate of the vehicle has higher credibility, and the video clips and the process photos from the detection start to the detection of the license plate are reserved and used as video evidence of illegal actions.
2) For the vehicles with target loss during tracking, namely the vehicles with re-tracking marks added in the step S33, the situation that irrelevant vehicles with very similar or identical appearance are mistakenly identified as target illegal vehicles cannot be excluded by the re-tracking module considering the potential occurrence area, such videos and process photos are used as reference basis of illegal behaviors, and the handover traffic management department carries out manual audit to judge whether the video and the process photos can be used as the evidence of the illegal vehicles.
3) And submitting the vehicles which can not acquire the license plates of the vehicles all the time to a traffic management department, performing manual verification, and observing whether the images or video evidence of the illegal behaviors of the vehicles exist through cameras at other positions.
Compared with the prior art, the method and the device for detecting the illegal vehicles according to the photos have the advantages that the illegal vehicles are detected firstly, then target tracking is carried out on the illegal vehicles through Kalman filtering and detection frame matching until the vehicle license plate is detected, and video evidence of illegal behaviors is generated, so that the effective field of view of the illegal monitoring cameras is increased to the vehicle detection position. In addition, in order to deal with the situation that the target is continuously shielded, harris angular points are extracted to serve as vehicle feature points, the position of the partially shielded target is matched with the position of the corresponding feature point in the current frame, and the vehicle position is fitted by using a least square method to carry out subsequent tracking, so that a tracking system keeps steady under the influence of shielding, a potential occurrence domain is designed aiming at the situation that the tracked target is lost, searching and re-tracking of the lost target are carried out in the potential occurrence domain, the tracking capability of the system is improved, the generated video judgment is classified according to the tracking process, and convenience is provided for traffic management departments to handle illegal behaviors.
Example two
The embodiment provides a traffic offence detection system based on visual target tracking, which comprises:
the area determining and target detecting module is used for determining an illegal detection area, capturing illegal vehicles in the illegal detection area based on a target detecting algorithm, obtaining clear vehicle license plates of the illegal vehicles, and outputting illegal vehicle illegal behavior images and vehicle license plate information.
And the characteristic extraction and target tracking module is used for extracting Harris angular points based on the captured illegal vehicle images to serve as illegal vehicle characteristic points when clear license plates of the illegal vehicles cannot be acquired, and carrying out target tracking on the illegal vehicles by utilizing Kalman filtering.
And the matching and tracking state judging module is used for matching the predicted target frame with the feature points of the illegal vehicle when the preset detection frame is reached, and judging the tracking state of the illegal vehicle according to a matching result.
The vehicle license plate acquisition and target tracking module is used for acquiring clear vehicle license plates in a vehicle target detection frame of a current detection frame when the tracking state is that target tracking is normal, and outputting the illegal vehicle illegal behavior image and the vehicle license plate information; and if the clear license plate of the illegal vehicle cannot be acquired, executing target tracking of the illegal vehicle by using Kalman filtering in the matching and tracking state judging module.
And the position fitting and target tracking module is used for determining the fitting position of the illegal vehicle of the current detection frame according to the characteristic points of the illegal vehicle of the current detection frame and the last detection frame when the tracking state is the target partial shielding, and executing ' utilizing Kalman filtering to track the target of the illegal vehicle ' in the matching and tracking state judging module '.
The re-tracking module is used for carrying out re-tracking on the illegal vehicle in the next detection frame based on the position and the size of the vehicle target detection frame of the last frame before the loss when the tracking state is that the target tracking is lost; and after the re-tracking is successful, executing target tracking on the illegal vehicle by utilizing Kalman filtering in the matching and tracking state judging module.
Example III
The embodiment provides an electronic device, including a memory and a processor, where the memory is configured to store a computer program, and the processor is configured to run the computer program to enable the electronic device to execute the traffic offence detection method based on visual target tracking according to the first embodiment.
Alternatively, the electronic device may be a server.
In addition, the embodiment of the invention also provides a computer readable storage medium, which stores a computer program, and the computer program realizes the traffic offence detection method based on visual target tracking of the first embodiment when being executed by a processor.
Embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.
Claims (10)
1. A traffic offence detection method based on visual target tracking, the method comprising:
determining an illegal detection area, capturing an illegal vehicle in the illegal detection area based on a target detection algorithm, acquiring a clear vehicle license plate of the illegal vehicle, and outputting an illegal vehicle illegal behavior image and vehicle license plate information;
when the clear license plate of the illegal vehicle cannot be obtained, extracting Harris corner points based on the captured illegal vehicle image to serve as characteristic points of the illegal vehicle, and carrying out target tracking on the illegal vehicle by utilizing Kalman filtering;
when a preset detection frame is carried out, matching a predicted target frame and feature points of the illegal vehicle is carried out on the illegal vehicle, and the tracking state of the illegal vehicle is judged according to a matching result;
When the tracking state is that target tracking is normal, acquiring clear license plates in a vehicle target detection frame of a current detection frame, and outputting the illegal vehicle illegal behavior image and the license plate information; if the clear license plate of the illegal vehicle cannot be obtained, returning to the step of tracking the target of the illegal vehicle by using Kalman filtering;
when the tracking state is that the target part is blocked, determining the fitting position of the illegal vehicle of the current detection frame according to the characteristic points of the illegal vehicle of the current detection frame and the last detection frame, and returning to the step of 'performing target tracking on the illegal vehicle by using Kalman filtering';
when the tracking state is that the target tracking is lost, in the next detection frame, re-tracking the illegal vehicle is carried out based on the position and the size of the vehicle target detection frame of the last frame before the loss; and returning to the step of 'tracking the target of the illegal vehicle by using Kalman filtering' after the re-tracking is successful.
2. The method according to claim 1, wherein determining the area of the violation detection when detecting for the violation of the lane-like violation, in particular comprises:
Image clipping is carried out on the basis of the image shot by the illegal detection system to obtain a pavement area image;
performing color space conversion on the pavement area image, and performing edge detection on the converted image by using a Canny operator;
extracting straight line segments by Hough transformation according to an edge detection result, determining lane boundaries according to the extracted straight line segments, and obtaining a plurality of lane areas;
optionally, one of the lane areas is the violation detection area.
3. The method according to claim 1, wherein the extracting Harris corner points as feature points of the illegal vehicle based on the captured illegal vehicle image, specifically comprises:
carrying out gray scale treatment on the illegal vehicle image, and respectively calculating the gradient in the x direction and the gradient in the y direction in the vehicle target detection frame through a difference operator;
constructing a first matrix based on the gradient in the x-direction and the gradient in the y-direction;
performing Gaussian smoothing filtering on elements in the first matrix;
selecting the size of a window and the corresponding weight, and calculating a second matrix according to the size of the window, the weight and the first matrix;
calculating corresponding Harris response values at each pixel point in the illegal vehicle image according to the second matrix;
And screening pixel points with the Harris response value larger than a threshold value as the Harris corner points to obtain the illegal vehicle feature points.
4. The method according to claim 1, wherein the matching of the predicted target frame and the feature points of the illegal vehicle is performed on the illegal vehicle, specifically comprising:
in the current detection frame, carrying out vehicle detection in the road range by utilizing the target detection algorithm to obtain a plurality of vehicle detection frames corresponding to the current detection frame;
the vehicle target frame offset speed vector obtained by the Kalman filtering is acted on a vehicle target detection frame extracted from the previous detection frame and the illegal vehicle characteristic points to obtain a predicted target frame of the current detection frame and predicted positions of the illegal vehicle characteristic points;
calculating the intersection ratio of the predicted target frame and each vehicle detection frame;
judging whether the cross-over ratio is larger than a cross-over ratio threshold value or not to obtain a first judging result;
if the first judgment result is yes, a vehicle target detection frame corresponding to the illegal vehicle is obtained, and feature points of the illegal vehicle in the vehicle target detection frame are extracted;
performing nearest neighbor association matching on the illegal vehicle characteristic points in the vehicle target detection frame and the illegal vehicle characteristic points in the prediction target frame to obtain characteristic point matching coefficients;
If the first judgment result is negative, expanding the current prediction target frame with a preset expansion coefficient, and extracting the illegal vehicle characteristic points in the expanded prediction frame;
performing nearest neighbor association matching on the illegal vehicle feature points in the predicted target frame before and after expansion to obtain the feature point matching coefficients;
when the characteristic point matching coefficient is larger than the coefficient threshold value, matching is successful;
and when the characteristic point matching coefficient is smaller than or equal to a coefficient threshold value, matching fails.
5. The method according to claim 4, wherein the nearest neighbor association matching is performed on the illegal vehicle feature points in the vehicle target detection frame and the illegal vehicle feature points in the prediction target frame to obtain feature point matching coefficients, and the method specifically comprises:
calculating the Euclidean distance between each illegal vehicle characteristic point in the vehicle target detection frame and the prediction target frame, and constructing an association cost matrix;
comparing matrix elements in the association cost matrix with a preset association threshold;
regarding two feature points corresponding to the matrix elements exceeding the preset association threshold as nearest neighbor association matching point pairs;
And calculating the characteristic point matching coefficient according to the number of nearest neighbor associated matching point pairs, the number of illegal vehicle characteristic points of the previous detection frame and the number of illegal vehicle characteristic points of the current detection frame.
6. The method according to claim 5, wherein determining the fitting position of the illegal vehicle of the current detection frame according to the characteristic points of the illegal vehicle of the current detection frame and the previous detection frame specifically comprises:
calculating the relative positions of the matching feature points of the previous detection frame and the center point of the target frame, the positions of the matching feature points of the current detection frame in the illegal vehicle image coordinates, and constructing an equation set by combining the illegal vehicle center point coordinates of the current detection frame; the equation set comprises n equations; n is the number of the matched feature points; the matching feature points are nearest-neighbor association matching point pairs obtained after nearest-neighbor association matching;
and solving the equation set by using a least square method to obtain the fitting position of the illegal vehicle of the current detection frame.
7. The method of claim 4, wherein determining the tracking status of the offending vehicle based on the matching result comprises:
when the first judgment result is yes and the second judgment result is yes, the tracking state is that the target tracking is normal;
When the first judgment result is NO and the second judgment result is yes, the tracking state is the target part shielding;
and when the first judgment result is NO and the second judgment result is NO, the tracking state is the target tracking loss.
8. The method of claim 5, wherein the re-tracking of the offending vehicle is based on the position and size of a vehicle target detection frame of a last frame prior to the loss, in particular comprising:
acquiring the position and the size of a vehicle target detection frame of the last frame before the loss;
determining the movement direction of the illegal vehicle by using the velocity vector when the Kalman filtering is used for tracking the target;
determining a target potential occurrence domain according to the vehicle target detection frame of the last frame before the loss and the movement direction of the illegal vehicle;
calculating the relative positions of the feature points of the illegal vehicles in the vehicle target detection frame of the last frame before the loss and the center point of the vehicle target detection frame, and marking the relative positions as first relative positions;
calculating the intersection ratio of each vehicle detection frame in the next detection frame and the target potential occurrence domain, and marking the intersection ratio as a heavy tracking intersection ratio;
When the vehicle target detection frame with the re-tracking intersection ratio being larger than the re-tracking intersection ratio threshold exists, extracting the illegal vehicle characteristic points in the vehicle target detection frame during re-tracking, calculating the relative coordinates of the illegal vehicle characteristic points extracted during re-tracking and the center point of the vehicle target detection frame during re-tracking, and recording the relative coordinates as a second relative position;
performing feature point matching according to the first relative position and the second relative position by using a nearest neighbor correlation matching method to obtain a feature point matching coefficient during re-tracking;
when the characteristic point matching coefficient in the re-tracking is larger than the coefficient threshold in the re-tracking, recapturing the illegal vehicle in the potential target occurrence domain;
and correcting the vehicle position by utilizing a vehicle target detection frame for capturing the illegal vehicle currently, and adding a re-tracking mark to the re-captured illegal vehicle.
9. The method according to claim 8, wherein determining a target potential occurrence domain based on the vehicle target detection frame of the last frame before the loss and the movement direction of the offending vehicle, in particular comprises:
calculating the intersection point of the detection frame center speed reverse ray of the vehicle target detection frame of the last frame before the loss and the rear frame line of the detection frame, and taking a perpendicular line of the lane line through the intersection point;
Judging the lane where the illegal vehicle is located according to the coordinates of the detection frame lines at two sides, and extracting lane boundary lines;
and regarding the area formed by the lane boundary line, the perpendicular line of the lane line and the visual field edge as the target potential occurrence domain.
10. A traffic offence detection system based on visual target tracking, the system comprising:
the area determining and target detecting module is used for determining an illegal detecting area, capturing illegal vehicles in the illegal detecting area based on a target detecting algorithm, obtaining clear vehicle license plates of the illegal vehicles, and outputting illegal vehicle illegal behavior images and vehicle license plate information;
the feature extraction and target tracking module is used for extracting Harris corner points based on the captured illegal vehicle images to serve as illegal vehicle feature points when clear license plates of the illegal vehicles cannot be obtained, and carrying out target tracking on the illegal vehicles by utilizing Kalman filtering;
the matching and tracking state judging module is used for carrying out matching between a predicted target frame and the feature points of the illegal vehicle when the preset detection frame is carried out, and judging the tracking state of the illegal vehicle according to a matching result;
The vehicle license plate acquisition and target tracking module is used for acquiring clear vehicle license plates in a vehicle target detection frame of a current detection frame when the tracking state is that target tracking is normal, and outputting the illegal vehicle illegal behavior image and the vehicle license plate information; if the clear license plate of the illegal vehicle cannot be obtained, performing target tracking on the illegal vehicle by using Kalman filtering in a matching and tracking state judging module;
the position fitting and target tracking module is used for determining the fitting position of the illegal vehicle of the current detection frame according to the characteristic points of the illegal vehicle of the current detection frame and the last detection frame when the tracking state is partially blocked, and executing 'utilizing Kalman filtering to track the target of the illegal vehicle' in the matching and tracking state judging module;
the re-tracking module is used for carrying out re-tracking on the illegal vehicle in the next detection frame based on the position and the size of the vehicle target detection frame of the last frame before the loss when the tracking state is that the target tracking is lost; and after the re-tracking is successful, executing target tracking on the illegal vehicle by utilizing Kalman filtering in the matching and tracking state judging module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311198251.0A CN117237883A (en) | 2023-09-18 | 2023-09-18 | Traffic illegal behavior detection method and system based on visual target tracking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311198251.0A CN117237883A (en) | 2023-09-18 | 2023-09-18 | Traffic illegal behavior detection method and system based on visual target tracking |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117237883A true CN117237883A (en) | 2023-12-15 |
Family
ID=89090559
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311198251.0A Pending CN117237883A (en) | 2023-09-18 | 2023-09-18 | Traffic illegal behavior detection method and system based on visual target tracking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117237883A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117689907A (en) * | 2024-02-04 | 2024-03-12 | 福瑞泰克智能系统有限公司 | Vehicle tracking method, device, computer equipment and storage medium |
-
2023
- 2023-09-18 CN CN202311198251.0A patent/CN117237883A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117689907A (en) * | 2024-02-04 | 2024-03-12 | 福瑞泰克智能系统有限公司 | Vehicle tracking method, device, computer equipment and storage medium |
| CN117689907B (en) * | 2024-02-04 | 2024-04-30 | 福瑞泰克智能系统有限公司 | Vehicle tracking method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102267335B1 (en) | Method for detecting a speed employing difference of distance between an object and a monitoring camera | |
| JP4587038B2 (en) | Vehicle position detection method, and vehicle speed detection method and apparatus | |
| US8208021B2 (en) | Vehicle and lane mark detection device | |
| CN101030256B (en) | Method and apparatus for cutting vehicle image | |
| US8737690B2 (en) | Video-based method for parking angle violation detection | |
| US9047518B2 (en) | Method for the detection and tracking of lane markings | |
| CN103927762B (en) | Target vehicle automatic tracking method and device | |
| KR101735365B1 (en) | The robust object tracking method for environment change and detecting an object of interest in images based on learning | |
| CN111145555B (en) | Method and device for detecting vehicle violation | |
| WO2014132729A1 (en) | Stereo camera device | |
| US11367349B2 (en) | Method of detecting speed using difference of distance between object and monitoring camera | |
| CN106650730A (en) | Turn signal lamp detection method and system in car lane change process | |
| KR101001302B1 (en) | A sensing system of a lane offense cars using the corner data and sensing method using the same | |
| Rodríguez et al. | An adaptive, real-time, traffic monitoring system | |
| CN117237883A (en) | Traffic illegal behavior detection method and system based on visual target tracking | |
| CN105303157A (en) | Algorithm to extend detecting range for AVM stop line detection | |
| CN108520528B (en) | Mobile vehicle tracking method based on improved difference threshold and displacement matching model | |
| CN107506753B (en) | Multi-vehicle tracking method for dynamic video monitoring | |
| Klubsuwan et al. | Traffic violation detection using multiple trajectories evaluation of vehicles | |
| CN111768630A (en) | Violation waste image detection method and device and electronic equipment | |
| CN116682268A (en) | Portable urban road vehicle violation inspection system and method based on machine vision | |
| Munajat et al. | Vehicle detection and tracking based on corner and lines adjacent detection features | |
| Abdagic et al. | Counting traffic using optical flow algorithm on video footage of a complex crossroad | |
| JP4788926B2 (en) | Vehicle number reading device | |
| CN113870185A (en) | Image processing method based on image snapshot, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |