CN114022791B - Vehicle track motion feature recognition method based on high-altitude visual angle recognition system - Google Patents

Vehicle track motion feature recognition method based on high-altitude visual angle recognition system Download PDF

Info

Publication number
CN114022791B
CN114022791B CN202111201539.XA CN202111201539A CN114022791B CN 114022791 B CN114022791 B CN 114022791B CN 202111201539 A CN202111201539 A CN 202111201539A CN 114022791 B CN114022791 B CN 114022791B
Authority
CN
China
Prior art keywords
frame
vehicle
vehicle target
mth
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111201539.XA
Other languages
Chinese (zh)
Other versions
CN114022791A (en
Inventor
贺宜
曹博
吴超仲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan University of Technology WUT
Original Assignee
Wuhan University of Technology WUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University of Technology WUT filed Critical Wuhan University of Technology WUT
Priority to CN202111201539.XA priority Critical patent/CN114022791B/en
Publication of CN114022791A publication Critical patent/CN114022791A/en
Application granted granted Critical
Publication of CN114022791B publication Critical patent/CN114022791B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a vehicle track motion feature recognition method based on a high-altitude visual angle recognition system. The high-altitude visual angle recognition system comprises: the device comprises an aerial photographing device, a calculation processing host and a display device. The method comprises the steps of utilizing an aerial image device to collect high-altitude video data to manufacture a high-altitude image training data set and a high-altitude image sequence data set; the high-altitude image training data set is used for training YOLOv models; vehicle identification is carried out on the high-altitude image sequence data set to obtain a high-altitude image sequence vehicle identification frame set; generating an original vehicle track motion characteristic recognition text data set by using Kalman filtering and Hungary matching algorithm; and finally forming a five-level vehicle track motion feature recognition text data set through four processes of data preprocessing, motion feature extraction, lane number detection and coordinate conversion. The method can solve the problem of missing of the relevant part of the vehicle target data, and provides a specific implementation method for extracting the characteristics of the vehicle position, speed, acceleration and lane number.

Description

Vehicle track motion feature recognition method based on high-altitude visual angle recognition system
Technical Field
The invention belongs to the field of intelligent transportation, and particularly relates to a vehicle track motion feature recognition method based on a high-altitude visual angle recognition system.
Background
In recent years, the rapid development of artificial intelligence technology and automatic driving industry promotes the intellectualization of road traffic on the one hand, and also puts higher demands on the acquisition of road traffic information on the other hand. The artificial intelligence technology plays a great role in the fields of feature extraction, data mining and decision control, and simultaneously, the research on the automatic driving technology is tightened in each country, and the automatic driving is generally classified into six grades: l0 manual driving, L1 auxiliary driving, L2 semiautomatic driving, L3 high automatic driving, L4 ultrahigh automatic driving and L5 full automatic driving. The assessment of these grades requires support from real vehicle travel data in real scenes, and vehicle trajectory data under real roads can be used to verify vehicles in autonomous driving mode to assess their grade. At present, related researches have been carried out to collect vehicle track data. The chinese patent application CN110751099a proposes a high-precision track extraction method using aerial video, which focuses on denoising, splicing and smoothing of vehicle tracks, but the extraction of vehicle motion parameters is not described in detail, and in the vehicle target association stage, the influence of the change condition of the vehicle motion state of the previous frame on the vehicle state of the current frame is not considered. The Chinese patent application CN111611918A also provides a traffic flow data set acquisition and construction method based on aerial video, but the method for extracting the traffic flow parameters is deficient, and the utilized target tracking means is a single target tracking method, and the tracking method is to be expanded; the Chinese patent application CN111145545A focuses on cross-camera monitoring in road traffic detection, and is deficient in a method for extracting the motion characteristics of traffic vehicles.
Disclosure of Invention
In order to solve the technical problems, the invention provides a vehicle track motion feature recognition method based on a high-altitude visual angle recognition system.
The high-altitude visual angle recognition system is characterized by comprising: the device comprises an aerial photographing device, a calculation processing host and a display projection device;
the aerial photographing device is connected with the computing processing host in a wireless mode; the computing processing host is connected with the display projection device in a wired mode;
The aerial photographing device is used for collecting video image data of the road vehicle under the high-altitude visual angle and transmitting the video image data to the calculation processing host in a wireless mode; the calculation processing host is used for processing the road vehicle video image data under the high-altitude visual angle acquired by the aerial photographing device, further obtaining a vehicle image recognition result and a track generation result through the vehicle track motion characteristic recognition method under the high-altitude visual angle, and transmitting the vehicle image recognition result and the track generation result to the display projection device for display;
The aerial photographing device is located right above a road surface during remote photographing, namely, the included angle between the line of sight photographed by a camera of the aerial photographing device and the road surface is 90 degrees.
The vehicle track movement characteristic recognition method is characterized by comprising the following steps of:
Step 1: the calculation processing host computer is positioned right above the road pavement by using the aerial image capturing device in a wireless mode to capture video image data, and is used for forming a high-altitude image training data set, manually marking the high-altitude image training data set, marking the circumscribed rectangular frame of the vehicle target and the vehicle type, and forming a high-altitude image training vehicle marking frame set;
step 2: the calculation processing host computer is positioned right above the road surface by using the aerial image device in a wireless mode to shoot video image data, and is used for forming an overhead image sequence data set for subsequently extracting vehicle track data; the road in the image picture of the high-altitude image sequence data set is positioned at the middle position of the image;
Step 3: introducing YOLOv deep learning network models, sequentially inputting each frame of image in a high-altitude image training dataset and a vehicle marking frame corresponding to each frame of image in a high-altitude image training vehicle marking frame set into the YOLOv deep learning network models for training, constructing a loss function model by using a GIOU method, optimizing a loss function value by an Adam optimization algorithm, and identifying a vehicle target in the high-altitude image sequence dataset by using the trained YOLOv deep learning network model to obtain a high-altitude image sequence vehicle identification frame set;
Step 4: starting from the first frame of vehicle target circumscribed rectangular frame data in the high-altitude image sequence vehicle target identification frame set, the following processing procedure is carried out: applying Kalman filtering to a vehicle target boundary frame of a previous frame to obtain vehicle target estimated frame data of a current frame, performing association matching on the vehicle target identified frame data of the current frame and the vehicle target boundary frame in the vehicle target estimated frame data by using a Hungary association algorithm, wherein a matching mechanism is an IOU distance to obtain an ID sequence number of the vehicle target identified frame data of the current frame, namely the ID sequence number of the vehicle target of the current frame, and marking a new ID sequence number on the incompletely matched vehicle target frame data of the current frame; until the high-altitude image sequence ends. Combining the video Frame sequence number, the vehicle ID sequence number and the high-altitude image sequence vehicle target Frame set after the association matching process to form an original vehicle track motion characteristic identification text data set;
Step 5: and sequentially carrying out four processing processes of data preprocessing, motion feature extraction, lane number detection and coordinate conversion on the original vehicle track motion feature recognition text data set to finally form a five-level vehicle track motion feature recognition text data set.
Preferably, the high-altitude image training data set in step 1 is:
{datae(x,y),e∈[1,E],x∈[1,X],y∈[1,Y]}
Wherein, data e (X, Y) represents the X-th row and Y-th column pixel information of the E-th frame image in the high-altitude image training data set, E is the total frame number of the high-altitude image training data set, X is the number of rows of the image in the high-altitude image training data set, and Y is the number of columns of the image in the high-altitude image training data set;
the high-altitude image training vehicle marking frame set in the step 1 is as follows:
Wherein, Representing the left upper-corner abscissa of the nth vehicle target mark rectangular frame in the e-th frame image in the high-altitude image training vehicle mark frame set,/>Representing the left upper corner ordinate of the nth vehicle target mark rectangular frame in the e-th frame image in the high-altitude image training vehicle mark frame set; /(I)Representing the right lower-corner abscissa of the rectangular frame of the n-th vehicle target mark in the e-th frame image in the high-altitude image training vehicle mark frame set,/>Representing the lower right corner ordinate of the nth vehicle target mark rectangular frame in the e-th frame image in the high-altitude image training vehicle mark frame set; type e,n represents the mark category of the nth vehicle object in the e-th frame image in the high-altitude image training vehicle mark frame set;
preferably, in the step 2, the fixed shooting frame rate of the aerial camera is FPS, the length of the shot road is L, and the number of coverage pixel units in the road length direction of the shot picture is G; the shooting size of the high-altitude image data is X and Y;
the high-altitude image sequence data set in the step2 is as follows:
{datat(x,y),t∈[1,T],x∈[1,X],y∈[1,Y]}
Wherein, data t (X, Y) represents the X-th row and Y-th column pixel information of the T-th frame image in the high-altitude image sequence data set, T is the total frame number of the high-altitude image sequence data set, X is the number of rows of the image in the high-altitude image sequence data set, and Y is the number of columns of the image in the high-altitude image sequence data set;
preferably, the YOLOv network framework in step 3 is a yolo5x network structure;
and 3, the high-altitude image sequence vehicle identification frame set is as follows:
Wherein, Representing the left upper-corner abscissa of the circumscribed rectangular frame of the nth vehicle object in the t frame image in the vehicle identification frame set of the high-altitude image sequence,/>Representing the left upper-corner ordinate of the circumscribed rectangular frame of the nth vehicle object in the t frame image in the high-altitude image sequence vehicle identification frame set; /(I)Representing the right lower-corner abscissa of the circumscribed rectangular frame of the nth vehicle object in the t frame image in the vehicle identification frame set of the high-altitude image sequence,/>Representing the lower right corner ordinate of the circumscribed rectangular frame of the nth vehicle object in the t frame image in the high-altitude image sequence vehicle identification frame set; type t,n represents the class of the nth vehicle object in the t-th frame image in the frame set of the high-altitude image sequence vehicle identification;
preferably, in the recording of the current video frame number in step 4, the set of recorded video frame numbers is
Framet,n{framet,n}
Wherein frame t,n represents the video sequence number corresponding to the nth vehicle object of the nth frame.
The Kalman filtering processing process in the step4 sequentially comprises the following steps: initializing a vehicle target state vector; initializing a state transition matrix, initializing a covariance matrix, initializing an observation matrix and initializing a system noise matrix; predicting the current frame of vehicle target state vector according to the optimal estimated value of the previous frame of vehicle target state vector to obtain the predicted value of the current frame of vehicle target state vector; predicting a current frame vehicle target system error covariance matrix according to a previous frame vehicle target system error covariance matrix to obtain a current frame vehicle target system error covariance matrix predicted value; updating a Kalman coefficient by using a covariance matrix predicted value of a current frame vehicle target system; estimating according to the predicted value of the current frame vehicle target state vector and the system observation value to obtain the optimal estimated value of the current frame vehicle target state vector; updating the current frame of the vehicle target system error covariance matrix; extracting a current frame vehicle target estimation frame set from the current frame vehicle target state vector optimal estimation value;
in the process of initializing the vehicle target state vector by the kalman filter in the step 4, the vehicle target boundary frame characteristics are described by using the abscissa of the center of the vehicle target boundary frame, the ordinate of the center of the boundary frame, the area of the boundary frame and the aspect ratio of the boundary frame, and the motion state information of the boundary frame is described by adopting a linear uniform velocity model, namely:
Wherein, The motion state information of the boundary box is represented by u, the abscissa of the center of the boundary box, v, the ordinate of the center of the boundary box, s, the area of the boundary box, r, the transverse-longitudinal ratio of the boundary box, which is generally constant,/>Representing the rate of change of the center abscissa of the bounding box,/>Representing the ordinate of the centre of the bounding box,/>Indicating the area change rate of the bounding box. The motion state information of the mth vehicle target bounding box of the t-1 th frame is described as:
Wherein, Motion state information indicating the mth vehicle target boundary frame of the t-1 frame, u t-1,m indicating the abscissa of the center of the mth vehicle target boundary frame of the t-1 frame, v t-1,m indicating the ordinate of the center of the mth vehicle target boundary frame of the t-1 frame, s t-1,m indicating the area of the mth vehicle target boundary frame of the t-1 frame, r t-1,m indicating the ratio of the abscissa to the ordinate of the mth vehicle target boundary frame of the t-1 frame,/>Represents the central abscissa change rate of the mth vehicle target boundary frame of the t-1 frame,/>Ordinate representing the center of the mth vehicle target bounding box of the t-1 th frame,/>Representing the area change rate of the mth vehicle target boundary frame of the t-1 th frame;
The calculation formulas of the abscissa, the ordinate and the boundary frame area of the center of the mth vehicle target frame of the t-1 frame are as follows:
Wherein, Representing the upper left-hand abscissa of the target frame of the mth vehicle in the t-1 frame,/>Represents the lower right-hand corner abscissa of the frame of the mth vehicle target in the t-1 frame,/>Representing the upper left vertical coordinate of the frame of the mth vehicle target of the t-1 frame,/>Representing the lower right vertical coordinate of the frame of the mth vehicle target in the t-1 frame;
In the initializing state transition matrix in step 4, the state transition matrix F models the motion of the target state vector, and the state transition matrix F corresponding to the uniform motion model is initialized as follows:
Initializing a covariance matrix, wherein the covariance matrix P represents uncertainty of target position information, and the covariance matrix is an experience parameter;
In the initialization system noise covariance matrix, because the process noise is not measurable, the system noise covariance matrix Q is generally assumed to accord with normal distribution;
in initializing the observation matrix, the observation matrix H is related to the observable variable, and its value is initialized to:
In the initialized observation noise covariance matrix, since the observation noise is not measurable, the observation noise covariance matrix R is generally assumed to accord with normal distribution;
and 4, predicting a current frame of vehicle target state vector according to the optimal estimated value of the last frame of vehicle target state vector by Kalman filtering, wherein the calculation formula of the obtained mth frame of vehicle target state vector predicted value is as follows:
Wherein, Represents the optimal estimated value of the mth vehicle target state vector of the t-1 frame,/>Representing the mth frame of the mth vehicle target state vector predicted value, wherein F is a state transition matrix, B is a control matrix, and u t-1,m is a control gain matrix;
and 4, predicting the current frame of the vehicle target system error covariance matrix according to the previous frame of the vehicle target system error covariance matrix by Kalman filtering, wherein the calculation formula of the obtained mth frame of the vehicle target system error covariance matrix is as follows:
Wherein P t-1,m represents the mth vehicle target systematic error covariance matrix of the t-1 frame, Representing the mth frame of the mth vehicle target system error covariance matrix predicted value, wherein Q is the covariance matrix of the process noise;
in the third step, the kalman filter updates the kalman coefficient by using the predicted value of the current frame system error covariance matrix, and the calculation formula of the mth vehicle target kalman coefficient in the t frame is as follows:
Wherein H is an observation matrix, R is a covariance matrix of observation noise, and K t,m is an mth frame mth vehicle target Kalman coefficient;
In the step 4, the kalman filtering calculates the optimal estimated value of the vehicle target state vector in the current frame according to the predicted value of the vehicle target state vector in the current frame and the observed value of the system, and the calculation formula of the optimal estimated value of the vehicle target state vector in the mth frame is as follows:
Wherein, The optimal estimated value of the target state vector of the mth vehicle in the t-th frame is z t, and the observed value is z t;
In the step 4, in the Kalman filtering updating the current frame system error covariance matrix, the mth frame mth vehicle target system error covariance matrix updating calculation formula is as follows:
wherein P t,m is the mth vehicle target system covariance matrix of the t frame;
And 4, extracting an optimal estimated value of the mth target state vector of the t-th frame from the optimal estimated value of the target state vector of the current frame in the vehicle target estimated frame set of the current frame, wherein the optimal estimated value of the mth target state vector of the t-th frame is described as follows:
Wherein u t,m represents the optimal estimated value of the abscissa of the center of the mth vehicle target boundary frame of the optimal estimation, v t,m represents the optimal estimated value of the ordinate of the center of the mth vehicle target boundary frame of the optimal estimation, s t,m represents the optimal estimated value of the area of the mth vehicle target boundary frame of the optimal estimation, r t,m represents the optimal estimated value of the aspect ratio of the mth vehicle target boundary frame of the optimal estimation, Optimal estimation value of central abscissa change rate of mth frame mth vehicle target boundary frame representing optimal estimation,/>Representing the optimal estimated ordinate rate of change of the center of the mth frame mth vehicle target bounding box,/>Representing the area change rate of the mth vehicle target boundary frame of the t frame of the optimal estimation;
the calculation formula of the current frame vehicle target estimated frame coordinates is as follows:
Wherein, The upper left-hand abscissa of the frame of the mth vehicle object representing the optimal estimate,/>T frame mth vehicle target frame upper left corner ordinate representing optimal estimation,/>The lower right-hand abscissa of the frame of the mth vehicle target of the t frame representing the optimal estimation,/>Representing the optimal estimated lower right-hand ordinate of the frame of the mth vehicle object,
Thus, the current frame vehicle target estimation bounding box set is:
preferably, the hungarian correlation algorithm in step 4 performs matching by calculating the intersection ratio of the target frame IOU of the vehicle;
And 4, calculating the intersection ratio matching of the vehicle target frame IOU by using the Hungary correlation algorithm as follows: calculating the IOU intersection ratio of the mth vehicle target estimation frame of the t frame in the vehicle target estimation frame set of the current frame and the nth vehicle target frame of the t frame in the vehicle target identification frame set of the current frame, wherein the intersection area calculation formula is as follows:
s 1 represents the intersection area of the mth frame of the vehicle target estimation frame set of the current frame and the nth frame of the vehicle target identification frame of the t frame of the vehicle target identification frame set of the current frame;
The combined calculation formula is as follows:
S 2 represents the combined area of the mth frame of the vehicle target estimation frame set of the current frame and the nth frame of the vehicle target identification frame set of the current frame;
the IOU cross ratio calculation formula is:
The Hungary correlation algorithm vehicle frame IOU intersection ratio matching principle is as follows: if the calculated IOU intersection ratio of the mth vehicle target estimated frame of the t frame and the nth vehicle target edge identification frame of the t frame is the largest and belongs to the same vehicle class, the mth vehicle target of the t-1 frame and the nth vehicle target of the t frame belong to the same vehicle target, and the ID serial number of the nth vehicle target of the t frame is marked as the same ID serial number as the mth vehicle target of the t-1 frame. The associated vehicle id sequence number set is:
IDt,n{idt,n}
Wherein id t,n represents the vehicle id number corresponding to the nth vehicle target in the t frame.
And step 4, merging the Frame serial number of the video Frame after the association process, the vehicle ID serial number and the high-altitude image sequence vehicle target Frame set to form an original vehicle track motion feature recognition text data set, wherein the formed original vehicle track motion feature recognition text data set is as follows:
Preferably, the data preprocessing in step 5 is as follows:
firstly, calculating the center point coordinates of a vehicle target frame, wherein the calculation formula of the center point coordinates is as follows:
Wherein, Represents the center point abscissa of the nth vehicle target recognition frame of the t frame,/>Representing the ordinate of the center point of the nth vehicle target recognition frame of the nth frame;
secondly, the width and the height of a vehicle target frame need to be calculated, and a calculation formula is as follows:
Where w t,n denotes the width of the nth vehicle target frame of the t-th frame, and h t,n denotes the height of the nth vehicle target frame of the t-th frame.
Forming a primary vehicle trajectory motion feature recognition text dataset:
When the first-level vehicle track motion feature recognition text data set is subjected to data preprocessing, the vehicle track motion feature recognition text data is screened out by a threshold value discrimination method to form a second-level vehicle track motion feature recognition text data set, and a discrimination formula is as follows:
Wherein, Represents the abscissa after threshold decision screening,/>X 1 represents a lower limit of the abscissa determination threshold, X 2 represents an upper limit of the abscissa determination threshold, Y 1 represents a lower limit of the ordinate determination threshold, and Y 2 represents an upper limit of the ordinate determination threshold;
secondly, counting the vehicle tracks with the same ID sequence number, if the number of the video frames is smaller than a fixed value, judging that the vehicle tracks are fragments of the track fragments, and cleaning the vehicle tracks, wherein a judgment formula is as follows:
Wherein, The number of video frames representing the value of the vehicle ID, threshold representing a fixed value;
the formed secondary vehicle track motion feature recognition text data set is as follows:
Wherein, Video frame number corresponding to vehicle target frame after data screening is represented,/>Vehicle id serial number corresponding to vehicle target frame after data screening is represented,/>Representing the width of the target frame of the vehicle after data screening,/>And representing the height of the target frame of the vehicle after data screening.
The motion characteristic extraction process in the step 5 is as follows:
The vehicle speed of each vehicle id sequence number under each video frame sequence number is calculated first, and the specific process is to calculate the speed of the current frame by using the position difference value and the time difference value of the current frame and the previous frame, including the transverse speed and the longitudinal speed of the vehicle. The formed data set is a three-level vehicle track motion characteristic identification text data set, and a vehicle transverse speed and a vehicle longitudinal speed calculation formula is as follows:
Wherein, Representing the abscissa of the center point of the frame of the vehicle target of the t frame under the condition that the nth vehicle target of the t frame corresponds to the vehicle id serial number,/>Representing the abscissa of the center point of the vehicle target frame of the t-1 frame under the corresponding vehicle id serial number of the nth vehicle target frame of the t frame,/>Representing the ordinate of the center point of the frame of the vehicle target of the t frame under the condition that the nth vehicle target of the t frame corresponds to the vehicle id serial number,/>Representing the ordinate of the center point of the vehicle target frame of the nth vehicle target frame of the t frame corresponding to the t-1 frame under the vehicle id serial number, v t,n,x represents the transverse speed of the center point of the nth vehicle target frame of the t frame, v t,n,y represents the longitudinal speed of the center point of the nth vehicle target frame of the t frame,/>A video frame number representing the t-1 frame after data screening;
Since the speed calculation of each frame uses the position data of the previous frame, the vehicle speed of the first frame in each id vehicle frame sequence cannot be calculated, and therefore, a cubic polynomial is adopted for fitting, and the calculation formula is as follows:
where f 3(vx,2,vx,3,vx,4,vx,5) is a cubic function with respect to v x,2,vx,3,vx,4,vx,5, v x,1 is the first frame x-direction speed, f 3(vy,2,vy,3,vy,4,vy,5) is a cubic function with respect to v y,2,vy,3,vy,4,vy,5, and v y,1 is the first frame y-direction speed; v x,2,vx,3,vx,4,vx,5 is the speed of the 2 nd, 3 rd, 4 th and 5 th frames under different id vehicles respectively;
Secondly, calculating the vehicle acceleration of each vehicle id sequence number under each video frame sequence number, wherein the specific process is to calculate the acceleration by using the speed difference value and the time difference value of the current frame and the previous frame, wherein the acceleration comprises the vehicle transverse acceleration and the vehicle longitudinal acceleration, so that a three-level vehicle track motion characteristic recognition text data set is formed, and the calculation formulas of the vehicle transverse acceleration and the vehicle longitudinal acceleration are as follows:
Wherein, Represents the transverse speed of the center point of the nth frame of the vehicle corresponding to the vehicle id sequence number,/>Represents the transverse speed of the center point of the mth target frame of the (t-1) frame under the corresponding vehicle id sequence number of the nth vehicle target frame of the (t-1)Representing longitudinal speed of center point of nth frame of vehicle under corresponding vehicle id sequence numberThe longitudinal speed of the center point of the mth target frame of the (t-1) th frame of the (t) th frame of the (n) th frame of the (vehicle) corresponding to the vehicle id sequence number is represented, a t,n,x represents the longitudinal acceleration of the center point of the nth frame of the (t) th frame of the (vehicle), and a t,n,y represents the longitudinal acceleration of the center point of the nth frame of the (t) th frame of the (vehicle);
and the acceleration of the first frame of different vehicles is fitted by using a cubic polynomial under the same principle as the speed, and the calculation formula is as follows:
The formed three-level vehicle track motion characteristic recognition text data set is as follows:
the lane number detection in step 5 is as follows:
firstly, linear fitting is carried out on vehicle position coordinate data in a three-level vehicle track motion characteristic identification text data set to obtain a fitting straight line, and the expression of the fitting straight line is as follows:
Wherein, Representation of/>And/>A represents the slope of a straight line and B represents the intercept of the straight line;
Secondly, respectively calculating the distance from the vehicle position coordinate data in the three-level vehicle track motion characteristic recognition text data set to the fitting straight line, wherein the calculation formula is as follows:
judging the lane number by using a threshold value judging method to form a four-level vehicle track movement characteristic identification text data set, wherein the judging lane number formula is as follows:
{lanet,n＝k,if distk,1≤dist≤distk,2}
Wherein Lane t,n represents the lane number where the nth vehicle target frame center point of the t frame is located, k represents the determined lane number, dist k,1 represents the kth lane boundary lower limit, dist k,2 represents the kth lane boundary upper limit;
the formed four-level vehicle track motion characteristic recognition text data set is as follows:
the coordinates in step 5 are converted into: the pixel unit number covered in the road length direction is converted with the actual road length, a five-level vehicle track motion characteristic recognition text data set is formed after conversion, and the conversion proportion is as follows:
Wherein q is the size of the conversion proportion;
The conversion process of the four-level vehicle track movement characteristic identification text data set parameters is as follows:
Wherein, Represents the abscissa of the center point of the nth vehicle target frame of the nth frame after coordinate conversion,/>Representing the ordinate of the center point of the nth vehicle target frame of the nth frame after coordinate conversion,/>Representing the frame width of the nth vehicle target frame of the t frame after coordinate conversion,/>The method comprises the steps of representing the height of an nth vehicle target frame of a t frame after coordinate conversion, v t,n,x,q representing the transverse speed of a central point of the nth vehicle target frame of the t frame after coordinate conversion, v t,n,y,q representing the longitudinal speed of the central point of the nth vehicle target frame of the t frame after coordinate conversion, a t,n,x,q representing the transverse acceleration of the central point of the nth vehicle target frame of the t frame after coordinate conversion, and a t,n,y,q representing the longitudinal acceleration of the central point of the nth vehicle target frame of the t frame after coordinate conversion;
The formed five-stage vehicle track motion feature recognition text data set is as follows:
The invention has the advantages that: firstly, a new vehicle track characteristic recognition method is provided, which is different from the prior patent, the method of the invention uses YOLOv recognition models, uses Kalman filtering and Hungary algorithm based on uniform motion models, and provides a method for extracting vehicle speed, acceleration and lane number characteristics; the method of the invention overcomes the defects of the existing patent in the flow of the vehicle motion characteristic method, the applied Kalman filtering can slow down the omission of the relevant part of the vehicle target data, and the proposed speed, acceleration and lane number extraction method can effectively extract the vehicle motion characteristics.
Drawings
Fig. 1: is a schematic diagram of the device of the invention;
Fig. 2: the working scene diagram is the working scene diagram of the invention;
fig. 3: is a flow chart of the method of the invention;
fig. 4: extracting a test chart for the vehicle track of the method;
fig. 5: the method is a lane number detection test chart.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions in the present invention will be clearly and completely described below, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1, a schematic diagram of an apparatus according to the present invention is provided, in which the technical scheme of the apparatus is a high-altitude visual angle recognition system, and the apparatus is characterized by comprising:
The device comprises an aerial photographing device, a calculation processing host and a display projection device;
the aerial photographing device is connected with the computing processing host in a wireless mode; the computing processing host is connected with the display projection device in a wired mode;
The aerial photographing device is used for collecting video image data of the road vehicle under the high-altitude visual angle and transmitting the video image data to the calculation processing host in a wireless mode; the calculation processing host is used for processing the road vehicle video image data under the high-altitude visual angle acquired by the aerial photographing device, further obtaining a vehicle image recognition result and a track generation result through the vehicle track motion characteristic recognition method under the high-altitude visual angle, and transmitting the vehicle image recognition result and the track generation result to the display projection device for display;
the aerial photographing device is selected as follows: xingjiang DJI Mavic Air;
The computing processing host is configured to: i9-9900K model CPU; NVIDA GeForce RTX 3080 model number 3080 GPU; hua Shuo PRIME Z390-A model motherboard; DDR4 3000HZ 16G memory strips; GW-EPS1250DA type power supply;
The display screen is selected from the following types: AOC22B2H type display screen;
As shown in fig. 2, the aerial image capturing device is located right above the road surface during remote capturing, that is, the included angle between the line of sight captured by the camera of the aerial image capturing device and the road surface is 90 degrees.
As shown in fig. 3, the vehicle track motion feature recognition method is characterized by comprising the following steps:
Step 1: the calculation processing host computer is positioned right above the road pavement by using the aerial image capturing device in a wireless mode to capture video image data, and is used for forming a high-altitude image training data set, manually marking the high-altitude image training data set, marking the circumscribed rectangular frame of the vehicle target and the vehicle type, and forming a high-altitude image training vehicle marking frame set;
the high-altitude image training data set in the step1 is as follows:
{datae(x,y),e∈[1,E],x∈[1,X],y∈[1,Y]}
Wherein, data e (X, Y) represents the X-th row and Y-th column pixel information of the E-th frame image in the high-altitude image training data set, E is the total frame number of the high-altitude image training data set, X is the number of rows of the image in the high-altitude image training data set, and Y is the number of columns of the image in the high-altitude image training data set;
the high-altitude image training vehicle marking frame set in the step 1 is as follows:
Wherein, Representing the left upper-corner abscissa of the nth vehicle target mark rectangular frame in the e-th frame image in the high-altitude image training vehicle mark frame set,/>Representing the left upper corner ordinate of the nth vehicle target mark rectangular frame in the e-th frame image in the high-altitude image training vehicle mark frame set; /(I)Representing the right lower-corner abscissa of the rectangular frame of the n-th vehicle target mark in the e-th frame image in the high-altitude image training vehicle mark frame set,/>Representing the lower right corner ordinate of the nth vehicle target mark rectangular frame in the e-th frame image in the high-altitude image training vehicle mark frame set; type e,n represents the mark category of the nth vehicle object in the e-th frame image in the high-altitude image training vehicle mark frame set;
Step 2: the calculation processing host computer is positioned right above the road surface by using the aerial image device in a wireless mode to shoot video image data, and is used for forming an overhead image sequence data set for subsequently extracting vehicle track data; the road in the image picture of the high-altitude image sequence data set is in the middle position of the image.
Step 2, the fixed shooting frame rate of the aerial imaging device is FPS, the length of the shot road is l=152 meters, and the unit number of coverage pixels in the road length direction of the shot picture is g=3840; the high-altitude image data shooting sizes are x=3840 and y=2160;
the high-altitude image sequence data set in the step2 is as follows:
{datat(x,y),t∈[1,T],x∈[1,X],y∈[1,Y]}
Wherein, data t (x, Y) represents the x-th row and Y-th column pixel information of the T-th frame image in the high-altitude image sequence data set, T is the total frame number of the high-altitude image sequence data set, t=19200, x is the number of rows of the image in the high-altitude image sequence data set, and Y is the number of columns of the image in the high-altitude image sequence data set;
Step 3: introducing YOLOv deep learning network models, sequentially inputting each frame of image in the high-altitude image training data set and a vehicle marking frame corresponding to each frame of image in the high-altitude image training vehicle marking frame set into the YOLOv deep learning network models for training, constructing a loss function model by using a GIOU method, optimizing a loss function value by an Adam optimization algorithm, and identifying a vehicle target in the high-altitude image sequence data set by using the trained YOLOv deep learning network model to obtain a high-altitude image sequence vehicle identification frame set.
The YOLOv network framework in the step 3 is specifically a yolo5x network structure;
and 3, the high-altitude image sequence vehicle identification frame set is as follows:
Wherein, Representing the left upper-corner abscissa of the circumscribed rectangular frame of the nth vehicle object in the t frame image in the vehicle identification frame set of the high-altitude image sequence,/>Representing the left upper-corner ordinate of the circumscribed rectangular frame of the nth vehicle object in the t frame image in the high-altitude image sequence vehicle identification frame set; /(I)Representing the right lower-corner abscissa of the circumscribed rectangular frame of the nth vehicle object in the t frame image in the vehicle identification frame set of the high-altitude image sequence,/>Representing the lower right corner ordinate of the circumscribed rectangular frame of the nth vehicle object in the t frame image in the high-altitude image sequence vehicle identification frame set; type t,n represents the class of the nth vehicle object in the t-th frame image in the frame set of the high-altitude image sequence vehicle identification;
Step 4: starting from the first frame of vehicle target circumscribed rectangular frame data in the high-altitude image sequence vehicle target identification frame set, the following processing procedure is carried out: applying Kalman filtering to a vehicle target boundary frame of a previous frame to obtain vehicle target estimated frame data of a current frame, performing association matching on the vehicle target identified frame data of the current frame and the vehicle target boundary frame in the vehicle target estimated frame data by using a Hungary association algorithm, wherein a matching mechanism is an IOU distance to obtain an ID sequence number of the vehicle target identified frame data of the current frame, namely the ID sequence number of the vehicle target of the current frame, and marking a new ID sequence number on the incompletely matched vehicle target frame data of the current frame; until the high-altitude image sequence ends. And merging the video Frame sequence number, the vehicle ID sequence number and the high-altitude image sequence vehicle target Frame set after the association matching process to form an original vehicle track motion characteristic recognition text data set.
In the step 4, in the step of recording the current video frame number, the recorded video frame number set is as follows
Framet,n{framet,n}
Wherein frame t,n represents the video sequence number corresponding to the nth vehicle object of the nth frame.
The Kalman filtering processing process in the step4 sequentially comprises the following steps: initializing a vehicle target state vector; initializing a state transition matrix, initializing a covariance matrix, initializing an observation matrix and initializing a system noise matrix; predicting the current frame of vehicle target state vector according to the optimal estimated value of the previous frame of vehicle target state vector to obtain the predicted value of the current frame of vehicle target state vector; predicting a current frame vehicle target system error covariance matrix according to a previous frame vehicle target system error covariance matrix to obtain a current frame vehicle target system error covariance matrix predicted value; updating a Kalman coefficient by using a covariance matrix predicted value of a current frame vehicle target system; estimating according to the predicted value of the current frame vehicle target state vector and the system observation value to obtain the optimal estimated value of the current frame vehicle target state vector; updating the current frame of the vehicle target system error covariance matrix; extracting a current frame vehicle target estimation frame set from the current frame vehicle target state vector optimal estimation value;
in the process of initializing the vehicle target state vector by the kalman filter in the step 4, the vehicle target boundary frame characteristics are described by using the abscissa of the center of the vehicle target boundary frame, the ordinate of the center of the boundary frame, the area of the boundary frame and the aspect ratio of the boundary frame, and the motion state information of the boundary frame is described by adopting a linear uniform velocity model, namely:
Wherein, The motion state information of the boundary box is represented by u, the abscissa of the center of the boundary box, v, the ordinate of the center of the boundary box, s, the area of the boundary box, r, the transverse-longitudinal ratio of the boundary box, which is generally constant,/>Representing the rate of change of the center abscissa of the bounding box,/>Representing the ordinate of the centre of the bounding box,/>Indicating the area change rate of the bounding box. The motion state information of the mth vehicle target bounding box of the t-1 th frame is described as:
Wherein, Motion state information indicating the mth vehicle target boundary frame of the t-1 frame, u t-1,m indicating the abscissa of the center of the mth vehicle target boundary frame of the t-1 frame, v t-1,m indicating the ordinate of the center of the mth vehicle target boundary frame of the t-1 frame, s t-1,m indicating the area of the mth vehicle target boundary frame of the t-1 frame, r t-1,m indicating the ratio of the abscissa to the ordinate of the mth vehicle target boundary frame of the t-1 frame,/>Represents the central abscissa change rate of the mth vehicle target boundary frame of the t-1 frame,/>Ordinate representing the center of the mth vehicle target bounding box of the t-1 th frame,/>Representing the area change rate of the mth vehicle target boundary frame of the t-1 th frame;
The calculation formulas of the abscissa, the ordinate and the boundary frame area of the center of the mth vehicle target frame of the t-1 frame are as follows:
Wherein, Representing the upper left-hand abscissa of the target frame of the mth vehicle in the t-1 frame,/>Represents the lower right-hand corner abscissa of the frame of the mth vehicle target in the t-1 frame,/>Representing the upper left vertical coordinate of the frame of the mth vehicle target of the t-1 frame,/>Representing the lower right vertical coordinate of the frame of the mth vehicle target in the t-1 frame;
In the initializing state transition matrix in step 4, the state transition matrix F models the motion of the target state vector, and the state transition matrix F corresponding to the uniform motion model is initialized as follows:
/>
Initializing a covariance matrix, wherein the covariance matrix P represents uncertainty of target position information, and the covariance matrix is an experience parameter;
In the initialization system noise covariance matrix, because the process noise is not measurable, the system noise covariance matrix Q is generally assumed to accord with normal distribution;
in initializing the observation matrix, the observation matrix H is related to the observable variable, and its value is initialized to:
In the initialized observation noise covariance matrix, since the observation noise is not measurable, the observation noise covariance matrix R is generally assumed to accord with normal distribution;
and 4, predicting a current frame of vehicle target state vector according to the optimal estimated value of the last frame of vehicle target state vector by Kalman filtering, wherein the calculation formula of the obtained mth frame of vehicle target state vector predicted value is as follows:
Wherein, Represents the optimal estimated value of the mth vehicle target state vector of the t-1 frame,/>Representing the mth frame of the mth vehicle target state vector predicted value, wherein F is a state transition matrix, B is a control matrix, and u t-1,m is a control gain matrix;
and 4, predicting the current frame of the vehicle target system error covariance matrix according to the previous frame of the vehicle target system error covariance matrix by Kalman filtering, wherein the calculation formula of the obtained mth frame of the vehicle target system error covariance matrix is as follows:
Wherein P t-1,m represents the mth vehicle target systematic error covariance matrix of the t-1 frame, Representing the mth frame of the mth vehicle target system error covariance matrix predicted value, wherein Q is the covariance matrix of the process noise;
in the third step, the kalman filter updates the kalman coefficient by using the predicted value of the current frame system error covariance matrix, and the calculation formula of the mth vehicle target kalman coefficient in the t frame is as follows:
Wherein H is an observation matrix, R is a covariance matrix of observation noise, and K t,m is an mth frame mth vehicle target Kalman coefficient;
In the step 4, the kalman filtering calculates the optimal estimated value of the vehicle target state vector in the current frame according to the predicted value of the vehicle target state vector in the current frame and the observed value of the system, and the calculation formula of the optimal estimated value of the vehicle target state vector in the mth frame is as follows:
Wherein, The optimal estimated value of the target state vector of the mth vehicle in the t-th frame is z t, and the observed value is z t;
In the step 4, in the Kalman filtering updating the current frame system error covariance matrix, the mth frame mth vehicle target system error covariance matrix updating calculation formula is as follows:
wherein P t,m is the mth vehicle target system covariance matrix of the t frame;
And 4, extracting an optimal estimated value of the mth target state vector of the t-th frame from the optimal estimated value of the target state vector of the current frame in the vehicle target estimated frame set of the current frame, wherein the optimal estimated value of the mth target state vector of the t-th frame is described as follows:
Wherein u t,m represents the optimal estimated value of the abscissa of the center of the mth vehicle target boundary frame of the optimal estimation, v t,m represents the optimal estimated value of the ordinate of the center of the mth vehicle target boundary frame of the optimal estimation, s t,m represents the optimal estimated value of the area of the mth vehicle target boundary frame of the optimal estimation, r t,m represents the optimal estimated value of the aspect ratio of the mth vehicle target boundary frame of the optimal estimation, Optimal estimation value of central abscissa change rate of mth frame mth vehicle target boundary frame representing optimal estimation,/>Representing the optimal estimated ordinate rate of change of the center of the mth frame mth vehicle target bounding box,/>Representing the area change rate of the mth vehicle target boundary frame of the t frame of the optimal estimation;
the calculation formula of the current frame vehicle target estimated frame coordinates is as follows:
Wherein, The upper left-hand abscissa of the frame of the mth vehicle object representing the optimal estimate,/>T frame mth vehicle target frame upper left corner ordinate representing optimal estimation,/>The lower right-hand abscissa of the frame of the mth vehicle target of the t frame representing the optimal estimation,/>Representing the optimal estimated lower right-hand ordinate of the frame of the mth vehicle object,
Thus, the current frame vehicle target estimation bounding box set is:
preferably, the hungarian correlation algorithm in step 4 performs matching by calculating the intersection ratio of the target frame IOU of the vehicle;
And 4, calculating the intersection ratio matching of the vehicle target frame IOU by using the Hungary correlation algorithm as follows: calculating the IOU intersection ratio of the mth vehicle target estimation frame of the t frame in the vehicle target estimation frame set of the current frame and the nth vehicle target frame of the t frame in the vehicle target identification frame set of the current frame, wherein the intersection area calculation formula is as follows:
s 1 represents the intersection area of the mth frame of the vehicle target estimation frame set of the current frame and the nth frame of the vehicle target identification frame of the t frame of the vehicle target identification frame set of the current frame;
The combined calculation formula is as follows:
S 2 represents the combined area of the mth frame of the vehicle target estimation frame set of the current frame and the nth frame of the vehicle target identification frame set of the current frame;
the IOU cross ratio calculation formula is:
The Hungary correlation algorithm vehicle frame IOU intersection ratio matching principle is as follows: if the calculated IOU intersection ratio of the mth vehicle target estimated frame of the t frame and the nth vehicle target edge identification frame of the t frame is the largest and belongs to the same vehicle class, the mth vehicle target of the t-1 frame and the nth vehicle target of the t frame belong to the same vehicle target, and the ID serial number of the nth vehicle target of the t frame is marked as the same ID serial number as the mth vehicle target of the t-1 frame. The associated vehicle id sequence number set is:
IDt,n{idt,n}
Wherein id t,n represents the vehicle id number corresponding to the nth vehicle target in the t frame.
And step 4, merging the Frame serial number of the video Frame after the association process, the vehicle ID serial number and the high-altitude image sequence vehicle target Frame set to form an original vehicle track motion feature recognition text data set, wherein the formed original vehicle track motion feature recognition text data set is as follows:
as shown in fig. 4, a vehicle track extraction test chart is provided for an original vehicle track motion feature recognition text data set;
Step 5: and sequentially carrying out four processing processes of data preprocessing, motion feature extraction, lane number detection and coordinate conversion on the original vehicle track motion feature recognition text data set to finally form a five-level vehicle track motion feature recognition text data set.
And 5, preprocessing data, namely:
firstly, calculating the center point coordinates of a vehicle target frame, wherein the calculation formula of the center point coordinates is as follows:
Wherein, Represents the center point abscissa of the nth vehicle target recognition frame of the t frame,/>Representing the ordinate of the center point of the nth vehicle target recognition frame of the nth frame;
secondly, the width and the height of a vehicle target frame need to be calculated, and a calculation formula is as follows:
Where w t,n denotes the width of the nth vehicle target frame of the t-th frame, and h t,n denotes the height of the nth vehicle target frame of the t-th frame.
Forming a primary vehicle trajectory motion feature recognition text dataset:
When the first-level vehicle track motion feature recognition text data set is subjected to data preprocessing, the vehicle track motion feature recognition text data is screened out by a threshold value discrimination method to form a second-level vehicle track motion feature recognition text data set, and a discrimination formula is as follows:
Wherein, Represents the abscissa after threshold decision screening,/>X 1 represents a lower limit of the abscissa determination threshold, X 2 represents an upper limit of the abscissa determination threshold, Y 1 represents a lower limit of the ordinate determination threshold, and Y 2 represents an upper limit of the ordinate determination threshold;
secondly, counting the vehicle tracks with the same ID sequence number, if the number of the video frames is smaller than a fixed value, judging that the vehicle tracks are fragments of the track fragments, and cleaning the vehicle tracks, wherein a judgment formula is as follows:
Wherein, The number of video frames representing the value of the vehicle ID, threshold representing a fixed value;
the formed secondary vehicle track motion feature recognition text data set is as follows:
Wherein, Video frame number corresponding to vehicle target frame after data screening is represented,/>Vehicle id serial number corresponding to vehicle target frame after data screening is represented,/>Representing the width of the target frame of the vehicle after data screening,/>And representing the height of the target frame of the vehicle after data screening.
The motion characteristic extraction process in the step 5 is as follows:
The vehicle speed of each vehicle id sequence number under each video frame sequence number is calculated first, and the specific process is to calculate the speed of the current frame by using the position difference value and the time difference value of the current frame and the previous frame, including the transverse speed and the longitudinal speed of the vehicle. The formed data set is a three-level vehicle track motion characteristic identification text data set, and a vehicle transverse speed and a vehicle longitudinal speed calculation formula is as follows:
Wherein, Representing the abscissa of the center point of the frame of the vehicle target of the t frame under the condition that the nth vehicle target of the t frame corresponds to the vehicle id serial number,/>Representing the abscissa of the center point of the vehicle target frame of the t-1 frame under the corresponding vehicle id serial number of the nth vehicle target frame of the t frame,/>Representing the ordinate of the center point of the frame of the vehicle target of the t frame under the condition that the nth vehicle target of the t frame corresponds to the vehicle id serial number,/>Representing the ordinate of the center point of the vehicle target frame of the nth vehicle target frame of the t frame corresponding to the t-1 frame under the vehicle id serial number, v t,n,x represents the transverse speed of the center point of the nth vehicle target frame of the t frame, v t,n,y represents the longitudinal speed of the center point of the nth vehicle target frame of the t frame,/>A video frame number representing the t-1 frame after data screening;
Since the speed calculation of each frame uses the position data of the previous frame, the vehicle speed of the first frame in each id vehicle frame sequence cannot be calculated, and therefore, a cubic polynomial is adopted for fitting, and the calculation formula is as follows:
where f 3(vx,2,vx,3,vx,4,vx,5) is a cubic function with respect to v x,2,vx,3,vx,4,vx,5, v x,1 is the first frame x-direction speed, f 3(vy,2,vy,3,vy,4,vy,5) is a cubic function with respect to v y,2,vy,3,vy,4,vy,5, and v y,1 is the first frame y-direction speed; v x,2,vx,3,vx,4,vx,5 is the speed of the 2 nd, 3 rd, 4 th and 5 th frames under different id vehicles respectively;
Secondly, calculating the vehicle acceleration of each vehicle id sequence number under each video frame sequence number, wherein the specific process is to calculate the acceleration by using the speed difference value and the time difference value of the current frame and the previous frame, wherein the acceleration comprises the vehicle transverse acceleration and the vehicle longitudinal acceleration, so that a three-level vehicle track motion characteristic recognition text data set is formed, and the calculation formulas of the vehicle transverse acceleration and the vehicle longitudinal acceleration are as follows:
Wherein, Represents the transverse speed of the center point of the nth frame of the vehicle corresponding to the vehicle id sequence number,/>Represents the transverse speed of the center point of the mth target frame of the (t-1) frame under the corresponding vehicle id sequence number of the nth vehicle target frame of the (t-1)Representing longitudinal speed of center point of nth frame of vehicle under corresponding vehicle id sequence numberThe longitudinal speed of the center point of the mth target frame of the (t-1) th frame of the (t) th frame of the (n) th frame of the (vehicle) corresponding to the vehicle id sequence number is represented, a t,n,x represents the longitudinal acceleration of the center point of the nth frame of the (t) th frame of the (vehicle), and a t,n,y represents the longitudinal acceleration of the center point of the nth frame of the (t) th frame of the (vehicle);
and the acceleration of the first frame of different vehicles is fitted by using a cubic polynomial under the same principle as the speed, and the calculation formula is as follows:
The formed three-level vehicle track motion characteristic recognition text data set is as follows:
the lane number detection in step 5 is as follows:
firstly, linear fitting is carried out on vehicle position coordinate data in a three-level vehicle track motion characteristic identification text data set to obtain a fitting straight line, and the expression of the fitting straight line is as follows:
Wherein, Representation of/>And/>A represents the slope of a straight line, B represents the intercept of a straight line, and a= -0.008725, b=1189 is calculated;
Secondly, respectively calculating the distance from the vehicle position coordinate data in the three-level vehicle track motion characteristic recognition text data set to the fitting straight line, wherein the calculation formula is as follows:
judging the lane number by using a threshold value judging method to form a four-level vehicle track movement characteristic identification text data set, wherein the judging lane number formula is as follows:
{lanet,n＝k,if distk,1≤dist≤distk,2}
Wherein Lane t,n represents the lane number where the nth vehicle target frame center point of the t frame is located, k represents the determined lane number, dist k,1 represents the kth lane boundary lower limit, dist k,2 represents the kth lane boundary upper limit;
the formed four-level vehicle track motion characteristic recognition text data set is as follows:
As shown in fig. 5, a lane number detection test chart of a text data set is identified for four-level vehicle track movement characteristics, wherein continuous straight lines in the chart represent detected lane interval lines;
the coordinates in step 5 are converted into: the pixel unit number covered in the road length direction is converted with the actual road length, a five-level vehicle track motion characteristic recognition text data set is formed after conversion, and the conversion proportion is as follows:
wherein q is the size of the conversion proportion, and q= 0.03958 is calculated;
The conversion process of the four-level vehicle track movement characteristic identification text data set parameters is as follows:
Wherein, Represents the abscissa of the center point of the nth vehicle target frame of the nth frame after coordinate conversion,/>Representing the ordinate of the center point of the nth vehicle target frame of the nth frame after coordinate conversion,/>Representing the frame width of the nth vehicle target frame of the t frame after coordinate conversion,/>The method comprises the steps of representing the height of an nth vehicle target frame of a t frame after coordinate conversion, v t,n,x,q representing the transverse speed of a central point of the nth vehicle target frame of the t frame after coordinate conversion, v t,n,y,q representing the longitudinal speed of the central point of the nth vehicle target frame of the t frame after coordinate conversion, a t,n,x,q representing the transverse acceleration of the central point of the nth vehicle target frame of the t frame after coordinate conversion, and a t,n,y,q representing the longitudinal acceleration of the central point of the nth vehicle target frame of the t frame after coordinate conversion;
The formed five-stage vehicle track motion feature recognition text data set is as follows:
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (5)

1. A vehicle track motion feature recognition method based on a high-altitude visual angle recognition system is characterized by comprising the following steps of:
The high-altitude visual angle recognition system is characterized by comprising: the device comprises an aerial photographing device, a calculation processing host and a display projection device;
the aerial photographing device is connected with the computing processing host in a wireless mode; the computing processing host is connected with the display projection device in a wired mode;
The aerial photographing device is used for collecting video image data of the road vehicle under the high-altitude visual angle and transmitting the video image data to the calculation processing host in a wireless mode; the calculation processing host is used for processing the road vehicle video image data under the high-altitude visual angle acquired by the aerial photographing device, further obtaining a vehicle image recognition result and a track generation result through the vehicle track motion characteristic recognition method under the high-altitude visual angle, and transmitting the vehicle image recognition result and the track generation result to the display projection device for display;
The aerial photographing device is positioned right above the road surface during remote photographing, namely, the included angle between the photographing line of sight of the aerial photographing device and the road surface is 90 degrees;
The vehicle track motion feature recognition method comprises the following steps:
Step 1: the calculation processing host computer is positioned right above the road pavement by using the aerial image capturing device in a wireless mode to capture video image data, and is used for forming a high-altitude image training data set, manually marking the high-altitude image training data set, marking the circumscribed rectangular frame of the vehicle target and the vehicle type, and forming a high-altitude image training vehicle marking frame set;
step 2: the calculation processing host computer is positioned right above the road surface by using the aerial image device in a wireless mode to shoot video image data, and is used for forming an overhead image sequence data set for subsequently extracting vehicle track data; the road in the image picture of the high-altitude image sequence data set is positioned at the middle position of the image;
Step 3: introducing YOLOv deep learning network models, sequentially inputting each frame of image in a high-altitude image training dataset and a vehicle marking frame corresponding to each frame of image in a high-altitude image training vehicle marking frame set into the YOLOv deep learning network models for training, constructing a loss function model by using a GIOU method, optimizing a loss function value by an Adam optimization algorithm, and identifying a vehicle target in the high-altitude image sequence dataset by using the trained YOLOv deep learning network model to obtain a high-altitude image sequence vehicle identification frame set;
Step 4: starting from the first frame of vehicle target circumscribed rectangular frame data in the high-altitude image sequence vehicle target identification frame set, the following processing procedure is carried out: applying Kalman filtering to a vehicle target boundary frame of a previous frame to obtain vehicle target estimated frame data of a current frame, performing association matching on the vehicle target identified frame data of the current frame and the vehicle target boundary frame in the vehicle target estimated frame data by using a Hungary association algorithm, wherein a matching mechanism is an IOU distance to obtain an ID sequence number of the vehicle target identified frame data of the current frame, namely the ID sequence number of the vehicle target of the current frame, and marking a new ID sequence number on the incompletely matched vehicle target frame data of the current frame; until the high-altitude image sequence is finished; combining the video Frame sequence number, the vehicle ID sequence number and the high-altitude image sequence vehicle target Frame set after the association matching process to form an original vehicle track motion characteristic identification text data set;
Step 5: sequentially performing four processing processes of data preprocessing, motion feature extraction, lane number detection and coordinate conversion on an original vehicle track motion feature recognition text data set to finally form a five-level vehicle track motion feature recognition text data set;
and 5, preprocessing data, namely:
firstly, calculating the center point coordinates of a vehicle target frame, wherein the calculation formula of the center point coordinates is as follows:
Wherein, Represents the center point abscissa of the nth vehicle target recognition frame of the t frame,/>Representing the ordinate of the center point of the nth vehicle target recognition frame of the nth frame;
secondly, the width and the height of a vehicle target frame need to be calculated, and a calculation formula is as follows:
Wherein w t,n represents the width of the nth vehicle target frame of the t frame, and h t,n represents the height of the nth vehicle target frame of the t frame;
forming a primary vehicle trajectory motion feature recognition text dataset:
When the first-level vehicle track motion feature recognition text data set is subjected to data preprocessing, the vehicle track motion feature recognition text data is screened out by a threshold value discrimination method to form a second-level vehicle track motion feature recognition text data set, and a discrimination formula is as follows:
Wherein, Represents the abscissa after threshold decision screening,/>X 1 represents a lower limit of the abscissa determination threshold, X 2 represents an upper limit of the abscissa determination threshold, Y 1 represents a lower limit of the ordinate determination threshold, and Y 2 represents an upper limit of the ordinate determination threshold;
secondly, counting the vehicle tracks with the same ID sequence number, if the number of the video frames is smaller than a fixed value, judging that the vehicle tracks are fragments of the track fragments, and cleaning the vehicle tracks, wherein a judgment formula is as follows:
Wherein, The number of video frames representing the value of the vehicle ID, threshold representing a fixed value;
the formed secondary vehicle track motion feature recognition text data set is as follows:
Wherein, Video frame number corresponding to vehicle target frame after data screening is represented,/>Vehicle id serial number corresponding to vehicle target frame after data screening is represented,/>Representing the width of the target frame of the vehicle after data screening,/>Representing the height of a vehicle target frame after data screening;
the motion characteristic extraction process in the step 5 is as follows:
firstly, calculating the speed of the vehicle under each video frame sequence number of each vehicle id sequence number, wherein the specific process is to calculate the speed of the current frame by using the position difference value and the time difference value of the current frame and the previous frame, and the speed comprises the transverse speed and the longitudinal speed of the vehicle; the formed data set is a three-level vehicle track motion characteristic identification text data set, and a vehicle transverse speed and a vehicle longitudinal speed calculation formula is as follows:
Wherein, Representing the abscissa of the center point of the frame of the vehicle target of the t frame under the condition that the nth vehicle target of the t frame corresponds to the vehicle id serial number,/>Representing the abscissa of the center point of the vehicle target frame of the t-1 frame under the corresponding vehicle id serial number of the nth vehicle target frame of the t frame,/>Representing the ordinate of the center point of the frame of the vehicle target of the t frame under the condition that the nth vehicle target of the t frame corresponds to the vehicle id serial number,/>Representing the ordinate of the center point of the vehicle target frame of the nth vehicle target frame of the t frame corresponding to the t-1 frame under the vehicle id serial number, v t,n,x represents the transverse speed of the center point of the nth vehicle target frame of the t frame, v t,n,y represents the longitudinal speed of the center point of the nth vehicle target frame of the t frame,/>A video frame number representing the t-1 frame after data screening;
Since the speed calculation of each frame uses the position data of the previous frame, the vehicle speed of the first frame in each id vehicle frame sequence cannot be calculated, and therefore, a cubic polynomial is adopted for fitting, and the calculation formula is as follows:
where f 3(vx,2,vx,3,vx,4,vx,5) is a cubic function with respect to v x,2,vx,3,vx,4,vx,5, v x,1 is the first frame x-direction speed, f 3(vy,2,vy,3,vy,4,vy,5) is a cubic function with respect to v y,2,vy,3,vy,4,vy,5, and v y,1 is the first frame y-direction speed; v x,2,vx,3,vx,4,vx,5 is the speed of the 2 nd, 3 rd, 4 th and 5 th frames under different id vehicles respectively;
Secondly, calculating the vehicle acceleration of each vehicle id sequence number under each video frame sequence number, wherein the specific process is to calculate the acceleration by using the speed difference value and the time difference value of the current frame and the previous frame, wherein the acceleration comprises the vehicle transverse acceleration and the vehicle longitudinal acceleration, so that a three-level vehicle track motion characteristic recognition text data set is formed, and the calculation formulas of the vehicle transverse acceleration and the vehicle longitudinal acceleration are as follows:
Wherein, Represents the transverse speed of the center point of the nth frame of the vehicle corresponding to the vehicle id sequence number,/>Represents the transverse speed of the center point of the mth target frame of the (t-1) frame under the corresponding vehicle id sequence number of the nth vehicle target frame of the (t-1)Representing longitudinal speed of center point of nth frame of vehicle under corresponding vehicle id sequence numberThe longitudinal speed of the center point of the mth target frame of the (t-1) th frame of the (t) th frame of the (n) th frame of the (vehicle) corresponding to the vehicle id sequence number is represented, a t,n,x represents the longitudinal acceleration of the center point of the nth frame of the (t) th frame of the (vehicle), and a t,n,y represents the longitudinal acceleration of the center point of the nth frame of the (t) th frame of the (vehicle);
and the acceleration of the first frame of different vehicles is fitted by using a cubic polynomial under the same principle as the speed, and the calculation formula is as follows:
The formed three-level vehicle track motion characteristic recognition text data set is as follows:
the lane number detection in step 5 is as follows:
firstly, linear fitting is carried out on vehicle position coordinate data in a three-level vehicle track motion characteristic identification text data set to obtain a fitting straight line, and the expression of the fitting straight line is as follows:
Wherein, Representation of/>And/>A represents the slope of a straight line and B represents the intercept of the straight line;
Secondly, respectively calculating the distance from the vehicle position coordinate data in the three-level vehicle track motion characteristic recognition text data set to the fitting straight line, wherein the calculation formula is as follows:
judging the lane number by using a threshold value judging method to form a four-level vehicle track movement characteristic identification text data set, wherein the judging lane number formula is as follows:
{lanet,n＝k,if distk,1≤dist≤distk,2}
Wherein Lane t,n represents the lane number where the nth vehicle target frame center point of the t frame is located, k represents the determined lane number, dist k,1 represents the kth lane boundary lower limit, dist k,2 represents the kth lane boundary upper limit;
the formed four-level vehicle track motion characteristic recognition text data set is as follows:
the coordinates in step 5 are converted into: the pixel unit number covered in the road length direction is converted with the actual road length, a five-level vehicle track motion characteristic recognition text data set is formed after conversion, and the conversion proportion is as follows:
Wherein q is the size of the conversion proportion;
The conversion process of the four-level vehicle track movement characteristic identification text data set parameters is as follows:
Wherein, Represents the abscissa of the center point of the nth vehicle target frame of the nth frame after coordinate conversion,/>Representing the ordinate of the center point of the nth vehicle target frame of the nth frame after coordinate conversion,/>Representing the frame width of the nth vehicle target frame of the t frame after coordinate conversion,/>The method comprises the steps of representing the height of an nth vehicle target frame of a t frame after coordinate conversion, v t,n,x,q representing the transverse speed of a central point of the nth vehicle target frame of the t frame after coordinate conversion, v t,n,y,q representing the longitudinal speed of the central point of the nth vehicle target frame of the t frame after coordinate conversion, a t,n,x,q representing the transverse acceleration of the central point of the nth vehicle target frame of the t frame after coordinate conversion, and a t,n,y,q representing the longitudinal acceleration of the central point of the nth vehicle target frame of the t frame after coordinate conversion;
The formed five-stage vehicle track motion feature recognition text data set is as follows:
2. the high-altitude visual recognition system-based vehicle trajectory motion feature recognition method according to claim 1, wherein: the high-altitude image training data set in the step 1 is as follows:
{datae(x,y),e∈[1,E],x∈[1,X],y∈[1,Y]}
Wherein, data e (X, Y) represents the X-th row and Y-th column pixel information of the E-th frame image in the high-altitude image training data set, E is the total frame number of the high-altitude image training data set, X is the number of rows of the image in the high-altitude image training data set, and Y is the number of columns of the image in the high-altitude image training data set;
the high-altitude image training vehicle marking frame set in the step 1 is as follows:
Wherein, Representing the left upper-corner abscissa of the nth vehicle target mark rectangular frame in the e-th frame image in the high-altitude image training vehicle mark frame set,/>Representing the left upper corner ordinate of the nth vehicle target mark rectangular frame in the e-th frame image in the high-altitude image training vehicle mark frame set; /(I)Representing the right lower-corner abscissa of the rectangular frame of the n-th vehicle target mark in the e-th frame image in the high-altitude image training vehicle mark frame set,/>Representing the lower right corner ordinate of the nth vehicle target mark rectangular frame in the e-th frame image in the high-altitude image training vehicle mark frame set; type e,n represents the tag class of the nth vehicle object in the e-th frame image in the high-altitude image training vehicle tag bezel set.
3. The high-altitude visual recognition system-based vehicle trajectory motion feature recognition method according to claim 1, wherein: step 2, the fixed shooting frame rate of the aerial imaging device is FPS, the length of a shot road is L, and the number of coverage pixel units in the length direction of the shot image road is G; the shooting size of the high-altitude image data is X and Y;
the high-altitude image sequence data set in the step2 is as follows:
{datat(x,y),t∈[1,T],x∈[1,X],y∈[1,Y]}
Wherein, data t (X, Y) represents the X-th row and Y-th column pixel information of the T-th frame image in the high-altitude image sequence data set, T is the total frame number of the high-altitude image sequence data set, X is the number of rows of the image in the high-altitude image sequence data set, and Y is the number of columns of the image in the high-altitude image sequence data set.
4. The high-altitude visual recognition system-based vehicle trajectory motion feature recognition method according to claim 1, wherein: the YOLOv network framework in the step 3 is specifically a yolo5x network structure;
and 3, the high-altitude image sequence vehicle identification frame set is as follows:
Wherein, Representing the left upper-corner abscissa of the circumscribed rectangular frame of the nth vehicle object in the t frame image in the vehicle identification frame set of the high-altitude image sequence,/>Representing the left upper-corner ordinate of the circumscribed rectangular frame of the nth vehicle object in the t frame image in the high-altitude image sequence vehicle identification frame set; /(I)Representing the right lower-corner abscissa of the circumscribed rectangular frame of the nth vehicle object in the t frame image in the vehicle identification frame set of the high-altitude image sequence,/>Representing the lower right corner ordinate of the circumscribed rectangular frame of the nth vehicle object in the t frame image in the high-altitude image sequence vehicle identification frame set; type t,n represents the class of the nth vehicle object in the t-th frame image in the high-altitude image sequence vehicle identification frame set.
5. The high-altitude visual recognition system-based vehicle trajectory motion feature recognition method according to claim 1, wherein: recording the current video Frame number, wherein the recorded video Frame number set is Frame t,n{framet,n }, and
Wherein frame t,n represents the video sequence number corresponding to the nth vehicle object of the nth frame;
The Kalman filtering processing process in the step4 sequentially comprises the following steps: initializing a vehicle target state vector; initializing a state transition matrix, initializing a covariance matrix, initializing an observation matrix and initializing a system noise matrix; predicting the current frame of vehicle target state vector according to the optimal estimated value of the previous frame of vehicle target state vector to obtain the predicted value of the current frame of vehicle target state vector; predicting a current frame vehicle target system error covariance matrix according to a previous frame vehicle target system error covariance matrix to obtain a current frame vehicle target system error covariance matrix predicted value; updating a Kalman coefficient by using a covariance matrix predicted value of a current frame vehicle target system; estimating according to the predicted value of the current frame vehicle target state vector and the system observation value to obtain the optimal estimated value of the current frame vehicle target state vector; updating the current frame of the vehicle target system error covariance matrix; extracting a current frame vehicle target estimation frame set from the current frame vehicle target state vector optimal estimation value;
in the process of initializing the vehicle target state vector by the kalman filter in the step 4, the vehicle target boundary frame characteristics are described by using the abscissa of the center of the vehicle target boundary frame, the ordinate of the center of the boundary frame, the area of the boundary frame and the aspect ratio of the boundary frame, and the motion state information of the boundary frame is described by adopting a linear uniform velocity model, namely:
Wherein, The motion state information of the boundary box is represented by u, the abscissa of the center of the boundary box, v, the ordinate of the center of the boundary box, s, the area of the boundary box, r, the transverse-longitudinal ratio of the boundary box, which is generally constant,/>Representing the rate of change of the center abscissa of the bounding box,/>Representing the ordinate of the centre of the bounding box,/>Representing the area change rate of the bounding box; the motion state information of the mth vehicle target bounding box of the t-1 th frame is described as:
Wherein, Motion state information indicating the mth vehicle target boundary frame of the t-1 frame, u t-1,m indicating the abscissa of the center of the mth vehicle target boundary frame of the t-1 frame, v t-1,m indicating the ordinate of the center of the mth vehicle target boundary frame of the t-1 frame, s t-1,m indicating the area of the mth vehicle target boundary frame of the t-1 frame, r t-1,m indicating the ratio of the abscissa to the ordinate of the mth vehicle target boundary frame of the t-1 frame,/>Represents the central abscissa change rate of the mth vehicle target boundary frame of the t-1 frame,/>Ordinate representing the center of the mth vehicle target bounding box of the t-1 th frame,/>Representing the area change rate of the mth vehicle target boundary frame of the t-1 th frame;
The calculation formulas of the abscissa, the ordinate and the boundary frame area of the center of the mth vehicle target frame of the t-1 frame are as follows:
Wherein, Representing the upper left-hand abscissa of the target frame of the mth vehicle in the t-1 frame,/>Represents the lower right-hand corner abscissa of the frame of the mth vehicle target in the t-1 frame,/>Representing the upper left vertical coordinate of the frame of the mth vehicle target of the t-1 frame,/>Representing the lower right vertical coordinate of the frame of the mth vehicle target in the t-1 frame;
In the initializing state transition matrix in step 4, the state transition matrix F models the motion of the target state vector, and the state transition matrix F corresponding to the uniform motion model is initialized as follows:
Initializing a covariance matrix, wherein the covariance matrix P represents uncertainty of target position information, and the covariance matrix is an experience parameter;
In the initialization system noise covariance matrix, because the process noise is not measurable, the system noise covariance matrix Q is generally assumed to accord with normal distribution;
in initializing the observation matrix, the observation matrix H is related to the observable variable, and its value is initialized to:
In the initialized observation noise covariance matrix, since the observation noise is not measurable, the observation noise covariance matrix R is generally assumed to accord with normal distribution;
and 4, predicting a current frame of vehicle target state vector according to the optimal estimated value of the last frame of vehicle target state vector by Kalman filtering, wherein the calculation formula of the obtained mth frame of vehicle target state vector predicted value is as follows:
Wherein, Represents the optimal estimated value of the mth vehicle target state vector of the t-1 frame,/>Representing the mth frame of the mth vehicle target state vector predicted value, wherein F is a state transition matrix, B is a control matrix, and u t-1,m is a control gain matrix;
and 4, predicting the current frame of the vehicle target system error covariance matrix according to the previous frame of the vehicle target system error covariance matrix by Kalman filtering, wherein the calculation formula of the obtained mth frame of the vehicle target system error covariance matrix is as follows:
Wherein P t-1,m represents the mth vehicle target systematic error covariance matrix of the t-1 frame, Representing the mth frame of the mth vehicle target system error covariance matrix predicted value, wherein Q is the covariance matrix of the process noise;
in the third step, the kalman filter updates the kalman coefficient by using the predicted value of the current frame system error covariance matrix, and the calculation formula of the mth vehicle target kalman coefficient in the t frame is as follows:
Wherein H is an observation matrix, R is a covariance matrix of observation noise, and K t,m is an mth frame mth vehicle target Kalman coefficient;
In the step 4, the kalman filtering calculates the optimal estimated value of the vehicle target state vector in the current frame according to the predicted value of the vehicle target state vector in the current frame and the observed value of the system, and the calculation formula of the optimal estimated value of the vehicle target state vector in the mth frame is as follows:
Wherein, The optimal estimated value of the target state vector of the mth vehicle in the t-th frame is z t, and the observed value is z t;
In the step 4, in the Kalman filtering updating the current frame system error covariance matrix, the mth frame mth vehicle target system error covariance matrix updating calculation formula is as follows:
wherein P t,m is the mth vehicle target system covariance matrix of the t frame;
And 4, extracting an optimal estimated value of the mth target state vector of the t-th frame from the optimal estimated value of the target state vector of the current frame in the vehicle target estimated frame set of the current frame, wherein the optimal estimated value of the mth target state vector of the t-th frame is described as follows:
Wherein u t,m represents the optimal estimated value of the abscissa of the center of the mth vehicle target boundary frame of the optimal estimation, v t,m represents the optimal estimated value of the ordinate of the center of the mth vehicle target boundary frame of the optimal estimation, s t,m represents the optimal estimated value of the area of the mth vehicle target boundary frame of the optimal estimation, r t,m represents the optimal estimated value of the aspect ratio of the mth vehicle target boundary frame of the optimal estimation, Optimal estimation value of central abscissa change rate of mth frame mth vehicle target boundary frame representing optimal estimation,/>Representing the optimal estimated ordinate rate of change of the center of the mth frame mth vehicle target bounding box,/>Representing the area change rate of the mth vehicle target boundary frame of the t frame of the optimal estimation;
the calculation formula of the current frame vehicle target estimated frame coordinates is as follows:
Wherein, The upper left-hand abscissa of the frame of the mth vehicle object representing the optimal estimate,/>T frame mth vehicle target frame upper left corner ordinate representing optimal estimation,/>The lower right-hand abscissa of the frame of the mth vehicle target of the t frame representing the optimal estimation,/>Representing the vertical coordinate of the lower right corner of the mth frame and the mth vehicle target frame of the optimal estimation;
thus, the current frame vehicle target estimation bounding box set is:
step 4, matching the Hungary correlation algorithm by calculating the vehicle target frame IOU intersection ratio;
And 4, calculating the intersection ratio matching of the vehicle target frame IOU by using the Hungary correlation algorithm as follows: calculating the IOU intersection ratio of the mth vehicle target estimation frame of the t frame in the vehicle target estimation frame set of the current frame and the nth vehicle target frame of the t frame in the vehicle target identification frame set of the current frame, wherein the intersection area calculation formula is as follows:
s 1 represents the intersection area of the mth frame of the vehicle target estimation frame set of the current frame and the nth frame of the vehicle target identification frame of the t frame of the vehicle target identification frame set of the current frame;
The combined calculation formula is as follows:
S 2 represents the combined area of the mth frame of the vehicle target estimation frame set of the current frame and the nth frame of the vehicle target identification frame set of the current frame;
the IOU cross ratio calculation formula is:
The Hungary correlation algorithm vehicle frame IOU intersection ratio matching principle is as follows: if the calculated IOU intersection ratio of the mth vehicle target estimated frame of the t frame and the nth vehicle target edge identification frame of the t frame is the largest and belongs to the same vehicle class, the mth vehicle target of the t-1 frame and the nth vehicle target of the t frame belong to the same vehicle target, and the ID serial number of the nth vehicle target of the t frame is marked as the same ID serial number as the mth vehicle target of the t-1 frame; the associated vehicle id sequence number set is:
IDt,n{idt,n}
Wherein id t,n represents a vehicle id number corresponding to the nth vehicle target in the t frame;
combining the Frame sequence number of the video Frame after the association process, the vehicle ID sequence number and the high-altitude image sequence vehicle target Frame set to form an original vehicle track motion characteristic identification text data set, wherein the formed original vehicle track motion characteristic identification text data set is as follows:
CN202111201539.XA 2021-10-15 2021-10-15 Vehicle track motion feature recognition method based on high-altitude visual angle recognition system Active CN114022791B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111201539.XA CN114022791B (en) 2021-10-15 2021-10-15 Vehicle track motion feature recognition method based on high-altitude visual angle recognition system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111201539.XA CN114022791B (en) 2021-10-15 2021-10-15 Vehicle track motion feature recognition method based on high-altitude visual angle recognition system

Publications (2)

Publication Number Publication Date
CN114022791A CN114022791A (en) 2022-02-08
CN114022791B true CN114022791B (en) 2024-05-28

Family

ID=80056377

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111201539.XA Active CN114022791B (en) 2021-10-15 2021-10-15 Vehicle track motion feature recognition method based on high-altitude visual angle recognition system

Country Status (1)

Country Link
CN (1) CN114022791B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20070105100A (en) * 2006-04-25 2007-10-30 유일정보시스템(주) System for tracking car objects using mosaic video image and a method thereof
WO2020000251A1 (en) * 2018-06-27 2020-01-02 潍坊学院 Method for identifying video involving violation at intersection based on coordinated relay of video cameras
CN112329569A (en) * 2020-10-27 2021-02-05 武汉理工大学 Freight vehicle state real-time identification method based on image deep learning system
CN112884816A (en) * 2021-03-23 2021-06-01 武汉理工大学 Vehicle feature deep learning recognition track tracking method based on image system
CN113269098A (en) * 2021-05-27 2021-08-17 中国人民解放军军事科学院国防科技创新研究院 Multi-target tracking positioning and motion state estimation method based on unmanned aerial vehicle

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10618673B2 (en) * 2016-04-15 2020-04-14 Massachusetts Institute Of Technology Systems and methods for dynamic planning and operation of autonomous systems using image observation and information theory
JP6770393B2 (en) * 2016-10-04 2020-10-14 株式会社豊田中央研究所 Tracking device and program

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20070105100A (en) * 2006-04-25 2007-10-30 유일정보시스템(주) System for tracking car objects using mosaic video image and a method thereof
WO2020000251A1 (en) * 2018-06-27 2020-01-02 潍坊学院 Method for identifying video involving violation at intersection based on coordinated relay of video cameras
CN112329569A (en) * 2020-10-27 2021-02-05 武汉理工大学 Freight vehicle state real-time identification method based on image deep learning system
CN112884816A (en) * 2021-03-23 2021-06-01 武汉理工大学 Vehicle feature deep learning recognition track tracking method based on image system
CN113269098A (en) * 2021-05-27 2021-08-17 中国人民解放军军事科学院国防科技创新研究院 Multi-target tracking positioning and motion state estimation method based on unmanned aerial vehicle

Also Published As

Publication number Publication date
CN114022791A (en) 2022-02-08

Similar Documents

Publication Publication Date Title
CN110348445B (en) Instance segmentation method fusing void convolution and edge information
CN109977812B (en) Vehicle-mounted video target detection method based on deep learning
CN104282020B (en) A kind of vehicle speed detection method based on target trajectory
CN106910203B (en) The quick determination method of moving target in a kind of video surveillance
CN113160192B (en) Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background
CN103164706B (en) Object counting method and device based on video signal analysis
CN104239867B (en) License plate locating method and system
CN108053419A (en) Inhibited and the jamproof multiscale target tracking of prospect based on background
CN111814621A (en) Multi-scale vehicle and pedestrian detection method and device based on attention mechanism
CN105279772B (en) A kind of trackability method of discrimination of infrared sequence image
CN114023062B (en) Traffic flow information monitoring method based on deep learning and edge calculation
CN105869178A (en) Method for unsupervised segmentation of complex targets from dynamic scene based on multi-scale combination feature convex optimization
CN101094413A (en) Real time movement detection method in use for video monitoring
CN112884816B (en) Vehicle feature deep learning recognition track tracking method based on image system
CN105512618B (en) Video tracing method
CN104978567A (en) Vehicle detection method based on scenario classification
CN113111722A (en) Automatic driving target identification method based on improved Mask R-CNN
CN104992429A (en) Mountain crack detection method based on image local reinforcement
CN109684986B (en) Vehicle analysis method and system based on vehicle detection and tracking
CN111666860A (en) Vehicle track tracking method integrating license plate information and vehicle characteristics
CN110555377A (en) pedestrian detection and tracking method based on fisheye camera overlook shooting
CN107808524A (en) A kind of intersection vehicle checking method based on unmanned plane
CN115359407A (en) Multi-vehicle tracking method in video
CN113516853B (en) Multi-lane traffic flow detection method for complex monitoring scene
CN110443142A (en) A kind of deep learning vehicle count method extracted based on road surface with segmentation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant