CN112967316B - Motion compensation optimization method and system for 3D multi-target tracking - Google Patents

Motion compensation optimization method and system for 3D multi-target tracking Download PDF

Info

Publication number
CN112967316B
CN112967316B CN202110245922.9A CN202110245922A CN112967316B CN 112967316 B CN112967316 B CN 112967316B CN 202110245922 A CN202110245922 A CN 202110245922A CN 112967316 B CN112967316 B CN 112967316B
Authority
CN
China
Prior art keywords
frame
motion
coordinate system
motion compensation
local coordinate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110245922.9A
Other languages
Chinese (zh)
Other versions
CN112967316A (en
Inventor
张昱
王顺洪
张燕咏
吉建民
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Science and Technology of China USTC
Original Assignee
University of Science and Technology of China USTC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Science and Technology of China USTC filed Critical University of Science and Technology of China USTC
Priority to CN202110245922.9A priority Critical patent/CN112967316B/en
Publication of CN112967316A publication Critical patent/CN112967316A/en
Application granted granted Critical
Publication of CN112967316B publication Critical patent/CN112967316B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/003—Navigation within 3D models or images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Abstract

The invention relates to a motion compensation optimization method and a system facing 3D multi-target tracking, wherein the method comprises the following steps: step S1: respectively obtaining the data (theta, v) of the local coordinate systems of the observed point object in the t frame and the t +1 frame f ) (ii) a Step S2: calculating the heading angle difference delta theta of the object in the t frame and the t +1 frame, and according to the advancing speed of the object in the t frame and the t +1 frame
Figure DDA0002964077410000011
And
Figure DDA0002964077410000012
calculating the position of the object after motion compensation at the time of the t frame and under the local coordinate system of the t +1 frame
Figure DDA0002964077410000013
Step S3: according to the position of the object after motion compensation
Figure DDA0002964077410000016
Combining with motion prediction, updating the position of the object at the moment of the t +1 th frame and in the local coordinate system of the t +1 th frame
Figure DDA0002964077410000017
Step S4: obtaining an object surrounding frame of the t +1 th frame according to detection
Figure DDA0002964077410000014
And the predicted position of the object movement
Figure DDA0002964077410000015
Carrying out data association to determine the objectEnclosing the final position of the frame.

Description

Motion compensation optimization method and system for 3D multi-target tracking
Technical Field
The invention relates to the fields of automatic driving and 3D multi-target tracking, in particular to a motion compensation optimization method and system for 3D multi-target tracking.
Background
Three-dimensional multi-object tracking (3D MOT) is a key module in an autopilot system. Three-dimensional multi-target tracking needs to associate an object in a current frame with an object in a historical frame, and the position of the same object appearing in each frame of data forms the motion track of the object.
The mainstream model for solving the problem of three-dimensional multi-target tracking is detection and tracking (tracking by detection). Under the model, each frame of data processing is divided into two steps of detection and association, wherein an object existing in a current frame is detected firstly, and then the detected object is associated with an object detected in a historical frame. The detection step corresponds to a three-dimensional object detection problem, while the correlation step is typically modeled as a maximum bipartite graph matching problem. The object historical track set and the object current position set form two vertex sets of a bipartite graph, and the weight of each edge in the bipartite graph is defined as the affinity of the object track and the position, namely the similarity of the historical object characteristics and the current object characteristics. The hungarian algorithm is the most common method to solve this problem.
The motion of an object follows a certain physical law, and a motion model is usually adopted to match with a recursive filter to realize accurate estimation of a motion state. In the field of automatic driving, the technology is widely applied to the problems of estimation of the position and the posture of a traffic participant, motion prediction, intention understanding and the like.
Common motion models include Constant Velocity (CV), Constant Acceleration (CA), constant rotation rate (CTRV), constant rotation rate (CTRA), and constant acceleration (CTRA).
CV and CA belong to a linear model, and are matched with a linear recursive filter, such as a Kalman filter, to estimate the motion state. In the process, the prediction and the update of the motion state are linear transformation, and the calculation is simple and direct. CV and CA models assume that an object always moves linearly, and cannot be well estimated for the case of object rotation.
The CTRV and CTRA consider the influence of the rotation of an object, are more accurate in motion modeling, belong to a nonlinear model, and need to be matched with a nonlinear recursive filter, such as an extended Kalman filter or an unscented Kalman filter, to estimate the motion state. The prediction and update of the motion state need to approximate to obtain linear transformation by means of solving a Jacobian matrix or sampling and the like, and the calculation is complex.
The recursive filter relies on continuous observation of the system environment, and when the error of an observation result is large, a complex motion model has no absolute advantage in accuracy compared with a simple motion model. For example, the three-dimensional target detector based on point cloud is difficult to accurately predict the complex situation, such as an object far away from the laser radar or an object partially occluded, and the error between the observed bounding box and the true value may be large.
In the 3D multi-object tracking problem, the correlation is usually performed according to the intersection ratio (IoU) between the current frame and the 3D bounding box of the historical frame, however, this approach has a serious drawback when the observation point object itself moves. When the observation point object moves, the observed object has extra motion amount in a local coordinate system, and the motion amount cannot be accurately predicted by using a motion model. As shown in fig. 1, especially when the observation point object itself performs a motion such as turning, accelerating, decelerating, etc., the observed position of the object in the local coordinate system may be changed drastically, which may cause the association using the 3D bounding box IoU to be nearly ineffective, or even cause a false association. This makes the original 3D multi-object tracking method quite unstable, performing well in scenes where the observation point is stationary, but performing poorly in scenes where the observation point is moving "violently".
Therefore, how to improve the tracking accuracy of the conventional 3D multi-target tracking method becomes a problem to be solved urgently.
Disclosure of Invention
In order to solve the above technical problems, the present invention provides a motion compensation optimization method and system for 3D multi-target tracking.
The technical solution of the invention is as follows: a motion compensation optimization method facing 3D multi-target tracking comprises the following steps:
step S1: respectively obtaining the data (theta, v) of the local coordinate systems of the observed point object in the t frame and the t +1 frame f ) (ii) a Wherein θ is the orientation of the object in the world coordinate system of the current frame, and θ ═ 0 represents the head orientation east; v. of f The advancing speed of the object in the current frame is obtained;
step S2: calculating the course angle difference delta theta of the object in the tth frame and the t +1 th frame, and according to the forward speed of the object in the tth frame and the t +1 th frame
Figure BDA0002964077390000021
And
Figure BDA0002964077390000022
calculating the position of the object after motion compensation at the time of the t frame and under the local coordinate system of the t +1 frame
Figure BDA0002964077390000023
Step S3: according to the position of the object after motion compensation
Figure BDA0002964077390000024
Combining with motion prediction, updating the position of the object at the time of the t +1 frame and in the local coordinate system of the t +1 frame
Figure BDA0002964077390000025
Step S4: obtaining an object surrounding frame of the t +1 th frame according to detection
Figure BDA0002964077390000026
And the predicted position of the object motion
Figure BDA0002964077390000027
Performing data correlation to determine the objectEnclosing the final position of the frame.
Compared with the prior art, the invention has the following advantages:
1. the invention discloses a motion compensation optimization method facing 3D multi-target tracking, which utilizes self Inertial Measurement Unit (IMU) data of an observation point object to perform motion compensation so as to enhance a correlation link, improve the correlation degree of a 3D surrounding frame during tracking and reduce the number of error correlations. The method can obviously improve the tracking precision of the 3D multi-target tracking method, and especially has the most obvious improvement effect in the scenes of object steering and acceleration at the observation point.
2. The method provided by the invention is light and simple, and only needs to perform simple coordinate transformation on each object state. The method is generally applicable to various 3D multi-target tracking methods which use object motion information for association, and can be realized by only slightly changing.
3. The method provided by the invention has the advantages that the motion compensation is carried out on the object, the obtained motion state of the object is closer to the actual motion state of the object, and the influence of the motion of the object at the observation point is avoided.
Drawings
FIG. 1 is a schematic diagram of an object motion state when an object at an observation point moves according to an embodiment of the present disclosure;
FIG. 2 is a flowchart illustrating a motion compensation optimization method for 3D multi-object tracking according to an embodiment of the present invention;
fig. 3 shows a step S2 of the motion compensation optimization method for 3D multi-object tracking according to the embodiment of the present invention: calculating the heading angle difference delta theta of the object in the t frame and the t +1 frame, and according to the advancing speed of the object in the t frame and the t +1 frame
Figure BDA0002964077390000031
And
Figure BDA0002964077390000032
calculating the position of the object after motion compensation at the time of the t frame and under the local coordinate system of the t +1 frame
Figure BDA0002964077390000033
A flow chart of (1);
fig. 4 is a block diagram of a motion compensation optimization system for 3D multi-object tracking according to an embodiment of the present invention.
Detailed Description
Motion compensation is a means of performing supplementary correction on observed data using the motion of an object at an observation point, that is, when the object itself moves. When the observation point object moves, data observed by different frames are in different local coordinate systems, which greatly affects data association between the frames, and the motion compensation can correct the data to enable the data to be in the same local coordinate system, so that the data association effect is optimized. Motion compensation is widely applied to various sensing and imaging devices, and the technology is used in cameras, projectors and even intelligent televisions.
The invention provides a motion compensation optimization method and system facing 3D multi-target tracking, which are based on the existing 3D multi-target tracking method, use the self Inertial Measurement Unit (IMU) data of an observation point object to perform motion compensation so as to enhance the association link, improve the association degree of a 3D bounding box during tracking, reduce the number of error associations and improve the tracking precision of the existing 3D multi-target tracking method.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings.
The embodiment of the invention adopts a server configured with 2 Intel Xeon CPUs E5-2690 v3 (each CPU contains 12 physical cores and starts a hyper thread), 4 GeForce RTX 2080Ti GPUs (each 4352 core and 12GB video memory) and 256GB memory. Server software configuration case: the operating system was Ubuntu 18.04, the Python version was 3.7.7, and the CUDA version was 10.2.
The method is used for constructing an embodiment on the basis of a self-developed 3D multi-target tracking system ThunderMOT, wherein the ThunderMOT uses SA-SSD as a detector and uses a constant-speed motion model. The tracking management parameters are set to min _ hits ═ 1 and max _ agents ═ 2.
The tracking accuracy evaluation adopts a KITTI multi-target tracking data set, 11 sequences in training data are selected as verification data sets, the data arrival time interval delta t is 100 milliseconds, 3908 frames and 387 objects are used totally, and the types of the objects comprise motor vehicles, bicycles and pedestrians. The embodiment of the invention only evaluates the most numerous vehicle types in all types, and 217 objects are used in total. The tracking accuracy evaluation adopts CLEAR MOT index, and index calculation changes the two-dimensional bounding box intersection ratio into three-dimensional bounding box intersection ratio calculation on the basis of KITTI official supply.
Example one
As shown in fig. 2, a motion compensation optimization method for 3D multi-object tracking according to an embodiment of the present invention includes the following steps:
step S1: respectively obtaining the data (theta, v) of the local coordinate systems of the observed point object in the t frame and the t +1 frame f ) (ii) a Wherein, θ is the orientation of the object in the world coordinate system of the current frame, and θ ═ 0 represents the head orientation east; v. of f The advancing speed of the object in the current frame is shown.
In order to solve the problem that when an observation point object moves per se in 3D multi-target tracking, the correlation is nearly failed by using the intersection ratio (IoU) of a current frame 3D surrounding frame and a historical frame 3D surrounding frame, an Inertial Measurement Unit (IMU) sensor in a vehicle-mounted system is introduced, and data (theta, v) of the object in local coordinate systems of the t frame and the t +1 frame can be obtained by using the IMU respectively f ) Where θ is used to indicate the orientation of the object in the t-th frame under the world coordinate system of the object at the current observation point, θ ═ 0 is used to indicate the orientation of the vehicle head in east, v f Indicating the advancing speed of the object in the current frame.
For the subsequent motion state prediction, the motion state of the t +1 th frame is predicted based on the motion state of the t-th frame. Due to the motion of the observation point itself, the objects observed in the t frame and the t +1 frame will be in different local coordinate systems, which is a direct cause of near failure of the correlation by the object 3D bounding box IoU in both frames. The invention provides a method for converting the coordinate system of the recorded motion state by using the data provided by the IMU, so that the recorded motion state is in the same coordinate system, and the obtained motion state of the object is closer to the actual motion state of the object and is not influenced by the motion of the object at the observation point.
Step S2: calculating the heading angle difference delta theta of the object in the t frame and the t +1 frame, and according to the advancing speed of the object in the t frame and the t +1 frame
Figure BDA0002964077390000041
And
Figure BDA0002964077390000042
calculating the position of the object after motion compensation at the time of the t frame and under the local coordinate system of the t +1 frame
Figure BDA0002964077390000043
Step S3: according to the position of the object after motion compensation
Figure BDA0002964077390000051
Combining with motion prediction, updating the position of the object at the moment of the t +1 th frame and in the local coordinate system of the t +1 th frame
Figure BDA0002964077390000052
Step S4: obtaining an object surrounding frame of the t +1 th frame according to detection
Figure BDA0002964077390000053
And the predicted position of the object movement
Figure BDA0002964077390000054
And performing data correlation to determine the final position of the object surrounding frame.
The motion state prediction needs to predict the motion state of the object in the t +1 th frame through the motion state of the object in the t th frame. However, the motion state of the object in the t +1 th frame obtained by motion prediction in the existing method is still in the local coordinate system of the t-th frame, so the heading angle difference delta theta and the forward speed between the t-th frame and the t + 1-th frame provided by the IMU in the embodiment of the invention
Figure BDA0002964077390000055
And
Figure BDA0002964077390000056
and performing motion compensation on the predicted motion state of the object.
As shown in fig. 3, in one embodiment, the step S2: calculating the course angle difference delta theta of the object in the tth frame and the t +1 th frame, and according to the forward speed of the object in the tth frame and the t +1 th frame
Figure BDA0002964077390000057
And
Figure BDA0002964077390000058
calculating the position of the object after motion compensation at the time of the t frame and under the local coordinate system of the t +1 frame
Figure BDA0002964077390000059
The method specifically comprises the following steps:
step S21: calculating the course angle difference delta theta of the object between the t frame and the t +1 frame, and obtaining a rotation matrix R according to the following formula (1);
Figure BDA00029640773900000510
wherein, the R size is 3 multiplied by 3, which represents the 3D coordinate rotation relation.
Step S22: according to the heading angle difference delta theta and the advancing speed of the object in the tth frame and the t +1 th frame
Figure BDA00029640773900000511
And
Figure BDA00029640773900000512
calculating the displacement delta X of the object between the t frame and the t +1 frame according to the following formula (2);
Figure BDA00029640773900000513
wherein, Δ X is 3 × 1, which represents the displacement of the 3D coordinate axis in each direction.
Step S23: according to the following formula (3), the position of the object in the local coordinate system of the t frame time and the t +1 frame is calculated
Figure BDA00029640773900000514
Figure BDA00029640773900000515
Wherein the content of the first and second substances,
Figure BDA00029640773900000516
the upper corner of the graph indicates the local coordinate system in which the object is located, and the lower corner indicates the time frame in which the object is located.
In addition, the coordinate system corresponding to the above formula and matrix is based on the observation point object as the origin, the forward direction of the observation point is the positive direction of the Z axis, the right side of the observation point object is the positive direction of the X axis, and the downward direction of the observation point object is the positive direction of the Y axis.
In one embodiment, the step S3: according to the position of the object after motion compensation
Figure BDA00029640773900000517
Combining with motion prediction, updating the position of the object at the moment of the t +1 th frame and in the local coordinate system of the t +1 th frame
Figure BDA00029640773900000518
The method specifically comprises the following steps:
in this step, the motion state of the object in the t +1 th frame needs to be updated. The position of the motion-compensated object obtained in step S2
Figure BDA0002964077390000061
Inputting a trained motion prediction model, and updating the position of the object at the moment of the t +1 frame and under the local coordinate system of the t +1 frame
Figure BDA0002964077390000062
In one embodiment, the step S4: obtaining an object enclosing frame of the t +1 th frame according to detection
Figure BDA0002964077390000063
And the predicted position of the object movement
Figure BDA0002964077390000064
Performing data association, and determining the position of the final object enclosure frame, specifically including:
according to the t +1 th frame detected by the existing target detection model, at least more than one object surrounds the frame
Figure BDA0002964077390000065
And the predicted position of the object movement
Figure BDA0002964077390000066
And performing data association, and finally determining the position of the object surrounding frame.
Table 1 below shows three-dimensional tracking accuracy data versus data for a ThunderMOT that was motion compensated using the present invention. Where nMC indicates that motion compensation is not used and MC indicates that motion compensation is used.
TABLE 1 comparison of three-dimensional tracking accuracy data for motion compensation of thunderMOT
Name ΔMOTA ΔMOTP MOTA MOTP MT ML IDS FRAG F1 Prec Recall FAR TP FP FN
nMC N/A N/A 0.8366 0.7572 0.7081 0.0486 0 45 0.9259 0.9676 0.8876 0.073 8553 286 1083
MC 0.0111 0.0005 0.8477 0.7577 0.7189 0.0486 0 40 0.9309 0.9741 0.8913 0.0582 8591 228 1048
Wherein, mota (multiple Object Tracking accuracy) is a main standard for measuring Tracking accuracy, and the calculation method takes into consideration the Object matching errors in all frames in the multi-target Tracking process, and is as the following formula (4):
Figure BDA0002964077390000067
wherein FP is the number of false positive cases, indicating that an object that does not actually exist is erroneously identified as the number of objects; FN is the number of false counterexamples, representing the number of objects that are actually present but not recognized as objects; IDSW is the number of times the ID of the tracked object is changed, and indicates the number of times the same object is assigned with different IDs in different frames. GT is Ground Truth, which indicates the number of all positive cases.
The major quantization of the motp (multiple Object Tracking precision) is the positioning accuracy of the detector, which is calculated as the following formula (5):
Figure BDA0002964077390000068
wherein d is i In order to detect the measurement distance between the object i and the matched real value, a two-dimensional bounding box intersection ratio or a three-dimensional bounding box intersection ratio can be adopted; c is the number of successful matches in the current frame.
The results in table 1 above show that the tracking accuracy MOTA and MOTP are improved, and the gains obtained by motion compensation mainly result from the reduction of the interruption times FRAG, False Positive (FP) and False Negative (FN). This shows that the method for optimizing 3D multi-object tracking by motion compensation proposed by the present invention is effective.
The invention discloses a motion compensation optimization method facing 3D multi-target tracking, which utilizes self Inertial Measurement Unit (IMU) data of an observation point object to perform motion compensation so as to enhance a correlation link, improve the correlation degree of a 3D surrounding frame during tracking and reduce the number of error correlations. The method can obviously improve the tracking precision of the 3D multi-target tracking method, and especially has the most obvious improvement effect in the scenes of object steering and acceleration at the observation point. Meanwhile, the method provided by the invention is light and simple, and only needs to perform simple coordinate transformation on each object state. The method is generally applicable to various 3D multi-target tracking methods which use object motion information for association, and can be realized by only slightly changing.
Example two
As shown in fig. 4, an embodiment of the present invention provides a motion compensation optimization method system for 3D multi-target tracking, including the following modules:
an object motion data acquiring module 51 for acquiring data (θ, v) of local coordinate systems of the object at the t-th frame and the t + 1-th frame, respectively f ) (ii) a Wherein θ is the orientation of the object in the world coordinate system of the current frame, and θ equals 0 to represent the head orientation east; v. of f The advancing speed of the object in the current frame is obtained;
object motion compensationA module 52, configured to calculate a heading angle difference Δ θ between the tth frame and the t +1 th frame of the object, and according to the forward speed of the object at the tth frame and the t +1 th frame
Figure BDA0002964077390000071
And
Figure BDA0002964077390000072
calculating the position of the object after motion compensation at the time of the t frame and in the local coordinate system of the t +1 frame
Figure BDA0002964077390000073
An update object motion data module 53 for motion-compensated position of the object
Figure BDA0002964077390000074
Combining with motion prediction, updating the position of the object at the time of the t +1 frame and in the local coordinate system of the t +1 frame
Figure BDA0002964077390000075
An object position determination module 54 for determining an object bounding box of the t +1 th frame according to the detection
Figure BDA0002964077390000076
And the predicted position of the object motion
Figure BDA0002964077390000077
And performing data association to determine the final position of the object enclosing frame.
The above examples are provided for the purpose of describing the present invention only and are not intended to limit the scope of the present invention. The scope of the invention is defined by the appended claims. Various equivalent substitutions and modifications can be made without departing from the spirit and principles of the invention, and are intended to be within the scope of the invention.

Claims (2)

1. A motion compensation optimization method facing 3D multi-target tracking is characterized by comprising the following steps:
step S1: respectively obtaining the data (theta, v) of the local coordinate systems of the observed point object in the t frame and the t +1 frame f ) (ii) a Wherein θ is the orientation of the object in the world coordinate system of the current frame, and θ equals 0 to represent the head orientation east; v. of f The advancing speed of the object in the current frame is obtained;
step S2: calculating the heading angle difference delta theta of the object in the t frame and the t +1 frame, and according to the advancing speed of the object in the t frame and the t +1 frame
Figure FDA0003698492790000011
And
Figure FDA0003698492790000012
calculating the position of the object after motion compensation at the t frame time and under the local coordinate system of the t +1 frame
Figure FDA0003698492790000013
The method specifically comprises the following steps:
step S21: calculating the course angle difference delta theta of the object at the t frame and the t +1 frame, and obtaining a rotation matrix R according to the following formula (1);
Figure FDA0003698492790000014
step S22: according to the course angular difference delta theta and the advancing speed of the object in the t frame and the t +1 frame
Figure FDA0003698492790000015
And
Figure FDA0003698492790000016
calculating the displacement delta X of the object between the t frame and the t +1 frame according to the following formula (2);
Figure FDA0003698492790000017
step S23: according to the following formula (3), the position of the object in the local coordinate system of the t frame time and the t +1 frame is calculated
Figure FDA0003698492790000018
Figure FDA0003698492790000019
Step S3: according to the position of the object after motion compensation
Figure FDA00036984927900000110
Combining the motion prediction to update the position of the object under the local coordinate system of the t +1 th frame and the t +1 th frame at the moment
Figure FDA00036984927900000111
Step S4: obtaining an object surrounding frame of the t +1 th frame according to detection
Figure FDA00036984927900000112
And the predicted position of the object motion
Figure FDA00036984927900000113
And performing data association to determine the final position of the object enclosing frame.
2. A motion compensation optimization system for 3D multi-target tracking is characterized by comprising the following modules:
an object motion data acquisition module for respectively acquiring the data (theta, v) of the local coordinate system of the object in the t frame and the t +1 frame f ) (ii) a Wherein θ is the orientation of the object in the world coordinate system of the current frame, and θ equals 0 to represent the head orientation east; v. of f The advancing speed of the object in the current frame is obtained;
an object motion compensation module for calculating the course angle difference delta theta of the object between the t frame and the t +1 frame and according to the advancing speed of the object between the t frame and the t +1 frame
Figure FDA0003698492790000021
And
Figure FDA0003698492790000022
calculating the position of the object after motion compensation at the time of the t frame and in the local coordinate system of the t +1 frame
Figure FDA0003698492790000023
The method specifically comprises the following steps:
step S21: calculating the course angle difference delta theta of the object at the t frame and the t +1 frame, and obtaining a rotation matrix R according to the following formula (1);
Figure FDA0003698492790000024
step S22: according to the course angular difference delta theta and the advancing speed of the object in the t frame and the t +1 frame
Figure FDA0003698492790000025
And
Figure FDA0003698492790000026
calculating the displacement delta X of the object between the t frame and the t +1 frame according to the following formula (2);
Figure FDA0003698492790000027
step S23: according to the following formula (3), the position of the object in the local coordinate system of the t frame time and the t +1 frame is calculated
Figure FDA0003698492790000028
Figure FDA0003698492790000029
An object motion data updating module for updating the object motion data according to the compensated position of the object
Figure FDA00036984927900000210
Combining the motion prediction to update the position of the object under the local coordinate system of the t +1 th frame and the t +1 th frame at the moment
Figure FDA00036984927900000211
An object position determining module for obtaining an object enclosing frame of the t +1 th frame according to detection
Figure FDA00036984927900000212
And the predicted position of the object motion
Figure FDA00036984927900000213
And performing data association to determine the final position of the object enclosing frame.
CN202110245922.9A 2021-03-05 2021-03-05 Motion compensation optimization method and system for 3D multi-target tracking Active CN112967316B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110245922.9A CN112967316B (en) 2021-03-05 2021-03-05 Motion compensation optimization method and system for 3D multi-target tracking

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110245922.9A CN112967316B (en) 2021-03-05 2021-03-05 Motion compensation optimization method and system for 3D multi-target tracking

Publications (2)

Publication Number Publication Date
CN112967316A CN112967316A (en) 2021-06-15
CN112967316B true CN112967316B (en) 2022-09-06

Family

ID=76276682

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110245922.9A Active CN112967316B (en) 2021-03-05 2021-03-05 Motion compensation optimization method and system for 3D multi-target tracking

Country Status (1)

Country Link
CN (1) CN112967316B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113610263B (en) * 2021-06-18 2023-06-09 广东能源集团科学技术研究院有限公司 Power plant operation vehicle track estimation method and system
CN116309696B (en) * 2022-12-23 2023-12-01 苏州驾驶宝智能科技有限公司 Multi-category multi-target tracking method and device based on improved generalized cross-over ratio

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101739686A (en) * 2009-02-11 2010-06-16 北京智安邦科技有限公司 Moving object tracking method and system thereof
CN106679649A (en) * 2016-12-12 2017-05-17 浙江大学 Hand movement tracking system and tracking method
CN107705319A (en) * 2017-10-16 2018-02-16 中国电子科技集团公司第二十八研究所 One kind is based on the real-time detecting and tracking method of empty day background Small object
CN109581365A (en) * 2018-12-14 2019-04-05 湖南华诺星空电子技术有限公司 A kind of life-detection system based on multi-rotor unmanned aerial vehicle, method
CN111899276A (en) * 2020-07-07 2020-11-06 武汉大学 SLAM method and system based on binocular event camera

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11300677B2 (en) * 2019-07-08 2022-04-12 GM Global Technology Operations LLC Automated driving systems and control logic for host vehicle velocity estimation using wide aperture radar

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101739686A (en) * 2009-02-11 2010-06-16 北京智安邦科技有限公司 Moving object tracking method and system thereof
CN106679649A (en) * 2016-12-12 2017-05-17 浙江大学 Hand movement tracking system and tracking method
CN107705319A (en) * 2017-10-16 2018-02-16 中国电子科技集团公司第二十八研究所 One kind is based on the real-time detecting and tracking method of empty day background Small object
CN109581365A (en) * 2018-12-14 2019-04-05 湖南华诺星空电子技术有限公司 A kind of life-detection system based on multi-rotor unmanned aerial vehicle, method
CN111899276A (en) * 2020-07-07 2020-11-06 武汉大学 SLAM method and system based on binocular event camera

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
《An Aircraft-Centered Multi-Frame TBD Method for Airborne Radar System》;Peng QY et al;《IEEE》;20200904;全文 *
《视觉运动追踪训练对运动决策的影响及其神经机制研究》;刘洋;《中国优秀博硕士学位论文全文数据库(博士)社会科学Ⅱ辑》;20190415(第2019年第04期);全文 *

Also Published As

Publication number Publication date
CN112967316A (en) 2021-06-15

Similar Documents

Publication Publication Date Title
Song et al. High accuracy monocular SFM and scale correction for autonomous driving
US8885049B2 (en) Method and device for determining calibration parameters of a camera
JP2021523443A (en) Association of lidar data and image data
EP3159125A1 (en) Device for recognizing position of mobile robot by using direct tracking, and method therefor
EP3159122A1 (en) Device and method for recognizing location of mobile robot by means of search-based correlation matching
CN112734852A (en) Robot mapping method and device and computing equipment
CN112967316B (en) Motion compensation optimization method and system for 3D multi-target tracking
Wedel et al. Realtime depth estimation and obstacle detection from monocular video
Yin et al. Dynam-SLAM: An accurate, robust stereo visual-inertial SLAM method in dynamic environments
CN103617636A (en) Automatic video-target detecting and tracking method based on motion information and sparse projection
CN113920198B (en) Coarse-to-fine multi-sensor fusion positioning method based on semantic edge alignment
CN113570629B (en) Semantic segmentation method and system for removing dynamic objects
US11663808B2 (en) Distance estimating device and storage medium storing computer program for distance estimation
McManus et al. Distraction suppression for vision-based pose estimation at city scales
CN111985300A (en) Automatic driving dynamic target positioning method and device, electronic equipment and storage medium
CN110426714B (en) Obstacle identification method
CN116415202A (en) Multi-source data fusion method, system, electronic equipment and storage medium
KR20100066952A (en) Apparatus for tracking obstacle using stereo vision and method thereof
CN113029185A (en) Road marking change detection method and system in crowdsourcing type high-precision map updating
CN115908498B (en) Multi-target tracking method and device based on category optimal matching
CN116563376A (en) LIDAR-IMU tight coupling semantic SLAM method based on deep learning and related device
JP2021081272A (en) Position estimating device and computer program for position estimation
Rozsa et al. Immediate vehicle movement estimation and 3D reconstruction for Mono cameras by utilizing epipolar geometry and direction prior
WO2023283929A1 (en) Method and apparatus for calibrating external parameters of binocular camera
CN114817765A (en) Map-based target course disambiguation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant