WO2024159751A1 - 一种车云协同的智能3d多目标跟踪方法及系统 - Google Patents

一种车云协同的智能3d多目标跟踪方法及系统 Download PDF

Info

Publication number
WO2024159751A1
WO2024159751A1 PCT/CN2023/118281 CN2023118281W WO2024159751A1 WO 2024159751 A1 WO2024159751 A1 WO 2024159751A1 CN 2023118281 W CN2023118281 W CN 2023118281W WO 2024159751 A1 WO2024159751 A1 WO 2024159751A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
vehicle
detection
tracking
detected
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/118281
Other languages
English (en)
French (fr)
Inventor
谭晓军
安亚松
张晓飞
刘群铭
石艳丽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sun Yat Sen University
Original Assignee
Sun Yat Sen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sun Yat Sen University filed Critical Sun Yat Sen University
Publication of WO2024159751A1 publication Critical patent/WO2024159751A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional [3D] objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the present invention relates to the field of vehicle-cloud collaborative control technology, and in particular to a vehicle-cloud collaborative intelligent 3D multi-target tracking method and system.
  • three-dimensional (3D) target tracking methods are widely used to capture the dynamic information of vehicles and pedestrians in complex scenes, while also providing motion information such as the accurate position and acceleration of objects in the scene.
  • tracking methods based on image data can be divided into three categories: tracking methods based on image data, 3D tracking methods based on radar point cloud data, and multimodal tracking methods based on image and radar data fusion.
  • tracking methods based on image data have achieved a lot of remarkable results, due to the lack of depth information in image data, tracking methods based on image data cannot effectively extract and utilize 3D data information to perceive stereo information and complex scenes.
  • 3D point cloud data can supplement richer stereo scene information.
  • Tracking methods based on multimodal data make full use of image data and point cloud data to achieve more accurate and robust tracking effects; however, in most multimodal tracking methods, data calibration is required for different modal data fusion, but in actual engineering applications, the calibration parameters cannot be effectively maintained stable for a long time. Therefore, tracking methods based on 2D image data and multimodal data fusion have certain limitations in actual engineering contexts.
  • the target tracking method based on point cloud data not only supplements the scene stereo information to enhance the tracking effect, but also ensures the stability of the method in actual engineering applications.
  • an embodiment of the present invention provides a fast and accurate vehicle-cloud collaborative intelligent 3D multi-target tracking method and system.
  • An aspect of an embodiment of the present invention provides a vehicle-cloud collaborative intelligent 3D multi-target tracking method, including:
  • Detect vehicle targets obtain the detection status of each vehicle in the scene at the initial moment, and then establish initial trajectory information based on the detection status and vehicle identity; wherein the detection status includes vehicle detection frame information and detection confidence score;
  • Detect vehicle objects obtain the detection status of each vehicle in the scene at the target time, and then calculate the data correlation between the detection box information of all detected objects at the target time and the predicted state of the target at the initial time;
  • the matching result is managed in a target state until the tracking target is completed.
  • detecting the vehicle objects, obtaining the detection status of each vehicle in the scene at the target time, and then calculating the data correlation between the detection frame information of all detected objects at the target time and the predicted status of the target at the initial time includes:
  • the motion difference includes the angular difference of the speed between the previous and next moments, the numerical difference of the speed between the previous and next moments, and the deviation of the vehicle direction between the previous and next moments.
  • the step of performing similarity matching on all detected objects and predicted objects according to the data association degree to obtain a matching result is specifically: generating a tracking matching result according to the similarity measurement of all targets, including:
  • the calculation formula of the estimated speed of the vehicle object is:
  • the expression for measuring the difference in motion between the previous and next moments is:
  • S M represents the measure of the difference in motion between the previous and next moments; represents the estimated speed of the vehicle object at time t; represents the estimated speed of the vehicle object at time t-1; ⁇ t represents the direction angle of the vehicle.
  • the performing target state management on the matching result includes:
  • the state of the vehicle object is configured as Tracking, and the detected target and the predicted target are given the same identity;
  • the match fails, if the detected target is from the previous moment, the life cycle of the detected target is updated; if the target is from the later moment, it is confirmed that the target has just appeared in the tracking field of view, and the identity, life cycle and status of the target are configured;
  • the target tracking task is completed according to the life cycle of the detection target.
  • completing the task of tracking the target according to the life cycle of the detected target includes:
  • the state of the detection target is configured as Death
  • the state of the detection target is configured as Miss.
  • the calculation formula for the life cycle of the target that failed to match is:
  • life′ represents the life cycle of the target that failed to match; life represents a fixed life cycle threshold; ⁇ represents the scale factor; ⁇ represents the offset factor; Ct represents the confidence score of the target being successfully detected.
  • Another aspect of the embodiment of the present invention further provides a vehicle-cloud collaborative intelligent 3D multi-target tracking system, including:
  • the first module is used to detect vehicle targets, obtain the detection status of each vehicle in the scene at the initial moment, and then establish initial trajectory information according to the detection status and vehicle identity; wherein the detection status includes vehicle detection frame information and detection confidence score;
  • the second module is used to predict the state of the target at the moment through a Kalman filter algorithm according to the target information of the tracking result in the scene at the initial moment;
  • the third module is used to detect vehicle objects, obtain the detection status of each vehicle in the scene at the target time, and then calculate the data correlation between the detection frame information of all detected objects at the target time and the predicted state of the target at the initial time;
  • the fourth module is used to perform similarity matching on all detected objects and predicted objects according to the data association degree to obtain a matching result
  • the fifth module is used to perform target state management on the matching results until the tracking target is completed.
  • Another aspect of an embodiment of the present invention further provides an electronic device, including a processor and a memory;
  • the memory is used to store programs
  • the processor executes the program to implement the method described above.
  • the embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium.
  • a processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.
  • the embodiments of the present invention detect vehicle targets, obtain the detection status of each vehicle in the initial moment scenario, and then establish initial trajectory information based on the detection status and vehicle identity; predict the state at the target moment through the Kalman filter algorithm based on the target information of the tracking result in the initial moment scenario; detect vehicle objects, obtain the detection status of each vehicle in the target moment scenario, and then calculate the data correlation between the detection frame information of all detection objects at the target moment and the predicted state of the target at the initial moment; perform similarity matching on all detection objects and predicted objects based on the data correlation to obtain matching results; perform target state management on the matching results until the tracking target is completed.
  • the present invention can quickly complete the tracking confirmation of multiple target vehicles, and improve the accuracy of tracking prediction based on vehicle data in vehicle-cloud collaboration.
  • FIG1 is a flowchart of the overall steps provided by an embodiment of the present invention.
  • FIG2 is a flow chart of target life cycle and state adaptive management based on detection confidence of the present invention
  • FIG3 is a flow chart of an intelligent 3D multi-target tracking system provided by an embodiment of the present invention.
  • the present invention adopts a two-stage tracking paradigm based on single point cloud data to achieve multi-point tracking.
  • the present invention also supports online updating of better target detection models, and realizes the online adjustment and upgrade functions of the offline tracking system.
  • the present invention provides a vehicle-cloud collaborative intelligent 3D multi-target tracking method, comprising:
  • Detect vehicle targets obtain the detection status of each vehicle in the scene at the initial moment, and then establish initial trajectory information based on the detection status and vehicle identity; wherein the detection status includes vehicle detection frame information and detection confidence score;
  • Detect vehicle objects obtain the detection status of each vehicle in the scene at the target time, and then calculate the data correlation between the detection box information of all detected objects at the target time and the predicted state of the target at the initial time;
  • the matching result is managed in a target state until the tracking target is completed.
  • detecting the vehicle objects, obtaining the detection status of each vehicle in the scene at the target time, and then calculating the data correlation between the detection frame information of all detected objects at the target time and the predicted status of the target at the initial time includes:
  • the motion difference includes the angular difference of the speed between the previous and next moments, the numerical difference of the speed between the previous and next moments, and the deviation of the vehicle direction between the previous and next moments.
  • the step of performing similarity matching on all detected objects and predicted objects according to the data association degree to obtain a matching result is specifically: generating a tracking matching result according to the similarity measurement of all targets, including:
  • the calculation formula of the estimated speed of the vehicle object is:
  • the expression for measuring the difference in motion between the previous and next moments is:
  • S M represents the measure of the difference in motion between the previous and next moments; represents the estimated speed of the vehicle object at time t; represents the estimated speed of the vehicle object at time t-1; ⁇ t represents the direction angle of the vehicle.
  • the performing target state management on the matching result includes:
  • the state of the vehicle object is configured as Tracking, and the detected target and the predicted target are given the same identity;
  • the match fails, if the detected target is from the previous moment, the life cycle of the detected target is updated; if the target is from the later moment, it is confirmed that the target has just appeared in the tracking field of view, and the identity, life cycle and status of the target are configured;
  • the target tracking task is completed according to the life cycle of the detection target.
  • completing the task of tracking the target according to the life cycle of the detected target includes:
  • the state of the detection target is configured as Death
  • the state of the detection target is configured as Miss.
  • the calculation formula for the life cycle of the target that failed to match is:
  • life′ represents the life cycle of the target that failed to match; life represents a fixed life cycle threshold; ⁇ represents the scale factor; ⁇ represents the offset factor; Ct represents the confidence score of the target being successfully detected.
  • Another aspect of the embodiment of the present invention further provides a vehicle-cloud collaborative intelligent 3D multi-target tracking system, including:
  • the first module is used to detect vehicle targets, obtain the detection status of each vehicle in the scene at the initial moment, and then establish initial trajectory information according to the detection status and vehicle identity; wherein the detection status includes vehicle detection frame information and detection confidence score;
  • the second module is used to predict the state of the target at the moment through a Kalman filter algorithm according to the target information of the tracking result in the scene at the initial moment;
  • the third module is used to detect vehicle objects, obtain the detection status of each vehicle in the scene at the target time, and then calculate the data correlation between the detection frame information of all detected objects at the target time and the predicted state of the target at the initial time;
  • the fourth module is used to perform similarity matching on all detected objects and predicted objects according to the data association degree, and obtain To the matching results;
  • the fifth module is used to perform target state management on the matching results until the tracking target is completed.
  • Another aspect of an embodiment of the present invention further provides an electronic device, including a processor and a memory;
  • the memory is used to store programs
  • the processor executes the program to implement the method described above.
  • the embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium.
  • a processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.
  • the present invention proposes a vehicle-cloud collaborative intelligent 3D multi-target tracking method and online update system, which solves the problem that 2D image data cannot provide stereoscopic information of the scene and multimodal data tracking is difficult to use in practical applications, and supports online adjustment and upgrading of offline tracking systems.
  • An intelligent 3D multi-target tracking method for vehicle-cloud collaboration is implemented by the following steps:
  • Step 1 Use the target detection module to detect vehicle targets and obtain the detection state D t-1 of each vehicle in the scene at the initial time t-1, including the vehicle detection frame (size, position, orientation) information B t-1 and the detection confidence score C t-1 , and initialize the vehicle target identity ID to establish the initial trajectory information.
  • Step 2 Use the Kalman filter algorithm to predict the state P t at time t based on the target information B t-1 in the tracking result at time t-1.
  • Step 3 Use the vehicle detection module to obtain the detection state D t of the vehicle in the scene at time t. According to the proposed measurement method based on target motion and geometric information, calculate the data association S between the detection box information B t of all detected targets at time t and the predicted state P t of the target at time t-1.
  • Step 4 Based on the obtained data association S, the Hungarian algorithm is used to perform similarity matching on all detected targets and predicted targets to obtain matching results.
  • Step 5 Use the adaptive target state management module to manage the vehicle target state of the matching results. If the match is successful, the target state is set to Tracking, and the detected target is given the same identity ID as the predicted target; if the match fails, the state management is performed according to its lifespan.
  • Step 6 Repeat steps 2 to 5 until the tracking is completed.
  • An intelligent 3D multi-target tracking system supporting vehicle-cloud collaborative online update specifically includes a vehicle-cloud collaborative online update module, a vehicle target detection module, a vehicle identity association calculation module, a vehicle identity matching module, and a vehicle status management module. piece.
  • the vehicle-cloud collaborative online update module adjusts and upgrades the vehicle target detection pre-trained model in the offline tracking system by training the latest and optimal vehicle target detection model in the cloud.
  • the vehicle detection module uses a pre-trained vehicle target detection model to detect vehicle targets in the input 3D radar point cloud frame data to obtain the vehicle's 3D detection frame, orientation angle, and detection confidence score.
  • the vehicle identity association calculation module is used to calculate the identity association between vehicles at the previous and next moments. Specifically, according to the target similarity proposed in the present invention, the similarity matrix between all targets at the previous and next moments is calculated.
  • the vehicle identity matching module uses the Hungarian algorithm to perform maximum matching between the targets at the previous and next moments based on the similarity matrix between the targets at the previous and next moments obtained by the vehicle identity association calculation module, thereby realizing vehicle identity association.
  • the vehicle status management module performs adaptive management of the vehicle target life cycle and status during the tracking process based on the vehicle identity association results at the previous and next moments. Specifically, the vehicle status management is performed according to the management strategy proposed in the present invention, and the final tracking result is updated.
  • An intelligent 3D multi-target tracking algorithm for vehicle-cloud collaboration includes three specific algorithm processes: target (vehicle) detection, target (vehicle) identity association, and target (vehicle) status management.
  • the present invention realizes multi-target real-time tracking based on the tracking-by-detection tracking paradigm, and target detection is the first step of the algorithm.
  • a cascaded 3D R-CNN model is used as the target detection algorithm, and the point cloud data at time t is input.
  • all target detection results D t-1 are obtained through the target detection module, and the identity IDs of all targets are initialized.
  • the Kalman filter is used to predict the state at time t
  • the detection state of the target is D t , and its speed is estimated based on the detection states before and after:
  • the measure of the motion difference between the previous and next moments is defined:
  • the first term measures the angular difference in velocity between two moments (frames); the second term measures the numerical difference in velocity between two moments; and the third term measures the offset in target orientation between two moments.
  • the metric of their geometric difference is defined based on the complete intersection overlap (complete-IoU):
  • ⁇ 2 represents the detected vehicle center position (x t , y t , z t ) and the target center position predicted by the Kalman filter
  • c 2 represents the minimum diagonal length of the detection bounding box B t and the predicted bounding box P t
  • ⁇ is used to adjust the weight between IoU and spatial volume metric
  • v is a parameter to measure the spatial volume metric, which are defined as:
  • the Hungarian algorithm After obtaining the similarity measurement of all targets before and after, the Hungarian algorithm is used to match all detected targets with all targets at the previous moment (including the detected targets at the previous moment and the targets that failed to match) to obtain the tracking results; its matching objective function is:
  • X is an assignment matrix with only one element of 1 in each row and column
  • the state management strategy is used to adaptively manage the life cycle and state of the target.
  • the target state is set to Tracking, and the detected target is given the same identity ID as the predicted target; otherwise, the target that fails to match enters the state management stage. If the target is from the previous moment, its life cycle is updated according to formula (7), and if life′ ⁇ 1, the target state is set to Death, and it is considered that the target has left the tracking field of view; otherwise, the target state is set to Miss, and the target will have a chance to be successfully tracked in the next survival cycle, and the problem of tracking failure caused by occlusion in the short term is solved. If the target comes from time t, then the target has just appeared in the tracking field of view, initialize its identity ID, update its life cycle life′, and set its state to birth.
  • the lifecycle of the unmatched prediction target will be updated as follows:
  • life is a fixed life cycle threshold
  • ⁇ is a scale factor
  • ⁇ is an offset factor
  • Ct is the confidence score of the target successfully detected output by the target detection module.
  • the targets in the Miss, birth, and Tracking states update the tracking result T t at time t together.
  • FIG. 1 is an intelligent 3D multi-target tracking system that supports vehicle-cloud collaborative online updates, specifically including a vehicle-cloud collaborative online update module, a vehicle target detection module, a vehicle identity association calculation module, a vehicle identity matching module and a vehicle status management module.
  • the vehicle-cloud collaborative online update module uploads data to the cloud through the vehicle-cloud collaborative communication system, and selects different vehicle target detection methods and models in the cloud for online training. If the model trained in the cloud is better than the performance (detection accuracy and speed, etc.) of the vehicle target detection model in the current offline tracking system, the vehicle target detection model in the offline tracking system is updated by downloading the model from the cloud. To ensure that the updated model can fit the offline tracking system, the vehicle detection model must satisfy the input of 3D radar point cloud data and the output of vehicle target detection results (including but not limited to 3D detection box, orientation angle and detection confidence score).
  • the vehicle detection module detects vehicle targets in the input 3D radar point cloud data and obtains the vehicle's 3D detection box, orientation angle, and detection confidence score. Specifically, a 3D R-CNN cascade detection model with three detection heads is initially used in this system, in which a fixed IoU threshold is set to 0.75.
  • the vehicle identity association calculation module is used to calculate the identity association between vehicles at the previous and next moments. Specifically, according to the target similarity described in formula (5), the similarity matrix between all targets at the previous and next moments is calculated.
  • the vehicle identity matching module uses the Hungarian algorithm to perform maximum matching between the targets at the previous and next moments based on the similarity matrix between the targets at the previous and next moments obtained by the vehicle identity association calculation module, thereby realizing vehicle identity association.
  • the vehicle status management module is used to adaptively manage the vehicle target life cycle and status of the vehicle identity association results at the previous and next moments, and update the final tracking results.
  • the fixed life cycle threshold, scale factor and offset factor in the adaptive life cycle calculation formula (7) are set to 5, 8 and -0.5 respectively.
  • the tracking method of the present invention integrates vehicle detection, data association and target life state management methods, and uses the detected target motion and geometric information to achieve real-time tracking of multiple targets in complex scenes; in the case of target occlusion or misdetection, the vehicle state management strategy is used to achieve target re-matching and misdetection result screening.
  • the vehicle-cloud collaborative system supports online calibration and upgrading of the tracking system. Therefore, the advantages of the present invention are as follows:
  • An intelligent 3D multi-target tracking method including designing a data association method based on target motion and geometric information to achieve fast and accurate multi-target tracking; designing an adaptive target state management method to reduce the mismatch of tracking targets and effectively solve the problem of tracking interruption caused by short-term vehicle occlusion.
  • the function/operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram.
  • the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order.
  • the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.
  • the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.
  • An ordered list of executable instructions for a logical function may be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (e.g., a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, apparatus or device and execute the instructions), or in conjunction with such instruction execution system, apparatus or device.
  • a "computer-readable medium” may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, apparatus or device, or in conjunction with such instruction execution system, apparatus or device.
  • computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM).
  • the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.
  • a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.
  • a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal
  • a dedicated integrated circuit having a suitable combination of logic gate circuits
  • PGA programmable gate array
  • FPGA field programmable gate array

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Traffic Control Systems (AREA)

Abstract

一种车云协同的智能3D多目标跟踪方法及系统，方法包括：对车辆目标进行检测，获取初始时刻场景下各个车辆的检测状态，建立初始轨迹信息；根据初始时刻场景下的跟踪结果的目标信息，通过卡尔曼滤波算法预测目标时刻的状态；对车辆对象进行检测，获取各个车辆的检测状态，进而计算目标时刻下所有检测对象的检测框信息与初始时刻下目标的预测状态之间的数据关联度；根据数据关联度，对所有检测对象和预测对象进行相似度匹配，得到匹配结果；对匹配结果进行目标状态管理，直至完成跟踪目标。

Description

一种车云协同的智能3D多目标跟踪方法及系统 技术领域
本发明涉及车云协同控制技术领域，尤其是一种车云协同的智能3D多目标跟踪方法及系统。
背景技术
在车云协同和自动驾驶任务中，3维(3D)目标跟踪方法被广泛用于捕获复杂场景中车辆和行人的动态信息，同时还能提供场景中物体的准确位置和加速度等运动信息。
按照使用的数据不同，现今的跟踪方法可分为三类：基于图像数据的跟踪方法、基于雷达点云数据的3D跟踪方法以及图像和雷达数据融合的多模态跟踪方法。虽然基于图像数据的跟踪方法已经取得了大量显著的成果，但由于图像数据缺乏深度信息，基于图像数据的跟踪方法并不能有效地提取和利用3D数据信息以感知立体信息和复杂场景。相比于2D的图像数据，3D的点云数据能够补充更丰富的立体场景信息。基于多模态数据的跟踪方法充分利用图像数据和点云数据实现了更加精准、更加鲁棒的跟踪效果；但在大多的多模态跟踪方法中，需要对数据进行标定用于不同模态数据融合，而实际工程应用中却不能长期有效地保持标定参数的稳定。因此，基于2D图像数据和多模态数据融合的跟踪方法在实际工程背景中都存在一定的局限性。而基于点云数据的目标跟踪方法则在补充场景立体信息增强跟踪效果的同时，保证了方法在实际工程应用中的稳定性。
现今的跟踪方法按照阶段可以大致分为两类：端到端的跟踪方法和检测-跟踪两阶段跟踪方法。端到端的方法中所有模块被集成为一个完整的过程，优化了算法的学习过程，但最终的子模块缺乏一定的理论支持，导致其安全可信度成为一个值得担忧的问题。现今3D目标检测方法效果显著提升，能够为目标跟踪提供足够可依赖的信息，所以检测-跟踪两阶段的跟踪方法在实际应用中更具有价值。
发明内容
有鉴于此，本发明实施例提供一种快速且精准的车云协同的智能3D多目标跟踪方法及系统。
本发明实施例的一方面提供了一种车云协同的智能3D多目标跟踪方法，包括：
对车辆目标进行检测，获取初始时刻场景下各个车辆的检测状态，进而根据所述检测状态和车辆身份建立初始轨迹信息；其中，所述检测状态包括车辆检测框信息、检测置信度分数；
根据所述初始时刻场景下的跟踪结果的目标信息，通过卡尔曼滤波算法预测目标时刻的状态；
对车辆对象进行检测，获取目标时刻场景下各个车辆的检测状态，进而计算目标时刻下所有检测对象的检测框信息与初始时刻下目标的预测状态之间的数据关联度；
根据所述数据关联度，对所有检测对象和预测对象进行相似度匹配，得到匹配结果；
对所述匹配结果进行目标状态管理，直至完成跟踪目标。
可选地，所述对车辆对象进行检测，获取目标时刻场景下各个车辆的检测状态，进而计算目标时刻下所有检测对象的检测框信息与初始时刻下目标的预测状态之间的数据关联度，包括：
根据车辆对象在前后时刻的检测状态，估计车辆对象的速度；
根据前后时刻的估计速度及检测角度信息，确定前后时刻运动差异的度量；
根据所述目标时刻场景下的检测信息和预测状态，确定几何差异的度量；
根据所述运动差异的度量和所述几何差异的度量，确定前后时刻的目标的相似性度量；
其中，所述运动差异包括前后时刻之间速度的角度差异、前后时刻之间速度的数值差异、前后时刻之间车辆朝向的偏移。
可选地，所述根据所述数据关联度，对所有检测对象和预测对象进行相似度匹配，得到匹配结果这一步骤具体为：根据所有目标的相似性度量，生成跟踪匹配结果，包括：
获取所有检测目标的相似性度量；
根据所述相似性度量，利用匈牙利算法对前后时刻的所有检测目标进行匹配，得到跟踪结果。
可选地，所述车辆对象的预估的速度的计算公式为：
其中，表示车辆对象的预估速度；t表示时刻；(xt,yt)表示表示t时刻下检测目标的空间位置；表示空间位置向量对时间求微分操作。
可选地，所述前后时刻运动差异的度量的表达式为：
其中，SM表示前后时刻运动差异的度量；表示车辆对象在t时刻的预估速度；表示车辆对象在t-1时刻的预估速度；θt表示车辆的朝向角度。
可选地，所述对所述匹配结果进行目标状态管理，包括：
获取数据关联阶段的匹配结果；
当匹配成功时，将车辆对象的状态配置为Tracking，并为检测目标与预测目标赋予相同的身份标识；
当匹配失败时，如果检测目标来自在前时刻，则对检测目标的生命周期进行更新；如果目标来自在后时刻，则确认目标刚出现在跟踪视野内，对目标的身份标识、生命周期和状态进行配置；
根据所述检测目标的生命周期完成对目标的跟踪任务。
可选地，所述根据所述检测目标的生命周期完成对目标的跟踪任务，包括：
当检测目标的生命周期小于1时，将检测目标的状态配置为Death；
当检测目标的生命周期大于或等于1时，将检测目标的状态配置为Miss。
可选地，所述匹配失败的目标的生命周期的计算公式为：
其中，life′表示匹配失败的目标的生命周期；life表示固定的生命周期阈值；α代表尺度因子；β代表偏移因子；Ct代表目标被成功检测的置信度分数。
本发明实施例的另一方面还提供了一种车云协同的智能3D多目标跟踪系统，包括：
第一模块，用于对车辆目标进行检测，获取初始时刻场景下各个车辆的检测状态，进而根据所述检测状态和车辆身份建立初始轨迹信息；其中，所述检测状态包括车辆检测框信息、检测置信度分数；
第二模块，用于根据所述初始时刻场景下的跟踪结果的目标信息，通过卡尔曼滤波算法预测目标时刻的状态；
第三模块，用于对车辆对象进行检测，获取目标时刻场景下各个车辆的检测状态，进而计算目标时刻下所有检测对象的检测框信息与初始时刻下目标的预测状态之间的数据关联度；
第四模块，用于根据所述数据关联度，对所有检测对象和预测对象进行相似度匹配，得到匹配结果；
第五模块，用于对所述匹配结果进行目标状态管理，直至完成跟踪目标。
本发明实施例的另一方面还提供了一种电子设备，包括处理器以及存储器；
所述存储器用于存储程序；
所述处理器执行所述程序实现如前面所述的方法。
本发明实施例还公开了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器可以从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行前面的方法。
本发明的实施例对车辆目标进行检测，获取初始时刻场景下各个车辆的检测状态，进而根据所述检测状态和车辆身份建立初始轨迹信息；根据所述初始时刻场景下的跟踪结果的目标信息，通过卡尔曼滤波算法预测目标时刻的状态；对车辆对象进行检测，获取目标时刻场景下各个车辆的检测状态，进而计算目标时刻下所有检测对象的检测框信息与初始时刻下目标的预测状态之间的数据关联度；根据所述数据关联度，对所有检测对象和预测对象进行相似度匹配，得到匹配结果；对所述匹配结果进行目标状态管理，直至完成跟踪目标。本发明能够快速完成对多个目标车辆的跟踪确认，基于车云协同的车辆数据，提高了跟踪预测的准确性。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明实施例提供的整体步骤流程图；
图2为本发明基于检测置信度的目标生命周期及状态自适应管理流程图；
图3为本发明实施例提供的智能3D多目标跟踪系统流程图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
针对现有技术存在的问题，本发明基于单一的点云数据，采用两阶段的跟踪范式实现多 目标的跟踪。此外，结合车云协同系统之间的数据交互优势，本发明还支持在线更新更好的目标检测模型，实现离线跟踪系统的在线调校和升级功能。
具体地，本发明提供了一种车云协同的智能3D多目标跟踪方法，包括：
对车辆目标进行检测，获取初始时刻场景下各个车辆的检测状态，进而根据所述检测状态和车辆身份建立初始轨迹信息；其中，所述检测状态包括车辆检测框信息、检测置信度分数；
根据所述初始时刻场景下的跟踪结果的目标信息，通过卡尔曼滤波算法预测目标时刻的状态；
对车辆对象进行检测，获取目标时刻场景下各个车辆的检测状态，进而计算目标时刻下所有检测对象的检测框信息与初始时刻下目标的预测状态之间的数据关联度；
根据所述数据关联度，对所有检测对象和预测对象进行相似度匹配，得到匹配结果；
对所述匹配结果进行目标状态管理，直至完成跟踪目标。
可选地，所述对车辆对象进行检测，获取目标时刻场景下各个车辆的检测状态，进而计算目标时刻下所有检测对象的检测框信息与初始时刻下目标的预测状态之间的数据关联度，包括：
根据车辆对象在前后时刻的检测状态，估计车辆对象的速度；
根据前后时刻的估计速度及检测角度信息，确定前后时刻运动差异的度量；
根据所述目标时刻场景下的检测信息和预测状态，确定几何差异的度量；
根据所述运动差异的度量和所述几何差异的度量，确定前后时刻的目标的相似性度量；
其中，所述运动差异包括前后时刻之间速度的角度差异、前后时刻之间速度的数值差异、前后时刻之间车辆朝向的偏移。
可选地，所述根据所述数据关联度，对所有检测对象和预测对象进行相似度匹配，得到匹配结果这一步骤具体为：根据所有目标的相似性度量，生成跟踪匹配结果，包括：
获取所有检测目标的相似性度量；
根据所述相似性度量，利用匈牙利算法对前后时刻的所有检测目标进行匹配，得到跟踪结果。
可选地，所述车辆对象的预估的速度的计算公式为：
其中，表示车辆对象的预估速度；t表示时刻；(xt,yt)表示表示t时刻下检测目标的空间位置；表示空间位置向量对时间求微分操作。
可选地，所述前后时刻运动差异的度量的表达式为：
其中，SM表示前后时刻运动差异的度量；表示车辆对象在t时刻的预估速度；表示车辆对象在t-1时刻的预估速度；θt表示车辆的朝向角度。
可选地，所述对所述匹配结果进行目标状态管理，包括：
获取数据关联阶段的匹配结果；
当匹配成功时，将车辆对象的状态配置为Tracking，并为检测目标与预测目标赋予相同的身份标识；
当匹配失败时，如果检测目标来自在前时刻，则对检测目标的生命周期进行更新；如果目标来自在后时刻，则确认目标刚出现在跟踪视野内，对目标的身份标识、生命周期和状态进行配置；
根据所述检测目标的生命周期完成对目标的跟踪任务。
可选地，所述根据所述检测目标的生命周期完成对目标的跟踪任务，包括：
当检测目标的生命周期小于1时，将检测目标的状态配置为Death；
当检测目标的生命周期大于或等于1时，将检测目标的状态配置为Miss。
可选地，所述匹配失败的目标的生命周期的计算公式为：
其中，life′表示匹配失败的目标的生命周期；life表示固定的生命周期阈值；α代表尺度因子；β代表偏移因子；Ct代表目标被成功检测的置信度分数。
本发明实施例的另一方面还提供了一种车云协同的智能3D多目标跟踪系统，包括：
第一模块，用于对车辆目标进行检测，获取初始时刻场景下各个车辆的检测状态，进而根据所述检测状态和车辆身份建立初始轨迹信息；其中，所述检测状态包括车辆检测框信息、检测置信度分数；
第二模块，用于根据所述初始时刻场景下的跟踪结果的目标信息，通过卡尔曼滤波算法预测目标时刻的状态；
第三模块，用于对车辆对象进行检测，获取目标时刻场景下各个车辆的检测状态，进而计算目标时刻下所有检测对象的检测框信息与初始时刻下目标的预测状态之间的数据关联度；
第四模块，用于根据所述数据关联度，对所有检测对象和预测对象进行相似度匹配，得 到匹配结果；
第五模块，用于对所述匹配结果进行目标状态管理，直至完成跟踪目标。
本发明实施例的另一方面还提供了一种电子设备，包括处理器以及存储器；
所述存储器用于存储程序；
所述处理器执行所述程序实现如前面所述的方法。
本发明实施例还公开了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器可以从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行前面的方法。
下面结合说明书附图，对本发明的具体实现过程进行详细描述：
本发明提出了一种车云协同的智能3D多目标跟踪方法及在线更新系统，解决了2D图像数据无法提供场景立体信息和多模态数据跟踪难以用于实际应用的问题，且支持离线跟踪系统的在线调校和升级。
一种车云协同的智能3D多目标跟踪方法，具体由以下步骤实现：
步骤一：采用目标检测模块检测车辆目标，获得初始时刻t-1场景中每辆车的检测状态Dt-1，包括车辆检测框(大小、位置、朝向)信息Bt-1，以及检测置信度分数Ct-1，并初始化车辆目标身份ID，建立初始轨迹信息。
步骤二：利用卡尔曼滤波算法基于t-1时刻跟踪结果中的目标信息Bt-1预测其t时刻的状态Pt。
步骤三：利用车辆检测模块获取t时刻场景中车辆的检测状态Dt，根据所提出的基于目标运动和几何信息的度量方法，计算t时刻所有检测目标的检测框信息Bt和t-1时刻目标的预测状态Pt之间的数据关联度S。
步骤四：基于所得的数据关联度S，采用匈牙利算法对所有的检测目标和预测目标进行相似度匹配，得到匹配结果。
步骤五：采用自适应目标状态管理模块对匹配结果进行车辆目标状态管理。如果匹配成功，目标状态置为Tracking，赋予检测目标与预测目标相同身份ID；如果匹配失败，则按照其生存寿命进行状态管理。
步骤六：重复步骤二至步骤五，直至跟踪结束。
一种支持车云协同在线更新的智能3D多目标跟踪系统，具体包括车云协同在线更新模块、车辆目标检测模块、车辆身份关联度计算模块、车辆身份匹配模块以及车辆状态管理模 块。
所述车云协同在线更新模块，通过云端训练最新且最优的车辆目标检测模型对离线跟踪系统中的车辆目标检测预训练模型进行调校升级。
所述车辆检测模块，利用预训练的车辆目标检测模型在输入的3D雷达点云帧数据中检测车辆目标，获得车辆的3D检测框、朝向角度以及检测置信度分数。
所述车辆身份关联度计算模块用于计算前后时刻车辆之间的身份关联度。具体地，按照本发明中所提出的目标相似度，计算所有前后时刻目标之间的相似度矩阵。
所述车辆身份匹配模块，基于车辆身份关联度计算模块所得前后时刻目标之间的相似度矩阵，利用匈牙利算法进行前后时刻目标之间的最大匹配，实现车辆身份关联。
所述车辆状态管理模块基于前后时刻车辆身份关联结果，进行跟踪过程中车辆目标生命周期及状态的自适应管理。具体地，按照本发明中所提出的管理策略进行车辆状态管理，并更新最终跟踪结果。
具体地，结合图1、图2和图3说明本实施例的完整实施过程：
一种车云协同的智能3D多目标跟踪算法，具体包括目标(车辆)检测、目标(车辆)身份关联和目标(车辆)状态管理三个具体算法过程。
一、目标检测
本发明基于tracking-by-detection的跟踪范式实现多目标实时跟踪，目标检测是算法的第一步。使用级联的3D R-CNN模型作为目标检测算法，输入t时刻的点云数据，经过3D点云卷积网络和级联多检测头网络得到最终的检测结果Dt＝[Bt,Ct]，其中Bt＝[xt,yt,zt,wt,ht,lt,θt]表示目标检测框的中心位置(xt,yt,zt)、宽高长(wt,ht,lt)以及朝向角度(θt)；Ct表示检测置信度分数。
二、目标身份关联
在初始时刻t-1经过目标检测模块得到所有的目标检测结果Dt-1，并为所有目标初始化身份ID。基于t-1时刻检测状态中的检测框信息Bt-1，采用卡尔曼滤波预测t时刻的状态
t时刻，目标的检测状态为Dt，根据前后时刻检测状态估计其速度：
根据前后时刻的估计速度及检测角度信息，定义前后时刻运动差异的度量：
式中第一项度量两个时刻(帧)之间速度的角度差异；第二项度量两个时刻速度的数值差异；第三项度量两个时刻之间目标朝向的偏移。
根据t时刻检测信息Bt和预测状态Pt，基于完整交叉重叠度(complete-IoU)定义其几何差异的度量：
其中ρ2表示检测的车辆中心位置(xt,yt,zt)与卡尔曼滤波预测的目标中心位置之间的欧式距离，c2表示包含检测包围框Bt和预测包围框Pt的最小对角线长度，α用于调节IoU和空间体积度量之间的权重，v则是衡量空间体积度量的参数，分别定义为：

最后，前后时刻目标的相似性被度量为：
S＝SM+SG    (5)
得到所有目标前后时刻的相似性度量后，利用匈牙利算法对所有检测目标与前一时刻所有目标(包括前一时刻检测目标与匹配失败目标)进行匹配，得到跟踪结果；其匹配目标函数：

其中X是每行每列只有一个元素为1的指派矩阵，Sij表示上一时刻的目标i与当前时刻检测的目标j之间的相似度。xij＝1，则表示匹配成功，否则匹配失败。
三、目标状态管理
对于数据关联阶段匹配的结果，利用状态管理策略对目标的生命周期及状态自适应管理。
如果匹配成功，目标状态置为Tracking，赋予检测目标与预测目标相同身份ID；否则匹配失败的目标进入状态管理阶段。如果目标来自上一时刻，按照式(7)更新其生命周期，且如果life′<1，则将目标状态置为Death，认为目标已经离开跟踪视野；否则将目标状态置为Miss，目标将在接下来存活周期内有跟踪成功的机会，并解决短期内遮挡导致跟踪失败的 问题。如果目标来自t时刻，则目标刚出现在跟踪视野内，初始化其身份ID、更新其生命周期life′，并将其状态置为Birth。
具体地，未匹配成功的预测目标生命周期将按照如下更新：
其中life是固定的生命周期阈值、α是尺度因子、β是偏移因子。Ct是目标检测模块输出的该目标被成功检测的置信度分数。
状态为Miss、Birth和Tracking的目标一同更新t时刻的跟踪结果Tt。
结合图1至图3说明发明的另一种本实施方式，一种支持车云协同在线更新的智能3D多目标跟踪系统，具体包括车云协同在线更新模块、车辆目标检测模块、车辆身份关联度计算模块、车辆身份匹配模块以及车辆状态管理模块。
所述车云协同在线更新模块，通过车云协同通信系统将数据上传至云端，在云端选择不同的车辆目标检测方法及模型进行在线训练。如果云端训练所得模型比当前离线跟踪系统中车辆目标检测模型性能(检测精度和速度等)更好，则通过云端下载模型更新离线跟踪系统中的车辆目标检测模型。为保证更新模型能够契合离线跟踪系统，车辆检测模型需满足输入为3D雷达点云数据，输出为车辆目标检测结果(包含不限于3D检测框、朝向角度及检测置信度分数)。
所述车辆检测模块，在输入的3D雷达点云数据中检测车辆目标，获得车辆的3D检测框、朝向角度以及检测置信度分数。具体地，在本系统中初始使用3个检测头的3D R-CNN级联检测模型，其中固定的IoU阈值设置为0.75。
所述车辆身份关联度计算模块用于计算前后时刻车辆之间的身份关联度。具体地，按照式(5)所述的目标相似度，计算所有前后时刻目标之间的相似度矩阵。
所述车辆身份匹配模块，基于车辆身份关联度计算模块所得前后时刻目标之间的相似度矩阵，利用匈牙利算法进行前后时刻目标之间的最大匹配，实现车辆身份关联。
所述车辆状态管理模块用于对前后时刻车辆身份关联结果进行车辆目标生命周期及状态的自适应管理，并更新最终跟踪结果。其中自适应生命周期计算式(7)中固定生命周期阈值、尺度因子和偏移因子分别设置为5、8和-0.5。
综上所述，本发明的跟踪方法，综合车辆检测、数据关联以及目标生命状态管理方法，使用检测的目标运动和几何信息实现复杂场景中的多目标实时跟踪；在目标被遮挡或误检的情况下，利用车辆状态管理策略实现目标的重新匹配和误检结果筛出。本发明所述的跟踪系 统，基于所述跟踪方法，结合车云协同系统支持跟踪系统在线调校和升级。因此，本发明的优点如下：
(1)提出一种智能3D多目标跟踪方法，包括设计基于目标运动和几何信息的数据关联方法，实现快速且精准的多目标跟踪；设计自适应的目标状态管理方法，降低跟踪目标误匹配，以及有效解决车辆短期遮挡导致跟踪中断问题。
(2)基于所提出的智能3D多目标跟踪方法，结合车云协同系统，提出支持在线更新的多目标实时跟踪系统，使系统支持在线远程调校升级，可供长期稳定使用。
在一些可选择的实施例中，在方框图中提到的功能/操作可以不按照操作示图提到的顺序发生。例如，取决于所涉及的功能/操作，连续示出的两个方框实际上可以被大体上同时地执行或所述方框有时能以相反顺序被执行。此外，在本发明的流程图中所呈现和描述的实施例以示例的方式被提供，目的在于提供对技术更全面的理解。所公开的方法不限于本文所呈现的操作和逻辑流程。可选择的实施例是可预期的，其中各种操作的顺序被改变以及其中被描述为较大操作的一部分的子操作被独立地执行。
此外，虽然在功能性模块的背景下描述了本发明，但应当理解的是，除非另有相反说明，所述的功能和/或特征中的一个或多个可以被集成在单个物理装置和/或软件模块中，或者一个或多个功能和/或特征可以在单独的物理装置或软件模块中被实现。还可以理解的是，有关每个模块的实际实现的详细讨论对于理解本发明是不必要的。更确切地说，考虑到在本文中公开的装置中各种功能模块的属性、功能和内部关系的情况下，在工程师的常规技术内将会了解该模块的实际实现。因此，本领域技术人员运用普通技术就能够在无需过度试验的情况下实现在权利要求书中所阐明的本发明。还可以理解的是，所公开的特定概念仅仅是说明性的，并不意在限制本发明的范围，本发明的范围由所附权利要求书及其等同方案的全部范围来决定。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现 逻辑功能的可执行指令的定序列表，可以具体实现在任何计算机可读介质中，以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统)使用，或结合这些指令执行系统、装置或设备而使用。就本说明书而言，“计算机可读介质”可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。
计算机可读介质的更具体的示例(非穷尽性列表)包括以下：具有一个或多个布线的电连接部(电子装置)、便携式计算机盘盒(磁装置)、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编辑只读存储器(EPROM或闪速存储器)、光纤装置以及便携式光盘只读存储器(CDROM)。另外，计算机可读介质甚至可以是可在其上打印所述程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序，然后将其存储在计算机存储器中。
应当理解，本发明的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。例如，如果用硬件来实现，和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或他们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施例或示例中以合适的方式结合。
尽管已经示出和描述了本发明的实施例，本领域的普通技术人员可以理解：在不脱离本发明的原理和宗旨的情况下可以对这些实施例进行多种变化、修改、替换和变型，本发明的范围由权利要求及其等同物限定。
以上是对本发明的较佳实施进行了具体说明，但本发明并不限于所述实施例，熟悉本领域的技术人员在不违背本发明精神的前提下还可做出种种的等同变形或替换，这些等同的变形或替换均包含在本申请权利要求所限定的范围内。

Claims (10)

  1. 一种车云协同的智能3D多目标跟踪方法，其特征在于，包括：
    对车辆目标进行检测，获取初始时刻场景下各个车辆的检测状态，进而根据所述检测状态和车辆身份建立初始轨迹信息；其中，所述检测状态包括车辆检测框信息、检测置信度分数；
    根据所述初始时刻场景下的跟踪结果的目标信息，通过卡尔曼滤波算法预测目标时刻的状态；
    对车辆对象进行检测，获取目标时刻场景下各个车辆的检测状态，进而计算目标时刻下所有检测对象的检测框信息与初始时刻下目标的预测状态之间的数据关联度；
    根据所述数据关联度，对所有检测对象和预测对象进行相似度匹配，得到匹配结果；
    对所述匹配结果进行目标状态管理，直至完成跟踪目标。
  2. 根据权利要求1所述的一种车云协同的智能3D多目标跟踪方法，其特征在于，所述对车辆对象进行检测，获取目标时刻场景下各个车辆的检测状态，进而计算目标时刻下所有检测对象的检测框信息与初始时刻下目标的预测状态之间的数据关联度，包括：
    根据车辆对象在前后时刻的检测状态，估计车辆对象的速度；
    根据前后时刻的估计速度及检测角度信息，确定前后时刻运动差异的度量；
    根据所述目标时刻场景下的检测信息和预测状态，确定几何差异的度量；
    根据所述运动差异的度量和所述几何差异的度量，确定前后时刻的目标的相似性度量；
    其中，所述运动差异包括前后时刻之间速度的角度差异、前后时刻之间速度的数值差异、前后时刻之间车辆朝向的偏移。
  3. 根据权利要求2所述的一种车云协同的智能3D多目标跟踪方法，其特征在于，所述根据所述数据关联度，对所有检测对象和预测对象进行相似度匹配，得到匹配结果这一步骤具体为：根据所有目标的相似性度量，生成跟踪匹配结果，包括：
    获取所有检测目标的相似性度量；
    根据所述相似性度量，利用匈牙利算法对前后时刻的所有检测目标进行匹配，得到跟踪结果。
  4. 根据权利要求2所述的一种车云协同的智能3D多目标跟踪方法，其特征在于，所述车辆对象的预估的速度的计算公式为：
    其中，表示车辆对象的预估速度；t表示时刻；(xt,yt)表示表示t时刻下检测目标的空间位置；表示空间位置向量对时间求微分操作。
  5. 根据权利要求4所述的一种车云协同的智能3D多目标跟踪方法，其特征在于，所述前后时刻运动差异的度量的表达式为：
    其中，SM表示前后时刻运动差异的度量；表示车辆对象在t时刻的预估速度；表示车辆对象在t-1时刻的预估速度；θt表示车辆的朝向角度。
  6. 根据权利要求1所述的一种车云协同的智能3D多目标跟踪方法，其特征在于，所述对所述匹配结果进行目标状态管理，包括：
    获取数据关联阶段的匹配结果；
    当匹配成功时，将车辆对象的状态配置为Tracking，并为检测目标与预测目标赋予相同的身份标识；
    当匹配失败时，如果检测目标来自在前时刻，则对检测目标的生命周期进行更新；如果目标来自在后时刻，则确认目标刚出现在跟踪视野内，对目标的身份标识、生命周期和状态进行配置；
    根据所述检测目标的生命周期完成对目标的跟踪任务。
  7. 根据权利要求6所述的一种车云协同的智能3D多目标跟踪方法，其特征在于，所述根据所述检测目标的生命周期完成对目标的跟踪任务，包括：
    当检测目标的生命周期小于1时，将检测目标的状态配置为Death；
    当检测目标的生命周期大于或等于1时，将检测目标的状态配置为Miss。
  8. 根据权利要求7所述的一种车云协同的智能3D多目标跟踪方法，其特征在于，所述匹配失败的目标的生命周期的计算公式为：
    其中，life′表示匹配失败的目标的生命周期；life表示固定的生命周期阈值；α代表尺度因子；β代表偏移因子；Ct代表目标被成功检测的置信度分数。
  9. 一种车云协同的智能3D多目标跟踪系统，其特征在于，包括：
    第一模块，用于对车辆目标进行检测，获取初始时刻场景下各个车辆的检测状态，进而 根据所述检测状态和车辆身份建立初始轨迹信息；其中，所述检测状态包括车辆检测框信息、检测置信度分数；
    第二模块，用于根据所述初始时刻场景下的跟踪结果的目标信息，通过卡尔曼滤波算法预测目标时刻的状态；
    第三模块，用于对车辆对象进行检测，获取目标时刻场景下各个车辆的检测状态，进而计算目标时刻下所有检测对象的检测框信息与初始时刻下目标的预测状态之间的数据关联度；
    第四模块，用于根据所述数据关联度，对所有检测对象和预测对象进行相似度匹配，得到匹配结果；
    第五模块，用于对所述匹配结果进行目标状态管理，直至完成跟踪目标。
  10. 一种电子设备，其特征在于，包括处理器以及存储器；
    所述存储器用于存储程序；
    所述处理器执行所述程序实现如权利要求1至8中任一项所述的方法。
PCT/CN2023/118281 2023-02-01 2023-09-12 一种车云协同的智能3d多目标跟踪方法及系统 Ceased WO2024159751A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310077195.9 2023-02-01
CN202310077195.9A CN116259015B (zh) 2023-02-01 2023-02-01 一种车云协同的智能3d多目标跟踪方法及系统

Publications (1)

Publication Number Publication Date
WO2024159751A1 true WO2024159751A1 (zh) 2024-08-08

Family

ID=86687352

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/118281 Ceased WO2024159751A1 (zh) 2023-02-01 2023-09-12 一种车云协同的智能3d多目标跟踪方法及系统

Country Status (2)

Country Link
CN (1) CN116259015B (zh)
WO (1) WO2024159751A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120526553A (zh) * 2025-07-24 2025-08-22 华诺星空技术股份有限公司 基于多维特征融合的光纤传感信号识别与跟踪方法及系统
CN120634986A (zh) * 2025-05-29 2025-09-12 哈尔滨工业大学 一种基于三维探地雷达的三视图地下管线智能检测方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116259015B (zh) * 2023-02-01 2026-03-20 中山大学 一种车云协同的智能3d多目标跟踪方法及系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110085702A1 (en) * 2009-10-08 2011-04-14 University Of Southern California Object tracking by hierarchical association of detection responses
US20190325223A1 (en) * 2018-04-18 2019-10-24 Baidu Usa Llc Tracking objects with multiple cues
CN111932580A (zh) * 2020-07-03 2020-11-13 江苏大学 一种基于卡尔曼滤波与匈牙利算法的道路3d车辆跟踪方法及系统
CN114882260A (zh) * 2022-05-31 2022-08-09 济南大学 一种图匹配方法及系统
CN115205333A (zh) * 2022-06-29 2022-10-18 南京航空航天大学 一种对热红外影像行人目标跟踪的方法
CN116259015A (zh) * 2023-02-01 2023-06-13 中山大学 一种车云协同的智能3d多目标跟踪方法及系统

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10685244B2 (en) * 2018-02-27 2020-06-16 Tusimple, Inc. System and method for online real-time multi-object tracking
CN114972418B (zh) * 2022-03-30 2023-11-21 北京航空航天大学 基于核自适应滤波与yolox检测结合的机动多目标跟踪方法
CN115511920A (zh) * 2022-07-12 2022-12-23 南京华康智能科技有限公司 一种基于DeepSort和DeepEMD的检测跟踪方法和系统
CN115457497A (zh) * 2022-09-19 2022-12-09 东南大学 一种基于3d目标检测和多目标追踪检测车辆速度的方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110085702A1 (en) * 2009-10-08 2011-04-14 University Of Southern California Object tracking by hierarchical association of detection responses
US20190325223A1 (en) * 2018-04-18 2019-10-24 Baidu Usa Llc Tracking objects with multiple cues
CN111932580A (zh) * 2020-07-03 2020-11-13 江苏大学 一种基于卡尔曼滤波与匈牙利算法的道路3d车辆跟踪方法及系统
CN114882260A (zh) * 2022-05-31 2022-08-09 济南大学 一种图匹配方法及系统
CN115205333A (zh) * 2022-06-29 2022-10-18 南京航空航天大学 一种对热红外影像行人目标跟踪的方法
CN116259015A (zh) * 2023-02-01 2023-06-13 中山大学 一种车云协同的智能3d多目标跟踪方法及系统

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
CHENG, XIAOQIANG: "LiDAR-based 3D Object Detection and Multiple Object Tracking for Unmanned Vehicles [Unmanned driving 3D target detection and multi-target tracking based on LiDAR]", ZHANGQIAOKEYAN, 1 October 2020 (2020-10-01), CN, pages 1 - 70, XP009558480, DOI: 10.27356/d.cnki.gtjdu.2020.004498 *
HENSCHEL ROBERTO, LEAL-TAIX LAURA, ROSENHAHN BODO, SCHINDLER KONRAD, LEAL-TAIXÉ L, SCHINDLER K : "Tracking with multi-level features", ARXIV.ORG, 25 July 2016 (2016-07-25), XP093197075 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120634986A (zh) * 2025-05-29 2025-09-12 哈尔滨工业大学 一种基于三维探地雷达的三视图地下管线智能检测方法
CN120526553A (zh) * 2025-07-24 2025-08-22 华诺星空技术股份有限公司 基于多维特征融合的光纤传感信号识别与跟踪方法及系统

Also Published As

Publication number Publication date
CN116259015A (zh) 2023-06-13
CN116259015B (zh) 2026-03-20

Similar Documents

Publication Publication Date Title
WO2024159751A1 (zh) 一种车云协同的智能3d多目标跟踪方法及系统
CN112991389B (zh) 一种目标跟踪方法、装置及移动机器人
CN111209978B (zh) 三维视觉重定位方法、装置及计算设备、存储介质
CN106774345B (zh) 一种进行多机器人协作的方法与设备
CN115965657B (zh) 目标跟踪方法、电子设备、存储介质及车辆
WO2019119328A1 (zh) 一种基于视觉的定位方法及飞行器
US12277721B2 (en) Method for optimizing depth estimation model, computer device, and storage medium
CN113112525A (zh) 目标跟踪方法、网络模型及其训练方法、设备和介质
Butt et al. Camera calibration through camera projection loss
US20230281867A1 (en) Methods performed by electronic devices, electronic devices, and storage media
JP6194995B2 (ja) 運動予測の最適化方法、装置及びシステム
CN116309719B (zh) 目标跟踪方法、装置、计算机设备和存储介质
CN107909024B (zh) 基于图像识别和红外避障的车辆跟踪系统、方法及车辆
Ryoo et al. MambaGlue: Fast and robust local feature matching with Mamba
CN113325415B (zh) 车辆雷达数据和相机数据的融合方法及系统
CN120071300B (zh) 一种多目标区域动态车辆行人识别方法及系统
Li et al. A neuro-inspired visual SLAM approach using AKAZE feature extraction in complex and dynamic environments
CN113516013B (zh) 目标检测方法、装置、电子设备、路侧设备和云控平台
WO2023273467A1 (zh) 真值数据确定、神经网络训练、行驶控制方法及装置
CN117150881B (zh) 仿生机器鱼导航任务的强化学习训练回路方法及装置
Xie et al. Multi-modal 3D multi-object tracking with robust association and track drift compensation
CN114973236B (zh) 一种对点云目标识别结果进行调整的处理方法
US20260065659A1 (en) Method for processing sensor data, object detection network, and method for adapting an object detection network
CN118439052A (zh) 一种智能驾驶感知方法、系统、存储介质及电子设备
CN120612554B (zh) 目标检测方法和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23919315

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 10-11-2025)