CN115965657A - Target tracking method, electronic device, storage medium, and vehicle - Google Patents
Target tracking method, electronic device, storage medium, and vehicle Download PDFInfo
- Publication number
- CN115965657A CN115965657A CN202310171954.8A CN202310171954A CN115965657A CN 115965657 A CN115965657 A CN 115965657A CN 202310171954 A CN202310171954 A CN 202310171954A CN 115965657 A CN115965657 A CN 115965657A
- Authority
- CN
- China
- Prior art keywords
- frame
- predicted
- track
- queue
- current frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Image Analysis (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The invention relates to the technical field of computers, in particular provides a target tracking method, electronic equipment, a storage medium and a vehicle, and aims to solve the technical problem that the accuracy of the existing target tracking method is poor. To this end, the target tracking method of the present invention includes: acquiring sensor data acquired by a vehicle-mounted sensor; inputting sensor data into a network model, and outputting a current frame detection frame and current frame predicted track information of at least one detection target; and tracking at least one detection target based on the current frame detection frame, the current frame predicted track information and the historical frame predicted track information to obtain a tracking result. Therefore, the accuracy and the stability of target tracking are improved.
Description
Technical Field
The invention relates to the technical field of computers, and particularly provides a target tracking method, electronic equipment, a storage medium and a vehicle.
Background
At present, the function of advanced assistant driving is more and more concerned, and multi-target tracking is a basic ring in an advanced assistant driving system. Most of the commonly used target tracking methods are optical flow methods and filtering algorithms, wherein the optical flow methods are greatly influenced by the environment, the tracking accuracy in a complex environment is poor, and the tracking accuracy of the filtering algorithms under the conditions of target form change, shielding or target disappearance and the like is poor.
Accordingly, there is a need in the art for a new target tracking solution to address the above-mentioned problems.
Disclosure of Invention
The present invention has been made to overcome the above-mentioned drawbacks, and provides a solution or at least a partial solution to the above-mentioned technical problem. The invention provides a target tracking method, electronic equipment, a storage medium and a vehicle.
In a first aspect, the present invention provides a method of target tracking, the method comprising: acquiring sensor data acquired by a vehicle-mounted sensor; inputting the sensor data into a network model, and outputting a current frame detection frame and current frame predicted track information of at least one detection target; and tracking the at least one detection target based on the current frame detection frame, the current frame prediction track information and the historical frame prediction track information to obtain a tracking result.
In one embodiment, the tracking the at least one detection target based on the current frame detection frame, the current frame prediction track information, and the historical frame prediction track information to obtain a tracking result includes: creating a tracker, and initializing a predicted trajectory queue based on the tracker; adding the current frame predicted track information to the initialized predicted track queue; judging whether the predicted track queue added with the current frame predicted track information only has the current frame predicted track information; if so, giving a new track ID to the detection target; if not, the tracker is used for matching the current frame detection frame with historical frame prediction track information in the prediction track queue, and the track ID of the detection target is determined according to the matching result.
In one embodiment, the matching of the current frame detection frame with the historical frame predicted track information in the predicted track queue by using the tracker and the determination of the track ID of the detection target according to the matching result comprise the steps of matching the current frame detection frame with a predicted frame of the 1 st track point of the previous historical frame, wherein the predicted frame of the 1 st track point of the previous historical frame is obtained based on the predicted track information of the previous historical frame; if the matching is successful, taking the track ID corresponding to the predicted track information of the previous historical frame as the track ID of the detection target; otherwise, continuously matching the current frame with other historical frames before the last historical frame in the prediction track queue frame by frame until the current frame detection frame is successfully matched with one frame of historical frame prediction track information in other historical frames, stopping matching, and taking the track ID corresponding to the one frame of historical frame prediction track information which is successfully matched as the track ID of the detection target.
In one embodiment, said continuously matching said current frame with other historical frames in said prediction trajectory queue before said last historical frame on a frame-by-frame basis includes: and taking the current frame as the t-th frame, and matching the current frame detection frame with the prediction frame of the ith track point of the t-i-th frame, wherein the prediction frame of the ith track point of the t-i-th frame is obtained based on the predicted track information of the t-i-th frame, t and i are positive integers, and 1 & lti & lt t.
In one embodiment, the matching, by the tracker, the current frame detection frame and historical frame predicted track information in the predicted track queue, and determining the track ID of the detection target according to a matching result includes: and under the condition that the current frame detection frame is unsuccessfully matched with all the historical frame predicted track information in the predicted track queue, giving a new track ID to the detection target.
In one embodiment, the initializing a predicted trajectory queue based on the tracker includes: setting a queue length threshold for the predicted trajectory queue and setting a time length threshold for the predicted trajectory queue based on the tracker; the determining whether the predicted trajectory queue added with the current frame predicted trajectory information is only before the current frame predicted trajectory information, the method further includes: performing effective frame check on the predicted track queue added with the current frame predicted track information based on the time length threshold, and selectively updating the predicted track queue based on a check result; and/or selectively updating the predicted trajectory queue based on the queue length threshold.
In one embodiment, performing valid frame check on the predicted track queue after adding the current frame predicted track information based on the time length threshold, and selectively updating the predicted track queue based on a check result includes: acquiring a first time stamp corresponding to current frame predicted track information and a second time stamp corresponding to previous historical frame predicted track information in the predicted track queue; determining a difference between the first timestamp and the second timestamp; judging whether the difference value is smaller than a time length threshold value or not; and if so, updating the predicted track queue, and if not, not updating the predicted track queue.
In one embodiment, said selectively updating said queue of predicted trajectories based on said queue length threshold comprises: and deleting the historical frame farthest from the current frame predicted track information under the condition that the length of the predicted track queue added with the current frame predicted track information exceeds the queue length threshold.
In one embodiment, before matching the current frame detection box with the historical frame prediction track information in the prediction track queue using the tracker, the method further comprises: and converting the historical frame predicted track information into a vehicle coordinate system corresponding to the current frame.
In a second aspect, an electronic device is provided, comprising at least one processor and at least one storage means adapted to store a plurality of program codes, said program codes being adapted to be loaded and run by said processor to perform the target tracking method of any of the preceding claims.
In a third aspect, a computer readable storage medium is provided, having stored therein a plurality of program codes adapted to be loaded and executed by a processor to perform the object tracking method of any of the preceding claims.
In a fourth aspect, a vehicle is provided, which comprises the aforementioned electronic device.
One or more technical schemes of the invention at least have one or more of the following beneficial effects:
the target tracking method of the invention acquires sensor data acquired by a vehicle-mounted sensor; inputting sensor data into a detection model, and outputting a current frame detection frame and current frame predicted track information of at least one detection target; and tracking at least one detection target based on the current frame detection frame, the current frame prediction track information and the historical frame prediction track information to obtain a tracking result. Therefore, the target is subjected to time sequence tracking by utilizing the predicted track, and the accuracy and the stability of target tracking are improved under the condition of ensuring the tracking speed.
Drawings
The present disclosure will become more readily understood with reference to the accompanying drawings. As is readily understood by those skilled in the art: these drawings are for illustrative purposes only and are not intended to constitute a limitation on the scope of the present invention. Moreover, in the drawings, like numerals are used to indicate like parts, and in which:
FIG. 1 is a flow diagram illustrating the main steps of a target tracking method according to one embodiment of the present invention;
FIG. 2 is a schematic flow chart of tracking at least one detection target in one embodiment;
FIG. 3 is a schematic diagram of a complete flow chart of a target tracking method in one embodiment;
FIG. 4 is a schematic diagram of the electronic device in one embodiment.
Detailed Description
Some embodiments of the invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only for explaining the technical principle of the present invention, and are not intended to limit the scope of the present invention.
In the description of the present invention, a "module" or "processor" may include hardware, software, or a combination of both. A module may comprise hardware circuitry, various suitable sensors, communication ports, memory, may comprise software components such as program code, or may be a combination of software and hardware. The processor may be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and/or signal processing functionality. The processor may be implemented in software, hardware, or a combination thereof. Non-transitory computer-readable storage media include any suitable medium that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random-access memory, and so forth. The term "a and/or B" denotes all possible combinations of a and B, such as a alone, B alone or a and B. The term "at least one A or B" or "at least one of A and B" means similar to "A and/or B" and may include only A, only B, or both A and B. The singular forms "a", "an" and "the" may include plural forms as well.
At present, most of traditional target tracking methods are optical flow methods and filtering algorithms, wherein the optical flow methods are greatly influenced by the environment, the tracking accuracy in a complex environment is poor, and the tracking accuracy of the filtering algorithms under the conditions of target form change, shielding or target disappearance and the like is poor.
Therefore, the application provides a target tracking method, electronic equipment, a storage medium and a vehicle, and the method is used for acquiring sensor data acquired by a vehicle-mounted sensor; inputting sensor data into a detection model, and outputting a current frame detection frame and current frame predicted track information of at least one detection target; and tracking at least one detection target based on the current frame detection frame, the current frame prediction track information and the historical frame prediction track information to obtain a tracking result. Therefore, the target is subjected to time sequence tracking by utilizing the predicted track, and the accuracy and the stability of target tracking are improved under the condition of ensuring the tracking speed.
Referring to fig. 1, fig. 1 is a flow chart illustrating main steps of a target tracking method according to an embodiment of the invention.
As shown in fig. 1, the target tracking method in the embodiment of the present invention mainly includes the following steps S101 to S103.
Step S101: and acquiring sensor data acquired by the vehicle-mounted sensor.
Step S102: and inputting the sensor data into the network model, and outputting a current frame detection frame and current frame predicted track information of at least one detection target.
Step S103: and tracking at least one detection target based on the current frame detection frame, the current frame prediction track information and the historical frame prediction track information to obtain a tracking result.
Based on the steps S101 to S103, acquiring sensor data acquired by a vehicle-mounted sensor; inputting sensor data into a detection model, and outputting a current frame detection frame and current frame predicted track information of at least one detection target; and tracking at least one detection target based on the current frame detection frame, the current frame prediction track information and the historical frame prediction track information to obtain a tracking result. Therefore, the target is subjected to time sequence tracking by utilizing the predicted track, and the accuracy and the stability of target tracking are improved under the condition of ensuring the tracking speed.
The following further describes steps S101 to S103.
In step S101, the in-vehicle sensor may be any one of a camera and a laser radar. The sensor data collected by the vehicle-mounted sensor can be a video frame image or a point cloud image collected by a laser radar.
In addition, the acquired sensor image collected by the vehicle-mounted sensor may be a frame image or a multi-frame image, for example, sensor data collected by sensors with different viewing angles. The input information comprises multi-frame data, the stability of the detection effect and the accuracy of the predicted track can be improved, but the requirement of tracking the multi-frame data on time cost and calculation power is high. In this embodiment, it is preferable to acquire one frame of sensor acquisition data.
The above is a further description of step S101, and the following is a further description of step S102.
The network model is an end-to-end network model that includes at least the tasks of detection and prediction. Inputting sensor data into the network model, and outputting a current frame detection frame and current frame predicted track information of at least one detection target corresponding to the sensor data through main network feature extraction and a multi-task detection head and track prediction head. The predicted track information includes track point coordinates, orientation angles and the like at future time.
IntentNet may be used as an example of the network model, but is not limited thereto.
The above is a further description of step S102, and the following is a further description of step S103.
Specifically, the above step S103 may be realized by the following steps S1031 to S1035.
Step S1031: a tracker is created, based on which a predicted trajectory queue is initialized.
The tracker is an object, and the process of creating the tracker is the process of creating a tracker object and initializing the tracker object.
The prediction track queue is used for storing current frame information and historical frame information, such as current frame prediction track information, historical frame detection boxes and the like.
In a specific embodiment, the initializing a predicted trajectory queue based on the tracker includes: setting a queue length threshold for the predicted trajectory queue and setting a time length threshold for the predicted trajectory queue based on the tracker; the determining whether the predicted trajectory queue added with the current frame predicted trajectory information is only before the current frame predicted trajectory information further includes: performing effective frame check on the predicted track queue added with the current frame predicted track information based on the time length threshold, and selectively updating the predicted track queue based on a check result; and/or selectively updating the predicted trajectory queue based on the queue length threshold.
Updating the prediction track queue can be deleting the historical frame information which does not meet the valid frame check or the queue length threshold value, or acquiring new historical frame information from the historical frame information set and storing the new historical frame information in the prediction track queue.
In a specific embodiment, performing valid frame check on the predicted track queue after the current frame predicted track information is added based on the time length threshold, and selectively updating the predicted track queue based on a check result includes: acquiring a first time stamp corresponding to current frame predicted track information and a second time stamp corresponding to previous historical frame predicted track information in the predicted track queue; determining a difference between the first timestamp and the second timestamp; judging whether the difference value is smaller than a time length threshold value or not; if yes, updating the predicted track queue, and if not, not updating the predicted track queue.
Specifically, each frame of information has a corresponding timestamp, and performing valid frame check on the predicted track queue refers to checking whether a difference value between a timestamp of a t-th frame (current frame) and a timestamp of a t-1-th frame is within a reasonable interval. Specifically, the current frame predicted track information corresponds to a first time stamp, the previous historical frame predicted track information corresponds to a second time stamp, a difference value between the first time stamp and the second time stamp is calculated according to the first time stamp and the second time stamp, and whether the difference value is within a time length threshold range is judged; if yes, updating the predicted track queue, and if not, not updating the predicted track queue.
By carrying out effective frame check on the predicted track queue, the difference value between the timestamp of the previous historical frame and the timestamp of the current frame can be ensured to be always in a reasonable time range, so that the accuracy of target tracking is improved.
In a specific embodiment, the selectively updating the predicted trajectory queue based on the queue length threshold includes: and deleting the historical frame farthest from the current frame predicted track information under the condition that the length of the predicted track queue added with the current frame predicted track information exceeds the queue length threshold.
And deleting the historical frame farthest from the current frame information under the condition that the length of the maintained prediction track queue exceeds a queue length threshold value.
The queue length threshold is used for maintaining a predicted track queue and maintaining the predicted track queue with a certain length, and the tracking effect under the condition of shielding or object disappearance can be improved.
Step S1032: and adding the current frame predicted track information to the initialized predicted track queue.
In one embodiment, the detection box information of the current frame may also be added to the initialized predicted trajectory queue. Or reconstructing a detection frame queue storing detection frames to store the current frame detection frame in the detection frame queue.
Step S1033: and judging whether the predicted track queue added with the current frame predicted track information only has the current frame predicted track information.
Generally speaking, when performing initial tracking, only the current frame predicted track information is in the predicted track queue. And tracking the target object in the subsequent process, wherein current frame predicted track information and historical frame predicted track information exist in the predicted track queue at the same time.
Step S1034: and if so, giving a new track ID to the detection target.
Step S1035: if not, the tracker is used for matching the current frame detection frame with historical frame predicted track information in the predicted track queue, and the track ID of the detection target is determined according to the matching result.
Track ID (track ID), specifically target tracking ID.
Specifically, matching of the current frame detection frame with the historical frame prediction track information in the prediction track queue mainly includes performing frame-by-frame matching on the current frame detection frame and a prediction frame corresponding to each frame of historical frame prediction track information, and thus determining whether a historical frame which can be successfully matched with the current frame detection frame exists in the prediction track queue.
In a specific embodiment, the matching of the current frame detection frame with the historical frame predicted track information in the predicted track queue by using the tracker and the determination of the track ID of the detection target according to the matching result include matching the current frame detection frame with the predicted frame of the 1 st track point of the previous historical frame, wherein the predicted frame of the 1 st track point of the previous historical frame is obtained based on the predicted track information of the previous historical frame; if the matching is successful, taking a track ID corresponding to the previous historical frame predicted track information as a track ID of the detection target; otherwise, continuously matching the current frame with other historical frames before the previous historical frame in the prediction track queue frame by frame until the current frame detection frame is successfully matched with one frame of historical frame prediction track information in other historical frames, stopping matching, and taking the track ID corresponding to the successfully matched one frame of historical frame prediction track information as the track ID of the detection target.
Each frame of historical frame predicted track information comprises a plurality of track points and the orientation of each track point. And combining the orientation of each track point and the direction of the track point with the detection frame of the frame to obtain a prediction frame corresponding to each track point of each frame.
Specifically, firstly, matching a detection frame of a current frame with a prediction frame of a 1 st track point of a previous historical frame, and if the matching is successful, taking a track ID corresponding to predicted track information of the previous historical frame as a track ID of a detection target; otherwise, continuously matching the current frame with other historical frames before the last historical frame in the prediction track queue frame by frame until the current frame detection frame is successfully matched with one frame of historical frame prediction track information in other historical frames, stopping matching, and taking the track ID corresponding to the successfully matched one frame of historical frame prediction track information as the track ID of the detection target.
When a plurality of historical frame predicted track information exists, a plurality of predicted frames exist at the moment, the detection frame and the plurality of predicted frames need to be matched respectively, specifically, the cross-over ratio Iou between the detection frame and each predicted frame is calculated, at least one cross-over ratio meeting the conditions is selected from the cross-over ratios, and then the predicted frame which can be successfully matched with the detection frame is determined by using a Hungary matching algorithm.
In a specific embodiment, said continuously matching, on a frame-by-frame basis, the current frame with other historical frames before the last historical frame in the prediction trajectory queue includes: and taking the current frame as the t-th frame, and matching the current frame detection frame with the prediction frame of the ith track point of the t-i-th frame, wherein the prediction frame of the ith track point of the t-i-th frame is obtained based on the predicted track information of the t-i-th frame, t and i are positive integers, and 1 & lti & lt t.
Specifically, as shown in fig. 2, when the current frame detection frame (t-th frame) fails to be matched with the previous history frame (t-1 th frame), the current frame detection frame is continuously matched with the prediction frame of the second track point of the t-2 th frame, and if the matching is successful, the track ID corresponding to the predicted track information of the t-2 th frame is used as the track ID of the detection target. And if the matching fails, continuing to match the current frame detection frame with the t-3 th frame, repeating the step until the current frame detection frame is successfully matched with the historical frame in the predicted track queue, and outputting the track ID of the detection target.
In a specific embodiment, the matching, by using the tracker, the current frame detection frame and the historical frame predicted track information in the predicted track queue, and determining the track ID of the detection target according to the matching result includes: and under the condition that the current frame detection frame is unsuccessfully matched with all the historical frame predicted track information in the predicted track queue, giving a new track ID to the detection target.
Specifically, when the current frame detection frame fails to match all the history frames in the predicted trajectory queue, a new trajectory ID is given to the detection target, which indicates that the detection target is a newly-appeared target.
The method and the device solve the problems of morphological change, shielding, disappearance and the like of a detected target in the tracking process to a certain extent by maintaining the predicted track queue and matching the current frame information with the historical predicted information to obtain the tracked target, thereby outputting the stable tracked target.
In an embodiment, as shown in fig. 3 in particular, after the tracker is built and before the target tracking, data preprocessing may be further performed on the current frame detection frame and the current frame predicted trajectory information of the at least one detection target output in step S102. Illustratively, the detection frame information and the predicted track information contain the category and the confidence of the frame, confidence thresholds of different categories are set, and the detection frame and the predicted track with lower confidence are filtered. In addition, in multi-modal prediction of the trajectory, that is, in a case where a plurality of trajectories are predicted in one frame, a predicted trajectory with the highest confidence needs to be selected. Therefore, the complexity of target tracking is reduced, and the efficiency and accuracy of target tracking are improved.
In one embodiment, before the tracker is used to match the current frame detection box with the historical frame prediction track information in the prediction track queue, the method further includes: and converting the historical frame prediction track information into a vehicle coordinate system corresponding to the current frame.
Specifically, before matching, a coordinate transformation matrix for converting historical frame information into a current frame can be determined according to the self-vehicle positioning information, and then the historical frame predicted track information is converted into a vehicle coordinate system corresponding to the current frame according to the coordinate transformation matrix. Therefore, the matching of the current frame detection frame and the historical frame information is facilitated.
It should be noted that, although the foregoing embodiments describe each step in a specific sequence, those skilled in the art will understand that, in order to achieve the effect of the present invention, different steps do not necessarily need to be executed in such a sequence, and they may be executed simultaneously (in parallel) or in other sequences, and these changes are all within the protection scope of the present invention.
It will be understood by those skilled in the art that all or part of the flow of the method of the above-described embodiment may be implemented by a computer program, which may be stored in a computer-readable storage medium, and the steps of the method embodiments may be implemented when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device capable of carrying said computer program code, media, usb disk, removable hard disk, magnetic diskette, optical disk, computer memory, read-only memory, random access memory, electrical carrier wave signals, telecommunication signals, software distribution media, etc. It should be noted that the computer readable storage medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable storage media that does not include electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
Furthermore, the invention also provides electronic equipment. In an embodiment of the electronic device according to the present invention, as shown in particular in fig. 4, the electronic device comprises at least one processor 41 and at least one storage device 42, the storage device may be configured to store a program for performing the object tracking method of the above-mentioned method embodiment, and the processor may be configured to execute the program in the storage device, the program including but not limited to the program for performing the object tracking method of the above-mentioned method embodiment. For convenience of explanation, only the parts related to the embodiments of the present invention are shown, and details of the specific techniques are not disclosed.
The electronic device in the embodiment of the present invention may be a control apparatus device including various devices. In some possible implementations, an electronic device may include multiple storage devices and multiple processors. The program for executing the target tracking method of the above method embodiment may be divided into a plurality of sub programs, and each sub program may be loaded and executed by a processor to execute different steps of the target tracking method of the above method embodiment. Specifically, each piece of sub program may be stored in a different storage device, and each processor may be configured to execute the program in one or more storage devices to implement the target tracking method of the foregoing method embodiment together, that is, each processor executes different steps of the target tracking method of the foregoing method embodiment to implement the target tracking method of the foregoing method embodiment together.
The multiple processors may be processors disposed on the same device, for example, the electronic device may be a high-performance device composed of multiple processors, and the multiple processors may be processors configured on the high-performance device. In addition, the multiple processors may also be processors disposed on different devices, for example, the electronic device may be a server cluster, and the multiple processors may be processors on different servers in the server cluster.
Further, the invention also provides a computer readable storage medium. In one computer-readable storage medium embodiment according to the present invention, a computer-readable storage medium may be configured to store a program that executes the target tracking method of the above-described method embodiment, and the program may be loaded and executed by a processor to implement the above-described target tracking method. For convenience of explanation, only the parts related to the embodiments of the present invention are shown, and details of the specific techniques are not disclosed. The computer readable storage medium may be a storage device formed by including various electronic devices, and optionally, the computer readable storage medium is a non-transitory computer readable storage medium in the embodiment of the present invention.
Further, the invention also provides a vehicle which comprises the electronic equipment.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (12)
1. A method of target tracking, the method comprising:
acquiring sensor data acquired by a vehicle-mounted sensor;
inputting the sensor data into a network model, and outputting a current frame detection frame and current frame predicted track information of at least one detection target;
and tracking the at least one detection target based on the current frame detection frame, the current frame prediction track information and the historical frame prediction track information to obtain a tracking result.
2. The target tracking method according to claim 1, wherein the tracking the at least one detected target based on the current frame detection box, the current frame predicted trajectory information and historical frame predicted trajectory information to obtain a tracking result comprises:
creating a tracker, and initializing a predicted trajectory queue based on the tracker;
adding the current frame predicted track information to the initialized predicted track queue;
judging whether the predicted track queue added with the current frame predicted track information only has the current frame predicted track information;
if so, giving a new track ID to the detection target;
if not, the tracker is utilized to match the current frame detection frame with historical frame predicted track information in the predicted track queue, and the track ID of the detection target is determined according to a matching result.
3. The target tracking method according to claim 2, wherein the matching, by the tracker, the current frame detection frame with the historical frame predicted track information in the predicted track queue, and determining the track ID of the detection target according to a matching result comprises:
matching the current frame detection frame with a prediction frame of the 1 st track point of the previous historical frame, wherein the prediction frame of the 1 st track point of the previous historical frame is obtained based on the predicted track information of the previous historical frame;
if the matching is successful, taking a track ID corresponding to the previous historical frame predicted track information as a track ID of the detection target;
otherwise, continuously matching the current frame with other historical frames before the last historical frame in the prediction track queue frame by frame until the current frame detection frame is successfully matched with one frame of historical frame prediction track information in other historical frames, stopping matching, and taking the track ID corresponding to the one frame of historical frame prediction track information which is successfully matched as the track ID of the detection target.
4. The method of claim 3, wherein said continuously matching said current frame with other historical frames in said predicted trajectory queue before said previous historical frame on a frame-by-frame basis comprises: and taking the current frame as a t frame, and matching the current frame detection frame with a prediction frame of the ith track point of the t-i frame, wherein the prediction frame of the ith track point of the t-i frame is obtained based on the predicted track information of the t-i frame, t and i are positive integers, and 1 & lti & ltt.
5. The target tracking method according to claim 4, wherein the matching, by the tracker, the current frame detection frame with the historical frame predicted track information in the predicted track queue and determining the track ID of the detection target according to the matching result comprises: and under the condition that the current frame detection frame is unsuccessfully matched with all the historical frame predicted track information in the predicted track queue, giving a new track ID to the detection target.
6. The method of claim 2, wherein initializing a predicted trajectory queue based on the tracker comprises:
setting a queue length threshold for the predicted trajectory queue and setting a time length threshold for the predicted trajectory queue based on the tracker;
the determining whether the predicted trajectory queue added with the current frame predicted trajectory information is only before the current frame predicted trajectory information, the method further includes:
performing effective frame check on the predicted track queue added with the current frame predicted track information based on the time length threshold, and selectively updating the predicted track queue based on a check result; and/or
Selectively updating the predicted trajectory queue based on the queue length threshold.
7. The object tracking method according to claim 6, wherein performing effective frame check on the predicted track queue after adding the current frame predicted track information based on the time length threshold, and selectively updating the predicted track queue based on a check result comprises:
acquiring a first time stamp corresponding to current frame predicted track information and a second time stamp corresponding to previous historical frame predicted track information in the predicted track queue;
determining a difference between the first timestamp and the second timestamp;
judging whether the difference value is smaller than the time length threshold value or not;
and if so, updating the predicted track queue, and if not, not updating the predicted track queue.
8. The method of claim 6, wherein selectively updating the predicted trajectory queue based on the queue length threshold comprises: and deleting the historical frame which is farthest away from the current frame predicted track information under the condition that the length of the predicted track queue after the current frame predicted track information is added exceeds the queue length threshold.
9. The target tracking method of claim 2, wherein before matching the current frame detection box with the historical frame predicted trajectory information in the predicted trajectory queue using the tracker, the method further comprises: and converting the historical frame prediction track information into a vehicle coordinate system corresponding to the current frame.
10. An electronic device comprising at least one processor and at least one storage means adapted to store a plurality of program codes, characterized in that said program codes are adapted to be loaded and run by said processor to perform the object tracking method of any of claims 1 to 9.
11. A computer readable storage medium having stored therein a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by a processor to perform the object tracking method of any of claims 1 to 9.
12. A vehicle characterized in that the vehicle comprises the electronic device of claim 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310171954.8A CN115965657B (en) | 2023-02-28 | 2023-02-28 | Target tracking method, electronic device, storage medium and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310171954.8A CN115965657B (en) | 2023-02-28 | 2023-02-28 | Target tracking method, electronic device, storage medium and vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115965657A true CN115965657A (en) | 2023-04-14 |
| CN115965657B CN115965657B (en) | 2023-06-02 |
Family
ID=85894647
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310171954.8A Active CN115965657B (en) | 2023-02-28 | 2023-02-28 | Target tracking method, electronic device, storage medium and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115965657B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117292306A (en) * | 2023-11-27 | 2023-12-26 | 四川迪晟新达类脑智能技术有限公司 | Edge equipment-oriented vehicle target detection optimization method and device |
| CN117351039A (en) * | 2023-12-06 | 2024-01-05 | 广州紫为云科技有限公司 | Nonlinear multi-target tracking method based on feature query |
Citations (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050185822A1 (en) * | 2004-02-20 | 2005-08-25 | James Slaski | Component association tracker system and method |
| CN102065275A (en) * | 2009-11-17 | 2011-05-18 | 中国科学院电子学研究所 | Multi-target tracking method in intelligent video monitoring system |
| US20190108613A1 (en) * | 2017-10-06 | 2019-04-11 | Ford Global Technologies, Llc | Fusion Of Motion And Appearance Features For Object Detection And Trajectory Prediction |
| CN110751674A (en) * | 2018-07-24 | 2020-02-04 | 北京深鉴智能科技有限公司 | Multi-target tracking method and corresponding video analysis system |
| CN112200830A (en) * | 2020-09-11 | 2021-01-08 | 山东信通电子股份有限公司 | Target tracking method and device |
| CN112435276A (en) * | 2020-11-13 | 2021-03-02 | 鹏城实验室 | Vehicle tracking method and device, intelligent terminal and storage medium |
| CN112507949A (en) * | 2020-12-18 | 2021-03-16 | 北京百度网讯科技有限公司 | Target tracking method and device, road side equipment and cloud control platform |
| CN112837349A (en) * | 2021-02-09 | 2021-05-25 | 普联技术有限公司 | Target tracking method, target tracking equipment and computer-readable storage medium |
| CN113033447A (en) * | 2021-04-02 | 2021-06-25 | 蔚来汽车科技(安徽)有限公司 | Method for tracking object in video frame sequence, automatic parking method and device thereof |
| WO2021134172A1 (en) * | 2019-12-30 | 2021-07-08 | 华为技术有限公司 | Trajectory prediction method and related device |
| CN113792697A (en) * | 2021-09-23 | 2021-12-14 | 重庆紫光华山智安科技有限公司 | Target detection method and device, electronic equipment and readable storage medium |
| CN114169241A (en) * | 2021-12-09 | 2022-03-11 | 北京邮电大学 | End-to-end multi-target identification, tracking and prediction method |
| CN114359341A (en) * | 2021-12-29 | 2022-04-15 | 湖南国科微电子股份有限公司 | Multi-target tracking method and device, terminal equipment and readable storage medium |
| CN114581491A (en) * | 2022-04-30 | 2022-06-03 | 苏州浪潮智能科技有限公司 | Pedestrian trajectory tracking method, system and related device |
| WO2022127180A1 (en) * | 2020-12-17 | 2022-06-23 | 深圳云天励飞技术股份有限公司 | Target tracking method and apparatus, and electronic device and storage medium |
| WO2022142417A1 (en) * | 2020-12-31 | 2022-07-07 | 深圳云天励飞技术股份有限公司 | Target tracking method and apparatus, electronic device, and storage medium |
| WO2022142918A1 (en) * | 2020-12-31 | 2022-07-07 | 深圳云天励飞技术股份有限公司 | Multi-target tracking system and method |
| US20220245924A1 (en) * | 2021-01-29 | 2022-08-04 | Beijing Tusen Zhitu Technology Co., Ltd. | Training method for multi-object tracking model and multi-object tracking method |
| CN115080551A (en) * | 2022-06-15 | 2022-09-20 | 苏州轻棹科技有限公司 | Target track management method and device |
| CN115311330A (en) * | 2019-10-11 | 2022-11-08 | 杭州云栖智慧视通科技有限公司 | Video multi-target tracking method based on position prediction |
| CN115423846A (en) * | 2022-09-26 | 2022-12-02 | 青岛以萨数据技术有限公司 | Multi-target track tracking method and device |
-
2023
- 2023-02-28 CN CN202310171954.8A patent/CN115965657B/en active Active
Patent Citations (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050185822A1 (en) * | 2004-02-20 | 2005-08-25 | James Slaski | Component association tracker system and method |
| CN102065275A (en) * | 2009-11-17 | 2011-05-18 | 中国科学院电子学研究所 | Multi-target tracking method in intelligent video monitoring system |
| US20190108613A1 (en) * | 2017-10-06 | 2019-04-11 | Ford Global Technologies, Llc | Fusion Of Motion And Appearance Features For Object Detection And Trajectory Prediction |
| CN110751674A (en) * | 2018-07-24 | 2020-02-04 | 北京深鉴智能科技有限公司 | Multi-target tracking method and corresponding video analysis system |
| CN115311330A (en) * | 2019-10-11 | 2022-11-08 | 杭州云栖智慧视通科技有限公司 | Video multi-target tracking method based on position prediction |
| WO2021134172A1 (en) * | 2019-12-30 | 2021-07-08 | 华为技术有限公司 | Trajectory prediction method and related device |
| CN112200830A (en) * | 2020-09-11 | 2021-01-08 | 山东信通电子股份有限公司 | Target tracking method and device |
| CN112435276A (en) * | 2020-11-13 | 2021-03-02 | 鹏城实验室 | Vehicle tracking method and device, intelligent terminal and storage medium |
| WO2022127180A1 (en) * | 2020-12-17 | 2022-06-23 | 深圳云天励飞技术股份有限公司 | Target tracking method and apparatus, and electronic device and storage medium |
| CN112507949A (en) * | 2020-12-18 | 2021-03-16 | 北京百度网讯科技有限公司 | Target tracking method and device, road side equipment and cloud control platform |
| WO2022142417A1 (en) * | 2020-12-31 | 2022-07-07 | 深圳云天励飞技术股份有限公司 | Target tracking method and apparatus, electronic device, and storage medium |
| WO2022142918A1 (en) * | 2020-12-31 | 2022-07-07 | 深圳云天励飞技术股份有限公司 | Multi-target tracking system and method |
| US20220245924A1 (en) * | 2021-01-29 | 2022-08-04 | Beijing Tusen Zhitu Technology Co., Ltd. | Training method for multi-object tracking model and multi-object tracking method |
| CN112837349A (en) * | 2021-02-09 | 2021-05-25 | 普联技术有限公司 | Target tracking method, target tracking equipment and computer-readable storage medium |
| CN113033447A (en) * | 2021-04-02 | 2021-06-25 | 蔚来汽车科技(安徽)有限公司 | Method for tracking object in video frame sequence, automatic parking method and device thereof |
| CN113792697A (en) * | 2021-09-23 | 2021-12-14 | 重庆紫光华山智安科技有限公司 | Target detection method and device, electronic equipment and readable storage medium |
| CN114169241A (en) * | 2021-12-09 | 2022-03-11 | 北京邮电大学 | End-to-end multi-target identification, tracking and prediction method |
| CN114359341A (en) * | 2021-12-29 | 2022-04-15 | 湖南国科微电子股份有限公司 | Multi-target tracking method and device, terminal equipment and readable storage medium |
| CN114581491A (en) * | 2022-04-30 | 2022-06-03 | 苏州浪潮智能科技有限公司 | Pedestrian trajectory tracking method, system and related device |
| CN115080551A (en) * | 2022-06-15 | 2022-09-20 | 苏州轻棹科技有限公司 | Target track management method and device |
| CN115423846A (en) * | 2022-09-26 | 2022-12-02 | 青岛以萨数据技术有限公司 | Multi-target track tracking method and device |
Non-Patent Citations (1)
| Title |
|---|
| 姜鑫;陈武雄;朱明;郝志成;高文;: "基于实时递推最小二乘的多目标编批研究", 国外电子测量技术, no. 02 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117292306A (en) * | 2023-11-27 | 2023-12-26 | 四川迪晟新达类脑智能技术有限公司 | Edge equipment-oriented vehicle target detection optimization method and device |
| CN117351039A (en) * | 2023-12-06 | 2024-01-05 | 广州紫为云科技有限公司 | Nonlinear multi-target tracking method based on feature query |
| CN117351039B (en) * | 2023-12-06 | 2024-02-02 | 广州紫为云科技有限公司 | Nonlinear multi-target tracking method based on feature query |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115965657B (en) | 2023-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115965657A (en) | Target tracking method, electronic device, storage medium, and vehicle | |
| US20220398845A1 (en) | Method and device for selecting keyframe based on motion state | |
| CN115294168A (en) | Target tracking method and device and electronic equipment | |
| CN112650300A (en) | Unmanned aerial vehicle obstacle avoidance method and device | |
| CN113592706B (en) | Method and device for adjusting homography matrix parameters | |
| CN113095228B (en) | Method and device for detecting target in image and computer readable storage medium | |
| CN115953434B (en) | Track matching method, track matching device, electronic equipment and storage medium | |
| CN113721240B (en) | Target association method, device, electronic equipment and storage medium | |
| CN113869163B (en) | Target tracking method and device, electronic equipment and storage medium | |
| CN113486907A (en) | Unmanned equipment obstacle avoidance method and device and unmanned equipment | |
| CN112991418A (en) | Image depth prediction and neural network training method and device, medium and equipment | |
| CN116558540B (en) | Model training method and device, and track generating method and device | |
| CN115965944B (en) | Target information detection method, device, driving device and medium | |
| CN115923847B (en) | Preprocessing method and device for perception information of automatic driving vehicle and vehicle | |
| CN116580063B (en) | Target tracking method, target tracking device, electronic equipment and storage medium | |
| CN111209837B (en) | Target tracking method and device | |
| CN117876432A (en) | Target tracking method, terminal device and computer readable storage medium | |
| CN115482422B (en) | Training method of deep learning model, image processing method and device | |
| CN113759331B (en) | Radar positioning method, device, equipment and storage medium | |
| CN117991283A (en) | Multi-target tracking detection method, equipment and medium | |
| CN116661504A (en) | Robot tracking method, control terminal, robot and storage medium | |
| CN117333790A (en) | Similarity judging method and device for video events and electronic equipment | |
| CN114708302A (en) | Target tracking method and device, terminal equipment and computer readable storage medium | |
| CN116051637A (en) | VO reliability assessment method, model training method, device, equipment and product | |
| CN116994174A (en) | Video identification method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |