CN115965657B - Target tracking method, electronic device, storage medium and vehicle - Google Patents
Target tracking method, electronic device, storage medium and vehicle Download PDFInfo
- Publication number
- CN115965657B CN115965657B CN202310171954.8A CN202310171954A CN115965657B CN 115965657 B CN115965657 B CN 115965657B CN 202310171954 A CN202310171954 A CN 202310171954A CN 115965657 B CN115965657 B CN 115965657B
- Authority
- CN
- China
- Prior art keywords
- frame
- track
- predicted
- queue
- current frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The invention relates to the technical field of computers, in particular to a target tracking method, electronic equipment, a storage medium and a vehicle, and aims to solve the technical problem that the accuracy of the existing target tracking method is poor. For this purpose, the object tracking method of the present invention comprises: acquiring sensor data acquired by a vehicle-mounted sensor; inputting sensor data into a network model, and outputting current frame detection frames and current frame prediction track information of at least one detection target; and tracking at least one detection target based on the current frame detection frame, the current frame prediction track information and the historical frame prediction track information to obtain a tracking result. Thus, the accuracy and stability of target tracking are improved.
Description
Technical Field
The invention relates to the technical field of computers, and particularly provides a target tracking method, electronic equipment, a storage medium and a vehicle.
Background
Currently, advanced driving assistance functions are increasingly focused, and multi-objective tracking is a fundamental loop in advanced driving assistance systems. Most of the commonly used target tracking methods are an optical flow method and a filtering algorithm, wherein the optical flow method is greatly influenced by the environment, the tracking accuracy is poor under the complex environment, and the tracking accuracy of the filtering algorithm is poor under the conditions of target morphological change, shielding or target disappearance and the like.
Accordingly, there is a need in the art for a new target tracking scheme to address the above-described problems.
Disclosure of Invention
The present invention has been made to overcome the above-mentioned drawbacks, and to provide a solution or at least partially solve the above-mentioned technical problems. The invention provides a target tracking method, electronic equipment, a storage medium and a vehicle.
In a first aspect, the present invention provides a target tracking method, the method comprising: acquiring sensor data acquired by a vehicle-mounted sensor; inputting the sensor data into a network model, and outputting a current frame detection frame and current frame prediction track information of at least one detection target; and tracking the at least one detection target based on the current frame detection frame, the current frame prediction track information and the historical frame prediction track information to obtain a tracking result.
In one embodiment, the tracking the at least one detection target based on the current frame detection frame, the current frame prediction track information and the historical frame prediction track information to obtain a tracking result includes: creating a tracker, and initializing a predicted track queue based on the tracker; adding the current frame predicted track information to the initialized predicted track queue; judging whether the predicted track queue added with the predicted track information of the current frame only has the predicted track information of the current frame or not; if yes, giving a new track ID to the detection target; and if not, matching the current frame detection frame with the historical frame predicted track information in the predicted track queue by using the tracker, and determining the track ID of the detection target according to a matching result.
In one embodiment, the matching the current frame detection frame with the historical frame predicted track information in the predicted track queue by using the tracker, and determining the track ID of the detection target according to the matching result includes matching the current frame detection frame with the predicted frame of the 1 st track point of the previous historical frame, wherein the predicted frame of the 1 st track point of the previous historical frame is obtained based on the previous historical frame predicted track information; if the matching is successful, taking the track ID corresponding to the track information predicted by the previous history frame as the track ID of the detection target; otherwise, continuing to match the current frame with other historical frames before the last historical frame in the predicted track queue frame by frame until the current frame detection frame is successfully matched with one frame of historical frame predicted track information in the other historical frames, stopping the matching, and taking the track ID corresponding to the one frame of historical frame predicted track information which is successfully matched as the track ID of the detection target.
In one embodiment, said matching the current frame with other historical frames in the prediction track queue that precede the last historical frame further comprises: and taking the current frame as a t frame, and matching the current frame detection frame with a predicted frame of an ith track point of a t-i frame, wherein the predicted frame of the ith track point of the t-i frame is obtained based on the predicted track information of the t-i frame, t and i are positive integers, and 1 < i < t.
In one embodiment, the matching, by using the tracker, the current frame detection frame with the historical frame predicted track information in the predicted track queue, and determining the track ID of the detection target according to the matching result includes: and giving a new track ID to the detection target under the condition that the matching of the current frame detection frame and all the historical frame prediction track information in the prediction track queue fails.
In one embodiment, the initializing a predicted track queue based on the tracker includes: setting a queue length threshold of the predicted track queue based on the tracker, and setting a time length threshold of the predicted track queue; before the judging whether the predicted track queue added with the predicted track information of the current frame only has the predicted track information of the current frame, the method further comprises: performing effective frame inspection on the predicted track queue added with the current frame predicted track information based on the time length threshold value, and selectively updating the predicted track queue based on an inspection result; and/or selectively updating the predicted track queue based on the queue length threshold.
In one embodiment, performing a valid frame check on the predicted track queue to which the current frame predicted track information is added based on the time length threshold, and selectively updating the predicted track queue based on a check result includes: acquiring a first time stamp corresponding to current frame predicted track information and a second time stamp corresponding to last historical frame predicted track information in the predicted track queue; determining a difference between the first timestamp and the second timestamp; judging whether the difference value is smaller than a time length threshold value or not; if yes, updating the predicted track queue, and if not, not updating the predicted track queue.
In one embodiment, the selectively updating the predicted track queue based on the queue length threshold comprises: and deleting the historical frame farthest from the current frame predicted track information under the condition that the length of the predicted track queue added with the current frame predicted track information exceeds the queue length threshold value.
In one embodiment, before matching the current frame detection box with the historical frame prediction track information in the prediction track queue with the tracker, the method further comprises: and converting the historical frame prediction track information into a vehicle coordinate system corresponding to the current frame.
In a second aspect, an electronic device is provided comprising at least one processor and at least one storage device adapted to store a plurality of program code adapted to be loaded and executed by the processor to perform the object tracking method of any of the preceding claims.
In a third aspect, there is provided a computer readable storage medium having stored therein a plurality of program codes adapted to be loaded and executed by a processor to perform the object tracking method of any of the preceding claims.
In a fourth aspect, a vehicle is provided, the vehicle comprising the aforementioned electronic device.
The technical scheme provided by the invention has at least one or more of the following beneficial effects:
according to the target tracking method, sensor data acquired by a vehicle-mounted sensor are acquired; inputting sensor data into a detection model, and outputting a current frame detection frame and current frame prediction track information of at least one detection target; and tracking at least one detection target based on the current frame detection frame, the current frame prediction track information and the historical frame prediction track information to obtain a tracking result. Therefore, the target is tracked in a time sequence by utilizing the predicted track, and the accuracy and the stability of target tracking are improved under the condition of ensuring the tracking speed.
Drawings
The present disclosure will become more readily understood with reference to the accompanying drawings. As will be readily appreciated by those skilled in the art: the drawings are for illustrative purposes only and are not intended to limit the scope of the present invention. Moreover, like numerals in the figures are used to designate like parts, wherein:
FIG. 1 is a flow chart of the main steps of a target tracking method according to one embodiment of the invention;
FIG. 2 is a flow diagram of tracking at least one detection target in one embodiment;
FIG. 3 is a complete flow diagram of a target tracking method in one embodiment;
fig. 4 is a schematic structural diagram of an electronic device in one embodiment.
Detailed Description
Some embodiments of the invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are merely for explaining the technical principles of the present invention, and are not intended to limit the scope of the present invention.
In the description of the present invention, a "module," "processor" may include hardware, software, or a combination of both. A module may comprise hardware circuitry, various suitable sensors, communication ports, memory, or software components, such as program code, or a combination of software and hardware. The processor may be a central processor, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and/or signal processing functions. The processor may be implemented in software, hardware, or a combination of both. Non-transitory computer readable storage media include any suitable medium that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, and the like. The term "a and/or B" means all possible combinations of a and B, such as a alone, B alone or a and B. The term "at least one A or B" or "at least one of A and B" has a meaning similar to "A and/or B" and may include A alone, B alone or A and B. The singular forms "a", "an" and "the" include plural referents.
At present, most of traditional target tracking methods are an optical flow method and a filtering algorithm, wherein the optical flow method is greatly influenced by the environment, the tracking accuracy under a complex environment is poor, and the tracking accuracy under the conditions of target morphological change, shielding or target disappearance and the like of the filtering algorithm is poor.
Therefore, the application provides a target tracking method, electronic equipment, a storage medium and a vehicle, and sensor data acquired by a vehicle-mounted sensor are acquired; inputting sensor data into a detection model, and outputting a current frame detection frame and current frame prediction track information of at least one detection target; and tracking at least one detection target based on the current frame detection frame, the current frame prediction track information and the historical frame prediction track information to obtain a tracking result. Therefore, the target is tracked in a time sequence by utilizing the predicted track, and the accuracy and the stability of target tracking are improved under the condition of ensuring the tracking speed.
Referring to fig. 1, fig. 1 is a schematic flow chart of main steps of a target tracking method according to an embodiment of the present invention.
As shown in fig. 1, the target tracking method in the embodiment of the present invention mainly includes the following steps S101 to S103.
Step S101: and acquiring sensor data acquired by the vehicle-mounted sensor.
Step S102: and inputting the sensor data into a network model, and outputting a current frame detection frame and current frame prediction track information of at least one detection target.
Step S103: and tracking at least one detection target based on the current frame detection frame, the current frame prediction track information and the historical frame prediction track information to obtain a tracking result.
Based on the steps S101-S103, acquiring sensor data acquired by the vehicle-mounted sensor; inputting sensor data into a detection model, and outputting a current frame detection frame and current frame prediction track information of at least one detection target; and tracking at least one detection target based on the current frame detection frame, the current frame prediction track information and the historical frame prediction track information to obtain a tracking result. Therefore, the target is tracked in a time sequence by utilizing the predicted track, and the accuracy and the stability of target tracking are improved under the condition of ensuring the tracking speed.
The above steps S101 to S103 are further described below.
In step S101, the in-vehicle sensor may be any one of a camera and a lidar. The sensor data collected by the vehicle-mounted sensor can be video frame images or point cloud images collected by the laser radar.
In addition, the acquired sensor image acquired by the vehicle-mounted sensor may be one-frame image or may be multi-frame image, for example, sensor data acquired by sensors with different viewing angles. The input information contains multi-frame data, so that the stability of the detection effect and the accuracy of the predicted track can be improved, but the tracking of the multi-frame data has higher requirements on time cost and calculation force. In this embodiment, it is preferable to acquire one frame of sensor acquisition data.
The above is a further explanation of step S101, and the following further explanation of step S102 is continued.
The network model is an end-to-end network model that includes at least two tasks, detection and prediction. And inputting the sensor data into the network model, and outputting the current frame detection frame and the current frame prediction track information of at least one corresponding detection target in the sensor data through the main network feature extraction, the multitasking detection head and the track prediction head. Wherein the predicted trajectory information includes trajectory point coordinates and orientation angles at future times, and the like.
IntntNet may be used as an example of the network model, but is not limited thereto.
The above is a further explanation of step S102, and the following further explanation of step S103 is continued.
Specifically, the above step S103 may be realized by the following steps S1031 to S1035.
Step S1031: creating a tracker, and initializing a predicted track queue based on the tracker.
The tracker is an object, and the process of creating the tracker is a process of creating a new tracker object and initializing at the same time.
The prediction track queue is used for storing current frame information and historical frame information, such as current frame prediction track information, historical frame detection frames and the like.
In a specific embodiment, the initializing a predicted track queue based on the tracker includes: setting a queue length threshold of the predicted track queue based on the tracker, and setting a time length threshold of the predicted track queue; before the judging whether the predicted track queue added with the predicted track information of the current frame only has the predicted track information of the current frame, the method further comprises: performing effective frame inspection on the predicted track queue added with the current frame predicted track information based on the time length threshold value, and selectively updating the predicted track queue based on an inspection result; and/or selectively updating the predicted track queue based on the queue length threshold.
Updating the predicted track queue may be deleting historical frame information that does not meet a valid frame check or that does not meet a queue length threshold, or acquiring new historical frame information from a set of historical frame information and storing the new historical frame information in the predicted track queue.
In a specific embodiment, performing a valid frame check on the predicted track queue added with the predicted track information of the current frame based on the time length threshold, and selectively updating the predicted track queue based on a check result, including: acquiring a first time stamp corresponding to current frame predicted track information and a second time stamp corresponding to last historical frame predicted track information in the predicted track queue; determining a difference between the first timestamp and the second timestamp; judging whether the difference value is smaller than a time length threshold value or not; if yes, updating the predicted track queue, and if not, not updating the predicted track queue.
Specifically, each frame of information has a corresponding time stamp, and the effective frame inspection of the prediction track queue refers to checking whether the difference between the time stamp of the t-th frame (current frame) and the time stamp of the t-1 st frame is within a reasonable interval. Specifically, the current frame prediction track information corresponds to a first time stamp, the previous history frame prediction track information corresponds to a second time stamp, and a difference value between the current frame prediction track information and the second time stamp is calculated according to the first time stamp and the second time stamp, and whether the difference value is within a time length threshold value range is judged; if yes, updating the predicted track queue, and if not, not updating the predicted track queue.
By carrying out effective frame inspection on the predicted track queue, the difference value between the previous historical frame time stamp and the current frame time stamp can be ensured to be always positioned in a reasonable time range, so that the accuracy of target tracking is improved.
In a specific embodiment, the selectively updating the predicted track queue based on the queue length threshold includes: and deleting the historical frame farthest from the current frame predicted track information under the condition that the length of the predicted track queue added with the current frame predicted track information exceeds the queue length threshold value.
And deleting the historical frame farthest from the current frame information under the condition that the length of the maintained predicted track queue exceeds a queue length threshold.
The queue length threshold is used for maintaining a predicted track queue, and maintaining the predicted track queue with a certain length, so that tracking effect under the condition of shielding or object disappearance can be improved.
Step S1032: and adding the current frame predicted track information to the initialized predicted track queue.
In one embodiment, the detection frame information of the current frame may also be added to the initialized predicted track queue. A detection frame queue storing detection frames may also be reconstructed to store the current frame detection frame into the detection frame queue.
Step S1033: and judging whether the predicted track queue added with the predicted track information of the current frame only has the predicted track information of the current frame or not.
Typically, when initial tracking is performed, only the current frame of predicted track information is in the predicted track queue. Tracking the target object in the subsequent process, wherein the predicted track information of the current frame and the predicted track information of the historical frame exist in the predicted track queue at the same time.
Step S1034: if yes, giving the new track ID to the detection target.
Step S1035: if not, matching the current frame detection frame with the historical frame predicted track information in the predicted track queue by using a tracker, and determining the track ID of the detection target according to a matching result.
Track ID (track ID), specifically, target track ID.
Specifically, the matching of the current frame detection frame and the historical frame prediction track information in the prediction track queue mainly comprises the step of performing frame-by-frame matching on the current frame detection frame and the prediction frame corresponding to each frame of the historical frame prediction track information, so as to determine whether a historical frame which can be successfully matched with the current frame detection frame exists in the prediction track queue.
In one specific embodiment, the matching the current frame detection frame with the predicted track information of the historical frames in the predicted track queue by using the tracker, and determining the track ID of the detection target according to the matching result, wherein the matching the current frame detection frame with the predicted frame of the 1 st track point of the previous historical frame, wherein the predicted frame of the 1 st track point of the previous historical frame is obtained based on the predicted track information of the previous historical frame; if the matching is successful, taking the track ID corresponding to the track information predicted by the previous history frame as the track ID of the detection target; otherwise, continuing to match the current frame with other historical frames before the last historical frame in the predicted track queue frame by frame until the current frame detection frame is successfully matched with one frame of historical frame predicted track information in the other historical frames, stopping the matching, and taking the track ID corresponding to the one frame of historical frame predicted track information which is successfully matched as the track ID of the detection target.
The per-frame historical frame prediction track information includes a plurality of track points and an orientation of each track point. And combining the track points and the orientations of the track points with the detection frame of the frame to obtain a prediction frame corresponding to each track point of each frame.
Specifically, firstly, matching a detection frame of a current frame with a prediction frame of a 1 st track point of a previous history frame, and if the matching is successful, taking a track ID corresponding to the predicted track information of the previous history frame as a track ID of a detection target; otherwise, the current frame is continuously matched with other historical frames before the previous historical frame in the predicted track queue frame by frame until the current frame detection frame is successfully matched with the predicted track information of one historical frame in the other historical frames, the matching is stopped, and the track ID corresponding to the predicted track information of the successfully matched one historical frame is used as the track ID of the detection target.
When the previous history frame has a plurality of predicted track information, a plurality of predicted frames are provided at the moment, the detected frames and the predicted frames are required to be matched respectively, the cross-over ratio Iou between the detected frames and each predicted frame is calculated specifically, at least one cross-over ratio meeting the condition is selected from the plurality of cross-over ratios, and then the predicted frames which can be successfully matched with the detected frames are determined by using a Hungary matching algorithm.
In one embodiment, said matching the current frame with other historical frames in the predicted track queue that precede the last historical frame further comprises: and taking the current frame as a t frame, and matching the current frame detection frame with a predicted frame of an ith track point of a t-i frame, wherein the predicted frame of the ith track point of the t-i frame is obtained based on the predicted track information of the t-i frame, t and i are positive integers, and 1 < i < t.
Specifically, as shown in fig. 2, when the matching between the current frame detection frame (t-th frame) and the previous history frame (t-1 st frame) fails, the matching between the current frame detection frame and the predicted frame of the second track point of the t-2 th frame is continued, and if the matching is successful, the track ID corresponding to the predicted track information of the t-2 th frame is used as the track ID of the detection target. If the matching fails, the current frame detection frame is matched with the t-3 frame, the step is repeatedly executed until the current frame detection frame is successfully matched with the historical frame in the predicted track queue, and the track ID of the detection target is output.
In a specific embodiment, the matching, by using the tracker, the current frame detection frame with the historical frame predicted track information in the predicted track queue, and determining the track ID of the detection target according to the matching result includes: and giving a new track ID to the detection target under the condition that the matching of the current frame detection frame and all the historical frame prediction track information in the prediction track queue fails.
Specifically, in the case where the current frame detection frame fails to match all of the history frames in the predicted track queue, a new track ID is given to the detection target, indicating that the detection target is a new emerging target.
The tracking target is obtained by maintaining the predicted track queue and matching the current frame information with the history predicted information, so that the problems of morphological change, shielding, disappearance and the like of the detection target in the tracking process are solved to a certain extent, and a stable tracking target is output.
In one embodiment, as shown in fig. 3 in particular, after the tracker is constructed and before the target tracking, the data preprocessing may be further performed on the current frame detection frame and the current frame prediction track information of the at least one detection target output in step S102. The detection frame information and the prediction track information comprise the types and the confidence degrees of the frames, confidence threshold values of different types are set, and detection frames and prediction tracks with smaller confidence degrees are filtered. In addition, multi-modal prediction of the track, that is, a case where one frame predicts a plurality of tracks, requires screening out a predicted track with the highest confidence. Therefore, the complexity of target tracking is reduced, and the efficiency and accuracy of target tracking are improved.
In one embodiment, before matching the current frame detection frame with the historical frame prediction track information in the prediction track queue using the tracker, the method further comprises: and converting the historical frame prediction track information into a vehicle coordinate system corresponding to the current frame.
Specifically, before matching, the coordinate transformation matrix of the historical frame information converted to the current frame can be determined according to the self-vehicle positioning information, and then the historical frame prediction track information is converted to the vehicle coordinate system corresponding to the current frame according to the coordinate transformation matrix. Thus, the matching of the current frame detection frame and the historical frame information is convenient to follow.
It should be noted that, although the foregoing embodiments describe the steps in a specific order, it will be understood by those skilled in the art that, in order to achieve the effects of the present invention, the steps are not necessarily performed in such an order, and may be performed simultaneously (in parallel) or in other orders, and these variations are within the scope of the present invention.
It will be appreciated by those skilled in the art that the present invention may implement all or part of the above-described methods according to the above-described embodiments, or may be implemented by means of a computer program for instructing relevant hardware, where the computer program may be stored in a computer readable storage medium, and where the computer program may implement the steps of the above-described embodiments of the method when executed by a processor. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable storage medium may include: any entity or device, medium, usb disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier wave signals, telecommunications signals, software distribution media, and the like capable of carrying the computer program code. It should be noted that the computer readable storage medium may include content that is subject to appropriate increases and decreases as required by jurisdictions and by jurisdictions in which such computer readable storage medium does not include electrical carrier signals and telecommunications signals.
The invention further provides electronic equipment. In one embodiment of an electronic device according to the present invention, as particularly shown in fig. 4, the electronic device includes at least one processor 41 and at least one storage device 42, the storage device may be configured to store a program for executing the object tracking method of the above-described method embodiment, and the processor may be configured to execute the program in the storage device, including, but not limited to, the program for executing the object tracking method of the above-described method embodiment. For convenience of explanation, only those portions of the embodiments of the present invention that are relevant to the embodiments of the present invention are shown, and specific technical details are not disclosed, please refer to the method portions of the embodiments of the present invention.
The electronic device in the embodiment of the invention can be a control device formed by various devices. In some possible implementations, the electronic device may include multiple storage devices and multiple processors. While the program for performing the object tracking method of the above-described method embodiment may be divided into a plurality of sub-programs, each of which may be loaded and executed by a processor to perform the different steps of the object tracking method of the above-described method embodiment, respectively. Specifically, each of the subroutines may be stored in different storage devices, respectively, and each of the processors may be configured to execute the programs in one or more storage devices to collectively implement the target tracking method of the above-described method embodiment, that is, each of the processors executes different steps of the target tracking method of the above-described method embodiment, respectively, to collectively implement the target tracking method of the above-described method embodiment.
The plurality of processors may be processors disposed on the same device, for example, the electronic device may be a high-performance device composed of a plurality of processors, and the plurality of processors may be processors configured on the high-performance device. In addition, the plurality of processors may be processors disposed on different devices, for example, the electronic device may be a server cluster, and the plurality of processors may be processors on different servers in the server cluster.
Further, the invention also provides a computer readable storage medium. In one computer-readable storage medium embodiment according to the present invention, the computer-readable storage medium may be configured to store a program for performing the target tracking method of the above-described method embodiment, which may be loaded and executed by a processor to implement the above-described target tracking method. For convenience of explanation, only those portions of the embodiments of the present invention that are relevant to the embodiments of the present invention are shown, and specific technical details are not disclosed, please refer to the method portions of the embodiments of the present invention. The computer readable storage medium may be a storage device including various electronic devices, and optionally, the computer readable storage medium in the embodiments of the present invention is a non-transitory computer readable storage medium.
Further, the invention also provides a vehicle, which comprises the electronic equipment.
Thus far, the technical solution of the present invention has been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present invention is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present invention, and such modifications and substitutions will fall within the scope of the present invention.
Claims (12)
1. A method of target tracking, the method comprising:
acquiring sensor data acquired by a vehicle-mounted sensor;
inputting the sensor data into a network model, and outputting a current frame detection frame and current frame prediction track information of at least one detection target;
tracking the at least one detection target based on the current frame detection frame, the current frame prediction track information and the historical frame prediction track information to obtain a tracking result, wherein the tracking result comprises the following steps:
judging whether the predicted track information of the current frame is only in a predicted track queue added with the predicted track information of the current frame;
if yes, giving a new track ID to the detection target;
and if not, matching the current frame detection frame with the historical frame predicted track information in the predicted track queue, and determining the track ID of the detection target according to a matching result.
2. The target tracking method of claim 1, wherein the method further comprises: creating a tracker, and initializing a predicted track queue based on the tracker;
the matching the current frame detection frame with the historical frame prediction track information in the prediction track queue comprises the following steps: and matching the current frame detection frame with the historical frame predicted track information in the predicted track queue by using the tracker.
3. The method according to claim 2, wherein the matching, by the tracker, the current frame detection frame with the historical frame predicted track information in the predicted track queue, and determining the track ID of the detection target according to the matching result, includes:
matching the current frame detection frame with a prediction frame of the 1 st track point of a previous history frame, wherein the prediction frame of the 1 st track point of the previous history frame is obtained based on the previous history frame prediction track information;
if the matching is successful, taking the track ID corresponding to the track information predicted by the previous history frame as the track ID of the detection target;
otherwise, continuing to match the current frame with other historical frames before the last historical frame in the predicted track queue frame by frame until the current frame detection frame is successfully matched with one frame of historical frame predicted track information in the other historical frames, stopping the matching, and taking the track ID corresponding to the one frame of historical frame predicted track information which is successfully matched as the track ID of the detection target.
4. The method of claim 3, wherein said continuing to match the current frame with other historical frames in the predicted track queue that precede the last historical frame comprises: and taking the current frame as a t frame, and matching the current frame detection frame with a predicted frame of an ith track point of a t-i frame, wherein the predicted frame of the ith track point of the t-i frame is obtained based on the predicted track information of the t-i frame, t and i are positive integers, and 1 < i < t.
5. The method according to claim 4, wherein the matching, by the tracker, the current frame detection frame with the historical frame predicted track information in the predicted track queue, and determining the track ID of the detection target according to the matching result, includes: and giving a new track ID to the detection target under the condition that the matching of the current frame detection frame and all the historical frame prediction track information in the prediction track queue fails.
6. The target tracking method of claim 2, wherein initializing a predicted trajectory queue based on the tracker comprises:
setting a queue length threshold of the predicted track queue based on the tracker, and setting a time length threshold of the predicted track queue;
before the judging whether the predicted track queue added with the predicted track information of the current frame only has the predicted track information of the current frame, the method further comprises:
performing effective frame inspection on the predicted track queue added with the current frame predicted track information based on the time length threshold value, and selectively updating the predicted track queue based on an inspection result; and/or
The predicted trajectory queue is selectively updated based on the queue length threshold.
7. The target tracking method according to claim 6, wherein the effective frame inspection of the predicted track queue to which the current frame predicted track information is added based on the time length threshold value, the updating of the predicted track queue based on the inspection result selectively, comprises:
acquiring a first time stamp corresponding to current frame predicted track information and a second time stamp corresponding to last historical frame predicted track information in the predicted track queue;
determining a difference between the first timestamp and the second timestamp;
judging whether the difference value is smaller than the time length threshold value or not;
if yes, updating the predicted track queue, and if not, not updating the predicted track queue.
8. The target tracking method of claim 6, wherein the selectively updating the predicted trajectory queue based on the queue length threshold comprises: and deleting the historical frame farthest from the current frame predicted track information under the condition that the length of the predicted track queue added with the current frame predicted track information exceeds the queue length threshold value.
9. The target tracking method of claim 2, wherein prior to matching the current frame detection box with the historical frame predicted track information in the predicted track queue with the tracker, the method further comprises: and converting the historical frame prediction track information into a vehicle coordinate system corresponding to the current frame.
10. An electronic device comprising at least one processor and at least one storage means, the storage means being adapted to store a plurality of program code, characterized in that the program code is adapted to be loaded and executed by the processor to perform the object tracking method of any of claims 1 to 9.
11. A computer readable storage medium having stored therein a plurality of program codes, characterized in that the program codes are adapted to be loaded and executed by a processor to perform the object tracking method of any one of claims 1 to 9.
12. A vehicle, characterized in that it comprises the electronic device of claim 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310171954.8A CN115965657B (en) | 2023-02-28 | 2023-02-28 | Target tracking method, electronic device, storage medium and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310171954.8A CN115965657B (en) | 2023-02-28 | 2023-02-28 | Target tracking method, electronic device, storage medium and vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115965657A CN115965657A (en) | 2023-04-14 |
| CN115965657B true CN115965657B (en) | 2023-06-02 |
Family
ID=85894647
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310171954.8A Active CN115965657B (en) | 2023-02-28 | 2023-02-28 | Target tracking method, electronic device, storage medium and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115965657B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117292306A (en) * | 2023-11-27 | 2023-12-26 | 四川迪晟新达类脑智能技术有限公司 | Edge equipment-oriented vehicle target detection optimization method and device |
| CN117351039B (en) * | 2023-12-06 | 2024-02-02 | 广州紫为云科技有限公司 | Nonlinear multi-target tracking method based on feature query |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102065275A (en) * | 2009-11-17 | 2011-05-18 | 中国科学院电子学研究所 | Multi-target tracking method in intelligent video monitoring system |
| WO2021134172A1 (en) * | 2019-12-30 | 2021-07-08 | 华为技术有限公司 | Trajectory prediction method and related device |
| CN114169241A (en) * | 2021-12-09 | 2022-03-11 | 北京邮电大学 | End-to-end multi-target identification, tracking and prediction method |
| WO2022127180A1 (en) * | 2020-12-17 | 2022-06-23 | 深圳云天励飞技术股份有限公司 | Target tracking method and apparatus, and electronic device and storage medium |
| WO2022142417A1 (en) * | 2020-12-31 | 2022-07-07 | 深圳云天励飞技术股份有限公司 | Target tracking method and apparatus, electronic device, and storage medium |
| WO2022142918A1 (en) * | 2020-12-31 | 2022-07-07 | 深圳云天励飞技术股份有限公司 | Multi-target tracking system and method |
| CN115311330A (en) * | 2019-10-11 | 2022-11-08 | 杭州云栖智慧视通科技有限公司 | Video multi-target tracking method based on position prediction |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7298867B2 (en) * | 2004-02-20 | 2007-11-20 | Lockheed Martin Corporation | Component association tracker system and method |
| US10482572B2 (en) * | 2017-10-06 | 2019-11-19 | Ford Global Technologies, Llc | Fusion of motion and appearance features for object detection and trajectory prediction |
| CN110751674A (en) * | 2018-07-24 | 2020-02-04 | 北京深鉴智能科技有限公司 | Multi-target tracking method and corresponding video analysis system |
| CN112200830A (en) * | 2020-09-11 | 2021-01-08 | 山东信通电子股份有限公司 | Target tracking method and device |
| CN112435276A (en) * | 2020-11-13 | 2021-03-02 | 鹏城实验室 | Vehicle tracking method and device, intelligent terminal and storage medium |
| CN112507949A (en) * | 2020-12-18 | 2021-03-16 | 北京百度网讯科技有限公司 | Target tracking method and device, road side equipment and cloud control platform |
| CN114913386A (en) * | 2021-01-29 | 2022-08-16 | 北京图森智途科技有限公司 | Training method of multi-target tracking model and multi-target tracking method |
| CN112837349A (en) * | 2021-02-09 | 2021-05-25 | 普联技术有限公司 | Target tracking method, target tracking equipment and computer-readable storage medium |
| CN113033447A (en) * | 2021-04-02 | 2021-06-25 | 蔚来汽车科技(安徽)有限公司 | Method for tracking object in video frame sequence, automatic parking method and device thereof |
| CN113792697B (en) * | 2021-09-23 | 2023-09-05 | 重庆紫光华山智安科技有限公司 | Target detection method, target detection device, electronic equipment and readable storage medium |
| CN114359341A (en) * | 2021-12-29 | 2022-04-15 | 湖南国科微电子股份有限公司 | Multi-target tracking method and device, terminal equipment and readable storage medium |
| CN114581491B (en) * | 2022-04-30 | 2022-07-22 | 苏州浪潮智能科技有限公司 | Pedestrian trajectory tracking method, system and related device |
| CN115080551A (en) * | 2022-06-15 | 2022-09-20 | 苏州轻棹科技有限公司 | Target track management method and device |
| CN115423846A (en) * | 2022-09-26 | 2022-12-02 | 青岛以萨数据技术有限公司 | Multi-target track tracking method and device |
-
2023
- 2023-02-28 CN CN202310171954.8A patent/CN115965657B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102065275A (en) * | 2009-11-17 | 2011-05-18 | 中国科学院电子学研究所 | Multi-target tracking method in intelligent video monitoring system |
| CN115311330A (en) * | 2019-10-11 | 2022-11-08 | 杭州云栖智慧视通科技有限公司 | Video multi-target tracking method based on position prediction |
| WO2021134172A1 (en) * | 2019-12-30 | 2021-07-08 | 华为技术有限公司 | Trajectory prediction method and related device |
| WO2022127180A1 (en) * | 2020-12-17 | 2022-06-23 | 深圳云天励飞技术股份有限公司 | Target tracking method and apparatus, and electronic device and storage medium |
| WO2022142417A1 (en) * | 2020-12-31 | 2022-07-07 | 深圳云天励飞技术股份有限公司 | Target tracking method and apparatus, electronic device, and storage medium |
| WO2022142918A1 (en) * | 2020-12-31 | 2022-07-07 | 深圳云天励飞技术股份有限公司 | Multi-target tracking system and method |
| CN114169241A (en) * | 2021-12-09 | 2022-03-11 | 北京邮电大学 | End-to-end multi-target identification, tracking and prediction method |
Non-Patent Citations (1)
| Title |
|---|
| 基于实时递推最小二乘的多目标编批研究;姜鑫;陈武雄;朱明;郝志成;高文;;国外电子测量技术(第02期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115965657A (en) | 2023-04-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115965657B (en) | Target tracking method, electronic device, storage medium and vehicle | |
| CN109188457B (en) | Object detection frame generation method, device, equipment, storage medium and vehicle | |
| CN109635861B (en) | Data fusion method and device, electronic equipment and storage medium | |
| CN112650300A (en) | Unmanned aerial vehicle obstacle avoidance method and device | |
| CN113392794A (en) | Vehicle over-line identification method and device, electronic equipment and storage medium | |
| CN110363193B (en) | Vehicle weight recognition method, device, equipment and computer storage medium | |
| CN112966654A (en) | Lip movement detection method and device, terminal equipment and computer readable storage medium | |
| CN112132015A (en) | Detection method, device, medium and electronic equipment for illegal driving posture | |
| CN112558036A (en) | Method and apparatus for outputting information | |
| CN113869163B (en) | Target tracking method and device, electronic equipment and storage medium | |
| CN113721240B (en) | Target association method, device, electronic equipment and storage medium | |
| CN110634155A (en) | Target detection method and device based on deep learning | |
| CN114359572A (en) | Training method and device of multi-task detection model and terminal equipment | |
| CN112418234A (en) | Method and device for identifying license plate number, electronic equipment and storage medium | |
| CN112634355A (en) | Target tracking method, device, equipment and storage medium | |
| CN116558540B (en) | Model training method and device, and track generating method and device | |
| CN115965944B (en) | Target information detection method, device, driving device and medium | |
| CN115923847B (en) | Preprocessing method and device for perception information of automatic driving vehicle and vehicle | |
| CN116778170B (en) | Point cloud panorama segmentation method, control device, readable storage medium and vehicle | |
| CN113759331B (en) | Radar positioning method, device, equipment and storage medium | |
| CN116580063B (en) | Target tracking method, target tracking device, electronic equipment and storage medium | |
| CN113051437B (en) | Target duplicate removal method and device, electronic equipment and storage medium | |
| CN111209837B (en) | Target tracking method and device | |
| US20230410661A1 (en) | Method for warning collision of vehicle, system, vehicle, and computer readable storage medium | |
| CN111862107A (en) | Method and system for processing images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |