CN117991283A - Multi-target tracking detection method, equipment and medium - Google Patents
Multi-target tracking detection method, equipment and medium Download PDFInfo
- Publication number
- CN117991283A CN117991283A CN202410141888.4A CN202410141888A CN117991283A CN 117991283 A CN117991283 A CN 117991283A CN 202410141888 A CN202410141888 A CN 202410141888A CN 117991283 A CN117991283 A CN 117991283A
- Authority
- CN
- China
- Prior art keywords
- truth
- observation
- box
- tracking
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 113
- 238000000034 method Methods 0.000 claims abstract description 51
- 238000002372 labelling Methods 0.000 claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 14
- 239000011159 matrix material Substances 0.000 claims description 25
- 238000003860 storage Methods 0.000 claims description 17
- 238000004422 calculation algorithm Methods 0.000 claims description 14
- 238000012216 screening Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 8
- 230000001133 acceleration Effects 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 2
- 238000012549 training Methods 0.000 description 17
- 238000012805 post-processing Methods 0.000 description 8
- 238000013135 deep learning Methods 0.000 description 6
- 238000011161 development Methods 0.000 description 6
- 230000018109 developmental process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000012360 testing method Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000001914 filtration Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000013475 authorization Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Landscapes
- Optical Radar Systems And Details Thereof (AREA)
Abstract
The application discloses a multi-target tracking detection method, equipment and medium, and relates to the technical field of automatic driving. The method comprises the following steps: acquiring laser radar data; obtaining a plurality of observation frames based on the laser radar data and a target detection network model; acquiring a plurality of truth labeling results obtained based on the laser radar data, wherein each truth labeling result comprises a truth box and a tracking identifier corresponding to the truth box; matching the truth box and the observation boxes, and determining corresponding tracking identification for each observation box according to a matching processing result; and determining a multi-target tracking detection result according to the plurality of observation frames and tracking identifiers corresponding to the observation frames. The multi-target tracking detection method provided by the application can show more accurate detection effect in the multi-target tracking task.
Description
Technical Field
The application relates to the technical field of automatic driving, in particular to a multi-target tracking detection method, equipment and medium.
Background
In recent years, with the development of deep learning, advanced technology adopts a method based on deep learning to realize target detection and tracking. The deep learning scheme requires a large amount of labeled training data to train the target detection network, and datasets such as NuScenes, waymo and KITTI are one of the most popular and widely used datasets in the current automatic driving field, and they make a great contribution to the development of automatic driving technology. While these datasets contain multimodal sensor data and collect a large amount of data to cover various weather conditions, geographic locations, and traffic scenarios, they suffer from a significant drawback: only the detection results of perfect labeling of multiple targets are provided. However, in an actual multi-target tracking application scenario, the input of the target detection network is often noisy data, and since these public data sets cannot provide the target detection network with data closer to the actual scenario, the multi-target tracking detection result obtained based on the target detection network is directly inaccurate.
Disclosure of Invention
In order to overcome the defects in the prior art, the application provides a multi-target tracking detection method, equipment and a medium.
In a first aspect, the present application provides a multi-target tracking detection method, the method comprising:
acquiring laser radar data;
acquiring a plurality of observation frames based on the laser radar data and a target detection network model;
acquiring a plurality of truth labeling results obtained based on the laser radar data, wherein each truth labeling result comprises a truth box and a tracking identifier corresponding to the truth box;
Matching the truth box and the observation boxes, and determining corresponding tracking identification for each observation box according to a matching processing result;
And determining a multi-target tracking detection result according to the plurality of observation frames and tracking identifiers corresponding to the observation frames.
Preferably, the matching processing is performed on the truth box and the observation box, and determining the corresponding tracking identifier for each observation box according to the matching processing result specifically includes:
Constructing an incidence matrix based on a plurality of the truth boxes and the observation boxes;
determining a matching corresponding relation between the truth box and the observation box based on the incidence matrix;
And assigning the tracking identification corresponding to the truth box to the tracking identification corresponding to the observation box matched with the truth box based on the matching corresponding relation.
Further, the building an association matrix based on the plurality of truth boxes and the plurality of observation boxes specifically includes:
For each observation frame, calculating the distance and the intersection ratio of the three-dimensional center point of the observation frame and each truth frame;
calculating according to the three-dimensional center point distance and the intersection ratio to obtain a correlation value;
and constructing an association matrix by taking the association value as a matrix element.
The matching corresponding relation of the truth box and the observation box based on the incidence matrix is specifically: and based on the incidence matrix, determining the matching corresponding relation of the truth box and the observation box by using a Hungary algorithm.
Assigning the tracking identifier corresponding to the truth box to the tracking identifier corresponding to the observation box matched with the truth box based on the matching corresponding relation specifically comprises the following steps:
Screening the matching corresponding relation based on a preset distance threshold;
and assigning tracking identifications corresponding to the truth boxes in the matched corresponding relation to the tracking identifications corresponding to the observation boxes in the screened matched corresponding relation.
And aiming at the observation frames in the matching corresponding relation which do not pass through screening, assigning a preset identification value to the tracking identification corresponding to the observation frames.
Preferably, the obtaining a plurality of true value labeling results based on the laser radar data specifically includes: manually labeling the laser radar data based on a label generating system to obtain a plurality of truth boxes and tracking identifications corresponding to each truth box, and storing the obtained truth boxes and the tracking identifications corresponding to each truth box into a data set system as a plurality of truth labeling results;
or specifically: and acquiring the multiple true value labeling results from a data set system according to the laser radar data.
Preferably, the method further comprises: and obtaining auxiliary characteristic information of each observation frame based on the laser radar data and the target detection network model, wherein the auxiliary characteristic information at least comprises one of the number of points in the frame, the size of the frame, the position, the speed, the acceleration, the angle and the angular speed of the frame.
Further, the method further comprises: and storing the accessory characteristic information and the detection result into a data set system.
Preferably, the method further comprises: and displaying the multiple true value labeling results, the multiple observation frames and the multi-target tracking detection result based on a visual interface.
In a second aspect, the present application provides a smart device, which includes a processor and a storage device, where the storage device is adapted to store a plurality of program codes, where the program codes are adapted to be loaded and executed by the processor to perform the multi-target tracking detection method according to any one of the above-mentioned methods.
In a third aspect, the present application provides a computer readable storage medium having stored therein a plurality of program codes adapted to be loaded by a processor and to run the multi-object tracking detection method according to any one of the above-mentioned method aspects.
The technical scheme provided by the application has at least one or more of the following beneficial effects: in the technical scheme of implementing the application, laser radar data are firstly acquired; obtaining a plurality of observation frames based on the laser radar data and a target detection network model; then, a plurality of true value labeling results obtained based on the laser radar data are obtained, and each true value labeling result comprises a true value box and a tracking identifier corresponding to the true value box; matching the truth box and the observation boxes, and determining corresponding tracking identification for each observation box according to a matching processing result; and finally, determining a multi-target tracking detection result according to the plurality of observation frames and tracking identifiers corresponding to the observation frames. According to the multi-target tracking detection method provided by the application, based on the fixed true value labeling result, the observation frame is matched with the perfect true value frame in the true value labeling result through an intelligent matching mechanism, so that the perfect tracking identification of the observation frame is obtained based on the target detection network model, and finally, the multi-target tracking detection result is determined according to the obtained observation frame and the corresponding tracking identification, so that the multi-target tracking detection method provided by the application can show more accurate detection effect in a multi-target tracking task.
Drawings
The present disclosure will become more readily understood with reference to the accompanying drawings. As will be readily appreciated by those skilled in the art: the drawings are for illustrative purposes only and are not intended to limit the scope of the present application. Moreover, like numerals in the figures are used to designate like parts, wherein:
FIG. 1 is a schematic diagram of the design principle of the multi-target tracking detection method provided by the application;
FIG. 2 is a flow chart of the main steps of a multi-target tracking detection method according to one embodiment of the application;
FIG. 3 is a step flow diagram of one particular implementation of step S14 shown in FIG. 2;
FIG. 4 is a schematic diagram of a visual interface based on the multi-target tracking detection method provided by the application;
Fig. 5 is a block diagram illustrating a multi-target tracking detection apparatus according to an embodiment of the present application.
Detailed Description
Some embodiments of the application are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are merely for explaining the technical principles of the present application, and are not intended to limit the scope of the present application.
In the description of the present application, a "module," "processor" may include hardware, software, or a combination of both. A module may comprise hardware circuitry, various suitable sensors, communication ports, memory, or software components, such as program code, or a combination of software and hardware. The processor may be a central processor, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and/or signal processing functions. The processor may be implemented in software, hardware, or a combination of both. Non-transitory computer readable storage media include any suitable medium that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, and the like. The term "a and/or B" means all possible combinations of a and B, such as a alone, B alone or a and B. The term "at least one A or B" or "at least one of A and B" has a meaning similar to "A and/or B" and may include A alone, B alone or A and B. The singular forms "a", "an" and "the" include plural referents.
Because the multi-target tracking task needs to realize real-time tracking of a plurality of targets, the continuity of the object track and the uniqueness of the Identification (ID) at the same moment are ensured. In the prior art, the target tracking algorithm based on the traditional method comprises the following steps:
Kalman filter: the Kalman filter is a recursive filtering algorithm based on a state space model and is widely applied to the field of target tracking. The method realizes estimation of the target position and speed by fusing the observation data with a system model. The kalman filter works well when dealing with linear dynamic systems and gaussian noise, but the performance is degraded when dealing with non-linear systems and non-gaussian noise.
Target detection and data association (Detection and Data Association): the method comprises the steps of firstly detecting a target in each frame by using a target detection algorithm, and then matching the target between different frames by a data association technology to form a target track. Common data association methods include nearest neighbor method, maximum posterior probability method, multi-hypothesis tracking method, and the like.
In recent years, with the development of deep learning, advanced technology adopts a method based on deep learning to realize target detection and tracking, but because the used public data set (such as NuScenes and the like) cannot realize that training data and application scene data are homologous data, a target detection network obtained based on the training of the public data set cannot realize accurate tracking detection. Therefore, the application designs a multi-target tracking detection method, and the perfect tracking ID is assigned to the imperfect detection frame in a mode of matching the imperfect detection frame obtained based on the target detection network with the perfect detection frame. The scheme can adapt to different automatic driving scenes and different versions of target detection network results, so that the performance and the efficiency of the multi-target tracking task are improved.
The design principle of the multi-target tracking detection method provided by the application is shown in figure 1, and mainly comprises a true value labeling link and a laser radar sensing link. The perfect true value labeling link can adopt a special labeling generation system in the prior art, wherein the input is laser radar (Lidar) data, and a true value labeling result with a true value box and a tracking ID is generated after manual labeling. The Lidar perception link is a cloud observation frame generation method based on a laser radar, and firstly, an observation frame of each frame of data is generated based on a target detection network model, and detection errors can be generated due to detection accuracy. And in the tracking post-processing module, carrying out post-processing matching on the observation frame and the truth frame of each frame, and giving the tracking ID of the corresponding truth frame to the observation frame.
Referring to fig. 2, fig. 2 is a schematic flow chart of main steps of a multi-target tracking detection method according to an embodiment of the present application. As shown in fig. 2, the multi-target tracking detection method in the embodiment of the application mainly includes the following steps S11 to S15.
Step S11: acquiring sensor data acquired by a vehicle-mounted sensor;
The vehicle-mounted sensor can be a camera or a laser radar, and sensor data collected by the vehicle-mounted sensor can be video frame images or point cloud data collected by the laser radar. In addition, the acquired sensor data acquired by the vehicle-mounted sensor may be one-frame image or multi-frame image.
Step S12: obtaining a plurality of observation frames based on the sensor data and a target detection network model;
In this embodiment, taking the sensor data as point cloud data as an example, a plurality of observation frames with noise of each frame of point cloud can be obtained by inputting each frame of point cloud data into the target detection network model. The target detection network model may be a network model for performing target detection on point cloud data in the prior art, for example, a CNN-SEG algorithm model. The input of the object detection network model is point cloud data, and the output is the size and position of the object, and is generally represented by a bounding box, for example, a bounding box capable of outputting a plurality of objects such as a person, a car, and a bicycle contained in the point cloud data. That is, the observation frame described in the embodiments of the present application may be obtained by using the object detection network model, and in some embodiments, the output of the object detection network model may further include the speed, acceleration, angle, angular velocity of the object, the number of points in the frame, the size of the frame, the position of the frame, and other accessory features.
Step S13: acquiring a plurality of truth labeling results obtained based on the sensor data, wherein each truth labeling result comprises a truth box and a tracking identifier corresponding to the truth box;
In this embodiment, a label generating system in the prior art may be directly utilized, and multiple true value labeling results including a real three-dimensional frame and a tracking identifier (tracking ID) are generated by inputting laser radar data, through manually labeling multiple targets in the laser radar data, where the laser radar data may be point cloud data acquired by a laser radar sensor.
Optionally, the obtaining the multiple true value labeling results based on the laser radar data may specifically be: manually marking the laser radar data to obtain a plurality of truth boxes and tracking identifications corresponding to each truth box, and storing the obtained truth boxes and the tracking identifications corresponding to each truth box into a data set system as a plurality of truth marking results;
In the embodiment of the application, the laser radar data is manually marked only once, the marked results are stored in the data set system, and then the true value marked results can be directly obtained from the data set system. Therefore, optionally, the obtaining the multiple true value labeling results based on the laser radar data may further specifically be: and acquiring the multiple true value labeling results from a data set system according to the laser radar data.
Step S14: matching the truth box and the observation boxes, and determining corresponding tracking identification for each observation box according to a matching processing result;
In the existing method, an online mode is generally adopted, the observation frame and the truth frame are matched through the fact that the intersection ratio (IOU) of the observation frame and the truth frame of the camera or the radar is larger than a certain threshold value, and the tracking ID of the successfully matched truth frame is directly given to the current observation frame. However, when the matching method is applied to a multi-target tracking task based on the detection result of laser radar data, the following problems are faced:
1. The 3D bounding box of the object is smaller and requires higher accuracy, so that it is likely that the three-dimensional intersection ratio (IOU) of the eligible observation box and the truth box is 0, resulting in failure to perform effective matching.
2. The laser radar detection results may have a jump condition at the boundary, that is, the position of the observation frame in the adjacent frame changes drastically, which may cause difficulty in the matching process.
Based on the above-mentioned problems, an offline data labeling post-processing algorithm is adopted in the present embodiment to process the matching problem between the observation frame and the truth frame. Unlike the existing method, the off-line processing mode can fully consider the accuracy requirement of the 3D bounding box and the possible jump situation in the laser radar detection result.
Optionally, as shown in fig. 3, one implementation of this step specifically includes the following steps S141 to S143.
Step S141: constructing an incidence matrix based on the multiple truth frames and the multiple observation frames;
Specifically, in this embodiment, the constructed correlation matrix may be represented as c= [ C ij]M*N, where M represents M observation frames, N represents N truth frames, and the matrix size is m×n.
The calculation formula of the matrix element c ij in the correlation matrix is:
Wherein 3D distance represents the three-dimensional center point distance of the i-th observation box and the j-th truth box, and iou represents the intersection ratio of the i-th observation box and the j-th truth box.
Optionally: the specific implementation mode of the step is as follows:
for each observation frame, calculating the distance and the intersection ratio of the three-dimensional center points of the observation frame and the truth frame;
calculating according to the three-dimensional center point distance and the intersection ratio to obtain a correlation value;
and constructing an association matrix by taking the association value as a matrix element.
Step S142: determining a matching corresponding relation between the truth box and the observation box based on the incidence matrix;
specifically, in this embodiment, based on the correlation matrix, a matching correspondence relationship between the truth box and the observation box is determined by using a hungarian algorithm.
In the embodiment of the application, a classical Hungary algorithm is firstly applied to matching of an observation frame and a truth frame of a single frame so as to solve the problem of maximum matching of bipartite graphs, and based on a constructed correlation matrix, a global optimal matching is found by adopting an optimization method, so that the cost between the observation frame and the truth frame is minimized, wherein the cost is represented by a value c ij in the matrix.
Therefore, the Hungary algorithm in the scheme has important significance in the detection and tracking association of a single frame, and provides an innovative solution for the further development of a multi-target tracking technology. By applying the algorithm, the method can overcome challenges possibly appearing in the laser radar detection result and improve the accuracy and performance of the multi-target tracking task system. This provides a powerful support for the development of fields such as achieving accurate target tracking and automatic driving.
Step S143: and assigning the tracking identification corresponding to the truth box to the tracking identification corresponding to the observation box matched with the truth box based on the matching corresponding relation.
A preferred implementation of this step is specifically as follows:
Screening the matching corresponding relation based on a preset distance threshold;
Specifically, whether the distance between the observation frame and the truth frame (such as 3D distance) is larger than the preset distance threshold value is judged, if so, the matching corresponding relation which does not meet the condition is identified, and screening is not passed; otherwise, screening the matching corresponding relation meeting the condition, namely screening.
And assigning tracking identifications corresponding to the truth boxes in the matched corresponding relation to the tracking identifications corresponding to the observation boxes in the screened matched corresponding relation.
For example, if the tracking identifier corresponding to the truth box in the matching correspondence is 10, the tracking identifier corresponding to the observation box after assignment is also 10.
And aiming at the observation frames in the matching corresponding relation which do not pass through screening, assigning a preset identification value to the tracking identification corresponding to the observation frames.
For example, the preset identification value is taken as-1.
Step S15: and determining a multi-target tracking detection result according to the plurality of observation frames and tracking identifiers corresponding to the observation frames.
Specifically, in this embodiment, a data set formed by a plurality of observation frames and tracking identifiers corresponding to the respective observation frames is determined as a multi-target tracking detection result. It is understood that tracking identifiers corresponding to a plurality of observation frames and each observation frame can be obtained based on each frame of point cloud, so that a multi-target tracking detection result of the current frame of point cloud is formed.
Further, the multi-target tracking detection result obtained based on the steps S11 to S15 may also be used as training data for training a target tracking model, where the target tracking model is used for implementing multi-target tracking tasks, so that the multi-target tracking detection method based on the embodiment of the application may also be used for generating training data of different application scenarios, and thus performance and efficiency for implementing multi-target tracking tasks based on the target tracking model may be effectively improved. Based on the multi-target tracking detection method provided by the embodiment of the application, accurate and reliable matching can be realized by successfully applying the Hungary algorithm in detection and tracking association of a single frame, and a reliable reference is provided for data annotation and training of training data. The innovative application fully plays the advantages of the Hungary algorithm, and the matching corresponding relation between the observation frame and the truth frame is established in a globally optimal mode. By the method, a corresponding tracking ID can be accurately assigned to each observation frame, so that high-quality training data is provided for multi-target tracking tasks.
In addition, in order to facilitate management of training data, the truth labeling results, the multitask tracking detection results and the accessory feature information of each observation frame obtained in the steps S11 to S15 may be stored in the data set system, so that the training data may be directly obtained from the data set system when training of the multitask tracking model is performed, or the truth labeling results may be directly obtained from the data set system when the training data needs to be updated based on the method of the embodiment of the present application. Alternatively, the ancillary characteristic information of each observation box may be information including one or more of the number of points within the box, the size of the box, the position, speed, acceleration, angle, and angular velocity of the box.
In practical application, the multi-target tracking detection method provided by the application is used for realizing pipelined end-to-end network data production when training data are generated, and therefore, the method provided by the application can further comprise the following steps: and displaying the multiple true value labeling results, the multiple observation frames and the finally obtained multi-target tracking detection result which are obtained in the steps S11 to S15 on the basis of a visual interface.
Fig. 4 is a schematic diagram of a visual interface of the multi-target tracking detection method according to the present application, in which only part of truth labeling results are shown by way of example, for example, a truth box (tracking ID is 16) and a truth box (tracking ID is 9) shown in the figure refer to one truth labeling result respectively, and the truth box, tracking ID and motion trail corresponding to each truth labeling result can be intuitively seen from the figure. Also illustrated is a portion of the multi-target tracking test results, where each tracking test result includes an observation box and a corresponding tracking ID, for example, the tracking test result of one target object indicated by the inspection observation box (tracking ID is 25) and the observation box (tracking ID is 35) respectively shown in the figure, and each tracking test result is obtained by finding a true box corresponding to the matching for the observation box through the matching process, and further determining the tracking ID for the observation box.
The multi-target tracking detection method provided by the embodiment of the application has wide applicability and can be suitable for three-dimensional target detection network results of different automatic driving scenes and different versions. It also provides an effective tool for researchers and developers to generate training data of high quality, diversity and robustness. By using the data set of the training data generated by the multi-target tracking detection method provided by the embodiment of the application, the deep learning network can better learn and understand the real-world multi-target tracking task, thereby improving the generalization capability and the application performance of the target tracking model. This will have a positive and important impact on further developments and practical applications of autopilot technology.
Further, in practical application, the multi-target tracking detection method based on the application can develop a data production line for pipelining different versions of training data for adapting to different detection frames and application scenes. The data line may comprise four steps: cloud detection framework, truth data set acquisition, post-processing matching and data writing into the data set. Specifically:
cloud detection frame: the method comprises a lidar detection network model and a post-detection processing part. The lidar detection network receives an original point cloud as data input and outputs three types: and detecting surrounding frames of people, vehicles and bicycles. The post-detection processing part uses the point cloud sequence to obtain the accessory characteristics of each detection bounding box, such as speed, acceleration, the number of points in the box and the like.
True value dataset acquisition: the manually marked truth data set comprises a perfect truth box of each frame, and after the post-processing of the truth generation, the truth box is provided with a perfect tracking ID to serve as an associated truth value of a downstream target tracking network, and filtering-based smoothing is performed according to the track to serve as a truth value of state estimation of the target tracking network. The data set of the perfect truth box only needs to be marked once, then the cloud detection frame is updated each time to generate different imperfect detection frames at the upstream, and different post-processing matching algorithms are updated, so that the original perfect truth value is not required to be changed, and only the matching is required to be carried out again. And each time the production line is operated, the true value labeling result of the current data is required to be downloaded from the data set system.
Post-processing matching: in the post-processing matching process, the tracking ID of the truth box is assigned to the observation box which can be associated with the truth box. And setting the tracking ID of the detection frame to-1 for the false detection result.
Writing a data set: and the results obtained by the data production line are written into the data set, so that version management is convenient. The content of the output of the production line requires that each frame has a detection and truth result, wherein the detection and truth result is a set of a plurality of frames, and each frame comprises the characteristics of size, position, angle, speed, acceleration, angular speed and the like.
Furthermore, the application also provides a multi-target tracking detection device.
Referring to fig. 5, fig. 5 is a main block diagram of a multi-target tracking detection apparatus according to an embodiment of the present application, which includes an acquisition module 201, a detection module 202, a matching module 203, and a determination module 204. Wherein each module functions as follows:
An acquisition module 201, configured to acquire sensor data acquired by a vehicle-mounted sensor; acquiring a plurality of truth labeling results obtained based on the sensor data, wherein each truth labeling result comprises a truth box and a tracking identifier corresponding to the truth box;
a detection module 202 for obtaining a plurality of observation frames based on the sensor data and a target detection network model;
the matching module 203 is configured to perform matching processing on the truth box acquired by the acquiring module 201 and the observation box acquired by the detecting module 202, and determine a corresponding tracking identifier for each observation box according to a matching processing result;
The determining module 204 is configured to determine a multi-target tracking detection result according to the multiple observation frames and tracking identifiers corresponding to the respective observation frames.
Optionally, the apparatus may further include a truth labeling module for generating a plurality of truth labeling results based on the sensor data.
For convenience of description, the description of the multi-target tracking detection apparatus only shows the parts related to the embodiments of the present application, and specific technical details are not disclosed, please refer to the method parts of the embodiments of the present application.
It will be appreciated by those skilled in the art that the present application may implement all or part of the above-described methods according to the above-described embodiments, or may be implemented by means of a computer program for instructing relevant hardware, where the computer program may be stored in a computer readable storage medium, and where the computer program may implement the steps of the above-described embodiments of the method when executed by a processor. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable storage medium may include: any entity or device, medium, usb disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier wave signals, telecommunications signals, software distribution media, and the like capable of carrying the computer program code.
Further, the application also provides an intelligent device, which can comprise at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores a computer program which, when executed by the at least one processor, implements the method of any of the embodiments described above. The intelligent equipment provided by the application can comprise driving equipment, intelligent vehicles, robots and other equipment.
In some embodiments of the application, the smart device further comprises at least one sensor for sensing information. The sensor is communicatively coupled to a processor of any of the types mentioned herein. Optionally, the intelligent device further comprises an automatic driving system, and the automatic driving system is used for guiding the intelligent device to drive by itself or assist driving. The processor communicates with the sensors and/or the autopilot system for performing the method of any one of the embodiments described above.
The intelligent device in the embodiment of the application can be a control device formed by various electronic devices. In some possible implementations, the smart device may include a plurality of storage devices and a plurality of processors. The program for executing the multi-target tracking detection method of the above-described method embodiment may be divided into a plurality of sub-programs, and each sub-program may be loaded and executed by a processor to execute the different steps of the multi-target tracking detection method of the above-described method embodiment. In particular, each segment of the sub-program may be stored in a different storage device, respectively, and each processor may be configured to execute the programs in one or more storage devices to collectively implement the multi-target tracking detection method of the above-described method embodiments.
The plurality of processors may be processors disposed on the same device, for example, the smart device may be a high-performance device composed of a plurality of processors, and the plurality of processors may be processors configured on the high-performance device. In addition, the multiple processors may be processors disposed on different devices, for example, the intelligent device may be a server cluster, and the multiple processors may be processors on different servers in the server cluster.
Further, the application also provides a computer readable storage medium.
In one embodiment of a computer readable storage medium according to the present application, the computer readable storage medium may be configured to store a program for performing the multi-target tracking detection method of the above-described method embodiment, which may be loaded and executed by a processor to implement the multi-target tracking detection method described above. For convenience of explanation, only those portions of the embodiments of the present application that are relevant to the embodiments of the present application are shown, and specific technical details are not disclosed, please refer to the method portions of the embodiments of the present application. The computer readable storage medium may be a storage device including various electronic devices, and optionally, the computer readable storage medium in the embodiments of the present application is a non-transitory computer readable storage medium.
The personal information of the related user possibly related in each embodiment of the application is personal information which is actively provided by the user or generated by using the product/service in the process of using the product/service and is obtained by authorization of the user, and is processed based on the reasonable purpose of the business scene according to legal, legal and necessary principles strictly according to the requirements of laws and regulations.
The personal information of the user processed by the application can be different according to specific product/service scenes, and the personal information of the user can relate to account information, equipment information, driving information, vehicle information or other related information of the user according to the specific scene of using the product/service by the user. The applicant would treat the user's personal information and its processing with a high diligence.
The application is very important to the safety of the personal information of the user, and adopts reasonable and feasible safety protection measures which accord with the industry standard to protect the information of the user and prevent the personal information from unauthorized access, disclosure, use, modification, damage or loss.
Thus far, the technical solution of the present application has been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present application is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present application, and such modifications and substitutions will fall within the scope of the present application.
Claims (10)
1. A multi-target tracking detection method, the method comprising:
acquiring laser radar data;
Obtaining a plurality of observation frames based on the laser radar data and a target detection network model;
acquiring a plurality of truth labeling results obtained based on the laser radar data, wherein each truth labeling result comprises a truth box and a tracking identifier corresponding to the truth box;
Matching the truth box and the observation boxes, and determining corresponding tracking identification for each observation box according to a matching processing result;
And determining a multi-target tracking detection result according to the plurality of observation frames and tracking identifiers corresponding to the observation frames.
2. The method according to claim 1, wherein the matching the truth box and the observation box, and determining the corresponding tracking identifier for each observation box according to the matching result specifically includes:
Constructing an incidence matrix based on a plurality of the truth boxes and a plurality of the observation boxes;
determining a matching corresponding relation between the truth box and the observation box based on the incidence matrix;
And assigning the tracking identification corresponding to the truth box to the tracking identification corresponding to the observation box matched with the truth box based on the matching corresponding relation.
3. The method according to claim 2, wherein said constructing an association matrix based on a plurality of said truth boxes and a plurality of said observation boxes comprises:
For each observation frame, calculating the distance and the intersection ratio of the three-dimensional center point of the observation frame and each truth frame;
calculating according to the three-dimensional center point distance and the intersection ratio to obtain a correlation value;
and constructing an association matrix by taking the association value as a matrix element.
4. The method according to claim 2, wherein the determining the matching correspondence between the truth box and the observation box based on the correlation matrix is specifically: and based on the incidence matrix, determining the matching corresponding relation of the truth box and the observation box by using a Hungary algorithm.
5. The method according to claim 2, wherein assigning the tracking identifier corresponding to the truth box to the tracking identifier corresponding to the observation box matched with the truth box based on the matching correspondence relation specifically comprises:
Screening the matching corresponding relation based on a preset distance threshold;
and assigning tracking identifications corresponding to the truth boxes in the matched corresponding relation to the tracking identifications corresponding to the observation boxes in the screened matched corresponding relation.
And aiming at the observation frames in the matching corresponding relation which do not pass through screening, assigning a preset identification value to the tracking identification corresponding to the observation frames.
6. The method according to claim 1, wherein the obtaining a plurality of true value labeling results based on the lidar data is specifically: manually labeling the laser radar data based on a label generating system to obtain a plurality of truth boxes and tracking identifications corresponding to each truth box, and storing the obtained truth boxes and the tracking identifications corresponding to each truth box into a data set system as a plurality of truth labeling results;
or specifically: and acquiring the multiple true value labeling results from a data set system according to the laser radar data.
7. The method according to claim 1, wherein the method further comprises: and obtaining auxiliary characteristic information of each observation frame based on the laser radar data and the target detection network model, wherein the auxiliary characteristic information at least comprises one of the number of points in the frame, the size of the frame, the position, the speed, the acceleration, the angle and the angular speed of the frame.
8. The method according to claim 1, wherein the method further comprises: and displaying the multiple true value labeling results, the multiple observation frames and the multi-target tracking detection result based on a visual interface.
9. An intelligent device, comprising:
At least one processor;
and a memory communicatively coupled to the at least one processor;
Wherein the memory has stored therein a computer program which, when executed by the at least one processor, implements the multi-target tracking detection method of any of claims 1 to 8.
10. A computer readable storage medium having stored therein a plurality of program codes, wherein the program codes are adapted to be loaded and executed by a processor to perform the multi-target tracking detection method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410141888.4A CN117991283A (en) | 2024-01-31 | 2024-01-31 | Multi-target tracking detection method, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410141888.4A CN117991283A (en) | 2024-01-31 | 2024-01-31 | Multi-target tracking detection method, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117991283A true CN117991283A (en) | 2024-05-07 |
Family
ID=90894825
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410141888.4A Pending CN117991283A (en) | 2024-01-31 | 2024-01-31 | Multi-target tracking detection method, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117991283A (en) |
-
2024
- 2024-01-31 CN CN202410141888.4A patent/CN117991283A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111222395B (en) | Target detection method and device and electronic equipment | |
| CN109300151B (en) | Image processing method and device and electronic equipment | |
| CN111928842B (en) | Monocular vision based SLAM positioning method and related device | |
| CN111928857B (en) | Method and related device for realizing SLAM positioning in dynamic environment | |
| CN114926766A (en) | Identification method and device, equipment and computer readable storage medium | |
| WO2019175532A1 (en) | Urban environment labelling | |
| CN112434566A (en) | Passenger flow statistical method and device, electronic equipment and storage medium | |
| Liu et al. | Vehicle-related distance estimation using customized YOLOv7 | |
| US11436452B2 (en) | System and method for label augmentation in video data | |
| CN114898314A (en) | Target detection method, device and equipment for driving scene and storage medium | |
| US20230350418A1 (en) | Position determination by means of neural networks | |
| CN111754388B (en) | Picture construction method and vehicle-mounted terminal | |
| CN113902047B (en) | Image element matching method, device, equipment and storage medium | |
| Lee et al. | SAM-net: LiDAR depth inpainting for 3D static map generation | |
| Barra et al. | Can Existing 3D Monocular Object Detection Methods Work in Roadside Contexts? A Reproducibility Study | |
| CN117991283A (en) | Multi-target tracking detection method, equipment and medium | |
| CN113469045B (en) | Visual positioning method and system for unmanned integrated card, electronic equipment and storage medium | |
| CN114937248A (en) | Vehicle tracking method and device for cross-camera, electronic equipment and storage medium | |
| CN114140497A (en) | Target vehicle 3D real-time tracking method and system | |
| Tamayo et al. | Improving object distance estimation in automated driving systems using camera images, LiDAR point clouds and hierarchical clustering | |
| US10896333B2 (en) | Method and device for aiding the navigation of a vehicle | |
| CN111401194A (en) | Data processing method and device for automatic driving vehicle | |
| Fan et al. | Autonomous Vehicle Vision 2021: ICCV Workshop Summary | |
| CN116385336B (en) | Deep learning-based weld joint detection method, system, device and storage medium | |
| Parimi et al. | Dynamic speed estimation of moving objects from camera data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |