CN111652907A - Multi-target tracking method and device based on data association and electronic equipment - Google Patents
Multi-target tracking method and device based on data association and electronic equipment Download PDFInfo
- Publication number
- CN111652907A CN111652907A CN201911358077.5A CN201911358077A CN111652907A CN 111652907 A CN111652907 A CN 111652907A CN 201911358077 A CN201911358077 A CN 201911358077A CN 111652907 A CN111652907 A CN 111652907A
- Authority
- CN
- China
- Prior art keywords
- target
- image data
- association
- continuous frames
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20076—Probabilistic image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a multi-target tracking method, a multi-target tracking device and electronic equipment based on data association, wherein the method comprises the following steps: acquiring any two continuous frames of image data in video data, wherein the any two continuous frames of image data are divided into current frame image data and next frame image data; detecting each target in any two continuous frames of image data according to a target detection algorithm; and correspondingly associating each target in any two continuous frames of image data according to a target association algorithm to form a plurality of target tracks. By implementing the method and the device, the multi-target can be quickly tracked.
Description
Technical Field
The invention relates to the field of machine vision, in particular to a multi-target tracking method and device based on data association and electronic equipment.
Background
The multi-target tracking means that a plurality of targets in a video image are detected and tracked to achieve the purpose of multi-target monitoring. With the progress of real-time monitoring technology, the monitoring of the target is no longer only tracked for a single target, and in many scenes, multiple targets in video data need to be tracked.
Disclosure of Invention
Therefore, the technical problem to be solved by the present invention is to overcome the defect that the single target tracking algorithm in the prior art cannot meet the requirement of multi-target tracking, so as to provide a multi-target tracking method, device and electronic device based on data association.
According to a first aspect, an embodiment of the present invention provides a multi-target tracking method based on data association, including the following steps: acquiring any two continuous frames of image data in video data, wherein the any two continuous frames of image data are divided into current frame image data and next frame image data; detecting each target in any two continuous frames of image data according to a target detection algorithm; and correspondingly associating each target in any two continuous frames of image data according to a target association algorithm to form a plurality of target tracks.
With reference to the first aspect, in a first implementation manner of the first aspect, the detecting each object in the arbitrary two consecutive frames of image data according to an object detection algorithm includes: and inputting any two continuous frames of image data in the video data into a YOLO neural network detection model, and acquiring each target in the any two continuous frames of image data.
With reference to the first aspect, in a second implementation manner of the first aspect, the associating the targets according to a target association algorithm includes: dividing each target in any two continuous frames of image data into two sets, dividing each target in the current frame of image data into a current set, and dividing each target in the next frame of image data into a next set; performing matching calculation on the targets in the two sets to obtain a matching value, wherein the matching value represents the probability that the targets in the two sets are the same target; selecting each target of which the matching value exceeds a first threshold value in the two sets, and pre-associating each target; and in the pre-association data, performing data association between any target in the current set and the pre-associated target in the latter set.
With reference to the second embodiment of the first aspect, in a third embodiment of the first aspect, the method further includes: when the pre-associated targets in the latter set are associated, selecting the targets in the current set and other pre-associated targets of the pre-associated targets in the latter set to form an augmentation path; exchanging the target in the augmented path, and performing data association.
With reference to the second implementation manner of the first aspect, in a fourth implementation manner of the first aspect, the performing matching calculation on the targets in the two sets to obtain a matching value specifically includes: obtaining a motion parameter of a target, predicting the motion track of the target according to the motion parameter, and obtaining a predicted position of the target; judging the motion matching degree according to the target prediction position and the target detection result; judging the appearance matching degree of the target detection result in the two continuous frames of image data according to the minimum cosine distance; and determining the matching weight according to the motion matching degree and the appearance matching degree.
According to a second aspect, an embodiment of the present invention provides a multi-target tracking apparatus based on data association, including: the image data acquisition module is used for acquiring any two continuous frames of image data in the video data, wherein the any two continuous frames of image data are divided into current frame image data and next frame image data; the target detection module is used for detecting each target in any two continuous frames of image data according to a target detection algorithm; and the target association module is used for correspondingly associating each target in any two continuous frames of image data according to a target association algorithm to form a plurality of target tracks.
With reference to the second aspect, in a first implementation manner of the second aspect, the object detection module includes: and the target detection submodule is used for inputting any two continuous frames of image data in the video data into a YOLO neural network detection model and acquiring targets in the any two continuous frames of image data.
With reference to the second aspect, in a second implementation manner of the second aspect, the target associating module includes: the set dividing module is used for dividing each target in any two continuous frames of image data into two sets, wherein each target in the current frame of image data is divided into a current set, and each target in the next frame of image data is divided into a next set; a matching value calculation module, configured to perform matching calculation on the targets in the two sets to obtain a matching value, where the matching value represents a probability that the targets in the two sets are the same target; the pre-association module is used for selecting each target of which the matching value exceeds a first threshold value in the two sets and pre-associating each target; and the target association submodule is used for performing data association on any target in the current set and the pre-associated target in the latter set in the pre-associated data.
According to a third aspect, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the data association-based multi-target tracking method according to the first aspect or any of the embodiments of the first aspect when executing the program.
According to a fourth aspect, an embodiment of the present invention provides a storage medium, on which computer instructions are stored, and the instructions, when executed by a processor, implement the steps of the data association-based multi-target tracking method according to the first aspect or any one of the embodiments of the first aspect.
The technical scheme of the invention has the following advantages:
1. the invention provides a multi-target tracking method based on data association, which is characterized in that each target in a data image is detected firstly, and each target is associated according to a target association algorithm so as to realize multi-target tracking based on data association and ensure the real-time performance of the multi-target tracking.
2. The method for acquiring the targets in any two continuous frames of image data by using the YOLO neural network detection model can realize one-time multi-target detection, and the YOLO neural network detection model is high in speed and high in detection efficiency, so that the whole multi-target tracking speed based on data association is improved.
3. According to the method, each target in any two continuous frames of image data is divided into two sets, the data in the two sets are correlated, a plurality of data are screened firstly, the data quantity of subsequent correlation is reduced, the data correlation speed is improved, and the real-time performance of multi-target tracking is guaranteed.
4. The method and the device determine the matching weight by utilizing the motion matching degree and the appearance matching degree, not only consider the motion condition of the target, but also consider the appearance matching degree, so that the matching degree of the obtained matching weight is higher.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flow chart of a specific example of a target tracking method in an embodiment of the present invention;
FIG. 2 is a functional block diagram of a specific example of a target tracking device in an embodiment of the present invention;
fig. 3 is a schematic block diagram of a specific example of an electronic device in the embodiment of the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; the two elements may be directly connected or indirectly connected through an intermediate medium, or may be communicated with each other inside the two elements, or may be wirelessly connected or wired connected. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
In addition, the technical features involved in the different embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The embodiment provides a multi-target tracking method based on data association, as shown in fig. 1, including the following steps:
s110, acquiring any two continuous frames of image data in the video data, wherein the any two continuous frames of image data are divided into current frame image data and next frame image data.
For example, the video data may be obtained in any manner of two consecutive frames of image data, and the two consecutive frames of image data after being deframed are divided into a current frame of image data and a next frame of image data. The embodiment does not limit the manner of acquiring any two consecutive frames of image data in the data, and those skilled in the art can determine the manner as needed.
And S120, detecting each target in any two continuous frames of image data according to a target detection algorithm.
Illustratively, the respective targets may represent ships, vehicles, and the like in the image data. The target detection algorithm can be that about 2000 candidate frames of the region are extracted from an original picture through Selective search, all the candidate frames are scaled into a fixed size, features are extracted through a CNN network, two full-connection layers are added on the basis of the feature layers, and classification is carried out through SVM classification. The embodiment does not limit the target detection algorithm and the specific representation of the target, and those skilled in the art can determine the target according to the needs.
S130, correspondingly associating each target in any two continuous frames of image data according to a target association algorithm to form a plurality of target tracks.
Illustratively, the target association algorithm may be an IOU algorithm, and the method includes acquiring each target of any two consecutive frames of image data, sequentially performing cross-over ratio calculation, and performing data association on two targets with the largest cross-over ratio calculation result; or dividing each target of any two continuous frames of image data into two sets, searching the maximum matching of the two sets, and performing data association on the matching result, thereby realizing the multi-target tracking based on the data association. The target association algorithm is not limited in this embodiment, and those skilled in the art can determine the target association algorithm as needed.
The embodiment provides a multi-target tracking method based on data association, which detects each target in a data image, associates each target according to a target association algorithm, and realizes multi-target tracking based on data association.
As an optional implementation manner of the present application, the step S120 includes:
and inputting any two continuous frames of image data in the video data into a YOLO neural network detection model to obtain each target in the any two continuous frames of image data.
Illustratively, the YOLO neural network detection model is a convolutional neural network that can predict a plurality of positions and types at a time, and any two consecutive frames of image data are input to the YOLO neural network detection model, so that each target in the two frames of image data can be detected respectively.
For example, in ship monitoring video data, image data of any two consecutive frames are unframed, and a data image is input into a YOLO neural network detection model, so that all ships in the two frames of image data can be detected.
The embodiment provides a method for acquiring each target in any two continuous frames of image data by using a YOLO neural network detection model, which can realize one-time multi-target detection, and the YOLO neural network detection model has the advantages of high speed and high detection efficiency, and improves the whole multi-target tracking speed based on data association.
As an optional implementation manner of the present application, the step S130 includes:
firstly, dividing each target in any two continuous frames of image data into two sets, dividing each target in the current frame of image data into a current set, and dividing each target in the next frame of image data into a next set.
Illustratively, all the targets in two continuous frames of image data are detected through a target detection algorithm, when all the targets form a target set as a current set, the targets in the next frame of image data form another target set as a next set.
Secondly, performing matching calculation on the targets in the two sets to obtain a matching value, wherein the matching value represents the probability that the targets in the two sets are the same target.
For example, the way of performing matching calculation on the targets in the two sets may be to perform intersection ratio calculation on each target in the two sets, and the calculation result of the intersection ratio is used as the matching value. Taking the target ship as an example, all ships in the current set are detected, the YOLO neural network detection model returns the coordinates of detection frames of all ships in the current set, the coordinates of the detection frames of all ships are sequentially subjected to intersection and comparison calculation, the calculated intersection and comparison values are used as matching values of the current ship, each ship has a plurality of matching values, and different matching values respectively correspond to intersection and comparison calculation results of the ship and each ship in the next set.
And thirdly, selecting each target with the matching value exceeding the first threshold value in the two sets, and pre-associating each target.
Illustratively, still taking the above target as a ship example, since each ship will have several matching values, different matching values respectively correspond to the intersection ratio calculation results of the ship and the ships of the latter set. In order to reduce the amount of associated data, the two sets are filtered according to a first threshold, and only the targets with matching values exceeding the first threshold are pre-associated. The first threshold may be 75% in size, and when the match exceeds 75%, the objects in both sets are pre-associated. The first threshold size in this embodiment is not limited, and can be determined by those skilled in the art as needed.
Then, in the pre-association data, data association is carried out between any target in the current set and the pre-associated target in the next set.
For example, the way of performing data association between any target in the current set and a target pre-associated in the next set may be to select a maximum matching value of the target in the current set as a target assignment, assign 0 to all the targets pre-associated in the next set, perform data association according to the assignments, and perform data association according to the principle of data association, that is, when the matching value is the same as the assignment to the target, perform data association on the target with the same assignment as the matching value, when data association conflicts, subtract a second value from the conflicting target assignment in the current set, add the second value to the conflicting target assignment in the next set, and perform data association again according to the principle described above until the target in the current set and the target data in the next set are associated completely, where the second value may be 0.1. The second numerical value and the data association method are not limited in this embodiment, and those skilled in the art can determine the second numerical value and the data association method as needed.
According to the method, each target in any two continuous frames of image data is divided into two sets, the data in the two sets are correlated, a plurality of data are screened firstly, the data quantity of subsequent correlation is reduced, the data correlation speed is improved, and the real-time performance of multi-target tracking is guaranteed.
As an optional embodiment of the present application, the multi-target tracking method based on data association includes:
when the pre-associated targets in the latter set are associated, selecting the targets in the current set and other pre-associated targets of the pre-associated targets in the latter set to form an augmentation path; and exchanging the targets in the augmented path to perform data association.
As an optional embodiment of the present application, performing matching calculation on targets in two sets to obtain a matching value specifically includes:
firstly, motion parameters of a target are obtained, a target motion track is predicted according to the motion parameters, and a target prediction position is obtained.
The motion parameters may be, for example, direction of motion, velocity, acceleration, resistance, etc. The motion parameter of the target may be obtained by calculation according to the motion of the target in the image data, or may be a preset target motion parameter. The target motion trajectory is predicted according to the motion parameters, and the mode of obtaining the target predicted position can be that a Kalman filter is constructed through the motion parameters, and prediction is carried out according to the Kalman filter. The specific manner of obtaining the target predicted position is not limited in this embodiment, and may be determined as needed.
Secondly, judging the motion matching degree according to the target prediction position and the target detection result; and judging the appearance matching degree of the target detection result in the two continuous frames of image data according to the minimum cosine distance.
For example, the manner of determining the degree of motion matching according to the target predicted position and the target detection result may be to calculate the degree of motion matching using the mahalanobis distance, or may be directly determined as a value of the degree of motion matching according to the intersection ratio using the target predicted position and the target detection result. The method for judging the appearance matching degree of the adjacent target detection result according to the minimum cosine distance can be completed through an appearance model (ReID model), a feature vector of a unit norm is extracted by using a depth network, and the minimum cosine distance between the feature vectors is used as the value of the appearance matching degree. The embodiment does not limit the specific manner of obtaining the motion matching degree and the appearance matching degree, and can determine the motion matching degree and the appearance matching degree as required.
Then, a matching weight is determined according to the degree of motion matching and the degree of appearance matching.
For example, the way of determining the matching weight according to the degree of motion matching and the degree of appearance matching may be by obtaining a weighted sum of the degree of motion matching and the degree of appearance matching. The embodiment does not limit the specific determination method of the matching weight, and may determine the matching weight as needed.
The matching weight is determined by using the motion matching degree and the appearance matching degree, the motion condition of the target is considered, the appearance matching degree is considered, and the matching degree of the obtained matching weight is higher.
The embodiment provides a multi-target tracking device based on data association, as shown in fig. 2, including:
the image data acquiring module 210 is configured to acquire any two consecutive frames of image data in the video data, where the two consecutive frames of image data are divided into a current frame of image data and a next frame of image data; the specific implementation manner is shown in the corresponding part of step S110 of the method of this embodiment, and is not described herein again.
The target detection module 220 is configured to detect each target in any two consecutive frames of image data according to a target detection algorithm; the specific implementation manner is shown in the corresponding part of step S120 of the method of this embodiment, and is not described herein again.
And the target association module 230 is configured to correspondingly associate each target in any two consecutive frames of image data according to a target association algorithm to form a plurality of target tracks. The specific implementation manner is shown in the corresponding part of step S130 of the method of this embodiment, and is not described herein again.
The embodiment provides a multi-target tracking device based on data association, which detects each target in a data image, associates each target according to a target association algorithm, and realizes multi-target tracking based on data association.
As an optional embodiment of the present application, the target detection module 220 includes:
and the target detection submodule is used for inputting any two continuous frames of image data in the video data into the YOLO neural network detection model and acquiring targets in any two continuous frames of image data. The specific implementation manner is shown in the corresponding part of the method of the embodiment, and is not described herein again.
As an optional embodiment of the present application, the target associating module 230 includes:
the set dividing module is used for dividing each target in any two continuous frames of image data into two sets, wherein each target in the current frame of image data is divided into a current set, and each target in the next frame of image data is divided into a next set; the specific implementation manner is shown in the corresponding part of the method of the embodiment, and is not described herein again.
The matching value calculation module is used for performing matching calculation on the targets in the two sets to obtain a matching value, and the matching value represents the probability that the targets in the two sets are the same target; the specific implementation manner is shown in the corresponding part of the method of the embodiment, and is not described herein again.
The pre-association module is used for selecting each target of which the matching value exceeds a first threshold value in the two sets and pre-associating each target; the specific implementation manner is shown in the corresponding part of the method of the embodiment, and is not described herein again.
And the target association submodule is used for performing data association between any target in the current set and a pre-associated target in the next set in the pre-associated data. The specific implementation manner is shown in the corresponding part of the method of the embodiment, and is not described herein again.
As an optional embodiment of the present application, the multi-target tracking apparatus based on data association further includes:
the augmented path selection module is used for selecting other pre-associated targets of the targets in the current set and the pre-associated targets in the next set to form an augmented path when the pre-associated targets in the next set are associated; the specific implementation manner is shown in the corresponding part of the method of the embodiment, and is not described herein again.
And the data association module is used for exchanging the targets in the augmented path and performing data association.
As an optional implementation manner of the present application, the matching value calculating module specifically includes:
the target prediction position acquisition module is used for acquiring the motion parameters of the target and predicting the motion track of the target according to the motion parameters to obtain a target prediction position; the specific implementation manner is shown in the corresponding part of the method of the embodiment, and is not described herein again.
The motion matching degree judging module is used for judging the motion matching degree according to the target prediction position and the target detection result; the specific implementation manner is shown in the corresponding part of the method of the embodiment, and is not described herein again.
Judging the appearance matching degree of target detection results in two continuous frames of image data according to the minimum cosine distance; the specific implementation manner is shown in the corresponding part of the method of the embodiment, and is not described herein again.
And determining the matching weight according to the motion matching degree and the appearance matching degree. The specific implementation manner is shown in the corresponding part of the method of the embodiment, and is not described herein again.
The embodiment of the present application also provides an electronic device, as shown in fig. 3, including a processor 310 and a memory 320, where the processor 310 and the memory 320 may be connected by a bus or in other manners.
Processor 310 may be a Central Processing Unit (CPU). The Processor 310 may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or any combination thereof.
The memory 320, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the multi-target tracking method based on data association in the embodiments of the present invention. The processor executes various functional applications and data processing of the processor by executing non-transitory software programs, instructions, and modules stored in the memory.
The memory 320 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor, and the like. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 320 may optionally include memory located remotely from the processor, which may be connected to the processor via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory 320 and, when executed by the processor 310, perform a multi-target tracking method based on data association as in the embodiment shown in FIG. 1.
The details of the electronic device may be understood with reference to the corresponding related description and effects in the embodiment shown in fig. 1, and are not described herein again.
The embodiment also provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions can execute the multi-target tracking method based on data association in any method embodiment. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard disk (Hard disk Drive, abbreviated as HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications therefrom are within the scope of the invention.
Claims (10)
1. A multi-target tracking method based on data association is characterized by comprising the following steps:
acquiring any two continuous frames of image data in video data, wherein the any two continuous frames of image data are divided into current frame image data and next frame image data;
detecting each target in any two continuous frames of image data according to a target detection algorithm;
and correspondingly associating each target in any two continuous frames of image data according to a target association algorithm to form a plurality of target tracks.
2. The method according to claim 1, wherein the detecting each object in the arbitrary two consecutive frames of image data according to an object detection algorithm comprises:
and inputting any two continuous frames of image data in the video data into a YOLO neural network detection model, and acquiring each target in the any two continuous frames of image data.
3. The method of claim 1, wherein said associating the respective targets according to a target association algorithm comprises:
dividing each target in any two continuous frames of image data into two sets, dividing each target in the current frame of image data into a current set, and dividing each target in the next frame of image data into a next set;
performing matching calculation on the targets in the two sets to obtain a matching value, wherein the matching value represents the probability that the targets in the two sets are the same target;
selecting each target of which the matching value exceeds a first threshold value in the two sets, and pre-associating each target;
and in the pre-association data, performing data association between any target in the current set and the pre-associated target in the latter set.
4. The method of claim 3, further comprising:
when the pre-associated targets in the latter set are associated, selecting the targets in the current set and other pre-associated targets of the pre-associated targets in the latter set to form an augmentation path;
exchanging the target in the augmented path, and performing data association.
5. The method according to claim 3, wherein the performing matching calculation on the targets in the two sets to obtain a matching value specifically includes:
obtaining a motion parameter of a target, predicting the motion track of the target according to the motion parameter, and obtaining a predicted position of the target;
judging the motion matching degree according to the target prediction position and the target detection result;
judging the appearance matching degree of the target detection result in the two continuous frames of image data according to the minimum cosine distance;
and determining the matching weight according to the motion matching degree and the appearance matching degree.
6. A multi-target tracking apparatus based on data association, comprising:
the image data acquisition module is used for acquiring any two continuous frames of image data in the video data, wherein the any two continuous frames of image data are divided into current frame image data and next frame image data;
the target detection module is used for detecting each target in any two continuous frames of image data according to a target detection algorithm;
and the target association module is used for correspondingly associating each target in any two continuous frames of image data according to a target association algorithm to form a plurality of target tracks.
7. The apparatus of claim 6, wherein the object detection module comprises:
and the target detection submodule is used for inputting any two continuous frames of image data in the video data into a YOLO neural network detection model and acquiring targets in the any two continuous frames of image data.
8. The apparatus of claim 6, wherein the target association module comprises:
the set dividing module is used for dividing each target in any two continuous frames of image data into two sets, wherein each target in the current frame of image data is divided into a current set, and each target in the next frame of image data is divided into a next set;
a matching value calculation module, configured to perform matching calculation on the targets in the two sets to obtain a matching value, where the matching value represents a probability that the targets in the two sets are the same target;
the pre-association module is used for selecting each target of which the matching value exceeds a first threshold value in the two sets and pre-associating each target;
and the target association submodule is used for performing data association on any target in the current set and the pre-associated target in the latter set in the pre-associated data.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the data association based multi-target tracking method of any one of claims 1 to 5 when executing the program.
10. A storage medium having stored thereon computer instructions, which when executed by a processor, perform the steps of the data association based multi-target tracking method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911358077.5A CN111652907B (en) | 2019-12-25 | 2019-12-25 | Multi-target tracking method and device based on data association and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911358077.5A CN111652907B (en) | 2019-12-25 | 2019-12-25 | Multi-target tracking method and device based on data association and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111652907A true CN111652907A (en) | 2020-09-11 |
| CN111652907B CN111652907B (en) | 2021-08-27 |
Family
ID=72347897
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911358077.5A Active CN111652907B (en) | 2019-12-25 | 2019-12-25 | Multi-target tracking method and device based on data association and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111652907B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023071273A1 (en) * | 2021-10-29 | 2023-05-04 | 上海商汤智能科技有限公司 | Point cloud data processing |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100124358A1 (en) * | 2008-11-17 | 2010-05-20 | Industrial Technology Research Institute | Method for tracking moving object |
| WO2013043020A2 (en) * | 2011-09-23 | 2013-03-28 | 주식회사 크리에이츠 | System and method for photographing moving subject by means of multiple cameras, and acquiring actual movement trajectory of subject based on photographed images |
| CN108875600A (en) * | 2018-05-31 | 2018-11-23 | 银江股份有限公司 | A kind of information of vehicles detection and tracking method, apparatus and computer storage medium based on YOLO |
| CN109816690A (en) * | 2018-12-25 | 2019-05-28 | 北京飞搜科技有限公司 | Multi-target tracking method and system based on depth characteristic |
| CN110047096A (en) * | 2019-04-28 | 2019-07-23 | 中南民族大学 | A kind of multi-object tracking method and system based on depth conditions random field models |
-
2019
- 2019-12-25 CN CN201911358077.5A patent/CN111652907B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100124358A1 (en) * | 2008-11-17 | 2010-05-20 | Industrial Technology Research Institute | Method for tracking moving object |
| WO2013043020A2 (en) * | 2011-09-23 | 2013-03-28 | 주식회사 크리에이츠 | System and method for photographing moving subject by means of multiple cameras, and acquiring actual movement trajectory of subject based on photographed images |
| CN108875600A (en) * | 2018-05-31 | 2018-11-23 | 银江股份有限公司 | A kind of information of vehicles detection and tracking method, apparatus and computer storage medium based on YOLO |
| CN109816690A (en) * | 2018-12-25 | 2019-05-28 | 北京飞搜科技有限公司 | Multi-target tracking method and system based on depth characteristic |
| CN110047096A (en) * | 2019-04-28 | 2019-07-23 | 中南民族大学 | A kind of multi-object tracking method and system based on depth conditions random field models |
Non-Patent Citations (1)
| Title |
|---|
| 陈慧岩等: "《智能车辆理论与应用》", 31 July 2018, 北京理工大学出版社 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023071273A1 (en) * | 2021-10-29 | 2023-05-04 | 上海商汤智能科技有限公司 | Point cloud data processing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111652907B (en) | 2021-08-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2016352215B2 (en) | Method and device for tracking location of human face, and electronic equipment | |
| JP2018508078A (en) | System and method for object tracking | |
| EP3186780B1 (en) | System and method for image scanning | |
| US20140205141A1 (en) | Systems and methods for tracking and detecting a target object | |
| JP2016517084A (en) | Computer-based method of dynamic category object recognition Computer-based method and system | |
| WO2016179808A1 (en) | An apparatus and a method for face parts and face detection | |
| KR102476022B1 (en) | Face detection method and apparatus thereof | |
| CN112749726B (en) | Training method and device for target detection model, computer equipment and storage medium | |
| US20210174116A1 (en) | Neural network system and operating method thereof | |
| CN112926461B (en) | Neural network training and driving control method and device | |
| CN111047088A (en) | Prediction image acquisition method and device, computer equipment and storage medium | |
| CN115147745A (en) | Small target detection method based on urban unmanned aerial vehicle image | |
| CN109685062A (en) | A kind of object detection method, device, equipment and storage medium | |
| CN112001948A (en) | Target tracking processing method and device | |
| CN110866428A (en) | Target tracking method and device, electronic equipment and storage medium | |
| CN111652907B (en) | Multi-target tracking method and device based on data association and electronic equipment | |
| CN116931583B (en) | Method, device, equipment and storage medium for determining and avoiding moving object | |
| CN111611836A (en) | Ship detection model training and ship tracking method based on background elimination method | |
| CN111553474A (en) | Ship detection model training method and ship tracking method based on unmanned aerial vehicle video | |
| CN112347818B (en) | Method and device for screening difficult sample images of video target detection model | |
| CN111681264A (en) | Real-time multi-target tracking method for monitoring scene | |
| CN111292331A (en) | Image processing method and device | |
| CN112149698A (en) | Method and device for screening difficult sample data | |
| CN113744518B (en) | Method and device for detecting vehicle travelable area | |
| EP3905116A1 (en) | Image processing system for identifying and tracking objects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |