CN111512317B - Multi-target real-time tracking method and device and electronic equipment - Google Patents
Multi-target real-time tracking method and device and electronic equipment Download PDFInfo
- Publication number
- CN111512317B CN111512317B CN201880083620.2A CN201880083620A CN111512317B CN 111512317 B CN111512317 B CN 111512317B CN 201880083620 A CN201880083620 A CN 201880083620A CN 111512317 B CN111512317 B CN 111512317B
- Authority
- CN
- China
- Prior art keywords
- information
- target
- current
- targets
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
Abstract
A multi-target real-time tracking method, a device and an electronic device, wherein the method comprises the following steps: acquiring image information (101), wherein the image information comprises current frame information and previous frame information of a plurality of targets; according to the current frame information and the last frame information of the targets, judging whether one time of matching of at least one target in the targets is successful or not (102); if the at least one target is not successfully matched once, carrying out secondary matching, and judging whether the at least one target is successfully matched twice (103); and forming output information (104) of the at least one target with successful primary matching and/or successful secondary matching, wherein the output information comprises current existence information and identification information. The tracking accuracy can be increased by matching at least one target twice in one frame of image.
Description
Technical Field
The invention relates to the field of software development, in particular to a multi-target real-time tracking method, a multi-target real-time tracking device and electronic equipment.
Background
Tracking may determine the motion profile of a target (object or person). Current single-target tracking algorithms, such as based on correlation filtering (KCF), can perform real-time single-target tracking on low-power devices. However, in the terminal device, due to limitation of power consumption, the current tracking algorithm has a real-time problem for multi-target tracking. Particularly, when the total number of targets is more than 10, the combination of a plurality of single-target tracking algorithms is large in calculation amount and high in data processing delay, so that the tracking accuracy obtained by using the tracking mode is low.
Disclosure of Invention
The invention aims at overcoming the defects in the prior art, and provides a multi-target real-time tracking method, a multi-target real-time tracking device and electronic equipment, which solve the problem of low tracking accuracy.
The aim of the invention is realized by the following technical scheme:
in a first aspect, a multi-target real-time tracking method is provided, the method comprising:
acquiring image information, wherein the image information comprises current frame information and previous frame information of a plurality of targets;
according to the current frame information and the previous frame information of the targets, performing primary matching, and judging whether at least one target in the targets is successfully matched once or not;
if the at least one target is not successfully matched once, carrying out secondary matching, and judging whether the at least one target is successfully matched twice or not, wherein the secondary matching comprises at least one of feature matching and distance matching;
and forming the information of the at least one target which is successfully matched once and/or successfully matched twice into output information, wherein the output information comprises current existence information and identification information.
Optionally, after the performing the secondary matching if the at least one target does not succeed in the primary matching, determining whether the at least one target succeeds in the secondary matching further includes:
if the secondary matching is unsuccessful, regenerating the current frame information of the residual target, and acquiring new image information, wherein the new image information comprises the current frame information and the next frame information.
Optionally, the current frame information includes current detection information of the multiple targets, and the previous frame information includes historical existence information and corresponding identification information of the multiple targets;
the step of performing primary matching according to the current frame information and the previous frame information of the plurality of targets, and the step of judging whether at least one target of the plurality of targets is successfully matched once comprises the following steps:
performing overlapping degree calculation on the current detection information of at least one target of the plurality of targets and the historical presence information of at least one target of the plurality of targets to obtain overlapping degree of the current detection information of the at least one target and the historical presence information of the at least one target;
and judging whether the at least one target is successfully matched once according to the overlapping degree.
Optionally, the determining whether the at least one target matches successfully according to the overlapping degree includes:
selecting the maximum overlapping degree and comparing with a preset overlapping degree threshold value, and judging whether the maximum overlapping degree is larger than the overlapping degree threshold value or not;
if the maximum overlapping degree is larger than the overlapping degree threshold value, the primary matching is successful, and if the maximum overlapping degree is smaller than the overlapping degree threshold value, the primary matching is unsuccessful.
Optionally, the current frame information includes current detection information of the multiple targets, and the previous frame information includes historical existence information and corresponding identification information of the multiple targets;
the performing secondary matching, and determining whether the at least one target is successfully matched secondarily includes:
extracting a current feature vector of current detection information of at least one of the plurality of targets, extracting a historical feature vector of historical presence information of at least one of the plurality of targets, and calculating the current feature vector and the historical feature vector to obtain cosine similarity of the at least one target;
extracting current coordinates of current detection information of the at least one target and historical coordinates of historical presence information of the at least one target, and calculating the current coordinates and the historical coordinates of the at least one target to obtain a distance value of the at least one target;
and judging whether the at least one target is successfully matched secondarily according to the cosine similarity and the distance value of the at least one target.
Optionally, the determining whether the at least one target is successfully matched twice according to the cosine similarity and the distance value of the at least one target includes:
comparing the cosine similarity with a preset cosine similarity threshold value, and comparing the distance value with a preset distance threshold value to obtain a comparison result;
and judging whether the secondary matching of the at least one target is successful or not according to the comparison result and a preset judging rule.
Optionally, the forming output information of the at least one target that is successfully matched once and/or successfully matched twice includes:
updating current detection information of the at least one target to the current presence information, and associating corresponding identification information of the at least one target to the current detection information;
and forming output information of the at least one target according to the current presence information and the corresponding identification information.
In a second aspect, there is provided a multi-target real-time tracking apparatus, the apparatus comprising:
the acquisition module is used for acquiring image information, wherein the image information comprises current frame information and previous frame information of a plurality of targets;
the first matching module is used for carrying out primary matching according to the current frame information and the previous frame information of the targets and judging whether the primary matching of at least one target in the targets is successful or not;
the second matching module is used for carrying out secondary matching if the at least one target is not successfully matched once, and judging whether the at least one target is successfully matched twice or not, wherein the secondary matching comprises at least one of feature matching and distance matching;
the output module is used for forming output information from the information of the at least one target which is successfully matched once and/or successfully matched twice, and the output information comprises current existence information and identification information.
In a third aspect, there is provided an electronic device comprising: the system comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the steps in the multi-target real-time tracking method provided by the embodiment of the invention when executing the computer program.
In a fourth aspect, a computer readable storage medium is provided, on which a computer program is stored, which when executed by a processor implements the steps in the multi-target real-time tracking method provided by the embodiment of the present invention.
The invention has the beneficial effects that: acquiring image information, wherein the image information comprises current frame information and previous frame information of a plurality of targets; according to the current frame information and the previous frame information of the targets, performing primary matching, and judging whether at least one target in the targets is successfully matched once or not; if the at least one target is not successfully matched once, carrying out secondary matching, and judging whether the at least one target is successfully matched twice; and forming output information of the at least one target with successful primary matching and/or successful secondary matching, wherein the output information comprises current existence information and identification information. The tracking accuracy can be increased by matching at least one target twice in one frame of image.
Drawings
FIG. 1 is a schematic flow chart of a multi-target real-time tracking method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of the present information according to an embodiment of the present invention;
FIG. 3 is a diagram of image information according to an embodiment of the present invention;
FIG. 4 is a flowchart of another multi-objective real-time tracking method according to an embodiment of the present invention;
FIG. 5 is a flowchart of another multi-objective real-time tracking method according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a multi-target real-time tracking device according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of another multi-target real-time tracking apparatus according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of another multi-target real-time tracking apparatus according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of another multi-target real-time tracking apparatus according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of another multi-target real-time tracking apparatus according to an embodiment of the present invention;
fig. 11 is a schematic diagram of another multi-target real-time tracking device according to an embodiment of the present invention.
Detailed Description
The following description of the preferred embodiments of the present invention will be provided to enable one of ordinary skill in the art to make and use the related art described below, and to further clarify the innovations and advantages of the present invention.
The invention provides a multi-target real-time tracking method and device and electronic equipment.
The aim of the invention is realized by the following technical scheme:
in a first aspect, referring to fig. 1, fig. 1 is a flow chart of a multi-target real-time tracking method according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
101. image information is acquired, wherein the image information comprises current frame information and last frame information of a plurality of targets.
In this step, the image information may be image information of a video frame acquired by the camera, and the image information may be identified according to time of the video frame, for example, when time of acquiring one frame of image in the video is 15.6789 seconds, the frame of image may be identified as 15S6789. Or the sequence number acquired by the frame in the video total frame, for example, the frame is 14567 th frame in the video total frame, and the frame image can be identified as 14567. Of course, the embodiment of the present invention is not limited to the two identification modes, but may also be other identification modes, such as a time stamp with date, a sequential identification with camera number, and the like.
The current frame information includes information such as feature coordinate values, feature range values, confidence values, and the like of a plurality of targets in the image, and the previous frame information includes information such as identifications, feature coordinate values, feature range values, confidence values, and the like of the plurality of targets, where the feature coordinate values and the feature range values may be measured by pixels or real dimensions, and the embodiment of the present invention is not limited specifically. As shown in fig. 2, the current frame information may be obtained by detecting multiple targets in an original image of the current frame in real time, and if the detected image includes information of the targets, a current feature frame is used to represent the target information, that is, the targets, and obtain central coordinate information, length-width (width-height) information and occurrence confidence of the current feature frame, where the confidence measures the reliability of the existence of the targets, that is, the higher the confidence is, the higher the possibility of existence of the targets is, the more reliable the current feature frame is, and the confidence may be obtained when the image is detected in real time. The feature range value includes an area value occupied by a feature image in a feature frame.
The above information of the previous frame may be information such as an identifier, a feature coordinate value, a feature range value, a confidence value, etc. of the multiple targets in the previous frame, and it should be noted that the identifiers of the multiple targets in the current frame and the multiple targets in the previous frame are different from each other, that is, the identifiers of the multiple targets in the current frame and the multiple targets in the previous frame do not overlap, for example, the identifiers of the multiple targets in the current frame are A, B, C, D, and the identifiers of the multiple targets in the previous frame may be a ', B ', C ', and D ', respectively, where a and a ' may be different targets. As shown in fig. 3, the previous frame of information may be obtained by detecting a plurality of targets in the original image of the previous frame in real time, or may be obtained by detecting a plurality of targets in the processed image of the previous frame in real time, where the target information in the previous frame of information is represented by a previous feature frame different from the feature frame in the current frame of information, for example, the current feature frame is a solid line frame, and then the previous feature frame is a dashed line frame, or may be distinguished by an identifier of the feature frame, for example, the identifier of the target is associated with the feature frame, and two different identifiers are configured for the feature frame in the current frame of information and the feature frame in the previous frame of information.
Specifically, optionally, the image information includes information such as an original image of the current frame, a current feature frame, and a previous feature frame, where the current feature frame includes a current identifier, current center coordinate information, current length-width (width-height) information, and confidence level of occurrence, and the previous feature frame includes a previous identifier, previous center coordinate information, and previous length-width (width-height) information.
It should be noted that, the feature frame may also be referred to as a target frame, the identifier may also be referred to as an ID, the current feature frame may also be referred to as a detection frame, the previous feature frame may also be referred to as a tracking frame, the value of the feature range is an area value occupied by a feature image in the feature frame, the real-time detection may be performed in a tracker or may be obtained by a tracking algorithm, and since the tracker and the tracking algorithm are known to those skilled in the art, and will not be described herein.
102. And according to the current frame information of the targets and the previous frame information, carrying out primary matching, and judging whether at least one target in the targets is successfully matched once.
The current frame information includes a current feature frame of the plurality of targets or a current feature vector of the plurality of targets, the previous frame information includes a previous feature frame of the plurality of targets or a previous feature vector of the plurality of targets, and at least one target of the plurality of targets is matched by using a deep learning target detection algorithm.
In the embodiment of the present invention, the current frame information in step 102 includes a plurality of current feature frames corresponding to a plurality of targets, the previous frame information includes a plurality of previous feature frames corresponding to a plurality of targets, information obtained by real-time detection is output to obtain a set of image information including a plurality of current feature frames and a plurality of previous feature frames, each previous feature frame is placed in the plurality of current feature frames to be matched, an overlapping area of each previous feature frame and the plurality of current feature frames is calculated, an overlapping degree (interaction-over-Union, ioU, also referred to as an Intersection ratio) of each previous feature frame and the plurality of current feature frames is calculated according to the overlapping area, and a current feature frame with a maximum overlapping degree of the previous feature frame is selected to be a set, as shown in fig. 3. And comparing the maximum overlapping degree of the last feature frame with a preset overlapping degree threshold value, wherein the case that the maximum overlapping degree meets the overlapping degree threshold value is marked as successful in one-time matching, the case that the maximum overlapping degree does not meet the overlapping degree threshold value is marked as unsuccessful in one-time matching, and the last feature frame enters step 103 to perform secondary matching.
In some possible embodiments, the similarity of the feature frames may be compared, and the current feature frame corresponding to the feature frame of the previous frame may be matched. The similarity includes: area similarity, length-width (width-height) similarity, and the like.
The first matching may be referred to as first matching, first tracking, and the like, and may be referred to as tracking.
103. If the at least one target is not successfully matched once, carrying out secondary matching, and judging whether the at least one target is successfully matched twice or not, wherein the secondary matching comprises at least one of feature matching and distance matching.
In the step, the target which is not successfully matched in the step 102 is subjected to secondary matching, wherein the secondary matching comprises at least one of feature matching and distance matching. Feature matching includes obtaining a current feature vector of a current feature frame, and obtaining a previous feature vector of a previous feature frame, and calculating a similarity between the previous feature vector and the current feature vector. Distance matching includes obtaining a distance value in a previous feature frame and a current feature frame. And selecting a previous feature frame with the similarity larger than a preset similarity threshold value to perform secondary matching with the current feature frame, or selecting a previous feature frame with the distance value smaller than the preset distance threshold value to perform secondary matching with the current feature frame, or selecting a previous feature frame with the similarity larger than the preset similarity threshold value and the distance value smaller than the preset distance threshold value to perform secondary matching with the current feature frame.
The above-mentioned secondary matching may also be referred to as first matching, re-matching, secondary tracking, second tracking, re-tracking, and the like.
104. And forming the information of the at least one target which is successfully matched once and/or successfully matched twice into output information, wherein the output information comprises current existence information and identification information.
The above current presence information includes a current feature frame, for example, if the matching is successful between the current feature frame a and the previous feature frame a', the information of the current feature frame a is output, where the information of the current feature frame includes: and the identification information comprises identification information of the current feature frame and is used for representing the current feature frame, for example, if the current feature frame is A, the A is output, and the identification information of the current feature frame is associated with the current feature frame.
In step 102, successfully matched targets may be placed in an active set, and unsuccessfully matched target information may be placed in a lost set. In step 103, after the targets in the lost set are subjected to secondary matching, successfully matched targets are obtained, and the successfully matched target information can be added into the active set to output the information of the targets in the active set.
Updating the successfully matched target to obtain current existence information and identification information, updating the identification information of the last feature frame to the current feature frame for the successfully matched current feature frame and the last feature frame, enabling the identification of the current feature frame to be unified with the identification of the last feature frame and used for representing the same target, namely, the successfully matched target is successfully tracked, and meanwhile deleting the successfully matched last feature frame, so that only the information of the current feature frame exists in the active set, and the current existence information and the identification information of the target are formed. For example: a is the identification of the current feature frame, A ' is the identification of the last feature frame, A and A ' are a pair of the current feature frame and the last feature frame which are successfully matched, the identification of the current feature frame is changed from A to A ', then the last feature frame is deleted from image information, the current feature frame A ' is recorded into an active set, and the output information is the identification A ' of the current feature frame, the central coordinate information, the length-width (width-height) information and the like of the current feature frame.
In the embodiment of the invention, image information is acquired, wherein the image information comprises current frame information and last frame information of a plurality of targets; according to the current frame information and the previous frame information of the targets, performing primary matching, and judging whether at least one target in the targets is successfully matched once or not; if the at least one target is not successfully matched once, carrying out secondary matching, and judging whether the at least one target is successfully matched twice; and forming output information of the at least one target with successful primary matching and/or successful secondary matching, wherein the output information comprises current existence information and identification information. Processing at least one object simultaneously in one frame of image can increase the tracking efficiency.
It should be noted that, the method for installing a container orchestration engine according to the embodiment of the present invention may be applied to an installation device of a container orchestration engine, for example: a computer, server, cell phone, etc. can perform the container orchestration engine installation.
Referring to fig. 4, fig. 4 is a flow chart of another multi-objective real-time tracking method according to an embodiment of the present invention, as shown in fig. 4, the method includes the following steps:
201. acquiring image information, wherein the image information comprises current frame information and previous frame information of a plurality of targets;
202. according to the current frame information and the previous frame information of the targets, performing primary matching, and judging whether at least one target in the targets is successfully matched once or not;
203. if the at least one target is not successfully matched once, carrying out secondary matching, and judging whether the at least one target is successfully matched twice or not, wherein the secondary matching comprises at least one of feature matching and distance matching;
204. forming output information from the information of the at least one target which is successfully matched once and/or successfully matched twice, wherein the output information comprises current existence information and identification information;
205. if the secondary matching is unsuccessful, regenerating the current frame information of the residual target, and acquiring new image information, wherein the new image information comprises the current frame information and the next frame information.
In step 202, successfully matched targets may be placed in an active set, and unsuccessfully matched target information may be placed in a lost set. After performing secondary matching on the targets in the lost set in step 203, a target with successful matching is obtained, target information with successful matching can be added into the active set, target information in the active set is output, and if the matching is unsuccessful, step 205 is performed.
In step 205, for the current feature frame and the previous feature frame for which the secondary matching has not been successful, the feature frame of the target is regenerated, the regenerated feature frame is recorded in the active set, and identification information corresponding to the active set is regenerated for the feature frames. For example: assuming that two elements of a current feature frame A 'and a current feature frame D' exist in an active set, B is a current feature frame which is not successfully matched, B 'is a last feature frame which is successfully matched, C is a current feature frame which is successfully matched with B', C 'is a last feature frame which is not successfully matched, the identification C of the current feature frame which is successfully matched is changed into an identification B' and is recorded into the active set to obtain a current feature frame B ', three elements of the current feature frame A', the current feature frame B 'and the current feature frame D' exist in the active set at the moment, the last feature frame B 'is deleted, the last feature frame C' is deleted for the current feature frame B which is not successfully matched with the last feature frame C ', the identification B of the current feature frame is rebuilt into E', and four elements of the current feature frame A ', the current feature frame B', the current feature frame D 'and the current feature frame E' exist in the active set at the moment, and the current frame information is obtained.
Acquiring new image information comprises acquiring new original image information, detecting the new original image in real time to obtain next frame information, adding the current feature frame in the active set into the new image information, and performing new tracking process circularly to obtain all tracking results, as shown in fig. 5. In addition, steps 201 to 205 may be performed in a loop, and multiple targets may be tracked.
It should be noted that, step 205 is optional, and in some embodiments, it is only necessary to form an output information output for the information of the at least one target that the primary matching is successful and/or the secondary matching is successful. Optionally, the current frame information includes current detection information of the multiple targets, and the previous frame information includes historical existence information and corresponding identification information of the multiple targets;
the step of performing primary matching according to the current frame information and the previous frame information of the plurality of targets, and the step of judging whether at least one target of the plurality of targets is successfully matched once comprises the following steps:
performing overlapping degree calculation on the current detection information of at least one target of the plurality of targets and the historical presence information of at least one target of the plurality of targets to obtain overlapping degree of the current detection information of the at least one target and the historical presence information of the at least one target;
and judging whether the at least one target is successfully matched once according to the overlapping degree.
The current detection information comprises current feature frame information obtained by detecting the current original image in real time and generated identifiers of all current feature frames, the current feature frame information comprises information such as center coordinate information, length and width (width and height) information, confidence and the like, and the identifiers of the current feature frames can be unique identifiers such as unique numerical identifiers and unique letter identifiers. The history presence information may be feature frame information existing in the previous frame image, and the corresponding identification information may be a unique identification of a feature frame existing in the previous frame image, or may be said to be a unique identification of the previous feature frame. The overlapping degree may be the overlapping degree of the current feature frame and the previous feature frame, including the coordinate overlapping degree of length and width (width and height), the area overlapping degree, and the like.
In some possible embodiments, the overlapping degree may also be the similarity of feature vectors or the similarity of feature frames.
When the overlapping degree or the similarity is larger than a preset threshold value, the at least one target can be judged to be successfully matched once, and when the overlapping degree or the similarity is smaller than the preset threshold value, the at least one target can be judged to be unsuccessfully matched once.
Optionally, the determining whether the at least one target matches successfully according to the overlapping degree includes:
selecting the maximum overlapping degree and comparing with a preset overlapping degree threshold value, and judging whether the maximum overlapping degree is larger than the overlapping degree threshold value or not;
if the maximum overlapping degree is larger than the overlapping degree threshold value, the primary matching is successful, and if the maximum overlapping degree is smaller than the overlapping degree threshold value, the primary matching is unsuccessful.
For each previous feature frame, placing the previous feature frame in a plurality of current feature frames for matching, calculating the overlapping area of each previous feature frame and the plurality of current feature frames, calculating the overlapping degree (also called cross-over-Union, ioU) of each previous feature frame and the plurality of current feature frames according to the overlapping area, selecting a current feature frame with the maximum overlapping degree of the previous feature frame as a group, and comparing the maximum overlapping degree of the previous feature frame with a preset overlapping degree threshold value, for example: a 'is the identification of the last feature frame, B' is the identification of the last feature frame, A is the identification of the current feature frame, B is the identification of the current feature frame, the overlapping degree of A 'and A is 0.4, the overlapping degree of A' and B is 0.8, A 'and B have the maximum overlapping degree, and are marked as a group, the overlapping degree of B' and A is 0.4, B 'and B have the maximum overlapping degree, and are marked as a group, the overlapping degree of A' and A is 0.4, the maximum overlapping degree meets the overlapping degree threshold value, and is marked as primary matching success, the overlapping degree threshold value is 0.6, the overlapping degree of B 'and B is less than 0.6, and the matching of B' and B is unsuccessful, if the maximum overlapping degree does not meet the overlapping degree threshold value, the primary matching is marked as unsuccessful, and the last feature frame enters step 203 to be matched secondarily.
Optionally, the current frame information includes current detection information of the multiple targets, and the previous frame information includes historical existence information and corresponding identification information of the multiple targets;
the performing secondary matching, and determining whether the at least one target is successfully matched secondarily includes:
extracting a current feature vector of current detection information of at least one of the plurality of targets, extracting a historical feature vector of historical presence information of at least one of the plurality of targets, and calculating the current feature vector and the historical feature vector to obtain cosine similarity of the at least one target;
extracting current coordinates of current detection information of the at least one target and historical coordinates of historical presence information of the at least one target, and calculating the current coordinates and the historical coordinates of the at least one target to obtain a distance value of the at least one target;
and judging whether the at least one target is successfully matched secondarily according to the cosine similarity and the distance value of the at least one target.
The current detection information comprises current feature frame information and a mark, and the historical existence information comprises the previous feature frame information and the mark. The current feature vector can obtain the current HOG feature vector by extracting a direction gradient histogram (Histogram of Oriented Gradient, abbreviated as HOG) of the current feature frame, and likewise, the last feature vector can obtain the last HOG feature vector by extracting a direction gradient histogram of the last feature frame, and the cosine similarity between the current HOG feature vector and the last HOG feature vector is obtained through calculation.
The current feature frame comprises current center coordinate information and current length and width (width and height) information, the previous feature frame comprises previous center coordinate information and previous length and width (width and height) information, and the distance value between the current feature frame and the previous feature frame can be calculated through the following formula:
D＝sqrt((x1-x2) 2 +(y1-y2) 2 )/min(w1,w2)
wherein D is a distance value between the current feature frame and the previous feature frame, x1, y1, w1 belong to the current feature frame, x1, y1 are center coordinates of the current feature frame, w1 is a width of the current feature frame, x2, y2, w2 belong to the previous feature frame, x2, y2 are center coordinates of the previous feature frame, and w2 is a width of the previous feature frame.
Optionally, the determining whether the at least one target is successfully matched twice according to the cosine similarity and the distance value of the at least one target includes:
comparing the cosine similarity with a preset cosine similarity threshold value, and comparing the distance value with a preset distance threshold value to obtain a comparison result;
and judging whether the secondary matching of the at least one target is successful or not according to the comparison result and a preset judging rule.
The judgment rule comprises: when the cosine similarity is larger than a preset cosine similarity threshold, the secondary matching can be considered to be successful. When the distance value is smaller than the set distance threshold value, the secondary matching may be considered to be successful. Of course, when the cosine similarity is greater than the preset cosine similarity threshold and the distance value is smaller than the preset distance threshold, the secondary matching can be considered to be successful.
Optionally, the forming output information of the at least one target that is successfully matched once and/or successfully matched twice includes:
updating current detection information of the at least one target to the current presence information, and associating corresponding identification information of the at least one target to the current detection information;
and forming output information of the at least one target according to the current presence information and the corresponding identification information.
The above current presence information includes a current feature frame, for example, if the matching is successful between the current feature frame a and the previous feature frame a', the information of the current feature frame a is output, where the information of the current feature frame includes: and the identification information comprises identification information of the current feature frame, and is used for representing the current feature frame, for example, if the current feature frame is A, the A is output, and if the current feature frame is A ', the A' is output, and the identification information of the current feature frame is associated with the current feature frame.
In step 202, successfully matched targets may be placed in an active set, and unsuccessfully matched target information may be placed in a lost set. In step 203, after performing secondary matching on the targets in the lost set, a target with a successful matching is obtained, and the target information with a successful matching can be added into the active set, and the information of the targets in the active set is output.
Updating the successfully matched target to obtain current existence information and identification information, updating the identification information of the last feature frame to the current feature frame for the successfully matched current feature frame and the last feature frame, enabling the identification of the current feature frame to be unified with the identification of the last feature frame and used for representing the same target, namely, the successfully matched target is successfully tracked, and meanwhile deleting the successfully matched last feature frame, so that only the information of the current feature frame exists in the active set, and the current existence information and the identification information of the target are formed. For example: a is the identification of the current feature frame, A ' is the identification of the last feature frame, A and A ' are a pair of the current feature frame and the last feature frame which are successfully matched, the identification of the current feature frame is changed from A to A ', then the last feature frame is deleted from image information, the current feature frame A ' is recorded into an active set, and the output information is the identification A ' of the current feature frame, the central coordinate information, the length-width (width-height) information and the like of the current feature frame.
In a second aspect, as shown in fig. 6, there is provided a multi-target real-time tracking apparatus, the apparatus comprising:
an obtaining module 401, configured to obtain image information, where the image information includes current frame information and previous frame information of a plurality of targets;
a first matching module 402, configured to perform a primary matching according to current frame information of the multiple targets and previous frame information, and determine whether at least one target of the multiple targets is successfully matched once;
a second matching module 403, configured to perform a second matching if the at least one target is not successfully matched, and determine whether the at least one target is successfully matched, where the second matching includes at least one of feature matching and distance matching;
and an output module 404, configured to form information of the at least one target that is successfully matched once and/or successfully matched twice into output information, where the output information includes current presence information and identification information.
Optionally, as shown in fig. 7, after the performing the secondary matching if the at least one target is not successfully matched, determining whether the at least one target is successfully matched, the apparatus further includes:
and the generating module 405 is configured to regenerate current frame information of the remaining targets if the second matching is unsuccessful, and acquire new image information, where the new image information includes the current frame information and the next frame information.
Optionally, as shown in fig. 8, the current frame information includes current detection information of the multiple targets, and the previous frame information includes historical presence information and corresponding identification information of the multiple targets;
the first matching module 402 includes:
the first processing unit 4021 is configured to perform overlapping degree calculation on current detection information of at least one of the plurality of targets and historical presence information of at least one of the plurality of targets, so as to obtain overlapping degree of the current detection information of the at least one target and the historical presence information of the at least one target;
the first judging unit 4022 is configured to judge whether the at least one target is successfully matched once according to the overlapping degree.
Alternatively, as shown in fig. 9, the first determining unit 4022 includes:
a comparing subunit 40221, configured to select a maximum overlapping degree and compare with a preset overlapping degree threshold, and determine whether the maximum overlapping degree is greater than the overlapping degree threshold;
the judging subunit 40222 is configured to succeed in one-time matching if the maximum overlapping degree is greater than the overlapping degree threshold, and unsuccessful in one-time matching if the maximum overlapping degree is less than the overlapping degree threshold.
Optionally, as shown in fig. 10, the current frame information includes current detection information of the multiple targets, and the previous frame information includes historical presence information and corresponding identification information of the multiple targets;
the second matching module 403 includes:
a second processing unit 4031, configured to extract a current feature vector of current detection information of at least one of the plurality of targets, extract a historical feature vector of historical presence information of at least one of the plurality of targets, and calculate the current feature vector and the historical feature vector to obtain cosine similarity of the at least one target;
a third processing unit 4032, configured to extract a current coordinate of the current detection information of the at least one target and a historical coordinate of the historical presence information of the at least one target, and calculate the current coordinate and the historical coordinate of the at least one target to obtain a distance value of the at least one target;
the second judging unit 4033 is configured to judge whether the at least one target is successfully matched twice according to the cosine similarity and the distance value of the at least one target.
Alternatively, as shown in fig. 11, the output module includes:
an updating unit 4041, configured to update current detection information of the at least one target to the current presence information, and associate corresponding identification information of the at least one target to the current detection information;
and an output unit 4042 for forming output information of the at least one target according to the current presence information and the corresponding identification information.
In a third aspect, an embodiment of the present invention provides an electronic device, including: the system comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the steps in the multi-target real-time tracking method provided by the embodiment of the invention when executing the computer program.
In a fourth aspect, embodiments of the present invention provide a computer readable storage medium having a computer program stored thereon, which when executed by a processor implements the steps in the multi-target real-time tracking method provided by the embodiments of the present invention.
The foregoing is a further detailed description of the invention in connection with specific preferred embodiments, and it is not intended that the invention be limited to these descriptions. It will be apparent to those skilled in the art that several simple deductions or substitutions may be made without departing from the spirit of the invention, and these should be considered to be within the scope of the invention.
Claims (10)
1. A multi-target real-time tracking method, the method comprising:
acquiring image information, wherein the image information comprises current frame information and previous frame information of a plurality of targets;
according to the current frame information and the previous frame information of the targets, performing primary matching, and judging whether at least one target in the targets is successfully matched once or not;
if the at least one target is not successfully matched once, carrying out secondary matching, and judging whether the at least one target is successfully matched twice or not, wherein the secondary matching comprises at least one of feature matching and distance matching;
and forming the information of the at least one target which is successfully matched once and/or successfully matched twice into output information, wherein the output information comprises current existence information and identification information.
2. The method of claim 1, wherein after performing the second match if the at least one target does not succeed in the first match, determining whether the at least one target succeeds in the second match further comprises:
if the secondary matching is unsuccessful, regenerating the current frame information of the residual target, and acquiring new image information, wherein the new image information comprises the current frame information and the next frame information.
3. The method of claim 2, wherein the current frame information includes current detection information of the plurality of targets, and the previous frame information includes historical presence information and corresponding identification information of the plurality of targets;
the step of performing primary matching according to the current frame information and the previous frame information of the plurality of targets, and the step of judging whether at least one target of the plurality of targets is successfully matched once comprises the following steps:
performing overlapping degree calculation on the current detection information of at least one target of the plurality of targets and the historical presence information of at least one target of the plurality of targets to obtain overlapping degree of the current detection information of the at least one target and the historical presence information of the at least one target;
and judging whether the at least one target is successfully matched once according to the overlapping degree.
4. The method of claim 3, wherein said determining whether the at least one target match was successful based on the degree of overlap comprises:
selecting the maximum overlapping degree and comparing with a preset overlapping degree threshold value, and judging whether the maximum overlapping degree is larger than the overlapping degree threshold value or not;
if the maximum overlapping degree is larger than the overlapping degree threshold value, the primary matching is successful, and if the maximum overlapping degree is smaller than the overlapping degree threshold value, the primary matching is unsuccessful.
5. The method of claim 2, wherein the current frame information includes current detection information of the plurality of targets, and the previous frame information includes historical presence information and corresponding identification information of the plurality of targets;
the performing secondary matching, and determining whether the at least one target is successfully matched secondarily includes:
extracting a current feature vector of current detection information of at least one of the plurality of targets, extracting a historical feature vector of historical presence information of at least one of the plurality of targets, and calculating the current feature vector and the historical feature vector to obtain cosine similarity of the at least one target;
extracting current coordinates of current detection information of the at least one target and historical coordinates of historical presence information of the at least one target, and calculating the current coordinates and the historical coordinates of the at least one target to obtain a distance value of the at least one target;
and judging whether the at least one target is successfully matched secondarily according to the cosine similarity and the distance value of the at least one target.
6. The method of claim 5, wherein determining whether the at least one object successfully matches twice based on the cosine similarity and distance values of the at least one object comprises:
comparing the cosine similarity with a preset cosine similarity threshold value, and comparing the distance value with a preset distance threshold value to obtain a comparison result;
and judging whether the secondary matching of the at least one target is successful or not according to the comparison result and a preset judging rule.
7. The method of claim 5, wherein the forming output information of the at least one target that successfully matches the primary and/or the secondary comprises:
updating current detection information of the at least one target to the current presence information, and associating corresponding identification information of the at least one target to the current detection information;
and forming output information of the at least one target according to the current presence information and the corresponding identification information.
8. A multi-target real-time tracking device, the device comprising:
the acquisition module is used for acquiring image information, wherein the image information comprises current frame information and previous frame information of a plurality of targets;
the first matching module is used for carrying out primary matching according to the current frame information and the previous frame information of the targets and judging whether the primary matching of at least one target in the targets is successful or not;
the second matching module is used for carrying out secondary matching if the at least one target is not successfully matched once, and judging whether the at least one target is successfully matched twice or not, wherein the secondary matching comprises at least one of feature matching and distance matching;
the output module is used for forming output information from the information of the at least one target which is successfully matched once and/or successfully matched twice, and the output information comprises current existence information and identification information.
9. An electronic device, comprising: memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps in the multi-target real time tracking method according to any one of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the multi-target real time tracking method according to any of claims 1 to 7.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2018/111589 WO2020082258A1 (en) | 2018-10-24 | 2018-10-24 | Multi-objective real-time tracking method and apparatus, and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111512317A CN111512317A (en) | 2020-08-07 |
| CN111512317B true CN111512317B (en) | 2023-06-06 |
Family
ID=70330247
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201880083620.2A Active CN111512317B (en) | 2018-10-24 | 2018-10-24 | Multi-target real-time tracking method and device and electronic equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111512317B (en) |
| WO (1) | WO2020082258A1 (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111914653B (en) * | 2020-07-02 | 2023-11-07 | 泰康保险集团股份有限公司 | Personnel marking method and device |
| CN112037256A (en) * | 2020-08-17 | 2020-12-04 | 中电科新型智慧城市研究院有限公司 | Target tracking method and device, terminal equipment and computer readable storage medium |
| CN112084914B (en) * | 2020-08-31 | 2024-04-26 | 的卢技术有限公司 | Multi-target tracking method integrating space motion and apparent feature learning |
| CN112070802B (en) * | 2020-09-02 | 2024-01-26 | 合肥英睿系统技术有限公司 | Target tracking method, device, equipment and computer readable storage medium |
| CN114185034A (en) * | 2020-09-15 | 2022-03-15 | 郑州宇通客车股份有限公司 | Target tracking method and system for millimeter wave radar |
| CN112101223B (en) * | 2020-09-16 | 2024-04-12 | 阿波罗智联(北京)科技有限公司 | Detection method, detection device, detection equipment and computer storage medium |
| CN112634327A (en) * | 2020-12-21 | 2021-04-09 | 合肥讯图信息科技有限公司 | Tracking method based on YOLOv4 model |
| CN113238209B (en) * | 2021-04-06 | 2024-01-16 | 宁波吉利汽车研究开发有限公司 | Road perception method, system, equipment and storage medium based on millimeter wave radar |
| CN113361456B (en) * | 2021-06-28 | 2024-05-07 | 北京影谱科技股份有限公司 | Face recognition method and system |
| CN113723311A (en) * | 2021-08-31 | 2021-11-30 | 浙江大华技术股份有限公司 | Target tracking method |
| CN114155275A (en) * | 2021-11-17 | 2022-03-08 | 深圳职业技术学院 | IOU-Tracker-based fish tracking method and device |
| CN115223135B (en) * | 2022-04-12 | 2023-11-21 | 广州汽车集团股份有限公司 | Parking space tracking method and device, vehicle and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108664930A (en) * | 2018-05-11 | 2018-10-16 | 西安天和防务技术股份有限公司 | A kind of intelligent multi-target detection tracking |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104424638A (en) * | 2013-08-27 | 2015-03-18 | 深圳市安芯数字发展有限公司 | Target tracking method based on shielding situation |
| CN104517275A (en) * | 2013-09-27 | 2015-04-15 | 株式会社理光 | Object detection method and system |
| CN106033613B (en) * | 2015-03-16 | 2019-04-30 | 北京大学 | Method for tracking target and device |
| CN104765886A (en) * | 2015-04-29 | 2015-07-08 | 百度在线网络技术（北京）有限公司 | Information acquisition method and device based on images |
| CN104778465B (en) * | 2015-05-06 | 2018-05-15 | 北京航空航天大学 | A kind of matched method for tracking target of distinguished point based |
| CN106097391B (en) * | 2016-06-13 | 2018-11-16 | 浙江工商大学 | A kind of multi-object tracking method of the identification auxiliary based on deep neural network |
| CN106203274B (en) * | 2016-06-29 | 2020-03-31 | 长沙慧联智能科技有限公司 | Real-time pedestrian detection system and method in video monitoring |
| CN107316322A (en) * | 2017-06-27 | 2017-11-03 | 上海智臻智能网络科技股份有限公司 | Video tracing method and device and object identifying method and device |
| CN108154118B (en) * | 2017-12-25 | 2018-12-18 | 北京航空航天大学 | A kind of target detection system and method based on adaptive combined filter and multistage detection |
-
2018
- 2018-10-24 CN CN201880083620.2A patent/CN111512317B/en active Active
- 2018-10-24 WO PCT/CN2018/111589 patent/WO2020082258A1/en active Application Filing
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108664930A (en) * | 2018-05-11 | 2018-10-16 | 西安天和防务技术股份有限公司 | A kind of intelligent multi-target detection tracking |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111512317A (en) | 2020-08-07 |
| WO2020082258A1 (en) | 2020-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111512317B (en) | Multi-target real-time tracking method and device and electronic equipment | |
| CN108960211B (en) | Multi-target human body posture detection method and system | |
| CN108304882B (en) | Image classification method and device, server, user terminal and storage medium | |
| CN110610166B (en) | Text region detection model training method and device, electronic equipment and storage medium | |
| CN109325964B (en) | Face tracking method and device and terminal | |
| US11205276B2 (en) | Object tracking method, object tracking device, electronic device and storage medium | |
| CN105164700B (en) | Detecting objects in visual data using a probabilistic model | |
| CN106971401B (en) | Multi-target tracking device and method | |
| US20120106784A1 (en) | Apparatus and method for tracking object in image processing system | |
| CN109919002B (en) | Yellow stop line identification method and device, computer equipment and storage medium | |
| CN111275040B (en) | Positioning method and device, electronic equipment and computer readable storage medium | |
| CN112597837A (en) | Image detection method, apparatus, device, storage medium and computer program product | |
| CN111126209B (en) | Lane line detection method and related equipment | |
| US20220207741A1 (en) | Methods, devices, apparatuses and storage media of detecting correlated objects invovled in images | |
| CN112150514A (en) | Pedestrian trajectory tracking method, device and equipment of video and storage medium | |
| CN110532895B (en) | Method, device and equipment for detecting fraudulent behavior in face recognition process | |
| CN113920158A (en) | Training and traffic object tracking method and device of tracking model | |
| CN113763466B (en) | Loop detection method and device, electronic equipment and storage medium | |
| CN111695404A (en) | Pedestrian falling detection method and device, electronic equipment and storage medium | |
| CN113744236B (en) | Loop detection method, device, storage medium and computer program product | |
| CN115797291A (en) | Circuit terminal identification method and device, computer equipment and storage medium | |
| CN115019152A (en) | Image shooting integrity judgment method and device | |
| CN114972465A (en) | Image target depth detection method and device, electronic equipment and storage medium | |
| CN111428565B (en) | Point cloud identification point positioning method and device based on deep learning | |
| CN110414845B (en) | Risk assessment method and device for target transaction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |