WO2020082258A1

WO2020082258A1 - Multi-objective real-time tracking method and apparatus, and electronic device

Info

Publication number: WO2020082258A1
Application number: PCT/CN2018/111589
Authority: WO
Inventors: 肖梦秋
Original assignee: 深圳鲲云信息科技有限公司
Priority date: 2018-10-24
Filing date: 2018-10-24
Publication date: 2020-04-30
Also published as: CN111512317B; CN111512317A

Abstract

Provided are a multi-objective real-time tracking method and apparatus, and an electronic device. The method comprises: acquiring image information (101), wherein the image information comprises information of the current frame and information of the previous frame of a plurality of objectives; matching the information of the current frame with the information of the previous frame of the plurality of objectives once, and determining whether at least one objective in the plurality of objectives is successfully matched a first time (102); if the at least one objective is not successfully matched a first time, then performing a second match, and determining whether the at least one objective is successfully matched a second time (103); and the information of the at least one objective that is successfully matched a first time and/or that is successfully matched a second time being formed as output information (104), wherein the output information comprises the current existence information and identification information. Performing matching on at least one objective twice in one frame image can increase the accuracy rate of tracking.

Description

Multi-target real-time tracking method, device and electronic equipment

Technical field

The invention relates to the field of software development, and more specifically, to a multi-target real-time tracking method, device and electronic equipment.

Background technique

Tracking can determine the trajectory of the target (object or person). Current single target tracking algorithms, such as based on correlation filtering (KCF), can perform real-time single target tracking on low-power devices. However, in the terminal device, due to the limitation of power consumption, the current tracking algorithm has a real-time problem for multi-target tracking. Especially when the total number of targets is greater than 10, the combination of multiple single target tracking algorithms requires a large amount of calculation and a high data processing delay, so that the tracking accuracy obtained using this tracking method is low.

Summary of the invention

The purpose of the present invention is to provide a multi-target real-time tracking method, device and electronic equipment in view of the above-mentioned defects in the prior art, which solves the problem of low tracking accuracy.

The purpose of the present invention is achieved by the following technical solutions:

In a first aspect, a multi-target real-time tracking method is provided. The method includes:

Obtain image information, where the image information includes current frame information and previous frame information of multiple targets;

Perform a match based on the current frame information of the multiple targets and the previous frame information, and determine whether at least one of the multiple targets is successfully matched at a time;

If the at least one target is not matched successfully once, then a second match is performed to determine whether the at least one target is successfully matched twice, and the second match includes at least one of feature matching and distance matching;

The information of the at least one target with a successful match and / or a successful second match is formed as output information, and the output information includes current presence information and identification information.

Optionally, after the at least one target is not successfully matched once, performing a second match to determine whether the at least one target is successfully matched a second time, further includes:

If the second match is unsuccessful, the current frame information of the remaining target is regenerated to obtain new image information, and the new image information includes the current frame information and the next frame information.

Optionally, the current frame includes current detection information of the multiple targets, and the previous frame information includes historical existence information and corresponding identification information of the multiple targets;

The matching the current frame information of the multiple targets with the previous frame information once to determine whether at least one of the multiple targets is successfully matched at a time includes:

Calculating the degree of overlap between the current detection information of at least one target of the plurality of targets and the historical presence information of at least one target of the plurality of targets to obtain the current detection information of the at least one target and the information of the at least one target The degree of overlap of information in history;

According to the degree of overlap, it is determined whether the at least one target is successfully matched at a time.

Optionally, the judging whether the at least one target is successfully matched according to the overlapping degree includes:

Selecting the maximum overlap degree to compare with the preset overlap degree threshold to determine whether the maximum overlap degree is greater than the overlap degree threshold;

If the maximum overlap degree is greater than the overlap degree threshold, a match is successful, and if the maximum overlap degree is less than the overlap degree threshold, a match is unsuccessful.

The performing second matching to determine whether the at least one target is successfully matched twice includes:

Extracting a current feature vector of current detection information of at least one target of the plurality of targets, extracting a historical feature vector of historical presence information of at least one target of the multiple targets, and combining the current feature vector with the historical feature vector Performing calculation to obtain the cosine similarity of the at least one target;

Extracting the current coordinates of the current detection information of the at least one target and the historical coordinates of the historical presence information of the at least one target, calculating the current coordinates of the at least one target and the historical coordinates to obtain the distance of the at least one target value;

According to the cosine similarity and distance values of the at least one target, it is determined whether the at least one target is successfully matched twice.

Optionally, the judging whether the at least one target is successfully matched twice according to the characteristic value and the distance value of the at least one target includes:

Comparing the characteristic value cosine similarity with a preset cosine similarity threshold, and comparing the distance value with a preset distance threshold to obtain a comparison result;

According to the comparison result, it is determined whether the at least one target is successfully matched twice according to a preset judgment rule.

Optionally, the output information of the formation of the at least one target with a successful first match and / or a successful second match includes:

Updating the current detection information of the at least one target to the current presence information, and associating the corresponding identification information of the at least one target to the current detection information;

The output information of the at least one target is formed according to the current presence information and the corresponding identification information.

In a second aspect, a multi-target real-time tracking device is provided. The device includes:

An acquisition module, for acquiring image information, the image information including current frame information and previous frame information of multiple targets;

The first matching module is configured to perform a match based on the current frame information of the multiple targets and the previous frame information, and determine whether at least one of the multiple targets is successfully matched at one time;

The second matching module is used to perform secondary matching if the at least one target has not been matched once, and determine whether the at least one target has been successfully matched twice. The secondary matching includes at least one of feature matching and distance matching. item;

The output module is configured to form information of the at least one target with a successful match and / or a successful second match into output information, where the output information includes current presence information and identification information.

In a third aspect, an electronic device is provided, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the implementation of the present invention is implemented Example provides the steps in the multi-target real-time tracking method.

According to a fourth aspect, a computer-readable storage medium is provided, and a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps in the multi-target real-time tracking method provided by the embodiments of the present invention are implemented .

Beneficial effects brought by the present invention: acquiring image information, the image information includes current frame information and previous frame information of multiple targets; according to the current frame information of the multiple targets and the previous frame information, a match is made to determine Whether at least one target among the plurality of targets is matched successfully once; if the at least one target is not matched successfully once, then a second match is performed to determine whether the at least one target is matched twice successfully; Or the output information of the formation of the at least one target for which the second match is successful, the output information includes current presence information and identification information. By matching at least one target twice in a frame of image, the tracking accuracy can be increased.

BRIEF DESCRIPTION

FIG. 1 is a schematic flowchart of a multi-target real-time tracking method according to an embodiment of the present invention;

2 is a schematic diagram of current information according to an embodiment of the present invention;

3 is a schematic diagram of image information according to an embodiment of the present invention;

4 is a schematic flowchart of another multi-target real-time tracking method provided by an embodiment of the present invention;

FIG. 5 is a schematic flowchart of another multi-target real-time tracking method according to an embodiment of the present invention;

6 is a schematic diagram of a multi-target real-time tracking device provided by an embodiment of the present invention;

7 is a schematic diagram of another multi-target real-time tracking device provided by an embodiment of the present invention;

8 is a schematic diagram of another multi-target real-time tracking device provided by an embodiment of the present invention;

9 is a schematic diagram of another multi-target real-time tracking device provided by an embodiment of the present invention;

10 is a schematic diagram of another multi-target real-time tracking device provided by an embodiment of the present invention;

FIG. 11 is a schematic diagram of another multi-target real-time tracking device provided by an embodiment of the present invention.

detailed description

The following describes the preferred embodiments of the present invention. Those of ordinary skill in the art will be able to implement related technologies in the art according to the description below, and will be able to more clearly understand the innovations and benefits of the present invention.

The invention provides a multi-target real-time tracking method, device and electronic equipment.

In the first aspect, please refer to FIG. 1. FIG. 1 is a schematic flowchart of a multi-target real-time tracking method according to an embodiment of the present invention. As shown in FIG. 1, the method includes the following steps:

101. Acquire image information, where the image information includes current frame information and previous frame information of multiple targets.

In this step, the above image information may be the image information of the video frame collected by the camera, and the image information may be identified according to the time of the video frame. For example, the time for acquiring one frame of image in the video is 15.6789 seconds. Identify the frame image as 15S6789. It may also be the sequence number obtained from the frame in the total video frame. For example, if the frame is the 14567th frame in the total video frame, the frame image may be identified as 14567. Of course, the embodiments of the present invention are not limited to the above two identification methods, and may also be other identification methods, such as a time stamp with a date and a sequential identification with a camera number.

The above current frame information includes the feature coordinate values, feature range values, and confidence values of multiple targets in the image. The previous frame information includes the identification, feature coordinate values, feature range values, and confidence levels of multiple targets. The feature coordinate value and the feature range value may be measured in pixels or measured in actual size, which is not specifically limited in the embodiment of the present invention. As shown in Figure 2, the current frame information can be obtained by real-time detection of multiple targets in the original image of the current frame. If the image contains target information, a current feature box is used to represent the target information. It is the target. Obtain the center coordinate information, length and width (width and height) information of the current feature box, and the confidence level that appears. The confidence level measures the credibility of the existence of the target. The higher it is, the more credible the current feature box is, and the confidence can be obtained when the image is detected in real time. The above feature range value includes the area value occupied by the feature image in the feature frame.

The above information of the previous frame may be the identification, feature coordinate value, feature range value, confidence value, etc. of multiple targets in the image in the previous frame. It should be noted that the identification of multiple targets in the current frame is the same as the previous The marks of multiple targets in the frame are different from each other. It can also be said that the marks of multiple targets in the current frame do not overlap with the marks of multiple targets in the previous frame. For example, the marks of multiple targets in the current frame are A and B, respectively. , C, D, the identification of multiple targets in the previous frame can be A ', B', C ', D', where A and A 'can be different targets. As shown in Figure 3, the information of the previous frame can be obtained by real-time detection of multiple targets in the original image of the previous frame, or by real-time detection of multiple targets in the processed image of the previous frame. The target information in the information is represented by a previous feature box that is different from the feature box in the current frame information. For example, if the current feature box is a solid box, the previous feature box is a dotted box. Of course, you can also use the feature box To distinguish between, for example, associating the target's identifier with the feature frame, configuring two different identifiers for the feature frame in the current frame information and the feature frame in the previous frame information, etc.

Specifically, the above image information includes the original image of the current frame, the current feature box, and the previous feature box. The current feature box includes the current identification, current center coordinate information, current length and width (width and height) information, and the appearance of the For the confidence level, the previous feature box includes the last logo, the last center coordinate information, and the last length and width (width and height) information.

It should be noted that the feature frame may also be referred to as a target frame, the aforementioned identifier may also be referred to as an ID, the aforementioned current feature frame may also be referred to as a detection frame, the aforementioned previous feature frame may also be referred to as a tracking frame, and the aforementioned feature range The value is the area value occupied by the feature image in the feature frame. The above-mentioned real-time detection can be performed in the tracker or can be obtained by a tracking algorithm. Those skilled in the art know that the tracker and the tracking algorithm will not repeat them here.

102. Perform a match based on the current frame information of the multiple targets and the previous frame information to determine whether at least one of the multiple targets is successfully matched at one time.

The current frame information includes the current feature boxes of multiple targets or the current feature vectors of multiple targets. The previous frame information includes the previous feature boxes of multiple targets or the previous feature vectors of multiple targets. The detection algorithm matches at least one target among multiple targets.

In the embodiment of the present invention, the current frame information in step 102 includes multiple current feature frames corresponding to multiple targets, and the previous frame information includes multiple previous feature frames corresponding to multiple targets. The information obtained by real-time detection is performed. Output, get a set of image information including multiple current feature boxes and multiple previous feature boxes, for each previous feature box, put it in multiple current feature boxes to match, calculate each The overlapping area of a feature box and multiple current feature boxes, according to the overlapping area, calculate the overlap degree of each previous feature box and multiple current feature boxes (Intersection-over-Union, IoU, also known as intersection and merge ratio), select and The maximum overlapping degree of the previous feature box and a current feature box become a group, as shown in FIG. 3. Compare the maximum overlap degree of the previous feature box with the preset overlap degree threshold. If the maximum overlap degree meets the overlap degree threshold, it is recorded as a successful match; if the maximum overlap degree does not meet the overlap degree threshold, it is recorded as a match. If it is unsuccessful, the previous feature box enters step 103 for secondary matching.

In some possible implementation manners, the similarity of the feature frames may also be compared to match the current feature frame corresponding to the feature frame of the previous frame. Similarity includes: area similarity, length-width (width-height) similarity, etc.

It should be noted that the above-mentioned one-time matching may also be called first-time matching, first-time matching, one-time tracking, first-time tracking, first-time tracking, etc., and may also be directly called tracking.

103. If none of the at least one target is successfully matched, perform a second match to determine whether the at least one target is successfully matched twice. The second match includes at least one of feature matching and distance matching.

In this step, the target that did not match successfully in step 102 is subjected to secondary matching, and the secondary matching includes at least one of feature matching and distance matching. Feature matching includes obtaining the current feature vector of the current feature box, and acquiring the previous feature vector of the previous feature box, and calculating the similarity between the previous feature vector and the current feature vector. Distance matching includes obtaining the distance value between the previous feature box and the current feature box. Select the previous feature box with a similarity greater than the preset similarity threshold to perform a second match with the current feature box, or select the previous feature box with a distance less than the preset distance threshold to perform a second match with the current feature box, or select both The previous feature frame that meets the tracking similarity greater than the preset similarity threshold and the distance value is less than the preset distance threshold performs a second match with the current feature frame.

It should be noted that the foregoing second matching may also be referred to as first matching, rematching, second tracking, second tracking, retracking, and so on.

104. Form information of the at least one target with a successful match and / or a successful second match into output information, where the output information includes current presence information and identification information.

The above-mentioned current presence information includes the current feature box. For example, if the successful match is the current feature box A and the previous feature box A ', the information of the current feature box A is output. The information of the current feature box includes: the center coordinates of the current feature box Information, length and width (width and height) information, etc. The above identification information includes the identification information of the current feature box, which is used to indicate the current feature box. For example, if the current feature box is A, output A. The identification information of the current feature box and the current feature Box.

In step 102, if the target is successfully matched, the target information of the successful match may be placed in an active set, and the target information of unsuccessful match may be placed in a lost set. In step 103, after the target in the missing set is matched for a second time, a target with a successful match is obtained, and the target information of the successful match may be added to the active set, and the information of the target in the active set is output.

Update the target that matches successfully to obtain the current presence information and identification information. For the current feature box and the previous feature box that match successfully, update the identification information of the previous feature box to the current feature box, so that the identification of the current feature box and The identity of the previous feature box is unified, which is used to represent the same target, that is to say, the target is successfully tracked, and at the same time, the previous feature box that matches successfully is deleted, so that only the information of the current feature box exists in the active set, forming the target Current presence information and identification information. For example: A is the identification of the current feature box, A 'is the identification of the previous feature box, A and A' are the pair of current feature boxes and the previous feature box that match successfully, and the identification of the current feature box is changed from A to A ', And then delete the previous feature box in the image information, and record the current feature box A' into the active set, then the output information is the current feature box identification A 'and the center coordinate information, length and width of the current feature box ( Width and height) information and other information.

In the embodiment of the present invention, image information is obtained, and the image information includes current frame information and previous frame information of multiple targets; according to the current frame information of the multiple targets and the previous frame information, a match is made to determine Whether at least one target among the plurality of targets is matched successfully once; if the at least one target is not matched successfully once, then a second match is performed to determine whether the at least one target is successfully matched twice; the first match is successful and / or The output information of the formation of the at least one target after the second match is successful, the output information includes current presence information and identification information. Simultaneous processing of at least one target in a frame of image can increase the efficiency of tracking.

It should be noted that the method for installing a container orchestration engine provided by an embodiment of the present invention can be applied to installation equipment for a container orchestration engine, such as computers, servers, mobile phones, and other devices that can install a container orchestration engine.

Please refer to FIG. 4. FIG. 4 is a schematic flowchart of another multi-target real-time tracking method provided by an embodiment of the present invention. As shown in FIG. 4, the method includes the following steps:

201. Acquire image information, where the image information includes current frame information and previous frame information of multiple targets;

202. Perform a match based on the current frame information of the multiple targets and the previous frame information to determine whether at least one of the multiple targets is successfully matched at one time;

203. If none of the at least one target is successfully matched, perform a second match to determine whether the at least one target is successfully matched twice. The second match includes at least one of feature matching and distance matching.

204. Form information of the at least one target with a successful match and / or a successful second match into output information, where the output information includes current presence information and identification information;

205. If the secondary matching is unsuccessful, regenerate the current frame information of the remaining target to obtain new image information, where the new image information includes the current frame information and the next frame information.

In step 202, if the target is successfully matched, the target information of the successful match may be placed in an active set, and the target information of unsuccessful match may be placed in a lost set. In step 203, after the target in the missing set is matched a second time, the target with a successful match can be obtained. The target information in the successful set can be added to the active set, and the target information in the active set can be output. If the match is not successful, Then go to step 205.

In step 205, for the current feature frame and the previous feature frame that have not been successfully matched for the second time, the target feature frame is regenerated, the regenerated feature frame is recorded in the active set, and the feature frames are regenerated It is consistent with the identification information in the active set. For example: suppose there are two elements of the current feature box A 'and the current feature box D' in the active set, B is the current feature box without matching success, B 'is the previous feature box with matching success, and C is the successful matching with B' In the current feature box, C 'is the previous feature box without a successful match, then the identifier C of the successfully matched current feature box is changed to the identifier B' and recorded in the active set, and the current feature box B 'is obtained. Feature box A ', current feature box B' and current feature box D 'three elements, delete the previous feature box B', delete the previous feature box for the current feature box B and the previous feature box C 'without matching C ', the identifier B of the current feature box is regenerated as E' and recorded in the active set, then there are three elements of the current feature box A ', the current feature box B', the current feature box D 'and the current feature in the active set Box E 'four elements, get the current frame information.

Obtaining new image information includes acquiring new original image information, performing real-time detection on the new original image to obtain the next frame information, adding the current feature frame in the active set to the new image information, and performing the new tracking process cyclically to obtain All tracking results are shown in Figure 5. In addition, step 201 to step 205 can also be executed cyclically, and multiple targets can be tracked.

It should be noted that step 205 is optional. In some embodiments, it is only necessary to form the output information output for the information of the at least one target with a successful match and / or a successful second match. Optionally, the current frame includes current detection information of the multiple targets, and the previous frame information includes historical existence information and corresponding identification information of the multiple targets;

The above current detection information includes the current feature frame information obtained by real-time detection of the current original image and the generated identifier of each current feature frame, and the current feature frame information information includes center coordinate information, length and width (width and height) information, and confidence information , The identification of the current feature box may be a unique identification such as a number unique identification, a letter unique identification, and so on. The above-mentioned historical existence information may be the characteristic frame information existing in the image of the previous frame, and the corresponding identification information is the unique identification of the characteristic frame existing in the image of the previous frame, and may also be said to be the unique identification of the previous characteristic frame. The degree of overlap may be the degree of overlap between the current feature frame and the previous feature frame, including the degree of overlap of the length, width (width and height) coordinates, and the degree of overlap of the area.

In some possible embodiments, the degree of overlap may also be the similarity of feature vectors or the similarity of feature frames.

When the above overlapping degree or similarity is greater than the preset threshold, the at least one target can be judged as a successful match, and when the above overlapping degree or similarity is less than the preset threshold, the at least one target can be determined as One match was unsuccessful.

For each previous feature box, place it in multiple current feature boxes to match, calculate the overlapping area of each previous feature box and multiple current feature boxes, and calculate each previous feature based on the overlapping area The overlap degree of the frame and multiple current feature frames (Intersection-over-Union, IoU, also known as cross-combination ratio), select the maximum overlap degree with the previous feature frame, one current feature frame becomes a group, and the previous feature frame The maximum overlap degree is compared with the preset overlap threshold, for example: A 'is the identifier of the previous feature box, B' is the identifier of the previous feature box, A is the identifier of the current feature box, and B is the current feature box. Mark, the overlapping degree of A 'and A is 0.4, and the overlapping degree of A' and B is 0.8, then A 'and B have the maximum overlapping degree, which is recorded as a group, the overlapping degree of B' and A is 0.4, B 'and If the overlap degree of B is 0.5, then B 'and B have the maximum overlap degree, which is recorded as a group, and the overlap degree of A' and A is 0.4. If the maximum overlap degree meets the overlap degree threshold, it is recorded as a successful match. Assuming the overlap degree The threshold is 0.6, A 'matches with B successfully, and the overlap between B' and B is less than 0.6, Then, the matching between B 'and B is unsuccessful. If the maximum degree of overlap does not meet the threshold of the degree of overlap, it is recorded as one-time unsuccessful matching, and the previous feature box proceeds to step 203 to perform the second-time matching.

The above-mentioned current detection information includes the current feature frame information and identification, and the historical presence information includes the previous feature frame information and identification. The current feature vector can be obtained by extracting the direction gradient histogram (Histogram of Oriented Gradient, HOG for short) of the current feature box. Similarly, the previous feature vector can be obtained by extracting the direction gradient histogram of the previous feature box On the previous HOG feature vector, the cosine similarity between the current HOG feature vector and the previous HOG feature vector is calculated.

The current feature box includes the current center coordinate information, current length and width (width and height) information, the previous feature box includes the previous center coordinate information, the previous length and width (width and height) information, and the distance value between the current feature box and the previous feature box It can be calculated by the following formula:

D = sqrt ((x1-x2) ² + (y1-y2) ² ) / min (w1, w2)

Where D is the distance between the current feature box and the previous feature box, x1, y1, w1 belong to the current feature box, x1, y1 are the center coordinates of the current feature box, w1 is the width of the current feature box, x2, y2, w2 Belongs to the previous feature frame, x2 and y2 are the center coordinates of the previous feature frame, and w2 is the width of the previous feature frame.

The judgment rule includes: when the cosine similarity is greater than the preset cosine similarity threshold, it can be considered that the second match is successful. In addition, when the distance value is less than the set distance threshold, it can also be considered that the secondary matching is successful. Of course, when the cosine similarity is greater than the preset cosine similarity threshold and the distance value is less than the set distance threshold, it can also be considered that the second match is successful.

The above current presence information includes the current feature box. For example, if the matching is successful between the current feature box A and the previous feature box A ', the information of the current feature box A is output. The information of the current feature box includes: the center coordinates of the current feature box Information, length and width (width and height) information, etc. The above identification information includes the identification information of the current feature box, used to represent the current feature box, for example, the current feature box is A, then output A, and the current feature box is A ', then output A ', the identification information of the current feature box is associated with the current feature box.

In step 202, if the target is successfully matched, the target information of the successful match may be placed in an active set, and the target information of unsuccessful match may be placed in a lost set. In step 203, after the target in the missing set is matched a second time, a target with a successful match is obtained, and the target information of the successful match can be added to the active set, and the information of the target in the active set is output.

Update the target that matches successfully to obtain the current presence information and identification information. For the current feature box and the previous feature box that match successfully, update the identification information of the previous feature box to the current feature box, so that the identification of the current feature box and The identity of the previous feature box is unified, which is used to represent the same target, that is to say, the target is successfully tracked, and at the same time, the previous feature box that matches successfully is deleted so that only the information of the current feature box exists in the active set, forming the target Current presence information and identification information. For example: A is the identification of the current feature box, A 'is the identification of the previous feature box, A and A' are the pair of current feature boxes and the previous feature box that match successfully, and the identification of the current feature box is changed from A to A ', And then delete the previous feature box in the image information, and record the current feature box A' into the active set, then the output information is the current feature box identification A 'and the center coordinate information, length and width of the current feature box ( Width and height) information and other information.

In a second aspect, as shown in FIG. 6, a multi-target real-time tracking device is provided. The device includes:

The obtaining module 401 is used to obtain image information, where the image information includes current frame information and previous frame information of multiple targets;

The first matching module 402 is configured to perform a match based on the current frame information of the multiple targets and the previous frame information, and determine whether at least one of the multiple targets is successfully matched at one time;

The second matching module 403 is configured to perform secondary matching if the at least one target has not been matched successfully once, and determine whether the at least one target has succeeded in secondary matching. The secondary matching includes at least one of feature matching and distance matching. One item

The output module 404 is configured to form information of the at least one target with a successful match and / or a successful second match into output information, where the output information includes current presence information and identification information.

Optionally, as shown in FIG. 7, after the at least one target is not successfully matched once, a second match is performed to determine whether the at least one target is successfully matched twice, the device further includes:

The generating module 405 is configured to regenerate the current frame information of the remaining target to obtain new image information if the second matching is unsuccessful, and the new image information includes the current frame information and the next frame information.

Optionally, as shown in FIG. 8, the current frame includes current detection information of the multiple targets, and the previous frame information includes historical existence information and corresponding identification information of the multiple targets;

The first matching module 402 includes:

The first processing unit 4021 is configured to calculate the degree of overlap between the current detection information of at least one target of the plurality of targets and the historical presence information of at least one target of the plurality of targets to obtain the current detection of the at least one target The degree of overlap between the information and the historical existence information of the at least one target;

The first determining unit 4022 is configured to determine whether the at least one target is successfully matched at a time according to the degree of overlap.

Optionally, as shown in FIG. 9, the first judgment unit 4022 includes:

The comparison subunit 40221 is used to select a maximum overlap degree and compare with a preset overlap degree threshold to determine whether the maximum overlap degree is greater than the overlap degree threshold;

The judgment subunit 40222 is configured to match once if the maximum overlap degree is greater than the overlap degree threshold, and fail to match once if the maximum overlap degree is less than the overlap degree threshold.

Optionally, as shown in FIG. 10, the current frame includes current detection information of the multiple targets, and the previous frame information includes historical existence information and corresponding identification information of the multiple targets;

The second matching module 403 includes:

The second processing unit 4031 is configured to extract a current feature vector of current detection information of at least one target of the plurality of targets, extract a historical feature vector of historical presence information of at least one target of the plurality of targets, and convert the current Calculating the feature vector and the historical feature vector to obtain the cosine similarity of the at least one target;

The third processing unit 4032 is configured to extract the current coordinates of the current detection information of the at least one target and the historical coordinates of the historical presence information of the at least one target, and calculate the current coordinates and the historical coordinates of the at least one target, Obtain the distance value of the at least one target;

The second determining unit 4033 is configured to determine whether the at least one target is successfully matched twice according to the cosine similarity and distance values of the at least one target.

Optionally, as shown in FIG. 11, the output module includes:

The updating unit 4041 is configured to update the current detection information of the at least one target to the current presence information, and associate the corresponding identification information of the at least one target to the current detection information;

The output unit 4042 forms the output information of the at least one target according to the current presence information and the corresponding identification information.

In a third aspect, an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, when the processor executes the computer program The steps in the multi-target real-time tracking method provided by the embodiments of the present invention are implemented.

According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium that stores a computer program on the computer-readable storage medium, and when the computer program is executed by a processor, realizes multi-target real-time tracking provided by an embodiment of the present invention Steps in the method.

The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and it cannot be assumed that the specific embodiments of the present invention are limited to these descriptions. For a person of ordinary skill in the technical field to which the present invention belongs, without deviating from the concept of the present invention, several simple deductions or replacements can be made, which should be regarded as falling within the protection scope of the present invention.

Claims

A multi-target real-time tracking method, characterized in that the method includes:

Obtain image information, where the image information includes current frame information and previous frame information of multiple targets;

Perform a match based on the current frame information of the multiple targets and the previous frame information, and determine whether at least one of the multiple targets is successfully matched at a time;

If the at least one target is not matched successfully once, then a second match is performed to determine whether the at least one target is successfully matched twice, and the second match includes at least one of feature matching and distance matching;

The information of the at least one target with a successful match and / or a successful second match is formed as output information, and the output information includes current presence information and identification information.
The method according to claim 1, wherein after the at least one target is not successfully matched once, performing a second match to determine whether the at least one target is successfully matched a second time, further comprising:

If the second matching is unsuccessful, the current frame information of the remaining target is regenerated to obtain new image information, where the new image information includes the current frame information and the next frame information.
The method of claim 2, wherein the current frame includes current detection information of the multiple targets, and the previous frame information includes historical presence information and corresponding identification information of the multiple targets;

The matching the current frame information of the multiple targets with the previous frame information once to determine whether at least one of the multiple targets is successfully matched at a time includes:

Calculating the degree of overlap between the current detection information of at least one target of the plurality of targets and the historical presence information of at least one target of the plurality of targets to obtain the current detection information of the at least one target and the information of the at least one target The degree of overlap of information in history;

According to the degree of overlap, it is determined whether the at least one target is successfully matched at a time.
The method according to claim 3, wherein the judging whether the at least one target is successfully matched according to the degree of overlap includes:

Selecting the maximum overlap degree to compare with the preset overlap degree threshold to determine whether the maximum overlap degree is greater than the overlap degree threshold;

If the maximum overlap degree is greater than the overlap degree threshold, a match is successful, and if the maximum overlap degree is less than the overlap degree threshold, a match is unsuccessful.
The method according to claim 2, wherein the current frame includes current detection information of the plurality of targets, and the previous frame information includes historical existence information and corresponding identification information of the plurality of targets;

The performing second matching to determine whether the at least one target is successfully matched twice includes:

Extracting a current feature vector of current detection information of at least one target of the plurality of targets, extracting a historical feature vector of historical presence information of at least one target of the multiple targets, and combining the current feature vector with the historical feature vector Performing calculation to obtain the cosine similarity of the at least one target;

Extracting the current coordinates of the current detection information of the at least one target and the historical coordinates of the historical presence information of the at least one target, calculating the current coordinates of the at least one target and the historical coordinates to obtain the distance of the at least one target value;

According to the cosine similarity and distance values of the at least one target, it is determined whether the at least one target is successfully matched twice.
The method according to claim 5, wherein the determining whether the at least one target is successfully matched twice according to the feature value and the distance value of the at least one target includes:

Comparing the characteristic value cosine similarity with a preset cosine similarity threshold, and comparing the distance value with a preset distance threshold to obtain a comparison result;

According to the comparison result, it is determined whether the at least one target is successfully matched twice according to a preset judgment rule.
The method according to claim 5, wherein the output information of the formation of the at least one target for which the first match is successful and / or the second match is successful includes:

Updating the current detection information of the at least one target to the current presence information, and associating the corresponding identification information of the at least one target to the current detection information;

The output information of the at least one target is formed according to the current presence information and the corresponding identification information.
A multi-target real-time tracking device, characterized in that the device includes:

An acquisition module, for acquiring image information, the image information including current frame information and previous frame information of multiple targets;

The first matching module is configured to perform a match based on the current frame information of the multiple targets and the previous frame information, and determine whether at least one of the multiple targets is successfully matched at one time;

The second matching module is used to perform secondary matching if the at least one target has not been matched once, and determine whether the at least one target has been successfully matched twice. The secondary matching includes at least one of feature matching and distance matching. item;

The output module is configured to form information of the at least one target with a successful match and / or a successful second match into output information, where the output information includes current presence information and identification information.
An electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the computer program as claimed in claim 1 Steps in the multi-target real-time tracking method described in any one of 7.
A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the multi-objective according to any one of claims 1 to 7 is realized Follow the steps in the method in real time.