WO2023184197A1

WO2023184197A1 - Target tracking method and apparatus, system, and storage medium

Info

Publication number: WO2023184197A1
Application number: PCT/CN2022/083887
Authority: WO
Inventors: 王杰; 孔繁昊
Original assignee: 京东方科技集团股份有限公司; 成都京东方智慧科技有限公司
Priority date: 2022-03-30
Filing date: 2022-03-30
Publication date: 2023-10-05
Also published as: CN117178292A

Abstract

A target tracking method and apparatus, a system, and a storage medium. The method comprises: at a first moment, acquiring an image to be subjected to detection; determining the detected positions of key points of detection targets in said image; acquiring the positions, at a second moment, of key points of targets to be tracked, wherein the second moment is a moment before the first moment; according to the positions, at the second moment, of the key points of the targets to be tracked, predicting the predicted positions, at the first moment, of the key points of the targets to be tracked; according to the detected positions and the predicted positions, determining a matching relationship between the detection targets and the targets to be tracked, wherein the matching relationship is used for matching the same targets among the detection targets and the targets to be tracked; and according to the matching relationship, tracking the targets to be tracked.

Description

Target tracking method, device, system and storage medium

Technical field

The present disclosure relates to the field of image processing, and in particular, to a target tracking method, device, system and storage medium.

Background technique

In the field of image processing, related solutions for detection and tracking of targets and key points are widely used in behavior recognition, posture analysis, and three-dimensional key point estimation. For example, tracking pedestrians can be used to determine the pedestrian's route trajectory, and tracking key points of the pedestrian can be used to determine whether the pedestrian has fallen.

Related technologies usually require target detection and key point detection of the target object respectively, and tracking the target object based on the outer box image obtained by the target detection, so as to indirectly detect the key points based on the relationship between the outer box image of the target object and the key points. Follow up. Another related technology detects key points of the target object and generates an outer box image of the target object based on the key points, thereby tracking the target object based on the generated outer box image.

Contents of the invention

On the one hand, a target tracking method is provided, which method includes: obtaining an image to be detected at a first moment; determining the detection position of a key point of the target to be detected in the image to be detected; and obtaining the position of the key point of the target to be tracked at a second moment. ; The second moment is the moment before the first moment; according to the position of the key point of the target to be tracked at the second moment, predict the predicted position of the key point of the target to be tracked at the first moment; according to the detection position and the predicted position, determine the detection The matching relationship between the target and the target to be tracked; the matching relationship is used to match the same target in the detected target and the target to be tracked; based on the matching relationship, the target to be tracked is tracked.

In some embodiments, the method includes: performing a first target operation on each detected target and each target to be tracked, and determining a distance between each detected target and each target to be tracked; the first target operation includes: according to The distance between the detected position of the key point in the i-th detection target and the predicted position of the key point of the j-th target to be tracked determines the distance between the i-th detection target and the j-th target to be tracked; where , i is a positive integer less than or equal to N; j is a positive integer less than or equal to M; the matching relationship is determined based on the distance between each detected target and each target to be tracked.

In some embodiments, the method includes: performing a second target operation on each corresponding key point of the i-th detected target and the j-th target to be tracked, and determining a distance between each corresponding key point; the second target operation It includes: determining the first key point and the second key point; determining the distance between the first key point and the second key point according to the detected position of the first key point and the predicted position of the second key point; the first key point is the Any key point among the i detection targets; the second key point is the key point corresponding to the first key point in the j-th target to be tracked; determine the i-th detection target based on the distance between each corresponding key point The distance between the jth target to be tracked.

In some embodiments, the distance between the i-th detected target and the j-th target to be tracked is calculated by the following formula:

Among them, D(i,j) represents the distance between the i-th detection target and the j-th target to be tracked, p _ik represents the detection position of the k-th key point of the i-th detection target, and p′ _jk represents the j-th The predicted position of the k-th key point of the target to be tracked, d(p _ik , p′ _jk ) represents the distance between the k-th key point of the i-th detection target and the k-th key point of the j-th target to be tracked _The distance _of δ(v _ik ,v′ _jk ) represents the detection situation of the k-th key point of the i-th detection target and the k-th key point of the j-th target to be tracked.

In some embodiments, the matching relationship includes at least one of the following: at least one target to be tracked matches the detected target, at least one target to be tracked does not match the detected target, and at least one detected target does not match the target to be tracked. match.

In some embodiments, the method includes: determining the weight value of each target to be tracked and the weight value of each detected target; wherein the weight value of any target to be tracked is the weight value of any target to be tracked and each detected target respectively. The value of the minimum distance among the distances; the weight value of each detection target is the preset weight value; perform the third target operation on each target to be tracked, and determine the detection target matching each target to be tracked; the third target operation includes : Determine the first detection target with the shortest distance from the mth target to be tracked; where the sum of the weight values of the first detection target and the mth target to be tracked is greater than or equal to the sum of the weight values of the first detection target and the mth target to be tracked distance; m is a positive integer; perform the fourth target operation on the mth target to be tracked and the first detection target to determine the detection target matching the mth target to be tracked; the fourth target operation includes: determining whether there is the first target to be tracked Tracking target; the first target to be tracked is the target to be tracked that matches the first detection target determined according to the third target operation; in the absence of the first target to be tracked, determine the mth target to be tracked and the first detection target Target matching; in the case where there is a first target to be tracked, determine the first influence value of the m-th target to be tracked, and the second influence value of the first target to be tracked; where the first influence value is based on the m-th target The difference between the target to be tracked and the second detection target; the difference is the difference between the distance between a target to be tracked and a detection target and the sum of the weights of a target to be tracked and a detection target; the second detection target is Except for the first detection target, it is the detection target with the smallest difference determined from the mth target to be tracked; the second influence value is the difference determined based on the first target to be tracked and the third detection target; the third detection target is the detection target except the mth target to be tracked. A detection target that has the smallest difference between a detection target and the first target to be tracked; determine a detection target that matches the mth target to be tracked based on the first influence value and the second influence value.

In some embodiments, the method includes: when the first influence value is less than the second influence value, performing a fourth target operation on the mth target to be tracked and the second detected target, and determining the relationship between the mth target to be tracked and the second detected target. Matching detection target; when the first influence value is greater than the second influence value, it is determined that the mth target to be tracked matches the first detection target.

In some embodiments, the method further includes: when the first influence value is greater than the second influence value, performing a fourth target operation on the first target to be tracked and the third target to be detected, and determining whether they match the first target to be tracked. detection target.

In some embodiments, the method further includes: when the first influence value is less than the second influence value, updating the weights of the mth target to be tracked, the first target to be tracked and the first detected target according to the first influence value. value; when the first influence value is greater than the second influence value, the weight values of the m-th target to be tracked, the first target to be tracked and the first detected target are updated according to the second influence value.

In some embodiments, the method includes: increasing the weight value of the mth target to be tracked by a first influence value, increasing the weight value of the first target to be tracked by a first influence value, and reducing the weight value of the first detected target. First influence value; increase the weight value of the mth target to be tracked by the second influence value, increase the weight value of the first target to be tracked by the second influence value, and reduce the weight value of the first detection target by the second influence value.

In some embodiments, the method includes: determining a list of targets to be tracked and a list of detected targets; wherein the list of targets to be tracked includes each target to be tracked, and the list of detected targets includes each detected target; the list of targets to be tracked and the list of detected targets The target to be tracked and the target to be detected in the target iteratively execute the fifth target operation until there is no target to be tracked in the list of targets to be tracked or there is no target to be detected in the list of detection targets, and the matching relationship is determined based on the execution result of the fifth target operation; fifth The target operation includes: determining the target distance between each target to be tracked in the current target list to be tracked and each target to be tracked in the current detected target list; determining the minimum distance among the target distances, and the second value corresponding to the minimum distance. The target to be tracked and the fourth detected target; the distance between the second target to be tracked and the fourth detected target is the minimum distance; the matching relationship between the second target to be tracked and the fourth detected target is determined; deleted from the current list of targets to be tracked The second target to be tracked deletes the fourth detection target from the current detection target list.

In some embodiments, the method includes: determining a third target to be tracked; each target to be tracked in the third target to be tracked has a matching detection target; and based on the key points of the detection target matching the third target to be tracked. Detect the position and the predicted position of the key point of the third target to be tracked, and update the position of the key point of the third target to be tracked.

In some embodiments, the method includes: determining a fifth detection target; each of the fifth detection targets does not have a matching target to be tracked; and adding a fifth detection target to the targets to be tracked.

In some embodiments, the method includes: determining the fourth target to be tracked and the number of mismatches for the fourth target to be tracked; there is no matching detection target for each target to be tracked in the fourth target to be tracked; the number of mismatches is The number of times that the target to be tracked has no matching detection target continuously; when the number of mismatches is greater than the preset tracking threshold, the fourth target to be tracked is deleted from the targets to be tracked.

In some embodiments, the method includes: when the number of mismatches is less than or equal to a preset tracking threshold, changing the predicted position of the key point of the fourth target to be tracked or the key point of the fourth target to be tracked at the second moment The position is used as the updated position of the key point of the fourth target to be tracked.

In some embodiments, the method includes: when there is a matching detection target for the fifth target to be tracked, updating the third target according to the detection position of the key point of the matching detection target and the predicted position of the key point of the fifth target to be tracked. The position of the key point of the fifth target to be tracked; the fifth target to be tracked is any target among the targets to be tracked; there is no matching detection target in the fifth target to be tracked, and the number of mismatches of the fifth target to be tracked is greater than the predetermined number When the tracking threshold is set, the fifth target to be tracked is deleted from the targets to be tracked; the number of mismatches is the number of consecutive times that the target to be tracked does not have a matching detection target; in the case where the sixth detection target does not have a matching target to be tracked Next, a sixth detection target is added to the targets to be tracked; the sixth detection target is any target among the detection targets.

In some embodiments, the method includes: when there is no matching detection target for the fifth target to be tracked, and the number of mismatches for the fifth target to be tracked is less than or equal to a preset tracking threshold, moving the fifth target to be tracked The predicted position of the key point or the position of the fifth target to be tracked at the second moment is used as the updated position of the key point of the fifth target to be tracked.

In some embodiments, the method includes: determining the motion state of the key points of the target to be tracked; and predicting the position and motion state of the key points of the target to be tracked at the first moment based on the position and motion state of the key points of the target to be tracked at the first moment. Predicted location.

In some embodiments, the method includes: when the number of detection targets is 0, determining that there is no matching detection target for each target to be tracked.

In some embodiments, the method includes: when the number of targets to be tracked is 0, determining that there is no matching target to be tracked for each detected target.

On the other hand, a target tracking device is provided, including: a processing unit and an acquisition unit. The acquisition unit is configured to acquire the image to be detected at the first moment; the processing unit is configured to determine the detection position of the key point of the detection target in the image to be detected; the acquisition unit is also configured to acquire the key point of the target to be tracked at The position at the second moment; the second moment is the moment before the first moment; the processing unit is also configured to predict the position of the key point of the target to be tracked at the first moment based on the position of the key point of the target to be tracked at the second moment. predicted position; the processing unit is further configured to determine a matching relationship between the detected target and the target to be tracked based on the detected position and the predicted position; the matching relationship is used to match the same target in the detected target and the target to be tracked; the processing unit is also configured to Configured to track the target to be tracked based on the matching relationship.

In some embodiments, the processing unit is configured to perform a first target operation on each detected target and each target to be tracked, and determine a distance between each detected target and each target to be tracked; the first target operation includes : Determine the distance between the i-th detection target and the j-th target to be tracked based on the distance between the detection position of the key point in the i-th detection target and the predicted position of the key point of the j-th target to be tracked ; where i is a positive integer less than or equal to N; j is a positive integer less than or equal to M; the processing unit is also configured to determine a matching relationship based on the distance between each detected target and each target to be tracked.

In some embodiments, the processing unit is configured to perform a second target operation on each corresponding key point of the i-th detected target and the j-th target to be tracked, and determine the distance between each corresponding key point; second The target operation includes: determining the first key point and the second key point; determining the distance between the first key point and the second key point based on the detected position of the first key point and the predicted position of the second key point; the first key point is any key point in the i-th detection target; the second key point is the key point corresponding to the first key point in the j-th target to be tracked; the processing unit is also configured to determine the relationship between each corresponding key point The distance determines the distance between the i-th detection target and the j-th target to be tracked.

In some embodiments, the processing unit is configured to determine the weight value of each target to be tracked and the weight value of each detected target; wherein the weight value of any target to be tracked is the weight value of any target to be tracked and each target respectively. The value of the minimum distance among the distances of the detected targets; the weight value of each detected target is the preset weight value; perform a third target operation on each target to be tracked, and determine the detection target that matches each target to be tracked; the third target The operation includes: determining the first detected target with the shortest distance from the mth target to be tracked; wherein the sum of the weight values of the first detected target and the mth target to be tracked is greater than or equal to the first detected target and the mth target to be tracked The distance of the target; m is a positive integer; perform the fourth target operation on the mth target to be tracked and the first detection target to determine the detection target matching the mth target to be tracked; the fourth target operation includes: determining whether there is the A target to be tracked; the first target to be tracked is a target to be tracked that matches the first detected target determined according to the third target operation; in the absence of the first target to be tracked, determine the mth target to be tracked and the th target to be tracked Detection target matching; in the case where the first target to be tracked exists, determine the first influence value of the mth target to be tracked, and the second influence value of the first target to be tracked; wherein the first influence value is based on the The difference between m targets to be tracked and the second detection target; the difference is the difference between the distance between a target to be tracked and a detection target and the sum of the weights of a target to be tracked and a detection target; the second detection The target is the detection target with the smallest difference between the mth target to be tracked and the first detection target; the second influence value is the difference determined based on the first target to be tracked and the third detection target; the third detection target is Except for the first detection target, the detection target with the smallest difference determined from the first target to be tracked is determined; based on the first influence value and the second influence value, the detection target matching the mth target to be tracked is determined.

In some embodiments, the processing unit is configured to perform a fourth target operation on the mth target to be tracked and the second detected target, and determine the relationship between the mth target to be tracked and the second detected target when the first influence value is less than the second influence value. The tracking target matches the detection target; when the first influence value is greater than the second influence value, it is determined that the mth target to be tracked matches the first detection target.

In some embodiments, the processing unit is configured to perform a fourth target operation on the first target to be tracked and the third target to be detected, and determine the relationship between the first target to be tracked and the first target to be tracked when the first influence value is greater than the second influence value. Matching detection targets.

In some embodiments, the processing unit is configured to update the mth target to be tracked, the first target to be tracked, and the first detected target according to the first influence value when the first influence value is less than the second influence value. Weight value; when the first influence value is greater than the second influence value, the weight values of the mth target to be tracked, the first target to be tracked and the first detected target are updated according to the second influence value.

In some embodiments, the processing unit is configured to increase the weight value of the mth target to be tracked by the first influence value, increase the weight value of the first target to be tracked by the first influence value, and increase the weight value of the first detected target by the first influence value. The value decreases the first influence value; the processing unit is further configured to increase the weight value of the mth target to be tracked by the second influence value, increase the weight value of the first target to be tracked by the second influence value, and increase the weight value of the first detection target by the second influence value. The weight value reduces the second influence value.

In some embodiments, the processing unit is configured to determine a list of targets to be tracked and a list of detected targets; wherein the list of targets to be tracked includes each target to be tracked, and the list of detected targets includes each detected target; the list of targets to be tracked and the list of detected targets are The target to be tracked and the detection target in the target list iteratively perform the fifth target operation until there is no target to be tracked in the target list to be tracked or there is no detection target in the detection target list, and the matching relationship is determined based on the execution result of the fifth target operation; The fifth target operation includes: determining the target distance between each target to be tracked in the current target list to be tracked and each target to be tracked in the current detected target list; determining the minimum distance in the target distance, and the minimum distance corresponding to The second target to be tracked and the fourth target to be detected; the distance between the second target to be tracked and the fourth target to be detected is the minimum distance; the matching relationship between the second target to be tracked and the fourth target to be detected is determined; from the current list of targets to be tracked Delete the second target to be tracked from , and delete the fourth detection target from the current detection target list.

In some embodiments, the processing unit is configured to determine a third target to be tracked; each of the third targets to be tracked has a matching detection target; the processing unit is further configured to determine the third target to be tracked according to The target matches the detected position of the key point of the detected target and the predicted position of the key point of the third target to be tracked, and updates the position of the key point of the third target to be tracked.

In some embodiments, the processing unit is configured to determine a fifth detection target; each detection target in the fifth detection target does not have a matching target to be tracked; the processing unit is further configured to add a third target to the target to be tracked. Five detection targets.

In some embodiments, the processing unit is configured to determine the fourth target to be tracked and the number of mismatches of the fourth target to be tracked; there is no matching detection target for each target to be tracked in the fourth target to be tracked; no matching The number of times is the number of consecutive times that the target to be tracked does not have a matching detection target; the processing unit is also configured to delete the fourth target to be tracked from the targets to be tracked when the number of mismatches is greater than the preset tracking threshold.

In some embodiments, the processing unit is configured to, when the number of mismatches is less than or equal to a preset tracking threshold, change the predicted position of the key point of the fourth target to be tracked or the key point of the fourth target to be tracked in the first The position at the second moment is used as the updated position of the key point of the fourth target to be tracked.

In some embodiments, the processing unit is configured to, in the case where a matching detection target exists in the fifth target to be tracked, based on the detection position of the key point of the matching detection target and the predicted position of the key point of the fifth target to be tracked. Update the position of the key point of the fifth target to be tracked; the fifth target to be tracked is any target among the targets to be tracked; the processing unit is also configured to have no matching detection target in the fifth target to be tracked, and the fifth target to be tracked is any one of the targets to be tracked; When the number of mismatches of the target to be tracked is greater than the preset tracking threshold, the fifth target to be tracked is deleted from the targets to be tracked; the number of mismatches is the number of consecutive times that the target to be tracked does not have a matching detection target; the processing unit is also It is configured to add a sixth detection target to the targets to be tracked when the sixth detection target does not have a matching target to be tracked; the sixth detection target is any target among the detection targets.

In some embodiments, the processing unit is configured to, when the fifth target to be tracked does not have a matching detection target and the number of mismatches of the fifth target to be tracked is less than or equal to a preset tracking threshold, The predicted position of the key point of the tracking target or the position of the fifth target to be tracked at the second moment is used as the updated position of the key point of the fifth target to be tracked.

In some embodiments, the processing unit is configured to determine the motion state of the key points of the target to be tracked; the processing unit is also configured to predict the position and motion state of the key points of the target to be tracked at the second moment. The predicted position of the key point of the target at the first moment.

In some embodiments, the processing unit is configured to determine that there is no matching detection target for each target to be tracked when the number of detection targets is 0.

In some embodiments, the processing unit is configured to determine that there is no matching target to be tracked for each detected target when the number of targets to be tracked is 0.

In yet another aspect, a non-transitory computer-readable storage medium is provided. The computer-readable storage medium stores computer program instructions. When the computer program instructions are run on a computer (for example, a target tracking device), they cause the computer to execute the target tracking method as described in any of the above embodiments.

In yet another aspect, a computer program product is provided. The computer program product includes computer program instructions. When the computer program instructions are executed on a computer (eg, a target tracking device), the computer program instructions cause the computer to perform the target tracking method as described in any of the above embodiments.

In yet another aspect, a computer program is provided. When the computer program is executed on a computer (eg, a target tracking device), the computer program causes the computer to perform the target tracking method as described in any of the above embodiments.

In another aspect, a target tracking device is provided. The device includes: a processor and a memory; the memory is used to store computer programs or instructions, and the processor is used to run the computer programs or instructions to achieve the goals described in any of the above embodiments. Tracking methods.

In another aspect, a chip is provided. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run computer programs or instructions to implement the target tracking method as described in any of the above embodiments.

Exemplarily, the chip provided in the present disclosure also includes a memory for storing computer programs or instructions.

It should be noted that the above computer instructions may be stored in whole or in part on a computer-readable storage medium. The computer-readable storage medium may be packaged together with the processor of the device, or may be packaged separately from the processor of the device, which is not limited in this disclosure.

In another aspect, a target tracking system is provided, including: a target tracking device and an image acquisition device, wherein the image acquisition device is used to collect images to be detected, and the target tracking device is used to perform target tracking as described in any of the above embodiments. method.

In this disclosure, the name of the above-mentioned target tracking device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the functions of each device or functional module are similar to the present disclosure, they fall within the scope of the claims of the present disclosure and its equivalent technology.

Description of drawings

In order to explain the technical solutions in the present disclosure more clearly, the drawings required to be used in some embodiments of the present disclosure will be briefly introduced below. Obviously, the drawings in the following description are only appendices of some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings. In addition, the drawings in the following description can be regarded as schematic diagrams and are not intended to limit the actual size of the product, the actual flow of the method, the actual timing of the signals, etc. involved in the embodiments of the present disclosure.

Figure 1 is an architectural diagram of a target tracking system 10 provided according to some embodiments;

Figure 2 is a relationship diagram between goals and key points provided according to some embodiments;

Figure 3 is a flow chart of a target tracking method provided according to some embodiments;

Figure 4 is a matching relationship diagram between a detected target and a target to be tracked according to some embodiments;

Figure 5 is a flow chart of another target tracking method provided according to some embodiments;

Figure 6 is a scene diagram of target tracking provided according to some embodiments;

Figure 7 is a schematic diagram of target tracking in related technology;

Figure 8 is a schematic diagram of a target tracking provided according to some embodiments of the present disclosure;

Figure 9 is a flow chart of another target tracking method provided according to some embodiments;

Figure 10 is a flowchart of a third target operation provided in accordance with some embodiments;

Figure 11 is a flowchart of a fourth target operation provided in accordance with some embodiments;

Figure 12 is a flow chart of another target tracking method provided according to some embodiments;

Figure 13 is a flowchart of a fifth target operation provided in accordance with some embodiments;

Figure 14 is a flow chart of another target tracking method provided according to some embodiments;

Figure 15 is a relationship diagram between a target to be tracked and key points of corresponding matching detected targets provided according to some embodiments;

Figure 16 is a flow chart of another target tracking method provided according to some embodiments;

Figure 17 is a structural diagram of a target tracking device provided according to some embodiments;

Figure 18 is a structural diagram of another target tracking device provided according to some embodiments.

Detailed ways

The technical solutions in some embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments provided by this disclosure, all other embodiments obtained by those of ordinary skill in the art fall within the scope of protection of this disclosure.

Unless the context otherwise requires, throughout the specification and claims, the term "comprise" and its other forms such as the third person singular "comprises" and the present participle "comprising" are used. Interpreted as open and inclusive, it means "including, but not limited to." In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "example", "specific "example" or "some examples" and the like are intended to indicate that a particular feature, structure, material or characteristic associated with the embodiment or example is included in at least one embodiment or example of the present disclosure. The schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be included in any suitable manner in any one or more embodiments or examples.

Hereinafter, the terms “first” and “second” are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Therefore, features defined as "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.

In describing some embodiments, expressions "coupled" and "connected" and their derivatives may be used. For example, some embodiments may be described using the term "connected" to indicate that two or more components are in direct physical or electrical contact with each other. As another example, the term "coupled" may be used when describing some embodiments to indicate that two or more components are in direct physical or electrical contact. However, the terms "coupled" or "communicatively coupled" may also refer to two or more components that are not in direct contact with each other but still cooperate or interact with each other. The embodiments disclosed herein are not necessarily limited by the content herein.

"At least one of A, B and C" has the same meaning as "at least one of A, B or C" and includes the following combinations of A, B and C: A only, B only, C only, A and B The combination of A and C, the combination of B and C, and the combination of A, B and C.

"A and/or B" includes the following three combinations: A only, B only, and a combination of A and B.

As used herein, the term "if" is optionally interpreted to mean "when" or "in response to" or "in response to determining" or "in response to detecting," depending on the context. Similarly, depending on the context, the phrase "if it is determined..." or "if [stated condition or event] is detected" is optionally interpreted to mean "when it is determined..." or "in response to the determination..." or “on detection of [stated condition or event]” or “in response to detection of [stated condition or event]”.

The use of "suitable for" or "configured to" in this document implies open and inclusive language that does not exclude devices that are suitable for or configured to perform additional tasks or steps.

Additionally, the use of "based on" is meant to be open and inclusive in that a process, step, calculation or other action "based on" one or more stated conditions or values may in practice be based on additional conditions or beyond the stated values.

As used herein, "about," "approximately," or "approximately" includes the stated value as well as an average within an acceptable range of deviations from the particular value, as determined by one of ordinary skill in the art. Determined taking into account the measurement in question and the errors associated with the measurement of the specific quantity (i.e., the limitations of the measurement system).

In the following, nouns involved in the embodiments of the present disclosure are explained to facilitate readers' understanding.

(1)Key point detection

Key point detection is used to detect specific nodes corresponding to the target, and can be used for behavior recognition, posture analysis, etc. Among them, key point detection algorithms include top-down detection methods and bottom-up detection methods. The top-down detection method is to detect targets in images and perform key point detection based on the detected targets. For example, top-down detection methods include the HRNet algorithm (Deep High-Resolution Representation Learning for Human Pose Estimation). The bottom-up detection method is to detect all key points from the image, and then cluster according to the position of each key point to obtain the target corresponding to the associated key point. For example, bottom-up detection methods include the OpenPose algorithm (Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields).

(2)Hungarian algorithm

The Hungarian algorithm is used to determine the matching relationship between elements in two sets, that is, to perform data association on the elements in the two sets to obtain matching results with as many combinations as possible.

(3)KM algorithm (Kuhn-Munkres algorithm)

Based on the Hungarian algorithm, the KM algorithm uses the correlation degree of the elements in the two sets as weights to perform data correlation on the elements in the two sets, thereby obtaining a matching result with the optimal matching degree.

(4) Cross-union ratio

The intersection over union (IoU) refers to the ratio of the intersection area of two image areas to the union area of the two image areas. IoU is usually used to detect the relationship between two image areas. For example, by calculating the IoU of the detection target in different frame images of the video data stream, the detection target is tracked.

Related technologies usually require target detection and key point detection of the target object respectively, and tracking the target object based on the outer box image obtained by the target detection, so as to indirectly detect the key points based on the relationship between the outer box image of the target object and the key points. Follow up. The related technology will also detect key points of the target object and generate an outer box image of the target object based on the key points, so as to track the target object based on the generated outer box image. However, the above scheme is greatly affected by changes in the outer box image, resulting in unstable tracking of the target object. At the same time, it is difficult to correspond one-to-one between the outer box image of the target object and the key points, so the tracking effect of the key points is poor. In addition, the above scheme requires target detection of the target object to indirectly achieve tracking of key points, and the calculation overhead is large, making the calculation cost of the scheme high.

In view of this, the present disclosure provides a target tracking method that matches the target to be tracked and the detected target according to the detected position of the key point of the detected target and the predicted position of the key point of the target to be tracked, and updates the target to be tracked according to the matching relationship and the key points of the target to be tracked. Therefore, the present disclosure can directly track targets based on key points, improve the stability of target tracking, and at the same time reduce computational overhead, thereby reducing computational costs.

The implementation of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

Figure 1 is an architectural diagram of a target tracking system 10 according to some embodiments. As shown in Figure 1, the target tracking system 10 includes: a target tracking device 101 and an image acquisition device 102. The target tracking device 101 and the image acquisition device 102 are connected through a communication link. The communication link may be a wired communication link or a wireless communication link, which is not limited by this disclosure.

It should be noted that the target tracking device 101 and the image acquisition device 102 in this disclosure may be one or multiple. For ease of understanding, only one target tracking device 101 and image acquisition device 102 are shown in FIG. 1 .

The image acquisition device 102 is configured to acquire an image to be detected of the target area and send the image to be detected to the target tracking device 101 . Correspondingly, the target tracking device 101 receives the image to be detected sent by the image acquisition device 102.

In a possible implementation, the image acquisition device 102 can acquire an image to be detected of the target area in real time, and send the image to be detected to the target tracking device 101 . The image acquisition device 102 can also acquire the image to be detected of the target area at a preset frequency, and send the video stream data including the image to be detected to the target tracking device 101 .

The image acquisition device 102 in the embodiment of the present disclosure is a device that converts image data into analog signals or digital signals through a photoreceptor, and can be deployed on land, including indoors or outdoors, handheld or vehicle-mounted. It can also be deployed on water (such as ships, etc.). It can also be deployed in the air (such as on airplanes, balloons, satellites, etc.). For example, the image capture device 102 includes a camera, a video camera, and a still camera. The image acquisition device 102 may also be a device with a camera function. For example, the image acquisition device 102 may be a mobile phone, a tablet computer, a notebook computer, a handheld computer, or a wearable device (such as a smart watch, a smart bracelet, or a pedometer) with a camera function. machines, etc.), vehicle-mounted equipment and flight equipment (such as intelligent robots, hot air balloons, drones, airplanes), etc.

Exemplarily, the image acquisition device 102 in the embodiment of the present disclosure may also be an infrared imager or a night vision device, used to acquire image data in dark areas.

Wherein, the target tracking device 101 is configured to obtain an image to be detected at the first moment, and determine the detection position of a key point of the detection target based on the image to be detected.

The target tracking device 101 is further configured to determine the predicted position of the key point of the target to be tracked at the first moment, and to track the target to be tracked and the key points of the target to be tracked based on the detected position and the predicted position.

The detection target is a target detected by the target tracking device 101 from the image to be detected, and the target to be tracked is a target tracked by the target tracking device 101 . Key points are nodes at specific parts corresponding to the target (detection target and/or target to be tracked). The targets in this disclosure may be physical substances that exist objectively in nature, such as people, animals, plants, vehicles, commodities, etc.

Taking the target as a person as an example, Figure 2 is a relationship diagram between targets and key points provided according to some embodiments. As shown in FIG. 2 , the target 201 may be a detection target or a target to be tracked in this disclosure, and the solid point 202 is the key point corresponding to the target 102 . The number and location of key points can be set according to actual conditions, and this disclosure does not limit this.

It should be noted that the number of detected targets and targets to be tracked at the first moment may be 0, or may be one or more.

For example, when the first time is the initial time (for example, the current image to be detected is the first frame image), the number of targets to be tracked at this time is 0. When the target tracking device 101 detects that there are no detection targets and key points of the detection targets in the image to be detected, the number of detection targets is 0. When the target tracking device 101 detects that there are detection targets and key points of the detection targets in the image to be detected, the number of detection targets is one or more.

The position of the key point in this disclosure may be the relative position of the key point in the image to be detected relative to the image to be detected, or it may be the actual position determined based on the relative position of the image to be detected and the area corresponding to the image to be detected.

The target tracking device 101 in the present disclosure can perform the operation of tracking the target in real time, that is, continuously acquiring the image to be detected at the current moment, and tracking the target to be tracked. The target tracking device 101 can also perform non-real-time tracking of the target, that is, track the target to be tracked based on the image data stream in the past period of time.

The target tracking device 101 in the embodiment of the present disclosure may be a server, including:

Processor, the processor can be a general central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more programs for controlling the disclosed solution implemented integrated circuit.

Transceiver, a transceiver can be any device such as a transceiver used to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks, WLAN) etc.

Memory, memory can be read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM) or other types that can store information and instructions. Type of dynamic storage device, it can also be electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disk storage, optical disc Storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or can be used to carry or store desired program code in the form of instructions or data structures and can be used by Any other media accessible by a computer, but not limited to this. The memory can exist independently and be connected to the processor through communication lines. Memory can also be integrated with the processor.

The target tracking device 101 can also be an application program installed on the server that provides a target tracking function, or the target tracking device 101 can be a central processing unit in the server, or the target tracking device 101 can be configured to execute in the server. Control module of target tracking method. The target tracking device 101 may also be coupled to the image acquisition device 102 as the same device.

It should be noted that the various embodiments of the present disclosure can be used for reference or reference to each other. For example, the same or similar steps, method embodiments, system embodiments and device embodiments can be referred to each other without limitation.

Figure 3 is a flow chart of a target tracking method provided according to some embodiments. As shown in Figure 3, the method includes the following steps:

Step 301: The target tracking device acquires the image to be detected at the first moment.

Among them, the image to be detected is an image collected for the target area.

For example, the target area can be an airport waiting area, a road, a scenic spot, a building stair passage, etc.

The first time may be the current time when the target tracking device obtains the image to be detected.

In a possible implementation, the target tracking device can acquire the image to be detected from the image acquisition device in real time, and use the current moment as the first moment.

Alternatively, the first moment may be the acquisition moment of the image to be detected. The collection time is the time when the image collection device collects the image to be detected.

In yet another possible implementation, the target tracking device may obtain video stream data including the image to be detected from the image acquisition device at a preset frequency, and use the acquisition time of the image frame of the image to be detected as the first time. .

Step 302: The target tracking device determines the detection position of the key point of the detection target in the image to be detected.

Among them, the number of detection targets can be 0, or one or more. The detection position of the key point of the detection target may be the relative position of the key point in the image to be detected relative to the image to be detected, or it may be the actual position determined based on the relative position of the image to be detected and the area corresponding to the image to be detected.

In a possible implementation, the target tracking device may determine the identification of key points of the detected target based on the location of the detected target. That is to say, the identification of key points of the same part in each detection target in this disclosure is the same. Therefore, the target tracking device can determine the corresponding relationship between the key points of different targets based on the identification.

It should be noted that the number of key points in this disclosure can be set according to actual conditions. However, when the target tracking device determines the position information of the key points of the detected target, some key points may not be detected due to factors such as occlusion. In this case, the target tracking device can set the position information of the undetected key points as preset values. Set the position value (for example (-1, -1)). This preset position value is used to indicate that the key point has not been detected.

In one possible implementation, the target tracking device inputs the image to be detected into the key point detection model to obtain the key point detection result.

Among them, the key point detection results include the detection target and the key points corresponding to the detection target.

For example, the key point detection result may be a data vector with a data size of N*K*A. Among them, N represents the number of detection targets, K represents the number of key points, and A represents the location information of the key points. When the position information is two-dimensional coordinates, the position information can be expressed as (x, y). For undetected key points, the target tracking device may set the location information of the undetected key points to (-1, -1). The key point detection model can be a top-down key point detection model, such as the HRnet model, and the key point detection model can also be a bottom-up key point detection model, such as the OpenPose model.

Step 303: The target tracking device obtains the position of the key point of the target to be tracked at the second moment.

Among them, the second moment is the moment before the first moment.

It should be noted that the target to be tracked is the target that needs to be tracked in this disclosure. Each target to be tracked has a unique identifier.

The number of targets to be tracked can be 0, one or more. For the number, location, correspondence relationship and other relevant contents of the key points of the target to be tracked, please refer to the description of the key points of the target detected in the above step 302, which will not be described again here.

In a possible implementation, the target tracking device in the present disclosure can perform a target tracking operation at the second moment, thereby obtaining the position of the key point of the target to be tracked at the second moment.

It should be noted that the target tracking method in the present disclosure not only needs to identify the target to be tracked and the key points of the target to be tracked, but also needs to determine the target to be tracked and the detected target and the target to be tracked in continuous images (that is, multiple images collected continuously). The relationship between the tracking target and the key points of the detection target is achieved, thereby achieving the effect of tracking the target to be tracked.

However, when the first time is the initial time, there is no second time at this time, and the number of key points of the target to be tracked obtained by the target tracking device at the second time is 0. The subsequent target tracking operation of the target tracking device is equivalent to performing an initial update operation on the target to be tracked. For example, the target tracking device uses the detection target determined at the initial time as the target to be tracked. Please refer to the subsequent description for specific content and will not be repeated here.

Step 304: The target tracking device predicts the predicted position of the key point of the target to be tracked at the first time based on the position of the key point of the target to be tracked at the second time.

In one possible implementation, the target tracking device can determine the motion state of key points of the target to be tracked.

The target tracking device can predict the predicted position of the key point of the target to be tracked at the first time based on the position and motion state of the key point of the target to be tracked at the second time.

The motion state may include the velocity of the key point, or the motion state may include the velocity and acceleration of the key point, or the motion state may include the velocity and direction of the key point.

Similar to the above process of determining the position of the key point of the target to be tracked at the second moment, the target tracking device may perform a target tracking operation at the second moment to determine the motion state of the key point of the target to be tracked. For specific content, please refer to the subsequent description and will not be repeated here.

It should be noted that the target tracking device can set the predicted position of an undetected key point in the target to be detected as a preset position value, thereby indicating that the key point has not been detected.

Step 305: The target tracking device determines the matching relationship between the detected target and the target to be tracked based on the detected position and the predicted position.

Among them, the matching relationship is used to match the same target among the detected target and the target to be tracked.

For example, the targets to be tracked include target A and target B. The detection targets include target C, target D and target E. Among them, target A and target C are the same target, which means that target A and target C match. If target B and target D are not the same target, it means that target B and target D do not match.

It should be noted that the detection position of the key point of the target to be detected can represent the detection position state of the corresponding target to be detected at the first moment. The predicted position of the key point of the target to be tracked can represent the predicted position state of the target to be tracked at the first moment. Therefore, the target tracking device can determine the matching relationship between the detected target and the target to be tracked based on the detected position state and the predicted position state of the key point.

The matching relationship includes at least one of the following: at least one target to be tracked matches the target to be tracked, at least one target to be tracked does not match the target to be tracked, and at least one target to be tracked does not match the target to be tracked.

Exemplarily, FIG. 4 is a matching relationship diagram between a detected target and a target to be tracked according to some embodiments. As shown in Figure 4, the target 4011 to be tracked matches the detection target 4021, the target 4013 to be tracked matches the detection target 4022, the target 4014 to be tracked matches the detection target 4023, and the target 4012 to be tracked does not match any of the detection targets. Matching, the target to be tracked 4015 does not match any of the detected targets, and the detected target 4024 does not match any of the targets to be tracked.

Step 306: The target tracking device tracks the target to be tracked according to the matching relationship.

In a possible implementation, the target tracking device can update the target identifier and the location of the key point of the target to be tracked at the first moment.

For a target to be tracked that has a matching detection target, the target tracking device may determine that the key point of the target to be tracked is at the first location based on the predicted position of the key point of the target to be tracked and the detection position of the key point of the corresponding matching detection target. moment position.

For a detected target that does not have a matching target to be tracked, the target tracking device can add the detected target to the target to be tracked.

It should be noted that when the first time is the initial time, the number of targets to be tracked at this time is 0, and the target tracking device can determine that there is no matching target to be tracked for each detected target. Therefore, the target tracking device can add each of the detected targets to the targets to be tracked.

In one possible implementation, when the first moment is the initial moment, the target tracking device sets the updated motion state of the key points of the target to be tracked as the initial motion state.

For example, the speed and acceleration of the initial motion state can be 0.

For a target to be tracked that does not have a matching detection target, the target tracking device may determine whether to delete the target from the targets to be tracked based on whether the matching situation of the target within a certain period of time satisfies the first preset condition.

For example, the first preset condition may be that the number of mismatches is less than or equal to the preset tracking threshold. The number of mismatches is the number of consecutive times that the target to be tracked does not have a matching detection target.

In one possible implementation, when the number of detection targets is 0, the target tracking device determines that there is no matching detection target for each target to be tracked.

Combined with the example in step 305 above, as shown in Figure 4, for the target to be tracked 4011, the target to be tracked 4013, and the target to be tracked 4014, the target tracking device can be based on the predicted position of the key point of the target to be tracked 4011 and the detected target 4021. The detection position of the key point determines the position of the key point of the target 4011 to be tracked at the first moment. The target tracking device determines the position of the key point of the target 4013 to be tracked at the first moment based on the predicted position of the key point of the target 4013 to be tracked and the detected position of the key point of the detection target 4022 . The target tracking device determines the position of the key point of the target 4014 to be tracked at the first moment based on the predicted position of the key point of the target 4014 to be tracked and the detected position of the key point of the detection target 4023 .

For the target to be tracked 4012 and the target to be tracked 4015, the target tracking device determines that the matching situation of the target to be tracked 4012 satisfies the first preset condition, and retains the target to be tracked 4012. The target tracking device determines that the matching situation of the target to be tracked 4015 does not meet the first preset condition, that is, the number of unmatched times is greater than the preset tracking threshold, and deletes the target to be tracked 4012.

For the detected target 4024, the target tracking device adds the detected target 4024 to the targets to be tracked.

In a possible implementation, the target tracking device can also send an updated tracking sequence of the target to be tracked to the target device, so that the target device performs relevant identification detection based on the tracking sequence.

The tracking sequence includes the target to be tracked and the key point information of the target to be tracked. For example, the identification of the target to be tracked and the locations of key points of the target to be tracked at different times.

For example, the target device can perform passerby action detection, workshop machine status detection and other related solutions based on the tracking sequence.

In a possible implementation, the target tracking device may determine the motion state of the key points of the target to be tracked based on the position of the key points of the target to be tracked at the first time and the position at the second time.

Based on the above technical solution, the target tracking device in the present disclosure can determine the detection position of the key point of the detection target based on the image to be detected at the first time, and predict the tracking target based on the position of the key point of the target to be tracked at the second time. The predicted position of the key point at the first moment. After that, the target tracking device can determine the matching relationship between the detection target and the target to be tracked based on the detection position of the key point of the target to be detected and the predicted position of the key point of the target to be tracked, and update the target to be tracked and the target to be tracked based on the matching relationship. Track the location of key points of the target to achieve target tracking. Therefore, the target tracking device in the present disclosure can match the detected target and the target to be tracked based on the location information of the key points of the target, and then update the target to be tracked and the key points of the target to be tracked, so that the target to be tracked at different times and The key points of the target to be tracked are related to each other. Compared with the technical solution in the related art that tracks a target based on the target outer frame image determined by target detection, the technical solution in this disclosure first predicts the position of the key point of the target to be tracked at the first moment, and then based on the predicted position and The detection position of the key points of the target is detected for target tracking, which is not affected by changes in the outer frame image, improving the stability and accuracy of target tracking. At the same time, the technical solution in this disclosure directly tracks the target based on key points, without the need for additional detection of the target, and has low computational overhead, reducing the computational cost.

It should be noted that steps 301 to 302 and steps 303 to 304 in the above embodiment are independent of each other. Steps 301 to 302 may be executed before steps 303 to 304, or may be executed after steps 303 to 304, or may be executed in parallel with steps 303 to 304. In FIG. 3 , the target tracking algorithm provided by the present disclosure is only explained by taking step 301 to step 302 before step 303 to step 304 as an example. This disclosure does not limit the execution order of the above steps.

Below, the process of determining the matching relationship between the detection target and the target to be tracked by the target tracking device is introduced.

As a possible embodiment of the present disclosure, as shown in Figure 5 in conjunction with Figure 3, the above step 305 can also be implemented through the following steps 501 to 502.

Step 501: The target tracking device performs a first target operation on each detected target and each target to be tracked, and determines the distance between each detected target and each target to be tracked.

Among them, the number of detection targets is N, and the number of targets to be tracked is M. M and N are positive integers.

In a possible implementation, the first target operation includes:

Step 5011: The target tracking device determines the i-th detected target and the j-th target to be tracked based on the distance between the detected position of the key point in the i-th detected target and the predicted position of the key point of the j-th target to be tracked. The distance between targets.

Among them, i is a positive integer less than or equal to N; j is a positive integer less than or equal to M.

Further, the process of the target tracking device determining the distance between the i-th detection target and the j-th target to be tracked can be implemented through the following steps 501a-step 501b:

Step 501a: The target tracking device may perform a second target operation on each corresponding key point of the i-th detected target and the j-th target to be tracked, and determine the distance between each corresponding key point.

Among them, the identification of the corresponding key points of the i-th detection target and the j-th target to be tracked is the same.

In a possible implementation, the second target operation includes the following steps 5012 to 5013:

Step 5012: The target tracking device determines the first key point and the second key point.

Among them, the first key point is any key point in the i-th detection target. The second key point is the key point corresponding to the first key point in the j-th target to be tracked.

Step 5013: The target tracking device determines the distance between the first key point and the second key point based on the detected position of the first key point and the predicted position of the second key point.

For example, the distance between the first key point and the second key point is calculated by the following formula 1:

Among them, p is the first key point, p′ is the second key point, d(p, p′) represents the distance between the first key point and the second key point, x and y are the position coordinates of the first key point , x′, y′ are the position coordinates of the second key point.

It should be noted that there may be undetected key points among the key points of the detection target and the target to be tracked. At this time, the distance between the undetected key points cannot be determined, so the target tracking device can set the distance between the undetected key points to a preset distance value. The preset distance value is used to indicate that at least one key point among the two corresponding key points has not been detected.

For example, the preset distance value may be -1.

Step 501b: The target tracking device determines the distance between the i-th detection target and the j-th target to be tracked based on the distance between each corresponding key point.

For example, the distance between the i-th detection target and the j-th target to be tracked is calculated by the following formula 2:

Among them, D(i,j) represents the distance between the i-th detection target and the j-th target to be tracked, p _ik represents the detection position of the k-th key point of the i-th detection target, and p′ _jk represents the j-th The predicted position of the k-th key point of the target to be tracked, d(p _ik , p′ _jk ) represents the distance between the k-th key point of the i-th detection target and the k-th key point of the j-th target to be tracked _The distance _of , δ(v _ik ,v′ _jk ) represents the detection situation of the k-th key point of the i-th detection target and the k-th key point of the j-th target to be tracked.

It should be noted that s can represent either the size information of the i-th detected target or the size information of the j-th target to be tracked, or it can also be composed of the size information of the i-th detected target and the j-th target to be tracked. The size information is determined. For example, s is the average value of the size information of the i-th detected target and the j-th size information to be tracked.

The size information can be expressed by the image area of the target, the positive square root of the image area of the target, or the number of pixels occupied by the target.

For example, when at least one of the k-th key point of the i-th detection target and the k-th key point of the j-th target to be tracked is not detected, δ(v _ik ,v′ The value of _jk ) can be 0. In the case where both the k-th key point of the i-th detection target and the k-th key point of the j-th target to be tracked are detected, the value of δ(v _ik ,v′ _jk ) can be 1.

Step 502: The target tracking device determines a matching relationship based on the distance between each detected target and each target to be tracked.

It should be noted that the target tracking device can determine the matching relationship between the target to be tracked and the detected target from an overall perspective based on the distance between each detected target and each target to be tracked. The target tracking device can also determine the matching relationship between the target to be tracked and the detection target from the perspective of the individual target to be tracked based on the distance between any target to be tracked and each detected target. For specific implementation methods, please refer to the subsequent description and will not be repeated here.

In one possible implementation, the target tracking device can determine whether each target to be tracked matches the detection target based on a preset matching algorithm and the distance between each detection target and each target to be tracked.

For example, the preset matching algorithm may be the KM algorithm.

Wherein, for the matching detection target and the target to be tracked, the target tracking device can also determine whether the distance between the detection target and the target to be tracked satisfies the second preset condition. When the distance between the detected target and the target to be tracked does not meet the second preset condition, it is determined that the detected target and the target to be tracked do not match. Through the above technical solution, the target tracking device can further filter the matching targets to be tracked and the detection targets, and exclude the targets to be tracked and the detection targets that do not actually match, thereby improving the accuracy of target tracking.

The second preset condition may be that the distance between the detection target and the target to be tracked is less than or equal to the target distance threshold.

For example, the target distance threshold is 3.5, the targets to be tracked include A1, A2, A3, and A4, and the detected targets include B1, B2, B3, and B4. A1 matches B1, and the distance between A1 and B1 is 3. A2 matches B2, and the distance between A2 and B2 is 4. A3 matches B4, and the distance between A3 and B4 is 3. A4 matches B3, and the distance between A4 and B3 is 2.

The distance between A2 and B2 does not meet the second preset condition. Therefore, the target tracking device determines that A2 and B2 do not match.

Figure 6 is a scene diagram of target tracking provided according to some embodiments. For ease of understanding, only one target is shown in FIG. 6 , and the technical solutions in this disclosure are applicable to target tracking scenarios of one or more targets. Target 601 is in jumping motion. At the second moment, the target 601 in the image a to be detected is in a preparatory action. At the first moment, the target 601 in the image b to be detected is in the process of jumping.

Figure 7 is a schematic diagram of a target tracking in the related art. As shown in Figure 7, in the related art, the outer frame image 701 of the target 601 at the second time is determined based on the image a to be detected at the second time in Figure 6, and the target is determined based on the image b to be detected at the first time in Figure 6 601 The outer box image 702 at the first moment. Calculate the IoU of the outer box image 701 and the outer box image 702, and determine whether the targets corresponding to the two outer box images are the same target based on the IoU. However, since the jumping action of the target 601 causes large changes in the position and size of the outer frame image, it is difficult to accurately track the target 601 in the related technology.

Figure 8 is a schematic diagram of a target tracking provided according to some embodiments of the present disclosure. As shown in Figure 8, the target tracking device in the present disclosure predicts the predicted position of the key point of the target 601 at the first time based on the position of the key point of the target 601 determined at the second time in Figure 6 (i.e., the hollow in Figure 8 point), and determine the detection position of the key point of the target 601 (ie, the solid point in Figure 8) based on the image b to be detected at the first moment in Figure 6. Afterwards, the target tracking device can determine the matching relationship of the target based on the detected position and the predicted position. In addition, even if there are some undetected key points among the obtained key points (for example, the predicted position of the key point at the right leg joint in Figure 8 is not detected), the target tracking method in the present disclosure can still be based on other key points The location information is used to track the target, which will not have too much impact on the target tracking of the disclosure.

Compared with the target tracking methods in the related art, the technical solution in the present disclosure is not affected by changes in the outer frame image, and improves the stability and accuracy of target tracking. At the same time, the technical solution in this disclosure directly tracks the target based on key points, without the need for additional detection of the target, and has low computational overhead, reducing the computational cost.

It should be noted that Figure 6 only takes a single target as an example to illustrate the differences between the tracking principles in this disclosure and those in related technologies. The technical solutions in this disclosure are also applicable to multi-target tracking. For details, please refer to the relevant embodiments. , the scene in Figure 6 does not limit the solution of the present disclosure.

The following describes the process by which the target tracking device determines the matching relationship based on the distance between each detected target and each target to be tracked.

As a possible embodiment of the present disclosure, as shown in Figure 9 in conjunction with Figure 5, the above step 502 can also be implemented through the following steps.

Step 901: The target tracking device determines the weight value of each target to be tracked and the weight value of each detected target.

Among them, the weight value of any target to be tracked is the value of the minimum distance between any target to be tracked and each detected target. The weight value of each detection target is a preset weight value.

It should be noted that the weight value of the target to be tracked is used to represent the expected value of the distance of the target to be tracked, and the weight value of the detection target is used to represent the expected value of the distance of the detection target.

The lower the weight value, the higher the distance requirements of the target to be tracked or the target to be detected. On the contrary, the higher the weight value, the lower the distance requirements of the target to be tracked or detected.

For example, the default weight value is 0, which can be set according to the actual situation, and this disclosure does not limit this.

Step 902: The target tracking device performs a third target operation on each target to be tracked, and determines a detection target matching each target to be tracked.

As shown in Figure 10, the third target operation includes the following steps 1001-1002.

Step 1001: The target tracking device determines the first detected target with the shortest distance from the mth target to be tracked.

Wherein, the sum of the weight values of the first detection target and the m-th target to be tracked is greater than or equal to the distance between the first detection target and the m-th target to be tracked. m is a positive integer.

It should be noted that if the sum of the weight values of the target to be tracked and the target to be detected is greater than or equal to the distance between the target to be tracked and the target to be detected, it means that the distance between the two meets the distance requirements of the two. In the case where the first detection target includes multiple detection targets, the target tracking device selects any one of the detection targets.

For example, the weight value of target A to be tracked is 4, the weight value of target B to be detected is -1, and the distance between target A to be tracked and target B to be detected is 2. Since the sum of the weight values of target A to be tracked and target B to be detected is 3, it is greater than the distance 2 between target A to be tracked and target B to be detected. Therefore, the distance meets the distance requirements between the target A to be tracked and the target B to be detected, that is, the target A to be tracked and the target B to be detected meet the matching conditions in distance. Afterwards, the target tracking device needs to further determine whether the distance is the smallest among the distances that meet the matching conditions, and perform subsequent operations.

In a possible implementation, when the first detection target does not exist among the detection targets, the target tracking device uses the second detection target as the first detection target, and updates the weight value of the mth target to be tracked.

Among them, the difference between the distance and the sum of weight values between the second detected target and the m-th target to be tracked is the smallest.

It should be noted that, in the case where the detection targets include multiple second detection targets, the target tracking device may select any one of the second detection targets as the first detection target. Since there is no detection target whose weight value sum to the mth target to be tracked is greater than or equal to the distance, that is, each detection target and the mth target to be tracked do not meet the matching condition in distance, therefore, the The m target to be tracked needs to reduce its expected distance value in order to match the detection target for the m target to be tracked. In order to ensure that the target to be tracked achieves optimal matching as a whole, the target tracking device uses the second detection target as the first detection target and updates the weight value of the mth target to be tracked.

Exemplarily, the target tracking device updates the weight value of the mth target to be tracked in the following manner: the target tracking device increases the weight value of the mth target to be tracked by the distance and weight between the second detected target and the mth target to be tracked. The difference between the sum of values.

Step 1002: The target tracking device performs a fourth target operation on the mth target to be tracked and the first detection target, and determines the detection target matching the mth target to be tracked.

As shown in Figure 11, the fourth target operation includes the following steps 1101-1104.

Step 1101: The target tracking device determines whether there is a first target to be tracked.

Wherein, the first target to be tracked is a target to be tracked that matches the first detected target and is determined according to the third target operation.

If there is no first target to be tracked, it means that there is currently no matching target to be tracked for the first detected target, so the target tracking device can perform step 1102.

Step 1102: The target tracking device determines that the mth target to be tracked matches the first detected target.

When there is a first target to be tracked, it means that the optimal matching between the mth target to be tracked and the first target to be tracked is the first detection target. That is to say, the target tracking device needs to re-determine the mth target to be tracked. The matching relationship between the tracking target and one of the first targets to be tracked and the detection targets other than the first detection target is such that the matching relationship between the target to be tracked and the detection target achieves optimal matching as a whole. Therefore, the target tracking device can perform step 1103.

Step 1103: The target tracking device determines the first influence value of the mth target to be tracked and the second influence value of the first target to be tracked.

The first influence value is the difference determined based on the mth target to be tracked and the second detected target. The difference is the difference between the distance between a target to be tracked and a detected target and the sum of the weights of a target to be tracked and a detected target. The second detection target is the detection target that has the smallest difference from the mth target to be tracked, except the first detection target. The second influence value is the difference determined based on the first target to be tracked and the third detected target. The third detection target is the detection target that has the smallest difference from the first target to be tracked except the first detection target.

Therefore, the first influence value is used to characterize the minimum degree of influence of the target tracking device in re-determining the detection target that matches the m-th target to be tracked; the second influence value is used to characterize the target tracking device in re-determining the detection target that matches the first target to be tracked. Minimum impact on detection target.

It should be noted that, when the difference between the sum of distances and weight values between multiple detection targets and the m-th target to be tracked is the smallest, the target tracking device selects any one of the detection targets as the second detection target. In the case where the difference between the sum of distances and weight values between multiple detection targets and the first target to be tracked is the smallest, the target tracking device selects any one of the detection targets as the third detection target.

Step 1104: The target tracking device determines the detection target matching the mth target to be tracked based on the first influence value and the second influence value.

If the first influence value is less than the second influence value, the target tracking device performs step 1105.

Step 1105: The target tracking device performs a fourth target operation on the mth target to be tracked and the second detection target, and determines the detection target matching the mth target to be tracked.

It should be noted that, since the first influence value is smaller than the second influence value, the influence of the target tracking device on the overall matching relationship caused by redetermining the matching relationship between the mth target to be tracked and the second detected target is smaller than that of redetermining the matching relationship. The matching relationship between the first target to be tracked and the third detected target has less impact on the overall matching relationship. In order to ensure that the target to be tracked and the detection target achieve optimal matching as a whole, the target tracking device can re-determine the detection target that matches the mth target to be tracked.

In a possible implementation, the target tracking device can also update the weight values of the mth target to be tracked, the first target to be tracked, and the first detected target according to the first influence value.

Exemplarily, the target tracking device increases the weight value of the mth target to be tracked by the first influence value, increases the weight value of the first target to be tracked by the first influence value, and reduces the weight value of the first detected target by the first influence value. value.

If the first influence value is greater than the second influence value, the target tracking device performs step 1106.

Step 1106: The target tracking device determines that the mth target to be tracked matches the first detected target.

In a possible implementation, the target tracking device may also perform a fourth target operation on the first target to be tracked and the third target to be detected, and determine the detection target that matches the first target to be tracked.

Similarly, since the first influence value is greater than the second influence value, the target tracking device re-determines the matching relationship between the m-th target to be tracked and the second detected target, which has a greater impact on the overall matching relationship than re-determining the first The matching relationship between the target to be tracked and the second detected target has a greater impact on the overall matching relationship. In order to ensure that the target to be tracked and the detection target achieve optimal matching as a whole, the target tracking device can re-determine the detection target that matches the first target to be tracked.

In a possible implementation, the target tracking device can also update the weight values of the mth target to be tracked, the first target to be tracked, and the first detected target according to the second influence value.

Exemplarily, the target tracking device increases the weight value of the mth target to be tracked by the second influence value, increases the weight value of the first target to be tracked by the second influence value, and reduces the weight value of the first detected target by the second influence value. value.

In the case where the first influence value is equal to the second influence value, the target tracking device performs step 1105, or the target tracking device performs step 1106.

It should be noted that the target tracking device in the above solution can determine a one-to-one matching relationship between the target to be tracked and the detected target as a whole based on the distance between each detected target and each target to be tracked. In the actual execution process, there is an unequal number of targets to be tracked and detected targets. In order to obtain a one-to-one matching relationship, some targets can be added to a smaller number of targets to make the number of targets to be tracked and detected targets Quantities are equal.

For example, if the number of detected targets is less than the number of targets to be tracked, the target tracking device increases the number of detected targets so that the number of detected targets is the same as the number of targets to be tracked. Among them, the increased distance between the detection target and each target to be tracked can be set to a larger preset distance to avoid affecting the matching relationship.

The value of the preset distance can theoretically be infinite. For example, the preset distance may be 1,000,000.

In addition, the above technical solution takes the target tracking device to determine the detection target that matches the target to be tracked as an example to illustrate the process of the target tracking device determining the matching relationship in the present disclosure. Target tracking can also determine the target to be tracked that matches the detection target through the above technical solution, thereby determining the matching relationship, that is, exchanging the execution object of the target to be tracked with the detection target in the above technical solution. The specific implementation method is the same as the above technical solution. , which will not be described again here.

Based on the above technical solution, the target tracking device in the present disclosure can, when determining the matching relationship between the target to be tracked and the detection target, consider whether the detection target has matched other targets to be tracked, and re-determine based on the current target to be tracked. The degree of impact that the matching relationship with the detection target has on the overall matching, and the degree of impact that the matched target to be tracked has on the overall matching by redetermining the matching relationship with the detection target. Adjust the relationship between the target to be tracked and the detection target at this time. Matching relationship, so that the matching relationship between the target to be tracked and the target to be detected achieves the overall optimal matching. Therefore, the target tracking device can determine the optimal matching relationship between the target to be tracked and the detected target from an overall perspective, thereby improving the stability and accuracy of target tracking.

For example, the distance between the target to be tracked and the detected target is as shown in Table 1 below:

Table 1 The distance between the target to be tracked and the target to be detected

Among them, the targets to be tracked include A1, A2, A3, and A4, and the detection targets include B1, B2, B3, and B4. The distance between A1 and B1 is 3, the distance between A1 and B2 is 4, the distance between A1 and B3 is 6, the distance between A1 and B4 is 4, and so on.

Taking the above solution as an example, the target tracking device determines the weight value of the target to be tracked, as shown in Table 2 below:

Table 2 Weight values of targets to be tracked

The weight value of the detection target is shown in Table 3 below:

Table 3 Weight values of detection targets

1. The target tracking device determines the matching relationship between A1 and the detection target.

The sum of the weight values of A1 and B1 is 3, which is equal to the distance between A1 and B1, and the distance between A1 and B1 is the shortest. Therefore, the target tracking device takes B1 as the first detection target. At the same time, B1 does not match other targets to be tracked. Therefore, the target tracking device determines that A1 matches B1.

The matching relationship at this time is: A1 matches B1.

2. The target tracking device determines the matching relationship between A2 and the detection target.

The sum of the weight values of A2 and B4 is 3, which is equal to the distance between A2 and B4, and the distance between A2 and B4 is the shortest. Therefore, the target tracking device takes B4 as the first detection target. At the same time, B4 does not match other targets to be tracked. Therefore, the target tracking device determines that A2 matches B4.

The matching relationships at this time are: A1 matches B1, and A2 matches B4.

3. The target tracking device determines the matching relationship between A3 and the detection target.

The sum of the weight values of A3 and B4 is 3, which is equal to the distance between A3 and B4, and the distance between A3 and B4 is the shortest. Therefore, the target tracking device takes B4 as the first detection target. However, since A2 and B4 have been matched, the target tracking device needs to determine the difference between the sum of the weight values of A3 and B1 and the distance, the difference between the sum of the weight values of A3 and B2 and the distance, and the difference between the sum of the weight values of A3 and B3. The difference between the sum and the distance, and the difference between the sum of the weight values of A2 and B1 and the distance, the difference between the sum of the weight values of A2 and B2 and the distance, and the difference between the sum of the weight values of A2 and B3 and the distance. value.

The results are shown in Table 4 below:

Table 4: The difference between the sum of weight values and the distance between A2, A3 and the detection target

Among them, the first influence value and the second influence value are both 1, the second detection target is B3, and the third detection target is B2. The target tracking device may execute step 9027 or step 9028. Here, the target tracking device executes step 9028 as an example.

The target tracking device determines that A3 and B4 match, and performs a fourth target operation on A2 and B2.

In addition, the target tracking device can also update the weight values of A2, A3 and B4 according to the second influence value.

The updated weight values of the targets to be tracked are shown in Table 5 below:

Table 5: Updated weight values of targets to be tracked

The updated weight values of the detection targets are shown in Table 6 below:

Table 6: Updated weight values of detection targets

Since B2 does not temporarily match other targets to be tracked, the target tracking device determines that A2 matches B2.

The matching relationships at this time are: A1 matches B1, A2 matches B2, and A3 matches B4.

4. The target tracking device determines the matching relationship between A4 and the detection target.

The sum of the weight values of A4 and B3 is equal to the distance between A4 and B3, and the distance between A4 and B3 is the shortest. At the same time, B3 does not match other targets to be tracked, so the target tracking device determines that A4 matches B3.

The matching relationships at this time are: A1 matches B1, A2 matches B2, A3 matches B4, and A4 matches B3.

At this point, the target tracking device determines the detection target matching each target to be tracked.

As another possible embodiment of the present disclosure, as shown in Figure 12 in conjunction with Figure 5, the above step 502 can also be implemented through the following steps.

Step 1201: The target tracking device determines a list of targets to be tracked and a list of detected targets.

The list of targets to be tracked includes each target to be tracked, and the list of detected targets includes each target to be detected.

Step 1202: The target tracking device performs the fifth target operation on the targets to be tracked and the detected targets in the target to be tracked list and the target to be detected list, until there is no target to be tracked in the target list to be tracked or there is no detection target in the target target list to be detected, determine Detected targets that match each target to be tracked.

It should be noted that during the actual execution process, the number of targets to be tracked and detected targets is not equal, so in the end there will be some targets to be tracked or some targets to be detected. The target tracking device may determine that there is no matching detection target for the part of the target to be tracked, or determine that there is no matching target to be tracked for the part of the detection target.

As shown in Figure 13, the fifth target operation includes the following steps 1301-1304.

Step 1301: The target tracking device determines the target distance between each target to be tracked in the current target list to be tracked and each target to be tracked in the current detected target list.

Step 1302: The target tracking device determines the minimum distance among the target distances, and the second target to be tracked and the fourth detection target corresponding to the minimum distance.

Wherein, the distance between the second target to be tracked and the fourth detection target is the minimum distance.

It should be noted that when there are multiple distances with the same minimum value, the target tracking device can select any one of the distances as the minimum distance, and use the target to be tracked corresponding to the minimum distance as the second target to be tracked. The corresponding detection target is used as the fourth detection target.

Step 1303: The target tracking device determines the matching relationship between the second target to be tracked and the fourth detected target.

Since the distance between the second target to be tracked and the fourth detected target is the minimum distance, the target tracking device can first determine that the second target to be tracked matches the fourth detected target, thereby obtaining the best value of the second target to be tracked. Excellent matching relationship.

Step 1304: The target tracking device deletes the second target to be tracked from the current list of targets to be tracked, and deletes the fourth detected target from the current list of detected targets.

That is to say, when the target tracking device determines the matching relationship between the subsequent target to be tracked and the detection target, it does not need to consider the distance between the target to be tracked and other detection targets that have already determined the matching relationship, nor does it need to consider the already determined matching relationship. The distance between the detection target and other targets to be tracked is reduced, thereby reducing the calculation amount of the target tracking device when determining the matching relationship.

Similarly, the above technical solution takes the target tracking device to determine the detection target that matches the target to be tracked as an example to explain the process of the target tracking device determining the matching relationship in the present disclosure. Target tracking can also determine the target to be tracked that matches the detection target through the above technical solution, thereby determining the matching relationship, that is, exchanging the execution object of the target to be tracked with the detection target in the above technical solution. The specific implementation method is the same as the above technical solution. , which will not be described again here.

Based on the above technical solution, after determining that the second target to be tracked and the fourth detected target with the smallest distance in the target list to be tracked and the list of detected targets match, the target tracking device in the present disclosure can match the second target to be tracked with the third target to be tracked. Four detected targets are removed from the list to determine the matching relationship between the remaining targets to be tracked and the detected targets. Therefore, the target tracking device can determine whether each target to be tracked matches the detection target from the perspective of the target individual to be tracked, further reducing the computational cost of determining the matching relationship by the target tracking device, thereby improving the target tracking efficiency.

Combined with the example in Table 1 above, the target tracking device determines that the list of targets to be tracked includes: A1, A2, A3, and A4, and the list of detected targets includes: B1, B2, B3, and B4. Among them, the distance between A4 and B3 and the distance between A4 and B4 are 2, which are both minimum values. Therefore, the target tracking device can select A4 and B3 to match, and delete A4 and B3 from the to-be-tracked target list and the detected target list. At this time, the target tracking device determines that the list of targets to be tracked includes: A1, A2, and A3, and the list of detected targets includes: B1, B2, and B4.

In the deleted target list to be tracked and the detected target list, the distances between A1 and B1, A2 and B4, and the distance between A3 and B4 are 3, which are all minimum values. Therefore, the target tracking device can select A2 and B4 to match, and delete A2 and B4 from the to-be-tracked target list and the detected target list. At this time, the target tracking device determines that the list of targets to be tracked includes: A1 and A3, and the list of detected targets includes: B1 and B2.

In the deleted target list to be tracked and the detected target list, the distance between A1 and B1 is the minimum value 3. Therefore, the target tracking device can select A1 and B1 to match, and delete A1 and B1 from the to-be-tracked target list and the detected target list. At this time, the target tracking device determines that the list of targets to be tracked includes: A3, and the list of detected targets includes: B2.

In the deleted target list and detection target list, the distance between A3 and B2 is the minimum value 5. Therefore, the target tracking device can select A3 and B2 to match, and delete A3 and B2 from the to-be-tracked target list and the detected target list. At this time, the target tracking device determines that there is no target to be tracked in the list of targets to be tracked, and there is no detection target in the list of detection targets.

Finally, the matching relationships determined by the target tracking device are: A1 matches B1, A2 matches B4, A3 matches B2, and A4 matches B3.

Below, the process of the target tracking device updating the positions of the target to be tracked and the key points of the target to be tracked based on the matching relationship is introduced.

As a possible embodiment of the present disclosure, as shown in Figure 14 in conjunction with Figure 3, the above step 306 can also be implemented through the following steps.

Among them, for the target to be tracked that has a matching detection target, the above step 306 can be implemented through the following steps 1401 to 1402:

Step 1401: The target tracking device determines the third target to be tracked.

Wherein, each of the third targets to be tracked has a matching detection target.

It should be noted that the third target to be tracked is a target that successfully matches the detected target among the targets to be tracked. Therefore, for the third target to be tracked, the target tracking device can associate the third target to be tracked with the matching detection target, thereby achieving the technical effect of target tracking.

Step 1402: The target tracking device updates the position of the key point of the third target to be tracked based on the detected position of the key point of the detected target that matches the third target to be tracked and the predicted position of the key point of the third target to be tracked.

For example, the target tracking device can correct the predicted position and the detected position according to the Kalman filtering algorithm to determine the position of the key point of the third target to be tracked at the first moment.

It should be noted that the third target to be tracked may have undetected key points, and the matching detection target may also have undetected key points. As shown in Figure 15, the left side is the third target to be tracked, and the right side is the matching detection target. Among them,

key points

1501 and 1504 are detected key points of the third target to be tracked (ie, the solid points in Figure 15).

Key points

1502 and 1503 are undetected key points of the third target to be tracked (ie, hollow points in Figure 15).

Key points

1505 and 1506 are detected key points of the detection target.

Key points

1507 and 1508 are undetected key points of the detection target. For ease of understanding, only one target to be tracked is shown in Figure 15 .

When the key point of the third target to be tracked has been detected, and the corresponding key point of the matching detection target has been detected (for example, key point 1501 and key point 1505 in Figure 15), the target tracking device The position of the key point of the third target to be tracked may be updated based on the detected position of the key point of the detected target that matches the third target to be tracked and the predicted position of the key point of the third target to be tracked.

When the key point of the third target to be tracked has been detected, and the corresponding key point of the matching detection target has not been detected (for example, key point 1504 and key point 1508 in Figure 15), the target tracking device The predicted position of the key point of the third target to be tracked or the position at the second time can be used as the position of the key point at the first time.

In the case where the key point of the third target to be tracked has not been detected and the corresponding key point of the matching detection target has been detected (for example, key point 1502 and key point 1506 in Figure 15), the target tracking device The detection position of the corresponding key point of the matching detection target may be used as the position of the undetected key point at the first moment.

When the key point of the third target to be tracked has not been detected, and the corresponding key point of the matching detection target has not been detected (for example, key point 1503 and key point 1507 in Figure 15), the target tracking device The position information of an undetected key point can be set to a preset position value to indicate that the key point has not been detected.

For a detected target that does not have a matching target to be tracked, the above step 306 can be implemented through the following steps 1403 to 1404:

Step 1403: The target tracking device determines the fifth detection target.

Wherein, each detection target in the fifth detection target does not have a matching target to be tracked.

Step 1404: The target tracking device adds a fifth detection target to the targets to be tracked.

It should be noted that a detected target for which there is no matching target to be tracked may be a new target that appears in the target area at the first moment. Therefore, the target tracking device needs to add it to the target to be tracked and subsequently track it.

In one possible implementation, step 1404 includes: the target tracking device uses the fifth detected target as a newly added target to be tracked at the first moment, and uses the key points of the fifth detected target as the key points of the newly added target to be tracked.

For a target to be tracked that does not have a matching detection target, the above step 306 can be implemented through the following steps 1405 to 1408:

Step 1405: The target tracking device determines the fourth target to be tracked and the number of mismatches between the fourth target to be tracked.

Wherein, each of the fourth targets to be tracked does not have a matching detection target, and the number of mismatches is the number of consecutive times that the target to be tracked does not have a matching detection target.

When there is no matching detection target for the target to be tracked, the target tracking device sets the number of mismatches for the target to be tracked plus one.

When the target to be tracked has a matching detection target, the target tracking device clears the number of mismatches of the target to be tracked.

For example, the target tracking device may set a matching counter for each target to be tracked. The match counter is used to count the number of mismatches for the target to be tracked.

Step 1406: The target tracking device determines whether the number of mismatches is greater than the preset tracking threshold.

The preset tracking threshold can be set according to actual conditions, and this disclosure does not limit this.

It should be noted that when the number of mismatches is greater than the preset tracking threshold, it means that the fourth target to be tracked has failed to track multiple times, and the target tracking device determines that the fourth target to be tracked has disappeared from the target area. The target tracking device may perform step 1407.

Step 1407: The target tracking device deletes the fourth target to be tracked from the targets to be tracked.

If the number of mismatches is less than or equal to the preset tracking threshold, it means that the fourth target to be tracked may fail to match due to other factors. The target tracking device may perform step 1408.

Step 1408: The target tracking device uses the predicted position of the key point of the fourth target to be tracked or the position of the key point of the fourth target to be tracked at the second time as the updated position of the key point of the fourth target to be tracked.

It should be noted that for the undetected key points in the fourth target to be tracked, the target tracking device can set the position information of the undetected key points as a preset position value to indicate that the key points have not been detected. detected.

Based on the above technical solution, the target tracking device in the present disclosure can divide the target to be tracked and the detection target into three categories according to the matching relationship, that is, the target to be tracked has a matching detection target and the target to be tracked does not have a matching detection target. The detection target of the target and the target to be tracked for which there is no matching detection target. At the same time, the target tracking device performs corresponding update operations based on the above three types of targets, effectively improving the stability and accuracy of target tracking.

It should be noted that in the above embodiment, steps 1401 to 1402, steps 1403 to 1404, and steps 1405 to 1408 are independent of each other. Steps 1401 to 1402, steps 1403 to 1404, and steps 1405 to 1408 may be executed in any combination of execution order, or may be executed in parallel. This disclosure does not limit the execution order of the above steps.

The following is an introduction to the process in which the target tracking device updates the positions of the target to be tracked and the key points of the target to be tracked based on the matching relationship.

As another possible embodiment of the present disclosure, as shown in Figure 16 in conjunction with Figure 3, in addition to the above steps 1401 to 1408, the above step 306 can also be implemented through the following steps.

Step 1601: The target tracking device determines whether there is a matching detection target for the fifth target to be tracked.

Wherein, the fifth target to be tracked is any target among the targets to be tracked.

If there is a matching detection target for the fifth target to be tracked, the target tracking device executes step 1602.

Step 1602: The target tracking device updates the position of the key point of the fifth target to be tracked based on the matching detected position of the key point of the detected target and the predicted position of the key point of the fifth target to be tracked.

For relevant implementation methods, please refer to the description in step 902, which will not be described again here.

In the case where there is no matching detected target for the fifth target to be tracked, the target tracking device executes step 1603.

Step 1603: The target tracking device determines whether the number of mismatches of the fifth target to be tracked is greater than a preset tracking threshold.

Among them, the number of mismatches is the number of consecutive times that the target to be tracked does not have a matching detection target.

In the case where the number of mismatches of the fifth target to be tracked is greater than the preset tracking threshold, the target tracking device may perform step 1604.

Step 1604: The target tracking device deletes the fifth target to be tracked from the targets to be tracked.

In the case where the number of mismatches of the fifth target to be tracked is less than or equal to the preset tracking threshold, the target tracking device may perform step 1605.

Step 1605: The target tracking device uses the predicted position of the key point of the fifth target to be tracked or the position of the key point of the fifth target to be tracked at the second time as the updated position of the key point of the fifth target to be tracked.

It should be noted that for the undetected key points in the fifth target to be tracked, the target tracking device can set the position information of the undetected key points as a preset position value to indicate that the key points have not been detected. detected.

Step 1606: The target tracking device determines whether there is a matching target to be tracked for the sixth detected target.

Wherein, the sixth detection target is any one of the detection targets.

In the case where the sixth detected target does not have a matching target to be tracked, the target tracking device may perform step 1607.

Step 1607: The target tracking device adds a sixth detection target to the targets to be tracked.

For relevant implementation methods, please refer to the description in step 1404, which will not be described again here.

Based on the above technical solution, the target tracking device in the present disclosure can determine the matching relationship between each target to be tracked and each detected target, and perform corresponding update operations for each target to be tracked and each detected target based on the matching relationship. , improving the stability and accuracy of target tracking.

It should be noted that step 1601 and step 1606 in the above embodiment are independent of each other. This disclosure does not limit the execution order of the above steps.

Embodiments of the present disclosure can divide the target tracking device into functional modules or functional units according to the above method examples. For example, each functional module or functional unit can be divided corresponding to each function, or two or more functions can be integrated into one in the processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules or functional units. Among them, the division of modules or units in the embodiments of the present disclosure is schematic and is only a logical function division. In actual implementation, there may be other division methods.

As shown in Figure 17, it is a structural diagram of a target tracking device 170 provided according to some embodiments. The device includes: a processing unit 1701 and an acquisition unit 1702.

The acquisition unit 1702 is configured to acquire the image to be detected at the first moment.

The processing unit 1701 is configured to determine the detection position of the key point of the detection target in the image to be detected.

The obtaining unit 1702 is also configured to obtain the position of the key point of the target to be tracked at the second moment. The second moment is the moment before the first moment.

The processing unit 1701 is also configured to predict the predicted position of the key point of the target to be tracked at the first time based on the position of the key point of the target to be tracked at the second time.

The processing unit 1701 is also configured to determine the matching relationship between the detected target and the target to be tracked based on the detected position and the predicted position. The matching relationship is used to match the same target among the detected target and the target to be tracked.

The processing unit 1701 is also configured to track the target to be tracked according to the matching relationship.

In some embodiments, the processing unit 1701 is configured to perform a first target operation on each detected target and each target to be tracked, and determine the distance between each detected target and each target to be tracked; the first target operation It includes: determining the distance between the i-th detection target and the j-th target to be tracked based on the distance between the detection position of the key point in the i-th detection target and the predicted position of the key point of the j-th target to be tracked. distance; where i is a positive integer less than or equal to N; j is a positive integer less than or equal to M; the processing unit 1701 is also configured to determine a match based on the distance between each detected target and each target to be tracked relation.

In some embodiments, the processing unit 1701 is configured to perform a second target operation on each corresponding key point of the i-th detected target and the j-th target to be tracked, and determine the distance between each corresponding key point; The second target operation includes: determining the first key point and the second key point; determining the distance between the first key point and the second key point based on the detected position of the first key point and the predicted position of the second key point; The point is any key point in the i-th detected target; the second key point is the key point corresponding to the first key point in the j-th target to be tracked; the processing unit 1701 is also configured to determine the key point according to each corresponding key point. The distance between them determines the distance between the i-th detection target and the j-th target to be tracked.

In some embodiments, the matching relationship includes at least one of the following: at least one target to be tracked matches the detected target, at least one target to be tracked does not match the detected target, and at least one detected target does not match the target to be tracked. At least one of the matches.

In some embodiments, the processing unit 1701 is configured to determine the weight value of each target to be tracked and the weight value of each detected target; wherein the weight value of any target to be tracked is the weight value of any target to be tracked and each target, respectively. The value of the minimum distance among the distances of the detection targets; the weight value of each detection target is the preset weight value; perform the third target operation on each target to be tracked, and determine the detection target matching each target to be tracked; third The target operation includes: determining the first detected target with the shortest distance from the mth target to be tracked; wherein the sum of the weight values of the first detected target and the mth target to be tracked is greater than or equal to the first detected target and the mth target to be tracked. The distance of the tracking target; m is a positive integer; perform the fourth target operation on the mth target to be tracked and the first detection target to determine the detection target that matches the mth target to be tracked; the fourth target operation includes: determining whether there is The first target to be tracked; the first target to be tracked is the target to be tracked that matches the first detected target determined according to the third target operation; in the absence of the first target to be tracked, determine the mth target to be tracked and the The first detected target matches; in the case where the first target to be tracked exists, determine the first influence value of the mth target to be tracked, and the second influence value of the first target to be tracked; wherein the first influence value is based on The difference between the mth target to be tracked and the second detection target; the difference is the difference between the distance between a target to be tracked and a detection target and the sum of the weights of a target to be tracked and a detection target; the second The detection target is the detection target that has the smallest difference with the mth target to be tracked except for the first detection target; the second influence value is the difference determined based on the first target to be tracked and the third detection target; the third detection target Except for the first detection target, it is the detection target with the smallest difference determined from the first target to be tracked; based on the first influence value and the second influence value, the detection target matching the mth target to be tracked is determined.

In some embodiments, the processing unit 1701 is configured to perform a fourth target operation on the m-th target to be tracked and the second detected target when the first influence value is less than the second influence value, and determine the relationship between the m-th target and the second detected target. The target to be tracked matches the detected target; when the first influence value is greater than the second influence value, it is determined that the mth target to be tracked matches the first detected target.

In some embodiments, the processing unit 1701 is configured to perform a fourth target operation on the first target to be tracked and the third detected target when the first influence value is greater than the second influence value, and determine the relationship between the first target to be tracked and the third target to be tracked. Target matching detection target.

In some embodiments, the processing unit 1701 is configured to update the mth target to be tracked, the first target to be tracked, and the first detection target according to the first influence value when the first influence value is less than the second influence value. weight value; when the first influence value is greater than the second influence value, the weight values of the mth target to be tracked, the first target to be tracked and the first detected target are updated according to the second influence value.

In some embodiments, the processing unit 1701 is configured to increase the weight value of the m-th target to be tracked by a first influence value, increase the weight value of the first target to be tracked by a first influence value, and increase the weight value of the first detected target by a first influence value. The weight value is reduced by the first influence value; the processing unit 1701 is also configured to increase the weight value of the m-th target to be tracked by the second influence value, increase the weight value of the first target to be tracked by the second influence value, and increase the weight value of the first target to be tracked by the second influence value. The weight value of the detection target is reduced by the second influence value.

In some embodiments, the processing unit 1701 is configured to determine a list of targets to be tracked and a list of detected targets; wherein the list of targets to be tracked includes each target to be tracked, and the list of detected targets includes each detected target; and the list of targets to be tracked and The target to be tracked and the detection target in the detection target list iteratively perform the fifth target operation until there is no target to be tracked in the target list to be tracked or there is no detection target in the detection target list, and the matching relationship is determined based on the execution result of the fifth target operation. ; The fifth target operation includes: determining the target distance between each target to be tracked in the current target list to be tracked and each target to be tracked in the current detected target list; determining the minimum distance in the target distance, and the minimum distance corresponding The second target to be tracked and the fourth target to be detected; the distance between the second target to be tracked and the fourth target to be detected is the minimum distance; determine the matching relationship between the second target to be tracked and the fourth target to be detected; from the current target to be tracked The second target to be tracked is deleted from the list, and the fourth detection target is deleted from the current detection target list.

In some embodiments, the processing unit 1701 is configured to determine a third target to be tracked; each of the third targets to be tracked has a matching detected target; the processing unit 1701 is further configured to determine the third target to be tracked according to the The detected position of the key point of the detected target to be tracked matches the predicted position of the key point of the third target to be tracked, and the position of the key point of the third target to be tracked is updated.

In some embodiments, the processing unit 1701 is configured to determine the fifth detection target; each detection target in the fifth detection target does not have a matching target to be tracked; the processing unit 1701 is also configured to determine among the targets to be tracked Added fifth detection target.

In some embodiments, the processing unit 1701 is configured to determine the fourth target to be tracked and the number of mismatches for the fourth target to be tracked; there is no matching detection target for each target to be tracked in the fourth target to be tracked; no The number of matches is the number of times that the target to be tracked has no matching detection target continuously; the processing unit 1701 is also configured to delete the fourth target to be tracked from the targets to be tracked when the number of mismatches is greater than the preset tracking threshold.

In some embodiments, the processing unit 1701 is configured to, when the number of mismatches is less than or equal to the preset tracking threshold, change the predicted position of the key point of the fourth target to be tracked or the key point of the fourth target to be tracked in The position at the second moment is used as the updated position of the key point of the fourth target to be tracked.

In some embodiments, the processing unit 1701 is configured to, in the case where a matching detection target exists for the fifth target to be tracked, based on the detection position of the key point of the matching detection target and the prediction of the key point of the fifth target to be tracked. The position updates the position of the key point of the fifth target to be tracked; the fifth target to be tracked is any one of the targets to be tracked; the processing unit 1701 is also configured to have no matching detection target in the fifth target to be tracked, and If the number of mismatches for the fifth target to be tracked is greater than the preset tracking threshold, the fifth target to be tracked is deleted from the targets to be tracked; the number of mismatches is the number of consecutive times that the target to be tracked does not have a matching detection target; the processing unit 1701 , is also configured to add a sixth detection target to the target to be tracked when the sixth detection target does not have a matching target to be tracked; the sixth detection target is any target among the detection targets.

In some embodiments, the processing unit 1701 is configured to, when the fifth target to be tracked does not have a matching detection target, and the number of mismatches of the fifth target to be tracked is less than or equal to the preset tracking threshold, The predicted position of the key point of the target to be tracked or the position of the fifth target to be tracked at the second moment is used as the updated position of the key point of the fifth target to be tracked.

In some embodiments, the processing unit 1701 is configured to determine the motion state of the key points of the target to be tracked; the processing unit 1701 is also configured to predict based on the position and motion state of the key points of the target to be tracked at the second moment. The predicted position of the key point of the target to be tracked at the first moment.

In some embodiments, the processing unit 1701 is configured to determine that there is no matching detection target for each target to be tracked when the number of detection targets is 0.

In some embodiments, the processing unit 1701 is configured to determine that there is no matching target to be tracked for each detected target when the number of targets to be tracked is 0.

When implemented by hardware, the acquisition unit 1702 in the embodiment of the present disclosure can be integrated on the communication interface, and the processing unit 1701 can be integrated on the processor. The specific implementation is shown in Figure 18.

Figure 18 shows another possible structural diagram of the target tracking device involved in the above embodiment. The target tracking device 180 includes: a processor 1802 and a communication interface 1803. The processor 1802 is configured to control and manage the actions of the target tracking device 180, for example, perform the steps performed by the processing unit 1701 described above, and/or be configured to perform other processes of the technology described herein. The communication interface 1803 is configured to support communication between the target tracking device 180 and other network entities, for example, performing the steps performed by the acquisition unit 1702 described above. Target tracking device 180 may also include memory 1801 and bus 1804 , memory 1801 configured to store program code and data for target tracking device 180 .

Among them, the memory 1801 can be the memory in the target tracking device 180, etc. The memory can include volatile memory, such as random access memory; the memory can also include non-volatile memory, such as read-only memory, flash memory, Hard disk or solid state drive; the memory may also include a combination of the above types of memory.

The processor 1802 described above may implement or execute various exemplary logical blocks, modules and circuits described in conjunction with the present disclosure. The processor may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various illustrative logical blocks, modules and circuits described in connection with this disclosure. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

The bus 1804 may be an Extended Industry Standard Architecture (EISA) bus or the like. Bus 1804 can be divided into address bus, data bus, control bus, etc. For ease of presentation, only one thick line is used in Figure 18, but it does not mean that there is only one bus or one type of bus.

The target tracking device 180 in Figure 18 can also be a chip. The chip includes one or more (including two) processors 1802 and communication interfaces 1803.

In some embodiments, the chip also includes memory 1801, which may include read-only memory and random access memory, and provides operating instructions and data to the processor 1802. Part of memory 1801 may also include non-volatile random access memory (NVRAM).

In some embodiments, memory 1801 stores elements, execution modules, or data structures, or subsets thereof, or extended sets thereof.

In the embodiment of the present disclosure, the corresponding operation is performed by calling the operation instructions stored in the memory 1801 (the operation instructions may be stored in the operating system).

Through the above description of the embodiments, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be allocated as needed. It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. For the specific working processes of the systems, devices and units described above, reference can be made to the corresponding processes in the foregoing method embodiments, which will not be described again here.

Some embodiments of the present disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) having computer program instructions stored therein, and the computer program instructions are stored in a computer (e.g., a target When running on the tracking device), the computer is caused to execute the target tracking method as described in any of the above embodiments.

Exemplarily, the above-mentioned computer-readable storage media may include, but are not limited to: magnetic storage devices (such as hard disks, floppy disks or tapes, etc.), optical disks (such as CD (Compact Disk, compressed disk), DVD (Digital Versatile Disk, etc.) Digital versatile disk), etc.), smart cards and flash memory devices (e.g., EPROM (Erasable Programmable Read-Only Memory, Erasable Programmable Read-Only Memory), cards, sticks or key drives, etc.). The various computer-readable storage media described in this disclosure may represent one or more devices and/or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing and/or carrying instructions and/or data.

Some embodiments of the present disclosure also provide a computer program product, for example, the computer program product is stored on a non-transitory computer-readable storage medium. The computer program product includes computer program instructions. When the computer program instructions are executed on a computer (eg, a target tracking device), the computer program instructions cause the computer to perform the target tracking method as described in the above embodiments.

Some embodiments of the present disclosure also provide a computer program. When the computer program is executed on a computer (eg, a target tracking device), the computer program causes the computer to perform the target tracking method as described in the above embodiments.

The beneficial effects of the above computer-readable storage medium, computer program product and computer program are the same as the beneficial effects of the target tracking method described in some of the above embodiments, and will not be described again here.

In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in various embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions that come to mind within the technical scope disclosed by the present disclosure by any person familiar with the technical field should be covered. within the scope of this disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

A target tracking method including:

Obtain the image to be detected at the first moment;

Determine the detection position of the key point of the detection target in the image to be detected;

Obtain the position of the key point of the target to be tracked at a second moment; the second moment is a moment before the first moment;

Predict the predicted position of the key point of the target to be tracked at the first time according to the position of the key point of the target to be tracked at the second time;

According to the detected position and the predicted position, a matching relationship between the detected target and the target to be tracked is determined; the matching relationship is used to match the same target among the detected target and the target to be tracked;

According to the matching relationship, the target to be tracked is tracked.
The method according to claim 1, wherein the number of detected targets is N, and the number of targets to be tracked is M; M and N are positive integers; Position, determine the matching relationship between the detected target and the target to be tracked, including:

Perform a first target operation on each detected target and each target to be tracked, and determine the distance between each detected target and each target to be tracked; the first target operation includes: according to the i-th detection The distance between the detected position of the key point in the target and the predicted position of the key point of the j-th target to be tracked determines the distance between the i-th detected target and the j-th target to be tracked; where, i is a positive integer less than or equal to N; j is a positive integer less than or equal to M;

The matching relationship is determined based on the distance between each detected target and each target to be tracked.
The method according to claim 2, wherein the i-th is determined based on the distance between the detected position of the key point in the i-th detected target and the predicted position of the key point of the j-th target to be tracked. The distance between the detected target and the jth target to be tracked includes:

Perform a second target operation on each corresponding key point of the i-th detection target and the j-th target to be tracked to determine the distance between each corresponding key point; the second target operation includes: Determine the first key point and the second key point; determine the distance between the first key point and the second key point according to the detected position of the first key point and the predicted position of the second key point; The first key point is any key point in the i-th detection target; the second key point is the key point corresponding to the first key point in the j-th target to be tracked;

According to the distance between each corresponding key point, the distance between the i-th detection target and the j-th target to be tracked is determined.
The method according to claim 2 or 3, wherein the distance between the i-th detection target and the j-th target to be tracked is calculated by the following formula:

Among them, D(i,j) represents the distance between the i-th detection target and the j-th target to be tracked, p ik represents the detection position of the k-th key point of the i-th detection target, p′ jk represents the predicted position of the k-th key point of the j-th target to be tracked, and d(p ik , p′ jk ) represents the k-th key point of the i-th detection target and the j-th key point of the j-th target to be tracked. The distance between k key points, s represents the size information of the target, v ik represents whether the k-th key point of the i-th detection target has been detected, v′ jk represents the k-th key of the j-th target to be tracked Whether the point is detected, δ(v ik ,v′ jk ) represents the detection situation of the k-th key point of the i-th detection target and the k-th key point of the j-th target to be tracked.
The method according to any one of claims 2 to 4, wherein the matching relationship includes at least one of the following: at least one target to be tracked matches the detection target, and at least one target to be tracked does not match the detection target. , and there is at least one detection target that does not match the target to be tracked.
The method according to any one of claims 2 to 5, wherein determining the matching relationship based on the distance between each detected target and each to-be-tracked target includes:

Determine the weight value of each target to be tracked and the weight value of each detection target; wherein the weight value of any target to be tracked is the distance between any target to be tracked and each detection target. The value of the minimum distance in; the weight value of each detection target is a preset weight value;

Perform a third target operation on each target to be tracked, and determine a detection target matching each target to be tracked;

The third target operation includes: determining the first detection target with the shortest distance from the mth target to be tracked; wherein the sum of the weight values of the first detection target and the mth target to be tracked is greater than or equal to The distance between the first detection target and the m-th target to be tracked; m is a positive integer;

Perform a fourth target operation on the mth target to be tracked and the first detection target to determine a detection target matching the mth target to be tracked;

The fourth target operation includes: determining whether there is a first target to be tracked; the first target to be tracked is a target to be tracked that matches the first detected target determined according to the third target operation;

In the absence of the first target to be tracked, determine that the mth target to be tracked matches the first detected target;

In the case where the first target to be tracked exists, determine the first influence value of the mth target to be tracked, and the second influence value of the first target to be tracked; wherein, the first influence value is the difference determined based on the mth target to be tracked and the second detected target; the difference is the distance between a target to be tracked and a detected target and the weight of the target to be tracked and the detected target The difference between the sum of The difference value determined according to the first target to be tracked and the third detection target; the third detection target is the detection with the smallest difference value determined from the first target to be tracked except the first detection target. Target;

According to the first influence value and the second influence value, a detection target matching the mth target to be tracked is determined.
The method according to claim 6, wherein determining, according to the first influence value and the second influence value, a detection target matching the mth target to be tracked includes:

In the case where the first influence value is less than the second influence value, perform the fourth target operation on the mth target to be tracked and the second detected target, and determine the relationship with the mth target to be tracked. The tracking target matches the detection target;

If the first influence value is greater than the second influence value, it is determined that the mth target to be tracked matches the first detected target.
The method of claim 7, further comprising:

In the case where the first influence value is greater than the second influence value, the fourth target operation is performed on the first target to be tracked and the third detection target, and it is determined that the relationship between the first target to be tracked and the first target to be tracked is Matching detection targets.
The method according to claim 7 or 8, wherein the method further includes:

When the first influence value is less than the second influence value, the mth target to be tracked, the first target to be tracked and the first detection target are updated according to the first influence value. Weights;

When the first influence value is greater than the second influence value, the mth target to be tracked, the first target to be tracked and the first detection target are updated according to the second influence value. Weights.
The method according to claim 9, wherein updating the weight values of the m-th target to be tracked, the first target to be tracked and the first detected target according to the first influence value includes:

The weight value of the mth target to be tracked is increased by the first influence value, the weight value of the first target to be tracked is increased by the first influence value, and the weight value of the first detection target is reduced. The first influence value;

The updating of the weight values of the mth target to be tracked, the first target to be tracked and the first detected target according to the second influence value includes:

The weight value of the mth target to be tracked is increased by the second influence value, the weight value of the first target to be tracked is increased by the second influence value, and the weight value of the first detection target is reduced. The second influence value.
The method according to any one of claims 2 to 5, wherein determining the matching relationship based on the distance between each detected target and each to-be-tracked target includes:

Determine a list of targets to be tracked and a list of detected targets; wherein the list of targets to be tracked includes each of the targets to be tracked, and the list of detected targets includes each of the detected targets;

Iteratively perform the fifth target operation on the targets to be tracked and the detected targets in the target to be tracked list and the detected target list, until there is no target to be tracked in the target to be tracked list or there is no target in the detected target list. Detect the target and determine the matching relationship according to the execution result of the fifth target operation;

The fifth target operation includes: determining a target distance between each target to be tracked in the current target list to be tracked and each target to be tracked in the current detected target list;

Determine the minimum distance among the target distances, and the second target to be tracked and the fourth detection target corresponding to the minimum distance; the distance between the second target to be tracked and the fourth detection target is the minimum distance distance;

Determine the matching relationship between the second target to be tracked and the fourth detected target;

The second target to be tracked is deleted from the current target list to be tracked, and the fourth detected target is deleted from the current detected target list.
The method according to any one of claims 1-11, wherein tracking the target to be tracked according to the matching relationship includes:

Determine a third target to be tracked; each target to be tracked in the third target to be tracked has a matching detection target;

The position of the key point of the third target to be tracked is updated according to the detected position of the key point of the detected target that matches the third target to be tracked and the predicted position of the key point of the third target to be tracked.
The method according to any one of claims 1-12, wherein tracking the target to be tracked according to the matching relationship includes:

Determine a fifth detection target; each detection target in the fifth detection target does not have a matching target to be tracked;

The fifth detection target is added to the targets to be tracked.
The method according to any one of claims 1-13, wherein tracking the target to be tracked according to the matching relationship includes:

Determine the number of mismatches between the fourth target to be tracked and the fourth target to be tracked; there is no matching detection target for each target to be tracked in the fourth target to be tracked; the number of mismatches is the number of mismatches for the target to be tracked. The number of times the target has no matching detection target in a row;

If the number of mismatches is greater than the preset tracking threshold, the fourth target to be tracked is deleted from the targets to be tracked.
The method according to any one of claims 1 to 14, wherein the position of the key point of the target to be tracked at the first time is predicted based on the position of the key point of the target to be tracked at the second time. Predicted locations, including:

Determine the motion status of the key points of the target to be tracked;

According to the position and motion state of the key point of the target to be tracked at the second time, the predicted position of the key point of the target to be tracked at the first time is predicted.
A target tracking device including:

an acquisition unit configured to acquire the image to be detected at the first moment;

a processing unit configured to determine the detection position of the key point of the detection target in the image to be detected;

The acquisition unit is further configured to acquire the position of the key point of the target to be tracked at a second moment; the second moment is a moment before the first moment;

The processing unit is further configured to predict the predicted position of the key point of the target to be tracked at the first time based on the position of the key point of the target to be tracked at the second time;

The processing unit is further configured to determine a matching relationship between the detected target and the target to be tracked based on the detected position and the predicted position; the matching relationship is used to match the detected target and the target. Describe the same target among the targets to be tracked;

The processing unit is further configured to track the target to be tracked according to the matching relationship.
A target tracking device, including: a processor and a memory; the memory is used to store computer programs or instructions, and the processor is used to run computer programs or instructions to implement the method described in any one of claims 1-15 Target tracking method.
A target tracking system, including a target tracking device and an image acquisition device, the image acquisition device is used to collect images to be detected, the target tracking device is used to perform target tracking as described in any one of claims 1-15 method.
A non-transitory computer-readable storage medium, wherein instructions are stored in the non-transitory computer-readable storage medium. When the computer executes the instructions, the computer executes any one of the above claims 1-15. The target tracking method.
A computer program product, the computer program product includes computer program instructions. When the computer program instructions are executed on a computer, the computer program instructions cause the computer to perform target tracking as described in any one of claims 1-15. method.