WO2022048053A1 - Target tracking method, apparatus, and device, and computer-readable storage medium - Google Patents

Abstract

A target tracking method, apparatus, and device, and a computer-readable storage medium. The method comprises: receiving starting frame image information and sub-frame image information (S101); determining a starting frame object and a sub-frame object according to the starting frame image information and the sub-frame image information, respectively, by means of a detection algorithm (S102); matching the starting frame object and the sub-frame object by means of an association algorithm, and determining whether a pending starting frame object and a pending sub-frame object are present, the pending starting frame object and the pending sub-frame object referring to the starting frame object and the sub-frame object that fail to be successfully matched, respectively (S103); when the pending starting frame object and the pending sub-frame object are present, determining, by means of a correlation filtering algorithm, an analog starting frame object corresponding to the pending starting frame object (S104); matching the analog starting frame object and the sub-frame object by means of the association algorithm, and determining whether a target sub-frame object is present, the target sub-frame object being a sub-frame object that is successfully matched with the analog starting frame object (S105); and when the target sub-frame object is present, determining a corresponding tracking object according to the target sub-frame object (S106). By using the described method, the stability and accuracy of target tracking are improved and the processing efficiency is improved.

Description

A target tracking method, apparatus, device and computer-readable storage medium

This application claims the priority of the Chinese patent application with the application number 202010908917.7 and the invention titled "A Target Tracking Method, Apparatus, Equipment and Computer-readable Storage Medium" filed with the China Patent Office on September 2, 2020, all of which are The contents are incorporated herein by reference.

technical field

The present invention relates to the field of automatic driving, and in particular, to a target tracking method, apparatus, device and computer-readable storage medium.

Background technique

Nowadays, the complex traffic situation and the hot development of automatic driving have put forward higher requirements for driving safety. The in-vehicle assistance system can respond to the surrounding environment, assist the driver or the vehicle to make judgments, and effectively avoid traffic accidents. occurrence, improve the safety factor of driving.

Target tracking is divided into single-target tracking and multi-target tracking. The application scenarios of the two tracking algorithms are different. Single-target tracking selects a target in the initial frame, and predicts the position and size of the target in subsequent frames. The tracking object of single-target tracking is only One, which greatly limits its application scenarios, but the single-target tracking algorithm, especially the single-target tracking algorithm based on correlation filtering, has the advantages of fast calculation speed and accurate tracking results. Compared with the well-developed single-target tracking algorithm, the research on multi-target tracking algorithm is still in the developing stage. Multi-target tracking is to track multiple targets at the same time to obtain the motion trajectory of each target, which is more suitable for application in vehicle-mounted assistance systems. However, the existing multi-target tracking algorithm is easy to miss targets and has a slow response speed, which is obviously not suitable for automatic driving scenarios where new targets often appear and old targets disappear.

Therefore, how to improve the accuracy, stability and response speed of the existing target tracking algorithm has become an urgent problem to be solved by those skilled in the art.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a target tracking method, apparatus, device and computer-readable storage medium, so as to solve the problems of unstable detection and tracking, slow response and missed target detection in the prior art.

In order to solve the above-mentioned technical problems, the present invention provides a target tracking method, comprising:

Receive the first frame image information and the second frame image information;

Determine the initial frame object and the sub-frame object respectively according to the initial frame image information and the sub-frame image information by a detection algorithm;

Match the initial frame object and the sub-frame object through an association algorithm, and determine whether there is a pending initial frame object and an pending sub-frame object, wherein the pending initial frame object and the pending sub-frame object respectively refer to the The initial frame object and the secondary frame object that are successfully matched;

When there are the pending start frame object and the pending sub-frame object, determine a simulated start frame object corresponding to the pending start frame object through a correlation filtering algorithm;

The simulation start frame object and the subframe object are matched by the association algorithm, and it is judged whether there is a target subframe object, and the target subframe object is a subframe object that is successfully matched with the simulation start frame object;

When the target sub-frame object exists, the corresponding tracking object is determined according to the target sub-frame object.

Optionally, in the target tracking method, after the simulation initial frame object and the secondary frame object are matched by the association algorithm, the method further includes:

Judging whether there is a suspicious initial frame object, the suspicious initial frame object is a simulated initial frame object for which the secondary frame object has not been successfully matched;

The suspect tracking object is determined according to the suspect initial frame object.

Optionally, in the target tracking method, after judging whether there are pending initial frame objects and pending secondary frame objects, the method further includes:

When only the pending start frame object exists and the pending secondary frame object does not exist, determine the simulated start frame object corresponding to the pending start frame object through a correlation filtering algorithm;

A suspect tracking object is determined according to the simulated start frame object.

Optionally, in the target tracking method, after determining the suspect tracking object, the method further includes:

determining the correlation filter response value of the suspect tracking object;

When the correlation filter response value is less than the first threshold, it is determined that the suspected tracking object disappears;

When the correlation filter response value is between the first threshold and the second threshold, it is determined that the suspicious tracking object is an unnecessary tracking object; wherein, after determining that the unnecessary tracking object is in the After the tracking information within the first number of frames is obtained, it is determined that the unnecessary tracking object disappears.

Optionally, in the target tracking method, the determining the simulated start frame object corresponding to the to-be-determined start frame object by using a correlation filtering algorithm includes:

obtaining the size information of the pending start frame object;

judging whether the size information is less than a size threshold;

When the size information is smaller than the size threshold, the target area corresponding to the to-be-determined start frame object is processed by the top-hat operator to obtain a preprocessing area;

A simulated start frame object corresponding to the to-be-determined start frame object is obtained through the correlation filtering algorithm according to the preprocessing area.

A target tracking device, comprising:

The receiving module is used to receive the image information of the first frame and the image information of the second frame;

a primary and secondary frame object determination module, configured to determine the primary frame object and the secondary frame object respectively according to the primary frame image information and the secondary frame image information through a detection algorithm;

The primary and secondary matching module is used to match the initial frame object and the secondary frame object through an association algorithm, and determine whether there is an undetermined initial frame object and an undetermined secondary frame object, wherein the pending initial frame object and the pending The secondary frame objects respectively refer to the initial frame objects and the secondary frame objects that have not been successfully matched;

a correlation filtering module, configured to determine a simulated start frame object corresponding to the pending start frame object through a correlation filtering algorithm when the pending start frame object and the pending secondary frame object exist;

The simulation matching module is used to match the simulation start frame object and the subframe object through the association algorithm, and judge whether there is a target subframe object, and the target subframe object is matched with the simulation start frame object successful subframe object;

A tracking determination module is configured to determine a corresponding tracking object according to the target sub-frame object when the target sub-frame object exists.

Optionally, in the target tracking device, the analog matching module further includes:

A doubtful judging unit for judging whether there is a doubtful initial frame object, and the doubtful initial frame object is a simulated initial frame object for which the secondary frame object has not been successfully matched;

The first suspect determination unit is configured to determine the suspect tracking object according to the suspect initial frame object.

Optionally, in the target tracking device, the primary and secondary matching module further includes:

An undetermined suspicious simulation unit, used for determining the simulated starting frame object corresponding to the undetermined starting frame object by a correlation filtering algorithm when only the undetermined starting frame object does not exist the undetermined secondary frame object;

The second suspect determination unit is configured to determine the suspect tracking object according to the simulated initial frame object.

A target tracking device comprising:

memory for storing computer programs;

The processor is configured to implement the steps of any one of the above-mentioned target tracking methods when executing the computer program.

A computer-readable storage medium storing a computer program on the computer-readable storage medium, when the computer program is executed by a processor, implements the steps of any one of the above-mentioned target tracking methods.

In the target tracking method provided by the present invention, the first frame image information and the second frame image information are received; the first frame object and the second frame object are respectively determined according to the first frame image information and the second frame image information through a detection algorithm; The algorithm matches the initial frame object and the sub-frame object, and determines whether there is a pending initial frame object and an pending sub-frame object, wherein the pending initial frame object and the pending sub-frame object respectively refer to an unsuccessful match. The initial frame object and the sub-frame object of the The association algorithm matches the simulated initial frame object and the sub-frame object, and determines whether there is a target sub-frame object, and the target sub-frame object is the sub-frame object successfully matched with the simulated initial frame object; when When the target sub-frame object exists, the corresponding tracking object is determined according to the target sub-frame object. In the present invention, by using the correlation filtering algorithm to calculate the to-be-determined initial-frame object, after obtaining the simulated initial-frame object, it is compared with the to-be-determined sub-frame object, thereby greatly improving the relationship between the initial frame object and all The matching success rate between the objects in the sub-frame is described, thereby improving the stability and accuracy of target tracking. At the same time, compared with other methods, it is not necessary to perform single-target tracking for each object in the image information, which can reduce the operation of the algorithm. to improve the running speed of the tracking algorithm. The present invention also provides a target tracking device, device and computer-readable storage medium with the above beneficial effects.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the following will briefly introduce the accompanying drawings used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only For some embodiments of the present invention, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a schematic flowchart of a specific embodiment of a target tracking method provided by the present invention;

2 is a schematic flowchart of another specific embodiment of the target tracking method provided by the present invention;

3 is a schematic flowchart of another specific implementation manner of the target tracking method provided by the present invention;

4 is a schematic structural diagram of a specific implementation manner of a target tracking device provided by the present invention;

FIG. 5 is a schematic structural diagram of a specific implementation manner of a target tracking system provided by the present invention.

detailed description

In some special scenes, such as bad weather at night, haze, etc., the driver's visual distance becomes smaller, and the imaging quality of visible light becomes worse, so the driving risk factor is greatly increased, and the infrared thermal imaging system provides a good solution for these scenes. s solution. The invention is an infrared multi-target tracking algorithm based on the infrared vehicle-mounted auxiliary system. In the vehicle-mounted auxiliary system, some important targets need to be detected and tracked, such as pedestrians and vehicles. Assist the driver or automatic driving to adjust the driving state of the vehicle to ensure the safety and stability of the vehicle. The tracking algorithm of the present invention can not only track multiple targets, but also improve the performance of target detection. Assist the vehicle to drive and improve the safety factor.

There are two main types of multi-target tracking, one is multi-target tracking combined with detection, and the other is multi-target tracking based on the initialization of the first frame. Both methods have their own advantages and disadvantages. The multi-target tracking combined with detection needs to obtain the detection target of each frame through the detection algorithm, and then use the multi-target tracking algorithm to associate the target of each frame to find the motion trajectory of each target. Common multi-target tracking algorithms include sort, Deep-sort and deep learning network JDE, FairMOT, etc., the multi-target tracking algorithm combined with detection is very dependent on the effect of detection. Currently, deep learning detection methods such as yolo, centernet are mostly used, and there are also classic frame difference method, optical flow. method, etc.; multi-target tracking based on the initialization of the first frame needs to initialize the target in the first frame, and use multiple single-target trackers to track in the subsequent frames. Obviously, such an algorithm is not suitable for new targets that often appear and old targets disappear. autonomous driving scenarios, and the speed is slower. Therefore, the multi-target tracking algorithm combined with detection can better meet the requirements of on-board assistance systems.

In addition to detecting the target accurately, the multi-target tracking combined with detection. It is also necessary to find the correlation matching between targets, correctly distinguish each target, and correctly deal with problems such as target occlusion, target disappearance, target appearance change, target appearance similarity, and new target appearance. The multi-target tracking algorithm is mainly composed of three parts: the detector, the tracker and the data association. The data association is the core part of the multi-target tracking algorithm, and it is also the difficulty of the multi-target tracking algorithm, especially when there are multiple objects that are close in distance and have similar appearances. The process of data association will be very complicated when the target is used. Commonly used data association methods include classical MHT, PDA and data association realized by matching algorithm combined with loss matrix. Adding a multi-target tracking algorithm to the vehicle-mounted auxiliary system can not only detect the target, but also obtain the information of a certain target by correlating the detection results, and further obtain the motion state information of the target, such as the target distance and movement direction. Etc., this information is of great significance in assisted driver driving and automatic driving. However, the current multi-target tracking combined with detection only correlates the detection results, and the multi-target algorithm itself does not have the ability to detect targets. Therefore, when the detection effect is not good, such as missing targets, unstable detection, etc., it will directly affect the results of the multi-target tracking algorithm and increase the risk factor of driving.

In order to make those skilled in the art better understand the solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The core of the present invention is to provide a target tracking method, and a schematic flowchart of a specific implementation of the method is shown in FIG.

S101: Receive the first frame image information and the second frame image information.

Since the tracking algorithm is based on video data, the image information of the first frame and the image information of the second frame are generally two adjacent frames of image information in a video. Of course, according to the situation, they may also be non-adjacent frames. Two frames of image information, for example, in order to reduce the burden on the processor, the image information of one frame interval is deliberately used for target tracking and so on.

S102: Determine the initial frame object and the sub-frame object respectively according to the initial frame image information and the sub-frame image information through a detection algorithm.

The current detection algorithm can be a machine learning method, such as HOG+SVM, Adaboost+ACF, etc.; it can also be a deep learning method, such as the yolo series or the SSD series, etc., which can be freely selected according to the actual situation.

S103: Match the initial frame object and the sub-frame object through an association algorithm, and determine whether there is a pending initial frame object and an pending sub-frame object, wherein the pending initial frame object and the pending sub-frame object respectively refer to The initial frame object and the secondary frame object that fail to match successfully.

It should be noted that the association algorithm can include a matching algorithm and a loss matrix, wherein the matching algorithm includes: Hungarian matching, KM matching, etc.; the loss matrix can be calculated by the intersection ratio, Euclidean distance, etc., and the loss matrix can also Calculation is replaced by deep features, of course, other suitable algorithms can also be selected according to the actual situation.

S104: When there are the pending start frame object and the pending secondary frame object, determine a simulated start frame object corresponding to the pending start frame object by using a correlation filtering algorithm.

The correlation filtering algorithm can adopt the existing single-target tracking algorithm, such as MOOSE, CSK, KCF, DCF and so on. The correlation filtering algorithm will process the results that are not successfully associated in the correlation algorithm of S103, and for the undetermined initial frame objects that are not successfully associated, the correlation filtering algorithm will be used to track each object with a single target, and obtain: The tracking results of each object in the current frame can also be used for single-target tracking algorithms implemented by deep learning networks.

For the detection and tracking of the successfully matched object in step S103 (here, the successfully matched sub-frame object can be used as the tracking object corresponding to the initial frame object), in order to avoid subsequent tracking due to changes in the shape of the object If it fails, if the continuous matching successfully exceeds a certain number of frames (such as 5-10 frames), it is necessary to update the relevant filtering template.

S105: Match the simulated initial frame object and the secondary frame object through the association algorithm, and determine whether there is a target secondary frame object, and the target secondary frame object is a secondary frame successfully matched with the simulated initial frame object object.

S106: When the target sub-frame object exists, determine a corresponding tracking object according to the target sub-frame object.

Since the target subframe object corresponds to the simulation start frame object, and the simulation start frame object corresponds to the pending start frame object, the target subframe object can be determined to correspond to the start frame object of the tracking object.

Certainly, if there is only the pending sub-frame object in the judgment process of S103 or after the matching of S105, there is a sub-frame object that is not successfully matched with the initial frame object, then the above-mentioned sub-frame object is identified as a new tracking object.

As a preferred implementation manner, in S104, determining the simulated start frame object corresponding to the to-be-determined start frame object by using the correlation filtering algorithm includes:

S1041: Acquire size information of the to-be-determined start frame object.

S1042 judges whether the size information is smaller than a size threshold.

S1043: When the size information is smaller than the size threshold, process the target area corresponding to the to-be-determined start frame object by using a top-hat operator to obtain a preprocessing area.

S1044: Obtain a simulated start frame object corresponding to the to-be-determined start frame object through the correlation filtering algorithm according to the preprocessing area.

In this specific implementation manner, for the detected small infrared targets, the top-hat operator is processed first to highlight the small infrared targets, and then the features are calculated for tracking, which can effectively improve the tracking accuracy.

Aiming at infrared images, the present invention can effectively perform multi-target detection and tracking in the night, dust, haze and other weather, improve the safety of driving in the appeal scene, and combine the detection algorithm, multi-target tracking and single-target tracking, It can improve the detection effect. When the detection algorithm does not detect the target, the tracking algorithm is used to track the target, supplement the detection results, stabilize the detection frame, and improve the detection accuracy. In addition, the present invention integrates the multi-target tracking algorithm combined with the detection algorithm and the single-target tracking, which is faster than multiple single-target trackers, and is tracked by the detection results of the detection algorithm, which is more accurate. For the entire video, the above process needs to be repeated cyclically until the video ends. When the sub-frame image information in this cycle is used as the initial frame image information in the next cycle, the tracking object obtained in this cycle is The start frame object in the next loop.

The algorithm of the present invention can be realized based on the Xsafe-II M series vehicle infrared night vision system. The system connects the Xsafe-II M series vehicle infrared camera with the ECU processing unit based on the Ambarella CV25 chip to realize the real-time infrared image Display, target recognition, target tracking and alarm functions.

The schematic diagram of the overall structure of the system is shown in Figure 5. The target tracking method of the present invention needs to work in the intelligent algorithm module in the ECU processing unit of the system in Figure 5, and the results output by the algorithm are displayed on the car display screen through the display module. .

Table 1 shows the specific data in a set of tracking tests:

video	video duration	Improved tracking accuracy
test set 1	90s	8.5%
test set 2	120s	15.2%
test set 3	60s	5.3%

Table 1 Tracking test results

Among them, the detection algorithm is mobilenet_yolov3, the matching algorithm is the Hungarian algorithm, the loss matrix is calculated by the intersection ratio, and the tracking algorithm is KCF. The four labeled infrared vehicle videos are tested. The results are as follows, the tracking accuracy is 3%-20% promote.

In the target tracking method provided by the present invention, the first frame image information and the second frame image information are received; the first frame object and the second frame object are respectively determined according to the first frame image information and the second frame image information through a detection algorithm; The algorithm matches the initial frame object and the sub-frame object, and determines whether there is a pending initial frame object and an pending sub-frame object, wherein the pending initial frame object and the pending sub-frame object respectively refer to an unsuccessful match. The initial frame object and the sub-frame object of the The association algorithm matches the simulated initial frame object and the sub-frame object, and determines whether there is a target sub-frame object, and the target sub-frame object is the sub-frame object successfully matched with the simulated initial frame object; when When the target sub-frame object exists, the corresponding tracking object is determined according to the target sub-frame object. In the present invention, only the undetermined initial frame object that fails to match is calculated by using the correlation filtering algorithm, and after the simulated initial frame object is obtained, it is compared with the undetermined sub-frame object, which greatly improves the performance of the initial frame. The matching success rate between the object and the sub-frame object, thereby improving the stability and accuracy of target tracking. At the same time, compared with other methods, it is not necessary to perform single-target tracking for each object in the image information, which can reduce The computational complexity of the algorithm improves the running speed of the tracking algorithm.

On the basis of the first embodiment, the tracking objects are further classified, and the second embodiment is obtained. The schematic flowchart is shown in FIG. 2 , including:

S201: Receive the first frame image information and the second frame image information.

S202: Determine the initial frame object and the sub-frame object respectively according to the initial frame image information and the sub-frame image information through a detection algorithm.

S203: Match the initial frame object and the sub-frame object through an association algorithm, and determine whether there is a pending initial frame object and a pending sub-frame object, wherein the pending initial frame object and the pending sub-frame object respectively refer to The initial frame object and the secondary frame object that fail to match successfully.

S2041: When there are the pending start frame object and the pending secondary frame object, determine a simulated start frame object corresponding to the pending start frame object through a correlation filtering algorithm.

S2051: When only the pending start frame object exists and the pending secondary frame object does not exist, determine a simulated start frame object corresponding to the pending start frame object through a correlation filtering algorithm;

S2052: Determine a suspected tracking object according to the simulated initial frame object.

S2042: Match the simulation start frame object and the subframe object by using the association algorithm, and determine whether there is a target subframe object, and the target subframe object is a subframe that is successfully matched with the simulation start frame object object.

S2043: When the target sub-frame object exists, determine a corresponding tracking object according to the target sub-frame object.

The difference between this specific implementation manner and the above specific implementation manner is that a new type of tracking object detection is added in this specific implementation manner, and the remaining steps are the same as the above specific implementation manner, which will not be repeated here.

It should be noted that S2041 , S2042 , and S2043 and S2051 and S2052 in this specific embodiment are two processing solutions in different situations, and there is no sequence relationship between them, and the sequence of steps can be exchanged at will.

It should also be noted that in addition to determining whether the suspect tracking object exists in S2052, in S2042, in addition to judging whether the target sub-frame object exists at this time, it is also possible to determine whether there is a suspect starting frame object, the suspect starting frame The object is a simulated initial frame object for which the secondary frame object has not been successfully matched; and then a suspicious tracking object is determined according to the suspicious initial frame object.

In this specific implementation manner, in addition to the tracking object, the suspect tracking object is also identified, and the suspect tracking object does not appear in every frame of the tracking object, but in practical applications, the object is in the video If a certain frame of a frame does not appear, it does not mean that the object disappears, but it may be blocked by other objects and other reasons and cannot be identified. In this specific implementation, an object that does not appear in a single frame is defined as a suspicious tracking object. Temporarily retaining the relevant information of the suspected tracking object in the system can avoid deleting the object if the object cannot be identified in a single frame. If the object reappears in the next frame, the trouble of recalculating the tracking object is greatly reduced. The computational complexity of the method is reduced, and the computational efficiency is improved.

On the basis of the second embodiment, the tracking objects are further classified, and the third embodiment is obtained. The schematic flowchart is shown in FIG. 3 , including:

S301: Receive the image information of the first frame and the image information of the second frame.

S302 : Determine an initial frame object and a sub-frame object respectively according to the initial frame image information and the sub-frame image information through a detection algorithm.

S303: Match the initial frame object and the sub-frame object through an association algorithm, and determine whether there is a pending initial frame object and an pending sub-frame object, wherein the pending initial frame object and the pending sub-frame object respectively refer to The initial frame object and the secondary frame object that fail to match successfully.

S3041: When there are the pending start frame object and the pending secondary frame object, use a correlation filtering algorithm to determine a simulated start frame object corresponding to the pending start frame object.

S3051: When only the pending start frame object exists and the pending secondary frame object does not exist, determine a simulated start frame object corresponding to the pending start frame object through a correlation filtering algorithm;

S3052: Determine a suspected tracking object according to the simulated initial frame object.

S3042: Match the simulated initial frame object and the secondary frame object by using the association algorithm, and determine whether there is a target secondary frame object, and the target secondary frame object is a secondary frame successfully matched with the simulated initial frame object object.

S3043: When the target sub-frame object exists, determine a corresponding tracking object according to the target sub-frame object.

S3053: Determine the correlation filter response value of the suspected tracking object.

S3054: When the correlation filter response value is less than the first threshold, determine that the suspected tracking object disappears.

S3055: When the correlation filtering response value is between the first threshold and the second threshold, determine that the suspected tracking object is an unnecessary tracking object; wherein, when determining the unnecessary tracking through the correlation filtering algorithm After the object has tracking information within the first number of frames, it is determined that the unnecessary tracking object disappears.

The difference between this specific implementation and the above specific implementation is that in this specific implementation, after the suspected tracking object is determined, the suspected tracking object is further classified and processed, and the remaining steps are the same as the above specific implementation. , and will not be repeated here.

In this specific embodiment, the suspected tracking object is further classified according to the correlation filter response value. If the correlation filter response value is less than a certain threshold (for example, 0.5-0.7), the object is recognized as disappearing and deleted. Tracking of the object; if it is between the two thresholds, it is considered that the tracking algorithm does not track the object well, and the object is only tracked for a certain number of frames (for example, 5-20 frames), and the tracking of the object is deleted. ; if it is greater than a certain threshold (e.g. 0.8-0.9), the tracking is considered good and the object will continue to be tracked. In this way, when the object disappears, the tracking frame will stay in the picture for a long time, or the system will judge that the object has disappeared just after the object disappears from the picture for a few frames, thus increasing the computational burden and reducing the computational efficiency when the object reappears. .

The following describes the target tracking apparatus provided by the embodiments of the present invention, and the target tracking apparatus described below and the target tracking method described above may refer to each other correspondingly.

FIG. 4 is a structural block diagram of a target tracking apparatus provided by an embodiment of the present invention, which is referred to as the fourth specific embodiment. Referring to FIG. 4, the target tracking apparatus may include:

The receiving module 100 is used for receiving the first frame image information and the second frame image information;

The primary and secondary frame object determination module 200 is configured to determine the primary frame object and the secondary frame object respectively according to the primary frame image information and the secondary frame image information through a detection algorithm;

The primary and secondary matching module 300 is configured to match the start frame object and the secondary frame object through an association algorithm, and determine whether there is an undetermined start frame object and an undetermined secondary frame object, wherein the pending start frame object and the The pending sub-frame objects respectively refer to the initial frame object and the sub-frame object that have not been successfully matched;

The correlation filtering module 400 is configured to determine, through a correlation filtering algorithm, a simulated initial frame object corresponding to the pending initial frame object when the pending initial frame object and the pending secondary frame object exist;

The simulation matching module 500 is used to match the simulation start frame object and the subframe object through the association algorithm, and determine whether there is a target subframe object, and the target subframe object is the same as the simulation start frame object. Match the successful subframe object;

The tracking determination module 600 is configured to determine a corresponding tracking object according to the target sub-frame object when the target sub-frame object exists.

As a preferred embodiment, the analog matching module 500 further includes:

A doubtful judging unit for judging whether there is a doubtful initial frame object, and the doubtful initial frame object is a simulated initial frame object for which the secondary frame object has not been successfully matched;

The first suspect determination unit is configured to determine the suspect tracking object according to the suspect initial frame object.

As a preferred embodiment, the primary and secondary matching module 300 further includes:

an undetermined suspicious simulation unit, configured to determine a simulated starting frame object corresponding to the undetermined starting frame object through a correlation filtering algorithm when only the undetermined starting frame object does not exist the undetermined secondary frame object;

The second suspect determination unit is configured to determine the suspect tracking object according to the simulated initial frame object.

As a preferred embodiment, the analog matching module 500 and/or the primary and secondary matching module 300 further includes:

a response value determination unit, used for determining the relevant filtering response value of the suspect tracking object;

A disappearance judgment unit, configured to determine that the suspected tracking object disappears when the correlation filter response value is less than a first threshold;

an unnecessary judging unit, configured to determine that the suspected tracking object is an unnecessary tracking object when the correlation filtering response value is between the first threshold and the second threshold; wherein, when the correlation filtering algorithm is used to determine It is determined that the unnecessary tracking object disappears after the tracking information of the unnecessary tracking object is within the first number of frames.

As a preferred embodiment, the correlation filtering module 400 includes:

an acquisition unit, used for acquiring the size information of the to-be-determined start frame object;

a size judging unit for judging whether the size information is less than a size threshold;

The top-hat unit is configured to process the target area corresponding to the to-be-determined start frame object by the top-hat operator to obtain a preprocessing area when the size information is less than the size threshold;

A simulation determination unit, configured to obtain a simulation start frame object corresponding to the to-be-determined start frame object through the correlation filtering algorithm according to the preprocessing area.

The target tracking device provided by the present invention, through the receiving module 100, is used to receive the initial frame image information and the secondary frame image information; the primary and secondary frame object determination module 200 is used for detecting The secondary frame image information respectively determines the initial frame object and the secondary frame object; the primary and secondary matching module 300 is used to match the initial frame object and the secondary frame object through an association algorithm, and determine whether there is an undetermined initial frame object and an undetermined secondary frame object. A frame object, wherein the pending start frame object and the pending sub-frame object respectively refer to the initial frame object and the sub-frame object that have not been successfully matched; the correlation filtering module 400 is configured to, when there is the pending start frame object, When the frame object and the undetermined sub-frame object, determine the simulation start frame object corresponding to the undetermined start frame object through a correlation filtering algorithm; the simulation matching module 500 is used for the simulation start frame object and the simulation start frame object through the correlation algorithm. The sub-frame objects are matched to determine whether there is a target sub-frame object, and the target sub-frame object is a sub-frame object that is successfully matched with the simulated initial frame object; the tracking determination module 600 is used for when the target sub-frame object exists. When the frame object is selected, the corresponding tracking object is determined according to the target sub-frame object. In the present invention, by using the correlation filtering algorithm to calculate the undetermined starting frame object, after obtaining the simulated starting frame object, and then comparing it with the undetermined sub-frame object, it greatly improves the relationship between the starting frame object and all other objects. The matching success rate between the above-mentioned sub-frame objects, thereby improving the stability and accuracy of target tracking, and at the same time, compared with other methods, the use of the correlation filtering algorithm can greatly improve the initial frame object and the sub-frame object. The matching speed between them can improve the processing efficiency.

The target tracking device of this embodiment is used to implement the aforementioned target tracking method, so the specific implementation of the target tracking device can be found in the embodiment part of the aforementioned target tracking method, for example, the receiving module 100, the primary and secondary frame object determination module 200 , the primary and secondary matching module 300, the correlation filtering module 400, the 500-level analog matching module and the total determination module 600 are respectively used to realize steps S101, S102, S103, S104, S105 and S106 in the above-mentioned target tracking method. Therefore, its specific implementation For the manner, reference may be made to the descriptions of the corresponding partial embodiments, which are not repeated here.

A target tracking device comprising:

memory for storing computer programs;

The processor is configured to implement the steps of any one of the above-mentioned target tracking methods when executing the computer program. In the target tracking method provided by the present invention, the first frame image information and the second frame image information are received; the first frame object and the second frame object are respectively determined according to the first frame image information and the second frame image information through a detection algorithm; The algorithm matches the initial frame object and the sub-frame object, and determines whether there is a pending initial frame object and an pending sub-frame object, wherein the pending initial frame object and the pending sub-frame object respectively refer to an unsuccessful match. The initial frame object and the sub-frame object of the The association algorithm matches the simulated initial frame object and the sub-frame object, and determines whether there is a target sub-frame object, and the target sub-frame object is the sub-frame object successfully matched with the simulated initial frame object; when When the target sub-frame object exists, the corresponding tracking object is determined according to the target sub-frame object. In the present invention, by using the correlation filtering algorithm to calculate the undetermined starting frame object, after obtaining the simulated starting frame object, and then comparing it with the undetermined sub-frame object, it greatly improves the relationship between the starting frame object and all other objects. The matching success rate between the above-mentioned sub-frame objects, thereby improving the stability and accuracy of target tracking, and at the same time, compared with other methods, the use of the correlation filtering algorithm can greatly improve the initial frame object and the sub-frame object. The matching speed between them can improve the processing efficiency.

A computer-readable storage medium storing a computer program on the computer-readable storage medium, when the computer program is executed by a processor, implements the steps of any one of the above-mentioned target tracking methods. In the target tracking method provided by the present invention, the first frame image information and the second frame image information are received; the first frame object and the second frame object are respectively determined according to the first frame image information and the second frame image information through a detection algorithm; The algorithm matches the initial frame object and the sub-frame object, and determines whether there is a pending initial frame object and an pending sub-frame object, wherein the pending initial frame object and the pending sub-frame object respectively refer to an unsuccessful match. The initial frame object and the sub-frame object of the The association algorithm matches the simulated initial frame object and the sub-frame object, and determines whether there is a target sub-frame object, and the target sub-frame object is the sub-frame object successfully matched with the simulated initial frame object; when When the target sub-frame object exists, the corresponding tracking object is determined according to the target sub-frame object. In the present invention, by using the correlation filtering algorithm to calculate the to-be-determined initial-frame object, after obtaining the simulated initial-frame object, it is compared with the to-be-determined sub-frame object, thereby greatly improving the relationship between the initial frame object and all The matching success rate between the above-mentioned sub-frame objects, thereby improving the stability and accuracy of target tracking, and at the same time, compared with other methods, the use of the correlation filtering algorithm can greatly improve the initial frame object and the sub-frame object. The matching speed between them is improved, and the processing efficiency is improved.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments may be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method.

It should be noted that in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these entities or operations There is no such actual relationship or order between them. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

Professionals may further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two, in order to clearly illustrate the possibilities of hardware and software. Interchangeability, the above description has generally described the components and steps of each example in terms of functionality. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

The steps of a method or algorithm described in conjunction with the embodiments disclosed herein may be directly implemented in hardware, a software module executed by a processor, or a combination of the two. A software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other in the technical field. in any other known form of storage medium.

The target tracking method, apparatus, device and computer-readable storage medium provided by the present invention have been described in detail above. The principles and implementations of the present invention are described herein by using specific examples, and the descriptions of the above embodiments are only used to help understand the method and the core idea of the present invention. It should be pointed out that for those skilled in the art, without departing from the principle of the present invention, several improvements and modifications can also be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims (10)

1. A target tracking method, comprising:

   Receive the first frame image information and the second frame image information;

   Determine the initial frame object and the sub-frame object respectively according to the initial frame image information and the sub-frame image information by a detection algorithm;

   Match the initial frame object and the sub-frame object through an association algorithm, and determine whether there is a pending initial frame object and an pending sub-frame object, wherein the pending initial frame object and the pending sub-frame object respectively refer to the The initial frame object and the secondary frame object that are successfully matched;

   When there are the pending start frame object and the pending sub-frame object, determine a simulated start frame object corresponding to the pending start frame object through a correlation filtering algorithm;

   The simulation start frame object and the subframe object are matched by the association algorithm, and it is judged whether there is a target subframe object, and the target subframe object is a subframe object that is successfully matched with the simulation start frame object;

   When the target sub-frame object exists, the corresponding tracking object is determined according to the target sub-frame object.
2. The target tracking method according to claim 1, wherein after matching the simulated initial frame object and the secondary frame object by the association algorithm, the method further comprises:

   Judging whether there is a suspicious initial frame object, the suspicious initial frame object is a simulated initial frame object for which the secondary frame object has not been successfully matched;

   The suspect tracking object is determined according to the suspect initial frame object.
3. The target tracking method according to claim 1, wherein after judging whether there is a pending initial frame object and an pending secondary frame object, the method further comprises:

   When only the pending start frame object exists and the pending secondary frame object does not exist, determine the simulated start frame object corresponding to the pending start frame object through a correlation filtering algorithm;

   A suspect tracking object is determined according to the simulated start frame object.
4. The target tracking method according to claim 2 or 3, characterized in that, after determining the suspect tracking object, further comprising:

   determining the correlation filter response value of the suspect tracking object;

   When the correlation filter response value is less than the first threshold, it is determined that the suspected tracking object disappears;

   When the correlation filter response value is between the first threshold and the second threshold, it is determined that the suspicious tracking object is an unnecessary tracking object; wherein, after determining that the unnecessary tracking object is in the After the tracking information within the first number of frames is obtained, it is determined that the unnecessary tracking object disappears.
5. The target tracking method according to claim 1, wherein the determining the simulated start frame object corresponding to the to-be-determined start frame object by using a correlation filtering algorithm comprises:

   obtaining the size information of the pending start frame object;

   judging whether the size information is less than a size threshold;

   When the size information is smaller than the size threshold, the target area corresponding to the to-be-determined start frame object is processed by the top-hat operator to obtain a preprocessing area;

   A simulated start frame object corresponding to the to-be-determined start frame object is obtained through the correlation filtering algorithm according to the preprocessing area.
6. A target tracking device, comprising:

   The receiving module is used to receive the image information of the first frame and the image information of the second frame;

   a primary and secondary frame object determination module, configured to determine the primary frame object and the secondary frame object respectively according to the primary frame image information and the secondary frame image information through a detection algorithm;

   The primary and secondary matching module is used to match the initial frame object and the secondary frame object through an association algorithm, and determine whether there is an undetermined initial frame object and an undetermined secondary frame object, wherein the pending initial frame object and the pending The secondary frame objects respectively refer to the initial frame objects and the secondary frame objects that have not been successfully matched;

   a correlation filtering module, configured to determine a simulated start frame object corresponding to the pending start frame object through a correlation filtering algorithm when the pending start frame object and the pending secondary frame object exist;

   The simulation matching module is used to match the simulation start frame object and the subframe object through the association algorithm, and judge whether there is a target subframe object, and the target subframe object is matched with the simulation start frame object successful subframe object;

   A tracking determination module is configured to determine a corresponding tracking object according to the target sub-frame object when the target sub-frame object exists.
7. The target tracking device of claim 6, wherein the analog matching module further comprises:

   Doubtful judging unit, for judging whether there is a doubtful initial frame object, and the doubtful initial frame object is the simulated initial frame object that the secondary frame object does not match successfully;

   The first suspect determination unit is configured to determine the suspect tracking object according to the suspect initial frame object.
8. The target tracking device according to claim 6, wherein the primary and secondary matching module further comprises:

   an undetermined suspicious simulation unit, configured to determine a simulated starting frame object corresponding to the undetermined starting frame object through a correlation filtering algorithm when only the undetermined starting frame object does not exist the undetermined secondary frame object;

   The second suspect determination unit is configured to determine the suspect tracking object according to the simulated initial frame object.
9. A target tracking device, comprising:

   memory for storing computer programs;

   The processor is configured to implement the steps of the target tracking method according to any one of claims 1 to 5 when executing the computer program.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the target tracking method according to any one of claims 1 to 5 is implemented A step of.

Images