CN111429478A

CN111429478A - Target tracking method and related equipment

Info

Publication number: CN111429478A
Application number: CN202010287132.2A
Authority: CN
Inventors: 王非凡; 何佳伟; 熊佳; 郭文彬; 刘小伟; 彭晓峰
Original assignee: Spreadtrum Communications Shanghai Co Ltd
Current assignee: Spreadtrum Communications Shanghai Co Ltd
Priority date: 2020-04-13
Filing date: 2020-04-13
Publication date: 2020-07-17
Anticipated expiration: 2040-04-13
Also published as: CN111429478B

Abstract

The application discloses a target tracking method and related equipment, which are applied to electronic equipment, wherein the method comprises the following steps: acquiring an image sequence to be tracked, wherein the image sequence to be tracked comprises a plurality of frames of images to be tracked, and each image to be tracked corresponds to one frame number; determining whether the area where the tracking object is located is a large-area flat area or whether the background of the tracking object and the background of the first frame image to be tracked are similar or not based on the tracking frame in the first frame image to be tracked of the image sequence to be tracked; if the area where the tracking object is located is not a large-area flat area or the tracking object is not similar to the background of the first frame of image to be tracked, tracking the tracking object; and if the area where the tracking object is located is a large-area flat area or the tracking object is similar to the background of the first frame of image to be tracked, stopping the tracking operation.

Description

Target tracking method and related equipment

Technical Field

The present application relates to the field of electronic technologies, and in particular, to a target tracking method and related devices.

Background

The tracking of moving objects is one of the subjects of common research in the field of computer vision, and plays an important role in intelligent transportation, video monitoring, medicine and the like. An algorithm commonly adopted for tracking a moving target is a mean shift algorithm, however, the mean shift algorithm is not suitable for a flat area in which a tracked object is not easily distinguished from a background in an image to be tracked, and in this case, the tracked object is easily shifted and unstable, so that the accuracy of target tracking is reduced.

Disclosure of Invention

The embodiment of the application provides a target tracking method and related equipment, which are used for improving the accuracy of target tracking.

In a first aspect, an embodiment of the present application provides a target tracking method applied to an electronic device, where the method includes:

acquiring an image sequence to be tracked, wherein the image sequence to be tracked comprises a plurality of frames of images to be tracked, and each image to be tracked corresponds to one frame number;

determining whether the area where the tracking object is located is a large-area flat area or whether the background of the tracking object and the background of the first frame image to be tracked are similar or not based on the tracking frame in the first frame image to be tracked of the image sequence to be tracked;

if the area where the tracking object is located is not a large-area flat area or the tracking object is not similar to the background of the first frame of image to be tracked, tracking the tracking object;

and if the area where the tracking object is located is a large-area flat area or the tracking object is similar to the background of the first frame of image to be tracked, stopping the tracking operation.

In a second aspect, an embodiment of the present application provides a target tracking apparatus applied to an electronic device, where the apparatus includes:

the device comprises an acquisition unit, a tracking unit and a tracking unit, wherein the acquisition unit is used for acquiring an image sequence to be tracked, the image sequence to be tracked comprises a plurality of frames of images to be tracked, and each image to be tracked corresponds to one frame number;

a determining unit, configured to determine, based on a tracking frame in a first to-be-tracked image of the to-be-tracked image sequence, whether a region where a tracking object is located is a large-area flat region or whether a background of the tracking object and a background of the first to-be-tracked image are similar;

the tracking unit is used for tracking the tracking object if the area where the tracking object is located is not a large-area flat area or the tracking object is not similar to the background of the first frame of image to be tracked; and if the area where the tracking object is located is a large-area flat area or the tracking object is similar to the background of the first frame of image to be tracked, stopping the tracking operation.

In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing steps in the method according to the first aspect of the embodiment of the present application.

In a fourth aspect, the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program for electronic data exchange, where the computer program makes a computer perform some or all of the steps described in the method according to the first aspect of the present application.

In a fifth aspect, the present application provides a computer program product, where the computer program product includes a non-transitory computer-readable storage medium storing a computer program, where the computer program is operable to cause a computer to perform some or all of the steps described in the method according to the first aspect of the present application. The computer program product may be a software installation package.

It can be seen that, in the embodiment of the present application, first, an image sequence to be tracked is obtained, where the image sequence to be tracked includes multiple frames of images to be tracked, and each image to be tracked corresponds to one frame number; then, determining whether the area where the tracking object is located is a large-area flat area or whether the background of the tracking object and the background of the first frame image to be tracked are similar or not based on a tracking frame in the first frame image to be tracked of the image sequence to be tracked; if the area where the tracking object is located is not a large-area flat area or the tracking object is not similar to the background of the first frame of image to be tracked, tracking the tracking object; if the area where the tracking object is located is a large-area flat area or the tracking object is similar to the background of the first frame of image to be tracked, stopping the tracking operation; before the tracking object is tracked, whether the area where the tracking object is located is a large-area flat area or whether the tracking object is similar to the background of the first frame of image to be tracked is determined, and the situations that the tracking object is shifted and unstable due to the fact that the tracking object and the background in the image to be tracked are not easily distinguished can be eliminated, so that the accuracy of target tracking is improved.

These and other aspects of the present application will be more readily apparent from the following description of the embodiments.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1A is a schematic flowchart of a target tracking method according to an embodiment of the present disclosure;

fig. 1B is a schematic structural diagram of q pre-detection regions according to an embodiment of the present disclosure;

FIG. 2 is a schematic flow chart of a target pre-detection algorithm provided in an embodiment of the present application;

FIG. 3 is a schematic flow chart of an anti-drift and anti-misdetection algorithm provided in an embodiment of the present application;

fig. 4 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

fig. 5 is a schematic structural diagram of a target tracking device according to an embodiment of the present disclosure.

Detailed Description

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The following are detailed below.

The terms "first," "second," "third," and "fourth," etc. in the description and claims of this application and in the accompanying drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

Hereinafter, some terms in the present application are explained to facilitate understanding by those skilled in the art.

The electronic device according to the embodiment of the present application may include various handheld devices, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to a wireless modem, which have wireless communication functions, and various forms of User Equipment (UE), Mobile Stations (MS), terminal devices (terminal device), and the like. For convenience of description, the above-mentioned devices are collectively referred to as electronic devices.

The tracking of moving objects is one of the subjects of common research in the field of computer vision, and plays an important role in intelligent transportation, video monitoring, medicine and the like. Current object tracking techniques typically require high accuracy, robustness and real-time performance in specific applications. The accuracy mainly refers to the accuracy of target detection and the accuracy in the target tracking process. Robustness means that uninterrupted target tracking is realized under the condition of background change, for example, long-time monitoring is needed in the video monitoring process. Real-time performance is a high requirement on the execution speed of the algorithm to ensure real-time and high-speed system monitoring. In practical application, the target tracking algorithm has the problems of scale change, form change, occlusion and disappearance, image blurring and the like, and has a certain difference from the ideal target tracking with high accuracy and robustness.

An algorithm commonly adopted for tracking the moving target is a mean shift algorithm, and the mean shift algorithm is a non-parameter estimation algorithm based on density gradient rise. The method comprises the following main algorithm steps: (1) determining the position, shape, size and characteristics of an initial frame; (2) calculating characteristic information of the initial frame as a target sample; (3) and comparing the information of the current frame tracking frame with the initial frame, calculating an offset mean value, and continuously moving by taking the offset mean value as a starting point until a certain condition is met.

The target tracking algorithm which takes the mean shift algorithm as a motion model and initial frame information calculated by the weight and characteristic probability distribution histogram algorithm as an appearance model can realize real-time tracking and rapid convergence, and can complete efficient and continuous target tracking tasks in simulation and simple scenes. However, in a complex scene or in actual application, drift and false detection are usually caused, the tracking of the initial object cannot be accurately completed, and even if the tracking continuity is maintained, the final result is still wrong and invalid.

In order to improve the accuracy of target tracking, the application provides a target tracking method, which includes the following two-part algorithm, and the two-part algorithm is implemented in the initial frame and the tracking process of target tracking respectively:

firstly, the initial detection of a tracking object is added in the target tracking taking a mean shift algorithm as a core, and the tracking object and a background are detected in an initial frame, so that the condition of a flat area which is not suitable for the mean shift algorithm and is not easily distinguished from the background is eliminated, and the easy shifting and unstable result of the tracking object in the subsequent tracking process caused by the flat area is avoided;

secondly, an anti-drift algorithm is added for the problem of object drift which is similar to or identical with the tracking object, so that continuous ineffective tracking caused by the drift of the tracking object is avoided.

The following examples are provided for the purpose of illustrating the present invention.

Referring to fig. 1A, fig. 1A is a schematic flowchart of a target tracking method according to an embodiment of the present application, where the method includes:

step 101: and acquiring an image sequence to be tracked, wherein the image sequence to be tracked comprises a plurality of frames of images to be tracked, and each image to be tracked corresponds to one frame number.

And 102, determining whether the area where the tracking object is located is a large-area flat area or whether the background of the tracking object and the background of the first frame image to be tracked are similar or not based on the tracking frame in the first frame image to be tracked of the image sequence to be tracked.

If not, go to step 103;

if yes, go to step 104.

And 103, tracking the tracking object.

And 104, stopping the tracking operation.

The multi-frame is at least two frames, and the first frame to be tracked of the image sequence to be tracked may be the 0 th frame, may also be the 1 st frame, and may also be other frames, which are determined based on the naming rule of the user, and are not limited herein.

The tracking frame in the first frame of image to be tracked is preset, and can be acquired before the image sequence to be tracked is acquired, or acquired after the image sequence to be tracked is acquired, or acquired simultaneously with the image sequence to be tracked.

Further, the tracking object is determined based on the image area framed by the tracking frame, and a specific implementation manner of determining the tracking object based on the image area framed by the tracking frame is as follows: and extracting the target image characteristics of the image area framed by the tracking frame, and determining an object associated with the target image characteristics as a tracking object if the target image characteristics are matched with preset image characteristics.

In an embodiment of the present application, the determining, based on a tracking frame in a first frame of image to be tracked in the sequence of images to be tracked, whether a region where a tracking object is located is a large-area flat region or whether a background of the tracking object and a background of the first frame of image to be tracked are similar includes:

determining q pre-detection areas based on a tracking frame in a first frame image to be tracked of the image sequence to be tracked, wherein each pre-detection area corresponds to a sequence number, and q is an integer greater than 1;

determining the maximum brightness, the minimum brightness and the average brightness of a first pre-detection area, wherein the serial number of the first pre-detection area is the minimum serial number of the q pre-detection areas, and the first pre-detection area is an image area selected by the tracking frame;

if the first difference value between the maximum brightness of the first pre-detection area and the minimum brightness of the first pre-detection area is larger than a first threshold value, determining that the area where the tracking object is located is not a large-area flat area or the tracking object is not similar to the background of the first frame of image to be tracked;

if the first difference is smaller than or equal to the first threshold, determining a first target confidence coefficient based on q-1 pre-detection regions except the first pre-detection region, wherein the first target confidence coefficient is used for indicating that the region where the tracking object is located is a large-area flat region or the confidence degree that the tracking object is similar to the background of the first frame of image to be tracked;

and determining whether the area where the tracking object is located is a large-area flat area or whether the tracking object is similar to the background of the first frame of image to be tracked or not based on the first target confidence.

Wherein q may be 2, 4, 6, 8, or other values; the sequence number of the first pre-detection area can be 0 or 1, and is determined based on a naming rule of a user; the q-1 pre-detection areas except the first pre-detection area are all larger than the image area framed by the tracking frame; the q pre-detection areas can be rectangular or circular; are not limited herein.

Further, a specific implementation manner of determining q pre-detection regions based on the tracking frame in the first image to be tracked of the image sequence to be tracked is as follows: determining a tracking object based on an image area framed by a tracking frame in the first frame of image to be tracked of the image sequence to be tracked; determining a geometric center of the tracked object; and determining the q pre-detection areas by taking the geometric center as the center of the q pre-detection areas and taking preset q side lengths as the target side length.

Further, a specific implementation manner of determining whether the region where the tracking object is located is a large-area flat region or whether the tracking object is similar to the background of the first frame of image to be tracked based on the first target confidence coefficient is as follows: determining whether the first target confidence is greater than or equal to a third threshold; if so, determining that the area where the tracking object is located is a large-area flat area or the tracking object is similar to the background of the first frame of image to be tracked; if not, determining that the area where the tracking object is located is not a large-area flat area or the tracking object is not similar to the background of the first frame of image to be tracked.

For example, as shown in fig. 1B, fig. 1B is a schematic structural diagram of q pre-detection regions provided in this embodiment of the present application, as shown in fig. 1B, a geometric center of a tracked object is O, 3 pre-detection regions determined based on O are a first pre-detection region (serial number 0), a second pre-detection region (serial number 1) and a third pre-detection region (serial number 2), respectively, the first pre-detection region is smaller than the second pre-detection region, the second pre-detection region is smaller than the third pre-detection region, if a maximum luminance of the first pre-detection region is Max L um0, a minimum luminance is Min L um0, and an average luminance is Avg L um0, a threshold is adopted for judgment, when Max L um 0-Min L um0> thrd0, it is determined that the region where the tracked object is not a flat region or the tracked object is tracked over a large area and the background of the tracked object is not similar to the first pre-detection region, wherein the tracked object is tracked by a drift detection algorithm 0.

It can be seen that, in the embodiment of the present application, before performing target tracking, a difference between the maximum brightness and the minimum brightness of the determined first pre-detection region is determined by a threshold method, and if the difference is greater than a first threshold, it is determined that a region where a tracking object is located is not a large-area flat region or the tracking object is not similar to a background of the first frame of image to be tracked; if the target tracking confidence coefficient is smaller than the first threshold value, whether the area where the tracking object is located is a large-area flat area or whether the tracking object is similar to the background of the first frame of image to be tracked is further judged through the first target confidence coefficient, so that the situations that the tracking object is shifted and unstable due to the fact that the tracking object and the background in the image to be tracked are not easily distinguished can be eliminated, and the target tracking accuracy is improved.

In an embodiment of the present application, the determining a first target confidence based on q-1 pre-detection regions other than the first pre-detection region includes:

s1: determining the maximum brightness, the minimum brightness and the average brightness of a second pre-detection area, wherein the serial number of the second pre-detection area is the serial number of a last pre-detection area plus 1, and the last pre-detection area is a pre-detection area which has a serial number smaller than that of the second pre-detection area and is adjacent to the serial number of the second pre-detection area;

s2: if the average brightness of the second pre-detection area is greater than the minimum brightness of the first pre-detection area and less than the maximum brightness of the first pre-detection area, increasing a first confidence coefficient, and determining whether a second difference value between the average brightness of the second pre-detection area and the average brightness of the first pre-detection area is within a first error allowable range;

if the average brightness of the second pre-detection area is less than or equal to the minimum brightness of the first pre-detection area, or greater than or equal to the maximum brightness of the first pre-detection area, determining whether a second difference between the average brightness of the second pre-detection area and the average brightness of the first pre-detection area is within the first error allowable range;

s3: if the second difference value is within the first error allowable range, increasing a first confidence coefficient;

if the second difference is not within the first error tolerance range, executing steps S1-S3 until the sequence number of the second pre-detection area is the maximum sequence number of the q pre-detection areas, and obtaining a first target confidence.

The initial value of the first target confidence is0, 5%, 10%, 15%, or other values, which is not limited herein; the first confidence is a preset value added to the first target confidence each time, and may be 5%, 10%, 15%, or other values, which is not limited herein; and after the first confidence coefficient is increased to obtain a first target confidence coefficient, taking the obtained new first target confidence coefficient as the first target confidence coefficient as an initial value of the next increase.

For example, as shown in fig. 1B, the second pre-detection area has a maximum brightness of Max um, a minimum brightness of Min um and an average brightness of Avg 0um, the third pre-detection area has a maximum brightness of Max 1um, a minimum brightness of Min 2um and an average brightness of Avg 3um, if Min 4um < Avg 5um < Max 6um, the target object is obtained by increasing 5%, and then determining whether Avg 7 um-Avg 8um is within a first error tolerance range (-k, k), if Min 9um > Avg um or Avg 0um > Max um, directly determining whether Avg um-Avg um is within (-k, k), if Avg um-Avg um is within (-k, k), increasing 5%, and then replacing the second pre-detection area with the third pre-detection area, executing the above steps, and if Avg um-Avg um is not within (-k, the target object is obtained by directly executing the third pre-detection area, and executing the above steps.

It can be seen that, in the embodiment of the present application, by calculating the first target confidence accumulated in q pre-detection regions, and then based on whether the region where the first target confidence tracking object is located is a large-area flat region or whether the tracking object is similar to the background of the first frame of image to be tracked, the situations that the tracking object drifts and is unstable due to the fact that the tracking object in the image to be tracked is not easily distinguished from the background can be eliminated, and the accuracy of target tracking is further improved.

In an embodiment of the present application, the tracking object includes:

if the tracking object is stably tracked in continuous p frames of images to be tracked by using the tracking frame through a mean shift algorithm, determining whether the position of the tracking object in the p frame of images to be tracked has shift or not through an anti-shift anti-error detection algorithm, wherein the sequence of images to be tracked comprises the p frames of images to be tracked, the p frames of images to be tracked comprise the p frame of images to be tracked, and p is an integer greater than 1;

if the position of the tracking object in the p-th frame of image to be tracked has drift, tracking the tracking object based on the position of the tracking object in the p-1-th frame of image to be tracked through the mean shift algorithm;

if the position of the target object in the image to be tracked in the p-th frame does not drift, tracking the tracking object based on the position of the tracking object in the image to be tracked in the p-th frame by the mean shift algorithm.

Further, if the displacement of the tracking object in each frame of image to be tracked in the continuous p frames of image to be tracked is less than or equal to a fourth threshold value, and/or the position relations of the tracking object in each two adjacent frames of image to be tracked in the continuous p frames of image to be tracked are intersected, it is determined that the tracking object is stably tracked in the continuous p frames of image to be tracked by using the tracking frame through a mean shift algorithm.

It can be seen that, in the embodiment of the present application, it is determined whether the position of the tracking object in the p-th image to be tracked has a drift through an anti-drift false detection algorithm, and if the position of the tracking object in the previous image to be tracked has a drift, the tracking object is tracked based on the position of the tracking object in the previous image to be tracked through a mean shift algorithm; if the image to be tracked does not have drift, tracking the tracked object based on the position of the tracked object in the current frame image to be tracked through a mean shift algorithm; therefore, the situation that the situation of the position drift of the tracked object is not suitable for the mean shift algorithm can be eliminated, and the accuracy of target tracking is improved.

In an embodiment of the present application, the determining, by an anti-drift false detection prevention algorithm, whether there is drift in the position of the tracking object in the p-th image to be tracked includes:

determining a first position of the tracking object in the p-1 th frame to-be-tracked image, and determining a second position of the tracking object in the p-1 th frame to-be-tracked image, wherein the first position is represented by a first ellipse with the circle center being (PreCenX, PreCenY), the minor axis being PreAxis0 and the major axis being PreAxis1, the second position is represented by an ellipse with the circle center being (CurCenX, CurCenY), the minor axis being CurAxis0 and the major axis being CurAxis1, and the first position and the second position are determined based on the mean shift algorithm;

if the third difference value between the PreAxis0 and the CurAxis0 is within a second allowable error range, and the fourth difference value between the PreAxis1 and the CurAxis1 is within a third allowable error range, determining whether the position of the tracking object in the p frame image to be tracked has drift;

and if the third difference is not within the second error allowable range and/or the fourth difference is not within the third error allowable range, determining that the position of the tracking object in the p frame image to be tracked has drift.

It can be seen that, in the embodiment of the present application, if a third difference between a short axis representing a position of the tracking object in the p-1 th frame to-be-tracked image and a short axis representing a position of the tracking object in the p-1 th frame to-be-tracked image is not within a second allowable error range, and/or a fourth difference between a long axis representing a position of the tracking object in the p-1 th frame to-be-tracked image and a long axis representing a position of the tracking object in the p-th frame to-be-tracked image is not within a third allowable error range, it is determined that there is a drift in a position of the tracking object in the p-th frame to-be-tracked image; when the position of the tracking object in the p-th frame of image to be tracked has drift, the tracking object is tracked based on the position of the tracking object in the previous frame of image to be tracked, so that the situation of the drift of the position of the tracking object is not suitable for the mean shift algorithm can be eliminated, and the accuracy of target tracking is further improved.

In an embodiment of the present application, the determining whether there is a drift in the position of the tracking object in the p-th image to be tracked includes:

determining a first average displacement of the tracking object in the p frames of images to be tracked and determining a first displacement of the tracking object in the p-th frame of images to be tracked;

if the fifth difference value of the first average displacement and the first displacement is within a fourth error allowable range, increasing a second confidence coefficient to obtain a second target confidence coefficient; if the fifth difference value is not within the fourth error allowable range, determining that the position of the target object in the p-th image to be tracked has a drift, where the second target confidence is used to indicate a confidence level that the position of the tracking object in the p-th image to be tracked has a drift;

if the second target confidence is greater than a second threshold, determining whether the position of the tracking object in the p-th frame of image to be tracked has drift or not based on a target position relationship, where the target position relationship is the position relationship between the tracking object in the p-1-th frame of image to be tracked and the tracking object in the p-th frame of image to be tracked.

Wherein the first displacement and the first average displacement may be determined by p circle centers representing positions of the tracked object in the image to be tracked.

In a first implementation, the first displacement is determined by a first equation as follows:

wherein the dis is a first displacement. It should be noted that p-1 second displacements of the tracked object in the p frames of images to be tracked can be respectively calculated through a first formula, and then the average displacement of the p-1 second displacements is obtained to obtain the first average displacement.

In a second implementation, the first displacement includes a first horizontal displacement and a first vertical displacement, and the first displacement is determined by a second equation as follows:

X＝PreCenX-CurCenX；Y＝PreCenY-CurCenY；

wherein X is the first horizontal displacement and Y is the first vertical displacement. It should be noted that p-1 second horizontal displacements and p-1 second vertical displacements of the tracked object in the p frames of images to be tracked can be calculated respectively through a second formula, and then the average horizontal displacement and the average vertical displacement of the p-1 second horizontal displacements and the p-1 second vertical displacements are obtained to obtain the first average displacement.

The initial value of the second target confidence is0, 5%, 10%, 15%, or other values, which is not limited herein; the second confidence is a preset value added to the second target confidence each time, and may be 5%, 10%, 15%, or other values, which is not limited herein; and after the second confidence degree is increased to obtain a second target confidence degree, taking the obtained new second target confidence degree as the initial value of the next increase.

It can be seen that, in the embodiment of the present application, the first average displacement of the tracking object in the p frames of images to be tracked is compared with the first displacement of the tracking object in the p th frame of images to be tracked, and when the difference between the first average displacement and the first displacement of the tracking object in the p th frame of images to be tracked is not within the fourth error tolerance range, it is determined that there is a drift in the position of the target object in the p th frame of images to be tracked; when the position of the tracking object in the p-th frame of image to be tracked has drift, the tracking object is tracked based on the position of the tracking object in the previous frame of image to be tracked, so that the situation of the drift of the position of the tracking object is not suitable for the mean shift algorithm can be eliminated, and the accuracy of target tracking is further improved.

In an embodiment of the present application, the determining whether there is a drift in the position of the target object in the p-th image to be tracked based on the target position relationship includes:

if the target position relationship is the phase separation, determining that the position of the target object in the p-th frame of image to be tracked has drift;

the target position relationship is determined based on the circle center distance between the first ellipse and the second ellipse and the average axial length of the first ellipse and the second ellipse, and if the circle center distance is larger than the average axial length, the target position relationship is determined to be a separation.

Further, the method further comprises: and determining the target position relation.

Specifically, a specific implementation manner of determining the target position relationship is as follows: determining an average axial length representing the first ellipse and the second ellipse based on a third formula; and if the distance between the centers of the first ellipse and the second ellipse is greater than the average axial length, determining that the target position relationship is a phase-to-phase relationship.

Wherein a circle center distance of the first ellipse from the second ellipse is determined based on the first formula.

The third formula is as follows:

axis_avg＝(CurAxis0+CurAxis1+PreAxis0+PreAxis1)/4

wherein, said axis_avgIs the average axial length.

It can be seen that, in the embodiment of the present application, assuming that the movement of the tracking object between each frame of the objects to be tracked is a small-range movement, if the target position relationship is a phase separation, it may be determined that there is a drift in the position of the target object in the p-th frame of the images to be tracked; when the position of the tracking object in the p-th frame of image to be tracked has drift, the tracking object is tracked based on the position of the tracking object in the previous frame of image to be tracked, so that the situation of the drift of the position of the tracking object is not suitable for the mean shift algorithm can be eliminated, and the accuracy of target tracking is further improved.

Referring to fig. 2, fig. 2 is a schematic flowchart of a target pre-detection algorithm provided in an embodiment of the present application, where the target pre-detection algorithm includes:

step 201: and acquiring an image sequence to be tracked, wherein the image sequence to be tracked comprises a plurality of frames of images to be tracked, and each image to be tracked corresponds to one frame number.

Step 202, determining q pre-detection areas based on a tracking frame in the first frame image to be tracked of the image sequence to be tracked, wherein each pre-detection area corresponds to a sequence number, and q is an integer greater than 1.

And 203, determining the maximum brightness, the minimum brightness and the average brightness of a 0 th pre-detection area, wherein the serial number of the 0 th pre-detection area is the minimum serial number of the q pre-detection areas, and the 0 th pre-detection area is the image area framed and selected by the tracking frame.

Step 204, determining whether a first difference value between the maximum brightness of the 0 th pre-detection area and the minimum brightness of the 0 th pre-detection area is greater than a first threshold value.

If yes, go to step 215;

if not, go to step 205.

Step 205: and determining the maximum brightness, the minimum brightness and the average brightness of the ith pre-detection area, wherein the initial value of i is 1.

Step 206: and determining whether the average brightness of the ith pre-detection area is greater than the minimum brightness of the 0 th pre-detection area and less than the maximum brightness of the 0 th pre-detection area.

If yes, go to step 207;

if not, go to step 208.

Step 207: the first confidence level is increased and then step 208 is performed.

Step 208: and determining whether a second difference value between the average brightness of the ith pre-detection area and the average brightness of the 0 th pre-detection area is within a first error allowable range.

If yes, go to step 209;

if not, go to step 210.

Step 209: the first confidence level is increased and then step 210 is performed.

Step 210: determining whether said i is equal to q-1;

if yes, go to step 212;

if not, go to step 211.

Step 211: i +1 is assigned to said i and step 205 is then executed.

Step 212: a first target confidence is determined.

Step 213: determining whether the first target confidence is greater than or equal to a third threshold;

if yes, go to step 214;

if not, go to step 215.

Step 214: and determining that the area where the tracking object is located is a large-area flat area or the tracking object is similar to the background of the first frame of image to be tracked.

Step 215: and determining that the area where the tracking object is located is not a large-area flat area or the tracking object is not similar to the background of the first frame of image to be tracked.

It should be noted that, for the specific implementation process of the present embodiment, reference may be made to the specific implementation process described in the above method embodiment, and a description thereof is omitted here.

Referring to fig. 3, fig. 3 is a schematic flow chart of an anti-drift and anti-error detection algorithm provided in the embodiment of the present application, where the anti-drift and anti-error detection algorithm includes:

step 301: if the tracking object is stably tracked in p frames of images to be tracked continuously by using the tracking frame through a mean shift algorithm, determining a first position of the tracking object in the p-1 frame of images to be tracked, the first position being represented by a first ellipse with a center of (PreCenX, PreCenY), a short axis of PreAxis0 and a long axis of PreAxis1, and determining a second position of the tracking object in the p frame of images to be tracked, the second position being represented by an ellipse with a center of (CurCenX, CurCenY), a short axis of CurAxis0 and a long axis of CurAs 1, the first position and the second position being determined based on the mean shift algorithm.

Step 302: determining whether a third difference between PreAxis0 and CurAxis0 is within a second allowable range of error, and a fourth difference between PreAxis1 and CurAxis1 is within a third allowable range of error;

if yes, go to step 303;

if not, go to step 309.

Step 303: determining a first average displacement of the tracking object in the p frames of images to be tracked, and determining a first displacement of the tracking object in the p-th frame of images to be tracked.

Step 304: determining whether a fifth difference of the first average displacement and the first displacement is within a fourth error tolerance;

if yes, go to step 305;

if not, go to step 309.

Step 305: and increasing a second confidence coefficient to obtain a second target confidence coefficient, wherein the second target confidence coefficient is used for representing the credibility that the position of the tracking object in the image to be tracked in the p-th frame is drifted.

Step 306: determining whether the second target confidence is greater than a second threshold;

if yes, go to step 307;

if not, go to step 309.

Step 307: and determining the target position relation between the tracking object in the image to be tracked of the p-1 th frame and the tracking object in the image to be tracked of the p-th frame.

Step 308: determining that the target position relationship is a phase separation.

Step 309: determining that the position of the target object in the p-th image to be tracked has drift.

In accordance with the embodiments shown in fig. 1A, fig. 2 and fig. 3, please refer to fig. 4, fig. 4 is a schematic structural diagram of an electronic device provided in an embodiment of the present application, and as shown in fig. 4, the electronic device includes a processor, a memory, a communication interface and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for performing the following steps:

In an embodiment of the present application, in determining whether a region where a tracking object is located is a large-area flat region or whether a background of the tracking object and a background of a first frame image to be tracked is similar based on a tracking frame in the first frame image to be tracked in the image sequence to be tracked, the program includes instructions specifically configured to:

In an embodiment of the application, in determining the first target confidence level based on q-1 pre-detected regions other than the first pre-detected region, the program comprises instructions specifically for performing the steps of:

In an embodiment of the application, in tracking the tracked object, the program includes instructions for performing the following steps:

In an embodiment of the present application, in determining whether there is a drift in the position of the tracking object in the p-th image to be tracked through an anti-drift false detection prevention algorithm, the program includes instructions specifically configured to:

In an embodiment of the present application, in determining whether there is a drift in the position of the tracking object in the image to be tracked in the p-th frame, the program includes instructions specifically configured to:

In an embodiment of the present application, in determining whether there is a drift in the position of the target object in the p-th image to be tracked based on the target position relationship, the program includes instructions specifically configured to:

In the embodiment of the present application, the electronic device may be divided into the functional units according to the method example, for example, each functional unit may be divided corresponding to each function, or two or more functions may be integrated into one processing unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit. It should be noted that the division of the unit in the embodiment of the present application is schematic, and is only a logic function division, and there may be another division manner in actual implementation.

The following is an embodiment of the apparatus of the present application, which is used to execute the method implemented by the embodiment of the method of the present application. Referring to fig. 5, fig. 5 is a schematic structural diagram of a target tracking apparatus applied to an electronic device according to an embodiment of the present application, the apparatus including:

an obtaining unit 501, configured to obtain an image sequence to be tracked, where the image sequence to be tracked includes multiple frames of images to be tracked, and each image to be tracked corresponds to one frame number;

a determining unit 502, configured to determine, based on a tracking frame in a first frame to-be-tracked image of the sequence of images to be tracked, whether a region where a tracking object is located is a large-area flat region or whether a background of the tracking object and a background of the first frame to-be-tracked image are similar;

a tracking unit 503, configured to track the tracked object if the area where the tracked object is located is not a large-area flat area or the tracked object is not similar to the background of the first frame of image to be tracked; and if the area where the tracking object is located is a large-area flat area or the tracking object is similar to the background of the first frame of image to be tracked, stopping the tracking operation.

In an embodiment of the application, in terms of determining whether a region where a tracking object is located is a large-area flat region or whether a background of the tracking object and a background of a first frame image to be tracked are similar based on a tracking frame in the first frame image to be tracked in the sequence of images to be tracked, the determining unit 502 is specifically configured to:

In an embodiment of the present application, in determining the first target confidence based on q-1 pre-detection regions except the first pre-detection region, the determining unit 502 is specifically configured to:

In an embodiment of the present application, in tracking the tracking object, the tracking unit 503 is specifically configured to:

In an embodiment of the present application, in determining whether there is a drift in the position of the tracking object in the p-th image to be tracked through an anti-drift anti-misdetection algorithm, the tracking unit 503 is specifically configured to:

In an embodiment of the present application, in determining whether there is a drift in the position of the tracking object in the image to be tracked in the p-th frame, the tracking unit 503 is specifically configured to:

In an embodiment of the present application, in determining whether there is a drift in the position of the target object in the p-th image to be tracked based on the target position relationship, the tracking unit 503 is specifically configured to:

It should be noted that the obtaining unit 501, the determining unit 502, and the tracking unit 503 may be implemented by a processor.

Embodiments of the present application also provide a computer storage medium, where the computer storage medium stores a computer program for electronic data exchange, the computer program enabling a computer to execute part or all of the steps of any one of the methods described in the above method embodiments, and the computer includes an electronic device.

Embodiments of the present application also provide a computer program product comprising a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods as described in the above method embodiments. The computer program product may be a software installation package, the computer comprising an electronic device.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.

In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the above-described division of the units is only one type of division of logical functions, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of some interfaces, devices or units, and may be an electric or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit may be stored in a computer readable memory if it is implemented in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a memory, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the above-mentioned method of the embodiments of the present application. And the aforementioned memory comprises: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable memory, which may include: flash Memory disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

The foregoing detailed description of the embodiments of the present application has been presented to illustrate the principles and implementations of the present application, and the above description of the embodiments is only provided to help understand the method and the core concept of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A target tracking method is applied to an electronic device, and comprises the following steps:

2. The method according to claim 1, wherein the determining whether a region where a tracking object is located is a large-area flat region or whether the background of the tracking object and the background of the first frame image to be tracked are similar based on a tracking frame in the first frame image to be tracked in the sequence of images to be tracked comprises:

3. The method of claim 2, wherein determining a first target confidence based on q-1 pre-detected regions other than the first pre-detected region comprises:

4. The method according to any one of claims 1-3, wherein said tracking said tracked object comprises:

5. The method according to claim 4, wherein the determining whether there is a drift in the position of the tracking object in the p-th image to be tracked through an anti-drift false detection prevention algorithm comprises:

determining a first position of the tracking object in the p-1 th frame to-be-tracked image, and determining a second position of the tracking object in the p-1 th frame to-be-tracked image, wherein the first position is represented by a first ellipse with a circle center of (PreCenX, PreCenY), a short axis of PreAxis0 and a long axis of PreAxis1, the second position is represented by an ellipse with a circle center of (CurCenX, CurCenY), a short axis of CurAxis0 and a long axis of CurAxis1, and the first position and the second position are determined based on the mean shift algorithm;

6. The method of claim 5, wherein the determining whether there is a drift in the position of the tracking object in the p-th image to be tracked comprises:

7. The method according to claim 6, wherein the determining whether there is a drift in the position of the target object in the p-th image to be tracked based on the target position relationship comprises:

8. An object tracking device applied to an electronic device, the device comprising:

9. An electronic device comprising a processor, a memory, a communication interface, and one or more programs stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps in the method of any of claims 1-7.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which is executed by a processor to implement the method of any one of claims 1-7.