CN111696132B - Target tracking method, device, computer readable storage medium and robot - Google Patents
Target tracking method, device, computer readable storage medium and robot Download PDFInfo
- Publication number
- CN111696132B CN111696132B CN202010413963.XA CN202010413963A CN111696132B CN 111696132 B CN111696132 B CN 111696132B CN 202010413963 A CN202010413963 A CN 202010413963A CN 111696132 B CN111696132 B CN 111696132B
- Authority
- CN
- China
- Prior art keywords
- target
- tracking
- response matrix
- threshold value
- confidence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000001514 detection method Methods 0.000 claims abstract description 58
- 230000008569 process Effects 0.000 claims abstract description 40
- 230000004044 response Effects 0.000 claims description 86
- 239000011159 matrix material Substances 0.000 claims description 60
- 230000006870 function Effects 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 22
- 238000012549 training Methods 0.000 claims description 19
- 230000000694 effects Effects 0.000 abstract description 6
- 238000004422 calculation algorithm Methods 0.000 description 26
- 238000010586 diagram Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000007774 longterm Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000739 chaotic effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The application belongs to the technical field of robots, and particularly relates to a target tracking method, a target tracking device, a computer-readable storage medium and a robot. The method comprises the steps of determining a search area according to position coordinates of a target and a detection frame; tracking the target in the search area to obtain the position coordinates and the confidence of the target; and ending the current tracking process when the confidence coefficient is smaller than a preset first threshold value. According to the method and the device, the confidence coefficient is introduced to evaluate the tracking effect in the tracking process of the target, when the confidence coefficient is lower, namely smaller than the preset first threshold value, the target is indicated to be lost, and at the moment, the current tracking process can be ended, so that the situation that the current tracking process is still continued after the target is lost is avoided. When the confidence coefficient is higher, namely, the confidence coefficient is larger than a preset second threshold value, the parameters of the related filter are updated, so that the error background information is prevented from being learned, and tracking of the target is kept.
Description
Technical Field
The application belongs to the technical field of robots, and particularly relates to a target tracking method, a target tracking device, a computer-readable storage medium and a robot.
Background
As a robot for assisting a human to complete a security work, a security robot is required to have the capability of detecting and tracking a target that may appear in a security place. However, the high complexity/low real-time nature of the detection algorithm and the susceptibility to occlusion limit the real-time tracking capabilities of the security robot to targets that appear within the security site, and thus the target tracking algorithm should be enabled after the target is detected. At present, a tracking algorithm of a target has the advantages of high instantaneity, certain shielding resistance and the like, but due to the limitations of the tracking algorithm, the current tracking algorithm cannot judge whether the target is lost, so that the situation that the current tracking process is still continued after the target is lost is caused. Moreover, when the related filter is trained in real time by the current tracking algorithm, as time goes by, deformation and shielding experienced by the target are more and more, the characteristics learned by the algorithm comprise more and more background information and more chaotic conditions, and the error background information can be learned, so that tracking of the target can be gradually lost.
Disclosure of Invention
In view of this, the embodiments of the present application provide a target tracking method, apparatus, computer readable storage medium, and robot, so as to solve the problem that the current tracking algorithm cannot determine whether the target is lost, so that the current tracking process is still continued after the target is lost, and tracking of the target is gradually lost over time.
A first aspect of an embodiment of the present application provides a target tracking method, which may include:
determining a search area according to the position coordinates of the target and the detection frame;
tracking the target in the search area to obtain the position coordinates and the confidence of the target;
the confidence coefficient is smaller than a preset first threshold value, the current tracking process is ended, and the follow-up information is fed back to a preset control center;
the confidence coefficient is larger than a preset second threshold value, parameters of the related filter are updated, and the second threshold value is larger than the first threshold value.
Further, before the target is tracked in the search area, the method further includes:
performing target detection in a designated area, and determining initial coordinates and an initial detection frame of a detected target;
performing an initialization process, the initialization process comprising: the correlation filter is initialized.
Further, initializing the correlation filter includes:
initializing parameters of the correlation filter according to:
wherein,x is the image of m times size area of the initial detection frame taking the initial coordinate as the center, m is a preset multiple parameter, y is the Gaussian function of m times size area of the initial detection frame taking the initial coordinate as the center, and sigma is used as the label of the training correlation filter 2 Variance of Gaussian function, F (&) is Fourier transform, F -1 (. Cndot.) Fourier inverse transform, # is the penalty constant for training, # represents the matrix term-wise multiplication operation, and α is the parameter of the correlation filter.
Further, the tracking the target in the search area to obtain the position coordinates and the confidence of the target includes:
tracking the target in the search area by using a correlation filter to obtain a response matrix as follows:
CF＝F -1 (F(k<x,z>)⊙F(α))
wherein z is the image of the search area, and CF is the response matrix;
determining a maximum response value in the response matrix as the maximum response value, and determining a position coordinate corresponding to the maximum response value as the position coordinate;
and calculating the confidence coefficient according to the response matrix.
Further, the calculating the confidence coefficient according to the response matrix includes:
calculating the confidence according to the following formula:
r＝p max
wherein p is max R is the confidence level, which is the maximum response value in the response matrix;
or alternatively
Calculating the confidence according to the following formula:
wherein CF is as follows w,h The value of the W row and the H column in the response matrix is 1-W, wherein W is the number of rows of the response matrix, H is 1-H, H is the number of columns of the response matrix, and p min And the minimum response value in the response matrix.
Further, the target tracking method may further include:
and if the confidence coefficient is smaller than or equal to the second threshold value and larger than or equal to the first threshold value, the parameters of the related filter are not updated.
Further, the updating the parameters of the correlation filter includes:
updating parameters of the correlation filter according to:
α＝(1-γ)α+γα′
wherein,and x is a Gaussian function taking the position coordinate of the target as the center, taking the image of the m-time-sized region of the detection frame, y is a Gaussian function taking the position coordinate of the target as the center, taking the Gaussian function of the m-time-sized region of the detection frame as a label for training a correlation filter, and gamma is a weight constant for parameter updating.
A second aspect of embodiments of the present application provides a target tracking apparatus, which may include:
the searching area determining module is used for determining a searching area according to the position coordinates of the target and the detection frame;
the tracking module is used for tracking the target in the search area to obtain the position coordinates and the confidence of the target;
the tracking loss judging module is used for judging that the target is tracked and lost if the confidence coefficient is smaller than a preset first threshold value, ending the current tracking process and feeding back tracking loss information to a preset control center;
the model updating module is used for updating parameters of the related filter when the confidence coefficient is larger than a preset second threshold value, and the second threshold value is larger than the first threshold value.
Further, the target tracking apparatus may further include:
the initial detection module is used for detecting the target in the designated area and determining initial coordinates and an initial detection frame of the detected target;
an initialization module, configured to perform an initialization procedure, where the initialization procedure includes: the correlation filter is initialized.
Further, the initializing module may include:
a correlation filter initializing unit, configured to initialize parameters of the correlation filter according to the following formula:
wherein,x is the image of m times size area of the initial detection frame taking the initial coordinate as the center, m is a preset multiple parameter, y is the Gaussian function of m times size area of the initial detection frame taking the initial coordinate as the center, and sigma is used as the label of the training correlation filter 2 Variance of Gaussian function, F (&) is Fourier transform, F -1 (. Cndot.) Fourier inverse transform, # is the penalty constant for training, # represents the matrix term-wise multiplication operation, and α is the parameter of the correlation filter.
Further, the tracking module may include:
the response matrix calculation unit is used for tracking the target in the search area by using a correlation filter to obtain a response matrix shown as follows:
CF＝F -1 (F(k<x,z>)⊙F(α))
wherein z is the image of the search area, and CF is the response matrix;
a position coordinate determining unit configured to determine a maximum response value in the response matrix as the maximum response value, and determine a position coordinate corresponding to the maximum response value as the position coordinate;
and the confidence calculating unit is used for calculating the confidence according to the response matrix.
Further, the confidence calculating unit is specifically configured to:
calculating the confidence according to the following formula:
r＝p max
wherein p is max R is the confidence level, which is the maximum response value in the response matrix;
or alternatively
Calculating the confidence according to the following formula:
wherein CF is as follows w,h The value of the W row and the H column in the response matrix is 1-W, wherein W is the number of rows of the response matrix, H is 1-H, H is the number of columns of the response matrix, and p min And the minimum response value in the response matrix.
Further, the model updating module is specifically configured to update parameters of the correlation filter according to the following formula:
α＝(1-γ)α+γα′
wherein,x is an image of an area with m times of the size of the detection frame taking the position coordinate of the target as the center, and y isTaking the position coordinates of the target as the center, taking a Gaussian function of an area with m times of the size of the detection frame as a tag for training a related filter, and taking gamma as a weight constant for parameter updating.
A third aspect of the embodiments of the present application provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of any one of the target tracking methods described above.
A fourth aspect of the embodiments of the present application provides a robot comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of any one of the object tracking methods described above when executing the computer program.
A fifth aspect of the embodiments of the present application provides a computer program product for, when run on a robot, causing the robot to perform the steps of any one of the object tracking methods described above.
Compared with the prior art, the embodiment of the application has the beneficial effects that: according to the embodiment of the application, a search area is determined according to the position coordinates of the target and the detection frame; tracking the target in the search area to obtain the position coordinates and the confidence of the target; and ending the current tracking process when the confidence coefficient is smaller than a preset first threshold value. According to the embodiment of the application, the confidence coefficient is introduced to evaluate the tracking effect in the tracking process of the target, when the confidence coefficient is lower, namely smaller than the preset first threshold value, the target is indicated to be lost, and at the moment, the current tracking process can be ended, so that the situation that the current tracking process is still continued after the lost tracking is avoided. When the confidence coefficient is higher, namely, the confidence coefficient is larger than a preset second threshold value, the parameters of the related filter are updated, so that the error background information is prevented from being learned, and tracking of the target is kept.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly introduce the drawings that are needed in the embodiments or the description of the prior art, it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of an initialization process;
FIG. 2 is a flow chart of one embodiment of a target tracking method in the embodiments of the present application;
FIG. 3 is a schematic diagram of a correlation of a target detection frame and a search area;
FIG. 4 is a block diagram of one embodiment of a target tracking device according to an embodiment of the present application;
fig. 5 is a schematic block diagram of a robot in an embodiment of the present application.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the embodiments described below are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in this specification and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
In addition, in the description of the present application, the terms "first," "second," "third," etc. are used merely to distinguish between descriptions and are not to be construed as indicating or implying relative importance.
In embodiments of the present application, the targets to be tracked may include, but are not limited to: personnel, vehicles, animals, and other moving objects.
The existing target tracking algorithm is divided into a short-time tracking algorithm and a long-time tracking algorithm: the short-time tracking algorithm can track the target well in a short time (less than 2 minutes), and as the time goes by, the deformation and shielding of the target are more and more, the learning characteristics of the algorithm are more and more disordered, and the tracking of the target is gradually lost; in the existing long-term Tracking algorithm (2-20 minutes), besides the real-time Tracking and training of the tracker, a detector is also provided, the real-time training is also carried out in the Tracking process, the most classical long-term Tracking algorithm is the Tracking-Learning-Detection (TLD) algorithm, the Tracking algorithm is not easy to lose under the condition of long-term Tracking due to the fact that the detector is trained simultaneously, but the complexity of the detector is high, and the complexity of the algorithm is greatly improved due to redundant training. The detection algorithm capable of being trained in real time is poor in detection effect, short-time tracking effect is not good, and therefore the overall tracking effect is not satisfactory. Whether the short-time tracking algorithm or the long-time tracking algorithm exists, the target is possibly lost, but the problem that whether the target is lost or not cannot be judged by the algorithm exists.
The embodiment of the application introduces confidence on the basis of the short-time tracker to realize efficient tracking and tracking loss judgment of the target: the relevant filter learns the characteristics of the target, and outputs the pixel point coordinate with the largest response to the characteristics of the target in the current frame to track the target, wherein the response value of the current frame to the characteristics of the target can be taken as the confidence level. Similarly, the confidence level is judged in the tracking process, and when the confidence level of tracking is low, the target is lost, and the target needs to be re-detected.
It is easy to understand that the target detection is a precondition for performing target tracking, and in the initial state, the target detection needs to be performed in the designated area by the target detector, where the detection method used may be any detection method in the prior art, and the embodiment of the present application is not limited specifically.
When a target is detected in a certain frame image, the position coordinates of the target in the frame image and a detection frame (bb) are determined, and this position is marked as initial coordinates and this detection frame is marked as initial detection frame.
After the target detection is completed, an initialization process of target tracking may be performed, as shown in fig. 1, and may include:
step S101, initializing the value of k.
In the embodiment of the present application, k is an image frame number for tracking the target, and k is a positive integer. Regarding the frame image in which the target is detected as the 0 th frame, after the target detection is completed, the value of k may be initialized to 1, that is, the following steps are performed: k=1.
Step S102, initializing the correlation filter (Correlation Filter, CF).
Specifically, the parameters of the correlation filter are initialized according to the following equation:
wherein,x is the image of the m-times size area of the initial detection frame with the initial coordinate as the center, m is a preset multiple parameter, the specific value of the m is set according to practical situations, preferably, the x is set to 2.5, namely, the search area is 2.5 times of the size of the detection frame, y is the Gaussian function of the m-times size area of the initial detection frame with the initial coordinate as the center, and the Gaussian function is used as a label of a training correlation filter, sigma 2 Variance of Gaussian function, F (&) is Fourier transform, F -1 (. Cndot.) Fourier inverse transform, # is the penalty constant for training, # represents the matrix term-wise multiplication operation, and α is the parameter of the correlation filter.
In tracking the target, the specific tracking process of each frame image is similar, and a specific example of any frame (kth frame) is described in detail below. As shown in fig. 2, the process of tracking the target in the kth frame image may include:
step S201, determining a search area in the kth frame of image according to the position coordinates of the target in the kth-1 frame of image and the detection frame.
As shown in fig. 3, the rectangle of the broken line indicates the detection frame of the target tracked in the kth-1 frame, and the search area in the kth frame is m times as large as the detection frame, and the search area is centered on the position coordinate of the target in the kth-1 frame image as shown by the solid line frame in the figure.
And step S202, tracking the target in the search area to obtain the position coordinate and the confidence of the target in the kth frame of image.
In this embodiment, the target is tracked using a correlation filter. In other embodiments, the target tracking algorithm may be, but is not limited to, such as Kalman filtering, target detection tracking, deep learning based tracking algorithms, multi-target tracking algorithms, and the like.
Specifically, the target may be tracked in the search area using a correlation filter, resulting in a response matrix as shown below:
CF＝F -1 (F(k<x,z>)⊙F(α))
wherein z is the image of the search area, and CF is the response matrix.
In the embodiment of the present application, the maximum response value in the response matrix may be determined as the maximum response value, and the position coordinate corresponding to the maximum response value may be determined as the position coordinate of the target in the kth frame image.
In a specific implementation of the embodiment of the present application, the confidence level may be calculated according to the following formula:
r＝p max
wherein p is max And r is the confidence level, and is the maximum response value in the response matrix.
In another specific implementation of the embodiment of the present application, the confidence level may be calculated according to the following formula:
wherein CF is as follows w,h The value of the W row and the H column in the response matrix is 1-W, wherein W is the number of rows of the response matrix, H is 1-H, H is the number of columns of the response matrix, and p min And the minimum response value in the response matrix.
Step S203, executing an operation corresponding to the confidence level.
Specifically, if the confidence coefficient is smaller than a preset first threshold value, judging that the target is lost, and ending the current tracking process. In the process, the following information can be fed back to a preset control center.
And if the confidence coefficient is larger than a preset second threshold value, updating parameters of the related filter.
The second threshold is greater than the first threshold, and specific values of the first threshold and the second threshold may be set according to actual situations, which is not specifically limited in the embodiment of the present application.
In the embodiment of the present application, the parameters of the correlation filter may be updated according to the following equation:
α＝(1-γ)α+γα′
wherein,and x is an image taking the position coordinate of the target in the kth frame image as the center, taking the m-times-sized region of the detection frame in the kth frame image, y is a Gaussian function taking the m-times-sized region of the detection frame in the kth frame image as the center, taking the label of the training related filter, and gamma is a weight constant for parameter updating.
After updating the parameters of the relevant filter, if the current tracking process has not been completed, continuing to track the target in the next frame of image, namely executing: k=k+1 and the tracking step shown in fig. 2 is re-performed until the current tracking process ends.
If the confidence coefficient is smaller than or equal to the second threshold value and larger than or equal to the first threshold value, the parameters of the related filter are not updated any more, and if the current tracking process is not finished, the step of directly continuing to track the target in the next frame of image is executed, namely: k=k+1 and the tracking step shown in fig. 2 is re-performed until the current tracking process ends.
In summary, in the embodiment of the present application, the search area is determined according to the position coordinates and the detection frame of the target; tracking the target in the search area to obtain the position coordinates and the confidence of the target; if the confidence coefficient is smaller than a preset first threshold value, judging that the target is lost, and ending the current tracking process. By the embodiment of the application, the confidence coefficient is introduced in the tracking process of the target to evaluate the tracking effect, and when the confidence coefficient is lower, the target is indicated to be lost, and the current tracking process can be ended at the moment, so that the situation that the current tracking process is still continued after the target is lost is avoided. When the confidence coefficient is higher, namely, the confidence coefficient is larger than a preset second threshold value, the parameters of the related filter are updated, so that the error background information is prevented from being learned, and tracking of the target is kept.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic of each process, and should not limit the implementation process of the embodiment of the present application in any way.
Fig. 4 shows a block diagram of an embodiment of a target tracking apparatus according to an embodiment of the present application, corresponding to a target tracking method described in the foregoing embodiment.
In this embodiment, a target tracking apparatus may include:
a search area determining module 401, configured to determine a search area according to the position coordinates of the target and the detection frame;
a tracking module 402, configured to track the target in the search area, to obtain a position coordinate and a confidence coefficient of the target;
a tracking loss determination module 403, configured to end the current tracking process, where the confidence coefficient is smaller than a preset first threshold;
the model updating module 404 is configured to update parameters of the relevant filter when the confidence coefficient is greater than a preset second threshold, where the second threshold is greater than the first threshold.
Further, the target tracking apparatus may further include:
the initial detection module is used for detecting the target in the designated area and determining initial coordinates and an initial detection frame of the detected target;
an initialization module, configured to perform an initialization procedure, where the initialization procedure includes: the correlation filter is initialized.
Further, the initializing module may include:
a correlation filter initializing unit, configured to initialize parameters of the correlation filter according to the following formula:
wherein,x is the image of m times size area of the initial detection frame taking the initial coordinate as the center, m is a preset multiple parameter, y is the Gaussian function of m times size area of the initial detection frame taking the initial coordinate as the center, and sigma is used as the label of the training correlation filter 2 Variance of Gaussian function, F (&) is Fourier transform, F -1 (. Cndot.) Fourier inverse transform, # is the penalty constant for training, # represents the matrix term-wise multiplication operation, and α is the parameter of the correlation filter.
Further, the tracking module may include:
the response matrix calculation unit is used for tracking the target in the search area by using a correlation filter to obtain a response matrix shown as follows:
CF＝F -1 (F(k<x,z>)⊙F(α))
wherein z is the image of the search area, and CF is the response matrix;
a position coordinate determining unit configured to determine a maximum response value in the response matrix as the maximum response value, and determine a position coordinate corresponding to the maximum response value as the position coordinate;
and the confidence calculating unit is used for calculating the confidence according to the response matrix.
Further, the confidence calculating unit is specifically configured to:
calculating the confidence according to the following formula:
r＝p max
wherein p is max R is the confidence level, which is the maximum response value in the response matrix;
or alternatively
Calculating the confidence according to the following formula:
wherein CF is as follows w,h The value of the W row and the H column in the response matrix is 1-W, wherein W is the number of rows of the response matrix, H is 1-H, H is the number of columns of the response matrix, and p min And the minimum response value in the response matrix.
Further, the model updating module is specifically configured to update parameters of the correlation filter according to the following formula:
α＝(1-γ)α+γα′
wherein,and x is a Gaussian function taking the position coordinate of the target as the center, taking the image of the m-time-sized region of the detection frame, y is a Gaussian function taking the position coordinate of the target as the center, taking the Gaussian function of the m-time-sized region of the detection frame as a label for training a correlation filter, and gamma is a weight constant for parameter updating.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described apparatus, modules and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Fig. 5 shows a schematic block diagram of a robot provided in an embodiment of the present application, and for convenience of explanation, only a portion related to the embodiment of the present application is shown.
As shown in fig. 5, the robot 5 of this embodiment includes: a processor 50, a memory 51 and a computer program 52 stored in said memory 51 and executable on said processor 50. The processor 50, when executing the computer program 52, implements the steps of the various target tracking method embodiments described above. Alternatively, the processor 50, when executing the computer program 52, performs the functions of the modules/units of the apparatus embodiments described above.
By way of example, the computer program 52 may be partitioned into one or more modules/units that are stored in the memory 51 and executed by the processor 50 to complete the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing a specific function for describing the execution of the computer program 52 in the robot 5.
It will be appreciated by those skilled in the art that fig. 5 is merely an example of the robot 5 and is not meant to be limiting of the robot 5, and may include more or fewer components than shown, or may combine certain components, or different components, e.g., the robot 5 may also include input and output devices, network access devices, buses, etc.
The processor 50 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), field programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 51 may be an internal storage unit of the robot 5, such as a hard disk or a memory of the robot 5. The memory 51 may be an external storage device of the robot 5, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the robot 5. Further, the memory 51 may also include both an internal memory unit and an external memory device of the robot 5. The memory 51 is used for storing the computer program as well as other programs and data required by the robot 5. The memory 51 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/robot and method may be implemented in other ways. For example, the apparatus/robot embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated modules/units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each method embodiment described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the computer readable medium contains content that can be appropriately scaled according to the requirements of jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is subject to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.
The above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.
Claims (7)
1. A target tracking method, comprising:
determining a search area according to the position coordinates of the target and the detection frame;
tracking the target in the search area by using a correlation filter to obtain a response matrix as follows: cf=f -1 (F(k<x,z>) As indicated by F (. Alpha.)), wherein x is an image of an m-times-sized region of the initial detection frame centered on the initial coordinates, m is a preset multiple parameter, z is an image of the search region,σ 2 as the variance of the gaussian function,f (·) is the Fourier transform, F -1 (. Cndot.) Fourier inverse transform, & gt represents the operation of multiplying the matrix item by item, alpha is the parameter of the correlation filter, and CF is the response matrix;
determining a position coordinate corresponding to the largest response value in the response matrix as the position coordinate of the target;
confidence was calculated according to the following: r=p max Or alternativelyWherein p is max For the maximum response value in the response matrix, r is the confidence, CF w,h The value of the W row and the H column in the response matrix is 1-W, wherein W is the number of rows of the response matrix, H is 1-H, H is the number of columns of the response matrix, and p min The minimum response value in the response matrix is obtained;
the confidence coefficient is smaller than a preset first threshold value, the current tracking process is ended, and the follow-up information is fed back to a preset control center;
the confidence coefficient is larger than a preset second threshold value, parameters of the related filter are updated, and the second threshold value is larger than the first threshold value;
the confidence coefficient is smaller than or equal to the second threshold value and larger than or equal to the first threshold value, and parameters of the relevant filter are not updated.
2. The target tracking method according to claim 1, characterized by further comprising, before tracking the target in the search area:
performing target detection in a designated area, and determining initial coordinates and an initial detection frame of a detected target;
performing an initialization process, the initialization process comprising: the correlation filter is initialized.
3. The method of claim 2, wherein initializing the correlation filter comprises:
initializing parameters of the correlation filter according to:
and taking a Gaussian function of an area with m times of the initial detection frame as a label of a training correlation filter, wherein y is a penalty constant for training by taking the initial coordinate as a center.
4. A target tracking method as claimed in any one of claims 1 to 3, wherein said updating parameters of the correlation filter comprises:
updating parameters of the correlation filter according to:
α＝(1-γ)α+γα′
wherein,and x' is a Gaussian function taking the position coordinates of the target as the center, taking the m-times-sized region of the detection frame as the tag of the training correlation filter, lambda is a training penalty constant, and gamma is a parameter updating weight constant.
5. An object tracking device, comprising:
the searching area determining module is used for determining a searching area according to the position coordinates of the target and the detection frame;
the tracking module is used for tracking the target in the search area by using a correlation filter to obtain a response matrix shown as follows: cf=f -1 (F(k<x,z>) As indicated by F (. Alpha.)), wherein x is an image of an m-times-sized region of the initial detection frame centered on the initial coordinates, m is a preset multiple parameter, z is an image of the search region,σ 2 variance of Gaussian function, F (&) is Fourier transform, F -1 (. Cndot.) Fourier inverse transform, & gt represents the operation of multiplying the matrix item by item, alpha is the parameter of the correlation filter, and CF is the response matrix; determining a position coordinate corresponding to the largest response value in the response matrix as the position coordinate of the target; confidence was calculated according to the following: r=p max Or->Wherein p is max For the maximum response value in the response matrix, r is the confidence, CF w,h The value of the W row and the H column in the response matrix is 1-W, wherein W is the number of rows of the response matrix, H is 1-H, H is the number of columns of the response matrix, and p min The minimum response value in the response matrix is obtained;
the tracking loss judging module is used for ending the current tracking process when the confidence coefficient is smaller than a preset first threshold value and feeding back the tracking loss information to a preset control center;
the model updating module is used for updating parameters of the related filter when the confidence coefficient is larger than a preset second threshold value, and the second threshold value is larger than the first threshold value; the confidence coefficient is smaller than or equal to the second threshold value and larger than or equal to the first threshold value, and parameters of the relevant filter are not updated.
6. A computer-readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the object tracking method according to any one of claims 1 to 4.
7. A robot comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the object tracking method according to any one of claims 1 to 4 when the computer program is executed.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010413963.XA CN111696132B (en) | 2020-05-15 | 2020-05-15 | Target tracking method, device, computer readable storage medium and robot |
| PCT/CN2020/140413 WO2021227519A1 (en) | 2020-05-15 | 2020-12-28 | Target tracking method and apparatus, and computer-readable storage medium and robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010413963.XA CN111696132B (en) | 2020-05-15 | 2020-05-15 | Target tracking method, device, computer readable storage medium and robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111696132A CN111696132A (en) | 2020-09-22 |
| CN111696132B true CN111696132B (en) | 2023-12-29 |
Family
ID=72477853
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010413963.XA Active CN111696132B (en) | 2020-05-15 | 2020-05-15 | Target tracking method, device, computer readable storage medium and robot |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111696132B (en) |
| WO (1) | WO2021227519A1 (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111696132B (en) * | 2020-05-15 | 2023-12-29 | 深圳市优必选科技股份有限公司 | Target tracking method, device, computer readable storage medium and robot |
| CN112561963A (en) * | 2020-12-18 | 2021-03-26 | 北京百度网讯科技有限公司 | Target tracking method and device, road side equipment and storage medium |
| CN114332157B (en) * | 2021-12-14 | 2024-05-24 | 北京理工大学 | Long-time tracking method for double-threshold control |
| CN114708300A (en) * | 2022-03-02 | 2022-07-05 | 北京理工大学 | Anti-blocking self-adaptive target tracking method and system |
| CN114549591B (en) * | 2022-04-27 | 2022-07-08 | 南京甄视智能科技有限公司 | Method and device for detecting and tracking time-space domain behaviors, storage medium and equipment |
| CN115222771B (en) * | 2022-07-05 | 2023-07-21 | 北京建筑大学 | Target tracking method and device |
| CN115147458B (en) * | 2022-07-21 | 2023-04-07 | 北京远度互联科技有限公司 | Target tracking method and device, electronic equipment and storage medium |
| CN115657654B (en) * | 2022-07-26 | 2023-12-08 | 东莞康视达自动化科技有限公司 | Visual recognition method for meal delivery robot |
| CN115221981B (en) * | 2022-09-20 | 2022-12-02 | 毫末智行科技有限公司 | Target tracking method and device, terminal equipment and storage medium |
| CN115984333B (en) * | 2023-02-14 | 2024-01-19 | 北京拙河科技有限公司 | Smooth tracking method and device for airplane target |
| CN116088580B (en) * | 2023-02-15 | 2023-11-07 | 北京拙河科技有限公司 | Flying object tracking method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107154024A (en) * | 2017-05-19 | 2017-09-12 | 南京理工大学 | Dimension self-adaption method for tracking target based on depth characteristic core correlation filter |
| CN110033472A (en) * | 2019-03-15 | 2019-07-19 | 电子科技大学 | A kind of stable objects tracking under the infrared ground environment of complexity |
| CN110084829A (en) * | 2019-03-12 | 2019-08-02 | 上海阅面网络科技有限公司 | Method for tracking target, device, electronic equipment and computer readable storage medium |
| CN110796676A (en) * | 2019-10-10 | 2020-02-14 | 太原理工大学 | Target tracking method combining high-confidence updating strategy with SVM (support vector machine) re-detection technology |
| CN110956653A (en) * | 2019-11-29 | 2020-04-03 | 中国科学院空间应用工程与技术中心 | Satellite video dynamic target tracking method with fusion of correlation filter and motion estimation |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104424634B (en) * | 2013-08-23 | 2017-05-03 | 株式会社理光 | Object tracking method and device |
| US9454827B2 (en) * | 2013-08-27 | 2016-09-27 | Qualcomm Incorporated | Systems, devices and methods for tracking objects on a display |
| US20160025850A1 (en) * | 2014-06-03 | 2016-01-28 | Watchstander, LLC | Autonomous Robotic Mobile Threat Security System |
| CN105469430A (en) * | 2015-12-10 | 2016-04-06 | 中国石油大学(华东) | Anti-shielding tracking method of small target in large-scale scene |
| CN111696132B (en) * | 2020-05-15 | 2023-12-29 | 深圳市优必选科技股份有限公司 | Target tracking method, device, computer readable storage medium and robot |
-
2020
- 2020-05-15 CN CN202010413963.XA patent/CN111696132B/en active Active
- 2020-12-28 WO PCT/CN2020/140413 patent/WO2021227519A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107154024A (en) * | 2017-05-19 | 2017-09-12 | 南京理工大学 | Dimension self-adaption method for tracking target based on depth characteristic core correlation filter |
| CN110084829A (en) * | 2019-03-12 | 2019-08-02 | 上海阅面网络科技有限公司 | Method for tracking target, device, electronic equipment and computer readable storage medium |
| CN110033472A (en) * | 2019-03-15 | 2019-07-19 | 电子科技大学 | A kind of stable objects tracking under the infrared ground environment of complexity |
| CN110796676A (en) * | 2019-10-10 | 2020-02-14 | 太原理工大学 | Target tracking method combining high-confidence updating strategy with SVM (support vector machine) re-detection technology |
| CN110956653A (en) * | 2019-11-29 | 2020-04-03 | 中国科学院空间应用工程与技术中心 | Satellite video dynamic target tracking method with fusion of correlation filter and motion estimation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111696132A (en) | 2020-09-22 |
| WO2021227519A1 (en) | 2021-11-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111696132B (en) | Target tracking method, device, computer readable storage medium and robot | |
| CN111563919B (en) | Target tracking method, device, computer readable storage medium and robot | |
| CN109544598B (en) | Target tracking method and device and readable storage medium | |
| US20210056715A1 (en) | Object tracking method, object tracking device, electronic device and storage medium | |
| CN111104925B (en) | Image processing method, image processing apparatus, storage medium, and electronic device | |
| CN111860398A (en) | Remote sensing image target detection method and system and terminal equipment | |
| CN112634246B (en) | Oral cavity image recognition method and related equipment | |
| CN113298152B (en) | Model training method, device, terminal equipment and computer readable storage medium | |
| CN111553234A (en) | Pedestrian tracking method and device integrating human face features and Re-ID feature sorting | |
| US20200349349A1 (en) | Human Body Recognition Method And Apparatus, And Storage Medium | |
| CN115546705B (en) | Target identification method, terminal device and storage medium | |
| CN112101156A (en) | Target identification method and device and electronic equipment | |
| CN113450329B (en) | Microcirculation image blood vessel branch erythrocyte flow rate calculation method and system | |
| CN112488982A (en) | Ultrasonic image detection method and device | |
| CN112053386B (en) | Target tracking method based on depth convolution characteristic self-adaptive integration | |
| CN111462176B (en) | Target tracking method, target tracking device and terminal equipment | |
| CN111260692A (en) | Face tracking method, device, equipment and storage medium | |
| CN113297995B (en) | Human body posture estimation method and terminal equipment | |
| CN113837930B (en) | Face image synthesis method, device and computer readable storage medium | |
| CN112749625B (en) | Time sequence behavior detection method, time sequence behavior detection device and terminal equipment | |
| CN116452011B (en) | Data processing method and device for agent decision | |
| CN117593536A (en) | Component characteristic acquisition method and device, electronic equipment and readable storage medium | |
| CN118097526A (en) | Flying object identification method and system based on image processing | |
| CN116052215A (en) | Human body detection model training method and device based on knowledge distillation | |
| CN116158962A (en) | Human body massage positioning method and device, readable storage medium and robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |