CN111696132A - Target tracking method and device, computer readable storage medium and robot - Google Patents
Target tracking method and device, computer readable storage medium and robot Download PDFInfo
- Publication number
- CN111696132A CN111696132A CN202010413963.XA CN202010413963A CN111696132A CN 111696132 A CN111696132 A CN 111696132A CN 202010413963 A CN202010413963 A CN 202010413963A CN 111696132 A CN111696132 A CN 111696132A
- Authority
- CN
- China
- Prior art keywords
- target
- tracking
- search area
- response matrix
- confidence coefficient
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 83
- 238000001514 detection method Methods 0.000 claims abstract description 56
- 230000008569 process Effects 0.000 claims abstract description 47
- 230000004044 response Effects 0.000 claims description 78
- 239000011159 matrix material Substances 0.000 claims description 53
- 230000006870 function Effects 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 20
- 238000012549 training Methods 0.000 claims description 17
- 239000000126 substance Substances 0.000 claims description 10
- 230000009466 transformation Effects 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 6
- 238000004422 calculation algorithm Methods 0.000 description 29
- 238000010586 diagram Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 4
- 230000007774 longterm Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 230000000739 chaotic effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Abstract
The present application relates to the field of robotics, and in particular, to a target tracking method and apparatus, a computer-readable storage medium, and a robot. The method comprises the steps of determining a search area according to position coordinates and a detection frame of a target; tracking the target in the search area to obtain a position coordinate and a confidence coefficient of the target; and if the confidence coefficient is smaller than a preset first threshold value, ending the current tracking process. Through the method and the device, the confidence coefficient is introduced to evaluate the tracking effect in the tracking process of the target, when the confidence coefficient is lower, namely smaller than a preset first threshold value, the target is lost, the current tracking process can be ended at the moment, and the situation that the current tracking process is continued after the target is lost is avoided. When the confidence is higher, namely greater than a preset second threshold, the parameters of the relevant filter are updated, so that wrong background information is prevented from being learned, and the target is kept to be tracked.
Description
Technical Field
The present application relates to the field of robotics, and in particular, to a target tracking method and apparatus, a computer-readable storage medium, and a robot.
Background
The security robot, as a robot for assisting a human in completing security protection work, needs to have the capability of detecting and tracking a target that may appear in a security place. However, the high complexity/low real-time performance of the detection algorithm and the characteristic of being easily affected by shielding limit the real-time tracking capability of the security robot on the target appearing in the security place, and therefore, the target tracking algorithm should be started after the target is detected. At present, a tracking algorithm of a target has the advantages of high real-time performance/certain anti-blocking capability and the like, but due to the limitation of the tracking algorithm, the current tracking algorithm cannot judge whether the target is lost, so that the situation that the current tracking process is continued after the target is lost occurs. Moreover, when the current tracking algorithm trains the relevant filter in real time, the deformation and the shielding of the target are more and more along with the time, the learned characteristics of the algorithm include more and more background information and more chaotic conditions, wrong background information can be learned, and the tracking of the target can be gradually lost.
Disclosure of Invention
In view of this, embodiments of the present application provide a target tracking method and apparatus, a computer-readable storage medium, and a robot, so as to solve the problems that a current tracking algorithm cannot determine whether a target is lost, so that a current tracking process continues after the target is lost, and the target is gradually lost as time goes on.
A first aspect of an embodiment of the present application provides a target tracking method, which may include:
determining a search area according to the position coordinates of the target and the detection frame;
tracking the target in the search area to obtain a position coordinate and a confidence coefficient of the target;
if the confidence coefficient is smaller than a preset first threshold value, ending the current tracking process;
and updating the parameters of the relevant filter when the confidence coefficient is greater than a preset second threshold value, wherein the second threshold value is greater than the first threshold value.
Further, before tracking the target in the search area, the method further includes:
carrying out target detection in the designated area, and determining an initial coordinate and an initial detection frame of the detected target;
performing an initialization process, the initialization process comprising: the correlation filter is initialized.
Further, the initializing the correlation filter includes:
parameters of the correlation filter are initialized according to:
wherein the content of the first and second substances,
taking the image of the m-time area of the initial detection frame by taking the initial coordinate as a center, taking m as a preset multiple parameter, taking the Gaussian function of the m-time area of the initial detection frame by taking the initial coordinate as a center, and taking the Gaussian function as a label of a training related filter, wherein the sigma is2Is the variance of a gaussian function and is,
in order to perform the fourier transformation, the method,
inverse Fourier transform, λ is a penalty term constant for training, ⊙ represents matrix term by termThe multiplication operation, α, is the parameter of the correlation filter.
Further, the tracking the target in the search area to obtain the position coordinate and the confidence of the target includes:
tracking the target in the search area by using a correlation filter to obtain a response matrix as shown in the following:
wherein z is the image of the search area and CF is the response matrix;
determining the maximum response value in the response matrix as the maximum response value, and determining the position coordinate corresponding to the maximum response value as the position coordinate;
and calculating the confidence degree according to the response matrix.
Further, said calculating said confidence level from said response matrix comprises:
calculating the confidence level according to:
r＝pmax
wherein p ismaxThe maximum response value in the response matrix is obtained, and r is the confidence coefficient;
or
Calculating the confidence level according to:
wherein, CFw,hIs the value of W row and H column in the response matrix, W is more than or equal to 1 and less than or equal to W, W is the row number of the response matrix, H is more than or equal to 1 and less than or equal to H, H is the column number of the response matrix, pminIs the smallest response value in the response matrix.
Further, the target tracking method may further include:
and if the confidence coefficient is less than or equal to the second threshold and greater than or equal to the first threshold, the parameters of the relevant filter are not updated.
Further, the updating the parameter of the correlation filter includes:
updating parameters of the correlation filter according to:
α＝(1-γ)α+γα′
wherein the content of the first and second substances,
and x is an image of an m-time area of the detection frame by taking the position coordinate of the target as a center, y is a Gaussian function of the m-time area of the detection frame by taking the position coordinate of the target as a center, and is used as a label for training a related filter, and gamma is a weight constant for updating the parameters.
A second aspect of an embodiment of the present application provides a target tracking apparatus, which may include:
the search area determining module is used for determining a search area according to the position coordinates of the target and the detection frame;
the tracking module is used for tracking the target in the search area to obtain the position coordinate and the confidence coefficient of the target;
the tracking loss judging module is used for judging that the target is tracked and lost and ending the current tracking process if the confidence coefficient is smaller than a preset first threshold value;
and the model updating module is used for updating the parameters of the relevant filter when the confidence coefficient is greater than a preset second threshold value, wherein the second threshold value is greater than the first threshold value.
Further, the target tracking apparatus may further include:
the initial detection module is used for detecting the target in the designated area and determining the initial coordinate and the initial detection frame of the detected target;
an initialization module to perform an initialization process, the initialization process comprising: the correlation filter is initialized.
Further, the initialization module may include:
a correlation filter initialization unit for initializing parameters of the correlation filter according to:
wherein the content of the first and second substances,
taking the image of the m-time area of the initial detection frame by taking the initial coordinate as a center, taking m as a preset multiple parameter, taking the Gaussian function of the m-time area of the initial detection frame by taking the initial coordinate as a center, and taking the Gaussian function as a label of a training related filter, wherein the sigma is2Is the variance of a gaussian function and is,
in order to perform the fourier transformation, the method,
and (3) performing inverse Fourier transform, wherein lambda is a penalty term constant of training, ⊙ represents a matrix multiplication operation item by item, and α is a parameter of a correlation filter.
Further, the tracking module may include:
a response matrix calculation unit, configured to track the target in the search area using a correlation filter, and obtain a response matrix shown as follows:
wherein z is the image of the search area and CF is the response matrix;
a position coordinate determination unit configured to determine a maximum response value in the response matrix as the maximum response value, and determine a position coordinate corresponding to the maximum response value as the position coordinate;
and the confidence coefficient calculating unit is used for calculating the confidence coefficient according to the response matrix.
Further, the confidence calculation unit is specifically configured to:
calculating the confidence level according to:
r＝pmax
wherein p ismaxThe maximum response value in the response matrix is obtained, and r is the confidence coefficient;
or
Calculating the confidence level according to:
wherein, CFw,hIs the value of W row and H column in the response matrix, W is more than or equal to 1 and less than or equal to W, W is the row number of the response matrix, H is more than or equal to 1 and less than or equal to H, H is the column number of the response matrix, pminIs the smallest response value in the response matrix.
Further, the model updating module is specifically configured to update the parameters of the correlation filter according to the following equation:
α＝(1-γ)α+γα′
wherein the content of the first and second substances,
and x is an image of an m-time area of the detection frame by taking the position coordinate of the target as a center, y is a Gaussian function of the m-time area of the detection frame by taking the position coordinate of the target as a center, and is used as a label for training a related filter, and gamma is a weight constant for updating the parameters.
A third aspect of embodiments of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps of any one of the above-mentioned target tracking methods.
A fourth aspect of embodiments of the present application provides a robot, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned target tracking methods when executing the computer program.
A fifth aspect of embodiments of the present application provides a computer program product, which, when run on a robot, causes the robot to perform the steps of any one of the above-mentioned target tracking methods.
Compared with the prior art, the embodiment of the application has the advantages that: according to the position coordinates and the detection frame of the target, the search area is determined; tracking the target in the search area to obtain a position coordinate and a confidence coefficient of the target; and if the confidence coefficient is smaller than a preset first threshold value, ending the current tracking process. According to the embodiment of the application, in the tracking process of the target, the confidence coefficient is introduced to evaluate the tracking effect, when the confidence coefficient is lower, namely smaller than a preset first threshold value, the target is lost, the current tracking process can be ended at the moment, and the situation that the current tracking process is continued after the target is lost is avoided. When the confidence is higher, namely greater than a preset second threshold, the parameters of the relevant filter are updated, so that wrong background information is prevented from being learned, and the target is kept to be tracked.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed for the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic flow diagram of an initialization process;
FIG. 2 is a flowchart of an embodiment of a target tracking method in an embodiment of the present application;
FIG. 3 is a schematic diagram illustrating a correlation between a detection frame and a search area of a target;
FIG. 4 is a block diagram of one embodiment of a target tracking device in an embodiment of the present application;
fig. 5 is a schematic block diagram of a robot according to an embodiment of the present application.
Detailed Description
In order to make the objects, features and advantages of the present invention more apparent and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the embodiments described below are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
In addition, in the description of the present application, the terms "first," "second," "third," and the like are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
In the embodiment of the present application, the target to be tracked may include, but is not limited to: people, vehicles, animals, and other moving objects.
The existing target tracking algorithm at present is divided into a short-time tracking algorithm and a long-time tracking algorithm: the short-time tracking algorithm can better track the target in a short time (less than 2 minutes), and as time goes on, the deformation and shielding of the target are more and more, the learning characteristics of the algorithm are more and more disordered, and the tracking of the target can be gradually lost; in the existing long-term Tracking algorithm (2-20 minutes), a tracker needs to be tracked and trained in real time, a detector is also provided, and real-time training is also carried out in the Tracking process, the most classical long-term Tracking algorithm is a Tracking-Learning-Detection (TLD) algorithm, the Tracking algorithm is not easy to lose due to simultaneous training of the detector under the condition of long-term Tracking, but the complexity of the detector is higher, and the complexity of the algorithm is greatly improved due to redundant training. The detection algorithm capable of being trained in real time is often poor in detection effect, and the short-time tracking effect is not good as well as the short-time tracking algorithm, so that the tracking effect does not meet the requirement overall. No matter the existing short-term tracking algorithm or the long-term tracking algorithm is adopted, the target is likely to be lost, but the problem that the algorithm cannot judge whether the target is lost or not exists.
The embodiment of the application introduces confidence coefficient on the basis of the short-time tracker to realize high-efficiency tracking and tracking loss judgment of the target: the method comprises the steps that a correlation filter learns characteristics of a target, and outputs pixel point coordinates which are most responsive to the target characteristics in a current frame to track the target, wherein the response value of the current frame to the target characteristics can be used as confidence, in order to deal with the situation that when a short-time tracking algorithm trains the correlation filter in real time, deformation and shielding of the target are more and more caused along with the time, the learned characteristics of the algorithm comprise more and more background information and more disorder, the confidence can be judged in the tracking process, and a tracker is updated only when the confidence of tracking is higher so as to prevent wrong background information from being learned. Similarly, the confidence level is judged in the tracking process, and when the tracking confidence level is low, the target is lost, and the target needs to be detected again.
It is easy to understand that the target detection is a precondition for performing target tracking, and in an initial state, the target detection needs to be performed in a specified area by a target detector first, where the detection method used may be any one of the detection methods in the prior art, and this is not specifically limited in this embodiment of the present application.
When an object is detected in a certain frame image, the position coordinate of the object in the frame image and a detection frame (bb) are determined, and here, the position coordinate is regarded as an initial coordinate, and the detection frame is regarded as an initial detection frame.
After the target detection is completed, an initialization process of target tracking may be performed, as shown in fig. 1, the initialization process may include:
and step S101, initializing the value of k.
In the embodiment of the present application, k is an image frame number for tracking a target, and k is a positive integer. Here, the frame image in which the target is detected is regarded as the 0 th frame, and after the target detection is completed, the value of k may be initialized to 1, that is, the following steps are performed: k is 1.
Step S102 is to initialize a Correlation Filter (CF).
Specifically, the parameters of the correlation filter are initialized according to:
wherein the content of the first and second substances,
x is an image of an m-time area of the initial detection frame by taking the initial coordinate as a center, and m is a preset multiple parameterThe number, the specific value of which may be set according to the actual situation, is preferably set to 2.5 here, that is, the search area is 2.5 times the size of the detection frame, y is a gaussian function with the initial coordinate as the center, the area m times the size of the initial detection frame is taken as the label of the training correlation filter, σ2Is the variance of a gaussian function and is,
in order to perform the fourier transformation, the method,
and (3) performing inverse Fourier transform, wherein lambda is a penalty term constant of training, ⊙ represents a matrix multiplication operation item by item, and α is a parameter of a correlation filter.
When tracking a target, the specific tracking process of each frame image is similar, and a detailed description will be given below by taking any one frame (k-th frame) as an example. As shown in fig. 2, the process of tracking the target in the k-th frame image may include:
step S201, determining a search area in the k frame image according to the position coordinate of the target in the k-1 frame image and the detection frame.
As shown in fig. 3, the rectangle frame with the dotted line indicates the detection frame of the target tracked in the k-1 th frame, and the search area in the k-1 th frame is m times the size of the detection frame, and as shown by the solid line frame in the figure, the search area is centered on the position coordinate of the target in the image of the k-1 th frame.
Step S202, tracking the target in the search area to obtain the position coordinate and the confidence of the target in the k frame image.
In this embodiment, the target is tracked using a correlation filter. In other embodiments, the target tracking algorithm may be, but is not limited to, a kalman filter, a target detection tracking, a deep learning based tracking algorithm, a multi-target tracking algorithm, and the like.
Specifically, the target may be tracked in the search area using a correlation filter, resulting in a response matrix as shown below:
wherein z is the image of the search area and CF is the response matrix.
In this embodiment, the maximum response value in the response matrix may be determined as the maximum response value, and the position coordinate corresponding to the maximum response value may be determined as the position coordinate of the target in the k-th frame image.
In a specific implementation of the embodiment of the present application, the confidence level may be calculated according to the following formula:
r＝pmax
wherein p ismaxAnd r is the confidence coefficient and is the maximum response value in the response matrix.
In another specific implementation of the embodiment of the present application, the confidence level may be calculated according to the following formula:
wherein, CFw,hIs the value of W row and H column in the response matrix, W is more than or equal to 1 and less than or equal to W, W is the row number of the response matrix, H is more than or equal to 1 and less than or equal to H, H is the column number of the response matrix, pminIs the smallest response value in the response matrix.
And step S203, executing the operation corresponding to the confidence coefficient.
Specifically, if the confidence is smaller than a preset first threshold, it is determined that the target is lost, and the current tracking process is ended. In the process, the information of the lost tracking can be fed back to a preset control center.
And if the confidence coefficient is greater than a preset second threshold value, updating the parameters of the relevant filter.
The second threshold is greater than the first threshold, and specific values of the first threshold and the second threshold may be set according to an actual situation, which is not specifically limited in this embodiment of the application.
In the embodiment of the present application, the parameters of the correlation filter may be updated according to the following formula:
α＝(1-γ)α+γα′
wherein the content of the first and second substances,
and x is an image of an m-time area of the detection frame in the k frame image by taking the position coordinate of the target in the k frame image as the center, y is a Gaussian function of the m-time area of the detection frame in the k frame image by taking the position coordinate of the target in the k frame image as the center, and gamma is a weight constant for updating the parameter.
After updating the parameters of the relevant filter, if the current tracking process is not finished, continuing to track the target in the next frame of image, namely executing: k is k +1, and the tracking step shown in fig. 2 is executed again until the current tracking process is finished.
If the confidence is less than or equal to the second threshold and greater than or equal to the first threshold, no updating is performed on the parameters of the relevant filter, and if the current tracking process is not finished, the step of directly continuing to track the target in the next frame of image is performed, that is to say: k is k +1, and the tracking step shown in fig. 2 is executed again until the current tracking process is finished.
In summary, the search area is determined according to the position coordinates of the target and the detection frame in the embodiment of the present application; tracking the target in the search area to obtain a position coordinate and a confidence coefficient of the target; and if the confidence coefficient is smaller than a preset first threshold value, judging that the target is lost, and ending the current tracking process. According to the embodiment of the application, the confidence coefficient is introduced to evaluate the tracking effect in the tracking process of the target, when the confidence coefficient is low, the target is lost, the current tracking process can be ended at the moment, and the situation that the current tracking process is continued after the target is lost is avoided. When the confidence is higher, namely greater than a preset second threshold, the parameters of the relevant filter are updated, so that wrong background information is prevented from being learned, and the target is kept to be tracked.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Fig. 4 is a block diagram of an embodiment of a target tracking device according to the present application, which corresponds to the target tracking method in the foregoing embodiment.
In this embodiment, a target tracking apparatus may include:
a search area determination module 401, configured to determine a search area according to the position coordinate of the target and the detection frame;
a tracking module 402, configured to track the target in the search area to obtain a position coordinate and a confidence of the target;
a tracking loss determining module 403, configured to end the current tracking process if the confidence is smaller than a preset first threshold;
and a model updating module 404, configured to update the parameter of the relevant filter when the confidence is greater than a preset second threshold, where the second threshold is greater than the first threshold.
Further, the target tracking apparatus may further include:
the initial detection module is used for detecting the target in the designated area and determining the initial coordinate and the initial detection frame of the detected target;
an initialization module to perform an initialization process, the initialization process comprising: the correlation filter is initialized.
Further, the initialization module may include:
a correlation filter initialization unit for initializing parameters of the correlation filter according to:
wherein the content of the first and second substances,
taking the image of the m-time area of the initial detection frame by taking the initial coordinate as a center, taking m as a preset multiple parameter, taking the Gaussian function of the m-time area of the initial detection frame by taking the initial coordinate as a center, and taking the Gaussian function as a label of a training related filter, wherein the sigma is2Is the variance of a gaussian function and is,
in order to perform the fourier transformation, the method,
and (3) performing inverse Fourier transform, wherein lambda is a penalty term constant of training, ⊙ represents a matrix multiplication operation item by item, and α is a parameter of a correlation filter.
Further, the tracking module may include:
a response matrix calculation unit, configured to track the target in the search area using a correlation filter, and obtain a response matrix shown as follows:
wherein z is the image of the search area and CF is the response matrix;
a position coordinate determination unit configured to determine a maximum response value in the response matrix as the maximum response value, and determine a position coordinate corresponding to the maximum response value as the position coordinate;
and the confidence coefficient calculating unit is used for calculating the confidence coefficient according to the response matrix.
Further, the confidence calculation unit is specifically configured to:
calculating the confidence level according to:
r＝pmax
wherein，pmaxThe maximum response value in the response matrix is obtained, and r is the confidence coefficient;
or
Calculating the confidence level according to:
wherein, CFw,hIs the value of W row and H column in the response matrix, W is more than or equal to 1 and less than or equal to W, W is the row number of the response matrix, H is more than or equal to 1 and less than or equal to H, H is the column number of the response matrix, pminIs the smallest response value in the response matrix.
Further, the model updating module is specifically configured to update the parameters of the correlation filter according to the following equation:
α＝(1-γ)α+γα′
wherein the content of the first and second substances,
and x is an image of an m-time area of the detection frame by taking the position coordinate of the target as a center, y is a Gaussian function of the m-time area of the detection frame by taking the position coordinate of the target as a center, and is used as a label for training a related filter, and gamma is a weight constant for updating the parameters.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses, modules and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Fig. 5 shows a schematic block diagram of a robot provided in an embodiment of the present application, and only a part related to the embodiment of the present application is shown for convenience of explanation.
As shown in fig. 5, the robot 5 of this embodiment includes: a processor 50, a memory 51 and a computer program 52 stored in said memory 51 and executable on said processor 50. The processor 50, when executing the computer program 52, implements the steps in the various target tracking method embodiments described above. Alternatively, the processor 50 implements the functions of the modules/units in the above-described device embodiments when executing the computer program 52.
Illustratively, the computer program 52 may be partitioned into one or more modules/units, which are stored in the memory 51 and executed by the processor 50 to accomplish the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of the computer program 52 in the robot 5.
Those skilled in the art will appreciate that fig. 5 is merely an example of a robot 5 and does not constitute a limitation of the robot 5 and may include more or fewer components than shown, or some components in combination, or different components, for example, the robot 5 may also include input and output devices, network access devices, buses, etc.
The Processor 50 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 51 may be an internal storage unit of the robot 5, such as a hard disk or a memory of the robot 5. The memory 51 may also be an external storage device of the robot 5, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, provided on the robot 5. Further, the memory 51 may also include both an internal storage unit and an external storage device of the robot 5. The memory 51 is used for storing the computer program and other programs and data required by the robot 5. The memory 51 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/robot and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/robot are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow in the method of the embodiments described above can be realized by a computer program, which can be stored in a computer-readable storage medium and can realize the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, etc. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A target tracking method, comprising:
determining a search area according to the position coordinates of the target and the detection frame;
tracking the target in the search area to obtain a position coordinate and a confidence coefficient of the target;
if the confidence coefficient is smaller than a preset first threshold value, ending the current tracking process;
and updating the parameters of the relevant filter when the confidence coefficient is greater than a preset second threshold value, wherein the second threshold value is greater than the first threshold value.
2. The target tracking method of claim 1, further comprising, prior to tracking the target in the search area:
carrying out target detection in the designated area, and determining an initial coordinate and an initial detection frame of the detected target;
performing an initialization process, the initialization process comprising: the correlation filter is initialized.
3. The method of claim 2, wherein initializing the correlation filter comprises:
parameters of the correlation filter are initialized according to:
wherein the content of the first and second substances,
taking the image of the m-time area of the initial detection frame by taking the initial coordinate as a center, taking m as a preset multiple parameter, taking the Gaussian function of the m-time area of the initial detection frame by taking the initial coordinate as a center, and taking the Gaussian function as a label of a training related filter, wherein the sigma is2Is the variance of a gaussian function and is,
in order to perform the fourier transformation, the method,
and (3) performing inverse Fourier transform, wherein lambda is a penalty term constant of training, ⊙ represents a matrix multiplication operation item by item, and α is a parameter of a correlation filter.
4. The target tracking method of claim 1, wherein the tracking the target in the search area to obtain the position coordinates and the confidence level of the target comprises:
tracking the target in the search area by using a correlation filter to obtain a response matrix as shown in the following:
wherein z is the image of the search area and CF is the response matrix;
determining the maximum response value in the response matrix as the maximum response value, and determining the position coordinate corresponding to the maximum response value as the position coordinate;
and calculating the confidence degree according to the response matrix.
5. The method of claim 4, wherein said calculating said confidence level from said response matrix comprises:
calculating the confidence level according to:
r＝pmax
wherein p ismaxThe maximum response value in the response matrix is obtained, and r is the confidence coefficient;
or
Calculating the confidence level according to:
wherein, CFw,hIs the value of W row and H column in the response matrix, W is more than or equal to 1 and less than or equal to W, W is the row number of the response matrix, H is more than or equal to 1 and less than or equal to H, H is the column number of the response matrix, pminIs the smallest response value in the response matrix.
6. The target tracking method of any one of claims 1 to 5, further comprising:
and when the confidence coefficient is less than or equal to the second threshold and greater than or equal to the first threshold, the parameters of the relevant filter are not updated any more.
7. The method of claim 6, wherein the updating the parameters of the correlation filter comprises:
updating parameters of the correlation filter according to:
α＝(1-γ)α+γα′
wherein the content of the first and second substances,
and x is an image of an m-time area of the detection frame by taking the position coordinate of the target as a center, y is a Gaussian function of the m-time area of the detection frame by taking the position coordinate of the target as a center, and is used as a label for training a related filter, and gamma is a weight constant for updating the parameters.
8. An object tracking device, comprising:
the search area determining module is used for determining a search area according to the position coordinates of the target and the detection frame;
the tracking module is used for tracking the target in the search area to obtain the position coordinate and the confidence coefficient of the target;
the tracking and losing judgment module is used for ending the current tracking process when the confidence coefficient is smaller than a preset first threshold value;
and the model updating module is used for updating the parameters of the relevant filter when the confidence coefficient is greater than a preset second threshold value, wherein the second threshold value is greater than the first threshold value.
9. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the object tracking method according to any one of claims 1 to 7.
10. A robot comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, carries out the steps of the object tracking method according to any of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010413963.XA CN111696132B (en) | 2020-05-15 | 2020-05-15 | Target tracking method, device, computer readable storage medium and robot |
| PCT/CN2020/140413 WO2021227519A1 (en) | 2020-05-15 | 2020-12-28 | Target tracking method and apparatus, and computer-readable storage medium and robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010413963.XA CN111696132B (en) | 2020-05-15 | 2020-05-15 | Target tracking method, device, computer readable storage medium and robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111696132A true CN111696132A (en) | 2020-09-22 |
| CN111696132B CN111696132B (en) | 2023-12-29 |
Family
ID=72477853
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010413963.XA Active CN111696132B (en) | 2020-05-15 | 2020-05-15 | Target tracking method, device, computer readable storage medium and robot |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111696132B (en) |
| WO (1) | WO2021227519A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112561963A (en) * | 2020-12-18 | 2021-03-26 | 北京百度网讯科技有限公司 | Target tracking method and device, road side equipment and storage medium |
| WO2021227519A1 (en) * | 2020-05-15 | 2021-11-18 | 深圳市优必选科技股份有限公司 | Target tracking method and apparatus, and computer-readable storage medium and robot |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114549591B (en) * | 2022-04-27 | 2022-07-08 | 南京甄视智能科技有限公司 | Method and device for detecting and tracking time-space domain behaviors, storage medium and equipment |
| CN115222771B (en) * | 2022-07-05 | 2023-07-21 | 北京建筑大学 | Target tracking method and device |
| CN115147458B (en) * | 2022-07-21 | 2023-04-07 | 北京远度互联科技有限公司 | Target tracking method and device, electronic equipment and storage medium |
| CN115657654B (en) * | 2022-07-26 | 2023-12-08 | 东莞康视达自动化科技有限公司 | Visual recognition method for meal delivery robot |
| CN115221981B (en) * | 2022-09-20 | 2022-12-02 | 毫末智行科技有限公司 | Target tracking method and device, terminal equipment and storage medium |
| CN115984333B (en) * | 2023-02-14 | 2024-01-19 | 北京拙河科技有限公司 | Smooth tracking method and device for airplane target |
| CN116088580B (en) * | 2023-02-15 | 2023-11-07 | 北京拙河科技有限公司 | Flying object tracking method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150063632A1 (en) * | 2013-08-27 | 2015-03-05 | Qualcomm Incorporated | Systems, devices and methods for tracking objects on a display |
| US20160025850A1 (en) * | 2014-06-03 | 2016-01-28 | Watchstander, LLC | Autonomous Robotic Mobile Threat Security System |
| CN107154024A (en) * | 2017-05-19 | 2017-09-12 | 南京理工大学 | Dimension self-adaption method for tracking target based on depth characteristic core correlation filter |
| CN110033472A (en) * | 2019-03-15 | 2019-07-19 | 电子科技大学 | A kind of stable objects tracking under the infrared ground environment of complexity |
| CN110084829A (en) * | 2019-03-12 | 2019-08-02 | 上海阅面网络科技有限公司 | Method for tracking target, device, electronic equipment and computer readable storage medium |
| CN110796676A (en) * | 2019-10-10 | 2020-02-14 | 太原理工大学 | Target tracking method combining high-confidence updating strategy with SVM (support vector machine) re-detection technology |
| CN110956653A (en) * | 2019-11-29 | 2020-04-03 | 中国科学院空间应用工程与技术中心 | Satellite video dynamic target tracking method with fusion of correlation filter and motion estimation |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104424634B (en) * | 2013-08-23 | 2017-05-03 | 株式会社理光 | Object tracking method and device |
| CN105469430A (en) * | 2015-12-10 | 2016-04-06 | 中国石油大学(华东) | Anti-shielding tracking method of small target in large-scale scene |
| CN111696132B (en) * | 2020-05-15 | 2023-12-29 | 深圳市优必选科技股份有限公司 | Target tracking method, device, computer readable storage medium and robot |
-
2020
- 2020-05-15 CN CN202010413963.XA patent/CN111696132B/en active Active
- 2020-12-28 WO PCT/CN2020/140413 patent/WO2021227519A1/en active Application Filing
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150063632A1 (en) * | 2013-08-27 | 2015-03-05 | Qualcomm Incorporated | Systems, devices and methods for tracking objects on a display |
| US20160025850A1 (en) * | 2014-06-03 | 2016-01-28 | Watchstander, LLC | Autonomous Robotic Mobile Threat Security System |
| CN107154024A (en) * | 2017-05-19 | 2017-09-12 | 南京理工大学 | Dimension self-adaption method for tracking target based on depth characteristic core correlation filter |
| CN110084829A (en) * | 2019-03-12 | 2019-08-02 | 上海阅面网络科技有限公司 | Method for tracking target, device, electronic equipment and computer readable storage medium |
| CN110033472A (en) * | 2019-03-15 | 2019-07-19 | 电子科技大学 | A kind of stable objects tracking under the infrared ground environment of complexity |
| CN110796676A (en) * | 2019-10-10 | 2020-02-14 | 太原理工大学 | Target tracking method combining high-confidence updating strategy with SVM (support vector machine) re-detection technology |
| CN110956653A (en) * | 2019-11-29 | 2020-04-03 | 中国科学院空间应用工程与技术中心 | Satellite video dynamic target tracking method with fusion of correlation filter and motion estimation |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021227519A1 (en) * | 2020-05-15 | 2021-11-18 | 深圳市优必选科技股份有限公司 | Target tracking method and apparatus, and computer-readable storage medium and robot |
| CN112561963A (en) * | 2020-12-18 | 2021-03-26 | 北京百度网讯科技有限公司 | Target tracking method and device, road side equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111696132B (en) | 2023-12-29 |
| WO2021227519A1 (en) | 2021-11-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111696132A (en) | Target tracking method and device, computer readable storage medium and robot | |
| US11100320B2 (en) | Image recognition method and apparatus | |
| CN111563919B (en) | Target tracking method, device, computer readable storage medium and robot | |
| CN111860398B (en) | Remote sensing image target detection method and system and terminal equipment | |
| CN113298152B (en) | Model training method, device, terminal equipment and computer readable storage medium | |
| CN111612822B (en) | Object tracking method, device, computer equipment and storage medium | |
| CN110349188B (en) | Multi-target tracking method, device and storage medium based on TSK fuzzy model | |
| CN115546705B (en) | Target identification method, terminal device and storage medium | |
| CN107944381B (en) | Face tracking method, face tracking device, terminal and storage medium | |
| CN112634246A (en) | Oral cavity image identification method and related equipment | |
| CN112037256A (en) | Target tracking method and device, terminal equipment and computer readable storage medium | |
| US10885635B2 (en) | Curvilinear object segmentation with noise priors | |
| CN109063567B (en) | Human body recognition method, human body recognition device and storage medium | |
| CN111126049B (en) | Object relation prediction method, device, terminal equipment and readable storage medium | |
| CN113971766A (en) | Robot control method and device, robot and storage medium | |
| CN110008881B (en) | Method and device for identifying cow behaviors of multiple moving targets | |
| US10832413B2 (en) | Curvilinear object segmentation with geometric priors | |
| CN116129504A (en) | Living body detection model training method and living body detection method | |
| CN113850843A (en) | Target tracking method and device, electronic equipment and storage medium | |
| CN111260692A (en) | Face tracking method, device, equipment and storage medium | |
| CN111199179B (en) | Target object tracking method, terminal equipment and medium | |
| CN111462176A (en) | Target tracking method, target tracking device and terminal equipment | |
| CN116580063B (en) | Target tracking method, target tracking device, electronic equipment and storage medium | |
| CN113297995B (en) | Human body posture estimation method and terminal equipment | |
| US20230298326A1 (en) | Image augmentation method, electronic device and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |