CN111563919B - Target tracking method, device, computer readable storage medium and robot - Google Patents
Target tracking method, device, computer readable storage medium and robot Download PDFInfo
- Publication number
- CN111563919B CN111563919B CN202010259829.9A CN202010259829A CN111563919B CN 111563919 B CN111563919 B CN 111563919B CN 202010259829 A CN202010259829 A CN 202010259829A CN 111563919 B CN111563919 B CN 111563919B
- Authority
- CN
- China
- Prior art keywords
- target
- position coordinate
- kth
- response value
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 230000004044 response Effects 0.000 claims abstract description 98
- 238000001514 detection method Methods 0.000 claims abstract description 52
- 239000011159 matrix material Substances 0.000 claims description 79
- 238000004590 computer program Methods 0.000 claims description 22
- 230000006870 function Effects 0.000 claims description 21
- 230000008569 process Effects 0.000 claims description 18
- 238000012549 training Methods 0.000 claims description 12
- 230000007704 transition Effects 0.000 claims description 4
- 230000000694 effects Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The application belongs to the technical field of robots, and particularly relates to a target tracking method, a target tracking device, a computer-readable storage medium and a robot. The method comprises the following steps: calculating the predicted position coordinates of the target in the kth frame of image by using a Kalman filter according to the position coordinates of the target in the kth-1 frame of image; determining a first search area in a kth frame image according to the predicted position coordinates and a detection frame in the kth-1 frame image; tracking the target in the first search area by using a correlation filter to obtain a first maximum response value and a first position coordinate; determining a second search area in the kth frame of image according to the position coordinates of the target in the kth-1 frame of image and the detection frame; tracking the target in the second search area by using a correlation filter to obtain a second maximum response value and a second position coordinate; and selecting the position coordinates of the target in the kth frame of image from the two position coordinates.
Description
Technical Field
The application belongs to the technical field of robots, and particularly relates to a target tracking method, a target tracking device, a computer-readable storage medium and a robot.
Background
As a robot for assisting a human to complete a security work, a security robot is required to have the capability of detecting and tracking a target that may appear in a security place. However, the high complexity/low real-time nature of the detection algorithm and the susceptibility to occlusion limit the real-time tracking capabilities of the security robot to targets that appear within the security site, and thus the target tracking algorithm should be enabled after the target is detected. At present, the tracking algorithm of the target has the advantages of high instantaneity, certain anti-occlusion capability and the like, but due to the limitation of the tracking algorithm, when a new image of each frame is tracked, whether the target exists near the pixel coordinates where the target possibly exists or not can only be tracked, which leads to the situation that when the target moves faster and reaches the outside of the tracking range or the edge, the tracking failure occurs, and the situation is generally called as a boundary effect. In order to cope with the "boundary effect", the simplest method is to expand the tracking range, but this may lead to the target tracker learning too much background information during the learning process, which adversely affects the tracking effect. At present, related documents try to solve the boundary effect, including SRDCF and CFLB algorithms, and the algorithms directly modify the optimized objective function of the target tracker, so that the target tracker has no closed solution, and the target tracker can only be solved by an iterative method in the learning and updating processes, which can cause the running speed of the target tracking algorithm to be reduced and cannot meet the higher real-time requirement.
Disclosure of Invention
In view of this, the embodiments of the present application provide a target tracking method, apparatus, computer readable storage medium, and robot, so as to solve the problem that the real-time performance of the existing target tracking method is poor.
A first aspect of an embodiment of the present application provides a target tracking method, which may include:
calculating the predicted position coordinate of the target in the kth frame image by using a Kalman filter according to the position coordinate of the target in the kth-1 frame image, wherein k is a positive integer;
determining a first search area in a kth frame image according to the predicted position coordinates and a detection frame in the kth-1 frame image;
tracking the target in the first search area by using a correlation filter to obtain a first maximum response value and a first position coordinate;
determining a second search area in the kth frame of image according to the position coordinates of the target in the kth-1 frame of image and the detection frame;
tracking the target in the second search area by using a correlation filter to obtain a second maximum response value and a second position coordinate;
comparing the first maximum response value with the second maximum response value, and selecting the position coordinate of the target in the kth frame image from the first position coordinate and the second position coordinate according to the comparison result.
Further, the comparing the first maximum response value and the second maximum response value, and selecting the position coordinate of the target in the kth frame image from the first position coordinate and the second position coordinate according to the comparison result includes:
if the first maximum response value is larger than the second maximum response value, determining the first position coordinate as the position coordinate of the target in the kth frame image;
and if the first maximum response value is smaller than or equal to the second maximum response value, determining the second position coordinate as the position coordinate of the target in the kth frame image.
Further, before calculating the predicted position coordinates of the target in the kth frame image using the kalman filter, the method further includes:
performing target detection in a designated area, and determining initial coordinates and an initial detection frame of a detected target;
performing an initialization process, the initialization process comprising: initializing the value of k; initializing a correlation filter; the kalman filter is initialized.
Further, initializing the correlation filter includes:
initializing parameters of the correlation filter according to:
wherein,x is the image of m times size area of the initial detection frame taking the initial coordinate as the center, m is a preset multiple parameter, y is the Gaussian function of m times size area of the initial detection frame taking the initial coordinate as the center, and sigma is used as the label of the training correlation filter 2 Variance of Gaussian function, F (&) is Fourier transform, F -1 (. Cndot.) Fourier inverse transform, λ is the penalty constant for training, by which is the matrix term-wise multiplication operation, α is the parameter of the correlation filter;
the initializing of the Kalman filter includes:
initializing a state transition matrix as:
initializing an observation matrix as:
initializing a prediction noise covariance matrix into a diagonal matrix of 4 rows and 4 columns, wherein diagonal elements are variances of Gaussian white noise;
initializing an observation noise covariance matrix into a diagonal matrix of 2 rows and 2 columns, wherein diagonal elements are variances of Gaussian white noise;
initializing a posterior error estimation covariance matrix into a unit diagonal matrix of 4 rows and 4 columns;
initializing the state of the target as:
w＝[u 0 v 0 Δu Δv] T
wherein, (u) 0 ,v 0 ) For the initial coordinates, (Δu, Δv) is the motion velocity obtained by random initialization.
Further, the tracking the target in the first search area by using the correlation filter to obtain a first maximum response value and a first position coordinate, including:
tracking the target in the first search area by using a correlation filter to obtain a first response matrix shown as follows:
CF 1 ＝F -1 (F(k<x,z 1 >)⊙F (α))
wherein z is 1 For the image of the first search area, CF 1 Is the first response matrix;
and determining the largest response value in the first response matrix as the first largest response value, and determining the position coordinate corresponding to the first largest response value as the first position coordinate.
Further, the tracking the target in the second search area by using the correlation filter to obtain a second maximum response value and a second position coordinate, including:
tracking the target in the second search area by using a correlation filter to obtain a second response matrix as follows:
CF 2 ＝F -1 (F(k<x,z 2 >)⊙F (α))
wherein z is 2 For the image of the second search area, CF 2 For the second response matrix;
and determining the largest response value in the second response matrix as the second largest response value, and determining the position coordinate corresponding to the second largest response value as the second position coordinate.
Further, after selecting the position coordinates of the target in the kth frame of image, the method further comprises:
updating parameters of the correlation filter according to:
α＝(1-γ)α+γα′
wherein,x is centered on the position coordinates of the object in the kth frame imageTaking an image of an m-time size region of a detection frame in a kth frame image, wherein y is a Gaussian function taking a position coordinate of the target in the kth frame image as a center, taking the m-time size region of the detection frame in the kth frame image as a label of a training related filter, and gamma is a weight constant of parameter updating;
updating the state of the object according to the following formula:
w＝w′+K′(q-Hw′)
wherein K' =ph T (HPH T +R) -1 W' is the predicted position coordinate of the target in the kth frame image, q is the position coordinate of the target in the kth frame image, H is the observation matrix, P is the posterior error estimation covariance matrix, and R is the observation noise covariance matrix;
updating the posterior error estimation covariance matrix according to:
P＝P′-K′HP
wherein P' =apa T +Q, Q is the prediction noise covariance matrix.
A second aspect of embodiments of the present application provides an object tracking device, which may include a module/unit implementing any of the above-mentioned object tracking methods.
A third aspect of the embodiments of the present application provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of any one of the target tracking methods described above.
A fourth aspect of the embodiments of the present application provides a robot comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of any one of the object tracking methods described above when executing the computer program.
A fifth aspect of the embodiments of the present application provides a computer program product for, when run on a robot, causing the robot to perform the steps of any one of the object tracking methods described above.
Compared with the prior art, the embodiment of the application has the beneficial effects that: in order to ensure the tracking effectiveness, the tracking of the target is not lost due to the fact that the target moves too fast or interference of other people occurs, a Kalman filter is used for predicting the position coordinates of the target possibly occurring in the next frame, the position coordinates of the target occurring in the previous frame are considered, and the position coordinates of the target in the next frame image are selected through the relevant filter. On the premise of ensuring higher real-time performance of the target tracking algorithm, the boundary effect is processed to influence the tracking algorithm, so that the performance of the tracking algorithm is improved, and the targets possibly appearing in the security and protection places are tracked in real time and efficiently.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly introduce the drawings that are needed in the embodiments or the description of the prior art, it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of an initialization process;
FIG. 2 is a flow chart of one embodiment of a target tracking method in the embodiments of the present application;
FIG. 3 is a schematic diagram of the correlation between a target detection frame and a region to be detected;
FIG. 4 is a block diagram of one embodiment of a target tracking device according to an embodiment of the present application;
fig. 5 is a schematic block diagram of a robot in an embodiment of the present application.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the embodiments described below are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in this specification and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
In addition, in the description of the present application, the terms "first," "second," "third," etc. are used merely to distinguish between descriptions and are not to be construed as indicating or implying relative importance.
In embodiments of the present application, the targets to be tracked may include, but are not limited to: personnel, vehicles, animals, and other moving objects.
It is easy to understand that the target detection is a precondition for performing target tracking, and in the initial state, the target detection needs to be performed in the designated area by the target detector, where the detection method used may be any detection method in the prior art, and the embodiment of the present application is not limited specifically.
When a target is detected in a certain frame image, the position coordinates of the target in the frame image and a detection frame (bb) are determined, and this position is marked as initial coordinates and this detection frame is marked as initial detection frame.
After the target detection is completed, an initialization process of target tracking may be performed, as shown in fig. 1, and may include:
step S101, initializing the value of k.
In the embodiment of the present application, k is an image frame number for tracking the target, and k is a positive integer. Regarding the frame image in which the target is detected as the 0 th frame, after the target detection is completed, the value of k may be initialized to 1, that is, the following steps are performed: k=1.
Step S102, initializing the correlation filter (Correlation Filter, CF).
Specifically, the parameters of the correlation filter are initialized according to the following equation:
wherein,x is the image of the m-times size area of the initial detection frame with the initial coordinate as the center, m is a preset multiple parameter, the specific value of the m is set according to practical situations, preferably, the x is set to 2.5, namely, the search area is 2.5 times of the size of the detection frame, y is the Gaussian function of the m-times size area of the initial detection frame with the initial coordinate as the center, and the Gaussian function is used as a label of a training correlation filter, sigma 2 Variance of Gaussian function, F (&) is Fourier transform, F -1 (. Cndot.) Fourier inverse transform, lambda is the penalty constant for training, and alpha represents the matrixAnd (3) performing a multiplication operation item by item, wherein alpha is a parameter of the correlation filter.
Step S103, initializing a Kalman Filter (KF).
Specifically, in the present embodiment, the state transition matrix (denoted herein as a) may be initialized to:
the observation matrix (denoted herein as H) is initialized to:
initializing a prediction noise covariance matrix into a diagonal matrix of 4 rows and 4 columns, wherein diagonal elements are variances of Gaussian white noise;
initializing an observation noise covariance matrix into a diagonal matrix of 2 rows and 2 columns, wherein diagonal elements are variances of Gaussian white noise;
initializing a posterior error estimation covariance matrix into a unit diagonal matrix of 4 rows and 4 columns;
the state of the target (denoted herein as w) is initialized to:
w＝[u 0 v 0 Δu Δv] T
wherein, (u) 0 ,v 0 ) For the initial coordinates, (Δu, Δv) is the motion velocity obtained by random initialization.
In tracking the target, the specific tracking process of each frame image is similar, and a specific example of any frame (kth frame) is described in detail below. As shown in fig. 2, the process of tracking the target in the kth frame image may include:
step S201, calculating the predicted position coordinates of the target in the kth frame of image by using a Kalman filter according to the position coordinates of the target in the kth-1 frame of image.
Specifically, the predicted position coordinates of the target in the kth frame image may be calculated according to the following equation:
w′＝Aw
(u KF ,v KF )＝(w′[0],w′[1])
wherein, (u) KF ,v KF ) And the predicted position coordinates are obtained.
And step S202, determining a first search area in the k-1 frame image according to the predicted position coordinates and the detection frame in the k-1 frame image.
As shown in fig. 3, the left image is the k-1 frame, the rectangle frame with a broken line indicates the detection frame of the target tracked in the k-1 frame, and then the area to be detected in the k frame (right image) is m times as large as the detection frame, that is, the size of the solid line frame in the image, and the coordinates of the target in the k frame in the image will move to the upper left corner according to the motion track of the target. The right image is the input k frame image, and the solid line box on the right is the tracking area of the classical correlation filter at the k frame, which is denoted herein as the second search area. It can be seen that due to the movement of the object, the object has moved in the image to the edge of the tracking area of the classical correlation filter at the kth frame, resulting in the occurrence of a "boundary effect", which may fail tracking at this time; when the kalman filter is introduced to predict the motion trail of the target, the kalman filter can predict the possible coordinates (moving to the upper left) of the target in the kth frame image by combining the motion trail of the target in the previous received image, so that the tracking area when the kalman filter outputs the kth frame to the relevant filter is shown by the solid line box on the left, and is marked as the first search area.
And step 203, tracking the target in the first search area by using a correlation filter to obtain a first maximum response value and a first position coordinate.
Specifically, the target may be tracked in the first search area using a correlation filter, resulting in a first response matrix as follows:
CF 1 ＝F -1 (F(k<x,z 1 >)⊙F (α))
wherein z is 1 Is the firstImage of a search area, CF 1 Is the first response matrix;
and determining the largest response value in the first response matrix as the first largest response value, and determining the position coordinate corresponding to the first largest response value as the first position coordinate. Here, the first maximum response value is denoted as p 1 Marking the first position as
And S204, determining a second search area in the k-1 frame image according to the position coordinates of the target in the k-1 frame image and the detection frame.
And step S205, tracking the target in the second search area by using a correlation filter to obtain a second maximum response value and a second position coordinate.
Specifically, the target may be tracked in the second search area using a correlation filter, resulting in a second response matrix as follows:
CF 2 ＝F -1 (F(k<x,z 2 >)⊙F (α))
wherein z is 2 For the image of the second search area, CF 2 For the second response matrix;
and determining the largest response value in the second response matrix as the second largest response value, and determining the position coordinate corresponding to the second largest response value as the second position coordinate. Here, the second maximum response value is denoted as p 2 Marking the second position as
And S206, selecting the position coordinates of the target in the kth frame of image from the first position coordinates and the second position coordinates according to the first maximum response value and the second maximum response value.
Specifically, if the first maximum response value is greater than the second maximum response value, determining the first position coordinate as the position coordinate of the target in the kth frame image, namely:
x＝z 1
wherein, (u) k ,v k ) Namely the position coordinates of the target in the kth frame of image.
If the first maximum response value is smaller than or equal to the second maximum response value, determining the second position coordinate as the position coordinate of the target in the kth frame image, namely:
x＝z 2 。
after the position coordinates of the target in the kth frame of image are selected, the correlation filter and the kalman filter can be updated respectively through the following processes:
when updating the correlation filter, the parameters of the correlation filter may be updated according to the following equation:
α＝(1-γ)α+γα′
wherein,and x is an image taking the position coordinate of the target in the kth frame image as the center, taking the m-times-sized region of the detection frame in the kth frame image, y is a Gaussian function taking the m-times-sized region of the detection frame in the kth frame image as the center, taking the label of the training related filter, and gamma is a weight constant for parameter updating.
When updating the kalman filter, the state of the object may be updated according to the following equation:
w＝w′+K′(q-Hw′)
wherein K' =ph T (HPH T +R) -1 W' is the predicted bit of the object in the kth frame imageSetting coordinates, q is the position coordinates of the target in the kth frame image, i.e. q= (u) k ,v k ) T H is an observation matrix, P is a posterior error estimation covariance matrix, and R is an observation noise covariance matrix.
The posterior error estimation covariance matrix may be updated according to:
P＝P′-K′HP
wherein P' =apa T +Q, Q is the prediction noise covariance matrix.
If the tracking process is not finished, continuing to track the target in the next frame of image, namely executing the following steps: k=k+1 and the tracking step shown in fig. 2 is re-performed until the entire tracking process ends.
In summary, in order to ensure the tracking effectiveness, the embodiments of the present application do not lose tracking of the target due to the fact that the target moves too fast or other person interference occurs, a kalman filter is used to predict the possible position coordinates of the target in the next frame, then the position coordinates of the target in the previous frame are considered, and the position coordinates of the target in the next frame image are selected from the position coordinates through the relevant filter. On the premise of ensuring higher real-time performance of the target tracking algorithm, the boundary effect is processed to influence the tracking algorithm, so that the performance of the tracking algorithm is improved, and the targets possibly appearing in the security and protection places are tracked in real time and efficiently.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic of each process, and should not limit the implementation process of the embodiment of the present application in any way.
Fig. 4 shows a block diagram of an embodiment of a target tracking apparatus according to an embodiment of the present application, corresponding to a target tracking method described in the foregoing embodiment.
In this embodiment, a target tracking apparatus may include:
a position coordinate prediction module 401, configured to calculate, according to a position coordinate of a target in a kth frame image, a predicted position coordinate of the target in the kth frame image using a kalman filter, where k is a positive integer;
a first search area determining module 402, configured to determine a first search area in a kth frame image according to the predicted position coordinates and a detection frame in the kth-1 frame image;
a first tracking module 403, configured to track the target in the first search area by using a correlation filter, to obtain a first maximum response value and a first position coordinate;
a second search area determining module 404, configured to determine a second search area in the kth frame image according to the position coordinates of the target in the kth-1 frame image and the detection frame;
a second tracking module 405, configured to track the target in the second search area by using a correlation filter, to obtain a second maximum response value and a second position coordinate;
and a position coordinate selecting module 406, configured to select, according to the first maximum response value and the second maximum response value, a position coordinate of the target in the kth frame image from the first position coordinate and the second position coordinate.
Further, the location coordinate selection module 406 may include:
a first selecting unit 4061, configured to determine the first position coordinate as a position coordinate of the target in a kth frame image if the first maximum response value is greater than the second maximum response value;
and a second selecting unit 4062, configured to determine the second position coordinate as the position coordinate of the target in the kth frame image if the first maximum response value is less than or equal to the second maximum response value.
Further, the target tracking apparatus may further include:
a target detection module 407, configured to perform target detection in a specified area, and determine initial coordinates and an initial detection frame of the detected target;
an initialization module 408, configured to perform an initialization procedure, where the initialization procedure includes: initializing the value of k; initializing a correlation filter; the kalman filter is initialized.
Further, the initialization module 408 may include:
a correlation filter parameter initializing unit 4081 for initializing parameters of the correlation filter according to the following formula:
wherein,x is the image of m times size area of the initial detection frame taking the initial coordinate as the center, m is a preset multiple parameter, y is the Gaussian function of m times size area of the initial detection frame taking the initial coordinate as the center, and sigma is used as the label of the training correlation filter 2 Variance of Gaussian function, F (&) is Fourier transform, F -1 (. Cndot.) Fourier inverse transform, λ is the penalty constant for training, by which is the matrix term-wise multiplication operation, α is the parameter of the correlation filter;
a kalman filter initializing unit 4082 for initializing the state transition matrix as:
initializing an observation matrix as:
initializing a prediction noise covariance matrix into a diagonal matrix of 4 rows and 4 columns, wherein diagonal elements are variances of Gaussian white noise;
initializing an observation noise covariance matrix into a diagonal matrix of 2 rows and 2 columns, wherein diagonal elements are variances of Gaussian white noise;
initializing a posterior error estimation covariance matrix into a unit diagonal matrix of 4 rows and 4 columns;
initializing the state of the target as:
w＝[u 0 v 0 Δu Δv] T
wherein, (u) 0 ,v 0 ) For the initial coordinates, (Δu, Δv) is the motion velocity obtained by random initialization.
Further, the first tracking module 403 is specifically configured to:
tracking the target in the first search area by using a correlation filter to obtain a first response matrix shown as follows:
CF 1 ＝F -1 (F(k<x,z 1 >)⊙F (α))
wherein z is 1 For the image of the first search area, CF 1 Is the first response matrix;
and determining the largest response value in the first response matrix as the first largest response value, and determining the position coordinate corresponding to the first largest response value as the first position coordinate.
Further, the second tracking module 405 is specifically configured to:
tracking the target in the second search area by using a correlation filter to obtain a second response matrix as follows:
CF 2 ＝F -1 (F(k<x,z 2 >)⊙F (α))
wherein z is 2 For the image of the second search area, CF 2 For the second response matrix;
and determining the largest response value in the second response matrix as the second largest response value, and determining the position coordinate corresponding to the second largest response value as the second position coordinate.
Further, the target tracking apparatus may further include:
a correlation filter updating module 409, configured to update parameters of the correlation filter according to the following formula:
α＝(1-γ)α+γα′
wherein,taking the position coordinate of the target in the kth frame image as the center, taking an image of an m-time size region of a detection frame in the kth frame image, taking the position coordinate of the target in the kth frame image as the center, taking a Gaussian function of the m-time size region of the detection frame in the kth frame image as the center, taking a label of a training related filter, and taking gamma as a weight constant of parameter updating;
a kalman filter update module 410 for:
updating the state of the object according to the following formula:
w＝w′+K′(q-Hw′)
wherein K' =ph T (HPH T +R) -1 W' is the predicted position coordinate of the target in the kth frame image, q is the position coordinate of the target in the kth frame image, H is the observation matrix, P is the posterior error estimation covariance matrix, and R is the observation noise covariance matrix;
updating the posterior error estimation covariance matrix according to:
P＝P′-K′HP
wherein P' =apa T +Q, Q is the prediction noise covariance matrix.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described apparatus, modules and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Fig. 5 shows a schematic block diagram of a robot provided in an embodiment of the present application, and for convenience of explanation, only a portion related to the embodiment of the present application is shown.
As shown in fig. 5, the robot 5 of this embodiment includes: a processor 50, a memory 51 and a computer program 52 stored in said memory 51 and executable on said processor 50. The processor 50, when executing the computer program 52, implements the steps of the various target tracking method embodiments described above, such as steps S201 to S206 shown in fig. 2. Alternatively, the processor 50 may perform the functions of the modules/units of the apparatus embodiments described above, such as the functions of the modules 401 to 406 shown in fig. 4, when executing the computer program 52.
By way of example, the computer program 52 may be partitioned into one or more modules/units that are stored in the memory 51 and executed by the processor 50 to complete the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing a specific function for describing the execution of the computer program 52 in the robot 5.
It will be appreciated by those skilled in the art that fig. 5 is merely an example of the robot 5 and is not meant to be limiting of the robot 5, and may include more or fewer components than shown, or may combine certain components, or different components, e.g., the robot 5 may also include input and output devices, network access devices, buses, etc.
The processor 50 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), field programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 51 may be an internal storage unit of the robot 5, such as a hard disk or a memory of the robot 5. The memory 51 may be an external storage device of the robot 5, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the robot 5. Further, the memory 51 may also include both an internal memory unit and an external memory device of the robot 5. The memory 51 is used for storing the computer program as well as other programs and data required by the robot 5. The memory 51 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/robot and method may be implemented in other ways. For example, the apparatus/robot embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated modules/units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each method embodiment described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the computer readable medium contains content that can be appropriately scaled according to the requirements of jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is subject to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.
The above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.
Claims (9)
1. A target tracking method, comprising the steps of:
calculating the predicted position coordinate of the target in the kth frame image by using a Kalman filter according to the position coordinate of the target in the kth-1 frame image, wherein k is a positive integer;
determining a first search area in a kth frame image according to the predicted position coordinates and a detection frame in the kth-1 frame image;
tracking the target in the first search area by using a correlation filter to obtain a first maximum response value and a first position coordinate;
determining a second search area in the kth frame of image according to the position coordinates of the target in the kth-1 frame of image and the detection frame;
tracking the target in the second search area by using a correlation filter to obtain a second maximum response value and a second position coordinate;
if the first maximum response value is larger than the second maximum response value, determining the first position coordinate as the position coordinate of the target in the kth frame image;
and if the first maximum response value is smaller than or equal to the second maximum response value, determining the second position coordinate as the position coordinate of the target in the kth frame image.
2. The object tracking method according to claim 1, characterized by further comprising, before calculating predicted position coordinates of the object in a kth frame image using a kalman filter:
performing target detection in a designated area, and determining initial coordinates and an initial detection frame of a detected target;
performing an initialization process, the initialization process comprising: initializing the value of k; initializing a correlation filter; the kalman filter is initialized.
3. The method of claim 2, wherein initializing the correlation filter comprises:
initializing parameters of the correlation filter according to:
wherein,x is the image of the m-times-sized region of the initial detection frame with the initial coordinate as the center, m is a preset multiple parameter, y is the Gaussian function of the m-times-sized region of the initial detection frame with the initial coordinate as the center, and the Gaussian function is used as a training correlation filterLabel, sigma 2 Variance of Gaussian function, F (&) is Fourier transform, F -1 (. Cndot.) Fourier inverse transform, λ is the penalty constant for training, by which is the matrix term-wise multiplication operation, α is the parameter of the correlation filter;
the initializing of the Kalman filter includes:
initializing a state transition matrix as:
initializing an observation matrix as:
initializing a prediction noise covariance matrix into a diagonal matrix of 4 rows and 4 columns, wherein diagonal elements are variances of Gaussian white noise;
initializing an observation noise covariance matrix into a diagonal matrix of 2 rows and 2 columns, wherein diagonal elements are variances of Gaussian white noise;
initializing a posterior error estimation covariance matrix into a unit diagonal matrix of 4 rows and 4 columns;
initializing the state of the target as:
w＝[u 0 v 0 Δu Δv] T
wherein, (u) 0 ,v 0 ) For the initial coordinates, (Δu, Δv) is the motion velocity obtained by random initialization.
4. The method of claim 1, wherein tracking the target in the first search area using a correlation filter to obtain a first maximum response value and a first position coordinate, comprises:
tracking the target in the first search area by using a correlation filter to obtain a first response matrix shown as follows:
CF 1 ＝F -1 (F(k<x,z 1 >)⊙F(α))
wherein z is 1 For the image of the first search area, CF 1 Is the first response matrix;
and determining the largest response value in the first response matrix as the first largest response value, and determining the position coordinate corresponding to the first largest response value as the first position coordinate.
5. The method of claim 1, wherein tracking the target in the second search area using a correlation filter to obtain a second maximum response value and a second position coordinate, comprising:
tracking the target in the second search area by using a correlation filter to obtain a second response matrix as follows:
CF 2 ＝F -1 (F(k<x,z 2 >)⊙F(α))
wherein z is 2 For the image of the second search area, CF 2 For the second response matrix;
and determining the largest response value in the second response matrix as the second largest response value, and determining the position coordinate corresponding to the second largest response value as the second position coordinate.
6. The object tracking method according to any one of claims 1 to 5, characterized by further comprising, after selecting the position coordinates of the object in the kth frame image:
updating parameters of the correlation filter according to:
α＝(1-γ)α+γα′
wherein,x is an image of an m-times-sized region of a detection frame in a kth frame image taking the position coordinate of the target in the kth frame image as the center, and y is an image of the targetTaking a Gaussian function of an m-time size area of a detection frame in a kth frame image as a label of a training related filter, wherein gamma is a weight constant of parameter updating;
updating the state of the object according to the following formula:
w＝w′+K′(q-Hw′)
wherein K' =ph T (HPH T +R) -1 W' is the predicted position coordinate of the target in the kth frame image, q is the position coordinate of the target in the kth frame image, H is the observation matrix, P is the posterior error estimation covariance matrix, and R is the observation noise covariance matrix;
updating the posterior error estimation covariance matrix according to:
P＝P′-K′HP
wherein P' =apa T +Q, Q is the prediction noise covariance matrix.
7. An object tracking device, characterized by comprising the following modules:
the position coordinate prediction module calculates the predicted position coordinate of the target in the kth frame image by using a Kalman filter according to the position coordinate of the target in the kth-1 frame image, wherein k is a positive integer;
the first search area determining module is used for determining a first search area in a kth frame image according to the predicted position coordinates and a detection frame in the kth-1 frame image;
the first tracking module is used for tracking the target in the first search area by using a correlation filter to obtain a first maximum response value and a first position coordinate;
the second search area determining module is used for determining a second search area in the kth frame of image according to the position coordinates of the target in the kth-1 frame of image and the detection frame;
the second tracking module is used for tracking the target in the second search area by using a correlation filter to obtain a second maximum response value and a second position coordinate;
the position coordinate selection module is used for determining the first position coordinate as the position coordinate of the target in the kth frame image if the first maximum response value is larger than the second maximum response value; and if the first maximum response value is smaller than or equal to the second maximum response value, determining the second position coordinate as the position coordinate of the target in the kth frame image.
8. A computer-readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the object tracking method according to any one of claims 1 to 6.
9. A robot comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the object tracking method according to any one of claims 1 to 6 when the computer program is executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010259829.9A CN111563919B (en) | 2020-04-03 | 2020-04-03 | Target tracking method, device, computer readable storage medium and robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010259829.9A CN111563919B (en) | 2020-04-03 | 2020-04-03 | Target tracking method, device, computer readable storage medium and robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111563919A CN111563919A (en) | 2020-08-21 |
| CN111563919B true CN111563919B (en) | 2023-12-29 |
Family
ID=72072950
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010259829.9A Active CN111563919B (en) | 2020-04-03 | 2020-04-03 | Target tracking method, device, computer readable storage medium and robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111563919B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113139416A (en) * | 2020-11-17 | 2021-07-20 | 西安天伟电子系统工程有限公司 | Object association method, computer device, and storage medium |
| CN112507906A (en) * | 2020-12-14 | 2021-03-16 | 北京澎思科技有限公司 | Target tracking method, device and computer readable storage medium |
| CN112750146B (en) * | 2020-12-31 | 2023-09-12 | 浙江大华技术股份有限公司 | Target object tracking method and device, storage medium and electronic equipment |
| CN112669351A (en) * | 2021-01-05 | 2021-04-16 | 北京声智科技有限公司 | Rapid movement detection method and device |
| CN112651377B (en) * | 2021-01-05 | 2023-06-09 | 河北建筑工程学院 | Ice and snow sport accident detection method and device and terminal equipment |
| CN113129339B (en) * | 2021-04-28 | 2023-03-10 | 北京市商汤科技开发有限公司 | Target tracking method and device, electronic equipment and storage medium |
| CN113313739A (en) * | 2021-06-23 | 2021-08-27 | 中国农业银行股份有限公司 | Target tracking method, device and storage medium |
| CN114399528B (en) * | 2021-11-29 | 2024-06-07 | 深圳先进技术研究院 | Three-dimensional space moving target tracking method and related device based on two-dimensional image |
| CN115690333B (en) * | 2022-12-30 | 2023-04-28 | 思看科技(杭州)股份有限公司 | Three-dimensional scanning method and system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104992451A (en) * | 2015-06-25 | 2015-10-21 | 河海大学 | Improved target tracking method |
| CN105469430A (en) * | 2015-12-10 | 2016-04-06 | 中国石油大学(华东) | Anti-shielding tracking method of small target in large-scale scene |
| CN108010067A (en) * | 2017-12-25 | 2018-05-08 | 北京航空航天大学 | A kind of visual target tracking method based on combination determination strategy |
| CN108198209A (en) * | 2017-12-22 | 2018-06-22 | 天津理工大学 | It is blocking and dimensional variation pedestrian tracking algorithm |
| CN110233667A (en) * | 2019-06-05 | 2019-09-13 | 华南理工大学 | VLC dynamic positioning method and system based on average drifting and Unscented kalman filtering |
| CN110276785A (en) * | 2019-06-24 | 2019-09-24 | 电子科技大学 | One kind is anti-to block infrared object tracking method |
| CN110706252A (en) * | 2019-09-09 | 2020-01-17 | 西安理工大学 | Robot nuclear correlation filtering tracking algorithm under guidance of motion model |
| CN110796687A (en) * | 2019-10-30 | 2020-02-14 | 电子科技大学 | Sky background infrared imaging multi-target tracking method |
-
2020
- 2020-04-03 CN CN202010259829.9A patent/CN111563919B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104992451A (en) * | 2015-06-25 | 2015-10-21 | 河海大学 | Improved target tracking method |
| CN105469430A (en) * | 2015-12-10 | 2016-04-06 | 中国石油大学(华东) | Anti-shielding tracking method of small target in large-scale scene |
| CN108198209A (en) * | 2017-12-22 | 2018-06-22 | 天津理工大学 | It is blocking and dimensional variation pedestrian tracking algorithm |
| CN108010067A (en) * | 2017-12-25 | 2018-05-08 | 北京航空航天大学 | A kind of visual target tracking method based on combination determination strategy |
| CN110233667A (en) * | 2019-06-05 | 2019-09-13 | 华南理工大学 | VLC dynamic positioning method and system based on average drifting and Unscented kalman filtering |
| CN110276785A (en) * | 2019-06-24 | 2019-09-24 | 电子科技大学 | One kind is anti-to block infrared object tracking method |
| CN110706252A (en) * | 2019-09-09 | 2020-01-17 | 西安理工大学 | Robot nuclear correlation filtering tracking algorithm under guidance of motion model |
| CN110796687A (en) * | 2019-10-30 | 2020-02-14 | 电子科技大学 | Sky background infrared imaging multi-target tracking method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111563919A (en) | 2020-08-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111563919B (en) | Target tracking method, device, computer readable storage medium and robot | |
| US10558891B2 (en) | Systems and methods for object tracking | |
| Chen et al. | Structure-adaptive fuzzy estimation for random-valued impulse noise suppression | |
| WO2021227519A1 (en) | Target tracking method and apparatus, and computer-readable storage medium and robot | |
| CN110766724B (en) | Target tracking network training and tracking method and device, electronic equipment and medium | |
| US11205276B2 (en) | Object tracking method, object tracking device, electronic device and storage medium | |
| CN111104925B (en) | Image processing method, image processing apparatus, storage medium, and electronic device | |
| WO2009082700A1 (en) | Online articulate object tracking with appearance and shape | |
| CN110349188B (en) | Multi-target tracking method, device and storage medium based on TSK fuzzy model | |
| CN111612822B (en) | Object tracking method, device, computer equipment and storage medium | |
| CN112733767A (en) | Human body key point detection method and device, storage medium and terminal equipment | |
| CN110766725B (en) | Template image updating method and device, target tracking method and device, electronic equipment and medium | |
| CN109410246B (en) | Visual tracking method and device based on correlation filtering | |
| CN110799984A (en) | Tracking control method, device and computer readable storage medium | |
| CN112700472A (en) | Target tracking method and related equipment | |
| CN111768427A (en) | Multi-moving-target tracking method and device and storage medium | |
| CN108447066B (en) | Biliary tract image segmentation method, terminal and storage medium | |
| CN116433722A (en) | Target tracking method, electronic device, storage medium, and program product | |
| CN109166138B (en) | Target tracking method and device based on high-order cumulant and storage medium | |
| CN108230284B (en) | Motion trail determination method and device | |
| CN111540016B (en) | Pose calculation method and device based on image feature matching, computer equipment and storage medium | |
| CN114463377A (en) | Target tracking optimization method and device, computer equipment and storage medium | |
| GB2457584A (en) | Template block matching using a similarity function and incremental threshold | |
| CN111260692A (en) | Face tracking method, device, equipment and storage medium | |
| CN112528925B (en) | Pedestrian tracking and image matching method and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |