CN111563919A - Target tracking method and device, computer readable storage medium and robot - Google Patents

Target tracking method and device, computer readable storage medium and robot Download PDF

Info

Publication number
CN111563919A
CN111563919A CN202010259829.9A CN202010259829A CN111563919A CN 111563919 A CN111563919 A CN 111563919A CN 202010259829 A CN202010259829 A CN 202010259829A CN 111563919 A CN111563919 A CN 111563919A
Authority
CN
China
Prior art keywords
target
position coordinate
frame image
tracking
response value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010259829.9A
Other languages
Chinese (zh)
Other versions
CN111563919B (en
Inventor
胡淑萍
程骏
张惊涛
郭渺辰
王东
顾在旺
庞建新
熊友军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ubtech Robotics Corp
Original Assignee
Ubtech Robotics Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ubtech Robotics Corp filed Critical Ubtech Robotics Corp
Priority to CN202010259829.9A priority Critical patent/CN111563919B/en
Publication of CN111563919A publication Critical patent/CN111563919A/en
Application granted granted Critical
Publication of CN111563919B publication Critical patent/CN111563919B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory

Abstract

The present application relates to the field of robotics, and in particular, to a target tracking method and apparatus, a computer-readable storage medium, and a robot. The method comprises the following steps: according to the position coordinates of the target in the k-1 frame image, calculating the predicted position coordinates of the target in the k frame image by using a Kalman filter; determining a first search area in the k frame image according to the predicted position coordinates and a detection frame in the k-1 frame image; tracking the target in the first search area by using a correlation filter to obtain a first maximum response value and a first position coordinate; determining a second search area in the kth frame image according to the position coordinate of the target in the kth-1 frame image and the detection frame; tracking the target in the second search area by using a correlation filter to obtain a second maximum response value and a second position coordinate; and selecting the position coordinates of the target in the k frame image from the two position coordinates.

Description

Target tracking method and device, computer readable storage medium and robot
Technical Field
The present application relates to the field of robotics, and in particular, to a target tracking method and apparatus, a computer-readable storage medium, and a robot.
Background
The security robot, as a robot for assisting a human in completing security protection work, needs to have the capability of detecting and tracking a target that may appear in a security place. However, the high complexity/low real-time performance of the detection algorithm and the characteristic of being easily affected by shielding limit the real-time tracking capability of the security robot on the target appearing in the security place, and therefore, the target tracking algorithm should be started after the target is detected. At present, a target tracking algorithm has the advantages of high real-time performance, certain anti-blocking capability and the like, but due to the limitation of the tracking algorithm, when a new image of each frame is tracked, only whether a target exists near a pixel coordinate where the target may appear or not can be tracked, which causes a situation that the tracking fails when the target moves fast and reaches the outside or edge of a tracking range, and the situation is generally called as a boundary effect. In order to cope with the boundary effect, the simplest method is to expand the tracking range, but this may cause the target tracker to learn too much background information in the learning process, and adversely affect the tracking effect. At present, related documents attempt to solve the boundary effect, including SRDCF and CFLB algorithms, which directly modify an optimized objective function of a target tracker, so that the target tracker has no closed solution, and the target tracker can only be solved by an iterative method in the learning and updating processes, which may reduce the operating speed of the target tracking algorithm and may not meet the high real-time requirement.
Disclosure of Invention
In view of this, embodiments of the present application provide a target tracking method, an apparatus, a computer-readable storage medium, and a robot, so as to solve the problem that the existing target tracking method is poor in real-time performance.
A first aspect of an embodiment of the present application provides a target tracking method, which may include:
according to the position coordinate of the target in the k-1 frame image, calculating the predicted position coordinate of the target in the k frame image by using a Kalman filter, wherein k is a positive integer;
determining a first search area in the k frame image according to the predicted position coordinates and a detection frame in the k-1 frame image;
tracking the target in the first search area by using a correlation filter to obtain a first maximum response value and a first position coordinate;
determining a second search area in the kth frame image according to the position coordinate of the target in the kth-1 frame image and the detection frame;
tracking the target in the second search area by using a correlation filter to obtain a second maximum response value and a second position coordinate;
and selecting the position coordinate of the target in the k frame image from the first position coordinate and the second position coordinate according to the first maximum response value and the second maximum response value.
Further, the selecting, according to the first maximum response value and the second maximum response value, a position coordinate of the target in a k-th frame image from the first position coordinate and the second position coordinate includes:
if the first maximum response value is larger than the second maximum response value, determining the first position coordinate as the position coordinate of the target in the k frame image;
and if the first maximum response value is smaller than or equal to the second maximum response value, determining the second position coordinate as the position coordinate of the target in the k frame image.
Further, before calculating the predicted position coordinates of the target in the k frame image using the kalman filter, the method further includes:
carrying out target detection in the designated area, and determining an initial coordinate and an initial detection frame of the detected target;
performing an initialization process, the initialization process comprising: initializing the value of k; initializing a correlation filter; the kalman filter is initialized.
Further, the initializing the correlation filter includes:
parameters of the correlation filter are initialized according to:
Figure BDA0002438877440000031
wherein the content of the first and second substances,
Figure BDA0002438877440000032
taking the image of the m-time area of the initial detection frame by taking the initial coordinate as a center, taking m as a preset multiple parameter, taking the Gaussian function of the m-time area of the initial detection frame by taking the initial coordinate as a center, and taking the Gaussian function as a label of a training related filter, wherein the sigma is2Is the variance of a gaussian function and is,
Figure BDA0002438877440000033
in order to perform the fourier transformation, the method,
Figure BDA0002438877440000034
an inverse Fourier transform, wherein lambda is a penalty term constant of training, ⊙ represents a matrix multiplication operation item by item, and α is a parameter of a correlation filter;
the initializing the kalman filter includes:
initializing the state transition matrix to:
Figure BDA0002438877440000035
the observation matrix is initialized to:
Figure BDA0002438877440000036
initializing a predictive noise covariance matrix into a diagonal matrix with 4 rows and 4 columns, wherein the diagonal elements are the variances of Gaussian white noise;
initializing an observation noise covariance matrix into a diagonal matrix with 2 rows and 2 columns, wherein the diagonal elements are the variances of Gaussian white noise;
initializing a posterior error estimation covariance matrix into a unit diagonal matrix with 4 rows and 4 columns;
initializing a state of the target to:
w＝[u0v0Δu Δv]T
wherein (u)0,v0) For the initial coordinates, (Δ u, Δ v) is the motion velocity obtained by random initialization.
Further, the tracking the target in the first search area by using a correlation filter to obtain a first maximum response value and a first position coordinate includes:
tracking the target in the first search area using a correlation filter, resulting in a first response matrix as shown below:
Figure BDA0002438877440000041
wherein z is1For the image of the first search area, CF1Is the first response matrix;
determining a maximum response value in the first response matrix as the first maximum response value, and determining a position coordinate corresponding to the first maximum response value as the first position coordinate.
Further, the tracking the target in the second search area using a correlation filter to obtain a second maximum response value and a second position coordinate includes:
tracking the target in the second search area by using a correlation filter to obtain a second response matrix as shown in the following:
Figure BDA0002438877440000042
wherein z is2For the image of the second search area, CF2Is the second response matrix;
and determining the maximum response value in the second response matrix as the second maximum response value, and determining the position coordinate corresponding to the second maximum response value as the second position coordinate.
Further, after the position coordinates of the target in the k frame image are selected, the method further comprises the following steps:
updating parameters of the correlation filter according to:
α＝(1-γ)α+γα′
wherein the content of the first and second substances,
Figure BDA0002438877440000043
taking the image of the m-times area of the detection frame in the k frame image by taking the position coordinate of the target in the k frame image as the center, taking the Gaussian function as a label for training a related filter, and taking gamma as a weight constant for updating parameters;
updating the state of the target according to:
w＝w′+K′(q-Hw′)
wherein, K ═ PHT(HPHT+R)-1W' is the predicted position coordinate of the target in the k frame image, q is the position coordinate of the target in the k frame image, H is an observation matrix, P is an a posteriori error estimation covariance matrix, and R is an observation noise covariance matrix;
updating the a posteriori error estimate covariance matrix according to:
P＝P′-K′HP
wherein, P' ═ APAT+ Q, Q is the predicted noise covariance matrix.
A second aspect of embodiments of the present application provides a target tracking apparatus, which may include a module/unit that implements any one of the target tracking methods described above.
A third aspect of embodiments of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps of any one of the above-mentioned target tracking methods.
A fourth aspect of embodiments of the present application provides a robot, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned target tracking methods when executing the computer program.
A fifth aspect of embodiments of the present application provides a computer program product, which, when run on a robot, causes the robot to perform the steps of any one of the above-mentioned target tracking methods.
Compared with the prior art, the embodiment of the application has the advantages that: in order to ensure the effectiveness of tracking, the tracking of the target cannot be lost due to too fast movement of the target or interference of other people, a Kalman filter is used for predicting the position coordinates of the target in the next frame, the position coordinates of the target in the previous frame are considered, and the position coordinates of the target in the next frame of image are selected through a related filter. On the premise of ensuring high real-time performance of the target tracking algorithm, the influence of the boundary effect on the tracking algorithm is processed, the performance of the tracking algorithm is improved, and the target which possibly appears in a security and protection place is tracked in real time and efficiently.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed for the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic flow diagram of an initialization process;
FIG. 2 is a flowchart of an embodiment of a target tracking method in an embodiment of the present application;
FIG. 3 is a schematic diagram of a correlation between a detection frame of a target and a region to be detected;
FIG. 4 is a block diagram of one embodiment of a target tracking device in an embodiment of the present application;
fig. 5 is a schematic block diagram of a robot according to an embodiment of the present application.
Detailed Description
In order to make the objects, features and advantages of the present invention more apparent and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the embodiments described below are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
In addition, in the description of the present application, the terms "first," "second," "third," and the like are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
In the embodiment of the present application, the target to be tracked may include, but is not limited to: people, vehicles, animals, and other moving objects.
It is easy to understand that the target detection is a precondition for performing target tracking, and in an initial state, the target detection needs to be performed in a specified area by a target detector first, where the detection method used may be any one of the detection methods in the prior art, and this is not specifically limited in this embodiment of the present application.
When an object is detected in a certain frame image, the position coordinate of the object in the frame image and a detection frame (bb) are determined, and here, the position coordinate is regarded as an initial coordinate, and the detection frame is regarded as an initial detection frame.
After the target detection is completed, an initialization process of target tracking may be performed, as shown in fig. 1, the initialization process may include:
and step S101, initializing the value of k.
In the embodiment of the present application, k is an image frame number for tracking a target, and k is a positive integer. Here, the frame image in which the target is detected is regarded as the 0 th frame, and after the target detection is completed, the value of k may be initialized to 1, that is, the following steps are performed: k is 1.
Step S102 is to initialize a Correlation Filter (CF).
Specifically, the parameters of the correlation filter are initialized according to:
Figure BDA0002438877440000071
wherein the content of the first and second substances,
Figure BDA0002438877440000072
taking x as an image of an m-time area of the initial detection frame by taking the initial coordinate as a center, taking m as a preset multiple parameter, wherein the specific value can be set according to the actual situation, preferably, the specific value is set to 2.5, namely, a search area is 2.5 times of the size of the detection frame, taking y as a Gaussian function of the m-time area of the initial detection frame by taking the initial coordinate as a center, and taking the Gaussian function as a label of a training related filter, wherein sigma is2Is the variance of a gaussian function and is,
Figure BDA0002438877440000081
in order to perform the fourier transformation, the method,
Figure BDA0002438877440000082
and (3) performing inverse Fourier transform, wherein lambda is a penalty term constant of training, ⊙ represents a matrix multiplication operation item by item, and α is a parameter of a correlation filter.
Step S103, initializing a Kalman Filter (KF).
Specifically, in the embodiment of the present application, the state transition matrix (denoted as a here) may be initialized as:
Figure BDA0002438877440000083
the observation matrix (denoted here as H) is initialized to:
Figure BDA0002438877440000084
initializing a predictive noise covariance matrix into a diagonal matrix with 4 rows and 4 columns, wherein the diagonal elements are the variances of Gaussian white noise;
initializing an observation noise covariance matrix into a diagonal matrix with 2 rows and 2 columns, wherein the diagonal elements are the variances of Gaussian white noise;
initializing a posterior error estimation covariance matrix into a unit diagonal matrix with 4 rows and 4 columns;
the state of the target (denoted here as w) is initialized to:
w＝[u0v0Δu Δv]T
wherein (u)0,v0) For the initial coordinates, (Δ u, Δ v) is the motion velocity obtained by random initialization.
When tracking a target, the specific tracking process of each frame image is similar, and a detailed description will be given below by taking any one frame (k-th frame) as an example. As shown in fig. 2, the process of tracking the target in the k-th frame image may include:
step S201, according to the position coordinate of the target in the k-1 frame image, calculating the predicted position coordinate of the target in the k frame image by using a Kalman filter.
Specifically, the predicted position coordinates of the target in the k-th frame image may be calculated according to the following equation:
w′＝Aw
(uKF,vKF)＝(w′[0],w′[1])
wherein (u)KF,vKF) I.e. the predicted position coordinates.
And S202, determining a first search area in the k frame image according to the predicted position coordinates and the detection frame in the k-1 frame image.
As shown in fig. 3, the left image is a k-1 th frame, a dashed rectangle frame represents a detection frame of the target tracked in the k-1 th frame, and then the region to be detected in the k-th frame (right image) is m times the size of the detection frame, i.e., the size of a solid frame in the image, and the coordinates of the target in the k-th frame in the image move to the upper left corner according to the motion trajectory of the target. The right image is the input image of the k frame, and the solid box on the right is the tracking area of the classical correlation filter at the k frame, which is denoted as the second search area here. It can be seen that due to the motion of the target, the target has moved to the edge of the tracking area of the classical correlation filter at the k-th frame in the image, resulting in the occurrence of "boundary effect", and the classical correlation filter may fail to track at this time; when the kalman filter is introduced to predict the motion trajectory of the target, the kalman filter may predict the coordinates (moving to the upper left corner) that may occur to the target in the image of the k-th frame by combining the motion trajectory of the target in the previously received image, so that the tracking area when the kalman filter outputs the k-th frame to the correlation filter is shown as the solid line frame on the left, and is referred to as the first search area here.
Step S203, tracking the target in the first search area by using a correlation filter, and obtaining a first maximum response value and a first position coordinate.
Specifically, the target may be tracked in the first search area using a correlation filter, resulting in a first response matrix as shown below:
Figure BDA0002438877440000091
wherein z is1For the image of the first search area, CF1Is the first response matrix;
determining a maximum response value in the first response matrix as the first maximum response value, and determining a position coordinate corresponding to the first maximum response value as the first position coordinate. Here, the first maximum response value is denoted as p1Marking the first position as
Figure BDA0002438877440000092
And S204, determining a second search area in the k frame image according to the position coordinate of the target in the k-1 frame image and the detection frame.
Step S205, tracking the target in the second search area by using a correlation filter, and obtaining a second maximum response value and a second position coordinate.
Specifically, the target may be tracked in the second search area using a correlation filter, resulting in a second response matrix as shown below:
Figure BDA0002438877440000101
wherein z is2For the image of the second search area, CF2Is the second response matrix;
and determining the maximum response value in the second response matrix as the second maximum response value, and determining the position coordinate corresponding to the second maximum response value as the second position coordinate. Here, the second maximum response value is denoted as p2Marking the second position as
Figure BDA0002438877440000102
Step S206, selecting the position coordinate of the target in the k frame image from the first position coordinate and the second position coordinate according to the first maximum response value and the second maximum response value.
Specifically, if the first maximum response value is greater than the second maximum response value, the first position coordinate is determined as the position coordinate of the target in the k-th frame image, that is:
Figure BDA0002438877440000103
wherein (u)k,vk) I.e. the position coordinates of the object in the k-th frame image.
If the first maximum response value is smaller than or equal to the second maximum response value, determining the second position coordinate as the position coordinate of the target in the k frame image, namely:
Figure BDA0002438877440000104
after the position coordinates of the target in the kth frame image are selected, the correlation filter and the Kalman filter can be respectively updated through the following processes:
in updating the correlation filter, the parameters of the correlation filter may be updated according to:
α＝(1-γ)α+γα′
wherein the content of the first and second substances,
Figure BDA0002438877440000105
and x is an image of an m-time area of the detection frame in the k frame image by taking the position coordinate of the target in the k frame image as the center, y is a Gaussian function of the m-time area of the detection frame in the k frame image by taking the position coordinate of the target in the k frame image as the center, and gamma is a weight constant for updating the parameter.
In updating the kalman filter, the state of the target may be updated according to the following equation:
w＝w′+K′(q-Hw′)
wherein, K ═ PHT(HPHT+R)-1W' is the predicted position coordinate of the target in the k-th frame image, q is the position coordinate of the target in the k-th frame image, i.e. q ═ uk,vk)TH is an observation matrix, P is an a posteriori error estimation covariance matrix, and R is an observation noise covariance matrix.
The a posteriori error estimate covariance matrix may be updated according to:
P＝P′-K′HP
wherein, P' ═ APAT+ Q, Q is the predicted noise covariance matrix.
If the tracking process is not finished, continuing to track the target in the next frame of image, namely executing the following steps: k equals to k +1, and the tracking step shown in fig. 2 is executed again until the whole tracking process is finished.
In summary, in order to ensure the effectiveness of tracking, the embodiment of the present application does not lose the tracking of the target due to too fast movement of the target or interference of other people, a kalman filter is used to predict the position coordinates of the target that may appear in the next frame, the position coordinates of the target that appears in the previous frame are considered, and the position coordinates of the target in the next frame of image are selected through a correlation filter. On the premise of ensuring high real-time performance of the target tracking algorithm, the influence of the boundary effect on the tracking algorithm is processed, the performance of the tracking algorithm is improved, and the target which possibly appears in a security and protection place is tracked in real time and efficiently.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Fig. 4 is a block diagram of an embodiment of a target tracking device according to the present application, which corresponds to the target tracking method in the foregoing embodiment.
In this embodiment, a target tracking apparatus may include:
a position coordinate prediction module 401, configured to calculate, according to a position coordinate of the target in the k-1 th frame image, a predicted position coordinate of the target in the k-1 th frame image by using a kalman filter, where k is a positive integer;
a first search area determining module 402, configured to determine a first search area in the kth frame image according to the predicted position coordinates and the detection frame in the kth-1 frame image;
a first tracking module 403, configured to track the target in the first search area using a correlation filter, so as to obtain a first maximum response value and a first position coordinate;
a second search area determining module 404, configured to determine a second search area in the kth frame image according to the position coordinate of the target in the kth-1 frame image and the detection frame;
a second tracking module 405, configured to track the target in the second search area by using a correlation filter, so as to obtain a second maximum response value and a second position coordinate;
a position coordinate selecting module 406, configured to select, according to the first maximum response value and the second maximum response value, a position coordinate of the target in the k-th frame image from the first position coordinate and the second position coordinate.
Further, the location coordinate selecting module 406 may include:
a first selecting unit 4061, configured to determine the first position coordinate as a position coordinate of the target in a k-th frame image if the first maximum response value is greater than the second maximum response value;
a second selecting unit 4062, configured to determine the second position coordinate as a position coordinate of the target in the k-th frame image if the first maximum response value is smaller than or equal to the second maximum response value.
Further, the target tracking apparatus may further include:
an object detection module 407, configured to perform object detection in the designated area, and determine an initial coordinate and an initial detection frame of the detected object;
an initialization module 408 configured to perform an initialization process, the initialization process comprising: initializing the value of k; initializing a correlation filter; the kalman filter is initialized.
Further, the initialization module 408 may include:
a correlation filter parameter initialization unit 4081 configured to initialize parameters of the correlation filter according to the following equation:
Figure BDA0002438877440000121
wherein the content of the first and second substances,
Figure BDA0002438877440000131
x is an image of an m-time area of the initial detection frame by taking the initial coordinate as a center, and m is a preset multiple parameterTaking the Gaussian function of the m-times area of the initial detection frame as a label of the training correlation filter by taking the initial coordinate as the center, wherein y is the number, sigma2Is the variance of a gaussian function and is,
Figure BDA0002438877440000132
in order to perform the fourier transformation, the method,
Figure BDA0002438877440000133
an inverse Fourier transform, wherein lambda is a penalty term constant of training, ⊙ represents a matrix multiplication operation item by item, and α is a parameter of a correlation filter;
a kalman filter initialization unit 4082, configured to initialize the state transition matrix as:
Figure BDA0002438877440000134
the observation matrix is initialized to:
Figure BDA0002438877440000135
initializing a predictive noise covariance matrix into a diagonal matrix with 4 rows and 4 columns, wherein the diagonal elements are the variances of Gaussian white noise;
initializing an observation noise covariance matrix into a diagonal matrix with 2 rows and 2 columns, wherein the diagonal elements are the variances of Gaussian white noise;
initializing a posterior error estimation covariance matrix into a unit diagonal matrix with 4 rows and 4 columns;
initializing a state of the target to:
w＝[u0v0Δu Δv]T
wherein (u)0,v0) For the initial coordinates, (Δ u, Δ v) is the motion velocity obtained by random initialization.
Further, the first tracking module 403 is specifically configured to:
tracking the target in the first search area using a correlation filter, resulting in a first response matrix as shown below:
Figure BDA0002438877440000136
wherein z is1For the image of the first search area, CF1Is the first response matrix;
determining a maximum response value in the first response matrix as the first maximum response value, and determining a position coordinate corresponding to the first maximum response value as the first position coordinate.
Further, the second tracking module 405 is specifically configured to:
tracking the target in the second search area by using a correlation filter to obtain a second response matrix as shown in the following:
Figure BDA0002438877440000141
wherein z is2For the image of the second search area, CF2Is the second response matrix;
and determining the maximum response value in the second response matrix as the second maximum response value, and determining the position coordinate corresponding to the second maximum response value as the second position coordinate.
Further, the target tracking apparatus may further include:
a correlation filter updating module 409, configured to update parameters of the correlation filter according to the following formula:
α＝(1-γ)α+γα′
wherein the content of the first and second substances,
Figure BDA0002438877440000142
taking the image of the m-times area of the detection frame in the k frame image by taking the position coordinate of the target in the k frame image as the center, taking the Gaussian function of the m-times area of the detection frame in the k frame image by taking the position coordinate of the target in the k frame image as the center, and taking the Gaussian function as the training correlationThe label of the filter, gamma is the weight constant of the parameter update;
a Kalman filter update module 410 to:
updating the state of the target according to:
w＝w′+K′(q-Hw′)
wherein, K ═ PHT(HPHT+R)-1W' is the predicted position coordinate of the target in the k frame image, q is the position coordinate of the target in the k frame image, H is an observation matrix, P is an a posteriori error estimation covariance matrix, and R is an observation noise covariance matrix;
updating the a posteriori error estimate covariance matrix according to:
P＝P′-K′HP
wherein, P' ═ APAT+ Q, Q is the predicted noise covariance matrix.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses, modules and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Fig. 5 shows a schematic block diagram of a robot provided in an embodiment of the present application, and only a part related to the embodiment of the present application is shown for convenience of explanation.
As shown in fig. 5, the robot 5 of this embodiment includes: a processor 50, a memory 51 and a computer program 52 stored in said memory 51 and executable on said processor 50. The processor 50, when executing the computer program 52, implements the steps in the above-mentioned embodiments of the object tracking method, such as the steps S201 to S206 shown in fig. 2. Alternatively, the processor 50, when executing the computer program 52, implements the functions of the modules/units in the above device embodiments, such as the functions of the modules 401 to 406 shown in fig. 4.
Illustratively, the computer program 52 may be partitioned into one or more modules/units, which are stored in the memory 51 and executed by the processor 50 to accomplish the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of the computer program 52 in the robot 5.
Those skilled in the art will appreciate that fig. 5 is merely an example of a robot 5 and does not constitute a limitation of the robot 5 and may include more or fewer components than shown, or some components in combination, or different components, for example, the robot 5 may also include input and output devices, network access devices, buses, etc.
The Processor 50 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 51 may be an internal storage unit of the robot 5, such as a hard disk or a memory of the robot 5. The memory 51 may also be an external storage device of the robot 5, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, provided on the robot 5. Further, the memory 51 may also include both an internal storage unit and an external storage device of the robot 5. The memory 51 is used for storing the computer program and other programs and data required by the robot 5. The memory 51 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/robot and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/robot are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow in the method of the embodiments described above can be realized by a computer program, which can be stored in a computer-readable storage medium and can realize the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, etc. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims (10)

1. A target tracking method, comprising:
according to the position coordinate of the target in the k-1 frame image, calculating the predicted position coordinate of the target in the k frame image by using a Kalman filter, wherein k is a positive integer;
determining a first search area in the k frame image according to the predicted position coordinates and a detection frame in the k-1 frame image;
tracking the target in the first search area by using a correlation filter to obtain a first maximum response value and a first position coordinate;
determining a second search area in the kth frame image according to the position coordinate of the target in the kth-1 frame image and the detection frame;
tracking the target in the second search area by using a correlation filter to obtain a second maximum response value and a second position coordinate;
and selecting the position coordinate of the target in the k frame image from the first position coordinate and the second position coordinate according to the first maximum response value and the second maximum response value.
2. The method for tracking the target according to claim 1, wherein the selecting the position coordinate of the target in the k-th frame image from the first position coordinate and the second position coordinate according to the first maximum response value and the second maximum response value comprises:
if the first maximum response value is larger than the second maximum response value, determining the first position coordinate as the position coordinate of the target in the k frame image;
and if the first maximum response value is smaller than or equal to the second maximum response value, determining the second position coordinate as the position coordinate of the target in the k frame image.
3. The target tracking method according to claim 1, before calculating the predicted position coordinates of the target in the k-th frame image using the kalman filter, further comprising:
carrying out target detection in the designated area, and determining an initial coordinate and an initial detection frame of the detected target;
performing an initialization process, the initialization process comprising: initializing the value of k; initializing a correlation filter; the kalman filter is initialized.
4. The method of claim 3, wherein initializing the correlation filter comprises:
parameters of the correlation filter are initialized according to:
Figure FDA0002438877430000021
wherein the content of the first and second substances,
Figure FDA0002438877430000022
taking an image of an m-time area of the initial detection frame by taking the initial coordinate as a center, taking m as a preset multiple parameter, taking y as a m-time cell of the initial detection frame by taking the initial coordinate as a centerGaussian function of the domain, as a label for training the correlation filter, σ2Is the variance of a gaussian function and is,
Figure FDA0002438877430000023
in order to perform the fourier transformation, the method,
Figure FDA0002438877430000024
an inverse Fourier transform, wherein lambda is a penalty term constant of training, ⊙ represents a matrix multiplication operation item by item, and α is a parameter of a correlation filter;
the initializing the kalman filter includes:
initializing the state transition matrix to:
Figure FDA0002438877430000025
the observation matrix is initialized to:
Figure FDA0002438877430000026
initializing a predictive noise covariance matrix into a diagonal matrix with 4 rows and 4 columns, wherein the diagonal elements are the variances of Gaussian white noise;
initializing an observation noise covariance matrix into a diagonal matrix with 2 rows and 2 columns, wherein the diagonal elements are the variances of Gaussian white noise;
initializing a posterior error estimation covariance matrix into a unit diagonal matrix with 4 rows and 4 columns;
initializing a state of the target to:
w＝[u0v0Δu Δv]T
wherein (u)0,v0) For the initial coordinates, (Δ u, Δ v) is the motion velocity obtained by random initialization.
5. The method of tracking an object according to claim 1, wherein the tracking the object in the first search area using a correlation filter to obtain a first maximum response value and a first location coordinate comprises:
tracking the target in the first search area using a correlation filter, resulting in a first response matrix as shown below:
Figure FDA0002438877430000031
wherein z is1For the image of the first search area, CF1Is the first response matrix;
determining a maximum response value in the first response matrix as the first maximum response value, and determining a position coordinate corresponding to the first maximum response value as the first position coordinate.
6. The method of tracking an object according to claim 1, wherein the tracking the object in the second search area using a correlation filter to obtain a second maximum response value and a second location coordinate comprises:
tracking the target in the second search area by using a correlation filter to obtain a second response matrix as shown in the following:
Figure FDA0002438877430000032
wherein z is2For the image of the second search area, CF2Is the second response matrix;
and determining the maximum response value in the second response matrix as the second maximum response value, and determining the position coordinate corresponding to the second maximum response value as the second position coordinate.
7. The target tracking method according to any one of claims 1 to 6, further comprising, after selecting the position coordinates of the target in the k-th frame image:
updating parameters of the correlation filter according to:
α＝(1-γ)α+γα′
wherein the content of the first and second substances,
Figure FDA0002438877430000033
taking the image of the m-times area of the detection frame in the k frame image by taking the position coordinate of the target in the k frame image as the center, taking the Gaussian function as a label for training a related filter, and taking gamma as a weight constant for updating parameters;
updating the state of the target according to:
w＝w′+K′(q-Hw′)
wherein, K ═ PHT(HPHT+R)-1W' is the predicted position coordinate of the target in the k frame image, q is the position coordinate of the target in the k frame image, H is an observation matrix, P is an a posteriori error estimation covariance matrix, and R is an observation noise covariance matrix;
updating the a posteriori error estimate covariance matrix according to:
P＝P′-K′HP
wherein, P' ═ APAT+ Q, Q is the predicted noise covariance matrix.
8. An object tracking device, comprising:
the position coordinate prediction module is used for calculating the predicted position coordinate of the target in the kth frame image by using a Kalman filter according to the position coordinate of the target in the kth-1 frame image, wherein k is a positive integer;
the first search area determining module is used for determining a first search area in the k frame image according to the predicted position coordinates and the detection frame in the k-1 frame image;
the first tracking module is used for tracking the target in the first search area by using a correlation filter to obtain a first maximum response value and a first position coordinate;
the second search area determining module is used for determining a second search area in the kth frame image according to the position coordinate of the target in the kth-1 frame image and the detection frame;
the second tracking module is used for tracking the target in the second search area by using a correlation filter to obtain a second maximum response value and a second position coordinate;
and the position coordinate selection module is used for selecting the position coordinate of the target in the k frame image from the first position coordinate and the second position coordinate according to the first maximum response value and the second maximum response value.
9. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the object tracking method according to any one of claims 1 to 7.
10. A robot comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, carries out the steps of the object tracking method according to any of claims 1 to 7.
CN202010259829.9A 2020-04-03 2020-04-03 Target tracking method, device, computer readable storage medium and robot Active CN111563919B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010259829.9A CN111563919B (en) 2020-04-03 2020-04-03 Target tracking method, device, computer readable storage medium and robot

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010259829.9A CN111563919B (en) 2020-04-03 2020-04-03 Target tracking method, device, computer readable storage medium and robot

Publications (2)

Publication Number Publication Date
CN111563919A true CN111563919A (en) 2020-08-21
CN111563919B CN111563919B (en) 2023-12-29

Family

ID=72072950

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010259829.9A Active CN111563919B (en) 2020-04-03 2020-04-03 Target tracking method, device, computer readable storage medium and robot

Country Status (1)

Country Link
CN (1) CN111563919B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112507906A (en) * 2020-12-14 2021-03-16 北京澎思科技有限公司 Target tracking method, device and computer readable storage medium
CN112651377A (en) * 2021-01-05 2021-04-13 河北建筑工程学院 Ice and snow movement accident detection method and device and terminal equipment
CN112750146A (en) * 2020-12-31 2021-05-04 浙江大华技术股份有限公司 Target object tracking method and device, storage medium and electronic equipment
CN113139416A (en) * 2020-11-17 2021-07-20 西安天伟电子系统工程有限公司 Object association method, computer device, and storage medium
CN113313739A (en) * 2021-06-23 2021-08-27 中国农业银行股份有限公司 Target tracking method, device and storage medium
WO2022227761A1 (en) * 2021-04-28 2022-11-03 上海商汤智能科技有限公司 Target tracking method and apparatus, electronic device, and storage medium
CN115690333A (en) * 2022-12-30 2023-02-03 思看科技(杭州)股份有限公司 Three-dimensional scanning method and system

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104992451A (en) * 2015-06-25 2015-10-21 河海大学 Improved target tracking method
CN105469430A (en) * 2015-12-10 2016-04-06 中国石油大学(华东) Anti-shielding tracking method of small target in large-scale scene
CN108010067A (en) * 2017-12-25 2018-05-08 北京航空航天大学 A kind of visual target tracking method based on combination determination strategy
CN108198209A (en) * 2017-12-22 2018-06-22 天津理工大学 It is blocking and dimensional variation pedestrian tracking algorithm
CN110233667A (en) * 2019-06-05 2019-09-13 华南理工大学 VLC dynamic positioning method and system based on average drifting and Unscented kalman filtering
CN110276785A (en) * 2019-06-24 2019-09-24 电子科技大学 One kind is anti-to block infrared object tracking method
CN110706252A (en) * 2019-09-09 2020-01-17 西安理工大学 Robot nuclear correlation filtering tracking algorithm under guidance of motion model
CN110796687A (en) * 2019-10-30 2020-02-14 电子科技大学 Sky background infrared imaging multi-target tracking method

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104992451A (en) * 2015-06-25 2015-10-21 河海大学 Improved target tracking method
CN105469430A (en) * 2015-12-10 2016-04-06 中国石油大学(华东) Anti-shielding tracking method of small target in large-scale scene
CN108198209A (en) * 2017-12-22 2018-06-22 天津理工大学 It is blocking and dimensional variation pedestrian tracking algorithm
CN108010067A (en) * 2017-12-25 2018-05-08 北京航空航天大学 A kind of visual target tracking method based on combination determination strategy
CN110233667A (en) * 2019-06-05 2019-09-13 华南理工大学 VLC dynamic positioning method and system based on average drifting and Unscented kalman filtering
CN110276785A (en) * 2019-06-24 2019-09-24 电子科技大学 One kind is anti-to block infrared object tracking method
CN110706252A (en) * 2019-09-09 2020-01-17 西安理工大学 Robot nuclear correlation filtering tracking algorithm under guidance of motion model
CN110796687A (en) * 2019-10-30 2020-02-14 电子科技大学 Sky background infrared imaging multi-target tracking method

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113139416A (en) * 2020-11-17 2021-07-20 西安天伟电子系统工程有限公司 Object association method, computer device, and storage medium
CN112507906A (en) * 2020-12-14 2021-03-16 北京澎思科技有限公司 Target tracking method, device and computer readable storage medium
CN112750146A (en) * 2020-12-31 2021-05-04 浙江大华技术股份有限公司 Target object tracking method and device, storage medium and electronic equipment
CN112750146B (en) * 2020-12-31 2023-09-12 浙江大华技术股份有限公司 Target object tracking method and device, storage medium and electronic equipment
CN112651377A (en) * 2021-01-05 2021-04-13 河北建筑工程学院 Ice and snow movement accident detection method and device and terminal equipment
WO2022227761A1 (en) * 2021-04-28 2022-11-03 上海商汤智能科技有限公司 Target tracking method and apparatus, electronic device, and storage medium
CN113313739A (en) * 2021-06-23 2021-08-27 中国农业银行股份有限公司 Target tracking method, device and storage medium
CN115690333A (en) * 2022-12-30 2023-02-03 思看科技(杭州)股份有限公司 Three-dimensional scanning method and system

Also Published As

Publication number Publication date
CN111563919B (en) 2023-12-29

Similar Documents

Publication Publication Date Title
CN111563919B (en) Target tracking method, device, computer readable storage medium and robot
US10558891B2 (en) Systems and methods for object tracking
Han et al. Incremental density approximation and kernel-based bayesian filtering for object tracking
CN110766724B (en) Target tracking network training and tracking method and device, electronic equipment and medium
EP1975879B1 (en) Computer implemented method for tracking object in sequence of frames of video
CN111696132A (en) Target tracking method and device, computer readable storage medium and robot
CN110349190B (en) Adaptive learning target tracking method, device, equipment and readable storage medium
US20120134586A1 (en) Device with datastream pipeline architecture for recognizing and locating objects in an image by detection window scanning
Frost et al. Detection and tracking of moving objects in a maritime environment using level set with shape priors
CN111612822B (en) Object tracking method, device, computer equipment and storage medium
CN110349188B (en) Multi-target tracking method, device and storage medium based on TSK fuzzy model
WO2009082700A1 (en) Online articulate object tracking with appearance and shape
CN112036381B (en) Visual tracking method, video monitoring method and terminal equipment
US20190043168A1 (en) An image processing device, an image processing method, and computer-readable recording medium
CN111126249A (en) Pedestrian re-identification method and device combining big data and Bayes
CN111223128A (en) Target tracking method, device, equipment and storage medium
CN115239760B (en) Target tracking method, system, equipment and storage medium
CN109410246B (en) Visual tracking method and device based on correlation filtering
CN110766725A (en) Template image updating method and device, target tracking method and device, electronic equipment and medium
CN113887699A (en) Knowledge distillation method, electronic device and storage medium
CN112907750A (en) Indoor scene layout estimation method and system based on convolutional neural network
CN112991394A (en) KCF target tracking method based on cubic spline interpolation and Markov chain
CN110147768B (en) Target tracking method and device
CN111768427A (en) Multi-moving-target tracking method and device and storage medium
CN116433722A (en) Target tracking method, electronic device, storage medium, and program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant