CN112001948A - Target tracking processing method and device - Google Patents

Target tracking processing method and device Download PDF

Info

Publication number
CN112001948A
CN112001948A CN202010753234.9A CN202010753234A CN112001948A CN 112001948 A CN112001948 A CN 112001948A CN 202010753234 A CN202010753234 A CN 202010753234A CN 112001948 A CN112001948 A CN 112001948A
Authority
CN
China
Prior art keywords
target
video frame
motion state
next video
state
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010753234.9A
Other languages
Chinese (zh)
Other versions
CN112001948B (en
Inventor
覃涛杰
韩建强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Dahua Technology Co Ltd
Original Assignee
Zhejiang Dahua Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Dahua Technology Co Ltd filed Critical Zhejiang Dahua Technology Co Ltd
Priority to CN202010753234.9A priority Critical patent/CN112001948B/en
Publication of CN112001948A publication Critical patent/CN112001948A/en
Application granted granted Critical
Publication of CN112001948B publication Critical patent/CN112001948B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/207—Analysis of motion for motion estimation over a hierarchy of resolutions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

The invention provides a target tracking processing method and a device, wherein the method comprises the following steps: carrying out target detection on a current video frame of video data to obtain a detection result of a target, wherein the detection result comprises target characteristics and a current motion state of the target; matching the characteristic information of the target with the predetermined characteristic information of the target to be tracked; under the condition of successful matching, determining the estimated motion state of the target in the next video frame according to the current motion state of the target; the target equipment for tracking the target in the next video frame is determined according to the estimated motion state, so that the problem that the tracking range is limited because the target is tracked based on a single camera in the related technology, the target searching is that the searching range is expanded from the current position according to a certain proportion, and the time is consumed when the corresponding target moves at a higher speed, is solved, the same target can be tracked among a plurality of camera equipment, and the target tracking range is expanded.

Description

Target tracking processing method and device
Technical Field
The invention relates to the field of image processing, in particular to a target tracking processing method and device.
Background
In the monitoring field, a target is tracked, and the motion state of the target is mostly tracked through a single camera. The prior art can work on the following general principles: firstly, detecting a tracking target in each frame, then determining whether the targets in each frame are the same through a specific technology, and finally determining the track of the target. The target detection technology can be RCNN, Yolo and the like, and the target association mostly determines whether the tracked targets in different frames are the same through the feature similarity of the targets. These techniques have some drawbacks, such as high complexity, long time consumption, poor generalization ability, etc., and cannot meet various requirements in practical situations.
The related technology provides a target tracking method, which utilizes a neural network to detect the target characteristics in a target frame and carry out similarity comparison with the given target characteristics, and the target is considered to be successfully tracked if the similarity is higher than a certain threshold value. When the target is searched, the searching range is gradually enlarged according to a certain proportion to search the target. The target tracking of a single-machine camera is mainly used, but the target searching method has the defects that the searching range is enlarged from the current position according to a certain proportion, the moving speed of the corresponding target is high, the searching is time-consuming, and the complexity of equipment is increased to a certain extent.
Aiming at the problem that in the related art, a target is tracked based on a single camera, the target searching is to enlarge the searching range from the current position according to a certain proportion, and the searching is more time-consuming when the corresponding target moves at a higher speed, so that the tracking range is limited, a solution is not provided.
Disclosure of Invention
The embodiment of the invention provides a target tracking processing method and a target tracking processing device, which are used for at least solving the problem that in the related technology, a target is tracked based on a single camera, the target searching is that the searching range is enlarged from the current position according to a certain proportion, and the searching is more time-consuming when the corresponding target moves at a higher speed, so that the tracking range is limited.
According to an embodiment of the present invention, there is provided a target tracking processing method including:
performing target detection on a current video frame of video data to obtain a detection result of a target, wherein the detection result comprises a target characteristic and a current motion state of the target;
matching the characteristic information of the target with the predetermined characteristic information of the target to be tracked;
under the condition of successful matching, determining the estimated motion state of the target in the next video frame according to the current motion state of the target;
and determining the target equipment tracking the target in the next video frame according to the estimated motion state.
Optionally, determining the estimated motion state of the target in the next video frame according to the current motion state of the target includes:
based on a Kalman motion estimation model, determining a state control vector of the target in the next video frame according to the position information of the target in a historical data frame;
and determining the estimated motion state of the target in the next video frame according to the state control vector and the current motion state.
Optionally, determining the estimated motion state of the target in the next video frame according to the state control vector and the current motion state includes:
correcting the current motion state to obtain a current corrected motion state;
determining an estimated motion state of the target in the next video frame according to the state control vector and the current modified motion state by:
Figure BDA0002610704590000021
wherein,
Figure BDA0002610704590000022
for said estimated motion state, FnState transition matrix, x, for the object in the next video framen-1For the current correction motion state, unFor the state control vector, BnControlling a matrix for the state of the target at the next video frame.
Optionally, the method further comprises:
determining a covariance matrix estimate of a state of the target in the next video frame by:
Figure BDA0002610704590000031
wherein,
Figure BDA0002610704590000032
covariance matrix estimation, P, for the state of the target in the next video framen-1Covariance matrix estimation, Q, for the state of the target after modification in the current video framenA covariance matrix of system noise in the next video frame;
and correcting the estimated motion state according to the covariance matrix estimation of the state of the target in the next video frame to obtain the corrected motion state of the target in the next video frame.
Optionally, the modifying the estimated motion state according to the covariance matrix estimation of the state of the target in the next video frame, and obtaining the modified motion state of the target in the next video frame includes:
acquiring the actual motion state of the target in the next video frame;
and correcting the estimated motion state according to the actual motion state of the target in the next video frame and the covariance matrix estimation of the state of the target in the next video frame to obtain the corrected motion state of the target in the next video frame.
Optionally, the method further comprises:
correcting the estimated motion state according to the covariance matrix estimation of the actual motion state of the target in the next video frame and the state of the target in the next video frame in the following way, so as to obtain the corrected motion state of the target in the next video frame:
Figure BDA0002610704590000033
Figure BDA0002610704590000034
Figure BDA0002610704590000035
Figure BDA0002610704590000036
wherein x isnFor the modified motion state of the object in the next video frame, znFor the state of motion of the object observed in the next video frame, SnMeasuring a cosine covariance matrix, R, for the target in the next video framenA covariance matrix of measured noise for the target in the next video frame; knFor the Carlman gain, H, of the object in the next video framenA mapping matrix of the motion state of the target to the device observations.
Optionally, the method further comprises:
correcting the covariance matrix estimate of the state of the target in the next video frame by the following method to obtain the covariance matrix estimate of the state of the target after correction in the next video frame:
Figure BDA0002610704590000041
wherein, PnCovariance matrix estimation for the modified state of the target in the next video frame.
Optionally, matching the feature information of the target with the predetermined feature information of the target to be tracked includes:
determining the similarity between the target characteristics and the predetermined characteristic information of the target to be tracked;
determining that the matching is successful under the condition that the similarity is greater than or equal to a preset threshold value;
and determining that the matching fails under the condition that the similarity is smaller than the preset threshold.
Optionally, determining, according to the estimated motion state, a target device that tracks the target in the next video frame includes:
determining a target monitoring area corresponding to the estimated motion state;
and determining the target equipment corresponding to the target monitoring area according to the corresponding relation between the monitoring area and the equipment.
According to another embodiment of the present invention, there is also provided a target tracking processing apparatus including:
the target detection module is used for carrying out target detection on a current video frame of video data to obtain a target detection result, wherein the detection result comprises a target characteristic and a current motion state of the target;
the matching module is used for matching the characteristic information of the target with the predetermined characteristic information of the target to be tracked;
the first determining module is used for determining the estimated motion state of the target in the next video frame according to the current motion state of the target under the condition of successful matching;
and the second determining module is used for determining target equipment for tracking the target in the next video frame according to the estimated motion state.
Optionally, the first determining module includes:
the first determining submodule is used for determining a state control vector of the target in the next video frame according to the position information of the target in a historical data frame based on a Kalman motion estimation model;
and the second determining submodule is used for determining the estimated motion state of the target in the next video frame according to the state control vector and the current motion state.
Optionally, the second determining sub-module includes:
the correction unit is used for correcting the current motion state to obtain a current corrected motion state;
a determining unit, configured to determine an estimated motion state of the target in the next video frame according to the state control vector and the current modified motion state by:
Figure BDA0002610704590000051
wherein,
Figure BDA0002610704590000052
for said estimated motion state, FnState transition matrix, x, for the object in the next video framen-1For the current correction motion state, unFor the state control vector, BnControlling a matrix for the state of the target at the next video frame.
Optionally, the apparatus further comprises:
a third determining module for determining a covariance matrix estimate of a state of the target in the next video frame by:
Figure BDA0002610704590000053
wherein,
Figure BDA0002610704590000054
covariance matrix estimation, P, for the state of the target in the next video framen-1Covariance matrix estimation, Q, for the state of the target after modification in the current video framenA covariance matrix of system noise in the next video frame;
and the correction module is used for correcting the estimated motion state according to the covariance matrix estimation of the state of the target in the next video frame to obtain the corrected motion state of the target in the next video frame.
Optionally, the correction module includes:
the acquisition submodule is used for acquiring the actual motion state of the target in the next video frame;
and the first correction submodule is used for correcting the pre-estimated motion state according to the actual motion state of the target in the next video frame and the covariance matrix estimation of the state of the target in the next video frame, so as to obtain the corrected motion state of the target in the next video frame.
Optionally, the first modification sub-module is further configured to modify the estimated motion state according to a covariance matrix estimate of an actual motion state of the target in the next video frame and a state of the target in the next video frame, so as to obtain a modified motion state of the target in the next video frame:
Figure BDA0002610704590000061
Figure BDA0002610704590000062
Figure BDA0002610704590000063
Figure BDA0002610704590000064
wherein x isnFor the modified motion state of the object in the next video frame, znFor the state of motion of the object observed in the next video frame, SnMeasuring a cosine covariance matrix, R, for the target in the next video framenA covariance matrix of measured noise for the target in the next video frame; knIn the next video frame for the targetMiddle Karman gain, HnA mapping matrix of the motion state of the target to the device observations.
Optionally, the apparatus further comprises:
a second modification submodule, configured to modify the covariance matrix estimate of the state of the target in the next video frame in the following manner, to obtain a covariance matrix estimate of the state of the target after modification in the next video frame:
Figure BDA0002610704590000065
wherein, PnCovariance matrix estimation for the modified state of the target in the next video frame.
Optionally, the matching module comprises:
the third determining submodule is used for determining the similarity between the target characteristics and the predetermined characteristic information of the target to be tracked;
the fourth determining submodule is used for determining that the matching is successful under the condition that the similarity is greater than or equal to a preset threshold value;
and the fifth determining submodule is used for determining that the matching fails under the condition that the similarity is smaller than the preset threshold.
Optionally, the second determining module is further configured to
Determining a target monitoring area corresponding to the estimated motion state;
and determining the target equipment corresponding to the target monitoring area according to the corresponding relation between the monitoring area and the equipment.
According to a further embodiment of the present invention, a computer-readable storage medium is also provided, in which a computer program is stored, wherein the computer program is configured to perform the steps of any of the above-described method embodiments when executed.
According to yet another embodiment of the present invention, there is also provided an electronic device, including a memory in which a computer program is stored and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.
By the invention, when the collected characteristic information of the target is matched with the characteristic information of the target to be tracked in the plurality of cameras, determining the estimated motion state of the target in the next video frame according to the current motion state of the target, the target equipment for tracking the target is determined according to the estimated motion state, the target equipment tracks the target in the video frame of the next moment, the problem that the target is tracked based on a single camera in the related technology can be solved, the target searching is to enlarge the searching range from the current position according to a certain proportion, the searching is more time-consuming when the moving speed of the corresponding target is higher, resulting in a problem that the tracking range is limited, achieving that when the object moves between the respective image pickup apparatuses, the same target can be tracked among a plurality of image pickup apparatuses, and the target tracking range is expanded.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention to a proper form. In the drawings:
fig. 1 is a block diagram of a hardware configuration of a mobile terminal of a target tracking processing method according to an embodiment of the present invention;
FIG. 2 is a flow diagram of a target tracking processing method according to an embodiment of the invention;
FIG. 3 is a flow diagram of target tracking during movement of a tracked target in accordance with an embodiment of the present invention;
FIG. 4 is a flow chart of a process of execution of a target tracking algorithm according to an embodiment of the present invention;
fig. 5 is a block diagram of a target tracking processing apparatus according to an embodiment of the present invention.
Detailed Description
The invention will be described in detail hereinafter with reference to the accompanying drawings in conjunction with embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
Example 1
The method provided by the first embodiment of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device. Taking a mobile terminal as an example, fig. 1 is a hardware structure block diagram of the mobile terminal of the target tracking processing method according to the embodiment of the present invention, as shown in fig. 1, the mobile terminal may include one or more processors 102 (only one is shown in fig. 1) (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 104 for storing data, and optionally, the mobile terminal may further include a transmission device 106 for a communication function and an input/output device 108. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration, and is not intended to limit the structure of the mobile terminal. For example, the mobile terminal may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The memory 104 may be used to store a computer program, for example, a software program and a module of an application software, such as a computer program corresponding to the target tracking processing method in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, so as to implement the method described above. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the mobile terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a Network adapter (NIC), which can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
In this embodiment, a target tracking processing method operating in the mobile terminal or the network architecture is provided, and fig. 2 is a flowchart of the target tracking processing method according to the embodiment of the present invention, as shown in fig. 2, the flowchart includes the following steps:
step S202, carrying out target detection on a current video frame of video data to obtain a detection result of a target, wherein the detection result comprises a target characteristic and a current motion state of the target;
the motion state in the embodiment of the present invention at least includes the speed and the position, i.e., the current motion state also at least includes the speed and the position of the target.
Step S204, matching the characteristic information of the target with the predetermined characteristic information of the target to be tracked;
in an embodiment of the present invention, the step S204 may specifically include: determining the similarity between the target characteristics and the predetermined characteristic information of the target to be tracked; determining that the matching is successful under the condition that the similarity is greater than or equal to a preset threshold value; and determining that the matching fails under the condition that the similarity is smaller than the preset threshold.
Step S206, under the condition of successful matching, determining the estimated motion state of the target in the next video frame according to the current motion state of the target;
and S208, determining the target equipment tracking the target in the next video frame according to the estimated motion state.
In an embodiment of the present invention, the step S208 may specifically include: determining a target monitoring area corresponding to the estimated motion state; and determining the target equipment corresponding to the target monitoring area according to the corresponding relation between the monitoring area and the equipment.
Through the above steps S202 to S208, when the collected feature information of the target matches with the feature information of the target to be tracked in the plurality of cameras, determining the estimated motion state of the target in the next video frame according to the current motion state of the target, the target equipment for tracking the target is determined according to the estimated motion state, the target equipment tracks the target in the video frame of the next moment, the problem that the target is tracked based on a single camera in the related technology can be solved, the target searching is to enlarge the searching range from the current position according to a certain proportion, the searching is more time-consuming when the moving speed of the corresponding target is higher, the problem that the tracking range is limited is caused, when the target moves among the camera devices, the same target can be tracked among a plurality of image pickup devices, and the target tracking range is expanded.
In an embodiment of the present invention, the step S206 may specifically include: determining a state control vector of the target in the next video frame according to the position information of the target in the historical data frame based on a Kalman motion estimation model; determining an estimated motion state of the target in the next video frame according to the state control vector and the current motion state, and further correcting the current motion state to obtain a current corrected motion state; determining a pre-estimated motion state of the object in the next video frame according to the state control vector and the current modified motion state by:
Figure BDA0002610704590000101
wherein,
Figure BDA0002610704590000102
for said estimated motion state, FnState transition matrix, x, for the target in the next video framen-1 is the current repairPositive motion state, unFor the state control vector, BnControlling a matrix for the state of the target at the next video frame.
In an alternative embodiment, the covariance matrix estimate of the state of the target in the next video frame is determined by:
Figure BDA0002610704590000103
wherein,
Figure BDA0002610704590000104
covariance matrix estimation, P, for the state of the target in the next video framen-1Covariance matrix estimation, Q, for the state of the target after modification in the current video framenA covariance matrix of system noise in the next video frame; correcting the estimated motion state according to covariance matrix estimation of the state of the target in the next video frame to obtain a corrected motion state of the target in the next video frame, and further acquiring an actual motion state of the target in the next video frame; correcting the estimated motion state according to the covariance matrix estimation of the actual motion state of the target in the next video frame and the state of the target in the next video frame to obtain a corrected motion state of the target in the next video frame, and further obtaining the corrected motion state of the target in the next video frame by the following method:
Figure BDA0002610704590000111
Figure BDA0002610704590000112
Figure BDA0002610704590000113
Figure BDA0002610704590000114
wherein x isnFor the modified motion state of the object in the next video frame, znFor the state of motion of the object observed in the next video frame, SnMeasuring a cosine covariance matrix, R, for the target in the next video framenA covariance matrix of measured noise in the next video frame for the target; knFor the Kalman gain, H, of the target in the next video framenA mapping matrix of the motion state of the object to device observations.
In another optional embodiment, the covariance matrix estimate of the state of the target in the next video frame is modified by:
Figure BDA0002610704590000115
wherein, PnCovariance matrix estimation for the modified state of the target in the next video frame.
In the embodiment of the invention, each camera device is connected to the same central controller, and the storage device at the controller end stores videos according to the tracked targets. Fig. 3 is a flowchart of target tracking during movement of a tracked target according to an embodiment of the present invention, as shown in fig. 3, including:
step S301, a source device tracks a target;
step S302, judging whether to switch the video equipment of the tracking target, if so, executing step S303, otherwise, returning to step S301;
step S303, the source equipment sends the characteristic information of the target to the target video equipment;
in step S304, the target video device acquires information of the target and tracks the target.
When the target is captured in a video device (source device), tracking according to a certain tracking algorithm, and switching the target if the tracked target leaves the current monitoring view of the camera and enters the monitoring view of other related devices (target video devices); otherwise, ending the target tracking. After the target is switched, the source device sends the related information (such as position information) of the tracked target to the target device through the central controller, and then the target device tracks the tracked target. Thus, the target tracking video stored in the central controller will be switched from the source device to the target device.
FIG. 4 is a flow chart of the execution process of the target tracking algorithm according to the embodiment of the invention, as shown in FIG. 4, including:
step S401, acquiring a current video frame acquired by video equipment;
step S402, carrying out target detection on the current video frame to obtain a target to be tracked;
s403, estimating the motion state of the target in the next video frame by adopting a Kalman motion model to obtain an estimated motion state;
and S404, determining target video equipment according to the target according to the estimated motion state, and sending the target characteristics of the target to the target video equipment.
When a target is tracked in given equipment, a frame of picture to be processed is obtained from camera equipment, then target detection is carried out on the picture, the tracked target is matched according to a target matching algorithm, and then a Kalman motion model is adopted to predict the position of the next frame of target according to history and the current position of the target. Finally, the target is tracked jointly based on its future position of occurrence and the monitored field of view of the associated camera device.
The target tracking algorithm in the embodiment of the invention mainly comprises the following steps:
and (4) inputting videos, wherein the videos need to be input according to a certain frame interval.
And target detection, wherein the target detection adopts a neural network to carry out target detection, such as YoloV 3. The input of the network is a given area in the picture, and the size of the area needs to be specifically determined by the target detection network; the center of the region is determined by a kalman motion estimation model.
And (3) depth feature extraction, wherein after the objects are detected, feature extraction is carried out on each object through a depth neural network, and the extracted features can avoid the target from being influenced by factors such as light, visual angle, image noise interference and the like.
And (3) matching the target depth characteristics to be tracked with the characteristics extracted from the object in the step (3), wherein the similarity can adopt a cosine similarity matrix, an Euclidean distance matrix and the like. If multiple tracking targets exist in the same equipment, the system adopts a Hungarian algorithm to obtain the best matching.
And performing Kalman motion estimation, namely predicting the position of the next frame of the target according to the current position and the historical position of the target after the target is detected through target matching.
Updating parameters, namely updating Kalman parameters according to the current position of a target through Kalman estimation; if the tracking target moves from the current equipment monitoring area to the video monitoring area of another equipment, the position parameters and the Kalman parameters of the target need to be updated to the other equipment through affine transformation.
The Kalman motion estimation consists of two stages, namely a prediction stage and a parameter correction stage, which can be respectively expressed as follows:
a prediction stage:
Figure BDA0002610704590000131
Figure BDA0002610704590000132
wherein,
Figure BDA0002610704590000133
representing an estimated matrix of the motion state (including position, velocity, etc.) of the target at time n,
Figure BDA0002610704590000134
representing a covariance matrix estimate for the target at time n; fnA state transition matrix representing the image pickup apparatus at time n; x is the number ofn-1Representing the motion state of the target at time n-1; b isnA state control matrix, u, representing the target at time nnA state control vector representing the target at time n; pn-1Represents the actual covariance matrix of the target at time n-1; qnA covariance matrix representing the system noise at time n.
And a correction stage:
Figure BDA0002610704590000141
Figure BDA0002610704590000142
Figure BDA0002610704590000143
Figure BDA0002610704590000144
Figure BDA0002610704590000145
wherein z isnRepresenting the motion state of the target actually observed by the video equipment at the moment n; hnA mapping matrix representing a target motion state to a device observation; snRepresenting a target measured cosine covariance matrix at time n; rnA covariance matrix representing the measurement noise at time n; knRepresenting the kalman gain of the target at time n.
The video device uses Kalman motion estimation iterative process comprising the following steps:
the video equipment adopts a model estimation of a Kalman prediction stage to obtain a motion state matrix and a covariance matrix at the moment n according to the current position of the target and the historical position of the target;
and calculating to obtain the Gaussian distribution of the target operation motion state at the next moment:
Figure BDA0002610704590000146
further, the expected motion state of the target is obtained according to the result;
observing the motion state z of the target at the moment nnThen updating the motion state x of the target according to the formula of the correction phase of Kalman motion estimationnSum covariance matrix Pn。
If the target moves linearly, the moving state xnConsisting of position and velocity, since position can be represented by a point, xnCan be expressed as a 2 x 1 dimensional vector, i.e. xn＝[pn,vn]T. Since the interval of processing frames by the video equipment is very short, in this period, the target can be regarded as uniform acceleration, uniform deceleration or uniform motion, and there are:
Figure BDA0002610704590000147
the motion state matrix at time 0 may refer to the actual situation, such as setting as:
Figure BDA0002610704590000151
the covariance matrix for the time 0 state may be set to
Figure BDA0002610704590000152
QnAnd RnA standard covariance distribution of 2 × 2 can be assumed; hnThe result obtained by the video apparatus, i.e., the actually required result, can be considered to be a 2 × 2 identity matrix. The iterative process of kalman motion estimation is performed, with the above assumptions. It is noted that u isnThe method can not be directly obtained through observation, and can be obtained through approximate calculation according to the position change of the target in the historical frame.
Expanding the motion estimation to a two-dimensional plane to estimate the motion of the target, and obtaining a motion state xnIs a 4 x 1 dimensional vector, i.e. xn＝[px,n,py,n,vx,n,vy,n]T。
According to the embodiment of the invention, deep reinforcement learning is directly applied to target tracking, and the characteristic of long time consumption and low speed of many traditional algorithms is abandoned; in the invention, a target track adopts a Kalman motion model in a camera to track a target so as to improve the speed; the characteristic extraction adopts a deep learning method, so that the speed is high; when the target moves among the camera devices, the same target can be tracked among the camera devices by adopting the radiation change, so that the target tracking range is expanded, and a plurality of targets can be tracked in the camera devices simultaneously, so that the target tracking has universality.
Example 2
According to another embodiment of the present invention, there is also provided a target tracking processing apparatus, and fig. 5 is a block diagram of the target tracking processing apparatus according to the embodiment of the present invention, as shown in fig. 5, including:
the target detection module 52 is configured to perform target detection on a current video frame of the video data to obtain a detection result of a target, where the detection result includes a target feature and a current motion state of the target;
a matching module 54, configured to match the feature information of the target with predetermined feature information of a target to be tracked;
a first determining module 56, configured to determine, according to the current motion state of the target, an estimated motion state of the target in a next video frame if the matching is successful;
a second determining module 58, configured to determine, according to the estimated motion state, a target device tracking the target in the next video frame.
Optionally, the first determining module 56 includes:
the first determining submodule is used for determining a state control vector of the target in the next video frame according to the position information of the target in a historical data frame based on a Kalman motion estimation model;
and the second determining submodule is used for determining the estimated motion state of the target in the next video frame according to the state control vector and the current motion state.
Optionally, the second determining sub-module includes:
the correction unit is used for correcting the current motion state to obtain a current corrected motion state;
a determining unit, configured to determine an estimated motion state of the target in the next video frame according to the state control vector and the current modified motion state by:
Figure BDA0002610704590000161
wherein,
Figure BDA0002610704590000162
for said estimated motion state, FnState transition matrix, x, for the object in the next video framen-1For the current correction motion state, unFor the state control vector, BnControlling a matrix for the state of the target at the next video frame.
Optionally, the apparatus further comprises:
a third determining module for determining a covariance matrix estimate of a state of the target in the next video frame by:
Figure BDA0002610704590000163
wherein,
Figure BDA0002610704590000164
covariance matrix estimation, P, for the state of the target in the next video framen-1In the current video frame for the targetCovariance matrix estimation of the corrected state, QnA covariance matrix of system noise in the next video frame;
and the correction module is used for correcting the estimated motion state according to the covariance matrix estimation of the state of the target in the next video frame to obtain the corrected motion state of the target in the next video frame.
Optionally, the correction module includes:
the acquisition submodule is used for acquiring the actual motion state of the target in the next video frame;
and the first correction submodule is used for correcting the pre-estimated motion state according to the actual motion state of the target in the next video frame and the covariance matrix estimation of the state of the target in the next video frame, so as to obtain the corrected motion state of the target in the next video frame.
Optionally, the first modification sub-module is further configured to modify the estimated motion state according to a covariance matrix estimate of an actual motion state of the target in the next video frame and a state of the target in the next video frame, so as to obtain a modified motion state of the target in the next video frame:
Figure BDA0002610704590000171
Figure BDA0002610704590000172
Figure BDA0002610704590000173
Figure BDA0002610704590000174
wherein，xnFor the modified motion state of the object in the next video frame, znFor the state of motion of the object observed in the next video frame, SnMeasuring a cosine covariance matrix, R, for the target in the next video framenA covariance matrix of measured noise for the target in the next video frame; knFor the Carlman gain, H, of the object in the next video framenA mapping matrix of the motion state of the target to the device observations.
Optionally, the apparatus further comprises:
a second modification submodule, configured to modify the covariance matrix estimate of the state of the target in the next video frame in the following manner, to obtain a covariance matrix estimate of the state of the target after modification in the next video frame:
Figure BDA0002610704590000175
wherein, PnCovariance matrix estimation for the modified state of the target in the next video frame.
Optionally, the matching module 54 includes:
the third determining submodule is used for determining the similarity between the target characteristics and the predetermined characteristic information of the target to be tracked;
the fourth determining submodule is used for determining that the matching is successful under the condition that the similarity is greater than or equal to a preset threshold value;
and the fifth determining submodule is used for determining that the matching fails under the condition that the similarity is smaller than the preset threshold.
Optionally, the second determining module 58 is further configured to
Determining a target monitoring area corresponding to the estimated motion state;
and determining the target equipment corresponding to the target monitoring area according to the corresponding relation between the monitoring area and the equipment.
It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are respectively located in different processors in any combination.
Example 3
Embodiments of the present invention also provide a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to perform the steps of any of the above method embodiments when executed.
Alternatively, in the present embodiment, the storage medium may be configured to store a computer program for executing the following steps:
s1, performing target detection on a current video frame of the video data to obtain a detection result of a target, wherein the detection result comprises a target characteristic and a current motion state of the target;
s2, matching the characteristic information of the target with the predetermined characteristic information of the target to be tracked;
s3, determining the estimated motion state of the target in the next video frame according to the current motion state of the target under the condition of successful matching;
and S4, determining the target equipment of the target in the next video frame according to the estimated motion state.
Optionally, in this embodiment, the storage medium may include, but is not limited to: various media capable of storing computer programs, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Example 4
Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, performing target detection on a current video frame of the video data to obtain a detection result of a target, wherein the detection result comprises a target characteristic and a current motion state of the target;
s2, matching the characteristic information of the target with the predetermined characteristic information of the target to be tracked;
s3, determining the estimated motion state of the target in the next video frame according to the current motion state of the target under the condition of successful matching;
and S4, determining the target equipment of the target in the next video frame according to the estimated motion state.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and this embodiment is not described herein again.
It will be apparent to those skilled in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be centralized on a single computing device or distributed across a network of computing devices, and alternatively, they may be implemented in program code that is executable by a computing device, such that it may be stored in a memory device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that shown or described herein, or separately fabricated into individual integrated circuit modules, or multiple ones of them fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims (12)

1. A target tracking processing method is characterized by comprising the following steps:
performing target detection on a current video frame of video data to obtain a detection result of a target, wherein the detection result comprises a target characteristic and a current motion state of the target;
matching the characteristic information of the target with the predetermined characteristic information of the target to be tracked;
under the condition of successful matching, determining the estimated motion state of the target in the next video frame according to the current motion state of the target;
and determining the target equipment tracking the target in the next video frame according to the estimated motion state.
2. The method of claim 1, wherein determining the estimated motion state of the object in the next video frame based on the current motion state of the object comprises:
determining a state control vector of the target in the next video frame according to the position information of the target in the historical data frame based on a Kalman motion estimation model;
and determining the estimated motion state of the target in the next video frame according to the state control vector and the current motion state.
3. The method of claim 2, wherein determining the estimated motion state of the target in the next video frame based on the state control vector and the current motion state comprises:
correcting the current motion state to obtain a current corrected motion state;
determining an estimated motion state of the target in the next video frame according to the state control vector and the current modified motion state by:
Figure FDA0002610704580000011
wherein,
Figure FDA0002610704580000021
for said estimated motion state, FnState transition matrix, x, for the target in the next video framen-1For the current correction motion state, unFor the state control vector, BnControlling a matrix for the state of the target at the next video frame.
4. The method of claim 2, further comprising:
determining a covariance matrix estimate of a state of the target in the next video frame by:
Figure FDA0002610704580000022
wherein,
Figure FDA0002610704580000023
covariance matrix estimation, P, for the state of the target in the next video framen-1Covariance matrix estimation, Q, for the state of the target after modification in the current video framenA covariance matrix of system noise in the next video frame;
and correcting the estimated motion state according to the covariance matrix estimation of the state of the target in the next video frame to obtain the corrected motion state of the target in the next video frame.
5. The method of claim 4, wherein modifying the estimated motion state based on a covariance matrix estimate of the state of the target in the next video frame comprises:
acquiring the actual motion state of the target in the next video frame;
and correcting the estimated motion state according to the actual motion state of the target in the next video frame and the covariance matrix estimation of the state of the target in the next video frame to obtain the corrected motion state of the target in the next video frame.
6. The method of claim 5, further comprising:
correcting the estimated motion state according to the covariance matrix estimation of the actual motion state of the target in the next video frame and the state of the target in the next video frame in the following way, so as to obtain the corrected motion state of the target in the next video frame:
Figure FDA0002610704580000031
Figure FDA0002610704580000032
Figure FDA0002610704580000033
Figure FDA0002610704580000034
wherein x isnFor the modified motion state of the object in the next video frame, znThe next video for the targetObserved target motion state in frames, SnMeasuring a cosine covariance matrix, R, for the target in the next video framenA covariance matrix of measured noise for the target in the next video frame; knFor the Kalman gain, H, of the target in the next video framenA mapping matrix of the motion state of the target to the device observations.
7. The method of claim 6, further comprising:
correcting the covariance matrix estimate of the state of the target in the next video frame by the following method to obtain the covariance matrix estimate of the state of the target after correction in the next video frame:
Figure FDA0002610704580000035
wherein, PnCovariance matrix estimation for the modified state of the target in the next video frame.
8. The method according to any one of claims 1 to 7, wherein matching the feature information of the target with predetermined feature information of a target to be tracked comprises:
determining the similarity between the target characteristics and the predetermined characteristic information of the target to be tracked;
determining that the matching is successful under the condition that the similarity is greater than or equal to a preset threshold value;
and determining that the matching fails under the condition that the similarity is smaller than the preset threshold.
9. The method of claim 8, wherein determining a target device in the next video frame that tracks the target based on the estimated motion state comprises:
determining a target monitoring area corresponding to the estimated motion state;
and determining the target equipment corresponding to the target monitoring area according to the corresponding relation between the monitoring area and the equipment.
10. An object tracking processing apparatus, characterized by comprising:
the target detection module is used for carrying out target detection on a current video frame of video data to obtain a detection result of a target, wherein the detection result comprises a target characteristic and a current motion state of the target;
the matching module is used for matching the characteristic information of the target with the predetermined characteristic information of the target to be tracked;
the first determining module is used for determining the estimated motion state of the target in the next video frame according to the current motion state of the target under the condition of successful matching;
and the second determining module is used for determining target equipment for tracking the target in the next video frame according to the estimated motion state.
11. A computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to carry out the method of any one of claims 1 to 9 when executed.
12. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and wherein the processor is arranged to execute the computer program to perform the method of any of claims 1 to 9.
CN202010753234.9A 2020-07-30 2020-07-30 Target tracking processing method and device Active CN112001948B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010753234.9A CN112001948B (en) 2020-07-30 2020-07-30 Target tracking processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010753234.9A CN112001948B (en) 2020-07-30 2020-07-30 Target tracking processing method and device

Publications (2)

Publication Number Publication Date
CN112001948A true CN112001948A (en) 2020-11-27
CN112001948B CN112001948B (en) 2024-06-11

Family

ID=73463320

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010753234.9A Active CN112001948B (en) 2020-07-30 2020-07-30 Target tracking processing method and device

Country Status (1)

Country Link
CN (1) CN112001948B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113092807A (en) * 2021-04-21 2021-07-09 上海浦江桥隧运营管理有限公司 Urban elevated road vehicle speed measuring method based on multi-target tracking algorithm
CN113344966A (en) * 2021-06-18 2021-09-03 北京星航科创技术有限公司 Target motion state correction method of motion camera
CN113628251A (en) * 2021-10-11 2021-11-09 北京中科金马科技股份有限公司 Smart hotel terminal monitoring method

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101783020B (en) * 2010-03-04 2011-08-17 湖南大学 Video multi-target fast tracking method based on joint probability data association
CN106780542A (en) * 2016-12-29 2017-05-31 北京理工大学 A kind of machine fish tracking of the Camshift based on embedded Kalman filter
CN106845385A (en) * 2017-01-17 2017-06-13 腾讯科技（上海）有限公司 The method and apparatus of video frequency object tracking
WO2019064375A1 (en) * 2017-09-27 2019-04-04 日本電気株式会社 Information processing device, control method, and program
CN109785368B (en) * 2017-11-13 2022-07-22 腾讯科技（深圳）有限公司 Target tracking method and device
CN109712173A (en) * 2018-12-05 2019-05-03 北京空间机电研究所 A kind of picture position method for estimating based on Kalman filter
CN110110649B (en) * 2019-05-02 2023-04-07 西安电子科技大学 Selective human face detection method based on speed direction
CN111145214A (en) * 2019-12-17 2020-05-12 深圳云天励飞技术有限公司 Target tracking method, device, terminal equipment and medium
CN111192296A (en) * 2019-12-30 2020-05-22 长沙品先信息技术有限公司 Pedestrian multi-target detection and tracking method based on video monitoring

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113092807A (en) * 2021-04-21 2021-07-09 上海浦江桥隧运营管理有限公司 Urban elevated road vehicle speed measuring method based on multi-target tracking algorithm
CN113092807B (en) * 2021-04-21 2024-05-14 上海浦江桥隧运营管理有限公司 Urban overhead road vehicle speed measuring method based on multi-target tracking algorithm
CN113344966A (en) * 2021-06-18 2021-09-03 北京星航科创技术有限公司 Target motion state correction method of motion camera
CN113344966B (en) * 2021-06-18 2023-09-22 北京星航科创技术有限公司 Target motion state correction method of motion camera
CN113628251A (en) * 2021-10-11 2021-11-09 北京中科金马科技股份有限公司 Smart hotel terminal monitoring method
CN113628251B (en) * 2021-10-11 2022-02-01 北京中科金马科技股份有限公司 Smart hotel terminal monitoring method

Also Published As

Publication number Publication date
CN112001948B (en) 2024-06-11

Similar Documents

Publication Publication Date Title
CN109035299B (en) Target tracking method and device, computer equipment and storage medium
CN109635657B (en) Target tracking method, device, equipment and storage medium
CN112001948B (en) Target tracking processing method and device
US9852511B2 (en) Systems and methods for tracking and detecting a target object
CN110610150B (en) Tracking method, device, computing equipment and medium of target moving object
CN111512317A (en) Multi-target real-time tracking method and device and electronic equipment
CN109035295B (en) Multi-target tracking method, device, computer equipment and storage medium
US10096123B2 (en) Method and device for establishing correspondence between objects in a multi-image source environment
CN110111364B (en) Motion detection method and device, electronic equipment and storage medium
CN115063454B (en) Multi-target tracking matching method, device, terminal and storage medium
CN111798487A (en) Target tracking method, device and computer readable storage medium
CN109636828A (en) Object tracking methods and device based on video image
CN116645396A (en) Track determination method, track determination device, computer-readable storage medium and electronic device
CN117896626B (en) Method, device, equipment and storage medium for detecting motion trail by multiple cameras
EP4038540B1 (en) Object detection
CN113989761A (en) Object tracking method and device, electronic equipment and storage medium
CN111652907B (en) Multi-target tracking method and device based on data association and electronic equipment
CN112819889A (en) Method and device for determining position information, storage medium and electronic device
KR101595334B1 (en) Method and apparatus for movement trajectory tracking of moving object on animal farm
CN112214629A (en) Loop detection method based on image recognition and movable equipment
CN112819859B (en) Multi-target tracking method and device applied to intelligent security
CN111783641A (en) Face clustering method and device
CN111862161B (en) Target tracking processing method and device
CN112528078B (en) Image archiving method, device and equipment
EP4198918A1 (en) Object detection method and object detection system for video

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant